* [PATCH 0/2] Add throttling detection to sev-guest @ 2022-10-13 16:00 Dionna Glaze 2022-10-13 16:00 ` [PATCH 1/2] x86/sev: Rename snp_guest_issue_request's fw_err Dionna Glaze 2022-10-13 16:00 ` [PATCH 2/2] virt/coco/sev-guest: interpret VMM errors from guest request Dionna Glaze 0 siblings, 2 replies; 5+ messages in thread From: Dionna Glaze @ 2022-10-13 16:00 UTC (permalink / raw) To: linux-kernel, x86 Cc: Dionna Glaze, Tom Lendacky, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen The guest request synchronous API from SEV-SNP VMs to the host's security processor consumes a global resource. For this reason, AMD's docs recommend that the host implements a throttling mechanism. In order for the guest to know it's been throttled and should try its request again, we need some good-faith communication from the host that the request has been throttled. These patches work with the existing /dev/sev-guest ABI to detect a throttling code. Cc: Tom Lendacky <Thomas.Lendacky@amd.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Joerg Roedel <jroedel@suse.de> Cc: Peter Gonda <pgonda@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Signed-off-by: Dionna Glaze <dionnaglaze@google.com> Dionna Glaze (2): x86/sev: Rename snp_guest_issue_request's fw_err virt/coco/sev-guest: interpret VMM errors from guest request arch/x86/include/asm/sev.h | 4 +-- arch/x86/kernel/sev.c | 6 ++--- drivers/virt/coco/sev-guest/sev-guest.c | 34 +++++++++++++++++-------- include/uapi/linux/sev-guest.h | 17 +++++++++++-- 4 files changed, 43 insertions(+), 18 deletions(-) -- 2.38.0.rc1.362.ged0d419d3c-goog ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/2] x86/sev: Rename snp_guest_issue_request's fw_err 2022-10-13 16:00 [PATCH 0/2] Add throttling detection to sev-guest Dionna Glaze @ 2022-10-13 16:00 ` Dionna Glaze 2022-10-13 16:00 ` [PATCH 2/2] virt/coco/sev-guest: interpret VMM errors from guest request Dionna Glaze 1 sibling, 0 replies; 5+ messages in thread From: Dionna Glaze @ 2022-10-13 16:00 UTC (permalink / raw) To: linux-kernel, x86 Cc: Dionna Glaze, Tom Lendacky, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen The GHCB specification declares that the firmware error value for a guest request will be stored in the lower 32 bits of EXIT_INFO_2. The upper 32 bits are for the VMM's own error code. The fw_err argument is thus a misnomer, and callers will need access to all 64 bits. Cc: Tom Lendacky <Thomas.Lendacky@amd.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Joerg Roedel <jroedel@suse.de> Cc: Peter Gonda <pgonda@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Signed-off-by: Dionna Glaze <dionnaglaze@google.com> --- arch/x86/include/asm/sev.h | 4 ++-- arch/x86/kernel/sev.c | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index ebc271bb6d8e..4312d6be129f 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -196,7 +196,7 @@ void snp_set_memory_private(unsigned long vaddr, unsigned int npages); void snp_set_wakeup_secondary_cpu(void); bool snp_init(struct boot_params *bp); void __init __noreturn snp_abort(void); -int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err); +int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *exitinfo2); #else static inline void sev_es_ist_enter(struct pt_regs *regs) { } static inline void sev_es_ist_exit(void) { } @@ -217,7 +217,7 @@ static inline void snp_set_wakeup_secondary_cpu(void) { } static inline bool snp_init(struct boot_params *bp) { return false; } static inline void snp_abort(void) { } static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, - unsigned long *fw_err) + unsigned long *exitinfo2) { return -ENOTTY; } diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index a428c62330d3..a4eca63e8b90 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -2175,7 +2175,7 @@ static int __init init_sev_config(char *str) } __setup("sev=", init_sev_config); -int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err) +int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *exitinfo2) { struct ghcb_state state; struct es_em_ctxt ctxt; @@ -2186,7 +2186,7 @@ int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) return -ENODEV; - if (!fw_err) + if (!exitinfo2) return -EINVAL; /* @@ -2218,7 +2218,7 @@ int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned ghcb->save.sw_exit_info_2 == SNP_GUEST_REQ_INVALID_LEN) input->data_npages = ghcb_get_rbx(ghcb); - *fw_err = ghcb->save.sw_exit_info_2; + *exitinfo2 = ghcb->save.sw_exit_info_2; ret = -EIO; } -- 2.38.0.rc1.362.ged0d419d3c-goog ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 2/2] virt/coco/sev-guest: interpret VMM errors from guest request 2022-10-13 16:00 [PATCH 0/2] Add throttling detection to sev-guest Dionna Glaze 2022-10-13 16:00 ` [PATCH 1/2] x86/sev: Rename snp_guest_issue_request's fw_err Dionna Glaze @ 2022-10-13 16:00 ` Dionna Glaze 2022-10-13 21:59 ` Tom Lendacky 1 sibling, 1 reply; 5+ messages in thread From: Dionna Glaze @ 2022-10-13 16:00 UTC (permalink / raw) To: linux-kernel, x86 Cc: Dionna Glaze, Tom Lendacky, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen The GHCB specification states that the upper 32 bits of exitinfo2 are for the VMM's error codes. The sev-guest ABI has already locked in that the fw_err status of the input will be 64 bits, and that BIT_ULL(32) means that the extended guest request's data buffer was too small, so we have to keep that ABI. This patch deprecates the fw_err field name in the request to exitinfo2. We can still interpret the upper 32 bits of exitinfo2 for the user anyway in case the request gets throttled. That makes the interface more user-friendly. Cc: Tom Lendacky <Thomas.Lendacky@amd.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Joerg Roedel <jroedel@suse.de> Cc: Peter Gonda <pgonda@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Signed-off-by: Dionna Glaze <dionnaglaze@google.com> --- drivers/virt/coco/sev-guest/sev-guest.c | 34 +++++++++++++++++-------- include/uapi/linux/sev-guest.h | 17 +++++++++++-- 2 files changed, 38 insertions(+), 13 deletions(-) diff --git a/drivers/virt/coco/sev-guest/sev-guest.c b/drivers/virt/coco/sev-guest/sev-guest.c index f422f9c58ba7..8bc435cbd7e3 100644 --- a/drivers/virt/coco/sev-guest/sev-guest.c +++ b/drivers/virt/coco/sev-guest/sev-guest.c @@ -303,9 +303,10 @@ static int enc_payload(struct snp_guest_dev *snp_dev, u64 seqno, int version, u8 static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, int msg_ver, u8 type, void *req_buf, size_t req_sz, void *resp_buf, - u32 resp_sz, __u64 *fw_err) + u32 resp_sz, __u64 *exitinfo2) { - unsigned long err; + unsigned long _exitinfo2 = 0; + unsigned int vmm_err; u64 seqno; int rc; @@ -322,9 +323,20 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in return rc; /* Call firmware to process the request */ - rc = snp_issue_guest_request(exit_code, &snp_dev->input, &err); - if (fw_err) - *fw_err = err; + rc = snp_issue_guest_request(exit_code, &snp_dev->input, &_exitinfo2); + if (exitinfo2) + *exitinfo2 = _exitinfo2; + + vmm_err = _exitinfo2 >> SNP_GUEST_VMM_ERR_SHIFT; + /* The host may return EBUSY if the request has been throttled. */ + if (vmm_err == SNP_GUEST_VMM_ERR_BUSY) + return -EAGAIN; + + if (vmm_err && vmm_err != SNP_GUEST_VMM_ERR_INVALID_LEN) { + pr_err("sev-guest: host returned unknown error code: %d\n", + vmm_err); + return -EINVAL; + } if (rc) return rc; @@ -378,7 +390,7 @@ static int get_report(struct snp_guest_dev *snp_dev, struct snp_guest_request_io rc = handle_guest_request(snp_dev, SVM_VMGEXIT_GUEST_REQUEST, arg->msg_version, SNP_MSG_REPORT_REQ, &req, sizeof(req), resp->data, - resp_len, &arg->fw_err); + resp_len, &arg->exitinfo2); if (rc) goto e_free; @@ -418,7 +430,7 @@ static int get_derived_key(struct snp_guest_dev *snp_dev, struct snp_guest_reque rc = handle_guest_request(snp_dev, SVM_VMGEXIT_GUEST_REQUEST, arg->msg_version, SNP_MSG_KEY_REQ, &req, sizeof(req), buf, resp_len, - &arg->fw_err); + &arg->exitinfo2); if (rc) return rc; @@ -480,10 +492,10 @@ static int get_ext_report(struct snp_guest_dev *snp_dev, struct snp_guest_reques snp_dev->input.data_npages = npages; ret = handle_guest_request(snp_dev, SVM_VMGEXIT_EXT_GUEST_REQUEST, arg->msg_version, SNP_MSG_REPORT_REQ, &req.data, - sizeof(req.data), resp->data, resp_len, &arg->fw_err); + sizeof(req.data), resp->data, resp_len, &arg->exitinfo2); /* If certs length is invalid then copy the returned length */ - if (arg->fw_err == SNP_GUEST_REQ_INVALID_LEN) { + if (arg->vmm_error == SNP_GUEST_VMM_ERR_INVALID_LEN) { req.certs_len = snp_dev->input.data_npages << PAGE_SHIFT; if (copy_to_user((void __user *)arg->req_data, &req, sizeof(req))) @@ -518,7 +530,7 @@ static long snp_guest_ioctl(struct file *file, unsigned int ioctl, unsigned long if (copy_from_user(&input, argp, sizeof(input))) return -EFAULT; - input.fw_err = 0xff; + input.exitinfo2 = 0xff; /* Message version must be non-zero */ if (!input.msg_version) @@ -549,7 +561,7 @@ static long snp_guest_ioctl(struct file *file, unsigned int ioctl, unsigned long mutex_unlock(&snp_cmd_mutex); - if (input.fw_err && copy_to_user(argp, &input, sizeof(input))) + if (input.exitinfo2 && copy_to_user(argp, &input, sizeof(input))) return -EFAULT; return ret; diff --git a/include/uapi/linux/sev-guest.h b/include/uapi/linux/sev-guest.h index 256aaeff7e65..0654a652e4dc 100644 --- a/include/uapi/linux/sev-guest.h +++ b/include/uapi/linux/sev-guest.h @@ -12,6 +12,7 @@ #ifndef __UAPI_LINUX_SEV_GUEST_H_ #define __UAPI_LINUX_SEV_GUEST_H_ +#include <linux/errno.h> #include <linux/types.h> struct snp_report_req { @@ -52,8 +53,15 @@ struct snp_guest_request_ioctl { __u64 req_data; __u64 resp_data; - /* firmware error code on failure (see psp-sev.h) */ - __u64 fw_err; + /* bits[63:32]: VMM error code, bits[31:0] firmware error code (see psp-sev.h) */ + union { + __u64 exitinfo2; + __u64 fw_err; /* Name deprecated in favor of others */ + struct { + __u32 fw_error; + __u32 vmm_error; + }; + }; }; struct snp_ext_report_req { @@ -77,4 +85,9 @@ struct snp_ext_report_req { /* Get SNP extended report as defined in the GHCB specification version 2. */ #define SNP_GET_EXT_REPORT _IOWR(SNP_GUEST_REQ_IOC_TYPE, 0x2, struct snp_guest_request_ioctl) + +#define SNP_GUEST_VMM_ERR_SHIFT 32 +#define SNP_GUEST_VMM_ERR_INVALID_LEN 1 +#define SNP_GUEST_VMM_ERR_BUSY EBUSY + #endif /* __UAPI_LINUX_SEV_GUEST_H_ */ -- 2.38.0.rc1.362.ged0d419d3c-goog ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 2/2] virt/coco/sev-guest: interpret VMM errors from guest request 2022-10-13 16:00 ` [PATCH 2/2] virt/coco/sev-guest: interpret VMM errors from guest request Dionna Glaze @ 2022-10-13 21:59 ` Tom Lendacky 2022-10-18 16:20 ` Dionna Amalie Glaze 0 siblings, 1 reply; 5+ messages in thread From: Tom Lendacky @ 2022-10-13 21:59 UTC (permalink / raw) To: Dionna Glaze, linux-kernel, x86 Cc: Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen On 10/13/22 11:00, Dionna Glaze wrote: > The GHCB specification states that the upper 32 bits of exitinfo2 are > for the VMM's error codes. The sev-guest ABI has already locked in I will have to update the GHCB specification for SNP Guest Request and SNP Extended Guest Request for this change. I believe that there isn't an impact from a backwards compatibility since software should be checking for specific values for success or the need for additional memory. Adding a (new) return code will result in an error return for older software, which is ok. > that the fw_err status of the input will be 64 bits, and that > BIT_ULL(32) means that the extended guest request's data buffer was too > small, so we have to keep that ABI. This patch deprecates the fw_err > field name in the request to exitinfo2. > > We can still interpret the upper 32 bits of exitinfo2 for the user > anyway in case the request gets throttled. That makes the interface more > user-friendly. > > Cc: Tom Lendacky <Thomas.Lendacky@amd.com> > Cc: Paolo Bonzini <pbonzini@redhat.com> > Cc: Joerg Roedel <jroedel@suse.de> > Cc: Peter Gonda <pgonda@google.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: Dave Hansen <dave.hansen@linux.intel.com> > Signed-off-by: Dionna Glaze <dionnaglaze@google.com> > --- > drivers/virt/coco/sev-guest/sev-guest.c | 34 +++++++++++++++++-------- > include/uapi/linux/sev-guest.h | 17 +++++++++++-- > 2 files changed, 38 insertions(+), 13 deletions(-) > > diff --git a/drivers/virt/coco/sev-guest/sev-guest.c b/drivers/virt/coco/sev-guest/sev-guest.c > index f422f9c58ba7..8bc435cbd7e3 100644 > --- a/drivers/virt/coco/sev-guest/sev-guest.c > +++ b/drivers/virt/coco/sev-guest/sev-guest.c > @@ -303,9 +303,10 @@ static int enc_payload(struct snp_guest_dev *snp_dev, u64 seqno, int version, u8 > > static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, int msg_ver, > u8 type, void *req_buf, size_t req_sz, void *resp_buf, > - u32 resp_sz, __u64 *fw_err) > + u32 resp_sz, __u64 *exitinfo2) > { > - unsigned long err; > + unsigned long _exitinfo2 = 0; > + unsigned int vmm_err; > u64 seqno; > int rc; > > @@ -322,9 +323,20 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in > return rc; > > /* Call firmware to process the request */ > - rc = snp_issue_guest_request(exit_code, &snp_dev->input, &err); > - if (fw_err) > - *fw_err = err; > + rc = snp_issue_guest_request(exit_code, &snp_dev->input, &_exitinfo2); > + if (exitinfo2) > + *exitinfo2 = _exitinfo2; > + > + vmm_err = _exitinfo2 >> SNP_GUEST_VMM_ERR_SHIFT; > + /* The host may return EBUSY if the request has been throttled. */ > + if (vmm_err == SNP_GUEST_VMM_ERR_BUSY) > + return -EAGAIN; > + > + if (vmm_err && vmm_err != SNP_GUEST_VMM_ERR_INVALID_LEN) { > + pr_err("sev-guest: host returned unknown error code: %d\n", > + vmm_err); > + return -EINVAL; > + } > > if (rc) > return rc; > @@ -378,7 +390,7 @@ static int get_report(struct snp_guest_dev *snp_dev, struct snp_guest_request_io > > rc = handle_guest_request(snp_dev, SVM_VMGEXIT_GUEST_REQUEST, arg->msg_version, > SNP_MSG_REPORT_REQ, &req, sizeof(req), resp->data, > - resp_len, &arg->fw_err); > + resp_len, &arg->exitinfo2); > if (rc) > goto e_free; > > @@ -418,7 +430,7 @@ static int get_derived_key(struct snp_guest_dev *snp_dev, struct snp_guest_reque > > rc = handle_guest_request(snp_dev, SVM_VMGEXIT_GUEST_REQUEST, arg->msg_version, > SNP_MSG_KEY_REQ, &req, sizeof(req), buf, resp_len, > - &arg->fw_err); > + &arg->exitinfo2); > if (rc) > return rc; > > @@ -480,10 +492,10 @@ static int get_ext_report(struct snp_guest_dev *snp_dev, struct snp_guest_reques > snp_dev->input.data_npages = npages; > ret = handle_guest_request(snp_dev, SVM_VMGEXIT_EXT_GUEST_REQUEST, arg->msg_version, > SNP_MSG_REPORT_REQ, &req.data, > - sizeof(req.data), resp->data, resp_len, &arg->fw_err); > + sizeof(req.data), resp->data, resp_len, &arg->exitinfo2); > > /* If certs length is invalid then copy the returned length */ > - if (arg->fw_err == SNP_GUEST_REQ_INVALID_LEN) { > + if (arg->vmm_error == SNP_GUEST_VMM_ERR_INVALID_LEN) { > req.certs_len = snp_dev->input.data_npages << PAGE_SHIFT; > > if (copy_to_user((void __user *)arg->req_data, &req, sizeof(req))) > @@ -518,7 +530,7 @@ static long snp_guest_ioctl(struct file *file, unsigned int ioctl, unsigned long > if (copy_from_user(&input, argp, sizeof(input))) > return -EFAULT; > > - input.fw_err = 0xff; > + input.exitinfo2 = 0xff; > > /* Message version must be non-zero */ > if (!input.msg_version) > @@ -549,7 +561,7 @@ static long snp_guest_ioctl(struct file *file, unsigned int ioctl, unsigned long > > mutex_unlock(&snp_cmd_mutex); > > - if (input.fw_err && copy_to_user(argp, &input, sizeof(input))) > + if (input.exitinfo2 && copy_to_user(argp, &input, sizeof(input))) > return -EFAULT; > > return ret; > diff --git a/include/uapi/linux/sev-guest.h b/include/uapi/linux/sev-guest.h > index 256aaeff7e65..0654a652e4dc 100644 > --- a/include/uapi/linux/sev-guest.h > +++ b/include/uapi/linux/sev-guest.h > @@ -12,6 +12,7 @@ > #ifndef __UAPI_LINUX_SEV_GUEST_H_ > #define __UAPI_LINUX_SEV_GUEST_H_ > > +#include <linux/errno.h> > #include <linux/types.h> > > struct snp_report_req { > @@ -52,8 +53,15 @@ struct snp_guest_request_ioctl { > __u64 req_data; > __u64 resp_data; > > - /* firmware error code on failure (see psp-sev.h) */ > - __u64 fw_err; > + /* bits[63:32]: VMM error code, bits[31:0] firmware error code (see psp-sev.h) */ > + union { > + __u64 exitinfo2; > + __u64 fw_err; /* Name deprecated in favor of others */ > + struct { > + __u32 fw_error; > + __u32 vmm_error; > + }; > + }; > }; > > struct snp_ext_report_req { > @@ -77,4 +85,9 @@ struct snp_ext_report_req { > /* Get SNP extended report as defined in the GHCB specification version 2. */ > #define SNP_GET_EXT_REPORT _IOWR(SNP_GUEST_REQ_IOC_TYPE, 0x2, struct snp_guest_request_ioctl) > > + > +#define SNP_GUEST_VMM_ERR_SHIFT 32 > +#define SNP_GUEST_VMM_ERR_INVALID_LEN 1 > +#define SNP_GUEST_VMM_ERR_BUSY EBUSY The GHCB spec has to be OS agnostic, so in the specification I would be adding a value of 16 so that this matches. Not sure we don't want to just use 2. We might be able to say the VMM error codes will follow the POSIX standard, but then the currently defined value of 1 doesn't quite match an EPERM. Thoughts? Thanks, Tom > + > #endif /* __UAPI_LINUX_SEV_GUEST_H_ */ ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 2/2] virt/coco/sev-guest: interpret VMM errors from guest request 2022-10-13 21:59 ` Tom Lendacky @ 2022-10-18 16:20 ` Dionna Amalie Glaze 0 siblings, 0 replies; 5+ messages in thread From: Dionna Amalie Glaze @ 2022-10-18 16:20 UTC (permalink / raw) To: Tom Lendacky Cc: linux-kernel, x86, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen > > The GHCB spec has to be OS agnostic, so in the specification I would be > adding a value of 16 so that this matches. Not sure we don't want to just > use 2. We might be able to say the VMM error codes will follow the POSIX > standard, but then the currently defined value of 1 doesn't quite match an > EPERM. > > Thoughts? > I also thought it was a bit weird. Let's use 2. I'll send a v2. -- -Dionna Glaze, PhD (she/her) ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2022-10-18 16:21 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-10-13 16:00 [PATCH 0/2] Add throttling detection to sev-guest Dionna Glaze 2022-10-13 16:00 ` [PATCH 1/2] x86/sev: Rename snp_guest_issue_request's fw_err Dionna Glaze 2022-10-13 16:00 ` [PATCH 2/2] virt/coco/sev-guest: interpret VMM errors from guest request Dionna Glaze 2022-10-13 21:59 ` Tom Lendacky 2022-10-18 16:20 ` Dionna Amalie Glaze
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®