From: Mark Brown <broonie@kernel.org>
To: Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>, Marc Zyngier <maz@kernel.org>,
James Morse <james.morse@arm.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
Oliver Upton <oliver.upton@linux.dev>,
Oleg Nesterov <oleg@redhat.com>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev,
kvmarm@lists.cs.columbia.edu, Mark Brown <broonie@kernel.org>,
stable@vger.kernel.org, Mark Rutland <mark.rutland@arm.com>,
Eric Auger <eauger@redhat.com>,
Wilco Dijkstra <wilco.dijkstra@arm.com>,
Eric Auger <eric.auger@redhat.com>,
Florian Weimer <fweimer@redhat.com>,
Fuad Tabba <tabba@google.com>,
Jeremy Linton <jeremy.linton@arm.com>,
Paolo Bonzini <pbonzini@redhat.com>
Subject: [PATCH RESEND 6.1 05/12] KVM: arm64: Unconditionally save+flush host FPSIMD/SVE/SME state
Date: Fri, 04 Apr 2025 14:23:38 +0100 [thread overview]
Message-ID: <20250404-stable-sve-6-1-v1-5-cd5c9eb52d49@kernel.org> (raw)
In-Reply-To: <20250404-stable-sve-6-1-v1-0-cd5c9eb52d49@kernel.org>
From: Mark Rutland <mark.rutland@arm.com>
[ Upstream commit fbc7e61195e23f744814e78524b73b59faa54ab4 ]
There are several problems with the way hyp code lazily saves the host's
FPSIMD/SVE state, including:
* Host SVE being discarded unexpectedly due to inconsistent
configuration of TIF_SVE and CPACR_ELx.ZEN. This has been seen to
result in QEMU crashes where SVE is used by memmove(), as reported by
Eric Auger:
https://issues.redhat.com/browse/RHEL-68997
* Host SVE state is discarded *after* modification by ptrace, which was an
unintentional ptrace ABI change introduced with lazy discarding of SVE state.
* The host FPMR value can be discarded when running a non-protected VM,
where FPMR support is not exposed to a VM, and that VM uses
FPSIMD/SVE. In these cases the hyp code does not save the host's FPMR
before unbinding the host's FPSIMD/SVE/SME state, leaving a stale
value in memory.
Avoid these by eagerly saving and "flushing" the host's FPSIMD/SVE/SME
state when loading a vCPU such that KVM does not need to save any of the
host's FPSIMD/SVE/SME state. For clarity, fpsimd_kvm_prepare() is
removed and the necessary call to fpsimd_save_and_flush_cpu_state() is
placed in kvm_arch_vcpu_load_fp(). As 'fpsimd_state' and 'fpmr_ptr'
should not be used, they are set to NULL; all uses of these will be
removed in subsequent patches.
Historical problems go back at least as far as v5.17, e.g. erroneous
assumptions about TIF_SVE being clear in commit:
8383741ab2e773a9 ("KVM: arm64: Get rid of host SVE tracking/saving")
... and so this eager save+flush probably needs to be backported to ALL
stable trees.
Fixes: 93ae6b01bafee8fa ("KVM: arm64: Discard any SVE state when entering KVM guests")
Fixes: 8c845e2731041f0f ("arm64/sve: Leave SVE enabled on syscall if we don't context switch")
Fixes: ef3be86021c3bdf3 ("KVM: arm64: Add save/restore support for FPMR")
Reported-by: Eric Auger <eauger@redhat.com>
Reported-by: Wilco Dijkstra <wilco.dijkstra@arm.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Tested-by: Mark Brown <broonie@kernel.org>
Tested-by: Eric Auger <eric.auger@redhat.com>
Acked-by: Will Deacon <will@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Florian Weimer <fweimer@redhat.com>
Cc: Fuad Tabba <tabba@google.com>
Cc: Jeremy Linton <jeremy.linton@arm.com>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Oliver Upton <oliver.upton@linux.dev>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Oliver Upton <oliver.upton@linux.dev>
Link: https://lore.kernel.org/r/20250210195226.1215254-2-mark.rutland@arm.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
[ Mark: Handle vcpu/host flag conflict, remove host_data_ptr() ]
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Mark Brown <broonie@kernel.org>
---
arch/arm64/kernel/fpsimd.c | 25 -------------------------
arch/arm64/kvm/fpsimd.c | 18 ++++++++++--------
2 files changed, 10 insertions(+), 33 deletions(-)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 3fcacbce5d42..47425311acc5 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -1671,31 +1671,6 @@ void fpsimd_signal_preserve_current_state(void)
sve_to_fpsimd(current);
}
-/*
- * Called by KVM when entering the guest.
- */
-void fpsimd_kvm_prepare(void)
-{
- if (!system_supports_sve())
- return;
-
- /*
- * KVM does not save host SVE state since we can only enter
- * the guest from a syscall so the ABI means that only the
- * non-saved SVE state needs to be saved. If we have left
- * SVE enabled for performance reasons then update the task
- * state to be FPSIMD only.
- */
- get_cpu_fpsimd_context();
-
- if (test_and_clear_thread_flag(TIF_SVE)) {
- sve_to_fpsimd(current);
- current->thread.fp_type = FP_STATE_FPSIMD;
- }
-
- put_cpu_fpsimd_context();
-}
-
/*
* Associate current's FPSIMD context with this cpu
* The caller must have ownership of the cpu FPSIMD context before calling
diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c
index ec82d0191f76..1765f723afd4 100644
--- a/arch/arm64/kvm/fpsimd.c
+++ b/arch/arm64/kvm/fpsimd.c
@@ -79,9 +79,16 @@ void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
if (!system_supports_fpsimd())
return;
- fpsimd_kvm_prepare();
-
- vcpu->arch.fp_state = FP_STATE_HOST_OWNED;
+ /*
+ * Ensure that any host FPSIMD/SVE/SME state is saved and unbound such
+ * that the host kernel is responsible for restoring this state upon
+ * return to userspace, and the hyp code doesn't need to save anything.
+ *
+ * When the host may use SME, fpsimd_save_and_flush_cpu_state() ensures
+ * that PSTATE.{SM,ZA} == {0,0}.
+ */
+ fpsimd_save_and_flush_cpu_state();
+ vcpu->arch.fp_state = FP_STATE_FREE;
vcpu_clear_flag(vcpu, HOST_SVE_ENABLED);
if (read_sysreg(cpacr_el1) & CPACR_EL1_ZEN_EL0EN)
@@ -100,11 +107,6 @@ void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
vcpu_clear_flag(vcpu, HOST_SME_ENABLED);
if (read_sysreg(cpacr_el1) & CPACR_EL1_SMEN_EL0EN)
vcpu_set_flag(vcpu, HOST_SME_ENABLED);
-
- if (read_sysreg_s(SYS_SVCR) & (SVCR_SM_MASK | SVCR_ZA_MASK)) {
- vcpu->arch.fp_state = FP_STATE_FREE;
- fpsimd_save_and_flush_cpu_state();
- }
}
}
--
2.39.5
next prev parent reply other threads:[~2025-04-04 13:27 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-04-04 13:23 [6.1 PATCH RESEND 00/12] KVM: arm64: Backport of SVE fixes to v6.1 Mark Brown
2025-04-04 13:23 ` [PATCH RESEND 6.1 01/12] KVM: arm64: Discard any SVE state when entering KVM guests Mark Brown
2025-04-04 13:23 ` [PATCH RESEND 6.1 02/12] arm64/fpsimd: Track the saved FPSIMD state type separately to TIF_SVE Mark Brown
2025-04-04 13:23 ` [PATCH RESEND 6.1 03/12] arm64/fpsimd: Have KVM explicitly say which FP registers to save Mark Brown
2025-04-04 13:23 ` [PATCH RESEND 6.1 04/12] arm64/fpsimd: Stop using TIF_SVE to manage register saving in KVM Mark Brown
2025-04-04 13:23 ` Mark Brown [this message]
2025-04-04 13:23 ` [PATCH RESEND 6.1 06/12] KVM: arm64: Remove host FPSIMD saving for non-protected KVM Mark Brown
2025-04-04 13:23 ` [PATCH RESEND 6.1 07/12] KVM: arm64: Remove VHE host restore of CPACR_EL1.ZEN Mark Brown
2025-04-04 13:23 ` [PATCH RESEND 6.1 08/12] KVM: arm64: Remove VHE host restore of CPACR_EL1.SMEN Mark Brown
2025-04-04 13:23 ` [PATCH RESEND 6.1 09/12] KVM: arm64: Refactor exit handlers Mark Brown
2025-04-04 13:23 ` [PATCH RESEND 6.1 10/12] KVM: arm64: Mark some header functions as inline Mark Brown
2025-04-04 13:23 ` [PATCH RESEND 6.1 11/12] KVM: arm64: Calculate cptr_el2 traps on activating traps Mark Brown
2025-04-04 13:23 ` [PATCH RESEND 6.1 12/12] KVM: arm64: Eagerly switch ZCR_EL{1,2} Mark Brown
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250404-stable-sve-6-1-v1-5-cd5c9eb52d49@kernel.org \
--to=broonie@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=eauger@redhat.com \
--cc=eric.auger@redhat.com \
--cc=fweimer@redhat.com \
--cc=gregkh@linuxfoundation.org \
--cc=james.morse@arm.com \
--cc=jeremy.linton@arm.com \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=kvmarm@lists.linux.dev \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=maz@kernel.org \
--cc=oleg@redhat.com \
--cc=oliver.upton@linux.dev \
--cc=pbonzini@redhat.com \
--cc=stable@vger.kernel.org \
--cc=suzuki.poulose@arm.com \
--cc=tabba@google.com \
--cc=wilco.dijkstra@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®