mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: David Woodhouse <dwmw2@infradead.org>
To: kvm@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
	linux-doc@vger.kernel.org, xen-devel@lists.xenproject.org,
	x86@kernel.org, Paolo Bonzini <pbonzini@redhat.com>,
	Sean Christopherson <seanjc@google.com>,
	Paul Durrant <paul@xen.org>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Thomas Gleixner <tglx@kernel.org>, Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	"H . Peter Anvin" <hpa@zytor.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Shuah Khan <skhan@linuxfoundation.org>,
	Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Jonathan Cameron <jic23@kernel.org>,
	Sascha Bischoff <Sascha.Bischoff@arm.com>,
	Marc Zyngier <maz@kernel.org>, Joey Gouly <joey.gouly@arm.com>,
	Jack Allister <jalliste@amazon.com>,
	Dongli Zhang <dongli.zhang@oracle.com>,
	joe.jin@oracle.com, David Woodhouse <dwmw@amazon.co.uk>
Subject: [PATCH v13 03/26] KVM: x86: Replace nr_vcpus_matched_tsc count with all_vcpus_matched_tsc bool
Date: Wed, 11 Nov 2026 10:08:59 -0800	[thread overview]
Message-ID: <20261111180922.4177927-4-dwmw2@infradead.org> (raw)
In-Reply-To: <20261111180922.4177927-1-dwmw2@infradead.org>

From: David Woodhouse <dwmw@amazon.co.uk>

Comparing nr_vcpus_matched_tsc against kvm->online_vcpus at read time
was always racy — a new vCPU can appear between the increment and its
first synchronization — and it made "all vCPUs matched" an accident of
arithmetic rather than a property with defined semantics. Replace it
with a boolean maintained at synchronization time under tsc_write_lock:
set in kvm_track_tsc_matching(), cleared when a new generation opens,
consumed by pvclock_update_vm_gtod_copy(). The count remains as the
input from which the flag is computed, and for the trace event.

The consistent interface is the point. The flag starts true at VM
creation, so a VM whose vCPUs never synchronize — a protected guest
TSC (TDX, SNP Secure TSC) returns early from __kvm_synchronize_tsc() —
is eligible for PVCLOCK_TSC_STABLE_BIT by design rather than by luck
of the read-time arithmetic, and use_master_clock is true from VM
creation on a TSC-clocksource host, before any vCPU exists; later
patches build on exactly that. The default scaling ratio must then be
initialized too (a zero ratio zeroes tsc_hz and turns the master clock
straight back off), and the khz/ratio bookkeeping recorded even for a
protected guest TSC: it is host-side state, which a scaled protected
guest must not leave at the host's unscaled frequency.

The cached state needs care at vCPU creation: the new vCPU is counted
in online_vcpus before it is synchronized, and userspace taking
vcpu->mutex first can hold that window open across a whole first
KVM_RUN. Clear the matched state up front in postcreate — the old
read-time comparison failed conservatively false throughout — and let
the synchronization (or a racing host-initiated TSC write on the
already-live fd) re-latch it; a postcreate aborted by a fatal signal
just propagates whatever the state then is with a masterclock update.
Only the instants between the online_vcpus increment and the clear
remain stale-true.

The explicit clear at new-generation start is currently overwritten by
the unconditional recompute in kvm_track_tsc_matching(); it is
preparation for later patches which consume the previous value.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 arch/x86/include/asm/kvm_host.h |  6 ++-
 arch/x86/kvm/x86.c              | 76 +++++++++++++++++++++++++++------
 2 files changed, 67 insertions(+), 15 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index ea48b1949316..18d7bb9fd8d2 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1232,8 +1232,9 @@ struct kvm_arch {
 	s64 kvmclock_offset;
 
 	/*
-	 * This also protects nr_vcpus_matched_tsc which is read from a
-	 * preemption-disabled region, so it must be a raw spinlock.
+	 * This also protects nr_vcpus_matched_tsc and
+	 * all_vcpus_matched_tsc, which are read from preemption-disabled
+	 * regions, so it must be a raw spinlock.
 	 */
 	raw_spinlock_t tsc_write_lock;
 	u32 cur_tsc_khz;
@@ -1242,6 +1243,7 @@ struct kvm_arch {
 	u64 cur_tsc_write;
 	u64 cur_tsc_offset;
 	u64 cur_tsc_generation;
+	bool all_vcpus_matched_tsc;
 	int nr_vcpus_matched_tsc;
 
 	u32 default_tsc_khz;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 12f465040efc..1bb0136042a6 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -1125,8 +1125,10 @@ static void kvm_track_tsc_matching(struct kvm_vcpu *vcpu, bool new_generation)
 	 * and all vCPUs must have matching TSCs.  Note, the count for matching
 	 * vCPUs doesn't include the reference vCPU, hence "+1".
 	 */
-	bool use_master_clock = (ka->nr_vcpus_matched_tsc + 1 ==
-				 atomic_read(&vcpu->kvm->online_vcpus)) &&
+	ka->all_vcpus_matched_tsc = (ka->nr_vcpus_matched_tsc + 1 ==
+				     atomic_read(&vcpu->kvm->online_vcpus));
+
+	bool use_master_clock = ka->all_vcpus_matched_tsc &&
 				gtod_is_based_on_tsc(gtod->clock.vclock_mode);
 
 	/*
@@ -1327,20 +1329,23 @@ static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc,
 
 	lockdep_assert_held(&kvm->arch.tsc_write_lock);
 
-	if (vcpu->arch.guest_tsc_protected)
-		return;
-
-	if (user_set_tsc)
-		vcpu->kvm->arch.user_set_tsc = true;
-
 	/*
-	 * Track the TSC frequency, scaling ratio, and offset for the current
+	 * Track the TSC frequency and scaling ratio for the current
 	 * generation. These are used to detect matching TSC writes and to
-	 * compute the guest TSC from the host clock.
+	 * compute the guest TSC from the host clock. This is host-side
+	 * bookkeeping, recorded even for a protected guest TSC — a scaled
+	 * protected guest (TDX, SNP Secure TSC) must not leave the master
+	 * clock computing with the host's unscaled frequency.
 	 */
 	kvm->arch.cur_tsc_khz = vcpu->arch.virtual_tsc_khz;
 	kvm->arch.cur_tsc_scaling_ratio = vcpu->arch.l1_tsc_scaling_ratio;
 
+	if (vcpu->arch.guest_tsc_protected)
+		return;
+
+	if (user_set_tsc)
+		vcpu->kvm->arch.user_set_tsc = true;
+
 	vcpu->arch.last_guest_tsc = tsc;
 
 	/*
@@ -1367,6 +1372,7 @@ static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc,
 		kvm->arch.cur_tsc_write = tsc;
 		kvm->arch.cur_tsc_offset = offset;
 		kvm->arch.nr_vcpus_matched_tsc = 0;
+		kvm->arch.all_vcpus_matched_tsc = false;
 	} else if (vcpu->arch.this_tsc_generation != kvm->arch.cur_tsc_generation) {
 		kvm->arch.nr_vcpus_matched_tsc++;
 	}
@@ -1593,8 +1599,7 @@ static void pvclock_update_vm_gtod_copy(struct kvm *kvm)
 	bool host_tsc_clocksource, vcpus_matched;
 
 	lockdep_assert_held(&kvm->arch.tsc_write_lock);
-	vcpus_matched = (ka->nr_vcpus_matched_tsc + 1 ==
-			atomic_read(&kvm->online_vcpus));
+	vcpus_matched = ka->all_vcpus_matched_tsc;
 
 	/*
 	 * If the host uses TSC clock, then passthrough TSC as stable
@@ -9584,8 +9589,38 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
 
 void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
 {
-	if (mutex_lock_killable(&vcpu->mutex))
+	struct kvm_arch *ka = &vcpu->kvm->arch;
+	unsigned long flags;
+
+	/*
+	 * The new vCPU is already counted in online_vcpus but has not
+	 * been synchronized. Clear the cached matched state before
+	 * trying the mutex: the old read-time comparison failed
+	 * conservatively false throughout this window, and userspace
+	 * can hold it open across a whole first KVM_RUN by taking
+	 * vcpu->mutex before we do. The synchronization below (or any
+	 * racing host-initiated TSC write on the already-live fd)
+	 * re-latches the flags. A protected guest TSC never
+	 * participates in offset matching, so for it there is nothing
+	 * to clear — and no later sync to re-latch.
+	 */
+	if (!vcpu->arch.guest_tsc_protected) {
+		raw_spin_lock_irqsave(&ka->tsc_write_lock, flags);
+		WRITE_ONCE(ka->all_vcpus_matched_tsc, false);
+		raw_spin_unlock_irqrestore(&ka->tsc_write_lock, flags);
+	}
+
+	if (mutex_lock_killable(&vcpu->mutex)) {
+		/*
+		 * Aborted by a fatal signal: the synchronization never
+		 * ran. Propagate whatever the matched state now is (a
+		 * racing TSC write may have re-latched it), so the
+		 * master clock does not stay in its previous mode with
+		 * running siblings keeping PVCLOCK_TSC_STABLE_BIT.
+		 */
+		kvm_update_masterclock(vcpu->kvm);
 		return;
+	}
 	vcpu_load(vcpu);
 	kvm_synchronize_tsc(vcpu, NULL);
 	if (kvm_check_request(KVM_REQ_MASTERCLOCK_UPDATE, vcpu))
@@ -10061,6 +10096,21 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	ratelimit_set_flags(&kvm->arch.kvmclock_update_rs, RATELIMIT_MSG_ON_RELEASE);
 	kvm->arch.kvmclock_offset = -get_kvmclock_base_ns();
 
+	/*
+	 * Start with all (zero) vCPUs considered matched, so that a VM whose
+	 * vCPUs never reach kvm_track_tsc_matching() — notably when the guest
+	 * TSC is protected (TDX, SNP Secure TSC) and __kvm_synchronize_tsc()
+	 * returns early — is still eligible for PVCLOCK_TSC_STABLE_BIT. The
+	 * scaling ratio must then also be initialized: with vcpus_matched
+	 * now true here, pvclock_update_vm_gtod_copy() reaches the scaling
+	 * path, and a zero ratio would zero tsc_hz and turn the master
+	 * clock straight back off (permanently so for protected-TSC guests,
+	 * whose syncs return before kvm_track_tsc_matching() ever requests
+	 * the masterclock update that would recompute it).
+	 */
+	kvm->arch.all_vcpus_matched_tsc = true;
+	kvm->arch.cur_tsc_scaling_ratio = kvm_caps.default_tsc_scaling_ratio;
+
 	raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags);
 	pvclock_update_vm_gtod_copy(kvm);
 	raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags);
-- 
2.43.0


  parent reply	other threads:[~2026-11-11 18:09 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-11 18:08 [PATCH v13 00/26] KVM: x86: Cleaning up the KVM clock mess, part 2 David Woodhouse
2026-11-11 18:08 ` [PATCH v13 01/26] KVM: x86: Improve synchronization in kvm_synchronize_tsc() David Woodhouse
2026-11-11 18:08 ` [PATCH v13 02/26] KVM: x86: Kill last_tsc_{nsec,write,offset} fields David Woodhouse
2026-11-11 18:08 ` David Woodhouse [this message]
2026-11-11 18:09 ` [PATCH v13 04/26] KVM: x86: Allow KVM master clock mode when TSCs are offset from each other David Woodhouse
2026-11-11 18:09 ` [PATCH v13 05/26] KVM: x86: Factor out kvm_use_master_clock() David Woodhouse
2026-11-11 18:09 ` [PATCH v13 06/26] KVM: x86: Avoid gratuitous global clock updates David Woodhouse
2026-11-11 18:09 ` [PATCH v13 07/26] KVM: x86: Don't latch the Hyper-V TSC page from a superseded snapshot David Woodhouse
2026-11-11 18:09 ` [PATCH v13 08/26] KVM: x86: Avoid redundant masterclock updates from multiple vCPUs David Woodhouse
2026-11-11 18:09 ` [PATCH v13 09/26] KVM: x86: Stop TSC catchup when a frequency change no longer needs it David Woodhouse
2026-11-11 18:09 ` [PATCH v13 10/26] KVM: x86: Re-synchronize TSC after KVM_SET_TSC_KHZ David Woodhouse
2026-11-11 18:09 ` [PATCH v13 11/26] KVM: x86: Cache host vclock_mode for masterclock eligibility checks David Woodhouse
2026-11-11 18:09 ` [PATCH v13 12/26] KVM: x86: Remove pvclock_gtod_data and private timekeeping code David Woodhouse
2026-11-11 18:09 ` [PATCH v13 13/26] KVM: x86: Keep the master clock trustworthy while the VM has no vCPUs David Woodhouse
2026-11-11 18:09 ` [PATCH v13 14/26] UAPI: x86: Move pvclock-abi to UAPI for x86 platforms David Woodhouse
2026-11-11 18:09 ` [PATCH v13 15/26] KVM: selftests: Use UAPI pvclock-abi.h in xen_shinfo_test David Woodhouse
2026-11-11 18:09 ` [PATCH v13 16/26] KVM: x86: Add KVM_[GS]ET_CLOCK_GUEST for accurate KVM clock migration David Woodhouse
2026-11-11 18:09 ` [PATCH v13 17/26] KVM: selftests: Add library helpers for clock and TSC-offset ioctls David Woodhouse
2026-11-11 18:09 ` [PATCH v13 18/26] KVM: x86: Add KVM_VCPU_TSC_SCALE and fix the documentation on TSC migration David Woodhouse
2026-11-11 18:09 ` [PATCH v13 19/26] KVM: selftests: Add KVM/PV clock selftest to prove timer correction David Woodhouse
2026-11-11 18:09 ` [PATCH v13 20/26] KVM: selftests: Add master clock offset test David Woodhouse
2026-11-11 18:09 ` [PATCH v13 21/26] KVM: selftests: Add Xen/generic CPUID timing leaf test David Woodhouse
2026-11-11 18:09 ` [PATCH v13 22/26] KVM: selftests: Add Xen runstate migration test David Woodhouse
2026-11-11 18:09 ` [PATCH v13 23/26] KVM: selftests: Test that the first user TSC write is honoured exactly David Woodhouse
2026-11-11 18:09 ` [PATCH v13 24/26] KVM: x86/xen: Drop open-coded fallback clock read in kvm_xen_start_timer() David Woodhouse
2026-11-11 18:09 ` [PATCH v13 25/26] KVM: x86: Use a single time snapshot in kvm_vm_ioctl_set_clock() David Woodhouse
2026-11-11 18:09 ` [PATCH v13 26/26] KVM: x86: Keep master clock for guests using legacy MSR_KVM_SYSTEM_TIME David Woodhouse

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261111180922.4177927-4-dwmw2@infradead.org \
    --to=dwmw2@infradead.org \
    --cc=Sascha.Bischoff@arm.com \
    --cc=boris.ostrovsky@oracle.com \
    --cc=bp@alien8.de \
    --cc=corbet@lwn.net \
    --cc=dave.hansen@linux.intel.com \
    --cc=dongli.zhang@oracle.com \
    --cc=dwmw@amazon.co.uk \
    --cc=hpa@zytor.com \
    --cc=jalliste@amazon.com \
    --cc=jgross@suse.com \
    --cc=jic23@kernel.org \
    --cc=joe.jin@oracle.com \
    --cc=joey.gouly@arm.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=mingo@redhat.com \
    --cc=paul@xen.org \
    --cc=pbonzini@redhat.com \
    --cc=seanjc@google.com \
    --cc=skhan@linuxfoundation.org \
    --cc=tglx@kernel.org \
    --cc=vkuznets@redhat.com \
    --cc=x86@kernel.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®