mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: David Woodhouse <dwmw2@infradead.org>
To: Sean Christopherson <seanjc@google.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
	Jonathan Corbet <corbet@lwn.net>,
	 Shuah Khan <skhan@linuxfoundation.org>,
	Thomas Gleixner <tglx@kernel.org>, Ingo Molnar <mingo@redhat.com>,
	 Borislav Petkov	 <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	x86@kernel.org, "H. Peter Anvin"	 <hpa@zytor.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Juergen Gross	 <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Paul Durrant	 <paul@xen.org>, Jonathan Cameron <jic23@kernel.org>,
	Sascha Bischoff	 <Sascha.Bischoff@arm.com>,
	Marc Zyngier <maz@kernel.org>, Joey Gouly	 <joey.gouly@arm.com>,
	Jack Allister <jalliste@amazon.com>,
	Dongli Zhang	 <dongli.zhang@oracle.com>,
	joe.jin@oracle.com, kvm@vger.kernel.org,
	 linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
	 xen-devel@lists.xenproject.org, linux-kselftest@vger.kernel.org
Subject: Re: [PATCH v7 17/36] KVM: x86: Allow KVM master clock mode when TSCs are offset from each other
Date: Tue, 11 Aug 2026 16:05:03 +0100	[thread overview]
Message-ID: <e256fa4af96e916ba30019cbba501fa896025fb0.camel@infradead.org> (raw)
In-Reply-To: <ansywxh0VX5rtfWc@google.com>

[-- Attachment #1: Type: text/plain, Size: 3633 bytes --]

On Tue, 2026-08-11 at 07:33 -0700, Sean Christopherson wrote:
> 
> Actually, why are KVM_{G,S}ET_CLOCK_GUEST vCPU-scoped?  Per the documentation,
> the API "Sets the KVM clock (for the whole VM) in terms of the vCPU TSC".  If
> the APIs are VM-scoped instead of vCPU-scoped, then KVM can simply save/restore
> what's in the per-VM masterclock state, no?

They're vCPU-scoped because they need to be tied to a guest TSC (on
live migration, neither ka->master_cycle_now nor ka->master_kernel_ns
are useful — those are the "per-VM masterclock state").

Theoretically, guest TSCs can be different on each vCPU (different
offset, different *rate* even. Not that we allow KVM_[GS]ET_CLOCK_GUEST
at different rates, I concede).

So they operate in the context of a given vCPU, and *its* TSC.

And I think I'm going to defend that 'theoretical they can be
different', because I *would* like to eliminate the ways that a *guest*
can force non-masterclock mode, and that does mean allowing the offset-
TSC case.

FWIW in my local tree I've just extended the pvclock_migration_test to
test precisely the thing you were concerned about: three vCPUs with
divergent TSC offsets, migrated by setting each vCPU's TSC and then
invoking KVM_SET_CLOCK_GUEST once, through vCPU0. Masterclock stays
active, TSC_STABLE_BIT is correctly clear, and all three vCPUs'
pvclocks (and KVM_GET_CLOCK) agree to within a nanosecond afterwards.
I'll include that in the next spin.

> That would also help address my concerns about sanity checking the TSC frequency
> against the kvmclock frequency, as the APIs are much more blatantly about saving
> and restoring masterclock state.  For whatever reason, it feels more natural for
> me to say that KVM_SET_CLOCK_GUEST will fail if the target frequency doesn't
> (fuzzily?) match the frequency at which the masterclock is already configured.
> Probably because use_master_clock directly gates that information?  Whereas the
> vCPU's frequency is independently configured but obviously influences masterclock
> mode.

I am perfectly happy to say that the *existing* check as I have coded
it, is matching against the frequency at which the masterclock is
configured. Because we can't *get* there if the guest is not in
masterclock mode, and it can't be in masterclock mode unless all its
vCPUs are running at the same rate, which *is* the master clock rate.
:)

I guess I could even concede to change the actual code rather than just
the comment... (untested)

Still needs the *offset* of the vCPU though.

--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3595,15 +3595,15 @@ static int kvm_vcpu_ioctl_set_clock_guest(struct kvm_vcpu *v, void __user *argp

 	if (kvm_caps.has_tsc_control)
 		curr_tsc_hz = kvm_scale_tsc(curr_tsc_hz,
-					    v->arch.l1_tsc_scaling_ratio);
+					    ka->master_tsc_scaling_ratio);

 	/*
 	 * The mul/shift in the provided pvclock structure encode the guest
 	 * TSC frequency at which it was generated. Sanity-check that it is
-	 * consistent with this vCPU's effective TSC frequency, allowing a
-	 * discrepancy of 1 kHz either way since independently calibrated
-	 * hosts will not measure precisely the same value even for the
-	 * same nominal frequency.
+	 * consistent with the frequency at which the masterclock is
+	 * configured, allowing a discrepancy of 1 kHz either way since
+	 * independently calibrated hosts will not measure precisely the
+	 * same value even for the same nominal frequency.
 	 */
 	if (user_tsc_hz < curr_tsc_hz - 1000 ||
 	    user_tsc_hz > curr_tsc_hz + 1000) {


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]

  reply	other threads:[~2026-08-11 15:05 UTC|newest]

Thread overview: 74+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-28 14:39 [PATCH v7 00/36] Cleaning up the KVM clock mess David Woodhouse
2026-07-28 14:39 ` [PATCH v7 01/36] KVM: x86: Improve accuracy of KVM clock when TSC scaling is in force David Woodhouse
2026-07-28 14:39 ` [PATCH v7 02/36] KVM: x86: Explicitly disable TSC scaling without CONSTANT_TSC David Woodhouse
2026-07-28 14:39 ` [PATCH v7 03/36] KVM: x86: Activate master clock immediately on vCPU creation David Woodhouse
2026-07-28 14:39 ` [PATCH v7 04/36] KVM: x86: Avoid NTP frequency skew for KVM clock on 32-bit host David Woodhouse
2026-07-28 14:39 ` [PATCH v7 05/36] KVM: x86: Fold __get_kvmclock() into get_kvmclock() David Woodhouse
2026-07-28 23:06   ` Sean Christopherson
2026-07-28 14:39 ` [PATCH v7 06/36] KVM: x86: Drop CPU pinning in get_kvmclock() David Woodhouse
2026-07-28 14:39 ` [PATCH v7 07/36] KVM: x86: Restructure get_kvmclock() David Woodhouse
2026-07-28 23:04   ` Sean Christopherson
2026-07-28 14:39 ` [PATCH v7 08/36] KVM: x86: Fix KVM clock precision in get_kvmclock() with TSC scaling David Woodhouse
2026-07-28 14:39 ` [PATCH v7 09/36] KVM: x86: Use get_kvmclock() in kvm_get_wall_clock_epoch() David Woodhouse
2026-07-28 14:39 ` [PATCH v7 10/36] KVM: x86: Fix compute_guest_tsc() to handle negative time deltas David Woodhouse
2026-07-28 14:39 ` [PATCH v7 11/36] KVM: x86: Restructure kvm_guest_time_update() for TSC upscaling David Woodhouse
2026-08-04  1:26   ` Sean Christopherson
2026-07-28 14:39 ` [PATCH v7 12/36] KVM: x86: Simplify and comment kvm_get_time_scale() David Woodhouse
2026-07-28 14:39 ` [PATCH v7 13/36] KVM: x86: Remove implicit rdtsc() from kvm_compute_l1_tsc_offset() David Woodhouse
2026-07-28 14:39 ` [PATCH v7 14/36] KVM: x86: Improve synchronization in kvm_synchronize_tsc() David Woodhouse
2026-07-28 14:39 ` [PATCH v7 15/36] KVM: x86: Kill last_tsc_{nsec,write,offset} fields David Woodhouse
2026-07-28 14:39 ` [PATCH v7 16/36] KVM: x86: Replace nr_vcpus_matched_tsc count with all_vcpus_matched_tsc bool David Woodhouse
2026-07-28 14:39 ` [PATCH v7 17/36] KVM: x86: Allow KVM master clock mode when TSCs are offset from each other David Woodhouse
2026-08-10 17:47   ` Sean Christopherson
2026-08-10 18:20     ` David Woodhouse
2026-08-10 20:56       ` Sean Christopherson
2026-08-10 21:05         ` David Woodhouse
2026-08-11 14:16           ` Sean Christopherson
2026-08-11 14:33             ` Sean Christopherson
2026-08-11 15:05               ` David Woodhouse [this message]
2026-08-11 16:40                 ` Sean Christopherson
2026-08-11 17:18                   ` David Woodhouse
2026-08-11 17:28                     ` Sean Christopherson
2026-08-11 17:32                       ` David Woodhouse
2026-08-11 18:41                         ` Sean Christopherson
2026-08-11 21:00                           ` David Woodhouse
2026-08-11 21:46                             ` Sean Christopherson
2026-08-11 21:56                               ` David Woodhouse
2026-08-11 23:14                                 ` Sean Christopherson
2026-08-14  7:01                                   ` David Woodhouse
2026-08-14 17:54                                     ` Sean Christopherson
2026-08-15  7:02                                       ` David Woodhouse
2026-08-13  8:44                           ` David Woodhouse
2026-07-28 14:39 ` [PATCH v7 18/36] KVM: x86: Factor out kvm_use_master_clock() David Woodhouse
2026-07-28 14:39 ` [PATCH v7 19/36] KVM: x86: Avoid gratuitous global clock updates David Woodhouse
2026-07-28 14:40 ` [PATCH v7 20/36] KVM: x86/xen: Prevent runstate times from becoming negative David Woodhouse
2026-07-28 14:40 ` [PATCH v7 21/36] KVM: x86: Avoid redundant masterclock updates from multiple vCPUs David Woodhouse
2026-07-28 14:40 ` [PATCH v7 22/36] KVM: x86: Remove runtime Xen TSC frequency CPUID update David Woodhouse
2026-07-28 14:40 ` [PATCH v7 23/36] KVM: x86: Re-synchronize TSC after KVM_SET_TSC_KHZ David Woodhouse
2026-07-28 14:40 ` [PATCH v7 24/36] KVM: x86: Use ktime_get_snapshot_id() for master clock David Woodhouse
2026-07-28 14:40 ` [PATCH v7 25/36] KVM: x86: Compute kvmclock base without pvclock_gtod_data David Woodhouse
2026-07-28 14:40 ` [PATCH v7 26/36] KVM: x86: Cache host vclock_mode for masterclock eligibility checks David Woodhouse
2026-07-28 14:40 ` [PATCH v7 27/36] KVM: x86: Remove pvclock_gtod_data and private timekeeping code David Woodhouse
2026-07-28 14:40 ` [PATCH v7 28/36] KVM: x86: Activate master clock from kvm_arch_init_vm() David Woodhouse
2026-07-28 14:40 ` [PATCH v7 29/36] UAPI: x86: Move pvclock-abi to UAPI for x86 platforms David Woodhouse
2026-07-28 14:40 ` [PATCH v7 30/36] KVM: selftests: Use UAPI pvclock-abi.h in xen_shinfo_test David Woodhouse
2026-07-28 14:40 ` [PATCH v7 31/36] KVM: x86: Add KVM_[GS]ET_CLOCK_GUEST for accurate KVM clock migration David Woodhouse
2026-07-31 23:24   ` Sean Christopherson
2026-08-01  8:07     ` David Woodhouse
2026-08-04 23:38       ` Sean Christopherson
2026-08-05  9:26         ` David Woodhouse
2026-08-11 23:40   ` Sean Christopherson
2026-08-11 23:47     ` David Woodhouse
2026-08-14  7:27     ` David Woodhouse
2026-08-28 23:26       ` David Woodhouse
2026-07-28 14:40 ` [PATCH v7 32/36] KVM: x86: Add KVM_VCPU_TSC_SCALE and fix the documentation on TSC migration David Woodhouse
2026-07-28 14:40 ` [PATCH v7 33/36] KVM: selftests: Add KVM/PV clock selftest to prove timer correction David Woodhouse
2026-07-31 23:32   ` Sean Christopherson
2026-07-28 14:40 ` [PATCH v7 34/36] KVM: selftests: Add master clock offset test David Woodhouse
2026-07-31 23:38   ` Sean Christopherson
2026-07-28 14:40 ` [PATCH v7 35/36] KVM: selftests: Add Xen/generic CPUID timing leaf test David Woodhouse
2026-07-28 14:40 ` [PATCH v7 36/36] KVM: selftests: Add Xen runstate migration test David Woodhouse
2026-07-28 23:18 ` [PATCH v7 00/36] Cleaning up the KVM clock mess Sean Christopherson
2026-07-29 10:42   ` David Woodhouse
2026-08-10 16:42 ` Sean Christopherson
2026-08-10 16:56   ` David Woodhouse

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e256fa4af96e916ba30019cbba501fa896025fb0.camel@infradead.org \
    --to=dwmw2@infradead.org \
    --cc=Sascha.Bischoff@arm.com \
    --cc=boris.ostrovsky@oracle.com \
    --cc=bp@alien8.de \
    --cc=corbet@lwn.net \
    --cc=dave.hansen@linux.intel.com \
    --cc=dongli.zhang@oracle.com \
    --cc=hpa@zytor.com \
    --cc=jalliste@amazon.com \
    --cc=jgross@suse.com \
    --cc=jic23@kernel.org \
    --cc=joe.jin@oracle.com \
    --cc=joey.gouly@arm.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=mingo@redhat.com \
    --cc=paul@xen.org \
    --cc=pbonzini@redhat.com \
    --cc=seanjc@google.com \
    --cc=skhan@linuxfoundation.org \
    --cc=tglx@kernel.org \
    --cc=vkuznets@redhat.com \
    --cc=x86@kernel.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®