From: David Woodhouse <dwmw2@infradead.org>
To: kvm@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
linux-doc@vger.kernel.org, xen-devel@lists.xenproject.org,
x86@kernel.org, Paolo Bonzini <pbonzini@redhat.com>,
Sean Christopherson <seanjc@google.com>,
Paul Durrant <paul@xen.org>,
Vitaly Kuznetsov <vkuznets@redhat.com>,
Thomas Gleixner <tglx@kernel.org>, Ingo Molnar <mingo@redhat.com>,
Borislav Petkov <bp@alien8.de>,
Dave Hansen <dave.hansen@linux.intel.com>,
"H . Peter Anvin" <hpa@zytor.com>,
Jonathan Corbet <corbet@lwn.net>,
Shuah Khan <skhan@linuxfoundation.org>,
Juergen Gross <jgross@suse.com>,
Boris Ostrovsky <boris.ostrovsky@oracle.com>,
Jonathan Cameron <jic23@kernel.org>,
Sascha Bischoff <Sascha.Bischoff@arm.com>,
Marc Zyngier <maz@kernel.org>, Joey Gouly <joey.gouly@arm.com>,
Jack Allister <jalliste@amazon.com>,
Dongli Zhang <dongli.zhang@oracle.com>,
joe.jin@oracle.com, David Woodhouse <dwmw@amazon.co.uk>
Subject: [PATCH v13 23/26] KVM: selftests: Test that the first user TSC write is honoured exactly
Date: Wed, 11 Nov 2026 10:09:19 -0800 [thread overview]
Message-ID: <20261111180922.4177927-24-dwmw2@infradead.org> (raw)
In-Reply-To: <20261111180922.4177927-1-dwmw2@infradead.org>
From: David Woodhouse <dwmw@amazon.co.uk>
The 1-second "slop" heuristic in kvm_synchronize_tsc() is only
supposed to apply between successive userspace TSC writes, i.e. once
kvm->arch.user_set_tsc has been set by a previous write. The kernel's
own internal TSC synchronization on vCPU creation (and frequency
changes) must not arm it, otherwise the first genuine userspace write
falling within 1 second of virtual cycle time of the kernel's default
generation is "corrected" back to that generation instead of being
honoured — breaking the restore of a guest with less than one second
of runtime.
Add a test which creates a vCPU and immediately performs a first
host-initiated write of a TSC value corresponding to ~500ms of guest
runtime, then verifies the value reads back exactly (modulo elapsed
time) rather than being snapped back to the kernel's default
generation near zero.
In the frequency-change case, a buggy kernel carries the creation-rate
TSC across the change, where (expressed at the halved rate) it reads as
the creation-to-change plus creation-to-read time; a further timing
gate keeps that sum comfortably below the written 500ms so a
snapped-back value can never sit above the written one.
Both cases report through the kselftest TAP machinery; an
inconclusive case (host timing repeatedly too slow, a property of
host load rather than of the kernel under test) is recorded as a skip
without discarding the other case's result. A conclusively buggy
kernel still fails via TEST_ASSERT before any skip is reachable.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
tools/testing/selftests/kvm/Makefile.kvm | 1 +
.../selftests/kvm/x86/tsc_user_set_test.c | 193 ++++++++++++++++++
2 files changed, 194 insertions(+)
create mode 100644 tools/testing/selftests/kvm/x86/tsc_user_set_test.c
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 1edc78845fd3..fd063806a7fb 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -151,6 +151,7 @@ TEST_GEN_PROGS_x86 += x86/xcr0_cpuid_test
TEST_GEN_PROGS_x86 += x86/xss_msr_test
TEST_GEN_PROGS_x86 += x86/debug_regs
TEST_GEN_PROGS_x86 += x86/tsc_msrs_test
+TEST_GEN_PROGS_x86 += x86/tsc_user_set_test
TEST_GEN_PROGS_x86 += x86/vmx_pmu_caps_test
TEST_GEN_PROGS_x86 += x86/xen_cpuid_timing_test
TEST_GEN_PROGS_x86 += x86/xen_migration_test
diff --git a/tools/testing/selftests/kvm/x86/tsc_user_set_test.c b/tools/testing/selftests/kvm/x86/tsc_user_set_test.c
new file mode 100644
index 000000000000..5e6eb0a26c8e
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/tsc_user_set_test.c
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Test that the first userspace write to MSR_IA32_TSC takes effect
+ * precisely, and is not subjected to the 1-second "slop" heuristic
+ * which is only supposed to apply between successive *userspace*
+ * writes (kvm->arch.user_set_tsc).
+ *
+ * The kernel synchronizes the TSC internally when a vCPU is created
+ * (and when its frequency is set). If any of those kernel-internal
+ * paths incorrectly sets user_set_tsc, then the first genuine write
+ * from userspace which happens to fall within 1 second of virtual
+ * cycle time of the kernel's default generation is "corrected" to
+ * match that generation instead of being honoured exactly. That
+ * breaks the restore of a guest which has genuinely been running
+ * for less than one second.
+ *
+ * Simulate exactly that: create a vCPU (kernel establishes a default
+ * TSC generation starting near zero), then perform the first
+ * host-initiated write with a value corresponding to ~500ms of guest
+ * runtime. If the slop heuristic fires, the written value is snapped
+ * back to the kernel's generation (near zero) and the TSC reads back
+ * hundreds of milliseconds low. Repeat with an explicit
+ * KVM_SET_TSC_KHZ on the never-run vCPU before the write: that is
+ * another kernel-internal synchronization which must not arm the
+ * heuristic (nor open a generation that swallows the first write).
+ */
+#include <time.h>
+
+#include "kvm_util.h"
+#include "processor.h"
+
+/*
+ * Returns false when the attempt was inconclusive: a snapped-back value
+ * tracks time since vCPU creation, so if creation-to-write time gets
+ * anywhere near the written 500ms, a buggy kernel's value could land
+ * inside the assertions' tolerance and pass. The caller retries.
+ */
+static bool test_first_write(bool set_freq)
+{
+ struct kvm_vcpu *vcpu;
+ struct kvm_vm *vm;
+ u64 freq_khz, t_write, t_read, slack_cycles, mono_ns, create_ns, read_ns;
+ u64 freq_ns;
+ struct timespec ts;
+ int r;
+
+ clock_gettime(CLOCK_MONOTONIC_RAW, &ts);
+ create_ns = ts.tv_sec * 1000000000ULL + ts.tv_nsec;
+ freq_ns = create_ns;
+
+ vm = vm_create_with_one_vcpu(&vcpu, NULL);
+
+ if (set_freq) {
+ /*
+ * Halve the frequency so this is a real change, exercising
+ * the frequency resynchronization path rather than the
+ * unchanged-frequency no-op.
+ */
+ r = __vcpu_ioctl(vcpu, KVM_GET_TSC_KHZ, NULL);
+ TEST_ASSERT(r > 0, "KVM_GET_TSC_KHZ failed");
+ vcpu_ioctl(vcpu, KVM_SET_TSC_KHZ,
+ (void *)(unsigned long)(r / 2));
+ clock_gettime(CLOCK_MONOTONIC_RAW, &ts);
+ freq_ns = ts.tv_sec * 1000000000ULL + ts.tv_nsec;
+ }
+
+ r = __vcpu_ioctl(vcpu, KVM_GET_TSC_KHZ, NULL);
+ TEST_ASSERT(r > 0, "KVM_GET_TSC_KHZ failed");
+ freq_khz = r;
+
+ /*
+ * The first user write: a guest TSC corresponding to 500ms of
+ * runtime. This is within 1s (of virtual cycle time) of the
+ * kernel's default generation, so it is exactly the case that
+ * the slop heuristic would destroy if it were (incorrectly)
+ * armed by the kernel's own internal TSC synchronization.
+ */
+ t_write = freq_khz * 500ULL; /* khz * ms = cycles */
+
+ /*
+ * Measure only the write..read window (a window measured from
+ * before creation would also bound a snapped-back value), and
+ * insist the write lands well before 500ms of creation time so
+ * a snapped-back value cannot sit inside the tolerance.
+ */
+ clock_gettime(CLOCK_MONOTONIC_RAW, &ts);
+ mono_ns = ts.tv_sec * 1000000000ULL + ts.tv_nsec;
+
+ if (mono_ns - create_ns > 250000000ULL) {
+ pr_info(" creation took %lu ms, retrying for a conclusive run\n",
+ (mono_ns - create_ns) / 1000000);
+ kvm_vm_free(vm);
+ return false;
+ }
+
+ vcpu_set_msr(vcpu, MSR_IA32_TSC, t_write);
+
+ t_read = vcpu_get_msr(vcpu, MSR_IA32_TSC);
+
+ clock_gettime(CLOCK_MONOTONIC_RAW, &ts);
+ read_ns = ts.tv_sec * 1000000000ULL + ts.tv_nsec;
+
+ /*
+ * Bound the whole creation..read interval too: a snapped-back
+ * value tracks it, so letting it approach the written 500ms
+ * (via a stall between the two MSR ioctls, which the slack
+ * below would otherwise absorb) would let a buggy kernel pass.
+ */
+ if (read_ns - create_ns > 400000000ULL) {
+ pr_info(" creation..read took %lu ms, retrying for a conclusive run\n",
+ (read_ns - create_ns) / 1000000);
+ kvm_vm_free(vm);
+ return false;
+ }
+
+ /*
+ * In the set_freq case a buggy kernel carries the creation-rate
+ * TSC across the frequency change, where it reads (expressed at
+ * the halved rate) as the creation..freq-change plus the
+ * creation..read time in milliseconds. Keep that sum comfortably
+ * below the written 500ms, or a snapped-back value could sit
+ * above t_write and pass the assertions.
+ */
+ if ((freq_ns - create_ns) + (read_ns - create_ns) > 430000000ULL) {
+ pr_info(" freq-change at %lu ms + read at %lu ms, retrying for a conclusive run\n",
+ (freq_ns - create_ns) / 1000000,
+ (read_ns - create_ns) / 1000000);
+ kvm_vm_free(vm);
+ return false;
+ }
+
+ mono_ns = read_ns - mono_ns;
+
+ /*
+ * The TSC advances between the write and the read; bound the
+ * difference by the measured elapsed time around the two ioctls
+ * plus 10ms of slop, so a descheduled test cannot flake. If the
+ * write was snapped to the kernel's default generation, the
+ * value read back is ~500ms low (i.e. below t_write), which is
+ * unambiguous.
+ */
+ slack_cycles = freq_khz * (mono_ns / 1000000ULL + 10);
+
+ TEST_ASSERT(t_read >= t_write,
+ "TSC went backwards from explicit write: wrote %lu, read %lu"
+ " (first userspace write was subjected to 1-second slop?)",
+ t_write, t_read);
+ TEST_ASSERT(t_read - t_write < slack_cycles,
+ "TSC read back too far from written value: wrote %lu, read %lu",
+ t_write, t_read);
+
+ kvm_vm_free(vm);
+ return true;
+}
+
+static void run_first_write(bool set_freq)
+{
+ const char *name = set_freq ? "first write after KVM_SET_TSC_KHZ"
+ : "first write";
+ int i;
+
+ for (i = 0; i < 5; i++) {
+ if (test_first_write(set_freq)) {
+ ksft_test_result_pass("%s\n", name);
+ return;
+ }
+ }
+
+ /*
+ * Inconclusive runs — either gate: slow vCPU creation, or a
+ * stall between the MSR write and read — reflect host
+ * scheduling load, not the behaviour under test; record a skip
+ * rather than report a kernel regression, without discarding a
+ * conclusive result the other case already established.
+ */
+ ksft_test_result_skip("%s: host timing repeatedly too slow for a conclusive run\n",
+ name);
+}
+
+int main(void)
+{
+ ksft_print_header();
+ ksft_set_plan(2);
+
+ run_first_write(false);
+
+ if (kvm_has_cap(KVM_CAP_TSC_CONTROL))
+ run_first_write(true);
+ else
+ ksft_test_result_skip("first write after KVM_SET_TSC_KHZ: no KVM_CAP_TSC_CONTROL\n");
+
+ ksft_finished();
+}
--
2.43.0
next prev parent reply other threads:[~2026-11-11 18:09 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-11 18:08 [PATCH v13 00/26] KVM: x86: Cleaning up the KVM clock mess, part 2 David Woodhouse
2026-11-11 18:08 ` [PATCH v13 01/26] KVM: x86: Improve synchronization in kvm_synchronize_tsc() David Woodhouse
2026-11-11 18:08 ` [PATCH v13 02/26] KVM: x86: Kill last_tsc_{nsec,write,offset} fields David Woodhouse
2026-11-11 18:08 ` [PATCH v13 03/26] KVM: x86: Replace nr_vcpus_matched_tsc count with all_vcpus_matched_tsc bool David Woodhouse
2026-11-11 18:09 ` [PATCH v13 04/26] KVM: x86: Allow KVM master clock mode when TSCs are offset from each other David Woodhouse
2026-11-11 18:09 ` [PATCH v13 05/26] KVM: x86: Factor out kvm_use_master_clock() David Woodhouse
2026-11-11 18:09 ` [PATCH v13 06/26] KVM: x86: Avoid gratuitous global clock updates David Woodhouse
2026-11-11 18:09 ` [PATCH v13 07/26] KVM: x86: Don't latch the Hyper-V TSC page from a superseded snapshot David Woodhouse
2026-11-11 18:09 ` [PATCH v13 08/26] KVM: x86: Avoid redundant masterclock updates from multiple vCPUs David Woodhouse
2026-11-11 18:09 ` [PATCH v13 09/26] KVM: x86: Stop TSC catchup when a frequency change no longer needs it David Woodhouse
2026-11-11 18:09 ` [PATCH v13 10/26] KVM: x86: Re-synchronize TSC after KVM_SET_TSC_KHZ David Woodhouse
2026-11-11 18:09 ` [PATCH v13 11/26] KVM: x86: Cache host vclock_mode for masterclock eligibility checks David Woodhouse
2026-11-11 18:09 ` [PATCH v13 12/26] KVM: x86: Remove pvclock_gtod_data and private timekeeping code David Woodhouse
2026-11-11 18:09 ` [PATCH v13 13/26] KVM: x86: Keep the master clock trustworthy while the VM has no vCPUs David Woodhouse
2026-11-11 18:09 ` [PATCH v13 14/26] UAPI: x86: Move pvclock-abi to UAPI for x86 platforms David Woodhouse
2026-11-11 18:09 ` [PATCH v13 15/26] KVM: selftests: Use UAPI pvclock-abi.h in xen_shinfo_test David Woodhouse
2026-11-11 18:09 ` [PATCH v13 16/26] KVM: x86: Add KVM_[GS]ET_CLOCK_GUEST for accurate KVM clock migration David Woodhouse
2026-11-11 18:09 ` [PATCH v13 17/26] KVM: selftests: Add library helpers for clock and TSC-offset ioctls David Woodhouse
2026-11-11 18:09 ` [PATCH v13 18/26] KVM: x86: Add KVM_VCPU_TSC_SCALE and fix the documentation on TSC migration David Woodhouse
2026-11-11 18:09 ` [PATCH v13 19/26] KVM: selftests: Add KVM/PV clock selftest to prove timer correction David Woodhouse
2026-11-11 18:09 ` [PATCH v13 20/26] KVM: selftests: Add master clock offset test David Woodhouse
2026-11-11 18:09 ` [PATCH v13 21/26] KVM: selftests: Add Xen/generic CPUID timing leaf test David Woodhouse
2026-11-11 18:09 ` [PATCH v13 22/26] KVM: selftests: Add Xen runstate migration test David Woodhouse
2026-11-11 18:09 ` David Woodhouse [this message]
2026-11-11 18:09 ` [PATCH v13 24/26] KVM: x86/xen: Drop open-coded fallback clock read in kvm_xen_start_timer() David Woodhouse
2026-11-11 18:09 ` [PATCH v13 25/26] KVM: x86: Use a single time snapshot in kvm_vm_ioctl_set_clock() David Woodhouse
2026-11-11 18:09 ` [PATCH v13 26/26] KVM: x86: Keep master clock for guests using legacy MSR_KVM_SYSTEM_TIME David Woodhouse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261111180922.4177927-24-dwmw2@infradead.org \
--to=dwmw2@infradead.org \
--cc=Sascha.Bischoff@arm.com \
--cc=boris.ostrovsky@oracle.com \
--cc=bp@alien8.de \
--cc=corbet@lwn.net \
--cc=dave.hansen@linux.intel.com \
--cc=dongli.zhang@oracle.com \
--cc=dwmw@amazon.co.uk \
--cc=hpa@zytor.com \
--cc=jalliste@amazon.com \
--cc=jgross@suse.com \
--cc=jic23@kernel.org \
--cc=joe.jin@oracle.com \
--cc=joey.gouly@arm.com \
--cc=kvm@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=maz@kernel.org \
--cc=mingo@redhat.com \
--cc=paul@xen.org \
--cc=pbonzini@redhat.com \
--cc=seanjc@google.com \
--cc=skhan@linuxfoundation.org \
--cc=tglx@kernel.org \
--cc=vkuznets@redhat.com \
--cc=x86@kernel.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®