mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: David Woodhouse <dwmw2@infradead.org>
To: kvm@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
	linux-doc@vger.kernel.org, xen-devel@lists.xenproject.org,
	x86@kernel.org, Paolo Bonzini <pbonzini@redhat.com>,
	Sean Christopherson <seanjc@google.com>,
	Paul Durrant <paul@xen.org>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Thomas Gleixner <tglx@kernel.org>, Ingo Molnar <mingo@redhat.com>,
	Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	"H . Peter Anvin" <hpa@zytor.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Shuah Khan <skhan@linuxfoundation.org>,
	Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Jonathan Cameron <jic23@kernel.org>,
	Sascha Bischoff <Sascha.Bischoff@arm.com>,
	Marc Zyngier <maz@kernel.org>, Joey Gouly <joey.gouly@arm.com>,
	Jack Allister <jalliste@amazon.com>,
	Dongli Zhang <dongli.zhang@oracle.com>,
	joe.jin@oracle.com, David Woodhouse <dwmw@amazon.co.uk>
Subject: [PATCH v13 23/26] KVM: selftests: Test that the first user TSC write is honoured exactly
Date: Wed, 11 Nov 2026 10:09:19 -0800	[thread overview]
Message-ID: <20261111180922.4177927-24-dwmw2@infradead.org> (raw)
In-Reply-To: <20261111180922.4177927-1-dwmw2@infradead.org>

From: David Woodhouse <dwmw@amazon.co.uk>

The 1-second "slop" heuristic in kvm_synchronize_tsc() is only
supposed to apply between successive userspace TSC writes, i.e. once
kvm->arch.user_set_tsc has been set by a previous write. The kernel's
own internal TSC synchronization on vCPU creation (and frequency
changes) must not arm it, otherwise the first genuine userspace write
falling within 1 second of virtual cycle time of the kernel's default
generation is "corrected" back to that generation instead of being
honoured — breaking the restore of a guest with less than one second
of runtime.

Add a test which creates a vCPU and immediately performs a first
host-initiated write of a TSC value corresponding to ~500ms of guest
runtime, then verifies the value reads back exactly (modulo elapsed
time) rather than being snapped back to the kernel's default
generation near zero.

In the frequency-change case, a buggy kernel carries the creation-rate
TSC across the change, where (expressed at the halved rate) it reads as
the creation-to-change plus creation-to-read time; a further timing
gate keeps that sum comfortably below the written 500ms so a
snapped-back value can never sit above the written one.

Both cases report through the kselftest TAP machinery; an
inconclusive case (host timing repeatedly too slow, a property of
host load rather than of the kernel under test) is recorded as a skip
without discarding the other case's result. A conclusively buggy
kernel still fails via TEST_ASSERT before any skip is reachable.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/x86/tsc_user_set_test.c     | 193 ++++++++++++++++++
 2 files changed, 194 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/tsc_user_set_test.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 1edc78845fd3..fd063806a7fb 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -151,6 +151,7 @@ TEST_GEN_PROGS_x86 += x86/xcr0_cpuid_test
 TEST_GEN_PROGS_x86 += x86/xss_msr_test
 TEST_GEN_PROGS_x86 += x86/debug_regs
 TEST_GEN_PROGS_x86 += x86/tsc_msrs_test
+TEST_GEN_PROGS_x86 += x86/tsc_user_set_test
 TEST_GEN_PROGS_x86 += x86/vmx_pmu_caps_test
 TEST_GEN_PROGS_x86 += x86/xen_cpuid_timing_test
 TEST_GEN_PROGS_x86 += x86/xen_migration_test
diff --git a/tools/testing/selftests/kvm/x86/tsc_user_set_test.c b/tools/testing/selftests/kvm/x86/tsc_user_set_test.c
new file mode 100644
index 000000000000..5e6eb0a26c8e
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/tsc_user_set_test.c
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Test that the first userspace write to MSR_IA32_TSC takes effect
+ * precisely, and is not subjected to the 1-second "slop" heuristic
+ * which is only supposed to apply between successive *userspace*
+ * writes (kvm->arch.user_set_tsc).
+ *
+ * The kernel synchronizes the TSC internally when a vCPU is created
+ * (and when its frequency is set). If any of those kernel-internal
+ * paths incorrectly sets user_set_tsc, then the first genuine write
+ * from userspace which happens to fall within 1 second of virtual
+ * cycle time of the kernel's default generation is "corrected" to
+ * match that generation instead of being honoured exactly. That
+ * breaks the restore of a guest which has genuinely been running
+ * for less than one second.
+ *
+ * Simulate exactly that: create a vCPU (kernel establishes a default
+ * TSC generation starting near zero), then perform the first
+ * host-initiated write with a value corresponding to ~500ms of guest
+ * runtime. If the slop heuristic fires, the written value is snapped
+ * back to the kernel's generation (near zero) and the TSC reads back
+ * hundreds of milliseconds low. Repeat with an explicit
+ * KVM_SET_TSC_KHZ on the never-run vCPU before the write: that is
+ * another kernel-internal synchronization which must not arm the
+ * heuristic (nor open a generation that swallows the first write).
+ */
+#include <time.h>
+
+#include "kvm_util.h"
+#include "processor.h"
+
+/*
+ * Returns false when the attempt was inconclusive: a snapped-back value
+ * tracks time since vCPU creation, so if creation-to-write time gets
+ * anywhere near the written 500ms, a buggy kernel's value could land
+ * inside the assertions' tolerance and pass. The caller retries.
+ */
+static bool test_first_write(bool set_freq)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	u64 freq_khz, t_write, t_read, slack_cycles, mono_ns, create_ns, read_ns;
+	u64 freq_ns;
+	struct timespec ts;
+	int r;
+
+	clock_gettime(CLOCK_MONOTONIC_RAW, &ts);
+	create_ns = ts.tv_sec * 1000000000ULL + ts.tv_nsec;
+	freq_ns = create_ns;
+
+	vm = vm_create_with_one_vcpu(&vcpu, NULL);
+
+	if (set_freq) {
+		/*
+		 * Halve the frequency so this is a real change, exercising
+		 * the frequency resynchronization path rather than the
+		 * unchanged-frequency no-op.
+		 */
+		r = __vcpu_ioctl(vcpu, KVM_GET_TSC_KHZ, NULL);
+		TEST_ASSERT(r > 0, "KVM_GET_TSC_KHZ failed");
+		vcpu_ioctl(vcpu, KVM_SET_TSC_KHZ,
+			   (void *)(unsigned long)(r / 2));
+		clock_gettime(CLOCK_MONOTONIC_RAW, &ts);
+		freq_ns = ts.tv_sec * 1000000000ULL + ts.tv_nsec;
+	}
+
+	r = __vcpu_ioctl(vcpu, KVM_GET_TSC_KHZ, NULL);
+	TEST_ASSERT(r > 0, "KVM_GET_TSC_KHZ failed");
+	freq_khz = r;
+
+	/*
+	 * The first user write: a guest TSC corresponding to 500ms of
+	 * runtime. This is within 1s (of virtual cycle time) of the
+	 * kernel's default generation, so it is exactly the case that
+	 * the slop heuristic would destroy if it were (incorrectly)
+	 * armed by the kernel's own internal TSC synchronization.
+	 */
+	t_write = freq_khz * 500ULL;		/* khz * ms = cycles */
+
+	/*
+	 * Measure only the write..read window (a window measured from
+	 * before creation would also bound a snapped-back value), and
+	 * insist the write lands well before 500ms of creation time so
+	 * a snapped-back value cannot sit inside the tolerance.
+	 */
+	clock_gettime(CLOCK_MONOTONIC_RAW, &ts);
+	mono_ns = ts.tv_sec * 1000000000ULL + ts.tv_nsec;
+
+	if (mono_ns - create_ns > 250000000ULL) {
+		pr_info("  creation took %lu ms, retrying for a conclusive run\n",
+			(mono_ns - create_ns) / 1000000);
+		kvm_vm_free(vm);
+		return false;
+	}
+
+	vcpu_set_msr(vcpu, MSR_IA32_TSC, t_write);
+
+	t_read = vcpu_get_msr(vcpu, MSR_IA32_TSC);
+
+	clock_gettime(CLOCK_MONOTONIC_RAW, &ts);
+	read_ns = ts.tv_sec * 1000000000ULL + ts.tv_nsec;
+
+	/*
+	 * Bound the whole creation..read interval too: a snapped-back
+	 * value tracks it, so letting it approach the written 500ms
+	 * (via a stall between the two MSR ioctls, which the slack
+	 * below would otherwise absorb) would let a buggy kernel pass.
+	 */
+	if (read_ns - create_ns > 400000000ULL) {
+		pr_info("  creation..read took %lu ms, retrying for a conclusive run\n",
+			(read_ns - create_ns) / 1000000);
+		kvm_vm_free(vm);
+		return false;
+	}
+
+	/*
+	 * In the set_freq case a buggy kernel carries the creation-rate
+	 * TSC across the frequency change, where it reads (expressed at
+	 * the halved rate) as the creation..freq-change plus the
+	 * creation..read time in milliseconds. Keep that sum comfortably
+	 * below the written 500ms, or a snapped-back value could sit
+	 * above t_write and pass the assertions.
+	 */
+	if ((freq_ns - create_ns) + (read_ns - create_ns) > 430000000ULL) {
+		pr_info("  freq-change at %lu ms + read at %lu ms, retrying for a conclusive run\n",
+			(freq_ns - create_ns) / 1000000,
+			(read_ns - create_ns) / 1000000);
+		kvm_vm_free(vm);
+		return false;
+	}
+
+	mono_ns = read_ns - mono_ns;
+
+	/*
+	 * The TSC advances between the write and the read; bound the
+	 * difference by the measured elapsed time around the two ioctls
+	 * plus 10ms of slop, so a descheduled test cannot flake. If the
+	 * write was snapped to the kernel's default generation, the
+	 * value read back is ~500ms low (i.e. below t_write), which is
+	 * unambiguous.
+	 */
+	slack_cycles = freq_khz * (mono_ns / 1000000ULL + 10);
+
+	TEST_ASSERT(t_read >= t_write,
+		    "TSC went backwards from explicit write: wrote %lu, read %lu"
+		    " (first userspace write was subjected to 1-second slop?)",
+		    t_write, t_read);
+	TEST_ASSERT(t_read - t_write < slack_cycles,
+		    "TSC read back too far from written value: wrote %lu, read %lu",
+		    t_write, t_read);
+
+	kvm_vm_free(vm);
+	return true;
+}
+
+static void run_first_write(bool set_freq)
+{
+	const char *name = set_freq ? "first write after KVM_SET_TSC_KHZ"
+				    : "first write";
+	int i;
+
+	for (i = 0; i < 5; i++) {
+		if (test_first_write(set_freq)) {
+			ksft_test_result_pass("%s\n", name);
+			return;
+		}
+	}
+
+	/*
+	 * Inconclusive runs — either gate: slow vCPU creation, or a
+	 * stall between the MSR write and read — reflect host
+	 * scheduling load, not the behaviour under test; record a skip
+	 * rather than report a kernel regression, without discarding a
+	 * conclusive result the other case already established.
+	 */
+	ksft_test_result_skip("%s: host timing repeatedly too slow for a conclusive run\n",
+			      name);
+}
+
+int main(void)
+{
+	ksft_print_header();
+	ksft_set_plan(2);
+
+	run_first_write(false);
+
+	if (kvm_has_cap(KVM_CAP_TSC_CONTROL))
+		run_first_write(true);
+	else
+		ksft_test_result_skip("first write after KVM_SET_TSC_KHZ: no KVM_CAP_TSC_CONTROL\n");
+
+	ksft_finished();
+}
-- 
2.43.0


  parent reply	other threads:[~2026-11-11 18:09 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-11 18:08 [PATCH v13 00/26] KVM: x86: Cleaning up the KVM clock mess, part 2 David Woodhouse
2026-11-11 18:08 ` [PATCH v13 01/26] KVM: x86: Improve synchronization in kvm_synchronize_tsc() David Woodhouse
2026-11-11 18:08 ` [PATCH v13 02/26] KVM: x86: Kill last_tsc_{nsec,write,offset} fields David Woodhouse
2026-11-11 18:08 ` [PATCH v13 03/26] KVM: x86: Replace nr_vcpus_matched_tsc count with all_vcpus_matched_tsc bool David Woodhouse
2026-11-11 18:09 ` [PATCH v13 04/26] KVM: x86: Allow KVM master clock mode when TSCs are offset from each other David Woodhouse
2026-11-11 18:09 ` [PATCH v13 05/26] KVM: x86: Factor out kvm_use_master_clock() David Woodhouse
2026-11-11 18:09 ` [PATCH v13 06/26] KVM: x86: Avoid gratuitous global clock updates David Woodhouse
2026-11-11 18:09 ` [PATCH v13 07/26] KVM: x86: Don't latch the Hyper-V TSC page from a superseded snapshot David Woodhouse
2026-11-11 18:09 ` [PATCH v13 08/26] KVM: x86: Avoid redundant masterclock updates from multiple vCPUs David Woodhouse
2026-11-11 18:09 ` [PATCH v13 09/26] KVM: x86: Stop TSC catchup when a frequency change no longer needs it David Woodhouse
2026-11-11 18:09 ` [PATCH v13 10/26] KVM: x86: Re-synchronize TSC after KVM_SET_TSC_KHZ David Woodhouse
2026-11-11 18:09 ` [PATCH v13 11/26] KVM: x86: Cache host vclock_mode for masterclock eligibility checks David Woodhouse
2026-11-11 18:09 ` [PATCH v13 12/26] KVM: x86: Remove pvclock_gtod_data and private timekeeping code David Woodhouse
2026-11-11 18:09 ` [PATCH v13 13/26] KVM: x86: Keep the master clock trustworthy while the VM has no vCPUs David Woodhouse
2026-11-11 18:09 ` [PATCH v13 14/26] UAPI: x86: Move pvclock-abi to UAPI for x86 platforms David Woodhouse
2026-11-11 18:09 ` [PATCH v13 15/26] KVM: selftests: Use UAPI pvclock-abi.h in xen_shinfo_test David Woodhouse
2026-11-11 18:09 ` [PATCH v13 16/26] KVM: x86: Add KVM_[GS]ET_CLOCK_GUEST for accurate KVM clock migration David Woodhouse
2026-11-11 18:09 ` [PATCH v13 17/26] KVM: selftests: Add library helpers for clock and TSC-offset ioctls David Woodhouse
2026-11-11 18:09 ` [PATCH v13 18/26] KVM: x86: Add KVM_VCPU_TSC_SCALE and fix the documentation on TSC migration David Woodhouse
2026-11-11 18:09 ` [PATCH v13 19/26] KVM: selftests: Add KVM/PV clock selftest to prove timer correction David Woodhouse
2026-11-11 18:09 ` [PATCH v13 20/26] KVM: selftests: Add master clock offset test David Woodhouse
2026-11-11 18:09 ` [PATCH v13 21/26] KVM: selftests: Add Xen/generic CPUID timing leaf test David Woodhouse
2026-11-11 18:09 ` [PATCH v13 22/26] KVM: selftests: Add Xen runstate migration test David Woodhouse
2026-11-11 18:09 ` David Woodhouse [this message]
2026-11-11 18:09 ` [PATCH v13 24/26] KVM: x86/xen: Drop open-coded fallback clock read in kvm_xen_start_timer() David Woodhouse
2026-11-11 18:09 ` [PATCH v13 25/26] KVM: x86: Use a single time snapshot in kvm_vm_ioctl_set_clock() David Woodhouse
2026-11-11 18:09 ` [PATCH v13 26/26] KVM: x86: Keep master clock for guests using legacy MSR_KVM_SYSTEM_TIME David Woodhouse

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261111180922.4177927-24-dwmw2@infradead.org \
    --to=dwmw2@infradead.org \
    --cc=Sascha.Bischoff@arm.com \
    --cc=boris.ostrovsky@oracle.com \
    --cc=bp@alien8.de \
    --cc=corbet@lwn.net \
    --cc=dave.hansen@linux.intel.com \
    --cc=dongli.zhang@oracle.com \
    --cc=dwmw@amazon.co.uk \
    --cc=hpa@zytor.com \
    --cc=jalliste@amazon.com \
    --cc=jgross@suse.com \
    --cc=jic23@kernel.org \
    --cc=joe.jin@oracle.com \
    --cc=joey.gouly@arm.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=mingo@redhat.com \
    --cc=paul@xen.org \
    --cc=pbonzini@redhat.com \
    --cc=seanjc@google.com \
    --cc=skhan@linuxfoundation.org \
    --cc=tglx@kernel.org \
    --cc=vkuznets@redhat.com \
    --cc=x86@kernel.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®