* [PATCH v7 1/5] vdso/math64: Use OPTIMIZER_HIDE_VAR() in __iter_div_u64_rem()
2026-10-02 15:30 [PATCH v7 0/5] vdso: Keep the CLOCK_AUX base at full precision Zhan Xusheng
@ 2026-10-02 15:30 ` Zhan Xusheng
2026-10-15 15:37 ` [tip: timers/vdso] " tip-bot2 for Zhan Xusheng
2026-10-02 15:31 ` [PATCH v7 2/5] vdso/math64: Add and use __iter_div64_u64_rem() Zhan Xusheng
` (3 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Zhan Xusheng @ 2026-10-02 15:30 UTC (permalink / raw)
To: Thomas Gleixner, Thomas Weißschuh
Cc: Yury Norov, Rasmus Villemoes, Andy Lutomirski, Vincenzo Frascino,
David Laight, linux-kernel, Zhan Xusheng, kernel test robot
__iter_div_u64_rem() open-codes an empty asm() to keep the compiler from
turning the subtraction loop into a modulo operation. That is what
OPTIMIZER_HIDE_VAR() is for, so use it and reflow the comment.
The macro lives in linux/compiler.h, which this header did not include.
vdso/math64.h is reached from linux/math64.h, whose includes are
linux/types.h, linux/math.h, asm/div64.h and vdso/math64.h; none of them
guarantees linux/compiler.h. x86 happens to get it through
asm/div64.h -> linux/log2.h, m68k's asm/div64.h only pulls in
asm-generic/div64.h and linux/types.h, so the macro is undeclared there
and the build fails with -Werror=implicit-function-declaration.
Include linux/compiler.h so the header no longer depends on its includer.
This is a no-op for the vDSO, where lib/vdso/gettimeofday.c already gets
it through linux/build_bug.h, and vdso/unaligned.h sets the same
precedent by including linux/compiler_types.h.
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202609300626.zwNjKaO6-lkp@intel.com/
Closes: https://lore.kernel.org/oe-kbuild-all/202609301300.uAawkHuh-lkp@intel.com/
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
---
include/vdso/math64.h | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/include/vdso/math64.h b/include/vdso/math64.h
index 22ae212f8b28..3c08909e742b 100644
--- a/include/vdso/math64.h
+++ b/include/vdso/math64.h
@@ -2,15 +2,18 @@
#ifndef __VDSO_MATH64_H
#define __VDSO_MATH64_H
-static __always_inline u32
-__iter_div_u64_rem(u64 dividend, u32 divisor, u64 *remainder)
+#include <linux/compiler.h>
+
+static __always_inline u32 __iter_div_u64_rem(u64 dividend, u32 divisor, u64 *remainder)
{
u32 ret = 0;
while (dividend >= divisor) {
- /* The following asm() prevents the compiler from
- optimising this loop into a modulo operation. */
- asm("" : "+rm"(dividend));
+ /*
+ * Prevent the compiler from optimising this loop into a
+ * modulo operation.
+ */
+ OPTIMIZER_HIDE_VAR(dividend);
dividend -= divisor;
ret++;
--
2.43.0
^ permalink raw reply [flat|nested] 10+ messages in thread* [tip: timers/vdso] vdso/math64: Use OPTIMIZER_HIDE_VAR() in __iter_div_u64_rem()
2026-10-02 15:30 ` [PATCH v7 1/5] vdso/math64: Use OPTIMIZER_HIDE_VAR() in __iter_div_u64_rem() Zhan Xusheng
@ 2026-10-15 15:37 ` tip-bot2 for Zhan Xusheng
0 siblings, 0 replies; 10+ messages in thread
From: tip-bot2 for Zhan Xusheng @ 2026-10-15 15:37 UTC (permalink / raw)
To: linux-tip-commits
Cc: kernel test robot, Zhan Xusheng, Thomas Gleixner, x86, linux-kernel
The following commit has been merged into the timers/vdso branch of tip:
Commit-ID: c1b910b270fee8930865410455f71e273f4c602c
Gitweb: https://git.kernel.org/tip/c1b910b270fee8930865410455f71e273f4c602c
Author: Zhan Xusheng <zhanxusheng1024@gmail.com>
AuthorDate: Fri, 02 Oct 2026 23:30:59 +08:00
Committer: Thomas Gleixner <tglx@kernel.org>
CommitterDate: Thu, 15 Oct 2026 17:33:20 +02:00
vdso/math64: Use OPTIMIZER_HIDE_VAR() in __iter_div_u64_rem()
__iter_div_u64_rem() open-codes an empty asm() to keep the compiler from
turning the subtraction loop into a modulo operation. That is what
OPTIMIZER_HIDE_VAR() is for, so use it and reflow the comment.
The macro lives in linux/compiler.h, which this header did not include.
vdso/math64.h is reached from linux/math64.h, whose includes are
linux/types.h, linux/math.h, asm/div64.h and vdso/math64.h; none of them
guarantees linux/compiler.h. x86 happens to get it through
asm/div64.h -> linux/log2.h, m68k's asm/div64.h only pulls in
asm-generic/div64.h and linux/types.h, so the macro is undeclared there
and the build fails with -Werror=implicit-function-declaration.
Include linux/compiler.h so the header no longer depends on its includer.
This is a no-op for the vDSO, where lib/vdso/gettimeofday.c already gets
it through linux/build_bug.h, and vdso/unaligned.h sets the same
precedent by including linux/compiler_types.h.
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20261002153104.1347978-2-zhanxusheng@xiaomi.com
Closes: https://lore.kernel.org/oe-kbuild-all/202609300626.zwNjKaO6-lkp@intel.com/
Closes: https://lore.kernel.org/oe-kbuild-all/202609301300.uAawkHuh-lkp@intel.com/
---
include/vdso/math64.h | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/include/vdso/math64.h b/include/vdso/math64.h
index 22ae212..3c08909 100644
--- a/include/vdso/math64.h
+++ b/include/vdso/math64.h
@@ -2,15 +2,18 @@
#ifndef __VDSO_MATH64_H
#define __VDSO_MATH64_H
-static __always_inline u32
-__iter_div_u64_rem(u64 dividend, u32 divisor, u64 *remainder)
+#include <linux/compiler.h>
+
+static __always_inline u32 __iter_div_u64_rem(u64 dividend, u32 divisor, u64 *remainder)
{
u32 ret = 0;
while (dividend >= divisor) {
- /* The following asm() prevents the compiler from
- optimising this loop into a modulo operation. */
- asm("" : "+rm"(dividend));
+ /*
+ * Prevent the compiler from optimising this loop into a
+ * modulo operation.
+ */
+ OPTIMIZER_HIDE_VAR(dividend);
dividend -= divisor;
ret++;
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v7 2/5] vdso/math64: Add and use __iter_div64_u64_rem()
2026-10-02 15:30 [PATCH v7 0/5] vdso: Keep the CLOCK_AUX base at full precision Zhan Xusheng
2026-10-02 15:30 ` [PATCH v7 1/5] vdso/math64: Use OPTIMIZER_HIDE_VAR() in __iter_div_u64_rem() Zhan Xusheng
@ 2026-10-02 15:31 ` Zhan Xusheng
2026-10-15 15:37 ` [tip: timers/vdso] " tip-bot2 for Zhan Xusheng
2026-10-02 15:31 ` [PATCH v7 3/5] vdso/vsyscall: Keep the CLOCK_AUX base scaled Zhan Xusheng
` (2 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Zhan Xusheng @ 2026-10-02 15:31 UTC (permalink / raw)
To: Thomas Gleixner, Thomas Weißschuh
Cc: Yury Norov, Rasmus Villemoes, Andy Lutomirski, Vincenzo Frascino,
David Laight, linux-kernel, Zhan Xusheng
The vDSO basetimes for CLOCK_MONOTONIC and CLOCK_BOOTTIME are stored in
the scaled nanoseconds of tkr_mono, so normalising them requires a
division by NSEC_PER_SEC << shift. That divisor does not fit the u32
parameter of __iter_div_u64_rem(), so update_vdso_time_data() open-codes
the same iterative division twice.
Repeated subtraction is the appropriate form at these two sites because
the quotient never exceeds one. accumulate_nsecs_to_secs() keeps
xtime_nsec below one scaled second, and the offset added to it is a
normalised timespec64 fraction, so the dividend stays below twice the
divisor. That bound holds on every architecture, which matters more here
than the cost of a division on any particular one.
Add __iter_div64_u64_rem(), the u64-divisor counterpart of
__iter_div_u64_rem(), and use it at both sites. Return the quotient as a
u32 for consistency with the u32-divisor variant; the bound above leaves
no use for a wider type.
Store the remainder directly into the basetime, as the coarse clocks
already do. The CLOCK_BOOTTIME copy of the CLOCK_MONOTONIC values then
takes both fields from the same place.
Folding the two open-coded loops into one inlined helper removes 16 bytes
of vsyscall.o text on x86-64 with gcc 13.
No functional change.
Suggested-by: David Laight <david.laight.linux@gmail.com>
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
include/vdso/math64.h | 20 ++++++++++++++++++++
kernel/time/vsyscall.c | 16 +++++-----------
2 files changed, 25 insertions(+), 11 deletions(-)
diff --git a/include/vdso/math64.h b/include/vdso/math64.h
index 3c08909e742b..6f1e311cfdbd 100644
--- a/include/vdso/math64.h
+++ b/include/vdso/math64.h
@@ -24,6 +24,26 @@ static __always_inline u32 __iter_div_u64_rem(u64 dividend, u32 divisor, u64 *re
return ret;
}
+static __always_inline u32 __iter_div64_u64_rem(u64 dividend, u64 divisor, u64 *remainder)
+{
+ u32 ret = 0;
+
+ while (dividend >= divisor) {
+ /*
+ * Prevent the compiler from optimising this loop into a
+ * modulo operation.
+ */
+ OPTIMIZER_HIDE_VAR(dividend);
+
+ dividend -= divisor;
+ ret++;
+ }
+
+ *remainder = dividend;
+
+ return ret;
+}
+
#if defined(CONFIG_ARCH_SUPPORTS_INT128) && defined(__SIZEOF_INT128__)
#ifndef mul_u64_u32_add_u64_shr
diff --git a/kernel/time/vsyscall.c b/kernel/time/vsyscall.c
index aa59919b8f2c..f43dd3f4744b 100644
--- a/kernel/time/vsyscall.c
+++ b/kernel/time/vsyscall.c
@@ -41,14 +41,12 @@ static inline void update_vdso_time_data(struct vdso_time_data *vdata, struct ti
nsec = tk->tkr_mono.xtime_nsec;
nsec += ((u64)tk->wall_to_monotonic.tv_nsec << tk->tkr_mono.shift);
- while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
- nsec -= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
- vdso_ts->sec++;
- }
- vdso_ts->nsec = nsec;
+ vdso_ts->sec += __iter_div64_u64_rem(nsec, (u64)NSEC_PER_SEC << tk->tkr_mono.shift,
+ &vdso_ts->nsec);
/* Copy MONOTONIC time for BOOTTIME */
sec = vdso_ts->sec;
+ nsec = vdso_ts->nsec;
/* Add the boot offset */
sec += tk->monotonic_to_boot.tv_sec;
nsec += (u64)tk->monotonic_to_boot.tv_nsec << tk->tkr_mono.shift;
@@ -56,12 +54,8 @@ static inline void update_vdso_time_data(struct vdso_time_data *vdata, struct ti
/* CLOCK_BOOTTIME */
vdso_ts = &vc[CS_HRES_COARSE].basetime[CLOCK_BOOTTIME];
vdso_ts->sec = sec;
-
- while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
- nsec -= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
- vdso_ts->sec++;
- }
- vdso_ts->nsec = nsec;
+ vdso_ts->sec += __iter_div64_u64_rem(nsec, (u64)NSEC_PER_SEC << tk->tkr_mono.shift,
+ &vdso_ts->nsec);
/* CLOCK_MONOTONIC_RAW */
vdso_ts = &vc[CS_RAW].basetime[CLOCK_MONOTONIC_RAW];
--
2.43.0
^ permalink raw reply [flat|nested] 10+ messages in thread* [tip: timers/vdso] vdso/math64: Add and use __iter_div64_u64_rem()
2026-10-02 15:31 ` [PATCH v7 2/5] vdso/math64: Add and use __iter_div64_u64_rem() Zhan Xusheng
@ 2026-10-15 15:37 ` tip-bot2 for Zhan Xusheng
0 siblings, 0 replies; 10+ messages in thread
From: tip-bot2 for Zhan Xusheng @ 2026-10-15 15:37 UTC (permalink / raw)
To: linux-tip-commits
Cc: David Laight, Zhan Xusheng, Thomas Gleixner, thomas.weissschuh,
x86, linux-kernel
The following commit has been merged into the timers/vdso branch of tip:
Commit-ID: dcd65074e2c593a06bed36178bba6d49d6b6883c
Gitweb: https://git.kernel.org/tip/dcd65074e2c593a06bed36178bba6d49d6b6883c
Author: Zhan Xusheng <zhanxusheng1024@gmail.com>
AuthorDate: Fri, 02 Oct 2026 23:31:00 +08:00
Committer: Thomas Gleixner <tglx@kernel.org>
CommitterDate: Thu, 15 Oct 2026 17:33:20 +02:00
vdso/math64: Add and use __iter_div64_u64_rem()
The vDSO basetimes for CLOCK_MONOTONIC and CLOCK_BOOTTIME are stored in
the scaled nanoseconds of tkr_mono, so normalising them requires a
division by NSEC_PER_SEC << shift. That divisor does not fit the u32
parameter of __iter_div_u64_rem(), so update_vdso_time_data() open-codes
the same iterative division twice.
Repeated subtraction is the appropriate form at these two sites because
the quotient never exceeds one. accumulate_nsecs_to_secs() keeps
xtime_nsec below one scaled second, and the offset added to it is a
normalised timespec64 fraction, so the dividend stays below twice the
divisor. That bound holds on every architecture, which matters more here
than the cost of a division on any particular one.
Add __iter_div64_u64_rem(), the u64-divisor counterpart of
__iter_div_u64_rem(), and use it at both sites. Return the quotient as a
u32 for consistency with the u32-divisor variant; the bound above leaves
no use for a wider type.
Store the remainder directly into the basetime, as the coarse clocks
already do. The CLOCK_BOOTTIME copy of the CLOCK_MONOTONIC values then
takes both fields from the same place.
Folding the two open-coded loops into one inlined helper removes 16 bytes
of vsyscall.o text on x86-64 with gcc 13.
No functional change.
Suggested-by: David Laight <david.laight.linux@gmail.com>
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Link: https://patch.msgid.link/20261002153104.1347978-3-zhanxusheng@xiaomi.com
---
include/vdso/math64.h | 20 ++++++++++++++++++++
kernel/time/vsyscall.c | 16 +++++-----------
2 files changed, 25 insertions(+), 11 deletions(-)
diff --git a/include/vdso/math64.h b/include/vdso/math64.h
index 3c08909..6f1e311 100644
--- a/include/vdso/math64.h
+++ b/include/vdso/math64.h
@@ -24,6 +24,26 @@ static __always_inline u32 __iter_div_u64_rem(u64 dividend, u32 divisor, u64 *re
return ret;
}
+static __always_inline u32 __iter_div64_u64_rem(u64 dividend, u64 divisor, u64 *remainder)
+{
+ u32 ret = 0;
+
+ while (dividend >= divisor) {
+ /*
+ * Prevent the compiler from optimising this loop into a
+ * modulo operation.
+ */
+ OPTIMIZER_HIDE_VAR(dividend);
+
+ dividend -= divisor;
+ ret++;
+ }
+
+ *remainder = dividend;
+
+ return ret;
+}
+
#if defined(CONFIG_ARCH_SUPPORTS_INT128) && defined(__SIZEOF_INT128__)
#ifndef mul_u64_u32_add_u64_shr
diff --git a/kernel/time/vsyscall.c b/kernel/time/vsyscall.c
index b314d6d..bd57c54 100644
--- a/kernel/time/vsyscall.c
+++ b/kernel/time/vsyscall.c
@@ -54,14 +54,12 @@ static inline void update_vdso_time_data(struct vdso_time_data *vdata, struct ti
nsec = tk->tkr_mono.xtime_nsec;
nsec += ((u64)tk->wall_to_monotonic.tv_nsec << tk->tkr_mono.shift);
- while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
- nsec -= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
- vdso_ts->sec++;
- }
- vdso_ts->nsec = nsec;
+ vdso_ts->sec += __iter_div64_u64_rem(nsec, (u64)NSEC_PER_SEC << tk->tkr_mono.shift,
+ &vdso_ts->nsec);
/* Copy MONOTONIC time for BOOTTIME */
sec = vdso_ts->sec;
+ nsec = vdso_ts->nsec;
/* Add the boot offset */
sec += tk->monotonic_to_boot.tv_sec;
nsec += (u64)tk->monotonic_to_boot.tv_nsec << tk->tkr_mono.shift;
@@ -69,12 +67,8 @@ static inline void update_vdso_time_data(struct vdso_time_data *vdata, struct ti
/* CLOCK_BOOTTIME */
vdso_ts = &vc[CS_HRES_COARSE].basetime[CLOCK_BOOTTIME];
vdso_ts->sec = sec;
-
- while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
- nsec -= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
- vdso_ts->sec++;
- }
- vdso_ts->nsec = nsec;
+ vdso_ts->sec += __iter_div64_u64_rem(nsec, (u64)NSEC_PER_SEC << tk->tkr_mono.shift,
+ &vdso_ts->nsec);
/* CLOCK_MONOTONIC_RAW */
vdso_ts = &vc[CS_RAW].basetime[CLOCK_MONOTONIC_RAW];
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v7 3/5] vdso/vsyscall: Keep the CLOCK_AUX base scaled
2026-10-02 15:30 [PATCH v7 0/5] vdso: Keep the CLOCK_AUX base at full precision Zhan Xusheng
2026-10-02 15:30 ` [PATCH v7 1/5] vdso/math64: Use OPTIMIZER_HIDE_VAR() in __iter_div_u64_rem() Zhan Xusheng
2026-10-02 15:31 ` [PATCH v7 2/5] vdso/math64: Add and use __iter_div64_u64_rem() Zhan Xusheng
@ 2026-10-02 15:31 ` Zhan Xusheng
2026-10-15 15:37 ` [tip: timers/vdso] " tip-bot2 for Zhan Xusheng
2026-10-02 15:31 ` [PATCH v7 4/5] vdso: Move BITS_PER_BYTE and BITS_PER_TYPE() to vdso/bits.h Zhan Xusheng
2026-10-02 15:31 ` [PATCH v7 5/5] vdso/gettimeofday: Assert that the clockid fits into the u32 bitmask Zhan Xusheng
4 siblings, 1 reply; 10+ messages in thread
From: Zhan Xusheng @ 2026-10-02 15:31 UTC (permalink / raw)
To: Thomas Gleixner, Thomas Weißschuh
Cc: Yury Norov, Rasmus Villemoes, Andy Lutomirski, Vincenzo Frascino,
David Laight, linux-kernel, Zhan Xusheng
The vDSO basetime of a clock is stored in the scaled nanoseconds of
tkr_mono, so that the reader can floor the base and the cycle delta
together in vdso_calc_ns().
vdso_time_update_aux() instead shifts the base down to nanoseconds, adds
the offset, and shifts it back up, which zeroes the fractional nanoseconds
of xtime_nsec. The reader then floors the base and the delta separately:
ktime_get_aux(): base + ((delta * mult + xtime_nsec) >> shift)
vdso: base + (xtime_nsec >> shift)
+ ((delta * mult) >> shift)
Since floor(a) + floor(b) <= floor(a + b), the vDSO reports 0 or 1 ns
below the syscall for the same clock. It is not a monotonicity problem:
across an update the step is floor(a + d) - floor(a) - floor(d), which is
0 or 1, never negative.
Add the offset in scaled nanoseconds as the other high resolution clocks
do, and normalise with __iter_div64_u64_rem() so that the stored base
stays below one second and the userspace fast-path does not iterate more
in __iter_div_u64_rem().
Only the sub-second field changes. (a + (b << shift)) >> shift is exactly
(a >> shift) + b, so the seconds carried out of the normalisation are the
same as before; what the old form dropped was the low shift bits of the
remainder.
monotonic_to_aux.tv_nsec is a normalised timespec64 fraction, so it stays
below NSEC_PER_SEC even for a negative offset, and the sum stays below
2 * (NSEC_PER_SEC << shift). The largest shift clocks_calc_mult_shift()
can pick is 32, which makes that 8.6e18 against a u64 limit of 1.8e19.
Fixes: 380b84e168e5 ("vdso/vsyscall: Update auxiliary clock data in the datapage")
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
kernel/time/vsyscall.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/kernel/time/vsyscall.c b/kernel/time/vsyscall.c
index f43dd3f4744b..0e4b499328c0 100644
--- a/kernel/time/vsyscall.c
+++ b/kernel/time/vsyscall.c
@@ -155,11 +155,11 @@ void vdso_time_update_aux(struct timekeeper *tk)
vdso_ts->sec = tk->xtime_sec + tk->monotonic_to_aux.tv_sec;
- nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
- nsec += tk->monotonic_to_aux.tv_nsec;
- vdso_ts->sec += __iter_div_u64_rem(nsec, NSEC_PER_SEC, &nsec);
- nsec = nsec << tk->tkr_mono.shift;
- vdso_ts->nsec = nsec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec += (u64)tk->monotonic_to_aux.tv_nsec << tk->tkr_mono.shift;
+ vdso_ts->sec += __iter_div64_u64_rem(nsec,
+ (u64)NSEC_PER_SEC << tk->tkr_mono.shift,
+ &vdso_ts->nsec);
}
__arch_update_vdso_clock(vc);
--
2.43.0
^ permalink raw reply [flat|nested] 10+ messages in thread* [tip: timers/vdso] vdso/vsyscall: Keep the CLOCK_AUX base scaled
2026-10-02 15:31 ` [PATCH v7 3/5] vdso/vsyscall: Keep the CLOCK_AUX base scaled Zhan Xusheng
@ 2026-10-15 15:37 ` tip-bot2 for Zhan Xusheng
0 siblings, 0 replies; 10+ messages in thread
From: tip-bot2 for Zhan Xusheng @ 2026-10-15 15:37 UTC (permalink / raw)
To: linux-tip-commits
Cc: Zhan Xusheng, Thomas Gleixner, thomas.weissschuh, x86, linux-kernel
The following commit has been merged into the timers/vdso branch of tip:
Commit-ID: 8d17190fae2caab89cd5df19257ed9dac0739e3b
Gitweb: https://git.kernel.org/tip/8d17190fae2caab89cd5df19257ed9dac0739e3b
Author: Zhan Xusheng <zhanxusheng1024@gmail.com>
AuthorDate: Fri, 02 Oct 2026 23:31:01 +08:00
Committer: Thomas Gleixner <tglx@kernel.org>
CommitterDate: Thu, 15 Oct 2026 17:33:20 +02:00
vdso/vsyscall: Keep the CLOCK_AUX base scaled
The vDSO basetime of a clock is stored in the scaled nanoseconds of
tkr_mono, so that the reader can floor the base and the cycle delta
together in vdso_calc_ns().
vdso_time_update_aux() instead shifts the base down to nanoseconds, adds
the offset, and shifts it back up, which zeroes the fractional nanoseconds
of xtime_nsec. The reader then floors the base and the delta separately:
ktime_get_aux(): base + ((delta * mult + xtime_nsec) >> shift)
vdso: base + (xtime_nsec >> shift)
+ ((delta * mult) >> shift)
Since floor(a) + floor(b) <= floor(a + b), the vDSO reports 0 or 1 ns
below the syscall for the same clock. It is not a monotonicity problem:
across an update the step is floor(a + d) - floor(a) - floor(d), which is
0 or 1, never negative.
Add the offset in scaled nanoseconds as the other high resolution clocks
do, and normalise with __iter_div64_u64_rem() so that the stored base
stays below one second and the userspace fast-path does not iterate more
in __iter_div_u64_rem().
Only the sub-second field changes. (a + (b << shift)) >> shift is exactly
(a >> shift) + b, so the seconds carried out of the normalisation are the
same as before; what the old form dropped was the low shift bits of the
remainder.
monotonic_to_aux.tv_nsec is a normalised timespec64 fraction, so it stays
below NSEC_PER_SEC even for a negative offset, and the sum stays below
2 * (NSEC_PER_SEC << shift). The largest shift clocks_calc_mult_shift()
can pick is 32, which makes that 8.6e18 against a u64 limit of 1.8e19.
Fixes: 380b84e168e5 ("vdso/vsyscall: Update auxiliary clock data in the datapage")
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Link: https://patch.msgid.link/20261002153104.1347978-4-zhanxusheng@xiaomi.com
---
kernel/time/vsyscall.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/kernel/time/vsyscall.c b/kernel/time/vsyscall.c
index bd57c54..658553f 100644
--- a/kernel/time/vsyscall.c
+++ b/kernel/time/vsyscall.c
@@ -168,11 +168,11 @@ void vdso_time_update_aux(struct timekeeper *tk)
vdso_ts->sec = tk->xtime_sec + tk->monotonic_to_aux.tv_sec;
- nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
- nsec += tk->monotonic_to_aux.tv_nsec;
- vdso_ts->sec += __iter_div_u64_rem(nsec, NSEC_PER_SEC, &nsec);
- nsec = nsec << tk->tkr_mono.shift;
- vdso_ts->nsec = nsec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec += (u64)tk->monotonic_to_aux.tv_nsec << tk->tkr_mono.shift;
+ vdso_ts->sec += __iter_div64_u64_rem(nsec,
+ (u64)NSEC_PER_SEC << tk->tkr_mono.shift,
+ &vdso_ts->nsec);
}
__arch_update_vdso_clock(vc);
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v7 4/5] vdso: Move BITS_PER_BYTE and BITS_PER_TYPE() to vdso/bits.h
2026-10-02 15:30 [PATCH v7 0/5] vdso: Keep the CLOCK_AUX base at full precision Zhan Xusheng
` (2 preceding siblings ...)
2026-10-02 15:31 ` [PATCH v7 3/5] vdso/vsyscall: Keep the CLOCK_AUX base scaled Zhan Xusheng
@ 2026-10-02 15:31 ` Zhan Xusheng
2026-10-02 15:31 ` [PATCH v7 5/5] vdso/gettimeofday: Assert that the clockid fits into the u32 bitmask Zhan Xusheng
4 siblings, 0 replies; 10+ messages in thread
From: Zhan Xusheng @ 2026-10-02 15:31 UTC (permalink / raw)
To: Thomas Gleixner, Thomas Weißschuh
Cc: Yury Norov, Rasmus Villemoes, Andy Lutomirski, Vincenzo Frascino,
David Laight, linux-kernel, Zhan Xusheng
lib/vdso/gettimeofday.c is built into the vDSO, where linux/bits.h is not
available. The next patch wants BITS_PER_TYPE() there.
Move BITS_PER_BYTE and BITS_PER_TYPE() from linux/bits.h to vdso/bits.h.
linux/bits.h already includes vdso/bits.h, so its users are unaffected, and
vdso/datapage.h already pulls vdso/bits.h into the vDSO translation units.
tools/perf/check-headers.sh tracks both headers, so update the tools copies
in lockstep to keep that check quiet.
Suggested-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
include/linux/bits.h | 2 --
include/vdso/bits.h | 2 ++
tools/include/linux/bits.h | 2 --
tools/include/vdso/bits.h | 2 ++
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/include/linux/bits.h b/include/linux/bits.h
index b7509f15718e..7f48e6af1fe7 100644
--- a/include/linux/bits.h
+++ b/include/linux/bits.h
@@ -9,8 +9,6 @@
#define BIT_WORD(nr) ((nr) / BITS_PER_LONG)
#define BIT_ULL_MASK(nr) (ULL(1) << ((nr) % BITS_PER_LONG_LONG))
#define BIT_ULL_WORD(nr) ((nr) / BITS_PER_LONG_LONG)
-#define BITS_PER_BYTE 8
-#define BITS_PER_TYPE(type) (sizeof(type) * BITS_PER_BYTE)
/*
* Create a contiguous bitmask starting at bit position @l and ending at
diff --git a/include/vdso/bits.h b/include/vdso/bits.h
index 388b212088ea..a0b6d404495a 100644
--- a/include/vdso/bits.h
+++ b/include/vdso/bits.h
@@ -6,5 +6,7 @@
#define BIT(nr) (UL(1) << (nr))
#define BIT_ULL(nr) (ULL(1) << (nr))
+#define BITS_PER_BYTE 8
+#define BITS_PER_TYPE(type) (sizeof(type) * BITS_PER_BYTE)
#endif /* __VDSO_BITS_H */
diff --git a/tools/include/linux/bits.h b/tools/include/linux/bits.h
index b7509f15718e..7f48e6af1fe7 100644
--- a/tools/include/linux/bits.h
+++ b/tools/include/linux/bits.h
@@ -9,8 +9,6 @@
#define BIT_WORD(nr) ((nr) / BITS_PER_LONG)
#define BIT_ULL_MASK(nr) (ULL(1) << ((nr) % BITS_PER_LONG_LONG))
#define BIT_ULL_WORD(nr) ((nr) / BITS_PER_LONG_LONG)
-#define BITS_PER_BYTE 8
-#define BITS_PER_TYPE(type) (sizeof(type) * BITS_PER_BYTE)
/*
* Create a contiguous bitmask starting at bit position @l and ending at
diff --git a/tools/include/vdso/bits.h b/tools/include/vdso/bits.h
index 388b212088ea..a0b6d404495a 100644
--- a/tools/include/vdso/bits.h
+++ b/tools/include/vdso/bits.h
@@ -6,5 +6,7 @@
#define BIT(nr) (UL(1) << (nr))
#define BIT_ULL(nr) (ULL(1) << (nr))
+#define BITS_PER_BYTE 8
+#define BITS_PER_TYPE(type) (sizeof(type) * BITS_PER_BYTE)
#endif /* __VDSO_BITS_H */
--
2.43.0
^ permalink raw reply [flat|nested] 10+ messages in thread* [PATCH v7 5/5] vdso/gettimeofday: Assert that the clockid fits into the u32 bitmask
2026-10-02 15:30 [PATCH v7 0/5] vdso: Keep the CLOCK_AUX base at full precision Zhan Xusheng
` (3 preceding siblings ...)
2026-10-02 15:31 ` [PATCH v7 4/5] vdso: Move BITS_PER_BYTE and BITS_PER_TYPE() to vdso/bits.h Zhan Xusheng
@ 2026-10-02 15:31 ` Zhan Xusheng
2026-10-15 13:54 ` Thomas Gleixner
4 siblings, 1 reply; 10+ messages in thread
From: Zhan Xusheng @ 2026-10-02 15:31 UTC (permalink / raw)
To: Thomas Gleixner, Thomas Weißschuh
Cc: Yury Norov, Rasmus Villemoes, Andy Lutomirski, Vincenzo Frascino,
David Laight, linux-kernel, Zhan Xusheng, kernel test robot
__cvdso_clock_gettime_common() and __cvdso_clock_getres_common() convert
the clockid into a bitmask and match it against VDSO_HRES, VDSO_COARSE,
VDSO_RAW and VDSO_AUX:
msk = 1U << clock;
vdso_clockid_valid() rejects anything above CLOCK_AUX_LAST beforehand,
and CLOCK_AUX_LAST is 23, so the shift count is in range. Nothing
records that dependency though. Raising MAX_AUX_CLOCKS beyond 16 moves
CLOCK_AUX_LAST to 32 and makes the shift undefined.
Add a BUILD_BUG_ON() at both conversion sites. The condition is on a
function parameter rather than a constant, so it relies on the compiler
deriving the range from the vdso_clockid_valid() bail-out above it. gcc
13 and clang 18 both do: the x86 vdso64 and vdso32 builds stay clean, and
raising MAX_AUX_CLOCKS to 17 trips the assert.
Suggested-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202609080117.XY9u2XZ9-lkp@intel.com/
Closes: https://lore.kernel.org/oe-kbuild-all/202609300510.Ej5t8KO8-lkp@intel.com/
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
lib/vdso/gettimeofday.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index f7a591aba59f..ef4dcc614489 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -285,6 +285,7 @@ __cvdso_clock_gettime_common(const struct vdso_time_data *vd, clockid_t clock,
* Convert the clockid to a bitmask and use it to check which
* clocks are handled in the VDSO directly.
*/
+ BUILD_BUG_ON(clock >= BITS_PER_TYPE(msk));
msk = 1U << clock;
if (likely(msk & VDSO_HRES))
vc = &vc[CS_HRES_COARSE];
@@ -438,6 +439,7 @@ bool __cvdso_clock_getres_common(const struct vdso_time_data *vd, clockid_t cloc
* Convert the clockid to a bitmask and use it to check which
* clocks are handled in the VDSO directly.
*/
+ BUILD_BUG_ON(clock >= BITS_PER_TYPE(msk));
msk = 1U << clock;
if (msk & (VDSO_HRES | VDSO_RAW)) {
/*
--
2.43.0
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCH v7 5/5] vdso/gettimeofday: Assert that the clockid fits into the u32 bitmask
2026-10-02 15:31 ` [PATCH v7 5/5] vdso/gettimeofday: Assert that the clockid fits into the u32 bitmask Zhan Xusheng
@ 2026-10-15 13:54 ` Thomas Gleixner
0 siblings, 0 replies; 10+ messages in thread
From: Thomas Gleixner @ 2026-10-15 13:54 UTC (permalink / raw)
To: Zhan Xusheng, Thomas Weißschuh
Cc: Yury Norov, Rasmus Villemoes, Andy Lutomirski, Vincenzo Frascino,
David Laight, linux-kernel, Zhan Xusheng, kernel test robot
On Fri, Oct 02 2026 at 23:31, Zhan Xusheng wrote:
> __cvdso_clock_gettime_common() and __cvdso_clock_getres_common() convert
> the clockid into a bitmask and match it against VDSO_HRES, VDSO_COARSE,
> VDSO_RAW and VDSO_AUX:
>
> msk = 1U << clock;
>
> vdso_clockid_valid() rejects anything above CLOCK_AUX_LAST beforehand,
> and CLOCK_AUX_LAST is 23, so the shift count is in range. Nothing
> records that dependency though. Raising MAX_AUX_CLOCKS beyond 16 moves
> CLOCK_AUX_LAST to 32 and makes the shift undefined.
>
> Add a BUILD_BUG_ON() at both conversion sites. The condition is on a
> function parameter rather than a constant, so it relies on the compiler
> deriving the range from the vdso_clockid_valid() bail-out above it. gcc
> 13 and clang 18 both do: the x86 vdso64 and vdso32 builds stay clean, and
> raising MAX_AUX_CLOCKS to 17 trips the assert.
But using 'clock' for the assert is fragile as it relies on the compiler
to defer the maximum 'clock' value from vdso_clockid_valid().
Checking for CLOCK_AUX_LAST >= BITS_PER_TYPE() should be sufficient, no?
> diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
> index f7a591aba59f..ef4dcc614489 100644
> --- a/lib/vdso/gettimeofday.c
> +++ b/lib/vdso/gettimeofday.c
> @@ -285,6 +285,7 @@ __cvdso_clock_gettime_common(const struct vdso_time_data *vd, clockid_t clock,
> * Convert the clockid to a bitmask and use it to check which
> * clocks are handled in the VDSO directly.
> */
> + BUILD_BUG_ON(clock >= BITS_PER_TYPE(msk));
> msk = 1U << clock;
> if (likely(msk & VDSO_HRES))
> vc = &vc[CS_HRES_COARSE];
> @@ -438,6 +439,7 @@ bool __cvdso_clock_getres_common(const struct vdso_time_data *vd, clockid_t cloc
> * Convert the clockid to a bitmask and use it to check which
> * clocks are handled in the VDSO directly.
> */
> + BUILD_BUG_ON(clock >= BITS_PER_TYPE(msk));
> msk = 1U << clock;
> if (msk & (VDSO_HRES | VDSO_RAW)) {
> /*
Aside of that why do we need the same BUILD_BUG_ON() in two functions?
Thanks,
tglx
^ permalink raw reply [flat|nested] 10+ messages in thread