mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Mark Rutland <mark.rutland@arm.com>
To: David Woodhouse <dwmw2@infradead.org>
Cc: Thomas Gleixner <tglx@kernel.org>,
	John Stultz <jstultz@google.com>, Stephen Boyd <sboyd@kernel.org>,
	Miroslav Lichvar <mlichvar@redhat.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Will Deacon <will@kernel.org>, Marc Zyngier <maz@kernel.org>,
	Daniel Lezcano <daniel.lezcano@kernel.org>,
	Rodolfo Giometti <giometti@enneenne.com>,
	linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org, David Woodhouse <dwmw@amazon.co.uk>
Subject: Re: [PATCH 1/2] arm64: Support inlined clocksource reads for the arch counter
Date: Fri, 2 Oct 2026 17:36:30 +0100	[thread overview]
Message-ID: <ar_djtAlVLAy0Gan@J2N7QTR9R3.cambridge.arm.com> (raw)
In-Reply-To: <20261002151938.69857-2-dwmw2@infradead.org>

Hi David,

On Fri, Oct 02, 2026 at 04:19:37PM +0100, David Woodhouse wrote:
> From: David Woodhouse <dwmw@amazon.co.uk>
> 
> Commit 2e27beeb66e43 ("timekeeping: Allow inlining clocksource::read()")
> allows an architecture's clocksource::read() to be inlined, and was only
> wired up for x86_64 thus far in commit b27801189f7fc ("x86: Inline TSC
> reads in timekeeping").
> 
> Add the corresponding ARCH_WANTS_CLOCKSOURCE_READ_INLINE support for the
> arm64 arch counter, allowing the indirect clock->read() call in the fast
> paths to be replaced with a direct sysreg read. A static key selects
> CNTPCT vs CNTVCT as appropriate, and the inline support is only
> advertised for the plain (no erratum workaround) read paths.
> 
> Measured on a Cortex-A53 at 1.35GHz (12.5MHz arch counter): the
> interval from a raw counter read in the caller to the counter read
> inside ktime_get_real_ts64() drops from 30.2ns to 23.4ns, mean of 1M
> iterations. Every ktime_get*() user of tk_clock_read() benefits
> similarly. Except ktime_get_snapshot_id(), which I shall get to shortly.
> 
> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> Assisted-by: LLM

On IRC you mentioned that the saving here was dominated by other costs
(in the microseconds range). Given that, as I mentioned there, I'd
prefer to avoid this.

The arm_arch_timer driver is complicated enough with all the errata
handling, with which we've had more than one painful problem. I don't
want more complication there just to game a microbenchmark.

Mark.

> ---
>  arch/arm64/Kconfig                     |  1 +
>  arch/arm64/include/asm/clock_inlined.h | 26 ++++++++++++++++++++++++++
>  drivers/clocksource/arm_arch_timer.c   | 21 +++++++++++++++++++++
>  3 files changed, 48 insertions(+)
>  create mode 100644 arch/arm64/include/asm/clock_inlined.h
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index b5a51b0ef944..25795b87cd41 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -94,6 +94,7 @@ config ARM64
>  	select ARCH_WANT_FRAME_POINTERS
>  	select ARCH_WANT_HUGE_PMD_SHARE if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36)
>  	select ARCH_WANT_LD_ORPHAN_WARN
> +	select ARCH_WANTS_CLOCKSOURCE_READ_INLINE
>  	select ARCH_WANTS_EXECMEM_LATE
>  	select ARCH_WANTS_NO_INSTR
>  	select ARCH_WANTS_THP_SWAP if ARM64_4K_PAGES
> diff --git a/arch/arm64/include/asm/clock_inlined.h b/arch/arm64/include/asm/clock_inlined.h
> new file mode 100644
> index 000000000000..fe997412bc7d
> --- /dev/null
> +++ b/arch/arm64/include/asm/clock_inlined.h
> @@ -0,0 +1,26 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _ASM_ARM64_CLOCK_INLINED_H
> +#define _ASM_ARM64_CLOCK_INLINED_H
> +
> +#include <linux/jump_label.h>
> +#include <asm/arch_timer.h>
> +
> +struct clocksource;
> +
> +/*
> + * Set by the arch timer driver when it selects the physical counter
> + * (kernel booted at EL2); otherwise the virtual counter is used. The
> + * driver only advertises CLOCK_SOURCE_CAN_INLINE_READ when it selected
> + * a plain (no erratum workaround) sysreg read matching this function.
> + */
> +DECLARE_STATIC_KEY_FALSE(arch_timer_inline_read_pct);
> +
> +static __always_inline u64 arch_inlined_clocksource_read(struct clocksource *cs)
> +{
> +	if (static_branch_likely(&arch_timer_inline_read_pct))
> +		return __arch_counter_get_cntpct();
> +
> +	return __arch_counter_get_cntvct();
> +}
> +
> +#endif
> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
> index cc9a8b399004..1c3d503d881a 100644
> --- a/drivers/clocksource/arm_arch_timer.c
> +++ b/drivers/clocksource/arm_arch_timer.c
> @@ -133,6 +133,13 @@ static noinstr u64 arch_counter_get_cntvct(void)
>  u64 (*arch_timer_read_counter)(void) __ro_after_init = arch_counter_get_cntvct;
>  EXPORT_SYMBOL_GPL(arch_timer_read_counter);
>  
> +#ifdef CONFIG_ARM64
> +#include <asm/clock_inlined.h>
> +
> +/* Selects CNTPCT in arch_inlined_clocksource_read() */
> +DEFINE_STATIC_KEY_FALSE(arch_timer_inline_read_pct);
> +#endif
> +
>  static u64 arch_counter_read(struct clocksource *cs)
>  {
>  	return arch_timer_read_counter();
> @@ -917,6 +924,15 @@ static void __init arch_counter_register(void)
>  		} else {
>  			rd = arch_counter_get_cntvct;
>  			scr = arch_counter_get_cntvct;
> +			/*
> +			 * A plain sysreg read with no erratum workaround
> +			 * matches arch_inlined_clocksource_read(); allow the
> +			 * core timekeeping code to bypass the indirect call.
> +			 */
> +#ifdef CONFIG_ARM64
> +			clocksource_counter.flags |=
> +				CLOCK_SOURCE_CAN_INLINE_READ;
> +#endif
>  		}
>  	} else {
>  		if (arch_timer_counter_has_wa()) {
> @@ -925,6 +941,11 @@ static void __init arch_counter_register(void)
>  		} else {
>  			rd = arch_counter_get_cntpct;
>  			scr = arch_counter_get_cntpct;
> +#ifdef CONFIG_ARM64
> +			static_branch_enable(&arch_timer_inline_read_pct);
> +			clocksource_counter.flags |=
> +				CLOCK_SOURCE_CAN_INLINE_READ;
> +#endif
>  		}
>  	}
>  
> -- 
> 2.43.0
> 

  reply	other threads:[~2026-10-02 16:36 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-02 15:19 [PATCH 0/2] Optimise ktime_get_snapshot_id() for capture latency David Woodhouse
2026-10-02 15:19 ` [PATCH 1/2] arm64: Support inlined clocksource reads for the arch counter David Woodhouse
2026-10-02 16:36   ` Mark Rutland [this message]
2026-10-02 15:19 ` [PATCH 2/2] timekeeping: Use inlined counter read in ktime_get_snapshot_id() David Woodhouse
2026-10-02 16:41 ` [PATCH 0/2] Optimise ktime_get_snapshot_id() for capture latency Mark Rutland
2026-10-02 17:11   ` David Woodhouse

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ar_djtAlVLAy0Gan@J2N7QTR9R3.cambridge.arm.com \
    --to=mark.rutland@arm.com \
    --cc=catalin.marinas@arm.com \
    --cc=daniel.lezcano@kernel.org \
    --cc=dwmw2@infradead.org \
    --cc=dwmw@amazon.co.uk \
    --cc=giometti@enneenne.com \
    --cc=jstultz@google.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=mlichvar@redhat.com \
    --cc=sboyd@kernel.org \
    --cc=tglx@kernel.org \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®