[PATCH 0/4] Micro optimise to get this_cpu once

[PATCH 0/4] Micro optimise to get this_cpu once

[Shrikanth Hegde]

It was observed that compiler doesn't optimise this block to hoist
this_cpu calculations out of the loop in preempt disabled sections. 

for_each_cpu(c, mask) {
	if (c == smp_processor_id())
		do_something
	do_something_else
}

smp_processor_id() could be compiled with CONFIG_DEBUG_PREEMPT=y where it
can be used for warnings. So maybe that's one of the reason it can't
optimize. __smp_processor_id is arch specific, that maybe another
reason.

Even on CONFIG_DEBUG_PREEMPT=n, compiler didn't optimize it out of the
loop.

find_new_ilb dis-assembly in powerpc(CONFIG_DEBUG_PREEMPT=n).
c00000000028cc7c:       bl      c000000000a93c98 <_find_next_and_bit>
c00000000028cc80:       nop
c00000000028cc84:       lwz     r5,0(r29)
c00000000028cc88:       extsw   r30,r3  
c00000000028cc8c:       mr      r31,r3  
c00000000028cc90:       mr      r26,r3  
c00000000028cc94:       cmplw   r5,r3   
c00000000028cc98:       mr      r3,r30  
c00000000028cc9c:       ble     c00000000028ccf8 <kick_ilb+0x10c>
c00000000028cca0:       lhz     r9,8(r13)
     #This is where smp_processor_id is fetched i.e within the loop body.
c00000000028cca4:       cmpw    r9,r31  
c00000000028cca8:       beq     c00000000028ccc0 <kick_ilb+0xd4> 
c00000000028ccac:       bl      c0000000002cd938 <idle_cpu+0x8>
c00000000028ccb0:       nop
c00000000028ccb4:       cmpwi   r3,0    
c00000000028ccb8:       bne     c00000000028cd30 <kick_ilb+0x144>

find_new_ilb dis-assembly in x86(CONFIG_DEBUG_PREEMPT=n).
ffffffff813588eb: 	call   ffffffff81367b30 <housekeeping_cpumask>
ffffffff813588f0: 	xor    %ecx,%ecx
ffffffff813588f2: 	mov    $0xffffffffffffffff,%rsi
ffffffff813588f9: 	mov    %rax,%r8
ffffffff813588fc: 	mov    %rsi,%rdx
ffffffff813588ff: 	mov    0x29258ba(%rip),%rax        # ffffffff83c7e1c0 <nohz>
ffffffff81358906: 	and    (%r8),%rax
ffffffff81358909: 	shl    %cl,%rdx
ffffffff8135890c: 	and    %rdx,%rax
ffffffff8135890f: 	je     ffffffff81358952 <sched_balance_trigger+0x142>
ffffffff81358911: 	tzcnt  %rax,%rbx
ffffffff81358916: 	cmp    $0x3f,%ebx
ffffffff81358919: 	ja     ffffffff81358952 <sched_balance_trigger+0x142>
ffffffff8135891b: 	cmp    %ebx,%gs:0x28e7712(%rip)        # ffffffff83c40034 <cpu_number>
    #This is smp_processor_id() in the loop.
ffffffff81358922: 	mov    %ebx,%edi
ffffffff81358924: 	je     ffffffff81358946 <sched_balance_trigger+0x136>
ffffffff81358926: 	mov    %r8,0x8(%rsp)
ffffffff8135892b: 	mov    %ebx,(%rsp)
ffffffff8135892e: 	call   ffffffff81365140 <idle_cpu>
ffffffff81358933: 	mov    $0xffffffffffffffff,%rsi
ffffffff8135893a: 	mov    (%rsp),%edi
ffffffff8135893d: 	mov    0x8(%rsp),%r8
ffffffff81358942: 	test   %eax,%eax
ffffffff81358944: 	jne    ffffffff813589a4 <sched_balance_trigger+0x194>
ffffffff81358946: 	lea    0x1(%rbx),%ecx

Patched kernel on powerpc find_new_ilb disassembly.
c00000000028cc5c:       08 00 4d a3     lhz     r26,8(r13)
It is fetched once.
...
c00000000028cc94:     bl      c000000000a93cd8 <_find_next_and_bit>
c00000000028cc98:     nop
c00000000028cc9c:     lwz     r5,0(r29)
c00000000028cca0:     extsw   r30,r3
c00000000028cca4:     mr      r31,r3
...
c00000000028cca8:     cmpw    cr7,r26,r3
c00000000028ccb8:     ble     c00000000028cd14 <kick_ilb+0x118>
c00000000028ccbc:     nop
c00000000028ccc0:     beq     cr7,c00000000028ccd8 <kick_ilb+0xdc>
c00000000028ccc4:     bl      c0000000002cd958 <idle_cpu+0x8>


In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
not print any warning.

In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
__smp_processor_id.

So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section
it is better to cache the value. It could save a few cycles. Though
tiny, repeated in loop could add up to a small value.

This is done only for hotpaths or function which gets called quite often.
It is skipped for init or conditional hotpaths such as tracing/events.

While it was sent out[1] along with other scheduler change, it made more sense
to send it out as separate series after observing a few more falling in
same bucket.
[1]: https://lore.kernel.org/all/20260319065314.343932-1-sshegde@linux.ibm.com/


Shrikanth Hegde (4):
  sched/fair: get this cpu once in find_new_ilb
  sched/core: get this cpu once in ttwu_queue_cond
  smp: get this_cpu once in smp_call_function
  timers: Get this_cpu once while clearing idle timer

 kernel/sched/core.c | 6 ++++--
 kernel/sched/fair.c | 4 ++--
 kernel/smp.c        | 4 ++--
 kernel/time/timer.c | 5 +++--
 4 files changed, 11 insertions(+), 8 deletions(-)

-- 
2.47.3

[PATCH 1/4] sched/fair: get this cpu once in find_new_ilb

[Shrikanth Hegde]

Calling smp_processor_id() on:
- In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
not print any warning.
- In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
__smp_processor_id

So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section
it is better to cache the value. It could save a few cycles. Though
tiny, repeated in loop could add up to a small value.

find_new_ilb is called in interrupt context. So preemption is disabled.
So Hoist the this_cpu out of loop

Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
Split it from v1 and found a few more places. Rest of the bits of v1 is
worked out separate
v1: https://lore.kernel.org/all/20260319065314.343932-1-sshegde@linux.ibm.com/

 kernel/sched/fair.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 1b62cb68faae..9298f49f842c 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -12626,14 +12626,14 @@ static inline int on_null_domain(struct rq *rq)
  */
 static inline int find_new_ilb(void)
 {
+	int this_cpu = smp_processor_id();
 	const struct cpumask *hk_mask;
 	int ilb_cpu;
 
 	hk_mask = housekeeping_cpumask(HK_TYPE_KERNEL_NOISE);
 
 	for_each_cpu_and(ilb_cpu, nohz.idle_cpus_mask, hk_mask) {
-
-		if (ilb_cpu == smp_processor_id())
+		if (ilb_cpu == this_cpu)
 			continue;
 
 		if (idle_cpu(ilb_cpu))
-- 
2.47.3

[tip: sched/core] sched/fair: Get this cpu once in find_new_ilb()

[tip-bot2 for Shrikanth Hegde]

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     76504bce4ee6b8757647e07bc1710dcac9acdc2e
Gitweb:        https://git.kernel.org/tip/76504bce4ee6b8757647e07bc1710dcac9acdc2e
Author:        Shrikanth Hegde <sshegde@linux.ibm.com>
AuthorDate:    Tue, 24 Mar 2026 01:06:27 +05:30
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Tue, 24 Mar 2026 10:07:04 +01:00

sched/fair: Get this cpu once in find_new_ilb()

Calling smp_processor_id() on:
 - In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
   not print any warning.
 - In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
   __smp_processor_id

So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section
it is better to cache the value. It could save a few cycles. Though
tiny, repeated in loop could add up to a small value.

find_new_ilb is called in interrupt context. So preemption is disabled.
So Hoist the this_cpu out of loop

Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Link: https://patch.msgid.link/20260323193630.640311-2-sshegde@linux.ibm.com
---
 kernel/sched/fair.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 0a35a82..2265092 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -12614,14 +12614,14 @@ static inline int on_null_domain(struct rq *rq)
  */
 static inline int find_new_ilb(void)
 {
+	int this_cpu = smp_processor_id();
 	const struct cpumask *hk_mask;
 	int ilb_cpu;
 
 	hk_mask = housekeeping_cpumask(HK_TYPE_KERNEL_NOISE);
 
 	for_each_cpu_and(ilb_cpu, nohz.idle_cpus_mask, hk_mask) {
-
-		if (ilb_cpu == smp_processor_id())
+		if (ilb_cpu == this_cpu)
 			continue;
 
 		if (idle_cpu(ilb_cpu))

[PATCH 2/4] sched/core: get this cpu once in ttwu_queue_cond

[Shrikanth Hegde]

Calling smp_processor_id() on:
- In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
not print any warning.
- In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
__smp_processor_id

So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section
it is better to cache the value. It could save a few cycles. Though
tiny, repeated could add up to a small value.

ttwu_queue_cond is called with interrupt disabled. So preemption is
disabled. Hence cache the value once instead.

Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
 kernel/sched/core.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 49af425f6a73..14996c7a0a31 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3900,6 +3900,8 @@ bool cpus_share_resources(int this_cpu, int that_cpu)
 
 static inline bool ttwu_queue_cond(struct task_struct *p, int cpu)
 {
+	int this_cpu = smp_processor_id();
+
 	/* See SCX_OPS_ALLOW_QUEUED_WAKEUP. */
 	if (!scx_allow_ttwu_queue(p))
 		return false;
@@ -3924,10 +3926,10 @@ static inline bool ttwu_queue_cond(struct task_struct *p, int cpu)
 	 * If the CPU does not share cache, then queue the task on the
 	 * remote rqs wakelist to avoid accessing remote data.
 	 */
-	if (!cpus_share_cache(smp_processor_id(), cpu))
+	if (!cpus_share_cache(this_cpu, cpu))
 		return true;
 
-	if (cpu == smp_processor_id())
+	if (cpu == this_cpu)
 		return false;
 
 	/*
-- 
2.47.3

Re: [PATCH 2/4] sched/core: get this cpu once in ttwu_queue_cond

[Mukesh Kumar Chaurasiya]

On Tue, Mar 24, 2026 at 01:06:28AM +0530, Shrikanth Hegde wrote:
> Calling smp_processor_id() on:
> - In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
> not print any warning.
> - In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
> __smp_processor_id
> 
> So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section
> it is better to cache the value. It could save a few cycles. Though
> tiny, repeated could add up to a small value.
> 
> ttwu_queue_cond is called with interrupt disabled. So preemption is
> disabled. Hence cache the value once instead.
> 
> Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
> ---
>  kernel/sched/core.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 49af425f6a73..14996c7a0a31 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3900,6 +3900,8 @@ bool cpus_share_resources(int this_cpu, int that_cpu)
>  
>  static inline bool ttwu_queue_cond(struct task_struct *p, int cpu)
>  {
> +	int this_cpu = smp_processor_id();
> +
>  	/* See SCX_OPS_ALLOW_QUEUED_WAKEUP. */
>  	if (!scx_allow_ttwu_queue(p))
>  		return false;
> @@ -3924,10 +3926,10 @@ static inline bool ttwu_queue_cond(struct task_struct *p, int cpu)
>  	 * If the CPU does not share cache, then queue the task on the
>  	 * remote rqs wakelist to avoid accessing remote data.
>  	 */
> -	if (!cpus_share_cache(smp_processor_id(), cpu))
> +	if (!cpus_share_cache(this_cpu, cpu))
>  		return true;
>  
> -	if (cpu == smp_processor_id())
> +	if (cpu == this_cpu)
>  		return false;
>  
>  	/*
> -- 
> 2.47.3
> 
LGTM

Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>

[tip: sched/core] sched/core: Get this cpu once in ttwu_queue_cond()

[tip-bot2 for Shrikanth Hegde]

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     0e81fe79fec5a639700f09f39c8ab680c3312ba2
Gitweb:        https://git.kernel.org/tip/0e81fe79fec5a639700f09f39c8ab680c3312ba2
Author:        Shrikanth Hegde <sshegde@linux.ibm.com>
AuthorDate:    Tue, 24 Mar 2026 01:06:28 +05:30
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Tue, 24 Mar 2026 10:07:05 +01:00

sched/core: Get this cpu once in ttwu_queue_cond()

Calling smp_processor_id() on:
 - In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
   not print any warning.
 - In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
   __smp_processor_id

So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section
it is better to cache the value. It could save a few cycles. Though
tiny, repeated could add up to a small value.

ttwu_queue_cond is called with interrupt disabled. So preemption is
disabled. Hence cache the value once instead.

Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Link: https://patch.msgid.link/20260323193630.640311-3-sshegde@linux.ibm.com
---
 kernel/sched/core.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 64b467c..7c7d4bf 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3842,6 +3842,8 @@ bool cpus_share_resources(int this_cpu, int that_cpu)
 
 static inline bool ttwu_queue_cond(struct task_struct *p, int cpu)
 {
+	int this_cpu = smp_processor_id();
+
 	/* See SCX_OPS_ALLOW_QUEUED_WAKEUP. */
 	if (!scx_allow_ttwu_queue(p))
 		return false;
@@ -3866,10 +3868,10 @@ static inline bool ttwu_queue_cond(struct task_struct *p, int cpu)
 	 * If the CPU does not share cache, then queue the task on the
 	 * remote rqs wakelist to avoid accessing remote data.
 	 */
-	if (!cpus_share_cache(smp_processor_id(), cpu))
+	if (!cpus_share_cache(this_cpu, cpu))
 		return true;
 
-	if (cpu == smp_processor_id())
+	if (cpu == this_cpu)
 		return false;
 
 	/*

[PATCH 3/4] smp: get this_cpu once in smp_call_function

[Shrikanth Hegde]

Calling smp_processor_id() on:
- In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
not print any warning.
- In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
__smp_processor_id

So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section
it is better to cache the value. It could save a few cycles. Though
tiny, repeated in loop could add up to a small value.

smp_call_function_single/smp_call_function_many_cond is called with
preemption disabled.
So cache the values once.

Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
 kernel/smp.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/smp.c b/kernel/smp.c
index f349960f79ca..4c57104d1cd3 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -677,7 +677,7 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
 	csd->func = func;
 	csd->info = info;
 #ifdef CONFIG_CSD_LOCK_WAIT_DEBUG
-	csd->node.src = smp_processor_id();
+	csd->node.src = this_cpu;
 	csd->node.dst = cpu;
 #endif
 
@@ -832,7 +832,7 @@ static void smp_call_function_many_cond(const struct cpumask *mask,
 			csd->func = func;
 			csd->info = info;
 #ifdef CONFIG_CSD_LOCK_WAIT_DEBUG
-			csd->node.src = smp_processor_id();
+			csd->node.src = this_cpu;
 			csd->node.dst = cpu;
 #endif
 			trace_csd_queue_cpu(cpu, _RET_IP_, func, csd);
-- 
2.47.3

Re: [PATCH 3/4] smp: get this_cpu once in smp_call_function

[Mukesh Kumar Chaurasiya]

On Tue, Mar 24, 2026 at 01:06:29AM +0530, Shrikanth Hegde wrote:
> Calling smp_processor_id() on:
> - In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
> not print any warning.
> - In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
> __smp_processor_id
> 
> So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section
> it is better to cache the value. It could save a few cycles. Though
> tiny, repeated in loop could add up to a small value.
> 
> smp_call_function_single/smp_call_function_many_cond is called with
> preemption disabled.
> So cache the values once.
> 
> Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
> ---
>  kernel/smp.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/smp.c b/kernel/smp.c
> index f349960f79ca..4c57104d1cd3 100644
> --- a/kernel/smp.c
> +++ b/kernel/smp.c
> @@ -677,7 +677,7 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
>  	csd->func = func;
>  	csd->info = info;
>  #ifdef CONFIG_CSD_LOCK_WAIT_DEBUG
> -	csd->node.src = smp_processor_id();
> +	csd->node.src = this_cpu;
>  	csd->node.dst = cpu;
>  #endif
>  
> @@ -832,7 +832,7 @@ static void smp_call_function_many_cond(const struct cpumask *mask,
>  			csd->func = func;
>  			csd->info = info;
>  #ifdef CONFIG_CSD_LOCK_WAIT_DEBUG
> -			csd->node.src = smp_processor_id();
> +			csd->node.src = this_cpu;
>  			csd->node.dst = cpu;
>  #endif
>  			trace_csd_queue_cpu(cpu, _RET_IP_, func, csd);
> -- 
> 2.47.3
> 
LGTM

Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>

[tip: smp/core] smp: Get this_cpu once in smp_call_function

[tip-bot2 for Shrikanth Hegde]

The following commit has been merged into the smp/core branch of tip:

Commit-ID:     ec39780d6a9e05e7e427008603b40efef515b775
Gitweb:        https://git.kernel.org/tip/ec39780d6a9e05e7e427008603b40efef515b775
Author:        Shrikanth Hegde <sshegde@linux.ibm.com>
AuthorDate:    Tue, 24 Mar 2026 01:06:29 +05:30
Committer:     Thomas Gleixner <tglx@kernel.org>
CommitterDate: Wed, 25 Mar 2026 20:11:30 +01:00

smp: Get this_cpu once in smp_call_function

smp_call_function_single() and smp_call_function_many_cond() disable
preemption and cache the CPU number via get_cpu().

Use this cached value throughout the function instead of invoking
smp_processor_id() again.

[ tglx: Make the copy&pasta'ed change log match the patch ]

Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Link: https://patch.msgid.link/20260323193630.640311-4-sshegde@linux.ibm.com
---
 kernel/smp.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/smp.c b/kernel/smp.c
index b179424..bdbf145 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -678,7 +678,7 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
 	csd->func = func;
 	csd->info = info;
 #ifdef CONFIG_CSD_LOCK_WAIT_DEBUG
-	csd->node.src = smp_processor_id();
+	csd->node.src = this_cpu;
 	csd->node.dst = cpu;
 #endif
 
@@ -833,7 +833,7 @@ static void smp_call_function_many_cond(const struct cpumask *mask,
 			csd->func = func;
 			csd->info = info;
 #ifdef CONFIG_CSD_LOCK_WAIT_DEBUG
-			csd->node.src = smp_processor_id();
+			csd->node.src = this_cpu;
 			csd->node.dst = cpu;
 #endif
 			trace_csd_queue_cpu(cpu, _RET_IP_, func, csd);

[PATCH 4/4] timers: Get this_cpu once while clearing idle timer

[Shrikanth Hegde]

Calling smp_processor_id() on:
- In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
not print any warning.
- In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
__smp_processor_id

So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section
it is better to cache the value. It could save a few cycles. Though
tiny, repeated could add up to a small value.

timer_clear_idle is called with interrupts disabled.
So cache the value once.

Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
 kernel/time/timer.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/kernel/time/timer.c b/kernel/time/timer.c
index 7e1e3bde6b8b..04d928c21aba 100644
--- a/kernel/time/timer.c
+++ b/kernel/time/timer.c
@@ -2319,6 +2319,7 @@ u64 timer_base_try_to_set_idle(unsigned long basej, u64 basem, bool *idle)
  */
 void timer_clear_idle(void)
 {
+	int this_cpu = smp_processor_id();
 	/*
 	 * We do this unlocked. The worst outcome is a remote pinned timer
 	 * enqueue sending a pointless IPI, but taking the lock would just
@@ -2327,9 +2328,9 @@ void timer_clear_idle(void)
 	 * path. Required for BASE_LOCAL only.
 	 */
 	__this_cpu_write(timer_bases[BASE_LOCAL].is_idle, false);
-	if (tick_nohz_full_cpu(smp_processor_id()))
+	if (tick_nohz_full_cpu(this_cpu))
 		__this_cpu_write(timer_bases[BASE_GLOBAL].is_idle, false);
-	trace_timer_base_idle(false, smp_processor_id());
+	trace_timer_base_idle(false, this_cpu);
 
 	/* Activate without holding the timer_base->lock */
 	tmigr_cpu_activate();
-- 
2.47.3

Re: [PATCH 4/4] timers: Get this_cpu once while clearing idle timer

[Mukesh Kumar Chaurasiya]

On Tue, Mar 24, 2026 at 01:06:30AM +0530, Shrikanth Hegde wrote:
> Calling smp_processor_id() on:
> - In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
> not print any warning.
> - In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
> __smp_processor_id
> 
> So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section
> it is better to cache the value. It could save a few cycles. Though
> tiny, repeated could add up to a small value.
> 
> timer_clear_idle is called with interrupts disabled.
> So cache the value once.
> 
> Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
> ---
>  kernel/time/timer.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/time/timer.c b/kernel/time/timer.c
> index 7e1e3bde6b8b..04d928c21aba 100644
> --- a/kernel/time/timer.c
> +++ b/kernel/time/timer.c
> @@ -2319,6 +2319,7 @@ u64 timer_base_try_to_set_idle(unsigned long basej, u64 basem, bool *idle)
>   */
>  void timer_clear_idle(void)
>  {
> +	int this_cpu = smp_processor_id();
>  	/*
>  	 * We do this unlocked. The worst outcome is a remote pinned timer
>  	 * enqueue sending a pointless IPI, but taking the lock would just
> @@ -2327,9 +2328,9 @@ void timer_clear_idle(void)
>  	 * path. Required for BASE_LOCAL only.
>  	 */
>  	__this_cpu_write(timer_bases[BASE_LOCAL].is_idle, false);
> -	if (tick_nohz_full_cpu(smp_processor_id()))
> +	if (tick_nohz_full_cpu(this_cpu))
>  		__this_cpu_write(timer_bases[BASE_GLOBAL].is_idle, false);
> -	trace_timer_base_idle(false, smp_processor_id());
> +	trace_timer_base_idle(false, this_cpu);
>  
>  	/* Activate without holding the timer_base->lock */
>  	tmigr_cpu_activate();
> -- 
> 2.47.3
> 

Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>

[tip: timers/core] timers: Get this_cpu once while clearing the idle state

[tip-bot2 for Shrikanth Hegde]

The following commit has been merged into the timers/core branch of tip:

Commit-ID:     551e49beb1752387aed09eb2a0ea4c82ed1f3a35
Gitweb:        https://git.kernel.org/tip/551e49beb1752387aed09eb2a0ea4c82ed1f3a35
Author:        Shrikanth Hegde <sshegde@linux.ibm.com>
AuthorDate:    Tue, 24 Mar 2026 01:06:30 +05:30
Committer:     Thomas Gleixner <tglx@kernel.org>
CommitterDate: Tue, 24 Mar 2026 23:21:30 +01:00

timers: Get this_cpu once while clearing the idle state

Calling smp_processor_id() on:
- In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
not print any warning.
- In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
__smp_processor_id

So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section it is
better to cache the value. It saves a few cycles. Though tiny, repeated
adds up.

timer_clear_idle() is called with interrupts disabled. So cache the value
once.

Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Link: https://patch.msgid.link/20260323193630.640311-5-sshegde@linux.ibm.com
---
 kernel/time/timer.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/kernel/time/timer.c b/kernel/time/timer.c
index 7e1e3bd..04d928c 100644
--- a/kernel/time/timer.c
+++ b/kernel/time/timer.c
@@ -2319,6 +2319,7 @@ u64 timer_base_try_to_set_idle(unsigned long basej, u64 basem, bool *idle)
  */
 void timer_clear_idle(void)
 {
+	int this_cpu = smp_processor_id();
 	/*
 	 * We do this unlocked. The worst outcome is a remote pinned timer
 	 * enqueue sending a pointless IPI, but taking the lock would just
@@ -2327,9 +2328,9 @@ void timer_clear_idle(void)
 	 * path. Required for BASE_LOCAL only.
 	 */
 	__this_cpu_write(timer_bases[BASE_LOCAL].is_idle, false);
-	if (tick_nohz_full_cpu(smp_processor_id()))
+	if (tick_nohz_full_cpu(this_cpu))
 		__this_cpu_write(timer_bases[BASE_GLOBAL].is_idle, false);
-	trace_timer_base_idle(false, smp_processor_id());
+	trace_timer_base_idle(false, this_cpu);
 
 	/* Activate without holding the timer_base->lock */
 	tmigr_cpu_activate();

Re: [PATCH 0/4] Micro optimise to get this_cpu once

[Mukesh Kumar Chaurasiya]

On Tue, Mar 24, 2026 at 01:06:26AM +0530, Shrikanth Hegde wrote:
> It was observed that compiler doesn't optimise this block to hoist
> this_cpu calculations out of the loop in preempt disabled sections. 
> 
> for_each_cpu(c, mask) {
> 	if (c == smp_processor_id())
> 		do_something
> 	do_something_else
> }
> 
> smp_processor_id() could be compiled with CONFIG_DEBUG_PREEMPT=y where it
> can be used for warnings. So maybe that's one of the reason it can't
> optimize. __smp_processor_id is arch specific, that maybe another
> reason.
> 
> Even on CONFIG_DEBUG_PREEMPT=n, compiler didn't optimize it out of the
> loop.
> 
> find_new_ilb dis-assembly in powerpc(CONFIG_DEBUG_PREEMPT=n).
> c00000000028cc7c:       bl      c000000000a93c98 <_find_next_and_bit>
> c00000000028cc80:       nop
> c00000000028cc84:       lwz     r5,0(r29)
> c00000000028cc88:       extsw   r30,r3  
> c00000000028cc8c:       mr      r31,r3  
> c00000000028cc90:       mr      r26,r3  
> c00000000028cc94:       cmplw   r5,r3   
> c00000000028cc98:       mr      r3,r30  
> c00000000028cc9c:       ble     c00000000028ccf8 <kick_ilb+0x10c>
> c00000000028cca0:       lhz     r9,8(r13)
>      #This is where smp_processor_id is fetched i.e within the loop body.
> c00000000028cca4:       cmpw    r9,r31  
> c00000000028cca8:       beq     c00000000028ccc0 <kick_ilb+0xd4> 
> c00000000028ccac:       bl      c0000000002cd938 <idle_cpu+0x8>
> c00000000028ccb0:       nop
> c00000000028ccb4:       cmpwi   r3,0    
> c00000000028ccb8:       bne     c00000000028cd30 <kick_ilb+0x144>
> 
> find_new_ilb dis-assembly in x86(CONFIG_DEBUG_PREEMPT=n).
> ffffffff813588eb: 	call   ffffffff81367b30 <housekeeping_cpumask>
> ffffffff813588f0: 	xor    %ecx,%ecx
> ffffffff813588f2: 	mov    $0xffffffffffffffff,%rsi
> ffffffff813588f9: 	mov    %rax,%r8
> ffffffff813588fc: 	mov    %rsi,%rdx
> ffffffff813588ff: 	mov    0x29258ba(%rip),%rax        # ffffffff83c7e1c0 <nohz>
> ffffffff81358906: 	and    (%r8),%rax
> ffffffff81358909: 	shl    %cl,%rdx
> ffffffff8135890c: 	and    %rdx,%rax
> ffffffff8135890f: 	je     ffffffff81358952 <sched_balance_trigger+0x142>
> ffffffff81358911: 	tzcnt  %rax,%rbx
> ffffffff81358916: 	cmp    $0x3f,%ebx
> ffffffff81358919: 	ja     ffffffff81358952 <sched_balance_trigger+0x142>
> ffffffff8135891b: 	cmp    %ebx,%gs:0x28e7712(%rip)        # ffffffff83c40034 <cpu_number>
>     #This is smp_processor_id() in the loop.
> ffffffff81358922: 	mov    %ebx,%edi
> ffffffff81358924: 	je     ffffffff81358946 <sched_balance_trigger+0x136>
> ffffffff81358926: 	mov    %r8,0x8(%rsp)
> ffffffff8135892b: 	mov    %ebx,(%rsp)
> ffffffff8135892e: 	call   ffffffff81365140 <idle_cpu>
> ffffffff81358933: 	mov    $0xffffffffffffffff,%rsi
> ffffffff8135893a: 	mov    (%rsp),%edi
> ffffffff8135893d: 	mov    0x8(%rsp),%r8
> ffffffff81358942: 	test   %eax,%eax
> ffffffff81358944: 	jne    ffffffff813589a4 <sched_balance_trigger+0x194>
> ffffffff81358946: 	lea    0x1(%rbx),%ecx
> 
> Patched kernel on powerpc find_new_ilb disassembly.
> c00000000028cc5c:       08 00 4d a3     lhz     r26,8(r13)
> It is fetched once.
> ...
> c00000000028cc94:     bl      c000000000a93cd8 <_find_next_and_bit>
> c00000000028cc98:     nop
> c00000000028cc9c:     lwz     r5,0(r29)
> c00000000028cca0:     extsw   r30,r3
> c00000000028cca4:     mr      r31,r3
> ...
> c00000000028cca8:     cmpw    cr7,r26,r3
> c00000000028ccb8:     ble     c00000000028cd14 <kick_ilb+0x118>
> c00000000028ccbc:     nop
> c00000000028ccc0:     beq     cr7,c00000000028ccd8 <kick_ilb+0xdc>
> c00000000028ccc4:     bl      c0000000002cd958 <idle_cpu+0x8>
> 
> 
> In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
> not print any warning.
> 
> In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
> __smp_processor_id.
> 
> So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section
> it is better to cache the value. It could save a few cycles. Though
> tiny, repeated in loop could add up to a small value.
> 
> This is done only for hotpaths or function which gets called quite often.
> It is skipped for init or conditional hotpaths such as tracing/events.
> 
> While it was sent out[1] along with other scheduler change, it made more sense
> to send it out as separate series after observing a few more falling in
> same bucket.
> [1]: https://lore.kernel.org/all/20260319065314.343932-1-sshegde@linux.ibm.com/
> 
The changes in all 4 patches are very similar and they are quite small.
IMO they can be clubbed together as a single patch.

Regards,
Mukesh
> 
> Shrikanth Hegde (4):
>   sched/fair: get this cpu once in find_new_ilb
>   sched/core: get this cpu once in ttwu_queue_cond
>   smp: get this_cpu once in smp_call_function
>   timers: Get this_cpu once while clearing idle timer
> 
>  kernel/sched/core.c | 6 ++++--
>  kernel/sched/fair.c | 4 ++--
>  kernel/smp.c        | 4 ++--
>  kernel/time/timer.c | 5 +++--
>  4 files changed, 11 insertions(+), 8 deletions(-)
> 
> -- 
> 2.47.3
>