[PATCH] sched: Add _TIF_NEED_RESCHED_LAZY to __resched_curr check

[PATCH] sched: Add _TIF_NEED_RESCHED_LAZY to __resched_curr check

[Jemmy Wong]

The TIF_NEED_RESCHED_LAZY flag can be set multiple times in a single
call path. For example:

entity_tick()
    update_curr(cfs_rq);
        resched_curr_lazy(rq);
    resched_curr_lazy(rq_of(cfs_rq));

Add a check in resched_curr_lazy() to return early if the flag is
already set, avoiding redundant operations.

Signed-off-by: Jemmy Wong <jemmywong512@gmail.com>

---
 include/linux/sched.h       | 2 +-
 include/linux/thread_info.h | 2 ++
 kernel/entry/common.c       | 2 +-
 kernel/entry/kvm.c          | 2 +-
 kernel/sched/core.c         | 2 +-
 5 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/include/linux/sched.h b/include/linux/sched.h
index e4ce0a76831e..5946434b2dc4 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -2041,7 +2041,7 @@ static inline void set_tsk_need_resched(struct task_struct *tsk)

 static inline void clear_tsk_need_resched(struct task_struct *tsk)
 {
-	atomic_long_andnot(_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY,
+	atomic_long_andnot(_TIF_NEED_RESCHED_MUSK,
 			  (atomic_long_t *)&task_thread_info(tsk)->flags);
 }

diff --git a/include/linux/thread_info.h b/include/linux/thread_info.h
index dd925d84fa46..a7512ab612ad 100644
--- a/include/linux/thread_info.h
+++ b/include/linux/thread_info.h
@@ -67,6 +67,8 @@ enum syscall_work_bit {
 #define _TIF_NEED_RESCHED_LAZY _TIF_NEED_RESCHED
 #endif

+#define _TIF_NEED_RESCHED_MUSK (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY)
+
 #ifdef __KERNEL__

 #ifndef arch_set_restart_data
diff --git a/kernel/entry/common.c b/kernel/entry/common.c
index 408d28b5179d..ac6eff43d07e 100644
--- a/kernel/entry/common.c
+++ b/kernel/entry/common.c
@@ -27,7 +27,7 @@ __always_inline unsigned long exit_to_user_mode_loop(struct pt_regs *regs,

 		local_irq_enable_exit_to_user(ti_work);

-		if (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY))
+		if (ti_work & _TIF_NEED_RESCHED_MUSK)
 			schedule();

 		if (ti_work & _TIF_UPROBE)
diff --git a/kernel/entry/kvm.c b/kernel/entry/kvm.c
index 8485f63863af..cacb24f0fc86 100644
--- a/kernel/entry/kvm.c
+++ b/kernel/entry/kvm.c
@@ -13,7 +13,7 @@ static int xfer_to_guest_mode_work(struct kvm_vcpu *vcpu, unsigned long ti_work)
 			return -EINTR;
 		}

-		if (ti_work & (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY))
+		if (ti_work & _TIF_NEED_RESCHED_MUSK)
 			schedule();

 		if (ti_work & _TIF_NOTIFY_RESUME)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index ccba6fc3c3fe..15bf4b132153 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1108,7 +1108,7 @@ static void __resched_curr(struct rq *rq, int tif)
 	if (is_idle_task(curr) && tif == TIF_NEED_RESCHED_LAZY)
 		tif = TIF_NEED_RESCHED;

-	if (cti->flags & ((1 << tif) | _TIF_NEED_RESCHED))
+	if (cti->flags & ((1 << tif) | _TIF_NEED_RESCHED_MUSK))
 		return;

 	cpu = cpu_of(rq);
--
2.50.1 (Apple Git-155)

Re: [PATCH] sched: Add _TIF_NEED_RESCHED_LAZY to __resched_curr check

[K Prateek Nayak]

Hello Jemmy,

On 9/28/2025 8:44 PM, Jemmy Wong wrote:
> The TIF_NEED_RESCHED_LAZY flag can be set multiple times in a single
> call path. For example:
> 
> entity_tick()
>     update_curr(cfs_rq);
>         resched_curr_lazy(rq);
>     resched_curr_lazy(rq_of(cfs_rq));
> 
> Add a check in resched_curr_lazy() to return early if the flag is
> already set, avoiding redundant operations.

That would have been a decent idea but then you decided to put that
check in __resched_curr() which makes it plain wrong.

[..snip..]

> --- a/include/linux/thread_info.h
> +++ b/include/linux/thread_info.h
> @@ -67,6 +67,8 @@ enum syscall_work_bit {
>  #define _TIF_NEED_RESCHED_LAZY _TIF_NEED_RESCHED
>  #endif
> 
> +#define _TIF_NEED_RESCHED_MUSK (_TIF_NEED_RESCHED | _TIF_NEED_RESCHED_LAZY)

s/MUSK/MASK/g

> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -1108,7 +1108,7 @@ static void __resched_curr(struct rq *rq, int tif)
>  	if (is_idle_task(curr) && tif == TIF_NEED_RESCHED_LAZY)
>  		tif = TIF_NEED_RESCHED;
> 
> -	if (cti->flags & ((1 << tif) | _TIF_NEED_RESCHED))
> +	if (cti->flags & ((1 << tif) | _TIF_NEED_RESCHED_MUSK))
>  		return;

__resched_curr() is used to set both TIF_NEED_RESCHED_LAZY and
TIF_NEED_RESCHED.

By putting this check here, any effort to set NEED_RESCHED and force an
early preemption will bail out if NEED_RESCHED_LAZY is already set which
will delay the preemption.

An example:

    /* New fair task wakes up. */
    check_preempt_wakeup_fair()
        resched_curr_lazy()
            __resched_curr(TIF_NEED_RESCHED_LAZY)

    /* New RT task wakes up. */
    wakeup_preempt()
        resched_curr()
            __resched_curr(TIF_NEED_RESCHED)
                /* Sees NEED_RESCHED_LAZY is already set. */
                /* Does not do a set_preempt_need_resched() */

    ... /* Added latency */
    sched_tick()
        if (tif_test_bit(TIF_NEED_RESCHED_LAZY))
            resched_curr()
                __resched_curr(TIF_NEED_RESCHED)
                    /* Again bails out early! */

    ... /* More latency! */


So, the tick doesn't even upgrade the LAZY flag to a full NEED_RESCHED
and the only time you actually schedule is either at exit to user mode
or if a kthread decides to yield.

Going back to your commit message, something like:

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 7f1e5cb94c53..3275abce9ca2 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1164,6 +1164,9 @@ static __always_inline int get_lazy_tif_bit(void)
 
 void resched_curr_lazy(struct rq *rq)
 {
+	if (task_thread_info(rq->curr)->flags & TIF_NEED_RESCHED_MASK)
+		return;
+
 	__resched_curr(rq, get_lazy_tif_bit());
 }
 
probably fits the bill better.

> 
>  	cpu = cpu_of(rq);
> --
> 2.50.1 (Apple Git-155)
> 

-- 
Thanks and Regards,
Prateek

Re: [PATCH] sched: Add _TIF_NEED_RESCHED_LAZY to __resched_curr check

[Jemmy Wong]

Hi Prateek,

Thank you for the very detailed example, which helped me clearly understand the 
difference between TIF_NEED_RESCHED and TIF_NEED_RESCHED_LAZY.

The source code logic seems correct and effectively prevents repeated flag settings:
> if (cti->flags & ((1 << tif) | _TIF_NEED_RESCHED))
>     return;

The inclusion of _TIF_NEED_RESCHED in the check indicates:
1. TIF_NEED_RESCHED has higher priority/urgent than TIF_NEED_RESCHED_LAZY.
2. TIF_NEED_RESCHED_LAZY is not needed if TIF_NEED_RESCHED is set.

> On Sep 29, 2025, at 12:37 PM, K Prateek Nayak <kprateek.nayak@amd.com> wrote:
> __resched_curr() is used to set both TIF_NEED_RESCHED_LAZY and
> TIF_NEED_RESCHED.
> 
> By putting this check here, any effort to set NEED_RESCHED and force an
> early preemption will bail out if NEED_RESCHED_LAZY is already set which
> will delay the preemption.
> 
> An example:
> 
>    /* New fair task wakes up. */
>    check_preempt_wakeup_fair()
>        resched_curr_lazy()
>            __resched_curr(TIF_NEED_RESCHED_LAZY)
> 
>    /* New RT task wakes up. */
>    wakeup_preempt()
>        resched_curr()
>            __resched_curr(TIF_NEED_RESCHED)
>                /* Sees NEED_RESCHED_LAZY is already set. */
>                /* Does not do a set_preempt_need_resched() */
> 
>    ... /* Added latency */
>    sched_tick()
>        if (tif_test_bit(TIF_NEED_RESCHED_LAZY))
>            resched_curr()
>                __resched_curr(TIF_NEED_RESCHED)
>                    /* Again bails out early! */
> 
>    ... /* More latency! */
> 
> 
> So, the tick doesn't even upgrade the LAZY flag to a full NEED_RESCHED
> and the only time you actually schedule is either at exit to user mode
> or if a kthread decides to yield.
> 
> Going back to your commit message, something like:
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 7f1e5cb94c53..3275abce9ca2 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -1164,6 +1164,9 @@ static __always_inline int get_lazy_tif_bit(void)
> 
> void resched_curr_lazy(struct rq *rq)
> {
> + if (task_thread_info(rq->curr)->flags & TIF_NEED_RESCHED_MASK)
> + return;
> +
> __resched_curr(rq, get_lazy_tif_bit());
> }
> 
> probably fits the bill better.
> 
>> 
>> cpu = cpu_of(rq);
>> --
>> 2.50.1 (Apple Git-155)
>> 
> 
> -- 
> Thanks and Regards,
> Prateek
> 

Best Regards,
Jemmy