* [PATCH 1/3] sched/fair: Exclude delayed-dequeued tasks from sum_nr_running
2026-11-20 1:49 [PATCH 0/3] sched: Fix load accounting and improve imbalance calculation Srikar Dronamraju
@ 2026-11-20 1:49 ` Srikar Dronamraju
2026-11-20 10:54 ` Peter Zijlstra
2026-11-20 1:49 ` [PATCH 2/3] sched/fair: Use runnable task imbalance for asymmetric groups Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 3/3] sched/topology: Simplify cache_nice_tries initialization Srikar Dronamraju
2 siblings, 1 reply; 6+ messages in thread
From: Srikar Dronamraju @ 2026-11-20 1:49 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra, Ingo Molnar, Juri Lelli,
Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
Mel Gorman, Valentin Schneider, K Prateek Nayak
Cc: LKML, Srikar Dronamraju
Delayed dequeue keeps a sleeping task on the CFS runqueue until its
accumulated lag is repaid. While such tasks remain accounted in
rq->nr_running, they are no longer runnable and are not eligible to
consume CPU time.
update_sg_lb_stats() currently uses rq->nr_running when computing
sum_nr_running. As a result, delayed-dequeued tasks contribute to group
load statistics even though they are effectively sleeping. This can
inflate the apparent load of a group and influence load balancing
decisions.
Other metrics collected by update_sg_lb_stats(), such as group_runnable
and h_nr_runnable, already focus on runnable work. Counting delayed
entities in sum_nr_running makes it inconsistent with the rest of the
load-balancing statistics.
Exclude cfs_h_nr_delayed() tasks from sum_nr_running so that load
balancing decisions are based on runnable tasks rather than entities
waiting to complete delayed dequeue accounting.
Signed-off-by: Srikar Dronamraju <srikar@linux.ibm.com>
---
kernel/sched/fair.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d4cc0caee626..21f2b17894e3 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -12296,7 +12296,7 @@ static inline void update_sg_lb_stats(struct lb_env *env,
sgs->group_runnable += cpu_runnable(rq);
sgs->sum_h_nr_running += rq->cfs.h_nr_runnable;
- nr_running = rq->nr_running;
+ nr_running = rq->nr_running - cfs_h_nr_delayed(rq);
sgs->sum_nr_running += nr_running;
if (cpu_overutilized(i))
--
2.55.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/3] sched/fair: Exclude delayed-dequeued tasks from sum_nr_running
2026-11-20 1:49 ` [PATCH 1/3] sched/fair: Exclude delayed-dequeued tasks from sum_nr_running Srikar Dronamraju
@ 2026-11-20 10:54 ` Peter Zijlstra
2026-11-20 10:59 ` Peter Zijlstra
0 siblings, 1 reply; 6+ messages in thread
From: Peter Zijlstra @ 2026-11-20 10:54 UTC (permalink / raw)
To: Srikar Dronamraju
Cc: Ingo Molnar, Ingo Molnar, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
Valentin Schneider, K Prateek Nayak, LKML
On Fri, Nov 20, 2026 at 07:19:48AM +0530, Srikar Dronamraju wrote:
> Delayed dequeue keeps a sleeping task on the CFS runqueue until its
> accumulated lag is repaid. While such tasks remain accounted in
> rq->nr_running, they are no longer runnable and are not eligible to
> consume CPU time.
>
> update_sg_lb_stats() currently uses rq->nr_running when computing
> sum_nr_running. As a result, delayed-dequeued tasks contribute to group
> load statistics even though they are effectively sleeping. This can
> inflate the apparent load of a group and influence load balancing
> decisions.
>
> Other metrics collected by update_sg_lb_stats(), such as group_runnable
> and h_nr_runnable, already focus on runnable work. Counting delayed
> entities in sum_nr_running makes it inconsistent with the rest of the
> load-balancing statistics.
>
> Exclude cfs_h_nr_delayed() tasks from sum_nr_running so that load
> balancing decisions are based on runnable tasks rather than entities
> waiting to complete delayed dequeue accounting.
>
> Signed-off-by: Srikar Dronamraju <srikar@linux.ibm.com>
> ---
> kernel/sched/fair.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index d4cc0caee626..21f2b17894e3 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -12296,7 +12296,7 @@ static inline void update_sg_lb_stats(struct lb_env *env,
> sgs->group_runnable += cpu_runnable(rq);
> sgs->sum_h_nr_running += rq->cfs.h_nr_runnable;
>
> - nr_running = rq->nr_running;
> + nr_running = rq->nr_running - cfs_h_nr_delayed(rq);
h_nr_runnable already excludes delayed; its is h_nr_queued that includes
it.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/3] sched/fair: Exclude delayed-dequeued tasks from sum_nr_running
2026-11-20 10:54 ` Peter Zijlstra
@ 2026-11-20 10:59 ` Peter Zijlstra
0 siblings, 0 replies; 6+ messages in thread
From: Peter Zijlstra @ 2026-11-20 10:59 UTC (permalink / raw)
To: Srikar Dronamraju
Cc: Ingo Molnar, Ingo Molnar, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
Valentin Schneider, K Prateek Nayak, LKML
On Fri, Nov 20, 2026 at 11:54:56AM +0100, Peter Zijlstra wrote:
> On Fri, Nov 20, 2026 at 07:19:48AM +0530, Srikar Dronamraju wrote:
> > Delayed dequeue keeps a sleeping task on the CFS runqueue until its
> > accumulated lag is repaid. While such tasks remain accounted in
> > rq->nr_running, they are no longer runnable and are not eligible to
> > consume CPU time.
> >
> > update_sg_lb_stats() currently uses rq->nr_running when computing
> > sum_nr_running. As a result, delayed-dequeued tasks contribute to group
> > load statistics even though they are effectively sleeping. This can
> > inflate the apparent load of a group and influence load balancing
> > decisions.
> >
> > Other metrics collected by update_sg_lb_stats(), such as group_runnable
> > and h_nr_runnable, already focus on runnable work. Counting delayed
> > entities in sum_nr_running makes it inconsistent with the rest of the
> > load-balancing statistics.
> >
> > Exclude cfs_h_nr_delayed() tasks from sum_nr_running so that load
> > balancing decisions are based on runnable tasks rather than entities
> > waiting to complete delayed dequeue accounting.
> >
> > Signed-off-by: Srikar Dronamraju <srikar@linux.ibm.com>
> > ---
> > kernel/sched/fair.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index d4cc0caee626..21f2b17894e3 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -12296,7 +12296,7 @@ static inline void update_sg_lb_stats(struct lb_env *env,
> > sgs->group_runnable += cpu_runnable(rq);
> > sgs->sum_h_nr_running += rq->cfs.h_nr_runnable;
> >
> > - nr_running = rq->nr_running;
> > + nr_running = rq->nr_running - cfs_h_nr_delayed(rq);
>
>
> h_nr_runnable already excludes delayed; its is h_nr_queued that includes
> it.
Argh, I cannot read, my bad. I looked at the line above that uses
h_nr_runnable.
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 2/3] sched/fair: Use runnable task imbalance for asymmetric groups
2026-11-20 1:49 [PATCH 0/3] sched: Fix load accounting and improve imbalance calculation Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 1/3] sched/fair: Exclude delayed-dequeued tasks from sum_nr_running Srikar Dronamraju
@ 2026-11-20 1:49 ` Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 3/3] sched/topology: Simplify cache_nice_tries initialization Srikar Dronamraju
2 siblings, 0 replies; 6+ messages in thread
From: Srikar Dronamraju @ 2026-11-20 1:49 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra, Ingo Molnar, Juri Lelli,
Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
Mel Gorman, Valentin Schneider, K Prateek Nayak
Cc: LKML, Srikar Dronamraju
The current load-balancing logic uses idle CPU counts to estimate
imbalance when SD_PREFER_SIBLING is not set. This works reasonably well
when groups within a sched_domain are symmetric, since the number of
idle CPUs closely tracks the distribution of runnable tasks.
However, idle CPU counts become misleading when groups contain different
numbers of cores. A larger group naturally has more idle CPUs than a
smaller group, even when its workload is proportionally heavier.
Consider a sched_domain containing two groups:
Group A: 2 cores, 1 runnable task
Group B: 4 cores, 4 runnable tasks
Although Group B is more heavily loaded, it may still report more idle
CPUs simply because it contains more cores. Balancing decisions based on
idle CPU counts can therefore favor the larger group and pull additional
work toward it.
The scheduler already uses runnable task distribution when
SD_PREFER_SIBLING is enabled, as this provides a more accurate view of
load distribution between groups. Apply the same approach when groups
have different numbers of cores and calculate imbalance based on
runnable tasks instead of idle CPU counts.
This improves load balancing on asymmetric topologies and avoids
overestimating the capacity available in larger groups.
Signed-off-by: Srikar Dronamraju <srikar@linux.ibm.com>
---
kernel/sched/fair.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 21f2b17894e3..f5fecbcd2d84 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -13164,7 +13164,8 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s
return;
}
- if (busiest->group_weight == 1 || sds->prefer_sibling) {
+ if (busiest->group_weight == 1 || sds->prefer_sibling ||
+ sds->busiest->cores != sds->local->cores) {
/*
* When prefer sibling, evenly spread running tasks on
* groups.
--
2.55.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 3/3] sched/topology: Simplify cache_nice_tries initialization
2026-11-20 1:49 [PATCH 0/3] sched: Fix load accounting and improve imbalance calculation Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 1/3] sched/fair: Exclude delayed-dequeued tasks from sum_nr_running Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 2/3] sched/fair: Use runnable task imbalance for asymmetric groups Srikar Dronamraju
@ 2026-11-20 1:49 ` Srikar Dronamraju
2 siblings, 0 replies; 6+ messages in thread
From: Srikar Dronamraju @ 2026-11-20 1:49 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra, Ingo Molnar, Juri Lelli,
Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
Mel Gorman, Valentin Schneider, K Prateek Nayak
Cc: LKML, Srikar Dronamraju
sched_domain initialization currently starts with cache_nice_tries = 0
and later updates it based on the domain type. Most sched domains
eventually end up with cache_nice_tries set to 1, with LLC-sharing
domains and most non-NUMA domains explicitly overriding the initial
value.
As a result, the current code initializes cache_nice_tries to a value
that is not the common case and subsequently reassigns it during domain
setup.
The only domain type that retains a value of 0 is a
SD_SHARE_CPUCAPACITY domain, while NUMA domains use a value of 2 and
most remaining domains use a value of 1.
Initialize cache_nice_tries to 1 by default and only override it for
the special cases that require a different value. This reduces the
number of assignments required during sched-domain construction and
makes the resulting configuration easier to follow.
No functional change is intended. The resulting values remain unchanged:
SD_SHARE_CPUCAPACITY : 0
SD_SHARE_LLC : 1
other non-NUMA : 1
SD_NUMA : 2
Signed-off-by: Srikar Dronamraju <srikar@linux.ibm.com>
---
kernel/sched/topology.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 99878da03bce..da4f1e127341 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -1979,7 +1979,7 @@ sd_init(struct sched_domain_topology_level *tl,
.busy_factor = 16,
.imbalance_pct = 117,
- .cache_nice_tries = 0,
+ .cache_nice_tries = 1,
.flags = 1*SD_BALANCE_NEWIDLE
| 1*SD_BALANCE_EXEC
@@ -2018,10 +2018,7 @@ sd_init(struct sched_domain_topology_level *tl,
*/
if (sd->flags & SD_SHARE_CPUCAPACITY) {
sd->imbalance_pct = 110;
-
- } else if (sd->flags & SD_SHARE_LLC) {
- sd->imbalance_pct = 117;
- sd->cache_nice_tries = 1;
+ sd->cache_nice_tries = 0;
#ifdef CONFIG_NUMA
} else if (sd->flags & SD_NUMA) {
@@ -2036,8 +2033,6 @@ sd_init(struct sched_domain_topology_level *tl,
}
#endif /* CONFIG_NUMA */
- } else {
- sd->cache_nice_tries = 1;
}
sd->private = sdd;
--
2.55.0
^ permalink raw reply [flat|nested] 6+ messages in thread