* [PATCH 0/3] sched: Fix load accounting and improve imbalance calculation
@ 2026-11-20 1:49 Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 1/3] sched/fair: Exclude delayed-dequeued tasks from sum_nr_running Srikar Dronamraju
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Srikar Dronamraju @ 2026-11-20 1:49 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra, Ingo Molnar, Juri Lelli,
Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
Mel Gorman, Valentin Schneider, K Prateek Nayak
Cc: LKML, Srikar Dronamraju
This series contains a few independent scheduler fixes and cleanups.
The first patch corrects load accounting in update_sg_lb_stats() by
excluding delayed-dequeued tasks from sum_nr_running. Delayed-dequeued
tasks remain visible in rq->nr_running even though they are no longer
runnable, causing load balancing statistics to overestimate runnable
load.
The second patch improves imbalance calculation for domains containing
groups with different numbers of cores. Idle CPU counts alone can be
misleading in such cases since larger groups naturally contain more idle
CPUs even when they are proportionally more heavily loaded. Use the
runnable task distribution for imbalance calculation when comparing
groups of differing sizes.
The final patch is a cleanup of cache_nice_tries initialization during
sched-domain construction. The resulting values remain unchanged but the
initialization becomes simpler and easier to follow.
Patches 1 and 2 contain functional fixes while patch 3 is a cleanup.
Srikar Dronamraju (3):
sched/fair: Exclude delayed-dequeued tasks from sum_nr_running
sched/fair: Use runnable task imbalance for asymmetric groups
sched/topology: Simplify cache_nice_tries initialization
kernel/sched/fair.c | 5 +++--
kernel/sched/topology.c | 9 ++-------
2 files changed, 5 insertions(+), 9 deletions(-)
--
2.55.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/3] sched/fair: Exclude delayed-dequeued tasks from sum_nr_running
2026-11-20 1:49 [PATCH 0/3] sched: Fix load accounting and improve imbalance calculation Srikar Dronamraju
@ 2026-11-20 1:49 ` Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 2/3] sched/fair: Use runnable task imbalance for asymmetric groups Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 3/3] sched/topology: Simplify cache_nice_tries initialization Srikar Dronamraju
2 siblings, 0 replies; 5+ messages in thread
From: Srikar Dronamraju @ 2026-11-20 1:49 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra, Ingo Molnar, Juri Lelli,
Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
Mel Gorman, Valentin Schneider, K Prateek Nayak
Cc: LKML, Srikar Dronamraju
Delayed dequeue keeps a sleeping task on the CFS runqueue until its
accumulated lag is repaid. While such tasks remain accounted in
rq->nr_running, they are no longer runnable and are not eligible to
consume CPU time.
update_sg_lb_stats() currently uses rq->nr_running when computing
sum_nr_running. As a result, delayed-dequeued tasks contribute to group
load statistics even though they are effectively sleeping. This can
inflate the apparent load of a group and influence load balancing
decisions.
Other metrics collected by update_sg_lb_stats(), such as group_runnable
and h_nr_runnable, already focus on runnable work. Counting delayed
entities in sum_nr_running makes it inconsistent with the rest of the
load-balancing statistics.
Exclude cfs_h_nr_delayed() tasks from sum_nr_running so that load
balancing decisions are based on runnable tasks rather than entities
waiting to complete delayed dequeue accounting.
Signed-off-by: Srikar Dronamraju <srikar@linux.ibm.com>
---
kernel/sched/fair.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d4cc0caee626..21f2b17894e3 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -12296,7 +12296,7 @@ static inline void update_sg_lb_stats(struct lb_env *env,
sgs->group_runnable += cpu_runnable(rq);
sgs->sum_h_nr_running += rq->cfs.h_nr_runnable;
- nr_running = rq->nr_running;
+ nr_running = rq->nr_running - cfs_h_nr_delayed(rq);
sgs->sum_nr_running += nr_running;
if (cpu_overutilized(i))
--
2.55.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 2/3] sched/fair: Use runnable task imbalance for asymmetric groups
2026-11-20 1:49 [PATCH 0/3] sched: Fix load accounting and improve imbalance calculation Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 1/3] sched/fair: Exclude delayed-dequeued tasks from sum_nr_running Srikar Dronamraju
@ 2026-11-20 1:49 ` Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 3/3] sched/topology: Simplify cache_nice_tries initialization Srikar Dronamraju
2 siblings, 0 replies; 5+ messages in thread
From: Srikar Dronamraju @ 2026-11-20 1:49 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra, Ingo Molnar, Juri Lelli,
Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
Mel Gorman, Valentin Schneider, K Prateek Nayak
Cc: LKML, Srikar Dronamraju
The current load-balancing logic uses idle CPU counts to estimate
imbalance when SD_PREFER_SIBLING is not set. This works reasonably well
when groups within a sched_domain are symmetric, since the number of
idle CPUs closely tracks the distribution of runnable tasks.
However, idle CPU counts become misleading when groups contain different
numbers of cores. A larger group naturally has more idle CPUs than a
smaller group, even when its workload is proportionally heavier.
Consider a sched_domain containing two groups:
Group A: 2 cores, 1 runnable task
Group B: 4 cores, 4 runnable tasks
Although Group B is more heavily loaded, it may still report more idle
CPUs simply because it contains more cores. Balancing decisions based on
idle CPU counts can therefore favor the larger group and pull additional
work toward it.
The scheduler already uses runnable task distribution when
SD_PREFER_SIBLING is enabled, as this provides a more accurate view of
load distribution between groups. Apply the same approach when groups
have different numbers of cores and calculate imbalance based on
runnable tasks instead of idle CPU counts.
This improves load balancing on asymmetric topologies and avoids
overestimating the capacity available in larger groups.
Signed-off-by: Srikar Dronamraju <srikar@linux.ibm.com>
---
kernel/sched/fair.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 21f2b17894e3..f5fecbcd2d84 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -13164,7 +13164,8 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s
return;
}
- if (busiest->group_weight == 1 || sds->prefer_sibling) {
+ if (busiest->group_weight == 1 || sds->prefer_sibling ||
+ sds->busiest->cores != sds->local->cores) {
/*
* When prefer sibling, evenly spread running tasks on
* groups.
--
2.55.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 3/3] sched/topology: Simplify cache_nice_tries initialization
2026-11-20 1:49 [PATCH 0/3] sched: Fix load accounting and improve imbalance calculation Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 1/3] sched/fair: Exclude delayed-dequeued tasks from sum_nr_running Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 2/3] sched/fair: Use runnable task imbalance for asymmetric groups Srikar Dronamraju
@ 2026-11-20 1:49 ` Srikar Dronamraju
2 siblings, 0 replies; 5+ messages in thread
From: Srikar Dronamraju @ 2026-11-20 1:49 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra, Ingo Molnar, Juri Lelli,
Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
Mel Gorman, Valentin Schneider, K Prateek Nayak
Cc: LKML, Srikar Dronamraju
sched_domain initialization currently starts with cache_nice_tries = 0
and later updates it based on the domain type. Most sched domains
eventually end up with cache_nice_tries set to 1, with LLC-sharing
domains and most non-NUMA domains explicitly overriding the initial
value.
As a result, the current code initializes cache_nice_tries to a value
that is not the common case and subsequently reassigns it during domain
setup.
The only domain type that retains a value of 0 is a
SD_SHARE_CPUCAPACITY domain, while NUMA domains use a value of 2 and
most remaining domains use a value of 1.
Initialize cache_nice_tries to 1 by default and only override it for
the special cases that require a different value. This reduces the
number of assignments required during sched-domain construction and
makes the resulting configuration easier to follow.
No functional change is intended. The resulting values remain unchanged:
SD_SHARE_CPUCAPACITY : 0
SD_SHARE_LLC : 1
other non-NUMA : 1
SD_NUMA : 2
Signed-off-by: Srikar Dronamraju <srikar@linux.ibm.com>
---
kernel/sched/topology.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 99878da03bce..da4f1e127341 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -1979,7 +1979,7 @@ sd_init(struct sched_domain_topology_level *tl,
.busy_factor = 16,
.imbalance_pct = 117,
- .cache_nice_tries = 0,
+ .cache_nice_tries = 1,
.flags = 1*SD_BALANCE_NEWIDLE
| 1*SD_BALANCE_EXEC
@@ -2018,10 +2018,7 @@ sd_init(struct sched_domain_topology_level *tl,
*/
if (sd->flags & SD_SHARE_CPUCAPACITY) {
sd->imbalance_pct = 110;
-
- } else if (sd->flags & SD_SHARE_LLC) {
- sd->imbalance_pct = 117;
- sd->cache_nice_tries = 1;
+ sd->cache_nice_tries = 0;
#ifdef CONFIG_NUMA
} else if (sd->flags & SD_NUMA) {
@@ -2036,8 +2033,6 @@ sd_init(struct sched_domain_topology_level *tl,
}
#endif /* CONFIG_NUMA */
- } else {
- sd->cache_nice_tries = 1;
}
sd->private = sdd;
--
2.55.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 0/3] sched: Fix load accounting and improve imbalance calculation
@ 2026-11-20 0:35 Srikar Dronamraju
0 siblings, 0 replies; 5+ messages in thread
From: Srikar Dronamraju @ 2026-11-20 0:35 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra, Ingo Molnar, Juri Lelli,
Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Ben Segall,
Mel Gorman, Valentin Schneider, K Prateek Nayak
Cc: LKML, Srikar Dronamraju
This series contains a few independent scheduler fixes and cleanups.
The first patch corrects load accounting in update_sg_lb_stats() by
excluding delayed-dequeued tasks from sum_nr_running. Delayed-dequeued
tasks remain visible in rq->nr_running even though they are no longer
runnable, causing load balancing statistics to overestimate runnable
load.
The second patch improves imbalance calculation for domains containing
groups with different numbers of cores. Idle CPU counts alone can be
misleading in such cases since larger groups naturally contain more idle
CPUs even when they are proportionally more heavily loaded. Use the
runnable task distribution for imbalance calculation when comparing
groups of differing sizes.
The final patch is a cleanup of cache_nice_tries initialization during
sched-domain construction. The resulting values remain unchanged but the
initialization becomes simpler and easier to follow.
Patches 1 and 2 contain functional fixes while patch 3 is a cleanup.
Srikar Dronamraju (3):
sched/fair: Exclude delayed-dequeued tasks from sum_nr_running
sched/fair: Use runnable task imbalance for asymmetric groups
sched/topology: Simplify cache_nice_tries initialization
kernel/sched/fair.c | 5 +++--
kernel/sched/topology.c | 9 ++-------
2 files changed, 5 insertions(+), 9 deletions(-)
--
2.55.0
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-11-20 1:50 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-20 1:49 [PATCH 0/3] sched: Fix load accounting and improve imbalance calculation Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 1/3] sched/fair: Exclude delayed-dequeued tasks from sum_nr_running Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 2/3] sched/fair: Use runnable task imbalance for asymmetric groups Srikar Dronamraju
2026-11-20 1:49 ` [PATCH 3/3] sched/topology: Simplify cache_nice_tries initialization Srikar Dronamraju
-- strict thread matches above, loose matches on Subject: below --
2026-11-20 0:35 [PATCH 0/3] sched: Fix load accounting and improve imbalance calculation Srikar Dronamraju
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®