[RFC PATCH] sched: Change nr_uninterruptible from unsigned to signed int

[RFC PATCH] sched: Change nr_uninterruptible from unsigned to signed int

[Aruna Ramakrishna]

We have encountered a bug where the load average displayed in top is
abnormally high and obviously incorrect. The real values look like this
(this is a production env, not a simulated one):

top - 13:54:24 up 68 days, 14:33,  7 users,  load average:
4294967298.80, 4294967298.55, 4294967298.58
Threads: 5764 total,   5 running, 5759 sleeping,   0 stopped,   0 zombie

From digging a bit into the vmcore:

crash> p calc_load_tasks
calc_load_tasks = $1 = {
  counter = 4294967297
}

which is:

crash> eval 4294967297
hexadecimal: 100000001

It seems like an overflow, since the value exceeds UINT_MAX.

Checking further:

The nr_uninterruptible values for each of the CPU runqueues is large,
and when they are summed up, the sum exceeds UINT_MAX, and the result
is stored in a long, which preserves this overflow.

long calc_load_fold_active(struct rq *this_rq, long adjust)
{
        long nr_active, delta = 0;

        nr_active = this_rq->nr_running - adjust;
        nr_active += (int)this_rq->nr_uninterruptible;
...

From the vmcore:

>>> sum=0
>>> for cpu in for_each_online_cpu(prog):
...     rq = per_cpu(prog["runqueues"], cpu)
...     nr_unint = rq.nr_uninterruptible.value_()
...     sum += nr_unint
...     print(f"CPU {cpu}: nr_uninterruptible = {hex(nr_unint)}")
...     print(f"sum {hex(sum)}")
...
CPU 0: nr_uninterruptible = 0x638dd3
sum 0x638dd3
CPU 1: nr_uninterruptible = 0x129fb26
sum 0x18d88f9
CPU 2: nr_uninterruptible = 0xd8281f
sum 0x265b118
...
CPU 94: nr_uninterruptible = 0xe0a86
sum 0xfff1e855
CPU 95: nr_uninterruptible = 0xe17ab
sum 0x100000000

This is what we see, stored in calc_load_tasks. The correct sum here would be 0.

From kernel/sched/loadavg.c:

 *  - cpu_rq()->nr_uninterruptible isn't accurately tracked per-CPU
because
 *    this would add another cross-CPU cacheline miss and atomic
operation
 *    to the wakeup path. Instead we increment on whatever CPU the task
ran
 *    when it went into uninterruptible state and decrement on whatever
CPU
 *    did the wakeup. This means that only the sum of nr_uninterruptible
over
 *    all CPUs yields the correct result.
 *

It seems that rq->nr_uninterruptible can go to large (positive) values
for one CPU if a lot of tasks were migrated off of that CPU after going
into an uninterruptible state. If they’re woken up on another CPU -
those target CPUs will have negative nr_uninterruptible values. I think
the casting of an unsigned int to signed int and adding to a long is
not preserving the sign, and results in a large positive value rather
than the correct sum of zero.

I suspect the bug surfaced as a side effect of this commit:

commit e6fe3f422be128b7d65de607f6ae67bedc55f0ca
Author: Alexey Dobriyan <adobriyan@gmail.com>
Date:   Thu Apr 22 23:02:28 2021 +0300

    sched: Make multiple runqueue task counters 32-bit

    Make:

            struct dl_rq::dl_nr_migratory
            struct dl_rq::dl_nr_running

            struct rt_rq::rt_nr_boosted
            struct rt_rq::rt_nr_migratory
            struct rt_rq::rt_nr_total

            struct rq::nr_uninterruptible

    32-bit.

    If total number of tasks can't exceed 2**32 (and less due to futex
pid
    limits), then per-runqueue counters can't as well.

    This patchset has been sponsored by REX Prefix Eradication Society.
...

which changed the counter nr_uninterruptible from unsigned long to unsigned
int.

Since nr_uninterruptible can be a positive or negative number, change
the type from unsigned int to signed int.

Another possible solution would be to partially rollback e6fe3f422be1,
and change nr_uninterruptible back to unsigned long.

Signed-off-by: Aruna Ramakrishna <aruna.ramakrishna@oracle.com>
---
 kernel/sched/sched.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 475bb5998295..f6d21278e64e 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1149,7 +1149,7 @@ struct rq {
 	 * one CPU and if it got migrated afterwards it may decrease
 	 * it on another CPU. Always updated under the runqueue lock:
 	 */
-	unsigned int		nr_uninterruptible;
+	int 			nr_uninterruptible;
 
 	union {
 		struct task_struct __rcu *donor; /* Scheduler context */

base-commit: 86731a2a651e58953fc949573895f2fa6d456841
prerequisite-patch-id: dd6db7012c5094dec89e689ba56fd3551d2b4a40
-- 
2.43.5

Re: [RFC PATCH] sched: Change nr_uninterruptible from unsigned to signed int

[Peter Zijlstra]

(Please, be careful not to wrap quoted text, unwrapped it for you)

On Wed, Jun 25, 2025 at 04:48:36AM +0000, Aruna Ramakrishna wrote:

> We have encountered a bug where the load average displayed in top is
> abnormally high and obviously incorrect. The real values look like this
> (this is a production env, not a simulated one):

Whoopie..

> The nr_uninterruptible values for each of the CPU runqueues is large,
> and when they are summed up, the sum exceeds UINT_MAX, and the result
> is stored in a long, which preserves this overflow.

Right, that's the problem spot.

> long calc_load_fold_active(struct rq *this_rq, long adjust)
> {
>         long nr_active, delta = 0;
> 
>         nr_active = this_rq->nr_running - adjust;
>         nr_active += (int)this_rq->nr_uninterruptible;
> ...

> From kernel/sched/loadavg.c:
> 
>  *  - cpu_rq()->nr_uninterruptible isn't accurately tracked per-CPU because
>  *    this would add another cross-CPU cacheline miss and atomic operation
>  *    to the wakeup path. Instead we increment on whatever CPU the task ran
>  *    when it went into uninterruptible state and decrement on whatever CPU
>  *    did the wakeup. This means that only the sum of nr_uninterruptible over
>  *    all CPUs yields the correct result.
>  *
> 
> It seems that rq->nr_uninterruptible can go to large (positive) values
> for one CPU if a lot of tasks were migrated off of that CPU after going
> into an uninterruptible state. If they’re woken up on another CPU -
> those target CPUs will have negative nr_uninterruptible values. I think
> the casting of an unsigned int to signed int and adding to a long is
> not preserving the sign, and results in a large positive value rather
> than the correct sum of zero.

So very close, yet so far...

> I suspect the bug surfaced as a side effect of this commit:
> 
> commit e6fe3f422be128b7d65de607f6ae67bedc55f0ca
> Author: Alexey Dobriyan <adobriyan@gmail.com>
> Date:   Thu Apr 22 23:02:28 2021 +0300
> 
>     sched: Make multiple runqueue task counters 32-bit
> 
>     Make:
> 
>             struct dl_rq::dl_nr_migratory
>             struct dl_rq::dl_nr_running
> 
>             struct rt_rq::rt_nr_boosted
>             struct rt_rq::rt_nr_migratory
>             struct rt_rq::rt_nr_total
> 
>             struct rq::nr_uninterruptible
> 
>     32-bit.
> 
>     If total number of tasks can't exceed 2**32 (and less due to futex pid
>     limits), then per-runqueue counters can't as well.
> 
>     This patchset has been sponsored by REX Prefix Eradication Society.
> ...
> 
> which changed the counter nr_uninterruptible from unsigned long to unsigned
> int.
> 
> Since nr_uninterruptible can be a positive or negative number, change
> the type from unsigned int to signed int.

(Strictly speaking it's making things worse, since signed overflow is UB
in regular C -- luckily we kernel folks have our own dialect and signed
and unsigned are both expected to wrap 2s-complement).

Also, we're already casting to (int) in the only place where we consume
the value. So changing the type should make no difference what so ever,
right?

> Another possible solution would be to partially rollback e6fe3f422be1,
> and change nr_uninterruptible back to unsigned long.

I think I prefer this.