From: "tip-bot2 for Frederic Weisbecker" <tip-bot2@linutronix.de>
To: linux-tip-commits@vger.kernel.org
Cc: Frederic Weisbecker <frederic@kernel.org>,
Thomas Gleixner <tglx@kernel.org>,
Shrikanth Hegde <sshegde@linux.ibm.com>,
x86@kernel.org, linux-kernel@vger.kernel.org
Subject: [tip: timers/nohz] sched/cputime: Handle dyntick-idle steal time correctly
Date: Tue, 02 Jun 2026 19:30:03 -0000 [thread overview]
Message-ID: <178042860321.710.17410796296233941593.tip-bot2@tip-bot2> (raw)
In-Reply-To: <20260508131647.43868-16-frederic@kernel.org>
The following commit has been merged into the timers/nohz branch of tip:
Commit-ID: 6199f9999a9b62b2b84a1bf5b52a9fd0bb8de5af
Gitweb: https://git.kernel.org/tip/6199f9999a9b62b2b84a1bf5b52a9fd0bb8de5af
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 08 May 2026 15:16:47 +02:00
Committer: Thomas Gleixner <tglx@kernel.org>
CommitterDate: Tue, 02 Jun 2026 21:27:26 +02:00
sched/cputime: Handle dyntick-idle steal time correctly
The dyntick-idle steal time is currently accounted when the tick restarts
but the stolen idle time is not subtracted from the idle time that was
already accounted. This is to avoid observing the idle time going backward
as the dyntick-idle cputime accessors can't reliably know in advance the
stolen idle time.
In order to maintain a forward progressing idle cputime while subtracting
idle steal time from it, keep track of the previously accounted idle stolen
time and substract it from _later_ idle cputime accounting.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Link: https://patch.msgid.link/20260508131647.43868-16-frederic@kernel.org
---
include/linux/kernel_stat.h | 1 +
kernel/sched/cputime.c | 28 +++++++++++++++++++++++-----
2 files changed, 24 insertions(+), 5 deletions(-)
diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h
index 512104b..fce1392 100644
--- a/include/linux/kernel_stat.h
+++ b/include/linux/kernel_stat.h
@@ -39,6 +39,7 @@ struct kernel_cpustat {
bool idle_elapse;
seqcount_t idle_sleeptime_seq;
u64 idle_entrytime;
+ u64 idle_stealtime[2];
#endif
u64 cpustat[NR_STATS];
};
diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c
index 94be22a..244b574 100644
--- a/kernel/sched/cputime.c
+++ b/kernel/sched/cputime.c
@@ -425,19 +425,32 @@ static inline void irqtime_account_process_tick(struct task_struct *p, int user_
static void kcpustat_idle_stop(struct kernel_cpustat *kc, u64 now)
{
u64 *cpustat = kc->cpustat;
- u64 delta;
+ u64 delta, steal, steal_delta;
+ int iowait;
if (!kc->idle_elapse)
return;
+ iowait = nr_iowait_cpu(smp_processor_id()) > 0;
delta = now - kc->idle_entrytime;
+ steal = steal_account_process_time(delta);
+ /*
+ * Record the idle time after substracting the steal time from
+ * previous update sequence. Don't substract the steal time from
+ * the current update sequence to avoid readers moving backward.
+ */
write_seqcount_begin(&kc->idle_sleeptime_seq);
- if (nr_iowait_cpu(smp_processor_id()) > 0)
+ steal_delta = min_t(u64, kc->idle_stealtime[iowait], delta);
+ delta -= steal_delta;
+ kc->idle_stealtime[iowait] -= steal_delta;
+
+ if (iowait)
cpustat[CPUTIME_IOWAIT] += delta;
else
cpustat[CPUTIME_IDLE] += delta;
+ kc->idle_stealtime[iowait] += steal;
kc->idle_entrytime = now;
kc->idle_elapse = false;
write_seqcount_end(&kc->idle_sleeptime_seq);
@@ -464,7 +477,6 @@ void kcpustat_dyntick_stop(u64 now)
kcpustat_idle_stop(kc, now);
kc->idle_dyntick = false;
vtime_dyntick_stop();
- steal_account_process_time(ULONG_MAX);
}
}
@@ -508,6 +520,7 @@ static u64 kcpustat_field_dyntick(int cpu, enum cpu_usage_stat idx,
bool compute_delta, u64 now)
{
struct kernel_cpustat *kc = &kcpustat_cpu(cpu);
+ int iowait = idx == CPUTIME_IOWAIT;
u64 *cpustat = kc->cpustat;
unsigned int seq;
u64 idle;
@@ -516,8 +529,13 @@ static u64 kcpustat_field_dyntick(int cpu, enum cpu_usage_stat idx,
seq = read_seqcount_begin(&kc->idle_sleeptime_seq);
idle = cpustat[idx];
- if (kc->idle_elapse && compute_delta && now > kc->idle_entrytime)
- idle += (now - kc->idle_entrytime);
+
+ if (kc->idle_elapse && compute_delta && now > kc->idle_entrytime) {
+ u64 delta = now - kc->idle_entrytime;
+
+ delta -= min_t(u64, kc->idle_stealtime[iowait], delta);
+ idle += delta;
+ }
} while (read_seqcount_retry(&kc->idle_sleeptime_seq, seq));
return idle;
next prev parent reply other threads:[~2026-06-02 19:30 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-05-08 13:16 [PATCH 00/15 v4] tick/sched: Refactor idle cputime accounting Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 01/15] tick/sched: Fix TOCTOU in nohz idle time fetch Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 02/15] sched/idle: Handle offlining first in idle loop Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 03/15] sched/cputime: Remove superfluous and error prone kcpustat_field() parameter Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 04/15] sched/cputime: Correctly support generic vtime idle time Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 05/15] powerpc/time: Prepare to stop elapsing in dynticks-idle Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 06/15] s390/time: " Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 07/15] tick/sched: Unify idle cputime accounting Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 08/15] tick/sched: Remove nohz disabled special case in cputime fetch Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 09/15] tick/sched: Move dyntick-idle cputime accounting to cputime code Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 10/15] tick/sched: Remove unused fields Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 11/15] tick/sched: Account tickless idle cputime only when tick is stopped Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 12/15] tick/sched: Consolidate idle time fetching APIs Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 13/15] sched/cputime: Provide get_cpu_[idle|iowait]_time_us() off-case Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 14/15] sched/cputime: Handle idle irqtime gracefully Frederic Weisbecker
2026-06-02 19:30 ` [tip: timers/nohz] " tip-bot2 for Frederic Weisbecker
2026-05-08 13:16 ` [PATCH 15/15] sched/cputime: Handle dyntick-idle steal time correctly Frederic Weisbecker
2026-06-02 19:30 ` tip-bot2 for Frederic Weisbecker [this message]
2026-05-26 10:42 ` [PATCH 00/15 v4] tick/sched: Refactor idle cputime accounting Frederic Weisbecker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=178042860321.710.17410796296233941593.tip-bot2@tip-bot2 \
--to=tip-bot2@linutronix.de \
--cc=frederic@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=sshegde@linux.ibm.com \
--cc=tglx@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®