From: "tip-bot2 for Peter Zijlstra" <tip-bot2@linutronix.de>
To: linux-tip-commits@vger.kernel.org
Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>,
x86@kernel.org, linux-kernel@vger.kernel.org
Subject: [tip: sched/core] sched/fair: Reflow sched_balance_rq()
Date: Tue, 30 Jun 2026 09:03:34 -0000 [thread overview]
Message-ID: <178281021463.3843924.17697492618086729638.tip-bot2@tip-bot2> (raw)
In-Reply-To: <20260618105627.GP49951@noisy.programming.kicks-ass.net>
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 296744614dd061287c9812a15521a64e292a90b4
Gitweb: https://git.kernel.org/tip/296744614dd061287c9812a15521a64e292a90b4
Author: Peter Zijlstra <peterz@infradead.org>
AuthorDate: Thu, 18 Jun 2026 10:51:49 +02:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Tue, 30 Jun 2026 10:56:56 +02:00
sched/fair: Reflow sched_balance_rq()
Reflow to reduce indenting.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260618105627.GP49951@noisy.programming.kicks-ass.net
---
kernel/sched/fair.c | 128 ++++++++++++++++++++----------------------
kernel/sched/sched.h | 19 +++++-
2 files changed, 78 insertions(+), 69 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index ce836ed..ce05acf 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -13539,82 +13539,78 @@ more_balance:
}
}
- if (!ld_moved) {
- schedstat_inc(sd->lb_failed[idle]);
- /*
- * Increment the failure counter only on periodic balance.
- * We do not want newidle balance, which can be very
- * frequent, pollute the failure counter causing
- * excessive cache_hot migrations and active balances.
- *
- * Similarly for migration_misfit which is not related to
- * load/util migration, don't pollute nr_balance_failed.
- *
- * The same for cache aware scheduling's allowance for
- * load imbalance. If regular load balance does not
- * migrate task due to LLC locality, it is a expected
- * behavior and don't pollute nr_balance_failed.
- * See can_migrate_task().
- */
- if (idle != CPU_NEWLY_IDLE &&
- env.migration_type != migrate_misfit &&
- !(env.flags & LBF_LLC_PINNED))
- sd->nr_balance_failed++;
+ if (ld_moved) {
+ sd->nr_balance_failed = 0;
+ goto out_unbalanced;
+ }
- if (need_active_balance(&env)) {
- unsigned long flags;
+ schedstat_inc(sd->lb_failed[idle]);
+ /*
+ * Increment the failure counter only on periodic balance.
+ * We do not want newidle balance, which can be very
+ * frequent, pollute the failure counter causing
+ * excessive cache_hot migrations and active balances.
+ *
+ * Similarly for migration_misfit which is not related to
+ * load/util migration, don't pollute nr_balance_failed.
+ *
+ * The same for cache aware scheduling's allowance for
+ * load imbalance. If regular load balance does not
+ * migrate task due to LLC locality, it is a expected
+ * behavior and don't pollute nr_balance_failed.
+ * See can_migrate_task().
+ */
+ if (idle != CPU_NEWLY_IDLE &&
+ env.migration_type != migrate_misfit &&
+ !(env.flags & LBF_LLC_PINNED))
+ sd->nr_balance_failed++;
- raw_spin_rq_lock_irqsave(busiest, flags);
+ if (!need_active_balance(&env))
+ goto out_unbalanced;
- /*
- * Don't kick the active_load_balance_cpu_stop,
- * if the curr task on busiest CPU can't be
- * moved to this_cpu:
- */
- if (!cpumask_test_cpu(this_cpu, busiest->curr->cpus_ptr)) {
- raw_spin_rq_unlock_irqrestore(busiest, flags);
- goto out_one_pinned;
- }
+ scoped_guard (raw_spin_rq_lock_irqsave, busiest) {
+ /*
+ * Don't kick the active_load_balance_cpu_stop,
+ * if the curr task on busiest CPU can't be
+ * moved to this_cpu:
+ */
+ if (!cpumask_test_cpu(this_cpu, busiest->curr->cpus_ptr))
+ goto out_one_pinned;
- /* Record that we found at least one task that could run on this_cpu */
- env.flags &= ~LBF_ALL_PINNED;
+ /* Record that we found at least one task that could run on this_cpu */
+ env.flags &= ~LBF_ALL_PINNED;
- /*
- * ->active_balance synchronizes accesses to
- * ->active_balance_work. Once set, it's cleared
- * only after active load balance is finished.
- */
- if (busiest->active_balance)
- goto no_active_balance;
+ /*
+ * ->active_balance synchronizes accesses to
+ * ->active_balance_work. Once set, it's cleared
+ * only after active load balance is finished.
+ */
+ if (busiest->active_balance)
+ goto out_unbalanced;
- /*
- * @busiest dropped its rq_lock in the middle of
- * scheduling out its ->curr task (->on_rq := 0), no
- * need to forcefully punt it away with active balance.
- */
- if (!busiest->curr->on_rq)
- goto no_active_balance;
-
- busiest->active_balance = 1;
- busiest->push_cpu = this_cpu;
- active_balance = 1;
-no_active_balance:
- preempt_disable();
- raw_spin_rq_unlock_irqrestore(busiest, flags);
- if (active_balance) {
- stop_one_cpu_nowait(cpu_of(busiest),
- active_load_balance_cpu_stop, busiest,
- &busiest->active_balance_work);
- }
- preempt_enable();
- }
- } else {
- sd->nr_balance_failed = 0;
+ /*
+ * @busiest dropped its rq_lock in the middle of
+ * scheduling out its ->curr task (->on_rq := 0), no
+ * need to forcefully punt it away with active balance.
+ */
+ if (!busiest->curr->on_rq)
+ goto out_unbalanced;
+
+ busiest->active_balance = 1;
+ busiest->push_cpu = this_cpu;
+ active_balance = 1;
+ preempt_disable();
+ }
+ if (active_balance) {
+ stop_one_cpu_nowait(cpu_of(busiest),
+ active_load_balance_cpu_stop, busiest,
+ &busiest->active_balance_work);
}
+ preempt_enable();
+out_unbalanced:
/* We were unbalanced, so reset the balancing interval */
sd->balance_interval = sd->min_interval;
-
goto out;
out_balanced:
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index e20823f..26ae13c 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -2021,7 +2021,8 @@ DEFINE_LOCK_GUARD_1(rq_lock, struct rq,
rq_unlock(_T->lock, &_T->rf),
struct rq_flags rf)
-DECLARE_LOCK_GUARD_1_ATTRS(rq_lock, __acquires(__rq_lockp(_T)), __releases(__rq_lockp(*(struct rq **)_T)));
+DECLARE_LOCK_GUARD_1_ATTRS(rq_lock, __acquires(__rq_lockp(_T)),
+ __releases(__rq_lockp(*(struct rq **)_T)));
#define class_rq_lock_constructor(_T) WITH_LOCK_GUARD_1_ATTRS(rq_lock, _T)
DEFINE_LOCK_GUARD_1(rq_lock_irq, struct rq,
@@ -2029,7 +2030,8 @@ DEFINE_LOCK_GUARD_1(rq_lock_irq, struct rq,
rq_unlock_irq(_T->lock, &_T->rf),
struct rq_flags rf)
-DECLARE_LOCK_GUARD_1_ATTRS(rq_lock_irq, __acquires(__rq_lockp(_T)), __releases(__rq_lockp(*(struct rq **)_T)));
+DECLARE_LOCK_GUARD_1_ATTRS(rq_lock_irq, __acquires(__rq_lockp(_T)),
+ __releases(__rq_lockp(*(struct rq **)_T)));
#define class_rq_lock_irq_constructor(_T) WITH_LOCK_GUARD_1_ATTRS(rq_lock_irq, _T)
DEFINE_LOCK_GUARD_1(rq_lock_irqsave, struct rq,
@@ -2037,9 +2039,20 @@ DEFINE_LOCK_GUARD_1(rq_lock_irqsave, struct rq,
rq_unlock_irqrestore(_T->lock, &_T->rf),
struct rq_flags rf)
-DECLARE_LOCK_GUARD_1_ATTRS(rq_lock_irqsave, __acquires(__rq_lockp(_T)), __releases(__rq_lockp(*(struct rq **)_T)));
+DECLARE_LOCK_GUARD_1_ATTRS(rq_lock_irqsave, __acquires(__rq_lockp(_T)),
+ __releases(__rq_lockp(*(struct rq **)_T)));
#define class_rq_lock_irqsave_constructor(_T) WITH_LOCK_GUARD_1_ATTRS(rq_lock_irqsave, _T)
+DEFINE_LOCK_GUARD_1(raw_spin_rq_lock_irqsave, struct rq,
+ raw_spin_rq_lock_irqsave(_T->lock, _T->flags),
+ raw_spin_rq_unlock_irqrestore(_T->lock, _T->flags),
+ unsigned long flags)
+
+DECLARE_LOCK_GUARD_1_ATTRS(raw_spin_rq_lock_irqsave, __acquires(__rq_lockp(_T)),
+ __releases(__rq_lockp(*(struct rq **)_T)));
+#define class_raw_spin_rq_lock_irqsave_constructor(_T) \
+ WITH_LOCK_GUARD_1_ATTRS(raw_spin_rq_lock_irqsave, _T)
+
#define this_rq_lock_irq(...) __acquire_ret(_this_rq_lock_irq(__VA_ARGS__), __rq_lockp(__ret))
static inline struct rq *_this_rq_lock_irq(struct rq_flags *rf) __acquires_ret
{
prev parent reply other threads:[~2026-06-30 9:03 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-17 7:21 [PATCH v5 0/2] sched/fair: Optimize some active balance logic Xin Zhao
2026-06-17 7:21 ` [PATCH v5 1/2] sched/fair: Don't trigger active lb if src_rq->curr is not on_rq Xin Zhao
2026-06-17 9:30 ` Valentin Schneider
2026-06-18 9:18 ` Peter Zijlstra
2026-06-18 10:09 ` Xin Zhao
2026-06-30 9:03 ` [tip: sched/core] " tip-bot2 for Xin Zhao
2026-06-17 7:21 ` [PATCH v5 2/2] sched/fair: Simplify balance_interval reset logic in sched_balance_rq() Xin Zhao
2026-06-18 9:40 ` Peter Zijlstra
2026-06-18 10:17 ` Xin Zhao
2026-06-18 10:31 ` Peter Zijlstra
2026-06-18 10:49 ` Xin Zhao
2026-06-30 9:03 ` [tip: sched/core] " tip-bot2 for Xin Zhao
2026-06-18 10:56 ` [PATCH v5 0/2] sched/fair: Optimize some active balance logic Peter Zijlstra
2026-06-18 13:56 ` Xin Zhao
2026-06-22 12:15 ` Aiqun(Maria) Yu
2026-06-30 9:03 ` tip-bot2 for Peter Zijlstra [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=178281021463.3843924.17697492618086729638.tip-bot2@tip-bot2 \
--to=tip-bot2@linutronix.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®