From: K Prateek Nayak <kprateek.nayak@amd.com>
To: <linux-kernel@vger.kernel.org>
Cc: <aubrey.li@linux.intel.com>, <efault@gmx.de>,
<gautham.shenoy@amd.com>, <libo.chen@oracle.com>,
<mgorman@techsingularity.net>, <mingo@kernel.org>,
<peterz@infradead.org>, <song.bao.hua@hisilicon.com>,
<srikar@linux.vnet.ibm.com>, <tglx@linutronix.de>,
<valentin.schneider@arm.com>, <vincent.guittot@linaro.org>,
<wuyun.abel@bytedance.com>, <wyes.karny@amd.com>,
<yu.c.chen@intel.com>, <yangyicong@huawei.com>
Subject: [PATCH 5/5] sched/fair: Add exception for hints in load balancing path
Date: Sat, 10 Sep 2022 16:23:26 +0530 [thread overview]
Message-ID: <20220910105326.1797-6-kprateek.nayak@amd.com> (raw)
In-Reply-To: <20220910105326.1797-1-kprateek.nayak@amd.com>
- Load balancing considerations
If we have more tasks than the CPUs in the MC Domain, ignore the hint
set by the user. This prevents losing the consolidation done at the
wakeup time.
- Considerations
Few trial and errors were done to find a good threshold to ignore hints.
Following are some of the wins and woes:
o Ignore hint if MC domain of src CPU does not have an idle core: This
metric is not very accurate and led to losing consolidation early on.
o Ignore hint if sd_shared->nr_llc_scan is 0: This too, like the
has_idle core metric was not always accurate.
o An atomic read of sd_shared->nr_busy_cpus doesn't encapsulate
overloaded run queues.
Best results were found by scanning LLC and finding the number of
running tasks and comparing it with size of LLC. If the LLC is beyond
fully loaded, safely ignore hint.
- Possible Improvements
o Consider the status of hint: If a wake affine hint was ignored in
the wakeup path, consider ignoring in the load balancer path as well
as the running LLC is not the desired LLC in fact.
Signed-off-by: K Prateek Nayak <kprateek.nayak@amd.com>
---
kernel/sched/fair.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 44 insertions(+)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 4c61bd0e93b3..8e1679b784fb 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7810,6 +7810,9 @@ struct lb_env {
unsigned int loop_break;
unsigned int loop_max;
+ /* Indicator to ignore hint if LLC is overloaded */
+ int ignore_hint;
+
enum fbq_type fbq_type;
enum migration_type migration_type;
struct list_head tasks;
@@ -7977,6 +7980,21 @@ int can_migrate_task(struct task_struct *p, struct lb_env *env)
return 0;
}
+ /*
+ * Hints are followed only if the MC Domain is still ideal
+ * for the task.
+ */
+ if (!env->ignore_hint) {
+ /*
+ * Only consider the hints from the wakeup path to maintain
+ * data locality.
+ */
+ if (READ_ONCE(p->hint) &
+ (PR_SCHED_HINT_WAKE_AFFINE | PR_SCHED_HINT_WAKE_HOLD))
+ return 0;
+ }
+
+
/* Record that we found at least one task that could run on dst_cpu */
env->flags &= ~LBF_ALL_PINNED;
@@ -10182,6 +10200,7 @@ static int load_balance(int this_cpu, struct rq *this_rq,
.cpus = cpus,
.fbq_type = all,
.tasks = LIST_HEAD_INIT(env.tasks),
+ .ignore_hint = 1,
};
cpumask_and(cpus, sched_domain_span(sd), cpu_active_mask);
@@ -10213,6 +10232,30 @@ static int load_balance(int this_cpu, struct rq *this_rq,
env.src_cpu = busiest->cpu;
env.src_rq = busiest;
+ /*
+ * Check if the hints can be followed during
+ * this load balancing cycle.
+ */
+ if (!(sd->flags & SD_SHARE_PKG_RESOURCES)) {
+ struct sched_domain *src_sd_llc = rcu_dereference(per_cpu(sd_llc, env.src_cpu));
+
+ if (src_sd_llc) {
+ int cpu, nr_llc_running = 0, llc_size = per_cpu(sd_llc_size, env.src_cpu);
+
+ for_each_cpu_wrap(cpu, sched_domain_span(src_sd_llc), env.src_cpu) {
+ struct rq *rq = cpu_rq(cpu);
+ nr_llc_running += rq->nr_running - rq->cfs.idle_h_nr_running;
+ }
+
+ /*
+ * Don't ignore hint if we can have one task
+ * per CPU in the LLC of the src_cpu.
+ */
+ if (nr_llc_running <= llc_size)
+ env.ignore_hint = 0;
+ }
+ }
+
ld_moved = 0;
/* Clear this flag as soon as we find a pullable task */
env.flags |= LBF_ALL_PINNED;
@@ -10520,6 +10563,7 @@ static int active_load_balance_cpu_stop(void *data)
.src_rq = busiest_rq,
.idle = CPU_IDLE,
.flags = LBF_ACTIVE_LB,
+ .ignore_hint = sd->flags & SD_SHARE_PKG_RESOURCES,
};
schedstat_inc(sd->alb_count);
--
2.25.1
next prev parent reply other threads:[~2022-09-10 10:56 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-10 10:53 [RFC PATCH 0/5] sched: Userspace Hinting for Task Placement K Prateek Nayak
2022-09-10 10:53 ` [PATCH 1/5] task_struct: Add field for per-task userspace scheduler hint K Prateek Nayak
2022-09-10 10:53 ` [PATCH 2/5] prctl: Add interface and helper functions to set hints K Prateek Nayak
2022-09-10 10:53 ` [PATCH 3/5] sched/fair: Add support for hints in the subsequent wakeup path K Prateek Nayak
2022-09-10 10:53 ` [PATCH 4/5] sched/fair: Consider hints in the initial task " K Prateek Nayak
2022-09-10 10:53 ` K Prateek Nayak [this message]
[not found] ` <20220911080547.2709-1-hdanton@sina.com>
2022-09-12 6:29 ` [PATCH 5/5] sched/fair: Add exception for hints in load balancing path K Prateek Nayak
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220910105326.1797-6-kprateek.nayak@amd.com \
--to=kprateek.nayak@amd.com \
--cc=aubrey.li@linux.intel.com \
--cc=efault@gmx.de \
--cc=gautham.shenoy@amd.com \
--cc=libo.chen@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@techsingularity.net \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=song.bao.hua@hisilicon.com \
--cc=srikar@linux.vnet.ibm.com \
--cc=tglx@linutronix.de \
--cc=valentin.schneider@arm.com \
--cc=vincent.guittot@linaro.org \
--cc=wuyun.abel@bytedance.com \
--cc=wyes.karny@amd.com \
--cc=yangyicong@huawei.com \
--cc=yu.c.chen@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®