* [PATCH v2] sched/fair: sanitize vruntime of entity being placed
@ 2023-01-30 12:22 Roman Kagan
2023-02-01 16:06 ` Peter Zijlstra
2023-02-11 10:30 ` [tip: sched/core] " tip-bot2 for Zhang Qiao
0 siblings, 2 replies; 3+ messages in thread
From: Roman Kagan @ 2023-01-30 12:22 UTC (permalink / raw)
To: linux-kernel
Cc: Juri Lelli, Zhang Qiao, Vincent Guittot, Waiman Long, Ben Segall,
Peter Zijlstra, Steven Rostedt, Mel Gorman, Valentin Schneider,
Dietmar Eggemann, Daniel Bristot de Oliveira, Ingo Molnar
From: Zhang Qiao <zhangqiao22@huawei.com>
When a scheduling entity is placed onto cfs_rq, its vruntime is pulled
to the base level (around cfs_rq->min_vruntime), so that the entity
doesn't gain extra boost when placed backwards.
However, if the entity being placed wasn't executed for a long time, its
vruntime may get too far behind (e.g. while cfs_rq was executing a
low-weight hog), which can inverse the vruntime comparison due to s64
overflow. This results in the entity being placed with its original
vruntime way forwards, so that it will effectively never get to the cpu.
To prevent that, ignore the vruntime of the entity being placed if it
didn't execute for much longer than the characteristic sheduler time
scale.
Signed-off-by: Zhang Qiao <zhangqiao22@huawei.com>
[rkagan: formatted, adjusted commit log, comments, cutoff value]
Co-developed-by: Roman Kagan <rkagan@amazon.de>
Signed-off-by: Roman Kagan <rkagan@amazon.de>
---
v1 -> v2:
- add Zhang Qiao's s-o-b
- fix constant promotion on 32bit
kernel/sched/fair.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 0f8736991427..717c3ca970e1 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -4656,6 +4656,7 @@ static void
place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial)
{
u64 vruntime = cfs_rq->min_vruntime;
+ u64 sleep_time;
/*
* The 'current' period is already promised to the current tasks,
@@ -4685,8 +4686,18 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial)
vruntime -= thresh;
}
- /* ensure we never gain time by being placed backwards. */
- se->vruntime = max_vruntime(se->vruntime, vruntime);
+ /*
+ * Pull vruntime of the entity being placed to the base level of
+ * cfs_rq, to prevent boosting it if placed backwards. If the entity
+ * slept for a long time, don't even try to compare its vruntime with
+ * the base as it may be too far off and the comparison may get
+ * inversed due to s64 overflow.
+ */
+ sleep_time = rq_clock_task(rq_of(cfs_rq)) - se->exec_start;
+ if ((s64)sleep_time > 60LL * NSEC_PER_SEC)
+ se->vruntime = vruntime;
+ else
+ se->vruntime = max_vruntime(se->vruntime, vruntime);
}
static void check_enqueue_throttle(struct cfs_rq *cfs_rq);
--
2.34.1
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH v2] sched/fair: sanitize vruntime of entity being placed
2023-01-30 12:22 [PATCH v2] sched/fair: sanitize vruntime of entity being placed Roman Kagan
@ 2023-02-01 16:06 ` Peter Zijlstra
2023-02-11 10:30 ` [tip: sched/core] " tip-bot2 for Zhang Qiao
1 sibling, 0 replies; 3+ messages in thread
From: Peter Zijlstra @ 2023-02-01 16:06 UTC (permalink / raw)
To: Roman Kagan
Cc: linux-kernel, Juri Lelli, Zhang Qiao, Vincent Guittot,
Waiman Long, Ben Segall, Steven Rostedt, Mel Gorman,
Valentin Schneider, Dietmar Eggemann, Daniel Bristot de Oliveira,
Ingo Molnar
On Mon, Jan 30, 2023 at 01:22:16PM +0100, Roman Kagan wrote:
> From: Zhang Qiao <zhangqiao22@huawei.com>
>
> When a scheduling entity is placed onto cfs_rq, its vruntime is pulled
> to the base level (around cfs_rq->min_vruntime), so that the entity
> doesn't gain extra boost when placed backwards.
>
> However, if the entity being placed wasn't executed for a long time, its
> vruntime may get too far behind (e.g. while cfs_rq was executing a
> low-weight hog), which can inverse the vruntime comparison due to s64
> overflow. This results in the entity being placed with its original
> vruntime way forwards, so that it will effectively never get to the cpu.
>
> To prevent that, ignore the vruntime of the entity being placed if it
> didn't execute for much longer than the characteristic sheduler time
> scale.
>
> Signed-off-by: Zhang Qiao <zhangqiao22@huawei.com>
> [rkagan: formatted, adjusted commit log, comments, cutoff value]
> Co-developed-by: Roman Kagan <rkagan@amazon.de>
> Signed-off-by: Roman Kagan <rkagan@amazon.de>
Thanks!
^ permalink raw reply [flat|nested] 3+ messages in thread
* [tip: sched/core] sched/fair: sanitize vruntime of entity being placed
2023-01-30 12:22 [PATCH v2] sched/fair: sanitize vruntime of entity being placed Roman Kagan
2023-02-01 16:06 ` Peter Zijlstra
@ 2023-02-11 10:30 ` tip-bot2 for Zhang Qiao
1 sibling, 0 replies; 3+ messages in thread
From: tip-bot2 for Zhang Qiao @ 2023-02-11 10:30 UTC (permalink / raw)
To: linux-tip-commits
Cc: Zhang Qiao, Roman Kagan, Peter Zijlstra (Intel), x86, linux-kernel
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 829c1651e9c4a6f78398d3e67651cef9bb6b42cc
Gitweb: https://git.kernel.org/tip/829c1651e9c4a6f78398d3e67651cef9bb6b42cc
Author: Zhang Qiao <zhangqiao22@huawei.com>
AuthorDate: Mon, 30 Jan 2023 13:22:16 +01:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Sat, 11 Feb 2023 11:18:09 +01:00
sched/fair: sanitize vruntime of entity being placed
When a scheduling entity is placed onto cfs_rq, its vruntime is pulled
to the base level (around cfs_rq->min_vruntime), so that the entity
doesn't gain extra boost when placed backwards.
However, if the entity being placed wasn't executed for a long time, its
vruntime may get too far behind (e.g. while cfs_rq was executing a
low-weight hog), which can inverse the vruntime comparison due to s64
overflow. This results in the entity being placed with its original
vruntime way forwards, so that it will effectively never get to the cpu.
To prevent that, ignore the vruntime of the entity being placed if it
didn't execute for much longer than the characteristic sheduler time
scale.
[rkagan: formatted, adjusted commit log, comments, cutoff value]
Signed-off-by: Zhang Qiao <zhangqiao22@huawei.com>
Co-developed-by: Roman Kagan <rkagan@amazon.de>
Signed-off-by: Roman Kagan <rkagan@amazon.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20230130122216.3555094-1-rkagan@amazon.de
---
kernel/sched/fair.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index c6c8e7f..ff4dbba 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -4652,6 +4652,7 @@ static void
place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial)
{
u64 vruntime = cfs_rq->min_vruntime;
+ u64 sleep_time;
/*
* The 'current' period is already promised to the current tasks,
@@ -4681,8 +4682,18 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial)
vruntime -= thresh;
}
- /* ensure we never gain time by being placed backwards. */
- se->vruntime = max_vruntime(se->vruntime, vruntime);
+ /*
+ * Pull vruntime of the entity being placed to the base level of
+ * cfs_rq, to prevent boosting it if placed backwards. If the entity
+ * slept for a long time, don't even try to compare its vruntime with
+ * the base as it may be too far off and the comparison may get
+ * inversed due to s64 overflow.
+ */
+ sleep_time = rq_clock_task(rq_of(cfs_rq)) - se->exec_start;
+ if ((s64)sleep_time > 60LL * NSEC_PER_SEC)
+ se->vruntime = vruntime;
+ else
+ se->vruntime = max_vruntime(se->vruntime, vruntime);
}
static void check_enqueue_throttle(struct cfs_rq *cfs_rq);
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2023-02-11 10:31 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-01-30 12:22 [PATCH v2] sched/fair: sanitize vruntime of entity being placed Roman Kagan
2023-02-01 16:06 ` Peter Zijlstra
2023-02-11 10:30 ` [tip: sched/core] " tip-bot2 for Zhang Qiao
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®