* [PATCH v4 0/3] sched: Get rid of select_task_rq()'s sd_flag parameter
@ 2020-11-02 18:45 Valentin Schneider
2020-11-02 18:45 ` [PATCH v4 1/3] sched: Add WF_TTWU, WF_EXEC wakeup flags Valentin Schneider
` (3 more replies)
0 siblings, 4 replies; 8+ messages in thread
From: Valentin Schneider @ 2020-11-02 18:45 UTC (permalink / raw)
To: linux-kernel
Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Steven Rostedt, Daniel Bristot de Oliveira
Hi folks,
This is v4 of that one series I've had stashed for a while [1]. I haven't
revisited the outstanding discussion bits, the most interesting part being that
we could get rid of the for_each_domain() loop in select_task_rq_fair() (or at
the very least simplify it). In any case, the first few patches are IMO a decent
cleanup on their own, hence me resubmitting them.
Links
=====
[1]: https://lore.kernel.org/lkml/20200415210512.805-1-valentin.schneider@arm.com/
Revisions
=========
v3 -> v4
--------
o Dropped want_affine related patches
o Made WF_{TTWU, FORK, EXEC} and SD_BALANCE_{WAKE, FORK, EXEC} share a nibble
(Peter)
v2 -> v3
--------
o Rebased on top of v5.7-rc1 (didn't re-run performance tests)
o Collected Reviewed-by (Dietmar)
o Updated changelog of 3/9 (Dietmar)
v1 -> v2
--------
o Removed the 'RFC' tag
o Made the sd_flags syctl read-only
o Removed the SD_LOAD_BALANCE flag
o Cleaned up ugly changes thanks to the above
Valentin Schneider (3):
sched: Add WF_TTWU, WF_EXEC wakeup flags
sched: Remove select_task_rq()'s sd_flag parameter
sched/fair: Dissociate wakeup decisions from SD flag value
kernel/sched/core.c | 10 +++++-----
kernel/sched/deadline.c | 4 ++--
kernel/sched/fair.c | 13 +++++++------
kernel/sched/idle.c | 2 +-
kernel/sched/rt.c | 4 ++--
kernel/sched/sched.h | 23 +++++++++++++++--------
kernel/sched/stop_task.c | 2 +-
7 files changed, 33 insertions(+), 25 deletions(-)
--
2.27.0
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH v4 1/3] sched: Add WF_TTWU, WF_EXEC wakeup flags 2020-11-02 18:45 [PATCH v4 0/3] sched: Get rid of select_task_rq()'s sd_flag parameter Valentin Schneider @ 2020-11-02 18:45 ` Valentin Schneider 2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Valentin Schneider 2020-11-02 18:45 ` [PATCH v4 2/3] sched: Remove select_task_rq()'s sd_flag parameter Valentin Schneider ` (2 subsequent siblings) 3 siblings, 1 reply; 8+ messages in thread From: Valentin Schneider @ 2020-11-02 18:45 UTC (permalink / raw) To: linux-kernel Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Daniel Bristot de Oliveira To remove the sd_flag parameter of select_task_rq(), we need another way of encoding wakeup types. There already is a WF_FORK flag, add the missing two. With that said, we still need an easy way to turn WF_foo into SD_bar (e.g. WF_TTWU into SD_BALANCE_WAKE). As suggested by Peter, let's make our lives easier and make them match exactly, and throw in some compile-time checks for good measure. Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> --- kernel/sched/sched.h | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index df80bfcea92e..65666c52f347 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1714,13 +1714,20 @@ static inline int task_on_rq_migrating(struct task_struct *p) return READ_ONCE(p->on_rq) == TASK_ON_RQ_MIGRATING; } -/* - * wake flags - */ -#define WF_SYNC 0x01 /* Waker goes to sleep after wakeup */ -#define WF_FORK 0x02 /* Child wakeup after fork */ -#define WF_MIGRATED 0x04 /* Internal use, task got migrated */ -#define WF_ON_CPU 0x08 /* Wakee is on_cpu */ +/* Wake flags. The first three directly map to some SD flag value */ +#define WF_EXEC 0x02 /* Wakeup after exec; maps to SD_BALANCE_EXEC */ +#define WF_FORK 0x04 /* Wakeup after fork; maps to SD_BALANCE_FORK */ +#define WF_TTWU 0x08 /* Wakeup; maps to SD_BALANCE_WAKE */ + +#define WF_SYNC 0x10 /* Waker goes to sleep after wakeup */ +#define WF_MIGRATED 0x20 /* Internal use, task got migrated */ +#define WF_ON_CPU 0x40 /* Wakee is on_cpu */ + +#ifdef CONFIG_SMP +static_assert(WF_EXEC == SD_BALANCE_EXEC); +static_assert(WF_FORK == SD_BALANCE_FORK); +static_assert(WF_TTWU == SD_BALANCE_WAKE); +#endif /* * To aid in avoiding the subversion of "niceness" due to uneven distribution -- 2.27.0 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [tip: sched/core] sched: Add WF_TTWU, WF_EXEC wakeup flags 2020-11-02 18:45 ` [PATCH v4 1/3] sched: Add WF_TTWU, WF_EXEC wakeup flags Valentin Schneider @ 2020-11-11 8:23 ` tip-bot2 for Valentin Schneider 0 siblings, 0 replies; 8+ messages in thread From: tip-bot2 for Valentin Schneider @ 2020-11-11 8:23 UTC (permalink / raw) To: linux-tip-commits Cc: Valentin Schneider, Peter Zijlstra (Intel), x86, linux-kernel The following commit has been merged into the sched/core branch of tip: Commit-ID: 17770579059258c5f1eef759e941af5f1a54f482 Gitweb: https://git.kernel.org/tip/17770579059258c5f1eef759e941af5f1a54f482 Author: Valentin Schneider <valentin.schneider@arm.com> AuthorDate: Mon, 02 Nov 2020 18:45:12 Committer: Peter Zijlstra <peterz@infradead.org> CommitterDate: Tue, 10 Nov 2020 18:39:06 +01:00 sched: Add WF_TTWU, WF_EXEC wakeup flags To remove the sd_flag parameter of select_task_rq(), we need another way of encoding wakeup types. There already is a WF_FORK flag, add the missing two. With that said, we still need an easy way to turn WF_foo into SD_bar (e.g. WF_TTWU into SD_BALANCE_WAKE). As suggested by Peter, let's make our lives easier and make them match exactly, and throw in some compile-time checks for good measure. Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/20201102184514.2733-2-valentin.schneider@arm.com --- kernel/sched/sched.h | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index e897d77..4725873 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1744,13 +1744,20 @@ static inline int task_on_rq_migrating(struct task_struct *p) return READ_ONCE(p->on_rq) == TASK_ON_RQ_MIGRATING; } -/* - * wake flags - */ -#define WF_SYNC 0x01 /* Waker goes to sleep after wakeup */ -#define WF_FORK 0x02 /* Child wakeup after fork */ -#define WF_MIGRATED 0x04 /* Internal use, task got migrated */ -#define WF_ON_CPU 0x08 /* Wakee is on_cpu */ +/* Wake flags. The first three directly map to some SD flag value */ +#define WF_EXEC 0x02 /* Wakeup after exec; maps to SD_BALANCE_EXEC */ +#define WF_FORK 0x04 /* Wakeup after fork; maps to SD_BALANCE_FORK */ +#define WF_TTWU 0x08 /* Wakeup; maps to SD_BALANCE_WAKE */ + +#define WF_SYNC 0x10 /* Waker goes to sleep after wakeup */ +#define WF_MIGRATED 0x20 /* Internal use, task got migrated */ +#define WF_ON_CPU 0x40 /* Wakee is on_cpu */ + +#ifdef CONFIG_SMP +static_assert(WF_EXEC == SD_BALANCE_EXEC); +static_assert(WF_FORK == SD_BALANCE_FORK); +static_assert(WF_TTWU == SD_BALANCE_WAKE); +#endif /* * To aid in avoiding the subversion of "niceness" due to uneven distribution ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v4 2/3] sched: Remove select_task_rq()'s sd_flag parameter 2020-11-02 18:45 [PATCH v4 0/3] sched: Get rid of select_task_rq()'s sd_flag parameter Valentin Schneider 2020-11-02 18:45 ` [PATCH v4 1/3] sched: Add WF_TTWU, WF_EXEC wakeup flags Valentin Schneider @ 2020-11-02 18:45 ` Valentin Schneider 2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Valentin Schneider 2020-11-02 18:45 ` [PATCH v4 3/3] sched/fair: Dissociate wakeup decisions from SD flag value Valentin Schneider 2020-11-10 12:52 ` [PATCH v4 0/3] sched: Get rid of select_task_rq()'s sd_flag parameter Peter Zijlstra 3 siblings, 1 reply; 8+ messages in thread From: Valentin Schneider @ 2020-11-02 18:45 UTC (permalink / raw) To: linux-kernel Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Daniel Bristot de Oliveira Only select_task_rq_fair() uses that parameter to do an actual domain search, other classes only care about what kind of wakeup is happening (fork, exec, or "regular") and thus just translate the flag into a wakeup type. WF_TTWU and WF_EXEC have just been added, use these along with WF_FORK to encode the wakeup types we care about. For select_task_rq_fair(), we can simply use the shiny new WF_flag : SD_flag mapping. Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> --- kernel/sched/core.c | 10 +++++----- kernel/sched/deadline.c | 4 ++-- kernel/sched/fair.c | 8 +++++--- kernel/sched/idle.c | 2 +- kernel/sched/rt.c | 4 ++-- kernel/sched/sched.h | 2 +- kernel/sched/stop_task.c | 2 +- 7 files changed, 17 insertions(+), 15 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index d2003a7d5ab5..d31eace9b5e2 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2348,12 +2348,12 @@ static int select_fallback_rq(int cpu, struct task_struct *p) * The caller (fork, wakeup) owns p->pi_lock, ->cpus_ptr is stable. */ static inline -int select_task_rq(struct task_struct *p, int cpu, int sd_flags, int wake_flags) +int select_task_rq(struct task_struct *p, int cpu, int wake_flags) { lockdep_assert_held(&p->pi_lock); if (p->nr_cpus_allowed > 1) - cpu = p->sched_class->select_task_rq(p, cpu, sd_flags, wake_flags); + cpu = p->sched_class->select_task_rq(p, cpu, wake_flags); else cpu = cpumask_any(p->cpus_ptr); @@ -2961,7 +2961,7 @@ try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags) */ smp_cond_load_acquire(&p->on_cpu, !VAL); - cpu = select_task_rq(p, p->wake_cpu, SD_BALANCE_WAKE, wake_flags); + cpu = select_task_rq(p, p->wake_cpu, wake_flags | WF_TTWU); if (task_cpu(p) != cpu) { wake_flags |= WF_MIGRATED; psi_ttwu_dequeue(p); @@ -3344,7 +3344,7 @@ void wake_up_new_task(struct task_struct *p) */ p->recent_used_cpu = task_cpu(p); rseq_migrate(p); - __set_task_cpu(p, select_task_rq(p, task_cpu(p), SD_BALANCE_FORK, 0)); + __set_task_cpu(p, select_task_rq(p, task_cpu(p), WF_FORK)); #endif rq = __task_rq_lock(p, &rf); update_rq_clock(rq); @@ -3888,7 +3888,7 @@ void sched_exec(void) int dest_cpu; raw_spin_lock_irqsave(&p->pi_lock, flags); - dest_cpu = p->sched_class->select_task_rq(p, task_cpu(p), SD_BALANCE_EXEC, 0); + dest_cpu = p->sched_class->select_task_rq(p, task_cpu(p), WF_EXEC); if (dest_cpu == smp_processor_id()) goto unlock; diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index f232305dcefe..a6933c7f9ba7 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1664,13 +1664,13 @@ static void yield_task_dl(struct rq *rq) static int find_later_rq(struct task_struct *task); static int -select_task_rq_dl(struct task_struct *p, int cpu, int sd_flag, int flags) +select_task_rq_dl(struct task_struct *p, int cpu, int flags) { struct task_struct *curr; bool select_rq; struct rq *rq; - if (sd_flag != SD_BALANCE_WAKE) + if (!(flags & WF_TTWU)) goto out; rq = cpu_rq(cpu); diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 290f9e38378c..c02f7af0a925 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6663,7 +6663,7 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu) /* * select_task_rq_fair: Select target runqueue for the waking task in domains - * that have the 'sd_flag' flag set. In practice, this is SD_BALANCE_WAKE, + * that have the relevant SD flag set. In practice, this is SD_BALANCE_WAKE, * SD_BALANCE_FORK, or SD_BALANCE_EXEC. * * Balances load by selecting the idlest CPU in the idlest group, or under @@ -6674,13 +6674,15 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu) * preempt must be disabled. */ static int -select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_flags) +select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags) { + int sync = (wake_flags & WF_SYNC) && !(current->flags & PF_EXITING); struct sched_domain *tmp, *sd = NULL; int cpu = smp_processor_id(); int new_cpu = prev_cpu; int want_affine = 0; - int sync = (wake_flags & WF_SYNC) && !(current->flags & PF_EXITING); + /* SD_flags and WF_flags share the first nibble */ + int sd_flag = wake_flags & 0xF; if (sd_flag & SD_BALANCE_WAKE) { record_wakee(p); diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 24d0ee26377d..56d12d9bbe47 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -375,7 +375,7 @@ void cpu_startup_entry(enum cpuhp_state state) #ifdef CONFIG_SMP static int -select_task_rq_idle(struct task_struct *p, int cpu, int sd_flag, int flags) +select_task_rq_idle(struct task_struct *p, int cpu, int flags) { return task_cpu(p); /* IDLE tasks as never migrated */ } diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 49ec096a8aa1..c3ca1ab9e011 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1428,14 +1428,14 @@ static void yield_task_rt(struct rq *rq) static int find_lowest_rq(struct task_struct *task); static int -select_task_rq_rt(struct task_struct *p, int cpu, int sd_flag, int flags) +select_task_rq_rt(struct task_struct *p, int cpu, int flags) { struct task_struct *curr; struct rq *rq; bool test; /* For anything but wake ups, just return the task_cpu */ - if (sd_flag != SD_BALANCE_WAKE && sd_flag != SD_BALANCE_FORK) + if (!(flags & (WF_TTWU | WF_FORK))) goto out; rq = cpu_rq(cpu); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 65666c52f347..f6d12c50d454 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1803,7 +1803,7 @@ struct sched_class { #ifdef CONFIG_SMP int (*balance)(struct rq *rq, struct task_struct *prev, struct rq_flags *rf); - int (*select_task_rq)(struct task_struct *p, int task_cpu, int sd_flag, int flags); + int (*select_task_rq)(struct task_struct *p, int task_cpu, int flags); void (*migrate_task_rq)(struct task_struct *p, int new_cpu); void (*task_woken)(struct rq *this_rq, struct task_struct *task); diff --git a/kernel/sched/stop_task.c b/kernel/sched/stop_task.c index ceb5b6b12561..c2ddcfee0483 100644 --- a/kernel/sched/stop_task.c +++ b/kernel/sched/stop_task.c @@ -11,7 +11,7 @@ #ifdef CONFIG_SMP static int -select_task_rq_stop(struct task_struct *p, int cpu, int sd_flag, int flags) +select_task_rq_stop(struct task_struct *p, int cpu, int flags) { return task_cpu(p); /* stop tasks as never migrate */ } -- 2.27.0 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [tip: sched/core] sched: Remove select_task_rq()'s sd_flag parameter 2020-11-02 18:45 ` [PATCH v4 2/3] sched: Remove select_task_rq()'s sd_flag parameter Valentin Schneider @ 2020-11-11 8:23 ` tip-bot2 for Valentin Schneider 0 siblings, 0 replies; 8+ messages in thread From: tip-bot2 for Valentin Schneider @ 2020-11-11 8:23 UTC (permalink / raw) To: linux-tip-commits Cc: Valentin Schneider, Peter Zijlstra (Intel), x86, linux-kernel The following commit has been merged into the sched/core branch of tip: Commit-ID: 3aef1551e942860a3881087171ef0cd45f6ebda7 Gitweb: https://git.kernel.org/tip/3aef1551e942860a3881087171ef0cd45f6ebda7 Author: Valentin Schneider <valentin.schneider@arm.com> AuthorDate: Mon, 02 Nov 2020 18:45:13 Committer: Peter Zijlstra <peterz@infradead.org> CommitterDate: Tue, 10 Nov 2020 18:39:06 +01:00 sched: Remove select_task_rq()'s sd_flag parameter Only select_task_rq_fair() uses that parameter to do an actual domain search, other classes only care about what kind of wakeup is happening (fork, exec, or "regular") and thus just translate the flag into a wakeup type. WF_TTWU and WF_EXEC have just been added, use these along with WF_FORK to encode the wakeup types we care about. For select_task_rq_fair(), we can simply use the shiny new WF_flag : SD_flag mapping. Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/20201102184514.2733-3-valentin.schneider@arm.com --- kernel/sched/core.c | 10 +++++----- kernel/sched/deadline.c | 4 ++-- kernel/sched/fair.c | 8 +++++--- kernel/sched/idle.c | 2 +- kernel/sched/rt.c | 4 ++-- kernel/sched/sched.h | 2 +- kernel/sched/stop_task.c | 2 +- 7 files changed, 17 insertions(+), 15 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 622f343..a6aaf9f 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2769,12 +2769,12 @@ out: * The caller (fork, wakeup) owns p->pi_lock, ->cpus_ptr is stable. */ static inline -int select_task_rq(struct task_struct *p, int cpu, int sd_flags, int wake_flags) +int select_task_rq(struct task_struct *p, int cpu, int wake_flags) { lockdep_assert_held(&p->pi_lock); if (p->nr_cpus_allowed > 1 && !is_migration_disabled(p)) - cpu = p->sched_class->select_task_rq(p, cpu, sd_flags, wake_flags); + cpu = p->sched_class->select_task_rq(p, cpu, wake_flags); else cpu = cpumask_any(p->cpus_ptr); @@ -3409,7 +3409,7 @@ try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags) */ smp_cond_load_acquire(&p->on_cpu, !VAL); - cpu = select_task_rq(p, p->wake_cpu, SD_BALANCE_WAKE, wake_flags); + cpu = select_task_rq(p, p->wake_cpu, wake_flags | WF_TTWU); if (task_cpu(p) != cpu) { wake_flags |= WF_MIGRATED; psi_ttwu_dequeue(p); @@ -3793,7 +3793,7 @@ void wake_up_new_task(struct task_struct *p) */ p->recent_used_cpu = task_cpu(p); rseq_migrate(p); - __set_task_cpu(p, select_task_rq(p, task_cpu(p), SD_BALANCE_FORK, 0)); + __set_task_cpu(p, select_task_rq(p, task_cpu(p), WF_FORK)); #endif rq = __task_rq_lock(p, &rf); update_rq_clock(rq); @@ -4384,7 +4384,7 @@ void sched_exec(void) int dest_cpu; raw_spin_lock_irqsave(&p->pi_lock, flags); - dest_cpu = p->sched_class->select_task_rq(p, task_cpu(p), SD_BALANCE_EXEC, 0); + dest_cpu = p->sched_class->select_task_rq(p, task_cpu(p), WF_EXEC); if (dest_cpu == smp_processor_id()) goto unlock; diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index cc1feb7..2a5836f 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1683,13 +1683,13 @@ static void yield_task_dl(struct rq *rq) static int find_later_rq(struct task_struct *task); static int -select_task_rq_dl(struct task_struct *p, int cpu, int sd_flag, int flags) +select_task_rq_dl(struct task_struct *p, int cpu, int flags) { struct task_struct *curr; bool select_rq; struct rq *rq; - if (sd_flag != SD_BALANCE_WAKE) + if (!(flags & WF_TTWU)) goto out; rq = cpu_rq(cpu); diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 3e5d98f..b1596fa 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6684,7 +6684,7 @@ fail: /* * select_task_rq_fair: Select target runqueue for the waking task in domains - * that have the 'sd_flag' flag set. In practice, this is SD_BALANCE_WAKE, + * that have the relevant SD flag set. In practice, this is SD_BALANCE_WAKE, * SD_BALANCE_FORK, or SD_BALANCE_EXEC. * * Balances load by selecting the idlest CPU in the idlest group, or under @@ -6695,13 +6695,15 @@ fail: * preempt must be disabled. */ static int -select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_flags) +select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags) { + int sync = (wake_flags & WF_SYNC) && !(current->flags & PF_EXITING); struct sched_domain *tmp, *sd = NULL; int cpu = smp_processor_id(); int new_cpu = prev_cpu; int want_affine = 0; - int sync = (wake_flags & WF_SYNC) && !(current->flags & PF_EXITING); + /* SD_flags and WF_flags share the first nibble */ + int sd_flag = wake_flags & 0xF; if (sd_flag & SD_BALANCE_WAKE) { record_wakee(p); diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 9da69c4..df91b19 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -376,7 +376,7 @@ void cpu_startup_entry(enum cpuhp_state state) #ifdef CONFIG_SMP static int -select_task_rq_idle(struct task_struct *p, int cpu, int sd_flag, int flags) +select_task_rq_idle(struct task_struct *p, int cpu, int flags) { return task_cpu(p); /* IDLE tasks as never migrated */ } diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index c961a3f..dbe4629 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1430,14 +1430,14 @@ static void yield_task_rt(struct rq *rq) static int find_lowest_rq(struct task_struct *task); static int -select_task_rq_rt(struct task_struct *p, int cpu, int sd_flag, int flags) +select_task_rq_rt(struct task_struct *p, int cpu, int flags) { struct task_struct *curr; struct rq *rq; bool test; /* For anything but wake ups, just return the task_cpu */ - if (sd_flag != SD_BALANCE_WAKE && sd_flag != SD_BALANCE_FORK) + if (!(flags & (WF_TTWU | WF_FORK))) goto out; rq = cpu_rq(cpu); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 4725873..590e6f2 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1833,7 +1833,7 @@ struct sched_class { #ifdef CONFIG_SMP int (*balance)(struct rq *rq, struct task_struct *prev, struct rq_flags *rf); - int (*select_task_rq)(struct task_struct *p, int task_cpu, int sd_flag, int flags); + int (*select_task_rq)(struct task_struct *p, int task_cpu, int flags); void (*migrate_task_rq)(struct task_struct *p, int new_cpu); void (*task_woken)(struct rq *this_rq, struct task_struct *task); diff --git a/kernel/sched/stop_task.c b/kernel/sched/stop_task.c index 91bb10c..55f3912 100644 --- a/kernel/sched/stop_task.c +++ b/kernel/sched/stop_task.c @@ -11,7 +11,7 @@ #ifdef CONFIG_SMP static int -select_task_rq_stop(struct task_struct *p, int cpu, int sd_flag, int flags) +select_task_rq_stop(struct task_struct *p, int cpu, int flags) { return task_cpu(p); /* stop tasks as never migrate */ } ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v4 3/3] sched/fair: Dissociate wakeup decisions from SD flag value 2020-11-02 18:45 [PATCH v4 0/3] sched: Get rid of select_task_rq()'s sd_flag parameter Valentin Schneider 2020-11-02 18:45 ` [PATCH v4 1/3] sched: Add WF_TTWU, WF_EXEC wakeup flags Valentin Schneider 2020-11-02 18:45 ` [PATCH v4 2/3] sched: Remove select_task_rq()'s sd_flag parameter Valentin Schneider @ 2020-11-02 18:45 ` Valentin Schneider 2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Valentin Schneider 2020-11-10 12:52 ` [PATCH v4 0/3] sched: Get rid of select_task_rq()'s sd_flag parameter Peter Zijlstra 3 siblings, 1 reply; 8+ messages in thread From: Valentin Schneider @ 2020-11-02 18:45 UTC (permalink / raw) To: linux-kernel Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Daniel Bristot de Oliveira The CFS wakeup code will only ever go through EAS / its fast path on "regular" wakeups (i.e. not on forks or execs). These are currently gated by a check against 'sd_flag', which would be SD_BALANCE_WAKE at wakeup. However, we now have a flag that explicitly tells us whether a wakeup is a "regular" one, so hinge those conditions on that flag instead. Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> --- kernel/sched/fair.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index c02f7af0a925..4f9ff5d85ab2 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6684,7 +6684,7 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags) /* SD_flags and WF_flags share the first nibble */ int sd_flag = wake_flags & 0xF; - if (sd_flag & SD_BALANCE_WAKE) { + if (wake_flags & WF_TTWU) { record_wakee(p); if (sched_energy_enabled()) { @@ -6721,9 +6721,8 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags) if (unlikely(sd)) { /* Slow path */ new_cpu = find_idlest_cpu(sd, p, cpu, prev_cpu, sd_flag); - } else if (sd_flag & SD_BALANCE_WAKE) { /* XXX always ? */ + } else if (wake_flags & WF_TTWU) { /* XXX always ? */ /* Fast path */ - new_cpu = select_idle_sibling(p, prev_cpu, new_cpu); if (want_affine) -- 2.27.0 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [tip: sched/core] sched/fair: Dissociate wakeup decisions from SD flag value 2020-11-02 18:45 ` [PATCH v4 3/3] sched/fair: Dissociate wakeup decisions from SD flag value Valentin Schneider @ 2020-11-11 8:23 ` tip-bot2 for Valentin Schneider 0 siblings, 0 replies; 8+ messages in thread From: tip-bot2 for Valentin Schneider @ 2020-11-11 8:23 UTC (permalink / raw) To: linux-tip-commits Cc: Valentin Schneider, Peter Zijlstra (Intel), x86, linux-kernel The following commit has been merged into the sched/core branch of tip: Commit-ID: dc824eb898534cd8e34582874dae3bb7cf2fa008 Gitweb: https://git.kernel.org/tip/dc824eb898534cd8e34582874dae3bb7cf2fa008 Author: Valentin Schneider <valentin.schneider@arm.com> AuthorDate: Mon, 02 Nov 2020 18:45:14 Committer: Peter Zijlstra <peterz@infradead.org> CommitterDate: Tue, 10 Nov 2020 18:39:06 +01:00 sched/fair: Dissociate wakeup decisions from SD flag value The CFS wakeup code will only ever go through EAS / its fast path on "regular" wakeups (i.e. not on forks or execs). These are currently gated by a check against 'sd_flag', which would be SD_BALANCE_WAKE at wakeup. However, we now have a flag that explicitly tells us whether a wakeup is a "regular" one, so hinge those conditions on that flag instead. Signed-off-by: Valentin Schneider <valentin.schneider@arm.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/20201102184514.2733-4-valentin.schneider@arm.com --- kernel/sched/fair.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b1596fa..6691e28 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6705,7 +6705,7 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags) /* SD_flags and WF_flags share the first nibble */ int sd_flag = wake_flags & 0xF; - if (sd_flag & SD_BALANCE_WAKE) { + if (wake_flags & WF_TTWU) { record_wakee(p); if (sched_energy_enabled()) { @@ -6742,9 +6742,8 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags) if (unlikely(sd)) { /* Slow path */ new_cpu = find_idlest_cpu(sd, p, cpu, prev_cpu, sd_flag); - } else if (sd_flag & SD_BALANCE_WAKE) { /* XXX always ? */ + } else if (wake_flags & WF_TTWU) { /* XXX always ? */ /* Fast path */ - new_cpu = select_idle_sibling(p, prev_cpu, new_cpu); if (want_affine) ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v4 0/3] sched: Get rid of select_task_rq()'s sd_flag parameter 2020-11-02 18:45 [PATCH v4 0/3] sched: Get rid of select_task_rq()'s sd_flag parameter Valentin Schneider ` (2 preceding siblings ...) 2020-11-02 18:45 ` [PATCH v4 3/3] sched/fair: Dissociate wakeup decisions from SD flag value Valentin Schneider @ 2020-11-10 12:52 ` Peter Zijlstra 3 siblings, 0 replies; 8+ messages in thread From: Peter Zijlstra @ 2020-11-10 12:52 UTC (permalink / raw) To: Valentin Schneider Cc: linux-kernel, Ingo Molnar, Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt, Daniel Bristot de Oliveira On Mon, Nov 02, 2020 at 06:45:11PM +0000, Valentin Schneider wrote: > Hi folks, > > This is v4 of that one series I've had stashed for a while [1]. I haven't > revisited the outstanding discussion bits, the most interesting part being that > we could get rid of the for_each_domain() loop in select_task_rq_fair() (or at > the very least simplify it). In any case, the first few patches are IMO a decent > cleanup on their own, hence me resubmitting them. Thanks! ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2020-11-11 8:25 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-11-02 18:45 [PATCH v4 0/3] sched: Get rid of select_task_rq()'s sd_flag parameter Valentin Schneider 2020-11-02 18:45 ` [PATCH v4 1/3] sched: Add WF_TTWU, WF_EXEC wakeup flags Valentin Schneider 2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Valentin Schneider 2020-11-02 18:45 ` [PATCH v4 2/3] sched: Remove select_task_rq()'s sd_flag parameter Valentin Schneider 2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Valentin Schneider 2020-11-02 18:45 ` [PATCH v4 3/3] sched/fair: Dissociate wakeup decisions from SD flag value Valentin Schneider 2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Valentin Schneider 2020-11-10 12:52 ` [PATCH v4 0/3] sched: Get rid of select_task_rq()'s sd_flag parameter Peter Zijlstra
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®