* [PATCH 1/5] sched/numa: Rename numabalancing_enabled to sched_numa_balancing
2016-01-07 17:57 [PATCH 4.3-stable 0/5] Disable automatic numa balancing on UMA Mel Gorman
@ 2016-01-07 17:57 ` Mel Gorman
2016-01-07 17:57 ` [PATCH 2/5] sched/numa: Disable sched_numa_balancing on UMA systems Mel Gorman
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Mel Gorman @ 2016-01-07 17:57 UTC (permalink / raw)
To: Linux-Stable
Cc: Srikar Dronamraju, Peter Zijlstra, Ingo Molnar, LKML, Mel Gorman
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
commit 78a9c54649ea220065aad9902460a1d137c7eafd upstream.
Simple rename of the 'numabalancing_enabled' variable to 'sched_numa_balancing'.
No functional changes.
Suggested-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1439290813-6683-2-git-send-email-srikar@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
kernel/sched/core.c | 6 +++---
kernel/sched/fair.c | 4 ++--
kernel/sched/sched.h | 6 +++---
3 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index bcd214e4b4d6..98594051d6d5 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2124,11 +2124,11 @@ void set_numabalancing_state(bool enabled)
sched_feat_set("NO_NUMA");
}
#else
-__read_mostly bool numabalancing_enabled;
+__read_mostly bool sched_numa_balancing;
void set_numabalancing_state(bool enabled)
{
- numabalancing_enabled = enabled;
+ sched_numa_balancing = enabled;
}
#endif /* CONFIG_SCHED_DEBUG */
@@ -2138,7 +2138,7 @@ int sysctl_numa_balancing(struct ctl_table *table, int write,
{
struct ctl_table t;
int err;
- int state = numabalancing_enabled;
+ int state = sched_numa_balancing;
if (write && !capable(CAP_SYS_ADMIN))
return -EPERM;
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 9a5e60fe721a..33d02d6a3cf4 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -2069,7 +2069,7 @@ void task_numa_fault(int last_cpupid, int mem_node, int pages, int flags)
int local = !!(flags & TNF_FAULT_LOCAL);
int priv;
- if (!numabalancing_enabled)
+ if (!sched_numa_balancing)
return;
/* for example, ksmd faulting in a user's mm */
@@ -7811,7 +7811,7 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued)
entity_tick(cfs_rq, se, queued);
}
- if (numabalancing_enabled)
+ if (sched_numa_balancing)
task_tick_numa(rq, curr);
}
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 6d2a119c7ad9..252cfbb78348 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1006,13 +1006,13 @@ extern struct static_key sched_feat_keys[__SCHED_FEAT_NR];
#ifdef CONFIG_NUMA_BALANCING
#define sched_feat_numa(x) sched_feat(x)
#ifdef CONFIG_SCHED_DEBUG
-#define numabalancing_enabled sched_feat_numa(NUMA)
+#define sched_numa_balancing sched_feat_numa(NUMA)
#else
-extern bool numabalancing_enabled;
+extern bool sched_numa_balancing;
#endif /* CONFIG_SCHED_DEBUG */
#else
#define sched_feat_numa(x) (0)
-#define numabalancing_enabled (0)
+#define sched_numa_balancing (0)
#endif /* CONFIG_NUMA_BALANCING */
static inline u64 global_rt_period(void)
--
2.6.4
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 2/5] sched/numa: Disable sched_numa_balancing on UMA systems
2016-01-07 17:57 [PATCH 4.3-stable 0/5] Disable automatic numa balancing on UMA Mel Gorman
2016-01-07 17:57 ` [PATCH 1/5] sched/numa: Rename numabalancing_enabled to sched_numa_balancing Mel Gorman
@ 2016-01-07 17:57 ` Mel Gorman
2016-01-07 17:57 ` [PATCH 3/5] sched/numa: Remove the NUMA sched_feature Mel Gorman
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Mel Gorman @ 2016-01-07 17:57 UTC (permalink / raw)
To: Linux-Stable
Cc: Srikar Dronamraju, Peter Zijlstra, Ingo Molnar, LKML, Mel Gorman
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
commit c3b9bc5bbfc3750570d788afffd431263ef695c6 upstream.
Commit 2a1ed24 ("sched/numa: Prefer NUMA hotness over cache hotness")
sets sched feature NUMA to true. However this can enable NUMA hinting
faults on a UMA system.
This commit ensures that NUMA hinting faults occur only on a NUMA system
by setting/resetting sched_numa_balancing.
This commit:
- Makes sched_numa_balancing common to CONFIG_SCHED_DEBUG and
!CONFIG_SCHED_DEBUG. Earlier it was only in !CONFIG_SCHED_DEBUG.
- Checks for sched_numa_balancing instead of sched_feat(NUMA).
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1439290813-6683-3-git-send-email-srikar@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
kernel/sched/core.c | 14 +++++---------
kernel/sched/fair.c | 4 ++--
kernel/sched/sched.h | 6 ------
3 files changed, 7 insertions(+), 17 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 98594051d6d5..6cd6ce1fe161 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2115,22 +2115,18 @@ static void __sched_fork(unsigned long clone_flags, struct task_struct *p)
}
#ifdef CONFIG_NUMA_BALANCING
-#ifdef CONFIG_SCHED_DEBUG
+__read_mostly bool sched_numa_balancing;
+
void set_numabalancing_state(bool enabled)
{
+ sched_numa_balancing = enabled;
+#ifdef CONFIG_SCHED_DEBUG
if (enabled)
sched_feat_set("NUMA");
else
sched_feat_set("NO_NUMA");
-}
-#else
-__read_mostly bool sched_numa_balancing;
-
-void set_numabalancing_state(bool enabled)
-{
- sched_numa_balancing = enabled;
-}
#endif /* CONFIG_SCHED_DEBUG */
+}
#ifdef CONFIG_PROC_SYSCTL
int sysctl_numa_balancing(struct ctl_table *table, int write,
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 33d02d6a3cf4..0d5987e8e329 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -5525,10 +5525,10 @@ static int migrate_degrades_locality(struct task_struct *p, struct lb_env *env)
unsigned long src_faults, dst_faults;
int src_nid, dst_nid;
- if (!p->numa_faults || !(env->sd->flags & SD_NUMA))
+ if (!sched_numa_balancing)
return -1;
- if (!sched_feat(NUMA))
+ if (!p->numa_faults || !(env->sd->flags & SD_NUMA))
return -1;
src_nid = cpu_to_node(env->src_cpu);
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 252cfbb78348..fb75db386adc 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1004,14 +1004,8 @@ extern struct static_key sched_feat_keys[__SCHED_FEAT_NR];
#endif /* SCHED_DEBUG && HAVE_JUMP_LABEL */
#ifdef CONFIG_NUMA_BALANCING
-#define sched_feat_numa(x) sched_feat(x)
-#ifdef CONFIG_SCHED_DEBUG
-#define sched_numa_balancing sched_feat_numa(NUMA)
-#else
extern bool sched_numa_balancing;
-#endif /* CONFIG_SCHED_DEBUG */
#else
-#define sched_feat_numa(x) (0)
#define sched_numa_balancing (0)
#endif /* CONFIG_NUMA_BALANCING */
--
2.6.4
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 3/5] sched/numa: Remove the NUMA sched_feature
2016-01-07 17:57 [PATCH 4.3-stable 0/5] Disable automatic numa balancing on UMA Mel Gorman
2016-01-07 17:57 ` [PATCH 1/5] sched/numa: Rename numabalancing_enabled to sched_numa_balancing Mel Gorman
2016-01-07 17:57 ` [PATCH 2/5] sched/numa: Disable sched_numa_balancing on UMA systems Mel Gorman
@ 2016-01-07 17:57 ` Mel Gorman
2016-01-07 17:57 ` [PATCH 4/5] sched/numa: Convert sched_numa_balancing to a static_branch Mel Gorman
2016-01-07 17:57 ` [PATCH 5/5] sched/numa: Fix task_tick_fair() from disabling numa_balancing Mel Gorman
4 siblings, 0 replies; 6+ messages in thread
From: Mel Gorman @ 2016-01-07 17:57 UTC (permalink / raw)
To: Linux-Stable
Cc: Srikar Dronamraju, Peter Zijlstra, Ingo Molnar, LKML, Mel Gorman
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
commit 2b49d84b259fc18e131026e5d38e7855352f71b9 upstream.
Variable sched_numa_balancing is available for both CONFIG_SCHED_DEBUG
and !CONFIG_SCHED_DEBUG. All code paths now check for
sched_numa_balancing. Hence remove sched_feat(NUMA).
Suggested-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1439290813-6683-4-git-send-email-srikar@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
kernel/sched/features.h | 16 ----------------
1 file changed, 16 deletions(-)
diff --git a/kernel/sched/features.h b/kernel/sched/features.h
index 83a50e7ca533..8baa708aa0d2 100644
--- a/kernel/sched/features.h
+++ b/kernel/sched/features.h
@@ -72,19 +72,3 @@ SCHED_FEAT(RT_PUSH_IPI, true)
SCHED_FEAT(FORCE_SD_OVERLAP, false)
SCHED_FEAT(RT_RUNTIME_SHARE, true)
SCHED_FEAT(LB_MIN, false)
-
-/*
- * Apply the automatic NUMA scheduling policy. Enabled automatically
- * at runtime if running on a NUMA machine. Can be controlled via
- * numa_balancing=
- */
-#ifdef CONFIG_NUMA_BALANCING
-
-/*
- * NUMA will favor moving tasks towards nodes where a higher number of
- * hinting faults are recorded during active load balancing. It will
- * resist moving tasks towards nodes where a lower number of hinting
- * faults have been recorded.
- */
-SCHED_FEAT(NUMA, true)
-#endif
--
2.6.4
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 4/5] sched/numa: Convert sched_numa_balancing to a static_branch
2016-01-07 17:57 [PATCH 4.3-stable 0/5] Disable automatic numa balancing on UMA Mel Gorman
` (2 preceding siblings ...)
2016-01-07 17:57 ` [PATCH 3/5] sched/numa: Remove the NUMA sched_feature Mel Gorman
@ 2016-01-07 17:57 ` Mel Gorman
2016-01-07 17:57 ` [PATCH 5/5] sched/numa: Fix task_tick_fair() from disabling numa_balancing Mel Gorman
4 siblings, 0 replies; 6+ messages in thread
From: Mel Gorman @ 2016-01-07 17:57 UTC (permalink / raw)
To: Linux-Stable
Cc: Srikar Dronamraju, Peter Zijlstra, Ingo Molnar, LKML, Mel Gorman
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
commit 2a595721a1fa6b684c1c818f379bef834ac3d65e upstream.
Variable sched_numa_balancing toggles numa_balancing feature. Hence
moving from a simple read mostly variable to a more apt static_branch.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1439310261-16124-1-git-send-email-srikar@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
kernel/sched/core.c | 12 +++++-------
kernel/sched/fair.c | 6 +++---
kernel/sched/sched.h | 6 +-----
3 files changed, 9 insertions(+), 15 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 6cd6ce1fe161..5998e12e14b4 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2114,18 +2114,16 @@ static void __sched_fork(unsigned long clone_flags, struct task_struct *p)
#endif /* CONFIG_NUMA_BALANCING */
}
+DEFINE_STATIC_KEY_FALSE(sched_numa_balancing);
+
#ifdef CONFIG_NUMA_BALANCING
-__read_mostly bool sched_numa_balancing;
void set_numabalancing_state(bool enabled)
{
- sched_numa_balancing = enabled;
-#ifdef CONFIG_SCHED_DEBUG
if (enabled)
- sched_feat_set("NUMA");
+ static_branch_enable(&sched_numa_balancing);
else
- sched_feat_set("NO_NUMA");
-#endif /* CONFIG_SCHED_DEBUG */
+ static_branch_disable(&sched_numa_balancing);
}
#ifdef CONFIG_PROC_SYSCTL
@@ -2134,7 +2132,7 @@ int sysctl_numa_balancing(struct ctl_table *table, int write,
{
struct ctl_table t;
int err;
- int state = sched_numa_balancing;
+ int state = static_branch_likely(&sched_numa_balancing);
if (write && !capable(CAP_SYS_ADMIN))
return -EPERM;
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 0d5987e8e329..7975932034f9 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -2069,7 +2069,7 @@ void task_numa_fault(int last_cpupid, int mem_node, int pages, int flags)
int local = !!(flags & TNF_FAULT_LOCAL);
int priv;
- if (!sched_numa_balancing)
+ if (!static_branch_likely(&sched_numa_balancing))
return;
/* for example, ksmd faulting in a user's mm */
@@ -5525,7 +5525,7 @@ static int migrate_degrades_locality(struct task_struct *p, struct lb_env *env)
unsigned long src_faults, dst_faults;
int src_nid, dst_nid;
- if (!sched_numa_balancing)
+ if (!static_branch_likely(&sched_numa_balancing))
return -1;
if (!p->numa_faults || !(env->sd->flags & SD_NUMA))
@@ -7811,7 +7811,7 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued)
entity_tick(cfs_rq, se, queued);
}
- if (sched_numa_balancing)
+ if (!static_branch_unlikely(&sched_numa_balancing))
task_tick_numa(rq, curr);
}
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index fb75db386adc..01875872cc94 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1003,11 +1003,7 @@ extern struct static_key sched_feat_keys[__SCHED_FEAT_NR];
#define sched_feat(x) (sysctl_sched_features & (1UL << __SCHED_FEAT_##x))
#endif /* SCHED_DEBUG && HAVE_JUMP_LABEL */
-#ifdef CONFIG_NUMA_BALANCING
-extern bool sched_numa_balancing;
-#else
-#define sched_numa_balancing (0)
-#endif /* CONFIG_NUMA_BALANCING */
+extern struct static_key_false sched_numa_balancing;
static inline u64 global_rt_period(void)
{
--
2.6.4
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH 5/5] sched/numa: Fix task_tick_fair() from disabling numa_balancing
2016-01-07 17:57 [PATCH 4.3-stable 0/5] Disable automatic numa balancing on UMA Mel Gorman
` (3 preceding siblings ...)
2016-01-07 17:57 ` [PATCH 4/5] sched/numa: Convert sched_numa_balancing to a static_branch Mel Gorman
@ 2016-01-07 17:57 ` Mel Gorman
4 siblings, 0 replies; 6+ messages in thread
From: Mel Gorman @ 2016-01-07 17:57 UTC (permalink / raw)
To: Linux-Stable
Cc: Srikar Dronamraju, Peter Zijlstra, Ingo Molnar, LKML, Mel Gorman
From: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
commit b52da86e0ad58f096710977fcda856fd84da9233 upstream.
If static branch 'sched_numa_balancing' is enabled, it should kickstart
NUMA balancing through task_tick_numa(). However the following commit:
2a595721a1fa ("sched/numa: Convert sched_numa_balancing to a static_branch")
erroneously disables this.
Fix this anomaly by enabling task_tick_numa() when the static branch
'sched_numa_balancing' is enabled.
Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Link: http://lkml.kernel.org/r/1443752305-27413-1-git-send-email-srikar@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
kernel/sched/fair.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 7975932034f9..98b6eacf09aa 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7811,7 +7811,7 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued)
entity_tick(cfs_rq, se, queued);
}
- if (!static_branch_unlikely(&sched_numa_balancing))
+ if (static_branch_unlikely(&sched_numa_balancing))
task_tick_numa(rq, curr);
}
--
2.6.4
^ permalink raw reply [flat|nested] 6+ messages in thread