[patch -mm] oom: fix constraint deadlock

[patch -mm] oom: fix constraint deadlock

[David Rientjes]

Fixes a deadlock in the OOM killer for allocations that are not
__GFP_HARDWALL.

Before the OOM killer checks for the allocation constraint, it takes
callback_mutex.

constrained_alloc() iterates through each zone in the allocation zonelist
and calls cpuset_zone_allowed_softwall() to determine whether an
allocation for gfp_mask is possible.  If a zone's node is not in the
OOM-triggering task's mems_allowed, it is not exiting, and we did not
fail on a __GFP_HARDWALL allocation, cpuset_zone_allowed_softwall()
attempts to take callback_mutex to check the nearest exclusive ancestor
of current's cpuset.  This results in deadlock.

We now take callback_mutex after iterating through the zonelist since we
don't need it yet.

Cc: Andi Kleen <ak@suse.de>
Cc: Nick Piggin <npiggin@suse.de>
Cc: Christoph Lameter <clameter@engr.sgi.com>
Cc: Martin J. Bligh <mbligh@mbligh.org>
Signed-off-by: David Rientjes <rientjes@google.com>
---
 mm/oom_kill.c |   10 ++++++----
 1 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/mm/oom_kill.c b/mm/oom_kill.c
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -397,6 +397,7 @@ void out_of_memory(struct zonelist *zonelist, gfp_t gfp_mask, int order)
 	struct task_struct *p;
 	unsigned long points = 0;
 	unsigned long freed = 0;
+	int constraint;
 
 	blocking_notifier_call_chain(&oom_notify_list, 0, &freed);
 	if (freed > 0)
@@ -411,14 +412,15 @@ void out_of_memory(struct zonelist *zonelist, gfp_t gfp_mask, int order)
 		show_mem();
 	}
 
-	cpuset_lock();
-	read_lock(&tasklist_lock);
-
 	/*
 	 * Check if there were limitations on the allocation (only relevant for
 	 * NUMA) that may require different handling.
 	 */
-	switch (constrained_alloc(zonelist, gfp_mask)) {
+	constraint = constrained_alloc(zonelist, gfp_mask);
+	cpuset_lock();
+	read_lock(&tasklist_lock);
+
+	switch (constraint) {
 	case CONSTRAINT_MEMORY_POLICY:
 		oom_kill_process(current, points,
 				"No available memory (MPOL_BIND)");

Re: [patch -mm] oom: fix constraint deadlock

[Andrew Morton]

On Thu, 26 Apr 2007 23:18:17 -0700 (PDT) David Rientjes <rientjes@google.com> wrote:

> Subject: [patch -mm] oom: fix constraint deadlock
> Date: Thu, 26 Apr 2007 23:18:17 -0700 (PDT)
> 
> Fixes a deadlock in the OOM killer for allocations that are not
> __GFP_HARDWALL.

Thanks.

Why is this described as "patch -mm"?  It applies to mainline.  If it's
fixing a -mm patch, do you know which one?

Re: [patch -mm] oom: fix constraint deadlock

[David Rientjes]

On Fri, 27 Apr 2007, Andrew Morton wrote:

> > Subject: [patch -mm] oom: fix constraint deadlock
> > Date: Thu, 26 Apr 2007 23:18:17 -0700 (PDT)
> > 
> > Fixes a deadlock in the OOM killer for allocations that are not
> > __GFP_HARDWALL.
> 
> Thanks.
> 
> Why is this described as "patch -mm"?  It applies to mainline.  If it's
> fixing a -mm patch, do you know which one?
> 

Fair enough.  It definitely is applicable to mainline and fixes a deadlock 
which we've been struggling with for awhile because not only does it block 
while holding callback_mutex, but also tasklist_lock.

Thanks.

Re: [patch -mm] oom: fix constraint deadlock

[Andrew Morton]

On Fri, 27 Apr 2007 00:19:43 -0700 (PDT) David Rientjes <rientjes@google.com> wrote:

> On Fri, 27 Apr 2007, Andrew Morton wrote:
> 
> > > Subject: [patch -mm] oom: fix constraint deadlock
> > > Date: Thu, 26 Apr 2007 23:18:17 -0700 (PDT)
> > > 
> > > Fixes a deadlock in the OOM killer for allocations that are not
> > > __GFP_HARDWALL.
> > 
> > Thanks.
> > 
> > Why is this described as "patch -mm"?  It applies to mainline.  If it's
> > fixing a -mm patch, do you know which one?
> > 
> 
> Fair enough.  It definitely is applicable to mainline and fixes a deadlock 
> which we've been struggling with for awhile because not only does it block 
> while holding callback_mutex, but also tasklist_lock.

OK, thanks.  I tagged it for a 2.6.20.x backport as well.