Re: [PATCH] cpuset semaphore depth check deadlock fix

[Nikita Danilov <nikita@clusterfs.com>]

Paul Jackson writes:
 > The cpusets-formalize-intermediate-gfp_kernel-containment patch
 > has a deadlock problem.

[...]

 >  
 >  /*
 > + * The global cpuset semaphore cpuset_sem can be needed by the
 > + * memory allocator to update a tasks mems_allowed (see the calls
 > + * to cpuset_update_current_mems_allowed()) or to walk up the
 > + * cpuset hierarchy to find a mem_exclusive cpuset see the calls
 > + * to cpuset_excl_nodes_overlap()).
 > + *
 > + * But if the memory allocation is being done by cpuset.c code, it
 > + * usually already holds cpuset_sem.  Double tripping on a kernel
 > + * semaphore deadlocks the current task, and any other task that
 > + * subsequently tries to obtain the lock.
 > + *
 > + * Run all up's and down's on cpuset_sem through the following
 > + * wrappers, which will detect this nested locking, and avoid
 > + * deadlocking.
 > + */
 > +
 > +static inline void cs_down(struct semaphore *psem)
 > +{
 > +	if (current->cpuset_sem_nest_depth == 0)
 > +		down(psem);
 > +	current->cpuset_sem_nest_depth++;
 > +}
 > +
 > +static inline void cs_up(struct semaphore *psem)
 > +{
 > +	current->cpuset_sem_nest_depth--;
 > +	if (current->cpuset_sem_nest_depth == 0)
 > +		up(psem);
 > +}

I am somewhat concerned that new fields are added to the struct
task_struct all the time: it's already over 1.3KB.

In that particular case, it seems that cs_{up,down}() (or however they
end up being named), are used only on &cpuset_sem, and adding new field
to the thread struct can be avoided by doing:

static DECLARE_MUTEX(cpuset_sem);
static struct task_struct *cpuset_sem_owner = NULL;
static int cpuset_sem_depth = 0;

static void cpusets_lock(void)
{
	if (cpuset_sem_owner != current) {
		down(&cpuset_sem);
		cpuset_sem_owner = current;
	}
	cpuset_sem_depth ++;
}

static void cpusets_unlock(void)
{
	if (-- cpuset_sem_depth == 0) {
		cpuset_sem_owner = NULL;
		up(&cpuset_sem);
	}
}

Nikita.