Re: [RFC][PATCH] CGroups: Add a per-subsystem hierarchy lock

["Paul Menage" <menage@google.com>]

On Wed, Jul 2, 2008 at 9:46 PM, Max Krasnyansky <maxk@qualcomm.com> wrote:
>
> mkdir /dev/cpuset
> mount -t cgroup -o cpuset cpuset /dev/cpuset
> mkdir /dev/cpuset/0
> mkdir /dev/cpuset/1
> echo 0-2 > /dev/cpuset/0/cpuset.cpus
> echo 3   > /dev/cpuset/1/cpuset.cpus
> echo 0 > /dev/cpuset/cpuset.sched_load_balance
> echo 0 > /sys/devices/system/cpu/cpu3/online
>

OK, I still can't reproduce this, on a 2-cpu system using one cpu for
each cpuset.

But the basic problem seemns to be that we have cpu_hotplug.lock taken
at the outer level (when offlining a CPU) and at the inner level (via
get_online_cpus() called from the guts of partition_sched_domains(),
if we didn't already take it at the outer level.

While looking at the code trying to figure out a nice way around this,
it struck me that we have the call path

cpuset_track_online_nodes() ->
  common_cpu_mem_hotplug_unplug() ->
    scan_for_empty_cpusets() ->
      access cpu_online_map with no calls to get_online_cpus()

Is that allowed? Maybe we need separate versions of
scan_for_empty_cpusets() that look at memory and cpus?

I think that we're going to want eventually a solution such pushing
the locking of cpuset_subsys.hierarchy_mutex down into the first part
of partition_sched_domains, that actually walks the cpuset tree

Paul