Re: [RFC/PATCH] cpuset: cpuset irq affinities

[Paul Jackson <pj@sgi.com>]

> But as long as nobody does CS_CPU_EXCLUSIVE they may overlap, right?

It's a bit stronger than that:

 1) They need non-overlapping cpusets at this level to control
    the sched_domain setup, if they want to avoid load balancing
    across almost all CPUs in the system.  Depending on the kernel
    version, sched_domain partitioning is controlled either by the
    cpuset flag cpu_exclusive, or the cpuset flag sched_load_balance.

 2) They need non-overlapping cpusets at this level to control
    memory placement of some kernel allocations, which are allowed
    outside the current tasks cpuset, to be confined by the nearest
    ancestor cpuset marked 'mem_exclusive'

 3) Some sysadmin tools are likely coded to expect a /dev/cpuset/boot
    cpuset, not a /dev/cpuset/system/boot cpuset, as that has been
    customary for a long time.

(1) and (2) would break the major batch schedulers.  They typically
mark their top cpuset, /dev/cpuset/pbs or /dev/cpuset/lfs or whatever
batch scheduler it is, as cpu_exclusive and mem_exclusive, by way of
expressing their intention to pretty much own those CPUs and memory
nodes.  If we fired them up on a system where that wasn't allowed due
to overlap with /dev/cpuset/system, they'd croak.  Such changes as that
are costly and unappreciated.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.940.382.4214