Re: 2.6.16 - sys_sched_getaffinity & hotplug

[Paul Jackson <pj@sgi.com>]

Nathan wrote:
> Which is problematic, because cpuset_cpus_allowed ->
> guarantee_online_cpus restricts the task->cpus_allowed mask to cpus
> which happen to be online at the time of the call to
> sched_setaffinity.  If more cpus come online later, that task can't be
> migrated to them.

Well, sort of.

A task could always migrate - just because a sched_getaffinity
the task did in the past doesn't show a CPU as valid, doesn't stop
the task from asking to pin to that CPU now.

One of three lessons could be taken from your example:
 1) return all possible CPUS (CPU_MASK_ALL, likely), as you recommend
 2) tell the task to not stash possibly stale returns from sched_getaffinity
 3) virtualize app CPU numbers relative to their containing cpuset,
      using an additional layer of user code.

I don't think we (or at least I ;) have an adequate understanding yet
of how hotplug will interact with the CPU affinity and Memory Node
mempolicy system calls, both of which are easier to use if things
don't come and go.  These calls are still, of course, usable, but
the possibilities for the task confusing itself with stale data
increase, and the simple system numbering of CPUs and Nodes by these
system calls makes (properly so) no effort to hide^Wvirtualize
these changes.

I tend to prefer lesson (3) above, but haven't yet delivered the
libraries or tools needed to support this as Open Source, so can't
really expect that preference to be very persuasive to others.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401