Re: [PATCH 3/4] 2.6.2-rc2-mm2 CPU Hotplug: The Core

[Rusty Russell <rusty@rustcorp.com.au>]

In message <Pine.LNX.4.58.0402020741250.16748@devserv.devel.redhat.com> you wri
te:
> 
> On Mon, 2 Feb 2004, Rusty Russell wrote:
> 
> > Unfortunately the __migrate_task() check won't go away: someone may have
> > asked to move from CPU 0 to 1, and by the time migration thread on 0
> > gets to the request, 1 has gone down.  We don't want all the callers to
> > hold the cpucontrol lock, because now the NUMA scheduler uses migration
> > as a common case 8(
> 
> well, when a CPU goes down it could process the migration request queue as
> well. (this would be a pretty natural thing to do if CPU-down executes in
> the migration-thread context.)

Wrong migration thread.  The migration thread on CPU 1 has been asked
to push into CPU 0, which is now going down.

Now I've slept on the "do it atomically" idea, I think it's a good
one.  I've even worked out how to maintain the "last thread on CPU is
the idle thread", although I'd need to test in code.

> Another question is user-space semantics - if user-space relies on CPU
> affinity, is the kernel allowed to violate it or should the process be
> notified. Sending it a signal (SIGTERM or anything similar) if the
> affinity was non-generic might be a good thing to do.

We've been there: we used to send SIGPWR with a new si_info field to
say what CPU it was.  But it turns out not to be useful, so we took it
out.

In practice, any app which wants to scale with # CPUs needs to know
when CPUs are coming up, as well as going down.  Ditto memory, etc.
This means they need to listen for the hotplug event (DBUS anyone?),
or we introduce a SIGRECONF (default ignored).  But AFAICT,
introducing a new signal isn't possible (at least on x86) without
breaking glibc.

Thanks!
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.