Re: [PATCH] Run PCI driver initialization on local node

[Christoph Lameter <christoph@lameter.com>]

On Wed, 6 Jul 2005, Andi Kleen wrote:

> On Wed, Jul 06, 2005 at 09:35:32AM -0700, Christoph Lameter wrote:
> > On Wed, 6 Jul 2005, Andi Kleen wrote:
> > 
> > > Instead of adding messy kmalloc_node()s everywhere run the 
> > > PCI driver probe on the node local to the device.
> > > Then the normal NUMA aware allocators do the right thing.
> > 
> > That depends on the architecture. Some do round robin allocs for periods 
> > of time during bootup. I think it is better to explicitly place control 
> 
> slab will usually do the right thing because it has a forced
> local node policy, but __gfp might not.

GFP allocs may not do the right thing. If you want to do this then it 
may be best to set the memory policy to restrict allocations to the node 
on which the device resides.

Plus there are CPU less nodes. What happens to those?

> Patching every driver in existence? That sounds like a lot of
> work. 

No just patch those that would benefit from it. The existing 
dma_alloc_coherent already takes care of many of the placement 
issues for driver memory. We would probably need to patch more locations 
where higher level control structure allocations are being done.

> The node local placement should be correct for nearly all drivers. I didn't 
> see any other fancy placement in your patches. If a driver still wants to do 
> fancy placement it is free to overwrite the policy. Having a good
> default is good.

Definitely.