Re: How can I optimize a process on a NUMA architecture(x86-64 specifically)?

[Jesse Barnes <jbarnes@engr.sgi.com>]

On Friday, May 21, 2004 2:14 pm, Brett E. wrote:
> So could process 0 run on processor 0, allocating local to processor 0,
> then run on processor 1, allocating local to processor 1, this way
> allocating to both processors?

Yep.

> So over time process 0's allocations 
> would be split up between both processors, defeating NUMA.

Well, I wouldn't put it that way, but it would give your app suboptimal 
performance.

> The homenode 
> concept + explicit CPU pinning seems useful in that they allow you to
> take advantage of NUMA better.

When you say homenode here, do you mean restricted memory allocation?  If so, 
then yes, that would be useful.  And we have it already with libnuma and 
sched_setaffinity.  If you mean some sort of preferred allocation node, then 
no, we don't have that, and it wouldn't be of much benefit I think (relative 
to what we already have).  The idea is to do "pretty good" by default, but 
still allow the user full control if they want to be sure.

> Without these two things the kernel will 
> just allocate on the currently running CPU whatever that may be when in
> fact a preference must be given to a CPU at some point, hopefully early
> on in the life of the process, in order to take advantage of NUMA.

The kernel does a pretty good job of keeping processes on the CPU they've been 
running on, and thus they'll probably stay close to their memory.  But 
without explicit pinning, there's no guarantee.

Jesse