3.1-rc2: CONFIG_NUMA Question

3.1-rc2: CONFIG_NUMA Question

[Justin Piszcz]

Hi,

I have a SuperMicro X8DTH-6F motherboard with 2 cpus it has NUMA options 
in the BIOS; what parameters/should I be using for the NUMA options in the 
kernel?

   | |    [*] Numa Memory Allocation and Scheduler Support                 | |
   | |    [ ] Old style AMD Opteron NUMA detection                         | |
   | |    [*] ACPI NUMA detection                                          | |
   | |    [ ] NUMA emulation                                               | |
   | |    (6) Maximum NUMA Nodes (as a power of 2)                         | |

Specifically, the maximum number of NUMA nodes?  Will this help to 
increase performance of large memory allocations/is there any downside
to enabling this feature?

Justin.

Re: 3.1-rc2: CONFIG_NUMA Question

[Américo Wang]

On Thu, Aug 18, 2011 at 6:51 PM, Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
> Hi,
>
> I have a SuperMicro X8DTH-6F motherboard with 2 cpus it has NUMA options in
> the BIOS; what parameters/should I be using for the NUMA options in the
> kernel?
>
>  | |    [*] Numa Memory Allocation and Scheduler Support                 | |
>  | |    [ ] Old style AMD Opteron NUMA detection                         | |
>  | |    [*] ACPI NUMA detection                                          | |
>  | |    [ ] NUMA emulation                                               | |
>  | |    (6) Maximum NUMA Nodes (as a power of 2)                         | |
>

That is Intel processor, right? If so, the things you selected here are enough,
the rest two is not what you need.

> Specifically, the maximum number of NUMA nodes?  Will this help to increase
> performance of large memory allocations/is there any downside
> to enabling this feature?

The max of NUMA nodes is specified by CONFIG_NODES_SHIFT,
in your case, it is 6, which means it supports up to 2^6 nodes.

If your hardware supports NUMA, yes, that certainly help to increase
the performance.

Thanks.

Re: 3.1-rc2: CONFIG_NUMA Question

[Justin Piszcz]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1470 bytes --]


On Thu, 18 Aug 2011, Américo Wang wrote:

> On Thu, Aug 18, 2011 at 6:51 PM, Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
>> Hi,
>>
>> I have a SuperMicro X8DTH-6F motherboard with 2 cpus it has NUMA options in
>> the BIOS; what parameters/should I be using for the NUMA options in the
>> kernel?
>>
>>  | |    [*] Numa Memory Allocation and Scheduler Support                 | |
>>  | |    [ ] Old style AMD Opteron NUMA detection                         | |
>>  | |    [*] ACPI NUMA detection                                          | |
>>  | |    [ ] NUMA emulation                                               | |
>>  | |    (6) Maximum NUMA Nodes (as a power of 2)                         | |
>>
>
> That is Intel processor, right? If so, the things you selected here are enough,
Hi, yes-- two Intel Xeon E5645s.

> the rest two is not what you need.
Ok.

>
>> Specifically, the maximum number of NUMA nodes?  Will this help to increase
>> performance of large memory allocations/is there any downside
>> to enabling this feature?
>
> The max of NUMA nodes is specified by CONFIG_NODES_SHIFT,
> in your case, it is 6, which means it supports up to 2^6 nodes.
Ok, is there any overhead having more than is needed?

>
> If your hardware supports NUMA, yes, that certainly help to increase
> the performance.
Nice, thanks.

Justin.

Re: 3.1-rc2: CONFIG_NUMA Question

[David Rientjes]

On Thu, 18 Aug 2011, Justin Piszcz wrote:

> > The max of NUMA nodes is specified by CONFIG_NODES_SHIFT,
> > in your case, it is 6, which means it supports up to 2^6 nodes.
> Ok, is there any overhead having more than is needed?
> 

Yeah, try to avoid going above 8 because that causes nodemasks that are 
typically allocated on the stack of kernel text to be dynamically 
allocated through the slab allocator to avoid the possibility of overflow.  
I doubt you have the need for more than 64 nodes, so I'd highly recommend 
leaving it at the default unless you want to debug or test some large 
numa=fake settings.