[PATCH] POWER 4 fails to boot with NUMA

[PATCH] POWER 4 fails to boot with NUMA

[Paul Mackerras]

From: Mike Kravetz <kravetz@us.ibm.com>

If CONFIG_NUMA is set, some POWER 4 systems will fail to boot.  This is
because of special processing needed to handle invalid node IDs (0xffff)
on POWER 4.  My previous patch to handle memory 'holes' within nodes
forgot to add this special case for POWER 4 in one place.

In reality, I'm not sure that configuring the kernel for NUMA on POWER 4
makes much sense.  Are there POWER 4 based systems with NUMA characteristics
that are presented by the firmware?  But, distros want one kernel for all
systems so NUMA is on by default in their kernels.  The patch handles those
cases.

Signed-off-by: Mike Kravetz <kravetz@us.ibm.com>
Signed-off-by: Paul Mackerras <paulus@samba.org>
---
diff -urN linux-2.6/arch/ppc64/mm/numa.c g5-ppc64/arch/ppc64/mm/numa.c
--- linux-2.6/arch/ppc64/mm/numa.c	2005-06-24 13:38:52.000000000 +1000
+++ g5-ppc64/arch/ppc64/mm/numa.c	2005-08-01 15:15:55.000000000 +1000
@@ -647,7 +647,12 @@
 new_range:
 			mem_start = read_n_cells(addr_cells, &memcell_buf);
 			mem_size = read_n_cells(size_cells, &memcell_buf);
-			numa_domain = numa_enabled ? of_node_numa_domain(memory) : 0;
+			if (numa_enabled) {
+				numa_domain = of_node_numa_domain(memory);
+				if (numa_domain  >= MAX_NUMNODES)
+					numa_domain = 0;
+			} else
+				numa_domain =  0;
 
 			if (numa_domain != nid)
 				continue;

Re: [PATCH] POWER 4 fails to boot with NUMA

[Sonny Rao]

On Mon, Aug 01, 2005 at 12:27:42AM -0500, Paul Mackerras wrote:
> From: Mike Kravetz <kravetz@us.ibm.com>
> 
> If CONFIG_NUMA is set, some POWER 4 systems will fail to boot.  This is
> because of special processing needed to handle invalid node IDs (0xffff)
> on POWER 4.  My previous patch to handle memory 'holes' within nodes
> forgot to add this special case for POWER 4 in one place.
> 
> In reality, I'm not sure that configuring the kernel for NUMA on POWER 4
> makes much sense.  Are there POWER 4 based systems with NUMA characteristics
> that are presented by the firmware?  But, distros want one kernel for all
> systems so NUMA is on by default in their kernels.  The patch handles those
> cases.

IIRC, In SMP mode the NUMA topology is exported.  I've tried this on a
p690 and it worked correctly on older kernels (2.6.10 or 2.6.11) 

I also noticed a nice speedup on a few things compared to LPAR mode :-)

Sonny

Re: [PATCH] POWER 4 fails to boot with NUMA

[Martin J. Bligh]

--Sonny Rao <sonny@burdell.org> wrote (on Monday, August 01, 2005 02:23:22 -0400):

> On Mon, Aug 01, 2005 at 12:27:42AM -0500, Paul Mackerras wrote:
>> From: Mike Kravetz <kravetz@us.ibm.com>
>> 
>> If CONFIG_NUMA is set, some POWER 4 systems will fail to boot.  This is
>> because of special processing needed to handle invalid node IDs (0xffff)
>> on POWER 4.  My previous patch to handle memory 'holes' within nodes
>> forgot to add this special case for POWER 4 in one place.
>> 
>> In reality, I'm not sure that configuring the kernel for NUMA on POWER 4
>> makes much sense.  Are there POWER 4 based systems with NUMA characteristics
>> that are presented by the firmware?  But, distros want one kernel for all
>> systems so NUMA is on by default in their kernels.  The patch handles those
>> cases.
> 
> IIRC, In SMP mode the NUMA topology is exported.  I've tried this on a
> p690 and it worked correctly on older kernels (2.6.10 or 2.6.11) 
> 
> I also noticed a nice speedup on a few things compared to LPAR mode :-)

Yeah, I have a p650 that's set up similarly .... I thought the auto-test
stuff was covering that, but it seems NUMA is not turned on for that box
like I thought it was. will fix ....

M.