[RFC][PATCH] hot add memory which is not aligned to section

[RFC][PATCH] hot add memory which is not aligned to section

[KAMEZAWA Hiroyuki]

This patch allows hot-add memory region which is not aligned to section.
Based on linux-2.6.17-rc2-mm1 + memory hotadd ioresource register patch.
http://www.uwsg.indiana.edu/hypermail/linux/kernel/0604.3/1188.html

Now, hot-added memory has to be aligned to section size. Considering big section
sized archs, this is not useful.They sometimes have not aligned memory.
For example, fujitsu's NUMA machine has 64MB firmware area at the lowest address
of each node.

If hot-added memory is registerd to iomem resoruce, we can make use
of that information to detect valid memory range.

Signed-Off-By: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>



Index: linux-2.6.17-rc2-mm1/kernel/resource.c
===================================================================
--- linux-2.6.17-rc2-mm1.orig/kernel/resource.c	2006-04-27 18:00:16.000000000 +0900
+++ linux-2.6.17-rc2-mm1/kernel/resource.c	2006-04-27 21:52:51.000000000 +0900
@@ -242,6 +242,44 @@
 
 EXPORT_SYMBOL(release_resource);
 
+#ifdef CONFIG_MEMORY_HOTPLUG
+/*
+ * Finds memory reosurce exists higher than  specified address.
+ * real_lock(&resource_lock) must be held before calling this.
+ */
+int find_next_system_ram(struct resource *res)
+{
+	u64 start, end;
+	struct resource *p;
+
+	BUG_ON(!res);
+
+	start = res->start;
+	end = res->end;
+
+	read_lock(&resource_lock);
+	for( p = iomem_resource.child; p ; p = p->sibling) {
+		/* system ram is just marked as IORESOURCE_MEM */
+		if (p->flags != IORESOURCE_MEM)
+			continue;
+		if (p->start > end) {
+			p = NULL;
+			break;
+		}
+		if (p->start >= start)
+			break;
+	}
+	read_unlock(&resource_lock);
+	if (!p)
+		return 0;
+	/* copy data */
+	res->start = p->start;
+	res->end = p->end;
+	return 1;
+}
+
+#endif
+
 /*
  * Find empty slot in the resource tree given range and alignment.
  */
Index: linux-2.6.17-rc2-mm1/include/linux/ioport.h
===================================================================
--- linux-2.6.17-rc2-mm1.orig/include/linux/ioport.h	2006-04-27 18:00:16.000000000 +0900
+++ linux-2.6.17-rc2-mm1/include/linux/ioport.h	2006-04-27 21:47:25.000000000 +0900
@@ -105,6 +105,10 @@
 			     void *alignf_data);
 int adjust_resource(struct resource *res, u64 start,
 		    u64 size);
+#ifdef CONFIG_MEMORY_HOTPLUG
+/* get registered SYSTEM_RAM resources in specified area */
+extern int find_next_system_ram(struct resource *res);
+#endif
 
 /* Convenience shorthand with allocation */
 #define request_region(start,n,name)	__request_region(&ioport_resource, (start), (n), (name))
Index: linux-2.6.17-rc2-mm1/mm/memory_hotplug.c
===================================================================
--- linux-2.6.17-rc2-mm1.orig/mm/memory_hotplug.c	2006-04-27 20:21:32.000000000 +0900
+++ linux-2.6.17-rc2-mm1/mm/memory_hotplug.c	2006-04-27 21:57:17.000000000 +0900
@@ -123,6 +123,9 @@
 	unsigned long i;
 	unsigned long flags;
 	unsigned long onlined_pages = 0;
+	struct resource res;
+	u64 section_end;
+	unsigned long start_pfn;
 	struct zone *zone;
 	int need_zonelists_rebuild = 0;
 
@@ -145,10 +148,26 @@
 	if (!populated_zone(zone))
 		need_zonelists_rebuild = 1;
 
-	for (i = 0; i < nr_pages; i++) {
-		struct page *page = pfn_to_page(pfn + i);
-		online_page(page);
-		onlined_pages++;
+	res.start = (u64)pfn << PAGE_SHIFT;
+	res.end = res.start + ((u64)nr_pages << PAGE_SHIFT) - 1;
+	section_end = res.end;
+
+	while (find_next_system_ram(&res)) {
+		start_pfn = (unsigned long)(res.start >> PAGE_SHIFT);
+		nr_pages = (unsigned long)
+                           ((res.end + 1 - res.start) >> PAGE_SHIFT);
+
+		if (PageReserved(pfn_to_page(start_pfn))) {
+			/* this region's page is not populated now */
+			for (i = 0; i < nr_pages; i++) {
+				struct page *page = pfn_to_page(start_pfn + i);
+				online_page(page);
+				onlined_pages++;
+			}
+		}
+
+		res.start = res.end + 1;
+		res.end = section_end;
 	}
 	zone->present_pages += onlined_pages;
 	zone->zone_pgdat->node_present_pages += onlined_pages;

Re: [Lhms-devel] [RFC][PATCH] hot add memory which is not aligned to section

[Dave Hansen]

On Thu, 2006-04-27 at 22:37 +0900, KAMEZAWA Hiroyuki wrote:
> @@ -145,10 +148,26 @@
>  	if (!populated_zone(zone))
>  		need_zonelists_rebuild = 1;
>  
> -	for (i = 0; i < nr_pages; i++) {
> -		struct page *page = pfn_to_page(pfn + i);
> -		online_page(page);
> -		onlined_pages++;
> +	res.start = (u64)pfn << PAGE_SHIFT;
> +	res.end = res.start + ((u64)nr_pages << PAGE_SHIFT) - 1;
> +	section_end = res.end;
> +
> +	while (find_next_system_ram(&res)) {
> +		start_pfn = (unsigned long)(res.start >> PAGE_SHIFT);
> +		nr_pages = (unsigned long)
> +                           ((res.end + 1 - res.start) >> PAGE_SHIFT);
> +
> +		if (PageReserved(pfn_to_page(start_pfn))) {
> +			/* this region's page is not populated now */
> +			for (i = 0; i < nr_pages; i++) {
> +				struct page *page = pfn_to_page(start_pfn + i);
> +				online_page(page);
> +				onlined_pages++;
> +			}
> +		}
> +
> +		res.start = res.end + 1;
> +		res.end = section_end;
>  	}
>  	zone->present_pages += onlined_pages;
>  	zone->zone_pgdat->node_present_pages += onlined_pages;

First of all, bravo for doing this in an architecture-independent way.
Very nice.

I'd really prefer to keep the 'struct resource' handling out of the
memory hotplug code.  This took a nice, little, comprehensible for loop,
and made it quite a bit more complex.  There is also a lot of casting
going on, which I don't really grasp in a first glance.

The 'struct resource' which gets passed into find_next_system_ram()
isn't a real resource.  Why not just pass a normal start and end address
in there, and let _it_ do the work?

It looks like that whole loop is optimized for being able to online a
really sparse area without diving into the iomem tables very often.
This seems like a premature complicating optimization to me.  

Why not do something like this:

	for (i = 0; i < nr_pages; i++) {
		struct page *page = pfn_to_page(pfn + i);

		if (page_is_in_io_resource(page))
			continue;

		online_page(page);
		onlined_pages++;
	}

That way, you keep the memory_hotplug.c file nice and neat.

Also, remind me again why you can't just make the SECTION_SIZE match
your 64MB I/O hole sizes.  I forget a lot/ :)

-- Dave

Re: [Lhms-devel] [RFC][PATCH] hot add memory which is not aligned to section

[KAMEZAWA Hiroyuki]

On Mon, 01 May 2006 07:56:32 -0700
Dave Hansen <haveblue@us.ibm.com> wrote:

> The 'struct resource' which gets passed into find_next_system_ram()
> isn't a real resource.  Why not just pass a normal start and end address
> in there, and let _it_ do the work?
> 
just I don't like return prural values by one function call, I needed start
and end.

> It looks like that whole loop is optimized for being able to online a
> really sparse area without diving into the iomem tables very often.
> This seems like a premature complicating optimization to me.  
> 
Hmm...complicating ?  

> Why not do something like this:
> 
> 	for (i = 0; i < nr_pages; i++) {
> 		struct page *page = pfn_to_page(pfn + i);
> 
> 		if (page_is_in_io_resource(page))
> 			continue;
> 
> 		online_page(page);
> 		onlined_pages++;
> 	}
> 
> That way, you keep the memory_hotplug.c file nice and neat.
> 
I'll clean up this later (if I can). maybe adding function like this will work.
-
	ioresouce_walk(scan_start, scan_end, callback_func);
-
I don't want to modify structures of resource just for memory hotplug.
Then, I want to avoid list-waliking as much as possible.


> Also, remind me again why you can't just make the SECTION_SIZE match
> your 64MB I/O hole sizes.  I forget a lot/ :)
> 
ia64 has 1GB SECTION_SIZE as default ;). I don't think our (fujitsu's)
customers can configure and recompile the kernel.


-Kame
P.S. I'm away from network until this week end.