Re: [PATCH] Add nid sanity on alloc_pages_node

[Joe Jin <joe.jin@oracle.com>]

On 2007-07-12 22:18, Andrew Morton wrote:
> On Fri, 13 Jul 2007 10:45:07 +0800 Joe Jin <joe.jin@oracle.com> wrote:
> 
> > This patch add nid sanity check on alloc_pages_node().
> > While two process change nr_hugepages at a system, alloc_fresh_huge_page()
> > been called, at this function, nid defined as a static variable, but, there
> > is not any protection of, if 2 process called at the same time, maybe pass a
> > invalid nid to alloc_pages_node. 
> > 
> > We have hit it by following scripts:
> > 
> > #!/bin/bash
> > while : ; do 
> > 	echo 1000000000000 > /proc/sys/vm/nr_hugepages 
> > 	echo 1 > /proc/sys/vm/nr_hugepages 
> > 	echo 10000000000000000000 > /proc/sys/vm/nr_hugepages 
> > 	echo 0 > /proc/sys/vm/nr_hugepages 
> > done 
> > 
> > 
> > Run the script at _two_ difference terminal, after a short time, a kernel panic
> > info will print.
> > 
> > 
> > Signed-off-by: Joe Jin <joe.jin@oracle.com>
> > ---
> > 
> > --- linux-2.6.22/include/linux/gfp.h.orig	2007-07-12 15:06:23.000000000 +0800
> > +++ linux-2.6.22/include/linux/gfp.h	2007-07-12 15:02:59.000000000 +0800
> > @@ -133,6 +133,9 @@
> >  	/* Unknown node is current node */
> >  	if (nid < 0)
> >  		nid = numa_node_id();
> > +	
> > +	if (unlikely(nid == MAX_NUMNODES))
> > +		nid = first_node(node_online_map);
> >  
> >  	return __alloc_pages(gfp_mask, order,
> >  		NODE_DATA(nid)->node_zonelists + gfp_zone(gfp_mask));
> 
> alloc_pages_node() is pretty much the last place where we want to fix this:
> it adds more cycles and more code to many important codepaths in the
> kernel.
> 
> It'd be much better to fix the race within alloc_fresh_huge_page().  That
> function is pretty pathetic.
> 
> Something like this?
> 
> --- a/mm/hugetlb.c~a
> +++ a/mm/hugetlb.c
> @@ -105,13 +105,20 @@ static void free_huge_page(struct page *
>  
>  static int alloc_fresh_huge_page(void)
>  {
> -	static int nid = 0;
> +	static int prev_nid;
> +	static DEFINE_SPINLOCK(nid_lock);
>  	struct page *page;
> -	page = alloc_pages_node(nid, htlb_alloc_mask|__GFP_COMP|__GFP_NOWARN,
> -					HUGETLB_PAGE_ORDER);
> -	nid = next_node(nid, node_online_map);
> +	int nid;
> +
> +	spin_lock(&nid_lock);
> +	nid = next_node(prev_nid, node_online_map);
>  	if (nid == MAX_NUMNODES)
>  		nid = first_node(node_online_map);
> +	prev_nid = nid;
> +	spin_unlock(&nid_lock);
> +
> +	page = alloc_pages_node(nid, htlb_alloc_mask|__GFP_COMP|__GFP_NOWARN,
> +					HUGETLB_PAGE_ORDER);
>  	if (page) {
>  		set_compound_page_dtor(page, free_huge_page);
>  		spin_lock(&hugetlb_lock);
> _
> 

The patch looks good for this bug, thanks :)

if other caller give a invalid nid to alloc_pages_node(), __alloc_pages
will crash again.

So I think we add some sanity check for nid at alloc_pages_node is
meaningful.  

another question, if (nid >= MAX_NUMNODES),  may I set nid to 0 directly
like following code?

	if (unlikly(nid >= MAX_NUMNODES)
		nid = 0

Thanks,
Joe