Re: [PATCH] Allow per-cpu variables to be page-aligned

[ebiederm@xmission.com (Eric W. Biederman)]

Rusty Russell <rusty@rustcorp.com.au> writes:

> [This was part of the GDT cleanups and per-cpu-> pda changes, which I
> have revised, but this stands on its own.  The only change is catching
> the x86-64 per-cpu allocator too].
> ==
> Let's allow page-alignment in general for per-cpu data (wanted by Xen,
> and Ingo suggested KVM as well).
>
> Because larger alignments can use more room, we increase the max
> per-cpu memory to 64k rather than 32k: it's getting a little tight.
>
> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
> Acked-by: Ingo Molnar <mingo@elte.hu>

> ===================================================================
> --- a/kernel/module.c
> +++ b/kernel/module.c
> @@ -346,10 +346,10 @@ static void *percpu_modalloc(unsigned lo
>  	unsigned int i;
>  	void *ptr;
>  
> -	if (align > SMP_CACHE_BYTES) {
> -		printk(KERN_WARNING "%s: per-cpu alignment %li > %i\n",
> -		       name, align, SMP_CACHE_BYTES);
> -		align = SMP_CACHE_BYTES;
> +	if (align > PAGE_SIZE) {
> +		printk(KERN_WARNING "%s: per-cpu alignment %li > %li\n",
> +		       name, align, PAGE_SIZE);
> +		align = PAGE_SIZE;
>  	}
>  
>  	ptr = __per_cpu_start;
> @@ -430,7 +430,7 @@ static int percpu_modinit(void)
>  	pcpu_size = kmalloc(sizeof(pcpu_size[0]) * pcpu_num_allocated,
>  			    GFP_KERNEL);
>  	/* Static in-kernel percpu data (used). */
> -	pcpu_size[0] = -ALIGN(__per_cpu_end-__per_cpu_start, SMP_CACHE_BYTES);
> +	pcpu_size[0] = -ALIGN(__per_cpu_end-__per_cpu_start, PAGE_SIZE);
>  	/* Free room. */
>  	pcpu_size[1] = PERCPU_ENOUGH_ROOM + pcpu_size[0];
>  	if (pcpu_size[1] < 0) {

Do we really want to allow modules to be able to allocate page sized
per cpu memory.  If my memory servers on how this code works we will wind
up allocating 1 page of per cpu memory for every module that allocates a
per cpu variable.  128 bytes sucks 4k is an order of magnitude worse.

On x86_64 we are only reserving 8K for modules...

Eric