[PATCH] [percpu] Make the unit size of the first chunk the same as other chunks

[PATCH] [percpu] Make the unit size of the first chunk the same as other chunks

[Zhihui Zhang]

Since we have already allocated the full unit size for the first chunk, we might as well use
it so that the unit size are the same for all chunks. The page first chunk allocator already
has this effect because it allocates one page at a time.

Signed-off-by: Zhihui Zhang <zzhsuny@gmail.com>
---
 mm/percpu.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/mm/percpu.c b/mm/percpu.c
index 014bab6..7242360 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -1960,6 +1960,7 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, size_t dyn_size,
 		return PTR_ERR(ai);
 
 	size_sum = ai->static_size + ai->reserved_size + ai->dyn_size;
+	ai->dyn_size += ai->unit_size - size_sum;
 	areas_size = PFN_ALIGN(ai->nr_groups * sizeof(void *));
 
 	areas = memblock_virt_alloc_nopanic(areas_size, 0);
@@ -2006,9 +2007,8 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, size_t dyn_size,
 				free_fn(ptr, ai->unit_size);
 				continue;
 			}
-			/* copy and return the unused part */
+			/* copy static data */
 			memcpy(ptr, __per_cpu_load, ai->static_size);
-			free_fn(ptr + size_sum, ai->unit_size - size_sum);
 		}
 	}
 
@@ -2034,7 +2034,7 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, size_t dyn_size,
 	}
 
 	pr_info("PERCPU: Embedded %zu pages/cpu @%p s%zu r%zu d%zu u%zu\n",
-		PFN_DOWN(size_sum), base, ai->static_size, ai->reserved_size,
+		PFN_DOWN(ai->unit_size), base, ai->static_size, ai->reserved_size,
 		ai->dyn_size, ai->unit_size);
 
 	rc = pcpu_setup_first_chunk(ai, base);
-- 
1.8.1.2

Re: [PATCH] [percpu] Make the unit size of the first chunk the same as other chunks

[Tejun Heo]

On Sat, Oct 25, 2014 at 11:05:58AM -0400, Zhihui Zhang wrote:
> Since we have already allocated the full unit size for the first chunk, we might as well use
> it so that the unit size are the same for all chunks. The page first chunk allocator already
> has this effect because it allocates one page at a time.

I'm not following.  Where do we allocate the full unit size for the
first chunk?

Thanks.

-- 
tejun

Re: [PATCH] [percpu] Make the unit size of the first chunk the same as other chunks

[Zhihui Zhang]

In pcpu_embed_first_chunk(), we allocate full unit size for each CPU
in the first chunk:

1981                 /* allocate space for the whole group */
1982                 ptr = alloc_fn(cpu, gi->nr_units * ai->unit_size,
atom_size);
1983                 if (!ptr) {
1984                         rc = -ENOMEM;
1985                         goto out_free_areas;
1986                 }


Later we freed unused part:


2009                         /* copy and return the unused part */
2010                         memcpy(ptr, __per_cpu_load, ai->static_size);
2011                         free_fn(ptr + size_sum, ai->unit_size - size_sum);

I am trying to make each CPU to have a full unit size in the first
chunk, same as in all other chunks. Does this make sense?

-Zhihui

On Mon, Oct 27, 2014 at 10:08 AM, Tejun Heo <tj@kernel.org> wrote:
> On Sat, Oct 25, 2014 at 11:05:58AM -0400, Zhihui Zhang wrote:
>> Since we have already allocated the full unit size for the first chunk, we might as well use
>> it so that the unit size are the same for all chunks. The page first chunk allocator already
>> has this effect because it allocates one page at a time.
>
> I'm not following.  Where do we allocate the full unit size for the
> first chunk?
>
> Thanks.
>
> --
> tejun

Re: [PATCH] [percpu] Make the unit size of the first chunk the same as other chunks

[Tejun Heo]

On Mon, Oct 27, 2014 at 07:32:47PM -0400, Zhihui Zhang wrote:
> I am trying to make each CPU to have a full unit size in the first
> chunk, same as in all other chunks. Does this make sense?

And why are you trying to do that?  Unit size may be highly variable.
Why would it make sense to reserve memory area according to a value
which can fluctuate for reasons completely unrelated to actual percpu
memory usage?

Thanks.

-- 
tejun

Re: [PATCH] [percpu] Make the unit size of the first chunk the same as other chunks

[Zhihui Zhang]

I see your point. Thanks.

-Zhihui

On Wed, Oct 29, 2014 at 12:18 AM, Tejun Heo <tj@kernel.org> wrote:
> Plesae restore lkml cc when replying.
>
> On Tue, Oct 28, 2014 at 08:12:30PM -0400, Zhihui Zhang wrote:
>> My patch just increases the dynamic area in the first chunk slightly
>> to cover the round up surplus. On my 64-bit laptop, it is 12288 bytes.
>
> As I wrote before, it's 12288 bytes on your laptop but it can be much
> larger on other setups.
>
>> It will mostly likely be used, and in fact, a second chunk will be
>> most likely needed as well.  So in theory you are right, but in
>> practice, it probably won't matter.
>
> If the initial dynamic reserve is too small, which could be the case
> given the overall increase in percpu memory, increase
> PERCPU_DYNAMIC_EARLY_SIZE.
>
> --
> tejun