[PATCH] align crash_notes allocation to make it be inside one physical page

[PATCH] align crash_notes allocation to make it be inside one physical page

[Baoquan He]

People reported that crash_notes in /proc/vmcore were corrupted and
this cause crash kdump failure. With code debugging and log we got
the root cause. This is because percpu variable crash_notes are
allocated in 2 vmalloc pages. As you know percpu is based on vmalloc
by default. Then vmalloc can't guarantee 2 continuous vmalloc pages
are also on 2 continuous physical pages. Then 1st kernel export the
starting addr and size, kdump kernel use the starting addr and size
to get the content of crash_notes, then 2nd part may not be in the
next neighbouring physical page as we think. That's why nhdr_ptr->n_namesz
or nhdr_ptr->n_descsz could be very huge in update_note_header_size_elf64()
and cause note header merging failure or some warnings.

In this patch change to call __alloc_percpu() to passed in the align
value which is nearest the the 2^log(sizeof(note_buf_t)). This align
value can make sure the crash_notes is allocated inside one physical
page since sizeof(note_buf_t) in all ARCHS is smaller PAGE_SIZE. But
add a WARN_ON in case it grow to be bigger than PAGE_SIZE in the future.

Signed-off-by: Baoquan He <bhe@redhat.com>
---
 kernel/kexec.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/kernel/kexec.c b/kernel/kexec.c
index a785c10..1740c42 100644
--- a/kernel/kexec.c
+++ b/kernel/kexec.c
@@ -1620,7 +1620,16 @@ void crash_save_cpu(struct pt_regs *regs, int cpu)
 static int __init crash_notes_memory_init(void)
 {
 	/* Allocate memory for saving cpu registers. */
-	crash_notes = alloc_percpu(note_buf_t);
+	size_t size, align;
+	int order;
+
+	size = sizeof(note_buf_t);
+	order = get_count_order(size);
+	align = 1<< order;
+
+	WARN_ON(size > PAGE_SIZE);
+
+	crash_notes = __alloc_percpu(size, align);
 	if (!crash_notes) {
 		pr_warn("Kexec: Memory allocation for saving cpu register states failed\n");
 		return -ENOMEM;
-- 
2.1.0

Re: [PATCH] align crash_notes allocation to make it be inside one physical page

[Minfei Huang]

On 07/30/15 at 11:07am, Baoquan He wrote:
> People reported that crash_notes in /proc/vmcore were corrupted and
> this cause crash kdump failure. With code debugging and log we got
> the root cause. This is because percpu variable crash_notes are
> allocated in 2 vmalloc pages. As you know percpu is based on vmalloc
> by default. Then vmalloc can't guarantee 2 continuous vmalloc pages
> are also on 2 continuous physical pages. Then 1st kernel export the
> starting addr and size, kdump kernel use the starting addr and size
> to get the content of crash_notes, then 2nd part may not be in the
> next neighbouring physical page as we think. That's why nhdr_ptr->n_namesz
> or nhdr_ptr->n_descsz could be very huge in update_note_header_size_elf64()
> and cause note header merging failure or some warnings.
> 
> In this patch change to call __alloc_percpu() to passed in the align
> value which is nearest the the 2^log(sizeof(note_buf_t)). This align
> value can make sure the crash_notes is allocated inside one physical
> page since sizeof(note_buf_t) in all ARCHS is smaller PAGE_SIZE. But
> add a WARN_ON in case it grow to be bigger than PAGE_SIZE in the future.
> 
> Signed-off-by: Baoquan He <bhe@redhat.com>
> ---
>  kernel/kexec.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/kexec.c b/kernel/kexec.c
> index a785c10..1740c42 100644
> --- a/kernel/kexec.c
> +++ b/kernel/kexec.c
> @@ -1620,7 +1620,16 @@ void crash_save_cpu(struct pt_regs *regs, int cpu)
>  static int __init crash_notes_memory_init(void)
>  {
>  	/* Allocate memory for saving cpu registers. */
> -	crash_notes = alloc_percpu(note_buf_t);
> +	size_t size, align;
> +	int order;
> +
> +	size = sizeof(note_buf_t);
> +	order = get_count_order(size);
> +	align = 1<< order;
> +
> +	WARN_ON(size > PAGE_SIZE);

It is fine without this warning, since percpu will fail to allocate the
memory larger than PAGE_SIZE.

Thanks
Minfei

Re: [PATCH] align crash_notes allocation to make it be inside one physical page

[Baoquan He]

On 07/30/15 at 01:15pm, Minfei Huang wrote:
> On 07/30/15 at 11:07am, Baoquan He wrote:
> > diff --git a/kernel/kexec.c b/kernel/kexec.c
> > index a785c10..1740c42 100644
> > --- a/kernel/kexec.c
> > +++ b/kernel/kexec.c
> > @@ -1620,7 +1620,16 @@ void crash_save_cpu(struct pt_regs *regs, int cpu)
> >  static int __init crash_notes_memory_init(void)
> >  {
> >  	/* Allocate memory for saving cpu registers. */
> > -	crash_notes = alloc_percpu(note_buf_t);
> > +	size_t size, align;
> > +	int order;
> > +
> > +	size = sizeof(note_buf_t);
> > +	order = get_count_order(size);
> > +	align = 1<< order;
> > +
> > +	WARN_ON(size > PAGE_SIZE);
> 
> It is fine without this warning, since percpu will fail to allocate the
> memory larger than PAGE_SIZE.

Thanks for your comment.

percpu will fail if align is larger than PAGE_SIZE. I will adjust it
as align = min(1<<order, PAGE_SIZE). But adding WARN_ON makes sense
in case sizeof(note_buf_t) is bigger than PAGE_SIZE in the future,
then we need consider changing  the design of storing crash_notes.

Thanks
Baoquan