[PATCHv3 0/4] x86: Reduce code duplication on page table initialization

[PATCHv3 0/4] x86: Reduce code duplication on page table initialization

[Kirill A. Shutemov]

Use kernel_ident_mapping_init() to initialize kernel page tables where
possible, replacing manual initialization, reducing code duplication.

v3:
 - Reviewed-bys from Tom;
 - Improve commit messages;
v2:
 - A separate patch to change what PA is mapped at relocate_kernel() VA.
 - Improve commit messages;
 - Add Reveiwed-by from Kai;

Kirill A. Shutemov (4):
  x86/mm/ident_map: Fix virtual address wrap to zero
  x86/acpi: Replace manual page table initialization with
    kernel_ident_mapping_init()
  x86/64/kexec: Map original relocate_kernel() in
    init_transition_pgtable()
  x86/64/kexec: Rewrite init_transition_pgtable() with
    kernel_ident_mapping_init()

 arch/x86/include/asm/kexec.h       |  5 +-
 arch/x86/kernel/acpi/madt_wakeup.c | 73 +++++-------------------
 arch/x86/kernel/machine_kexec_64.c | 89 +++++++++++-------------------
 arch/x86/mm/ident_map.c            | 14 +----
 4 files changed, 50 insertions(+), 131 deletions(-)

-- 
2.43.0

[PATCHv3 1/4] x86/mm/ident_map: Fix virtual address wrap to zero

[Kirill A. Shutemov]

Calculation of 'next' virtual address doesn't protect against wrapping
to zero. It can result in page table corruption and hang. The
problematic case is possible if user sets high x86_mapping_info::offset.

The wrapping to zero only occurs if the top PGD entry is accessed.
There are no such users in the upstream. Only hibernate_64.c uses
x86_mapping_info::offset, and it operates on the direct mapping range,
which is not the top PGD entry.

Replace manual 'next' calculation with p?d_addr_end() which handles
wrapping correctly.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
---
 arch/x86/mm/ident_map.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/arch/x86/mm/ident_map.c b/arch/x86/mm/ident_map.c
index 437e96fb4977..5872f3ee863c 100644
--- a/arch/x86/mm/ident_map.c
+++ b/arch/x86/mm/ident_map.c
@@ -101,9 +101,7 @@ static int ident_pud_init(struct x86_mapping_info *info, pud_t *pud_page,
 		pmd_t *pmd;
 		bool use_gbpage;
 
-		next = (addr & PUD_MASK) + PUD_SIZE;
-		if (next > end)
-			next = end;
+		next = pud_addr_end(addr, end);
 
 		/* if this is already a gbpage, this portion is already mapped */
 		if (pud_leaf(*pud))
@@ -154,10 +152,7 @@ static int ident_p4d_init(struct x86_mapping_info *info, p4d_t *p4d_page,
 		p4d_t *p4d = p4d_page + p4d_index(addr);
 		pud_t *pud;
 
-		next = (addr & P4D_MASK) + P4D_SIZE;
-		if (next > end)
-			next = end;
-
+		next = p4d_addr_end(addr, end);
 		if (p4d_present(*p4d)) {
 			pud = pud_offset(p4d, 0);
 			result = ident_pud_init(info, pud, addr, next);
@@ -199,10 +194,7 @@ int kernel_ident_mapping_init(struct x86_mapping_info *info, pgd_t *pgd_page,
 		pgd_t *pgd = pgd_page + pgd_index(addr);
 		p4d_t *p4d;
 
-		next = (addr & PGDIR_MASK) + PGDIR_SIZE;
-		if (next > end)
-			next = end;
-
+		next = pgd_addr_end(addr, end);
 		if (pgd_present(*pgd)) {
 			p4d = p4d_offset(pgd, 0);
 			result = ident_p4d_init(info, p4d, addr, next);
-- 
2.43.0

[PATCHv3 2/4] x86/acpi: Replace manual page table initialization with kernel_ident_mapping_init()

[Kirill A. Shutemov]

The function init_transition_pgtable() maps the page with
asm_acpi_mp_play_dead() into an identity mapping.

Replace manual page table initialization with kernel_ident_mapping_init()
to avoid code duplication. Use x86_mapping_info::offset to get the page
mapped at the correct location.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
---
 arch/x86/kernel/acpi/madt_wakeup.c | 73 ++++++------------------------
 1 file changed, 15 insertions(+), 58 deletions(-)

diff --git a/arch/x86/kernel/acpi/madt_wakeup.c b/arch/x86/kernel/acpi/madt_wakeup.c
index d5ef6215583b..78960b338be9 100644
--- a/arch/x86/kernel/acpi/madt_wakeup.c
+++ b/arch/x86/kernel/acpi/madt_wakeup.c
@@ -70,58 +70,6 @@ static void __init free_pgt_page(void *pgt, void *dummy)
 	return memblock_free(pgt, PAGE_SIZE);
 }
 
-/*
- * Make sure asm_acpi_mp_play_dead() is present in the identity mapping at
- * the same place as in the kernel page tables. asm_acpi_mp_play_dead() switches
- * to the identity mapping and the function has be present at the same spot in
- * the virtual address space before and after switching page tables.
- */
-static int __init init_transition_pgtable(pgd_t *pgd)
-{
-	pgprot_t prot = PAGE_KERNEL_EXEC_NOENC;
-	unsigned long vaddr, paddr;
-	p4d_t *p4d;
-	pud_t *pud;
-	pmd_t *pmd;
-	pte_t *pte;
-
-	vaddr = (unsigned long)asm_acpi_mp_play_dead;
-	pgd += pgd_index(vaddr);
-	if (!pgd_present(*pgd)) {
-		p4d = (p4d_t *)alloc_pgt_page(NULL);
-		if (!p4d)
-			return -ENOMEM;
-		set_pgd(pgd, __pgd(__pa(p4d) | _KERNPG_TABLE));
-	}
-	p4d = p4d_offset(pgd, vaddr);
-	if (!p4d_present(*p4d)) {
-		pud = (pud_t *)alloc_pgt_page(NULL);
-		if (!pud)
-			return -ENOMEM;
-		set_p4d(p4d, __p4d(__pa(pud) | _KERNPG_TABLE));
-	}
-	pud = pud_offset(p4d, vaddr);
-	if (!pud_present(*pud)) {
-		pmd = (pmd_t *)alloc_pgt_page(NULL);
-		if (!pmd)
-			return -ENOMEM;
-		set_pud(pud, __pud(__pa(pmd) | _KERNPG_TABLE));
-	}
-	pmd = pmd_offset(pud, vaddr);
-	if (!pmd_present(*pmd)) {
-		pte = (pte_t *)alloc_pgt_page(NULL);
-		if (!pte)
-			return -ENOMEM;
-		set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
-	}
-	pte = pte_offset_kernel(pmd, vaddr);
-
-	paddr = __pa(vaddr);
-	set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, prot));
-
-	return 0;
-}
-
 static int __init acpi_mp_setup_reset(u64 reset_vector)
 {
 	struct x86_mapping_info info = {
@@ -130,6 +78,7 @@ static int __init acpi_mp_setup_reset(u64 reset_vector)
 		.page_flag      = __PAGE_KERNEL_LARGE_EXEC,
 		.kernpg_flag    = _KERNPG_TABLE_NOENC,
 	};
+	unsigned long mstart, mend;
 	pgd_t *pgd;
 
 	pgd = alloc_pgt_page(NULL);
@@ -137,8 +86,6 @@ static int __init acpi_mp_setup_reset(u64 reset_vector)
 		return -ENOMEM;
 
 	for (int i = 0; i < nr_pfn_mapped; i++) {
-		unsigned long mstart, mend;
-
 		mstart = pfn_mapped[i].start << PAGE_SHIFT;
 		mend   = pfn_mapped[i].end << PAGE_SHIFT;
 		if (kernel_ident_mapping_init(&info, pgd, mstart, mend)) {
@@ -147,14 +94,24 @@ static int __init acpi_mp_setup_reset(u64 reset_vector)
 		}
 	}
 
-	if (kernel_ident_mapping_init(&info, pgd,
-				      PAGE_ALIGN_DOWN(reset_vector),
-				      PAGE_ALIGN(reset_vector + 1))) {
+	mstart = PAGE_ALIGN_DOWN(reset_vector);
+	mend = mstart + PAGE_SIZE;
+	if (kernel_ident_mapping_init(&info, pgd, mstart, mend)) {
 		kernel_ident_mapping_free(&info, pgd);
 		return -ENOMEM;
 	}
 
-	if (init_transition_pgtable(pgd)) {
+	/*
+	 * Make sure asm_acpi_mp_play_dead() is present in the identity mapping
+	 * at the same place as in the kernel page tables.
+	 * asm_acpi_mp_play_dead() switches to the identity mapping and the
+	 * function has be present at the same spot in the virtual address space
+	 * before and after switching page tables.
+	 */
+	info.offset = __START_KERNEL_map - phys_base;
+	mstart = PAGE_ALIGN_DOWN(__pa(asm_acpi_mp_play_dead));
+	mend = mstart + PAGE_SIZE;
+	if (kernel_ident_mapping_init(&info, pgd, mstart, mend)) {
 		kernel_ident_mapping_free(&info, pgd);
 		return -ENOMEM;
 	}
-- 
2.43.0

[PATCHv3 3/4] x86/64/kexec: Map original relocate_kernel() in init_transition_pgtable()

[Kirill A. Shutemov]

The init_transition_pgtable() function sets up transitional page tables.
It ensures that the relocate_kernel() function is present in the
identity mapping at the same location as in the kernel page tables.
relocate_kernel() switches to the identity mapping, and the function
must be present at the same location in the virtual address space before
and after switching page tables.

init_transition_pgtable() maps a copy of relocate_kernel() in
image->control_code_page at the relocate_kernel() virtual address, but
the original physical address of relocate_kernel() would also work.

It is safe to use original relocate_kernel() physical address cannot be
overwritten until swap_pages() is called, and the relocate_kernel()
virtual address will not be used by then.

Map the original relocate_kernel() at the relocate_kernel() virtual
address in the identity mapping. It is preparation to replace the
init_transition_pgtable() implementation with a call to
kernel_ident_mapping_init().

Note that while relocate_kernel() switches to the identity mapping, it
does not flush global TLB entries (CR4.PGE is not cleared). This means
that in most cases, the kernel still runs relocate_kernel() from the
original physical address before the change.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
 arch/x86/kernel/machine_kexec_64.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index 9c9ac606893e..645690e81c2d 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -157,7 +157,7 @@ static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
 	pte_t *pte;
 
 	vaddr = (unsigned long)relocate_kernel;
-	paddr = __pa(page_address(image->control_code_page)+PAGE_SIZE);
+	paddr = __pa(relocate_kernel);
 	pgd += pgd_index(vaddr);
 	if (!pgd_present(*pgd)) {
 		p4d = (p4d_t *)get_zeroed_page(GFP_KERNEL);
-- 
2.43.0

[PATCHv3 4/4] x86/64/kexec: Rewrite init_transition_pgtable() with kernel_ident_mapping_init()

[Kirill A. Shutemov]

init_transition_pgtable() sets up transitional page tables. Rewrite it
using kernel_ident_mapping_init() to avoid code duplication.

Change struct kimage_arch to track allocated page tables as a list, not
linking them to specific page table levels.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
---
 arch/x86/include/asm/kexec.h       |  5 +-
 arch/x86/kernel/machine_kexec_64.c | 89 +++++++++++-------------------
 2 files changed, 32 insertions(+), 62 deletions(-)

diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h
index ae5482a2f0ca..7f9287f371e6 100644
--- a/arch/x86/include/asm/kexec.h
+++ b/arch/x86/include/asm/kexec.h
@@ -145,10 +145,7 @@ struct kimage_arch {
 };
 #else
 struct kimage_arch {
-	p4d_t *p4d;
-	pud_t *pud;
-	pmd_t *pmd;
-	pte_t *pte;
+	struct list_head pages;
 };
 #endif /* CONFIG_X86_32 */
 
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index 645690e81c2d..fb350372835c 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -134,71 +134,42 @@ map_efi_systab(struct x86_mapping_info *info, pgd_t *level4p)
 	return 0;
 }
 
+static void *alloc_transition_pgt_page(void *data)
+{
+	struct kimage *image = (struct kimage *)data;
+	unsigned long virt;
+
+	virt = get_zeroed_page(GFP_KERNEL);
+	if (!virt)
+		return NULL;
+
+	list_add(&virt_to_page(virt)->lru, &image->arch.pages);
+	return (void *)virt;
+}
+
 static void free_transition_pgtable(struct kimage *image)
 {
-	free_page((unsigned long)image->arch.p4d);
-	image->arch.p4d = NULL;
-	free_page((unsigned long)image->arch.pud);
-	image->arch.pud = NULL;
-	free_page((unsigned long)image->arch.pmd);
-	image->arch.pmd = NULL;
-	free_page((unsigned long)image->arch.pte);
-	image->arch.pte = NULL;
+	struct page *page, *tmp;
+
+	list_for_each_entry_safe(page, tmp, &image->arch.pages, lru) {
+		list_del(&page->lru);
+		free_page((unsigned long)page_address(page));
+	}
 }
 
 static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
 {
-	pgprot_t prot = PAGE_KERNEL_EXEC_NOENC;
-	unsigned long vaddr, paddr;
-	int result = -ENOMEM;
-	p4d_t *p4d;
-	pud_t *pud;
-	pmd_t *pmd;
-	pte_t *pte;
+	struct x86_mapping_info info = {
+		.alloc_pgt_page	= alloc_transition_pgt_page,
+		.context	= image,
+		.page_flag	= __PAGE_KERNEL_LARGE_EXEC,
+		.kernpg_flag	= _KERNPG_TABLE_NOENC,
+		.offset = __START_KERNEL_map - phys_base,
+	};
+	unsigned long mstart = PAGE_ALIGN_DOWN(__pa(relocate_kernel));
+	unsigned long mend = mstart + PAGE_SIZE;
 
-	vaddr = (unsigned long)relocate_kernel;
-	paddr = __pa(relocate_kernel);
-	pgd += pgd_index(vaddr);
-	if (!pgd_present(*pgd)) {
-		p4d = (p4d_t *)get_zeroed_page(GFP_KERNEL);
-		if (!p4d)
-			goto err;
-		image->arch.p4d = p4d;
-		set_pgd(pgd, __pgd(__pa(p4d) | _KERNPG_TABLE));
-	}
-	p4d = p4d_offset(pgd, vaddr);
-	if (!p4d_present(*p4d)) {
-		pud = (pud_t *)get_zeroed_page(GFP_KERNEL);
-		if (!pud)
-			goto err;
-		image->arch.pud = pud;
-		set_p4d(p4d, __p4d(__pa(pud) | _KERNPG_TABLE));
-	}
-	pud = pud_offset(p4d, vaddr);
-	if (!pud_present(*pud)) {
-		pmd = (pmd_t *)get_zeroed_page(GFP_KERNEL);
-		if (!pmd)
-			goto err;
-		image->arch.pmd = pmd;
-		set_pud(pud, __pud(__pa(pmd) | _KERNPG_TABLE));
-	}
-	pmd = pmd_offset(pud, vaddr);
-	if (!pmd_present(*pmd)) {
-		pte = (pte_t *)get_zeroed_page(GFP_KERNEL);
-		if (!pte)
-			goto err;
-		image->arch.pte = pte;
-		set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
-	}
-	pte = pte_offset_kernel(pmd, vaddr);
-
-	if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
-		prot = PAGE_KERNEL_EXEC;
-
-	set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, prot));
-	return 0;
-err:
-	return result;
+	return kernel_ident_mapping_init(&info, pgd, mstart, mend);
 }
 
 static void *alloc_pgt_page(void *data)
@@ -299,6 +270,8 @@ int machine_kexec_prepare(struct kimage *image)
 	unsigned long start_pgtable;
 	int result;
 
+	INIT_LIST_HEAD(&image->arch.pages);
+
 	/* Calculate the offsets */
 	start_pgtable = page_to_pfn(image->control_code_page) << PAGE_SHIFT;
 
-- 
2.43.0

Re: [PATCHv3 3/4] x86/64/kexec: Map original relocate_kernel() in init_transition_pgtable()

[Huang, Kai]

On Mon, 2024-08-19 at 10:08 +0300, Kirill A. Shutemov wrote:
> The init_transition_pgtable() function sets up transitional page tables.
> It ensures that the relocate_kernel() function is present in the
> identity mapping at the same location as in the kernel page tables.
> relocate_kernel() switches to the identity mapping, and the function
> must be present at the same location in the virtual address space before
> and after switching page tables.
> 
> init_transition_pgtable() maps a copy of relocate_kernel() in
> image->control_code_page at the relocate_kernel() virtual address, but
> the original physical address of relocate_kernel() would also work.
> 
> It is safe to use original relocate_kernel() physical address cannot be
> overwritten until swap_pages() is called, and the relocate_kernel()
> virtual address will not be used by then.
> 
> Map the original relocate_kernel() at the relocate_kernel() virtual
> address in the identity mapping. It is preparation to replace the
> init_transition_pgtable() implementation with a call to
> kernel_ident_mapping_init().
> 
> Note that while relocate_kernel() switches to the identity mapping, it
> does not flush global TLB entries (CR4.PGE is not cleared). This means
> that in most cases, the kernel still runs relocate_kernel() from the
> original physical address before the change.
> 
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> ---
>  arch/x86/kernel/machine_kexec_64.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
> index 9c9ac606893e..645690e81c2d 100644
> --- a/arch/x86/kernel/machine_kexec_64.c
> +++ b/arch/x86/kernel/machine_kexec_64.c
> @@ -157,7 +157,7 @@ static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
>  	pte_t *pte;
>  
>  	vaddr = (unsigned long)relocate_kernel;
> -	paddr = __pa(page_address(image->control_code_page)+PAGE_SIZE);
> +	paddr = __pa(relocate_kernel);
>  	pgd += pgd_index(vaddr);
>  	if (!pgd_present(*pgd)) {
>  		p4d = (p4d_t *)get_zeroed_page(GFP_KERNEL);


IIUC, this breaks KEXEC_JUMP (image->preserve_context is true).

The relocate_kernel() first saves couple of regs and some other data like PA
of swap page to the control page.  Note here the VA_CONTROL_PAGE is used to
access the control page, so those data are saved to the control page.

SYM_CODE_START_NOALIGN(relocate_kernel)
        UNWIND_HINT_END_OF_STACK
        ANNOTATE_NOENDBR
        /*      
         * %rdi indirection_page
         * %rsi page_list
         * %rdx start address
         * %rcx preserve_context
         * %r8  bare_metal
         */

	...

        movq    PTR(VA_CONTROL_PAGE)(%rsi), %r11                             
        movq    %rsp, RSP(%r11)                                              
        movq    %cr0, %rax
        movq    %rax, CR0(%r11)
        movq    %cr3, %rax
        movq    %rax, CR3(%r11)
        movq    %cr4, %rax
        movq    %rax, CR4(%r11)

	...

	/*
         * get physical address of control page now
         * this is impossible after page table switch
         */
        movq    PTR(PA_CONTROL_PAGE)(%rsi), %r8

        /* get physical address of page table now too */
        movq    PTR(PA_TABLE_PAGE)(%rsi), %r9

        /* get physical address of swap page now */
        movq    PTR(PA_SWAP_PAGE)(%rsi), %r10

        /* save some information for jumping back */
        movq    %r9, CP_PA_TABLE_PAGE(%r11)
        movq    %r10, CP_PA_SWAP_PAGE(%r11)
        movq    %rdi, CP_PA_BACKUP_PAGES_MAP(%r11)

	...

And after jumping back from the second kernel, relocate_kernel() tries to
restore the saved data:

	...

        /* get the re-entry point of the peer system */
        movq    0(%rsp), %rbp
        leaq    relocate_kernel(%rip), %r8		<---------  (*) 
        movq    CP_PA_SWAP_PAGE(%r8), %r10
        movq    CP_PA_BACKUP_PAGES_MAP(%r8), %rdi
        movq    CP_PA_TABLE_PAGE(%r8), %rax
        movq    %rax, %cr3
        lea     PAGE_SIZE(%r8), %rsp
        call    swap_pages
        movq    $virtual_mapped, %rax
        pushq   %rax
        ANNOTATE_UNRET_SAFE
        ret
        int3
SYM_CODE_END(identity_mapped)

Note the above code (*) uses the VA of relocate_kernel() to access the control
page.  IIUC, that means if we map VA of relocate_kernel() to the original PA
where the code relocate_kernel() resides, then the above code will never be
able to read those data back since they were saved to the control page.

Did I miss anything?

Re: [PATCHv3 3/4] x86/64/kexec: Map original relocate_kernel() in init_transition_pgtable()

[kirill.shutemov]

On Mon, Aug 19, 2024 at 11:16:52AM +0000, Huang, Kai wrote:
> On Mon, 2024-08-19 at 10:08 +0300, Kirill A. Shutemov wrote:
> > The init_transition_pgtable() function sets up transitional page tables.
> > It ensures that the relocate_kernel() function is present in the
> > identity mapping at the same location as in the kernel page tables.
> > relocate_kernel() switches to the identity mapping, and the function
> > must be present at the same location in the virtual address space before
> > and after switching page tables.
> > 
> > init_transition_pgtable() maps a copy of relocate_kernel() in
> > image->control_code_page at the relocate_kernel() virtual address, but
> > the original physical address of relocate_kernel() would also work.
> > 
> > It is safe to use original relocate_kernel() physical address cannot be
> > overwritten until swap_pages() is called, and the relocate_kernel()
> > virtual address will not be used by then.
> > 
> > Map the original relocate_kernel() at the relocate_kernel() virtual
> > address in the identity mapping. It is preparation to replace the
> > init_transition_pgtable() implementation with a call to
> > kernel_ident_mapping_init().
> > 
> > Note that while relocate_kernel() switches to the identity mapping, it
> > does not flush global TLB entries (CR4.PGE is not cleared). This means
> > that in most cases, the kernel still runs relocate_kernel() from the
> > original physical address before the change.
> > 
> > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > ---
> >  arch/x86/kernel/machine_kexec_64.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
> > index 9c9ac606893e..645690e81c2d 100644
> > --- a/arch/x86/kernel/machine_kexec_64.c
> > +++ b/arch/x86/kernel/machine_kexec_64.c
> > @@ -157,7 +157,7 @@ static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
> >  	pte_t *pte;
> >  
> >  	vaddr = (unsigned long)relocate_kernel;
> > -	paddr = __pa(page_address(image->control_code_page)+PAGE_SIZE);
> > +	paddr = __pa(relocate_kernel);
> >  	pgd += pgd_index(vaddr);
> >  	if (!pgd_present(*pgd)) {
> >  		p4d = (p4d_t *)get_zeroed_page(GFP_KERNEL);
> 
> 
> IIUC, this breaks KEXEC_JUMP (image->preserve_context is true).
> 
> The relocate_kernel() first saves couple of regs and some other data like PA
> of swap page to the control page.  Note here the VA_CONTROL_PAGE is used to
> access the control page, so those data are saved to the control page.
> 
> SYM_CODE_START_NOALIGN(relocate_kernel)
>         UNWIND_HINT_END_OF_STACK
>         ANNOTATE_NOENDBR
>         /*      
>          * %rdi indirection_page
>          * %rsi page_list
>          * %rdx start address
>          * %rcx preserve_context
>          * %r8  bare_metal
>          */
> 
> 	...
> 
>         movq    PTR(VA_CONTROL_PAGE)(%rsi), %r11                             
>         movq    %rsp, RSP(%r11)                                              
>         movq    %cr0, %rax
>         movq    %rax, CR0(%r11)
>         movq    %cr3, %rax
>         movq    %rax, CR3(%r11)
>         movq    %cr4, %rax
>         movq    %rax, CR4(%r11)
> 
> 	...
> 
> 	/*
>          * get physical address of control page now
>          * this is impossible after page table switch
>          */
>         movq    PTR(PA_CONTROL_PAGE)(%rsi), %r8
> 
>         /* get physical address of page table now too */
>         movq    PTR(PA_TABLE_PAGE)(%rsi), %r9
> 
>         /* get physical address of swap page now */
>         movq    PTR(PA_SWAP_PAGE)(%rsi), %r10
> 
>         /* save some information for jumping back */
>         movq    %r9, CP_PA_TABLE_PAGE(%r11)
>         movq    %r10, CP_PA_SWAP_PAGE(%r11)
>         movq    %rdi, CP_PA_BACKUP_PAGES_MAP(%r11)
> 
> 	...
> 
> And after jumping back from the second kernel, relocate_kernel() tries to
> restore the saved data:
> 
> 	...
> 
>         /* get the re-entry point of the peer system */
>         movq    0(%rsp), %rbp
>         leaq    relocate_kernel(%rip), %r8		<---------  (*) 
>         movq    CP_PA_SWAP_PAGE(%r8), %r10
>         movq    CP_PA_BACKUP_PAGES_MAP(%r8), %rdi
>         movq    CP_PA_TABLE_PAGE(%r8), %rax
>         movq    %rax, %cr3
>         lea     PAGE_SIZE(%r8), %rsp
>         call    swap_pages
>         movq    $virtual_mapped, %rax
>         pushq   %rax
>         ANNOTATE_UNRET_SAFE
>         ret
>         int3
> SYM_CODE_END(identity_mapped)
> 
> Note the above code (*) uses the VA of relocate_kernel() to access the control
> page.  IIUC, that means if we map VA of relocate_kernel() to the original PA
> where the code relocate_kernel() resides, then the above code will never be
> able to read those data back since they were saved to the control page.
> 
> Did I miss anything?

Note that relocate_kernel() usage at (*) is inside identity_mapped(). We
run from identity mapping there. Nothing changed to identity mapping
around relocate_kernel(), only top mapping (at __START_KERNEL_map) is
affected.

But I didn't test kexec jump thing. Do you (or anybody else) have setup to
test it?

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv3 3/4] x86/64/kexec: Map original relocate_kernel() in init_transition_pgtable()

[Huang, Kai]

On Mon, 2024-08-19 at 14:57 +0300, kirill.shutemov@linux.intel.com wrote:
> On Mon, Aug 19, 2024 at 11:16:52AM +0000, Huang, Kai wrote:
> > On Mon, 2024-08-19 at 10:08 +0300, Kirill A. Shutemov wrote:
> > > The init_transition_pgtable() function sets up transitional page tables.
> > > It ensures that the relocate_kernel() function is present in the
> > > identity mapping at the same location as in the kernel page tables.
> > > relocate_kernel() switches to the identity mapping, and the function
> > > must be present at the same location in the virtual address space before
> > > and after switching page tables.
> > > 
> > > init_transition_pgtable() maps a copy of relocate_kernel() in
> > > image->control_code_page at the relocate_kernel() virtual address, but
> > > the original physical address of relocate_kernel() would also work.
> > > 
> > > It is safe to use original relocate_kernel() physical address cannot be
> > > overwritten until swap_pages() is called, and the relocate_kernel()
> > > virtual address will not be used by then.
> > > 
> > > Map the original relocate_kernel() at the relocate_kernel() virtual
> > > address in the identity mapping. It is preparation to replace the
> > > init_transition_pgtable() implementation with a call to
> > > kernel_ident_mapping_init().
> > > 
> > > Note that while relocate_kernel() switches to the identity mapping, it
> > > does not flush global TLB entries (CR4.PGE is not cleared). This means
> > > that in most cases, the kernel still runs relocate_kernel() from the
> > > original physical address before the change.
> > > 
> > > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > > ---
> > >  arch/x86/kernel/machine_kexec_64.c | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > 
> > > diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
> > > index 9c9ac606893e..645690e81c2d 100644
> > > --- a/arch/x86/kernel/machine_kexec_64.c
> > > +++ b/arch/x86/kernel/machine_kexec_64.c
> > > @@ -157,7 +157,7 @@ static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
> > >  	pte_t *pte;
> > >  
> > >  	vaddr = (unsigned long)relocate_kernel;
> > > -	paddr = __pa(page_address(image->control_code_page)+PAGE_SIZE);
> > > +	paddr = __pa(relocate_kernel);
> > >  	pgd += pgd_index(vaddr);
> > >  	if (!pgd_present(*pgd)) {
> > >  		p4d = (p4d_t *)get_zeroed_page(GFP_KERNEL);
> > 
> > 
> > IIUC, this breaks KEXEC_JUMP (image->preserve_context is true).
> > 
> > The relocate_kernel() first saves couple of regs and some other data like PA
> > of swap page to the control page.  Note here the VA_CONTROL_PAGE is used to
> > access the control page, so those data are saved to the control page.
> > 
> > SYM_CODE_START_NOALIGN(relocate_kernel)
> >         UNWIND_HINT_END_OF_STACK
> >         ANNOTATE_NOENDBR
> >         /*      
> >          * %rdi indirection_page
> >          * %rsi page_list
> >          * %rdx start address
> >          * %rcx preserve_context
> >          * %r8  bare_metal
> >          */
> > 
> > 	...
> > 
> >         movq    PTR(VA_CONTROL_PAGE)(%rsi), %r11                             
> >         movq    %rsp, RSP(%r11)                                              
> >         movq    %cr0, %rax
> >         movq    %rax, CR0(%r11)
> >         movq    %cr3, %rax
> >         movq    %rax, CR3(%r11)
> >         movq    %cr4, %rax
> >         movq    %rax, CR4(%r11)
> > 
> > 	...
> > 
> > 	/*
> >          * get physical address of control page now
> >          * this is impossible after page table switch
> >          */
> >         movq    PTR(PA_CONTROL_PAGE)(%rsi), %r8
> > 
> >         /* get physical address of page table now too */
> >         movq    PTR(PA_TABLE_PAGE)(%rsi), %r9
> > 
> >         /* get physical address of swap page now */
> >         movq    PTR(PA_SWAP_PAGE)(%rsi), %r10
> > 
> >         /* save some information for jumping back */
> >         movq    %r9, CP_PA_TABLE_PAGE(%r11)
> >         movq    %r10, CP_PA_SWAP_PAGE(%r11)
> >         movq    %rdi, CP_PA_BACKUP_PAGES_MAP(%r11)
> > 
> > 	...
> > 
> > And after jumping back from the second kernel, relocate_kernel() tries to
> > restore the saved data:
> > 
> > 	...
> > 
> >         /* get the re-entry point of the peer system */
> >         movq    0(%rsp), %rbp
> >         leaq    relocate_kernel(%rip), %r8		<---------  (*) 
> >         movq    CP_PA_SWAP_PAGE(%r8), %r10
> >         movq    CP_PA_BACKUP_PAGES_MAP(%r8), %rdi
> >         movq    CP_PA_TABLE_PAGE(%r8), %rax
> >         movq    %rax, %cr3
> >         lea     PAGE_SIZE(%r8), %rsp
> >         call    swap_pages
> >         movq    $virtual_mapped, %rax
> >         pushq   %rax
> >         ANNOTATE_UNRET_SAFE
> >         ret
> >         int3
> > SYM_CODE_END(identity_mapped)
> > 
> > Note the above code (*) uses the VA of relocate_kernel() to access the control
> > page.  IIUC, that means if we map VA of relocate_kernel() to the original PA
> > where the code relocate_kernel() resides, then the above code will never be
> > able to read those data back since they were saved to the control page.
> > 
> > Did I miss anything?
> 
> Note that relocate_kernel() usage at (*) is inside identity_mapped(). We
> run from identity mapping there. Nothing changed to identity mapping
> around relocate_kernel(), only top mapping (at __START_KERNEL_map) is
> affected.

Yes, but before this patch the VA of relocate_kernel() is mapped to the copied
one, which resides in the control page:

        control_page = page_address(image->control_code_page) + PAGE_SIZE;
        __memcpy(control_page, relocate_kernel, KEXEC_CONTROL_CODE_MAX_SIZE);
        
        page_list[PA_CONTROL_PAGE] = virt_to_phys(control_page);
        page_list[VA_CONTROL_PAGE] = (unsigned long)control_page;    

So the (*) can actually access to the control page IIUC.

Now if we change to map VA of relocate_kernel() to the original one, then (*)
won't be able to access the control page.

> 
> But I didn't test kexec jump thing. Do you (or anybody else) have setup to
> test it?
> 

No I don't know how to test either, just my understanding on the code :-(

Git blame says Ying is the original author, so +Ying here hoping he can
provide some insight.

Anyway, my opinion is we should do patch 4 first but still map VA of
relocate_kernel() to control page so there will be no functional change.  This
patchset is about to reduce duplicated code anyway.

Re: [PATCHv3 2/4] x86/acpi: Replace manual page table initialization with kernel_ident_mapping_init()

[Rafael J. Wysocki]

On Mon, Aug 19, 2024 at 9:08 AM Kirill A. Shutemov
<kirill.shutemov@linux.intel.com> wrote:
>
> The function init_transition_pgtable() maps the page with
> asm_acpi_mp_play_dead() into an identity mapping.
>
> Replace manual page table initialization with kernel_ident_mapping_init()
> to avoid code duplication. Use x86_mapping_info::offset to get the page
> mapped at the correct location.
>
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Reviewed-by: Kai Huang <kai.huang@intel.com>
> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
> ---
>  arch/x86/kernel/acpi/madt_wakeup.c | 73 ++++++------------------------
>  1 file changed, 15 insertions(+), 58 deletions(-)
>
> diff --git a/arch/x86/kernel/acpi/madt_wakeup.c b/arch/x86/kernel/acpi/madt_wakeup.c
> index d5ef6215583b..78960b338be9 100644
> --- a/arch/x86/kernel/acpi/madt_wakeup.c
> +++ b/arch/x86/kernel/acpi/madt_wakeup.c
> @@ -70,58 +70,6 @@ static void __init free_pgt_page(void *pgt, void *dummy)
>         return memblock_free(pgt, PAGE_SIZE);
>  }
>
> -/*
> - * Make sure asm_acpi_mp_play_dead() is present in the identity mapping at
> - * the same place as in the kernel page tables. asm_acpi_mp_play_dead() switches
> - * to the identity mapping and the function has be present at the same spot in
> - * the virtual address space before and after switching page tables.
> - */
> -static int __init init_transition_pgtable(pgd_t *pgd)
> -{
> -       pgprot_t prot = PAGE_KERNEL_EXEC_NOENC;
> -       unsigned long vaddr, paddr;
> -       p4d_t *p4d;
> -       pud_t *pud;
> -       pmd_t *pmd;
> -       pte_t *pte;
> -
> -       vaddr = (unsigned long)asm_acpi_mp_play_dead;
> -       pgd += pgd_index(vaddr);
> -       if (!pgd_present(*pgd)) {
> -               p4d = (p4d_t *)alloc_pgt_page(NULL);
> -               if (!p4d)
> -                       return -ENOMEM;
> -               set_pgd(pgd, __pgd(__pa(p4d) | _KERNPG_TABLE));
> -       }
> -       p4d = p4d_offset(pgd, vaddr);
> -       if (!p4d_present(*p4d)) {
> -               pud = (pud_t *)alloc_pgt_page(NULL);
> -               if (!pud)
> -                       return -ENOMEM;
> -               set_p4d(p4d, __p4d(__pa(pud) | _KERNPG_TABLE));
> -       }
> -       pud = pud_offset(p4d, vaddr);
> -       if (!pud_present(*pud)) {
> -               pmd = (pmd_t *)alloc_pgt_page(NULL);
> -               if (!pmd)
> -                       return -ENOMEM;
> -               set_pud(pud, __pud(__pa(pmd) | _KERNPG_TABLE));
> -       }
> -       pmd = pmd_offset(pud, vaddr);
> -       if (!pmd_present(*pmd)) {
> -               pte = (pte_t *)alloc_pgt_page(NULL);
> -               if (!pte)
> -                       return -ENOMEM;
> -               set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
> -       }
> -       pte = pte_offset_kernel(pmd, vaddr);
> -
> -       paddr = __pa(vaddr);
> -       set_pte(pte, pfn_pte(paddr >> PAGE_SHIFT, prot));
> -
> -       return 0;
> -}
> -
>  static int __init acpi_mp_setup_reset(u64 reset_vector)
>  {
>         struct x86_mapping_info info = {
> @@ -130,6 +78,7 @@ static int __init acpi_mp_setup_reset(u64 reset_vector)
>                 .page_flag      = __PAGE_KERNEL_LARGE_EXEC,
>                 .kernpg_flag    = _KERNPG_TABLE_NOENC,
>         };
> +       unsigned long mstart, mend;
>         pgd_t *pgd;
>
>         pgd = alloc_pgt_page(NULL);
> @@ -137,8 +86,6 @@ static int __init acpi_mp_setup_reset(u64 reset_vector)
>                 return -ENOMEM;
>
>         for (int i = 0; i < nr_pfn_mapped; i++) {
> -               unsigned long mstart, mend;
> -
>                 mstart = pfn_mapped[i].start << PAGE_SHIFT;
>                 mend   = pfn_mapped[i].end << PAGE_SHIFT;
>                 if (kernel_ident_mapping_init(&info, pgd, mstart, mend)) {
> @@ -147,14 +94,24 @@ static int __init acpi_mp_setup_reset(u64 reset_vector)
>                 }
>         }
>
> -       if (kernel_ident_mapping_init(&info, pgd,
> -                                     PAGE_ALIGN_DOWN(reset_vector),
> -                                     PAGE_ALIGN(reset_vector + 1))) {
> +       mstart = PAGE_ALIGN_DOWN(reset_vector);
> +       mend = mstart + PAGE_SIZE;
> +       if (kernel_ident_mapping_init(&info, pgd, mstart, mend)) {
>                 kernel_ident_mapping_free(&info, pgd);
>                 return -ENOMEM;
>         }
>
> -       if (init_transition_pgtable(pgd)) {
> +       /*
> +        * Make sure asm_acpi_mp_play_dead() is present in the identity mapping
> +        * at the same place as in the kernel page tables.
> +        * asm_acpi_mp_play_dead() switches to the identity mapping and the
> +        * function has be present at the same spot in the virtual address space

s/has be/must/

Otherwise LGTM

> +        * before and after switching page tables.
> +        */
> +       info.offset = __START_KERNEL_map - phys_base;
> +       mstart = PAGE_ALIGN_DOWN(__pa(asm_acpi_mp_play_dead));
> +       mend = mstart + PAGE_SIZE;
> +       if (kernel_ident_mapping_init(&info, pgd, mstart, mend)) {
>                 kernel_ident_mapping_free(&info, pgd);
>                 return -ENOMEM;
>         }
> --
> 2.43.0
>

Re: [PATCHv3 3/4] x86/64/kexec: Map original relocate_kernel() in init_transition_pgtable()

[kirill.shutemov]

On Mon, Aug 19, 2024 at 12:39:23PM +0000, Huang, Kai wrote:
> On Mon, 2024-08-19 at 14:57 +0300, kirill.shutemov@linux.intel.com wrote:
> > On Mon, Aug 19, 2024 at 11:16:52AM +0000, Huang, Kai wrote:
> > > On Mon, 2024-08-19 at 10:08 +0300, Kirill A. Shutemov wrote:
> > > > The init_transition_pgtable() function sets up transitional page tables.
> > > > It ensures that the relocate_kernel() function is present in the
> > > > identity mapping at the same location as in the kernel page tables.
> > > > relocate_kernel() switches to the identity mapping, and the function
> > > > must be present at the same location in the virtual address space before
> > > > and after switching page tables.
> > > > 
> > > > init_transition_pgtable() maps a copy of relocate_kernel() in
> > > > image->control_code_page at the relocate_kernel() virtual address, but
> > > > the original physical address of relocate_kernel() would also work.
> > > > 
> > > > It is safe to use original relocate_kernel() physical address cannot be
> > > > overwritten until swap_pages() is called, and the relocate_kernel()
> > > > virtual address will not be used by then.
> > > > 
> > > > Map the original relocate_kernel() at the relocate_kernel() virtual
> > > > address in the identity mapping. It is preparation to replace the
> > > > init_transition_pgtable() implementation with a call to
> > > > kernel_ident_mapping_init().
> > > > 
> > > > Note that while relocate_kernel() switches to the identity mapping, it
> > > > does not flush global TLB entries (CR4.PGE is not cleared). This means
> > > > that in most cases, the kernel still runs relocate_kernel() from the
> > > > original physical address before the change.
> > > > 
> > > > Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> > > > ---
> > > >  arch/x86/kernel/machine_kexec_64.c | 2 +-
> > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > 
> > > > diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
> > > > index 9c9ac606893e..645690e81c2d 100644
> > > > --- a/arch/x86/kernel/machine_kexec_64.c
> > > > +++ b/arch/x86/kernel/machine_kexec_64.c
> > > > @@ -157,7 +157,7 @@ static int init_transition_pgtable(struct kimage *image, pgd_t *pgd)
> > > >  	pte_t *pte;
> > > >  
> > > >  	vaddr = (unsigned long)relocate_kernel;
> > > > -	paddr = __pa(page_address(image->control_code_page)+PAGE_SIZE);
> > > > +	paddr = __pa(relocate_kernel);
> > > >  	pgd += pgd_index(vaddr);
> > > >  	if (!pgd_present(*pgd)) {
> > > >  		p4d = (p4d_t *)get_zeroed_page(GFP_KERNEL);
> > > 
> > > 
> > > IIUC, this breaks KEXEC_JUMP (image->preserve_context is true).
> > > 
> > > The relocate_kernel() first saves couple of regs and some other data like PA
> > > of swap page to the control page.  Note here the VA_CONTROL_PAGE is used to
> > > access the control page, so those data are saved to the control page.
> > > 
> > > SYM_CODE_START_NOALIGN(relocate_kernel)
> > >         UNWIND_HINT_END_OF_STACK
> > >         ANNOTATE_NOENDBR
> > >         /*      
> > >          * %rdi indirection_page
> > >          * %rsi page_list
> > >          * %rdx start address
> > >          * %rcx preserve_context
> > >          * %r8  bare_metal
> > >          */
> > > 
> > > 	...
> > > 
> > >         movq    PTR(VA_CONTROL_PAGE)(%rsi), %r11                             
> > >         movq    %rsp, RSP(%r11)                                              
> > >         movq    %cr0, %rax
> > >         movq    %rax, CR0(%r11)
> > >         movq    %cr3, %rax
> > >         movq    %rax, CR3(%r11)
> > >         movq    %cr4, %rax
> > >         movq    %rax, CR4(%r11)
> > > 
> > > 	...
> > > 
> > > 	/*
> > >          * get physical address of control page now
> > >          * this is impossible after page table switch
> > >          */
> > >         movq    PTR(PA_CONTROL_PAGE)(%rsi), %r8
> > > 
> > >         /* get physical address of page table now too */
> > >         movq    PTR(PA_TABLE_PAGE)(%rsi), %r9
> > > 
> > >         /* get physical address of swap page now */
> > >         movq    PTR(PA_SWAP_PAGE)(%rsi), %r10
> > > 
> > >         /* save some information for jumping back */
> > >         movq    %r9, CP_PA_TABLE_PAGE(%r11)
> > >         movq    %r10, CP_PA_SWAP_PAGE(%r11)
> > >         movq    %rdi, CP_PA_BACKUP_PAGES_MAP(%r11)
> > > 
> > > 	...
> > > 
> > > And after jumping back from the second kernel, relocate_kernel() tries to
> > > restore the saved data:
> > > 
> > > 	...
> > > 
> > >         /* get the re-entry point of the peer system */
> > >         movq    0(%rsp), %rbp
> > >         leaq    relocate_kernel(%rip), %r8		<---------  (*) 
> > >         movq    CP_PA_SWAP_PAGE(%r8), %r10
> > >         movq    CP_PA_BACKUP_PAGES_MAP(%r8), %rdi
> > >         movq    CP_PA_TABLE_PAGE(%r8), %rax
> > >         movq    %rax, %cr3
> > >         lea     PAGE_SIZE(%r8), %rsp
> > >         call    swap_pages
> > >         movq    $virtual_mapped, %rax
> > >         pushq   %rax
> > >         ANNOTATE_UNRET_SAFE
> > >         ret
> > >         int3
> > > SYM_CODE_END(identity_mapped)
> > > 
> > > Note the above code (*) uses the VA of relocate_kernel() to access the control
> > > page.  IIUC, that means if we map VA of relocate_kernel() to the original PA
> > > where the code relocate_kernel() resides, then the above code will never be
> > > able to read those data back since they were saved to the control page.
> > > 
> > > Did I miss anything?
> > 
> > Note that relocate_kernel() usage at (*) is inside identity_mapped(). We
> > run from identity mapping there. Nothing changed to identity mapping
> > around relocate_kernel(), only top mapping (at __START_KERNEL_map) is
> > affected.
> 
> Yes, but before this patch the VA of relocate_kernel() is mapped to the copied
> one, which resides in the control page:
> 
>         control_page = page_address(image->control_code_page) + PAGE_SIZE;
>         __memcpy(control_page, relocate_kernel, KEXEC_CONTROL_CODE_MAX_SIZE);
>         
>         page_list[PA_CONTROL_PAGE] = virt_to_phys(control_page);
>         page_list[VA_CONTROL_PAGE] = (unsigned long)control_page;    
> 
> So the (*) can actually access to the control page IIUC.
> 
> Now if we change to map VA of relocate_kernel() to the original one, then (*)
> won't be able to access the control page.

No, it still will be able to access control page.

So we call relocate_kernel() in normal kernel text (within
__START_KERNEL_map).

relocate_kernel() switches to identity mapping, VA is still the same.

relocate_kernel() jumps to identity_mapped() in the control page:


	/*
	 * get physical address of control page now
	 * this is impossible after page table switch
	 */
	movq	PTR(PA_CONTROL_PAGE)(%rsi), %r8

	...

	/* jump to identity mapped page */
	addq	$(identity_mapped - relocate_kernel), %r8
	pushq	%r8
	ANNOTATE_UNRET_SAFE
	ret

The ADDQ finds offset of identity_mapped() in the control page.

identity_mapping() finds start of the control page from *relative*
position of relocate_page() to the current RIP in the control page:

	leaq	relocate_kernel(%rip), %r8

It looks like this in my kernel binary:

	lea    -0xfa(%rip),%r8

What PA is mapped at the normal kernel text VA of relocate_kernel() makes
zero affect to the calculation.

Does it make sense?

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv3 3/4] x86/64/kexec: Map original relocate_kernel() in init_transition_pgtable()

[Huang, Kai]

> > 
> > So the (*) can actually access to the control page IIUC.
> > 
> > Now if we change to map VA of relocate_kernel() to the original one, then (*)
> > won't be able to access the control page.
> 
> No, it still will be able to access control page.
> 
> So we call relocate_kernel() in normal kernel text (within
> __START_KERNEL_map).
> 
> relocate_kernel() switches to identity mapping, VA is still the same.
> 
> relocate_kernel() jumps to identity_mapped() in the control page:
> 
> 
> 	/*
> 	 * get physical address of control page now
> 	 * this is impossible after page table switch
> 	 */
> 	movq	PTR(PA_CONTROL_PAGE)(%rsi), %r8
> 
> 	...
> 
> 	/* jump to identity mapped page */
> 	addq	$(identity_mapped - relocate_kernel), %r8
> 	pushq	%r8
> 	ANNOTATE_UNRET_SAFE
> 	ret
> 
> The ADDQ finds offset of identity_mapped() in the control page.
> 
> identity_mapping() finds start of the control page from *relative*
> position of relocate_page() to the current RIP in the control page:
> 
> 	leaq	relocate_kernel(%rip), %r8
> 
> It looks like this in my kernel binary:
> 
> 	lea    -0xfa(%rip),%r8

Ah I see.  I missed the *relative* addressing. :-)

> 
> What PA is mapped at the normal kernel text VA of relocate_kernel() makes
> zero affect to the calculation.

Yeah.

> 
> Does it make sense?
> 

Yes.  Thanks for explanation.

At later time:

        call    swap_pages                                                   
        movq    $virtual_mapped, %rax  	<---- (1)                            
        pushq   %rax
        ANNOTATE_UNRET_SAFE
        ret  				<---- (2)

(1) will load the VA which has __START_KERNEL_map to %rax, and after (2) the
kernel will run at VA of the original relocate_kernel() which maps to the PA
of the original relcoate_kernel().  But I think the memory page of the
original relocate_kernel() won't get corrupted after returning from the second
kernel, so should be safe to use?

Re: [PATCHv3 3/4] x86/64/kexec: Map original relocate_kernel() in init_transition_pgtable()

[kirill.shutemov]

On Tue, Aug 20, 2024 at 11:06:34AM +0000, Huang, Kai wrote:
> At later time:
> 
>         call    swap_pages                                                   
>         movq    $virtual_mapped, %rax  	<---- (1)                            
>         pushq   %rax
>         ANNOTATE_UNRET_SAFE
>         ret  				<---- (2)
> 
> (1) will load the VA which has __START_KERNEL_map to %rax, and after (2) the
> kernel will run at VA of the original relocate_kernel() which maps to the PA
> of the original relcoate_kernel().  But I think the memory page of the
> original relocate_kernel() won't get corrupted after returning from the second
> kernel, so should be safe to use?

Yes.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCHv3 3/4] x86/64/kexec: Map original relocate_kernel() in init_transition_pgtable()

[Huang, Kai]

On Tue, 2024-08-20 at 14:14 +0300, kirill.shutemov@linux.intel.com wrote:
> On Tue, Aug 20, 2024 at 11:06:34AM +0000, Huang, Kai wrote:
> > At later time:
> > 
> >         call    swap_pages                                                   
> >         movq    $virtual_mapped, %rax  	<---- (1)                            
> >         pushq   %rax
> >         ANNOTATE_UNRET_SAFE
> >         ret  				<---- (2)
> > 
> > (1) will load the VA which has __START_KERNEL_map to %rax, and after (2) the
> > kernel will run at VA of the original relocate_kernel() which maps to the PA
> > of the original relcoate_kernel().  But I think the memory page of the
> > original relocate_kernel() won't get corrupted after returning from the second
> > kernel, so should be safe to use?
> 
> Yes.
> 

Reviewed-by: Kai Huang <kai.huang@intel.com>

Re: [PATCHv3 4/4] x86/64/kexec: Rewrite init_transition_pgtable() with kernel_ident_mapping_init()

[Huang, Kai]

On Mon, 2024-08-19 at 10:08 +0300, Kirill A. Shutemov wrote:
> init_transition_pgtable() sets up transitional page tables. Rewrite it
> using kernel_ident_mapping_init() to avoid code duplication.
> 
> Change struct kimage_arch to track allocated page tables as a list, not
> linking them to specific page table levels.
> 
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
> 

Reviewed-by: Kai Huang <kai.huang@intel.com>