[PATCH 0/2] Map initrd as encrypted when relocating if SME is enabled

[PATCH 0/2] Map initrd as encrypted when relocating if SME is enabled

[Zelin Deng]

I found an issue on SME enabled AMD machine when initrd is relocated if
it was located in e820 reserved area.
For example key dmesg output:
...
[mem 0x000000005aafe000-0x000000006005ffff] reserved //e820 mapping
Move RAMDISK from [mem 0x5aafe000-0x5ccd5167] //relocate_initrd()
...

Early initrd will be copied by copy_from_early_mem() which will clear
encrypted pgprot flag as initrd source address is not in kernel usable
area. As initrd has been encrypted at earlier stage, encrypted data is
copied, which leads new initrd cannot be unpacked, then rootfs cannot be
mounted.
dmesg output:
...
[   11.296725] Trying to unpack rootfs image as initramfs...
[   11.302127] Initramfs unpacking failed: invalid magic at start of compressed archive
...
[   16.698152] /dev/root: Can't open blockdev
[   16.702255] VFS: Cannot open root device "PARTUUID=0ad58d87-05c7-43f8-b147-93140ad315e5" or unknown-block(0,0): error -6
[   16.713114] Please append a correct "root=" boot option; here are the available partitions:
[   16.721462] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[   16.729716] CPU: 9 PID: 1 Comm: swapper/0 Not tainted 6.1.0-rc5-next-20221114 #3
[   16.737099] Hardware name: AMD Corporation DAYTONA_X/DAYTONA_X, BIOS RYM1008B 01/19/2022
[   16.745175] Call Trace:
[   16.747623]  <TASK>
[   16.749727]  dump_stack_lvl+0x38/0x4c
[   16.753393]  panic+0xfb/0x28a
[   16.771999]  ? _printk+0x4c/0x52
[   16.775224]  mount_block_root+0x143/0x1dd
[   16.779237]  prepare_namespace+0x13f/0x16e
[   16.783334]  kernel_init_freeable+0x15a/0x164
[   16.787687]  ? __pfx_kernel_init+0x10/0x10
[   16.791785]  kernel_init+0x1a/0x130
[   16.795268]  ret_from_fork+0x29/0x50
[   16.798840]  </TASK>

To fix this issue, early initrd must be mapped as encrypted when it is
being relocated.

Zelin Deng (2):
  mm/early_ioremap.c: Always build early_memremap_prot() in x86
  x86/setup: Preserve _ENC flag when initrd is being relocated

 arch/x86/Kconfig                    |  1 +
 arch/x86/kernel/setup.c             | 30 ++++++++++++++++++++++++++++-
 include/asm-generic/early_ioremap.h |  6 ------
 mm/early_ioremap.c                  | 21 --------------------
 4 files changed, 30 insertions(+), 28 deletions(-)

-- 
2.27.0

[PATCH 1/2] mm/early_ioremap.c: Always build early_memremap_prot() in x86

[Zelin Deng]

In some scenarios, we'd want to specify protection attributes when we
are doing early memory map. As early_memremap_prot() is also defined in
loongarch it's better to select ARCH_USE_MEMREMAP_PROT on X86 config to
avoid redefined error.

Signed-off-by: Zelin Deng <zelin.deng@linux.alibaba.com>
---
 arch/x86/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 48d98a5fe06b..133d7d660b2e 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -97,6 +97,7 @@ config X86
 	select ARCH_HAS_SYNC_CORE_BEFORE_USERMODE
 	select ARCH_HAS_SYSCALL_WRAPPER
 	select ARCH_HAS_UBSAN_SANITIZE_ALL
+	select ARCH_USE_MEMREMAP_PROT
 	select ARCH_HAS_DEBUG_WX
 	select ARCH_HAS_ZONE_DMA_SET if EXPERT
 	select ARCH_HAVE_NMI_SAFE_CMPXCHG
-- 
2.27.0

[PATCH 2/2] x86/setup: Preserve _ENC flag when initrd is being relocated

[Zelin Deng]

Commit 107cd2532181 ("Encrypt the initrd earlier for BSP microcode update")
when SME is enabled, initrd will be encrypted at earlier stage. If
initrd is located at e820 reserved area the initrd will be copied to
direct mapping area in relocate_initrd().

In this case source address of initrd should be mapped as encrypted
while copy_from_early_mem() will clear encrypted attribute as the source
address is not in kernel usable area, therefore relocated initrd is
encrypted data and is not able to be unpacked later.

Add new function copy_early_initrd() to preserve _ENC flag in setup.c
and remove copy_from_early_mem() as it's only used once here by x86.

Signed-off-by: Zelin Deng <zelin.deng@linux.alibaba.com>
---
 arch/x86/kernel/setup.c             | 30 ++++++++++++++++++++++++++++-
 include/asm-generic/early_ioremap.h |  6 ------
 mm/early_ioremap.c                  | 21 --------------------
 3 files changed, 29 insertions(+), 28 deletions(-)

diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
index 56deaf37e508..f9996982f026 100644
--- a/arch/x86/kernel/setup.c
+++ b/arch/x86/kernel/setup.c
@@ -260,6 +260,34 @@ static u64 __init get_ramdisk_size(void)
 	return ramdisk_size;
 }
 
+#define MAX_MAP_CHUNK	(NR_FIX_BTMAPS << PAGE_SHIFT)
+
+static void __init copy_early_initrd(void *dest, phys_addr_t src,
+				     unsigned long size)
+{
+	unsigned long slop, clen;
+	char *p;
+
+	while (size) {
+		slop = offset_in_page(src);
+		clen = size;
+		if (clen > MAX_MAP_CHUNK - slop)
+			clen = MAX_MAP_CHUNK - slop;
+		/*
+		 * _ENC flag should be preserved so that when SME is enabled initrd
+		 * can be mapped as encrypted, as it had been encrypted earlier.
+		 * This flag won't impact on other platforms like TDX/SEV enabled.
+		 */
+		p = early_memremap_prot(src & PAGE_MASK, clen + slop,
+					pgprot_val(FIXMAP_PAGE_NORMAL));
+		memcpy(dest, p + slop, clen);
+		early_memunmap(p, clen + slop);
+		dest += clen;
+		src += clen;
+		size -= clen;
+	}
+}
+
 static void __init relocate_initrd(void)
 {
 	/* Assume only end is not page aligned */
@@ -279,7 +307,7 @@ static void __init relocate_initrd(void)
 	printk(KERN_INFO "Allocated new RAMDISK: [mem %#010llx-%#010llx]\n",
 	       relocated_ramdisk, relocated_ramdisk + ramdisk_size - 1);
 
-	copy_from_early_mem((void *)initrd_start, ramdisk_image, ramdisk_size);
+	copy_early_initrd((void *)initrd_start, ramdisk_image, ramdisk_size);
 
 	printk(KERN_INFO "Move RAMDISK from [mem %#010llx-%#010llx] to"
 		" [mem %#010llx-%#010llx]\n",
diff --git a/include/asm-generic/early_ioremap.h b/include/asm-generic/early_ioremap.h
index 9d0479f50f97..be1ce406f481 100644
--- a/include/asm-generic/early_ioremap.h
+++ b/include/asm-generic/early_ioremap.h
@@ -32,12 +32,6 @@ extern void early_ioremap_setup(void);
  */
 extern void early_ioremap_reset(void);
 
-/*
- * Early copy from unmapped memory to kernel mapped memory.
- */
-extern void copy_from_early_mem(void *dest, phys_addr_t src,
-				unsigned long size);
-
 #else
 static inline void early_ioremap_init(void) { }
 static inline void early_ioremap_setup(void) { }
diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c
index 9bc12e526ed0..86b68d63ad35 100644
--- a/mm/early_ioremap.c
+++ b/mm/early_ioremap.c
@@ -245,27 +245,6 @@ early_memremap_prot(resource_size_t phys_addr, unsigned long size,
 }
 #endif
 
-#define MAX_MAP_CHUNK	(NR_FIX_BTMAPS << PAGE_SHIFT)
-
-void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size)
-{
-	unsigned long slop, clen;
-	char *p;
-
-	while (size) {
-		slop = offset_in_page(src);
-		clen = size;
-		if (clen > MAX_MAP_CHUNK - slop)
-			clen = MAX_MAP_CHUNK - slop;
-		p = early_memremap(src & PAGE_MASK, clen + slop);
-		memcpy(dest, p + slop, clen);
-		early_memunmap(p, clen + slop);
-		dest += clen;
-		src += clen;
-		size -= clen;
-	}
-}
-
 #else /* CONFIG_MMU */
 
 void __init __iomem *
-- 
2.27.0

Re: [PATCH 2/2] x86/setup: Preserve _ENC flag when initrd is being relocated

[Zelin Deng]

在 2022/11/24 17:12, Zelin Deng 写道:
> Commit 107cd2532181 ("Encrypt the initrd earlier for BSP microcode update")
> when SME is enabled, initrd will be encrypted at earlier stage. If
> initrd is located at e820 reserved area the initrd will be copied to
> direct mapping area in relocate_initrd().
>
> In this case source address of initrd should be mapped as encrypted
> while copy_from_early_mem() will clear encrypted attribute as the source
> address is not in kernel usable area, therefore relocated initrd is
> encrypted data and is not able to be unpacked later.
>
> Add new function copy_early_initrd() to preserve _ENC flag in setup.c
> and remove copy_from_early_mem() as it's only used once here by x86.
>
> Signed-off-by: Zelin Deng <zelin.deng@linux.alibaba.com>
> ---
>   arch/x86/kernel/setup.c             | 30 ++++++++++++++++++++++++++++-
>   include/asm-generic/early_ioremap.h |  6 ------
>   mm/early_ioremap.c                  | 21 --------------------
>   3 files changed, 29 insertions(+), 28 deletions(-)
>
> diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c
> index 56deaf37e508..f9996982f026 100644
> --- a/arch/x86/kernel/setup.c
> +++ b/arch/x86/kernel/setup.c
> @@ -260,6 +260,34 @@ static u64 __init get_ramdisk_size(void)
>   	return ramdisk_size;
>   }
>   
> +#define MAX_MAP_CHUNK	(NR_FIX_BTMAPS << PAGE_SHIFT)
> +
> +static void __init copy_early_initrd(void *dest, phys_addr_t src,
> +				     unsigned long size)
> +{
> +	unsigned long slop, clen;
> +	char *p;
> +
> +	while (size) {
> +		slop = offset_in_page(src);
> +		clen = size;
> +		if (clen > MAX_MAP_CHUNK - slop)
> +			clen = MAX_MAP_CHUNK - slop;
> +		/*
> +		 * _ENC flag should be preserved so that when SME is enabled initrd
> +		 * can be mapped as encrypted, as it had been encrypted earlier.
> +		 * This flag won't impact on other platforms like TDX/SEV enabled.
> +		 */
> +		p = early_memremap_prot(src & PAGE_MASK, clen + slop,
> +					pgprot_val(FIXMAP_PAGE_NORMAL));

Sorry my mistake, perhaps cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) 
should be added here to determine if we are in guest to adjust the pgprot.

The scenarios in which initrd should not be encrypted are "SME not 
enabled" and in tdx host? Looks like _ENC flag is not enough.

> +		memcpy(dest, p + slop, clen);
> +		early_memunmap(p, clen + slop);
> +		dest += clen;
> +		src += clen;
> +		size -= clen;
> +	}
> +}
> +
>   static void __init relocate_initrd(void)
>   {
>   	/* Assume only end is not page aligned */
> @@ -279,7 +307,7 @@ static void __init relocate_initrd(void)
>   	printk(KERN_INFO "Allocated new RAMDISK: [mem %#010llx-%#010llx]\n",
>   	       relocated_ramdisk, relocated_ramdisk + ramdisk_size - 1);
>   
> -	copy_from_early_mem((void *)initrd_start, ramdisk_image, ramdisk_size);
> +	copy_early_initrd((void *)initrd_start, ramdisk_image, ramdisk_size);
>   
>   	printk(KERN_INFO "Move RAMDISK from [mem %#010llx-%#010llx] to"
>   		" [mem %#010llx-%#010llx]\n",
> diff --git a/include/asm-generic/early_ioremap.h b/include/asm-generic/early_ioremap.h
> index 9d0479f50f97..be1ce406f481 100644
> --- a/include/asm-generic/early_ioremap.h
> +++ b/include/asm-generic/early_ioremap.h
> @@ -32,12 +32,6 @@ extern void early_ioremap_setup(void);
>    */
>   extern void early_ioremap_reset(void);
>   
> -/*
> - * Early copy from unmapped memory to kernel mapped memory.
> - */
> -extern void copy_from_early_mem(void *dest, phys_addr_t src,
> -				unsigned long size);
> -
>   #else
>   static inline void early_ioremap_init(void) { }
>   static inline void early_ioremap_setup(void) { }
> diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c
> index 9bc12e526ed0..86b68d63ad35 100644
> --- a/mm/early_ioremap.c
> +++ b/mm/early_ioremap.c
> @@ -245,27 +245,6 @@ early_memremap_prot(resource_size_t phys_addr, unsigned long size,
>   }
>   #endif
>   
> -#define MAX_MAP_CHUNK	(NR_FIX_BTMAPS << PAGE_SHIFT)
> -
> -void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size)
> -{
> -	unsigned long slop, clen;
> -	char *p;
> -
> -	while (size) {
> -		slop = offset_in_page(src);
> -		clen = size;
> -		if (clen > MAX_MAP_CHUNK - slop)
> -			clen = MAX_MAP_CHUNK - slop;
> -		p = early_memremap(src & PAGE_MASK, clen + slop);
> -		memcpy(dest, p + slop, clen);
> -		early_memunmap(p, clen + slop);
> -		dest += clen;
> -		src += clen;
> -		size -= clen;
> -	}
> -}
> -
>   #else /* CONFIG_MMU */
>   
>   void __init __iomem *

Re: [PATCH 0/2] Map initrd as encrypted when relocating if SME is enabled

[Tom Lendacky]

On 11/24/22 03:12, Zelin Deng wrote:
> I found an issue on SME enabled AMD machine when initrd is relocated if
> it was located in e820 reserved area.
> For example key dmesg output:
> ...
> [mem 0x000000005aafe000-0x000000006005ffff] reserved //e820 mapping
> Move RAMDISK from [mem 0x5aafe000-0x5ccd5167] //relocate_initrd()
> ...
> 
> Early initrd will be copied by copy_from_early_mem() which will clear
> encrypted pgprot flag as initrd source address is not in kernel usable
> area. As initrd has been encrypted at earlier stage, encrypted data is
> copied, which leads new initrd cannot be unpacked, then rootfs cannot be
> mounted.

This is actually a bug in Grub where the kernel and initrd was moved out 
of loader_code/data and into runtime_service_code/data. This commit has 
since been reverted because it goes against the UEFI specification. It was 
a small window, but the bad version was picked up by some distros. No need 
for a kernel change, please update your version of Grub.

Thanks,
Tom

> dmesg output:
> ...
> [   11.296725] Trying to unpack rootfs image as initramfs...
> [   11.302127] Initramfs unpacking failed: invalid magic at start of compressed archive
> ...
> [   16.698152] /dev/root: Can't open blockdev
> [   16.702255] VFS: Cannot open root device "PARTUUID=0ad58d87-05c7-43f8-b147-93140ad315e5" or unknown-block(0,0): error -6
> [   16.713114] Please append a correct "root=" boot option; here are the available partitions:
> [   16.721462] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
> [   16.729716] CPU: 9 PID: 1 Comm: swapper/0 Not tainted 6.1.0-rc5-next-20221114 #3
> [   16.737099] Hardware name: AMD Corporation DAYTONA_X/DAYTONA_X, BIOS RYM1008B 01/19/2022
> [   16.745175] Call Trace:
> [   16.747623]  <TASK>
> [   16.749727]  dump_stack_lvl+0x38/0x4c
> [   16.753393]  panic+0xfb/0x28a
> [   16.771999]  ? _printk+0x4c/0x52
> [   16.775224]  mount_block_root+0x143/0x1dd
> [   16.779237]  prepare_namespace+0x13f/0x16e
> [   16.783334]  kernel_init_freeable+0x15a/0x164
> [   16.787687]  ? __pfx_kernel_init+0x10/0x10
> [   16.791785]  kernel_init+0x1a/0x130
> [   16.795268]  ret_from_fork+0x29/0x50
> [   16.798840]  </TASK>
> 
> To fix this issue, early initrd must be mapped as encrypted when it is
> being relocated.
> 
> Zelin Deng (2):
>    mm/early_ioremap.c: Always build early_memremap_prot() in x86
>    x86/setup: Preserve _ENC flag when initrd is being relocated
> 
>   arch/x86/Kconfig                    |  1 +
>   arch/x86/kernel/setup.c             | 30 ++++++++++++++++++++++++++++-
>   include/asm-generic/early_ioremap.h |  6 ------
>   mm/early_ioremap.c                  | 21 --------------------
>   4 files changed, 30 insertions(+), 28 deletions(-)
> 

Re: [PATCH 0/2] Map initrd as encrypted when relocating if SME is enabled

[Zelin Deng]

在 2022/11/24 22:26, Tom Lendacky 写道:
> On 11/24/22 03:12, Zelin Deng wrote:
>> I found an issue on SME enabled AMD machine when initrd is relocated if
>> it was located in e820 reserved area.
>> For example key dmesg output:
>> ...
>> [mem 0x000000005aafe000-0x000000006005ffff] reserved //e820 mapping
>> Move RAMDISK from [mem 0x5aafe000-0x5ccd5167] //relocate_initrd()
>> ...
>>
>> Early initrd will be copied by copy_from_early_mem() which will clear
>> encrypted pgprot flag as initrd source address is not in kernel usable
>> area. As initrd has been encrypted at earlier stage, encrypted data is
>> copied, which leads new initrd cannot be unpacked, then rootfs cannot be
>> mounted.
>
> This is actually a bug in Grub where the kernel and initrd was moved 
> out of loader_code/data and into runtime_service_code/data. This 
> commit has since been reverted because it goes against the UEFI 
> specification. It was a small window, but the bad version was picked 
> up by some distros. No need for a kernel change, please update your 
> version of Grub.
>
> Thanks,
> Tom
>
Hi Tom,

Thank you for clarification, I will update my grub and try again.

Thanks,

Zelin Deng

>> dmesg output:
>> ...
>> [   11.296725] Trying to unpack rootfs image as initramfs...
>> [   11.302127] Initramfs unpacking failed: invalid magic at start of 
>> compressed archive
>> ...
>> [   16.698152] /dev/root: Can't open blockdev
>> [   16.702255] VFS: Cannot open root device 
>> "PARTUUID=0ad58d87-05c7-43f8-b147-93140ad315e5" or 
>> unknown-block(0,0): error -6
>> [   16.713114] Please append a correct "root=" boot option; here are 
>> the available partitions:
>> [   16.721462] Kernel panic - not syncing: VFS: Unable to mount root 
>> fs on unknown-block(0,0)
>> [   16.729716] CPU: 9 PID: 1 Comm: swapper/0 Not tainted 
>> 6.1.0-rc5-next-20221114 #3
>> [   16.737099] Hardware name: AMD Corporation DAYTONA_X/DAYTONA_X, 
>> BIOS RYM1008B 01/19/2022
>> [   16.745175] Call Trace:
>> [   16.747623]  <TASK>
>> [   16.749727]  dump_stack_lvl+0x38/0x4c
>> [   16.753393]  panic+0xfb/0x28a
>> [   16.771999]  ? _printk+0x4c/0x52
>> [   16.775224]  mount_block_root+0x143/0x1dd
>> [   16.779237]  prepare_namespace+0x13f/0x16e
>> [   16.783334]  kernel_init_freeable+0x15a/0x164
>> [   16.787687]  ? __pfx_kernel_init+0x10/0x10
>> [   16.791785]  kernel_init+0x1a/0x130
>> [   16.795268]  ret_from_fork+0x29/0x50
>> [   16.798840]  </TASK>
>>
>> To fix this issue, early initrd must be mapped as encrypted when it is
>> being relocated.
>>
>> Zelin Deng (2):
>>    mm/early_ioremap.c: Always build early_memremap_prot() in x86
>>    x86/setup: Preserve _ENC flag when initrd is being relocated
>>
>>   arch/x86/Kconfig                    |  1 +
>>   arch/x86/kernel/setup.c             | 30 ++++++++++++++++++++++++++++-
>>   include/asm-generic/early_ioremap.h |  6 ------
>>   mm/early_ioremap.c                  | 21 --------------------
>>   4 files changed, 30 insertions(+), 28 deletions(-)
>>