[PATCH v2 1/2] x86/build: only align ENTRY_TEXT to PMD_SIZE if necessary

[PATCH v2 1/2] x86/build: only align ENTRY_TEXT to PMD_SIZE if necessary

[Hamza Mahfooz]

PTI requires the begin and end of ENTRY_TEXT be aligned to PMD_SIZE.
SRSO requires srso_alias_untrain_ret to be 2M aligned. This costs
between 2-4 MiB of RAM (depending on the size of the preceding section).
So, only align when PTI is enabled or SRSO is enabled.

Co-developed-by: Jared White <jaredwhite@microsoft.com>
Signed-off-by: Jared White <jaredwhite@microsoft.com>
Signed-off-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
---
 arch/x86/kernel/vmlinux.lds.S | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 4711a35e706c..53519fa7c772 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -65,8 +65,14 @@ const_cpu_current_top_of_stack = cpu_current_top_of_stack;
 		__end_rodata_hpage_align = .;			\
 		__end_rodata_aligned = .;
 
+#if defined(CONFIG_MITIGATION_PAGE_TABLE_ISOLATION) || defined(CONFIG_MITIGATION_SRSO)
 #define ALIGN_ENTRY_TEXT_BEGIN	. = ALIGN(PMD_SIZE);
 #define ALIGN_ENTRY_TEXT_END	. = ALIGN(PMD_SIZE);
+#else
+#define ALIGN_ENTRY_TEXT_BEGIN
+#define ALIGN_ENTRY_TEXT_END
+#endif
+
 #else
 
 #define X86_ALIGN_RODATA_BEGIN
-- 
2.54.0

[PATCH v2 2/2] x86/mm/64: free the gap between BSS_MAIN and BSS_DECRYPTED

[Hamza Mahfooz]

This region is unused after mark_rodata_ro() and takes up, up to 2 MiB
of memory. So, free it in mark_rodata_ro().

Co-developed-by: Jared White <jaredwhite@microsoft.com>
Signed-off-by: Jared White <jaredwhite@microsoft.com>
Signed-off-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
---
 arch/x86/include/asm/sections.h | 1 +
 arch/x86/kernel/vmlinux.lds.S   | 2 ++
 arch/x86/mm/init_64.c           | 4 ++++
 3 files changed, 7 insertions(+)

diff --git a/arch/x86/include/asm/sections.h b/arch/x86/include/asm/sections.h
index 30e8ee7006f9..1d6589bc0f68 100644
--- a/arch/x86/include/asm/sections.h
+++ b/arch/x86/include/asm/sections.h
@@ -8,6 +8,7 @@
 extern char __relocate_kernel_start[], __relocate_kernel_end[];
 extern char __brk_base[], __brk_limit[];
 extern char __end_rodata_aligned[];
+extern char __start_bss_decrypted_gap[];
 
 #if defined(CONFIG_X86_64)
 extern char __end_rodata_hpage_align[];
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 53519fa7c772..de9901bda7d0 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -94,6 +94,8 @@ const_cpu_current_top_of_stack = cpu_current_top_of_stack;
  * decrypted to avoid exposing more than we wish.
  */
 #define BSS_DECRYPTED						\
+	. = ALIGN(PAGE_SIZE);					\
+	__start_bss_decrypted_gap = .;				\
 	. = ALIGN(PMD_SIZE);					\
 	__start_bss_decrypted = .;				\
 	__pi___start_bss_decrypted = .;				\
diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c
index df2261fa4f98..3d90bd593637 100644
--- a/arch/x86/mm/init_64.c
+++ b/arch/x86/mm/init_64.c
@@ -1396,6 +1396,8 @@ void mark_rodata_ro(void)
 	unsigned long end = (unsigned long)__end_rodata_hpage_align;
 	unsigned long text_end = PFN_ALIGN(_etext);
 	unsigned long rodata_end = PFN_ALIGN(__end_rodata);
+	unsigned long bss_decrypted_gap_start = PFN_ALIGN(__start_bss_decrypted_gap);
+	unsigned long bss_decrypted_gap_stop = PFN_ALIGN(__start_bss_decrypted);
 	unsigned long all_end;
 
 	printk(KERN_INFO "Write protecting the kernel read-only data: %luk\n",
@@ -1433,6 +1435,8 @@ void mark_rodata_ro(void)
 				(void *)text_end, (void *)rodata_start);
 	free_kernel_image_pages("unused kernel image (rodata/data gap)",
 				(void *)rodata_end, (void *)_sdata);
+	free_kernel_image_pages("unused kernel image (bss_decrypted gap)",
+				(void *)bss_decrypted_gap_start, (void *)bss_decrypted_gap_stop);
 }
 
 /*
-- 
2.54.0

Re: [PATCH v2 2/2] x86/mm/64: free the gap between BSS_MAIN and BSS_DECRYPTED

[Borislav Petkov]

On Tue, Jun 23, 2026 at 11:24:30AM -0400, Hamza Mahfooz wrote:
> This region is unused after mark_rodata_ro() and takes up, up to 2 MiB
> of memory. So, free it in mark_rodata_ro().
> 
> Co-developed-by: Jared White <jaredwhite@microsoft.com>
> Signed-off-by: Jared White <jaredwhite@microsoft.com>
> Signed-off-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
> ---
>  arch/x86/include/asm/sections.h | 1 +
>  arch/x86/kernel/vmlinux.lds.S   | 2 ++
>  arch/x86/mm/init_64.c           | 4 ++++
>  3 files changed, 7 insertions(+)

Sashiko has comments:

https://sashiko.dev/#/patchset/20260623152430.1473481-1-hamzamahfooz%40linux.microsoft.com

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

[tip: x86/cleanups] x86/build: Only align ENTRY_TEXT to PMD_SIZE if necessary

[tip-bot2 for Hamza Mahfooz]

The following commit has been merged into the x86/cleanups branch of tip:

Commit-ID:     fa84683b892e2590401e655bebe6ba461367adc7
Gitweb:        https://git.kernel.org/tip/fa84683b892e2590401e655bebe6ba461367adc7
Author:        Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
AuthorDate:    Tue, 23 Jun 2026 11:24:29 -04:00
Committer:     Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Thu, 09 Jul 2026 15:34:23 -07:00

x86/build: Only align ENTRY_TEXT to PMD_SIZE if necessary

PTI requires the begin and end of ENTRY_TEXT be aligned to PMD_SIZE.  SRSO
requires srso_alias_untrain_ret to be 2M aligned. This costs between 2-4 MiB
of RAM (depending on the size of the preceding section).  So, only align when
either of the two are enabled.

Co-developed-by: Jared White <jaredwhite@microsoft.com>
Signed-off-by: Jared White <jaredwhite@microsoft.com>
Signed-off-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://patch.msgid.link/20260623152430.1473481-1-hamzamahfooz@linux.microsoft.com
---
 arch/x86/kernel/vmlinux.lds.S | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 74e336d..10caf92 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -65,9 +65,15 @@ const_cpu_current_top_of_stack = cpu_current_top_of_stack;
 		__end_rodata_hpage_align = .;			\
 		__end_rodata_aligned = .;
 
+#if defined(CONFIG_MITIGATION_PAGE_TABLE_ISOLATION) || defined(CONFIG_MITIGATION_SRSO)
 #define ALIGN_ENTRY_TEXT_BEGIN	. = ALIGN(PMD_SIZE);
 #define ALIGN_ENTRY_TEXT_END	. = ALIGN(PMD_SIZE);
 #else
+#define ALIGN_ENTRY_TEXT_BEGIN
+#define ALIGN_ENTRY_TEXT_END
+#endif
+
+#else
 
 #define X86_ALIGN_RODATA_BEGIN
 #define X86_ALIGN_RODATA_END					\

[tip: x86/build] x86/build: Only align ENTRY_TEXT to PMD_SIZE if necessary

[tip-bot2 for Hamza Mahfooz]

The following commit has been merged into the x86/build branch of tip:

Commit-ID:     f6cff820877ff34839920ff9de9b912359cfb607
Gitweb:        https://git.kernel.org/tip/f6cff820877ff34839920ff9de9b912359cfb607
Author:        Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
AuthorDate:    Tue, 23 Jun 2026 11:24:29 -04:00
Committer:     Ingo Molnar <mingo@kernel.org>
CommitterDate: Fri, 10 Jul 2026 11:21:17 +02:00

x86/build: Only align ENTRY_TEXT to PMD_SIZE if necessary

PTI requires the begin and end of ENTRY_TEXT be aligned to PMD_SIZE.  SRSO
requires srso_alias_untrain_ret to be 2M aligned. This costs between 2-4 MiB
of RAM (depending on the size of the preceding section).  So, only align when
either of the two are enabled.

Co-developed-by: Jared White <jaredwhite@microsoft.com>
Signed-off-by: Jared White <jaredwhite@microsoft.com>
Signed-off-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260623152430.1473481-1-hamzamahfooz@linux.microsoft.com
---
 arch/x86/kernel/vmlinux.lds.S | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 74e336d..10caf92 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -65,9 +65,15 @@ const_cpu_current_top_of_stack = cpu_current_top_of_stack;
 		__end_rodata_hpage_align = .;			\
 		__end_rodata_aligned = .;
 
+#if defined(CONFIG_MITIGATION_PAGE_TABLE_ISOLATION) || defined(CONFIG_MITIGATION_SRSO)
 #define ALIGN_ENTRY_TEXT_BEGIN	. = ALIGN(PMD_SIZE);
 #define ALIGN_ENTRY_TEXT_END	. = ALIGN(PMD_SIZE);
 #else
+#define ALIGN_ENTRY_TEXT_BEGIN
+#define ALIGN_ENTRY_TEXT_END
+#endif
+
+#else
 
 #define X86_ALIGN_RODATA_BEGIN
 #define X86_ALIGN_RODATA_END					\