Re: [Bug Report] Linux v6.15-rc7 boot failure on Xen-4.17

[Gupta Pawan <pawan.kumar.gupta@linux.intel.com>]

[-- Attachment #1: Type: text/plain, Size: 4367 bytes --]

On Wed, May 28, 2025 at 11:19:19AM +0200, Juergen Gross wrote:
> On 28.05.25 10:57, Jürgen Groß wrote:
> > On 28.05.25 10:26, Xin Li wrote:
> > > On 5/28/2025 12:27 AM, Xin Li wrote:
> > > > On 5/27/2025 11:49 PM, Juergen Gross wrote:
> > > > > On 28.05.25 07:11, Jürgen Groß wrote:
> > > > > > On 27.05.25 21:29, Andrew Cooper wrote:
> > > > > > > On 27/05/2025 8:21 pm, Xin Li wrote:
> > > > > > > > > On May 27, 2025, at 11:36 AM, Jürgen Groß <jgross@suse.com> wrote:
> > > > > > > > > 
> > > > > > > > > On 27.05.25 19:54, Xin Li wrote:
> > > > > > > > > > On 5/27/2025 10:46 AM, Pawan Gupta wrote:
> > > > > > > > > > > > Attached is the serial console log and my kernel config.
> > > > > > > > > > > Serial logs aren't telling much. I
> > > > > > > > > > > do not have a Xen setup to test,
> > > > > > > > > > > without
> > > > > > > > > > > Xen the config that you provided is booting a KVM guest just fine.
> > > > > > > > > > Yeah, as I replied to Juergen, the same kernel binary boots fine as
> > > > > > > > > > "native".
> > > > > > > > > > Unfortunately when booting as dom0 on Xen, it keeps rebooting w/o
> > > > > > > > > > helpful log.
> > > > > > > > > What about booting Xen on bare metal, i.e. no KVM being involved?
> > > > > > > > The same exact problem happens on Intel Simics. 
> > > > > > > > And I got to see it’s a NX page fault in dom0
> > > > > > > > kernel during apply alternatives.
> > > > > > > 
> > > > > > > In which case it's likely that there's an opencoded PTE update, rather
> > > > > > > than using the hooks (which are suitably paravirt'd).
> > > > > > 
> > > > > > I'd suspect a bug when NOT using 2M pages for execmem.
> > > > > > 
> > > > > > I'll have a look.
> > > > > 
> > > > > Could you have a try using "nohugevmalloc" dom0 kernel boot parameter?
> > > > > 
> > > > 
> > > > Tried in a KVM guest, still the same problem, and nothing new in the
> > > > serial log.
> > > 
> > > Attached is a dom0 log with stack traces.
> > > 
> > > But I really did NOT change anything to make it happen...
> > 
> > Thanks.
> > 
> > I think this might be related to Xen not advertising X86_FEATURE_PSE.
> > 
> > This will use PAGE_KERNEL page protection for execmem_alloc() page protection,
> > while with X86_FEATURE_PSE PAGE_KERNEL_ROX is being used.
> > 
> > For the kernel (so not in a module) there is no execmem_restore_rox() call
> > involved, so the NX bit will be kept for kernel side ITS thunks.
> > 
> > Peter, can you confirm my suspicion?
> 
> I just made a small test on my (rather old) system:
> 
> I verified that kernel 6.15 is booting fine as Xen dom0 (ITS mitigation
> not needed due to old cpu). Then I modified alternative.c to apply the
> ITS mitigations nevertheless, which made the kernel crash as Xen dom0.
> 
> With the following additional modification boot was working again:
> 
> diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
> index bfa444a7dbb0..fac4f9d26132 100644
> --- a/arch/x86/mm/init.c
> +++ b/arch/x86/mm/init.c
> @@ -1090,7 +1090,7 @@ struct execmem_info __init *execmem_arch_setup(void)
>                 pgprot = PAGE_KERNEL_ROX;
>                 flags = EXECMEM_KASAN_SHADOW | EXECMEM_ROX_CACHE;
>         } else {
> -               pgprot = PAGE_KERNEL;
> +               pgprot = PAGE_KERNEL_EXEC;
>                 flags = EXECMEM_KASAN_SHADOW;
>         }

I am not sure if returning a RWX page post-boot is a good idea.

Another option that might work is to set the executable permission when we
know that the allocated page is for kernel ITS thunk, and not modules?

---
diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index ecfe7b497cad..706464103856 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -211,6 +211,14 @@ static void *its_alloc(void)
 		its_mod->its_page_array[its_mod->its_num_pages++] = page;
 
 		execmem_make_temp_rw(page, PAGE_SIZE);
+	} else if (!IS_ENABLED(CONFIG_ARCH_HAS_EXECMEM_ROX) ||
+		   !cpu_feature_enabled(X86_FEATURE_PSE)) {
+		set_memory_x((unsigned long)page, 1);
+	}
+#else /* CONFIG_MODULES */
+	if (!IS_ENABLED(CONFIG_ARCH_HAS_EXECMEM_ROX) ||
+	    !cpu_feature_enabled(X86_FEATURE_PSE)) {
+		set_memory_x((unsigned long)page, 1);
 	}
 #endif /* CONFIG_MODULES */
 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]