[PATCH] ARM: initialize jump labels before setup_machine_fdt()

[PATCH] ARM: initialize jump labels before setup_machine_fdt()

[Jason A. Donenfeld]

Stephen reported that a static key warning splat appears during early
boot on arm64 systems that credit randomness from device trees that
contain an "rng-seed" property, because setup_machine_fdt() is called
before jump_label_init() during setup_arch(), which was fixed by
73e2d827a501 ("arm64: Initialize jump labels before
setup_machine_fdt()").

Upon cursory inspection, the same basic issue appears to apply to arm32
as well. In this case, we reorder setup_arch() to do things in the same
order as is now the case on arm64.

Reported-by: Stephen Boyd <swboyd@chromium.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Ard Biesheuvel <ardb@kernel.org>
Cc: stable@vger.kernel.org
Fixes: f5bda35fba61 ("random: use static branch for crng_ready()")
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
---
 arch/arm/kernel/setup.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 1e8a50a97edf..ef40d9f5d5a7 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -1097,10 +1097,15 @@ void __init setup_arch(char **cmdline_p)
 	const struct machine_desc *mdesc = NULL;
 	void *atags_vaddr = NULL;
 
+	setup_initial_init_mm(_text, _etext, _edata, _end);
+	setup_processor();
+	early_fixmap_init();
+	early_ioremap_init();
+	jump_label_init();
+
 	if (__atags_pointer)
 		atags_vaddr = FDT_VIRT_BASE(__atags_pointer);
 
-	setup_processor();
 	if (atags_vaddr) {
 		mdesc = setup_machine_fdt(atags_vaddr);
 		if (mdesc)
@@ -1125,15 +1130,10 @@ void __init setup_arch(char **cmdline_p)
 	if (mdesc->reboot_mode != REBOOT_HARD)
 		reboot_mode = mdesc->reboot_mode;
 
-	setup_initial_init_mm(_text, _etext, _edata, _end);
-
 	/* populate cmd_line too for later use, preserving boot_command_line */
 	strlcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE);
 	*cmdline_p = cmd_line;
 
-	early_fixmap_init();
-	early_ioremap_init();
-
 	parse_early_param();
 
 #ifdef CONFIG_MMU
-- 
2.35.1

Re: [PATCH] ARM: initialize jump labels before setup_machine_fdt()

[Ard Biesheuvel]

On Thu, 2 Jun 2022 at 23:22, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>
> Stephen reported that a static key warning splat appears during early
> boot on arm64 systems that credit randomness from device trees that
> contain an "rng-seed" property, because setup_machine_fdt() is called
> before jump_label_init() during setup_arch(), which was fixed by
> 73e2d827a501 ("arm64: Initialize jump labels before
> setup_machine_fdt()").
>
> Upon cursory inspection, the same basic issue appears to apply to arm32
> as well. In this case, we reorder setup_arch() to do things in the same
> order as is now the case on arm64.
>
> Reported-by: Stephen Boyd <swboyd@chromium.org>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Ard Biesheuvel <ardb@kernel.org>
> Cc: stable@vger.kernel.org
> Fixes: f5bda35fba61 ("random: use static branch for crng_ready()")

Wouldn't it be better to defer the
static_branch_enable(&crng_is_ready) call to later in the boot (e.g.,
using an initcall()), rather than going around 'fixing' fragile,
working early boot code across multiple architectures?

> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
> ---
>  arch/arm/kernel/setup.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
> index 1e8a50a97edf..ef40d9f5d5a7 100644
> --- a/arch/arm/kernel/setup.c
> +++ b/arch/arm/kernel/setup.c
> @@ -1097,10 +1097,15 @@ void __init setup_arch(char **cmdline_p)
>         const struct machine_desc *mdesc = NULL;
>         void *atags_vaddr = NULL;
>
> +       setup_initial_init_mm(_text, _etext, _edata, _end);
> +       setup_processor();
> +       early_fixmap_init();
> +       early_ioremap_init();
> +       jump_label_init();
> +

Is it really necessary to reorder all these calls? What does
jump_label_init() actually need?

If this is related to the code patching, I wonder whether it wouldn't
be better not to rewrite all the NOPs (this is a x86-ism as every new
x86 uarch appears to have a better [faster?] NOP than the previous
one)

The issue with changes like these is that we might end up with bug
report in ~3 months' time that 'obscure platform X no longer boots or
produces any output'. In the best case, we'll have a bisect report
identifying this patch, but we won't be able to simply revert it as it
would reintroduce this issue into a kernel that is now stable.



>         if (__atags_pointer)
>                 atags_vaddr = FDT_VIRT_BASE(__atags_pointer);
>
> -       setup_processor();
>         if (atags_vaddr) {
>                 mdesc = setup_machine_fdt(atags_vaddr);
>                 if (mdesc)
> @@ -1125,15 +1130,10 @@ void __init setup_arch(char **cmdline_p)
>         if (mdesc->reboot_mode != REBOOT_HARD)
>                 reboot_mode = mdesc->reboot_mode;
>
> -       setup_initial_init_mm(_text, _etext, _edata, _end);
> -
>         /* populate cmd_line too for later use, preserving boot_command_line */
>         strlcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE);
>         *cmdline_p = cmd_line;
>
> -       early_fixmap_init();
> -       early_ioremap_init();
> -
>         parse_early_param();
>
>  #ifdef CONFIG_MMU
> --
> 2.35.1
>

Re: [PATCH] ARM: initialize jump labels before setup_machine_fdt()

[Jason A. Donenfeld]

Hi Ard,

On 6/3/22, Ard Biesheuvel <ardb@kernel.org> wrote:
> On Thu, 2 Jun 2022 at 23:22, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>>
>> Stephen reported that a static key warning splat appears during early
>> boot on arm64 systems that credit randomness from device trees that
>> contain an "rng-seed" property, because setup_machine_fdt() is called
>> before jump_label_init() during setup_arch(), which was fixed by
>> 73e2d827a501 ("arm64: Initialize jump labels before
>> setup_machine_fdt()").
>>
>> Upon cursory inspection, the same basic issue appears to apply to arm32
>> as well. In this case, we reorder setup_arch() to do things in the same
>> order as is now the case on arm64.
>>
>> Reported-by: Stephen Boyd <swboyd@chromium.org>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Ard Biesheuvel <ardb@kernel.org>
>> Cc: stable@vger.kernel.org
>> Fixes: f5bda35fba61 ("random: use static branch for crng_ready()")
>
> Wouldn't it be better to defer the
> static_branch_enable(&crng_is_ready) call to later in the boot (e.g.,
> using an initcall()), rather than going around 'fixing' fragile,
> working early boot code across multiple architectures?

Yes, maybe. It's just more book keeping that's potentially
unnecessary, which would be nice to avoid. I wrote a patch for this
before, but it wasn't beautiful. And Catalin got a pretty easy arm64
patch queued up sufficiently fast that I figured this was better.

>
>> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
>> ---
>>  arch/arm/kernel/setup.c | 12 ++++++------
>>  1 file changed, 6 insertions(+), 6 deletions(-)
>>
>> diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
>> index 1e8a50a97edf..ef40d9f5d5a7 100644
>> --- a/arch/arm/kernel/setup.c
>> +++ b/arch/arm/kernel/setup.c
>> @@ -1097,10 +1097,15 @@ void __init setup_arch(char **cmdline_p)
>>         const struct machine_desc *mdesc = NULL;
>>         void *atags_vaddr = NULL;
>>
>> +       setup_initial_init_mm(_text, _etext, _edata, _end);
>> +       setup_processor();
>> +       early_fixmap_init();
>> +       early_ioremap_init();
>> +       jump_label_init();
>> +
>
> Is it really necessary to reorder all these calls? What does
> jump_label_init() actually need?

I'm not quite sure, but it matched how arm64 does things now. Was
hoping somebody with deep arm32 knowledge (e.g. you or rmk) would be
able to eyeball that to confirm.

Jason

Re: [PATCH] ARM: initialize jump labels before setup_machine_fdt()

[Ard Biesheuvel]

(+ Greg)

On Fri, 3 Jun 2022 at 09:37, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>
> Hi Ard,
>
> On 6/3/22, Ard Biesheuvel <ardb@kernel.org> wrote:
> > On Thu, 2 Jun 2022 at 23:22, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> >>
> >> Stephen reported that a static key warning splat appears during early
> >> boot on arm64 systems that credit randomness from device trees that
> >> contain an "rng-seed" property, because setup_machine_fdt() is called
> >> before jump_label_init() during setup_arch(), which was fixed by
> >> 73e2d827a501 ("arm64: Initialize jump labels before
> >> setup_machine_fdt()").
> >>
> >> Upon cursory inspection, the same basic issue appears to apply to arm32
> >> as well. In this case, we reorder setup_arch() to do things in the same
> >> order as is now the case on arm64.
> >>
> >> Reported-by: Stephen Boyd <swboyd@chromium.org>
> >> Cc: Catalin Marinas <catalin.marinas@arm.com>
> >> Cc: Ard Biesheuvel <ardb@kernel.org>
> >> Cc: stable@vger.kernel.org
> >> Fixes: f5bda35fba61 ("random: use static branch for crng_ready()")
> >
> > Wouldn't it be better to defer the
> > static_branch_enable(&crng_is_ready) call to later in the boot (e.g.,
> > using an initcall()), rather than going around 'fixing' fragile,
> > working early boot code across multiple architectures?
>
> Yes, maybe. It's just more book keeping that's potentially
> unnecessary, which would be nice to avoid. I wrote a patch for this
> before, but it wasn't beautiful. And Catalin got a pretty easy arm64
> patch queued up sufficiently fast that I figured this was better.
>

The problem is that your original patch was already backported as far
back as 5.10, and so this fix will need to be as well.

Playing with the code that runs before the call to setup_machine_fdt()
is risky because it implies that issues that are introduced are likely
to limit the ability of the system to generate diagnostic output of
any kind, given that the device tree is what describes the topology of
the system to the kernel. Before that, there is no serial or graphical
console, and the only way to figure out what goes on is to connect a
JTAG debugger and single step through the code or dump the contents of
__log_buf[].

I like the /dev/random work you have been doing but as you know, I was
skeptical about the need to backport all of that work to -stable, and
it appears my skepticism may have been justified.

The patch in question is an unquantified performance optimization,
which means it does not meet the stable-kernel-rules.rst criteria, but
it was backported nonetheless. Now, we are in a situation where we
must refactor very early boot code to address a regression introduced
by that backport.

> >
> >> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
> >> ---
> >>  arch/arm/kernel/setup.c | 12 ++++++------
> >>  1 file changed, 6 insertions(+), 6 deletions(-)
> >>
> >> diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
> >> index 1e8a50a97edf..ef40d9f5d5a7 100644
> >> --- a/arch/arm/kernel/setup.c
> >> +++ b/arch/arm/kernel/setup.c
> >> @@ -1097,10 +1097,15 @@ void __init setup_arch(char **cmdline_p)
> >>         const struct machine_desc *mdesc = NULL;
> >>         void *atags_vaddr = NULL;
> >>
> >> +       setup_initial_init_mm(_text, _etext, _edata, _end);
> >> +       setup_processor();
> >> +       early_fixmap_init();
> >> +       early_ioremap_init();
> >> +       jump_label_init();
> >> +
> >
> > Is it really necessary to reorder all these calls? What does
> > jump_label_init() actually need?
>
> I'm not quite sure, but it matched how arm64 does things now. Was
> hoping somebody with deep arm32 knowledge (e.g. you or rmk) would be
> able to eyeball that to confirm.
>

As far as I can tell, the early patching code on ARM does not rely on
the early fixmap code. Did you try just moving jump_label_init()
earlier in the function?

Also, how did you test this change?

Re: [PATCH] ARM: initialize jump labels before setup_machine_fdt()

[Ard Biesheuvel]

On Fri, 3 Jun 2022 at 09:51, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> (+ Greg)
>
> On Fri, 3 Jun 2022 at 09:37, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> >
> > Hi Ard,
> >
> > On 6/3/22, Ard Biesheuvel <ardb@kernel.org> wrote:
> > > On Thu, 2 Jun 2022 at 23:22, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> > >>
> > >> Stephen reported that a static key warning splat appears during early
> > >> boot on arm64 systems that credit randomness from device trees that
> > >> contain an "rng-seed" property, because setup_machine_fdt() is called
> > >> before jump_label_init() during setup_arch(), which was fixed by
> > >> 73e2d827a501 ("arm64: Initialize jump labels before
> > >> setup_machine_fdt()").
> > >>
> > >> Upon cursory inspection, the same basic issue appears to apply to arm32
> > >> as well. In this case, we reorder setup_arch() to do things in the same
> > >> order as is now the case on arm64.
> > >>
> > >> Reported-by: Stephen Boyd <swboyd@chromium.org>
> > >> Cc: Catalin Marinas <catalin.marinas@arm.com>
> > >> Cc: Ard Biesheuvel <ardb@kernel.org>
> > >> Cc: stable@vger.kernel.org
> > >> Fixes: f5bda35fba61 ("random: use static branch for crng_ready()")
> > >
> > > Wouldn't it be better to defer the
> > > static_branch_enable(&crng_is_ready) call to later in the boot (e.g.,
> > > using an initcall()), rather than going around 'fixing' fragile,
> > > working early boot code across multiple architectures?
> >
> > Yes, maybe. It's just more book keeping that's potentially
> > unnecessary, which would be nice to avoid. I wrote a patch for this
> > before, but it wasn't beautiful. And Catalin got a pretty easy arm64
> > patch queued up sufficiently fast that I figured this was better.
> >
>
> The problem is that your original patch was already backported as far
> back as 5.10, and so this fix will need to be as well.
>
> Playing with the code that runs before the call to setup_machine_fdt()
> is risky because it implies that issues that are introduced are likely
> to limit the ability of the system to generate diagnostic output of
> any kind, given that the device tree is what describes the topology of
> the system to the kernel. Before that, there is no serial or graphical
> console, and the only way to figure out what goes on is to connect a
> JTAG debugger and single step through the code or dump the contents of
> __log_buf[].
>
> I like the /dev/random work you have been doing but as you know, I was
> skeptical about the need to backport all of that work to -stable, and
> it appears my skepticism may have been justified.
>
> The patch in question is an unquantified performance optimization,
> which means it does not meet the stable-kernel-rules.rst criteria, but
> it was backported nonetheless. Now, we are in a situation where we
> must refactor very early boot code to address a regression introduced
> by that backport.
>
> > >
> > >> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
> > >> ---
> > >>  arch/arm/kernel/setup.c | 12 ++++++------
> > >>  1 file changed, 6 insertions(+), 6 deletions(-)
> > >>
> > >> diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
> > >> index 1e8a50a97edf..ef40d9f5d5a7 100644
> > >> --- a/arch/arm/kernel/setup.c
> > >> +++ b/arch/arm/kernel/setup.c
> > >> @@ -1097,10 +1097,15 @@ void __init setup_arch(char **cmdline_p)
> > >>         const struct machine_desc *mdesc = NULL;
> > >>         void *atags_vaddr = NULL;
> > >>
> > >> +       setup_initial_init_mm(_text, _etext, _edata, _end);
> > >> +       setup_processor();
> > >> +       early_fixmap_init();
> > >> +       early_ioremap_init();
> > >> +       jump_label_init();
> > >> +
> > >
> > > Is it really necessary to reorder all these calls? What does
> > > jump_label_init() actually need?
> >
> > I'm not quite sure, but it matched how arm64 does things now. Was
> > hoping somebody with deep arm32 knowledge (e.g. you or rmk) would be
> > able to eyeball that to confirm.
> >
>
> As far as I can tell, the early patching code on ARM does not rely on
> the early fixmap code. Did you try just moving jump_label_init()
> earlier in the function?
>

The below seems to work too:

--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -1101,6 +1101,7 @@ void __init setup_arch(char **cmdline_p)
                atags_vaddr = FDT_VIRT_BASE(__atags_pointer);

        setup_processor();
+       jump_label_init();
        if (atags_vaddr) {
                mdesc = setup_machine_fdt(atags_vaddr);
                if (mdesc)

Re: [PATCH] ARM: initialize jump labels before setup_machine_fdt()

[Jason A. Donenfeld]

Hi Ard,

On 6/3/22, Ard Biesheuvel <ardb@kernel.org> wrote:
> On Fri, 3 Jun 2022 at 09:51, Ard Biesheuvel <ardb@kernel.org> wrote:
>>
>> (+ Greg)
>>
>> On Fri, 3 Jun 2022 at 09:37, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>> >
>> > Hi Ard,
>> >
>> > On 6/3/22, Ard Biesheuvel <ardb@kernel.org> wrote:
>> > > On Thu, 2 Jun 2022 at 23:22, Jason A. Donenfeld <Jason@zx2c4.com>
>> > > wrote:
>> > >>
>> > >> Stephen reported that a static key warning splat appears during
>> > >> early
>> > >> boot on arm64 systems that credit randomness from device trees that
>> > >> contain an "rng-seed" property, because setup_machine_fdt() is
>> > >> called
>> > >> before jump_label_init() during setup_arch(), which was fixed by
>> > >> 73e2d827a501 ("arm64: Initialize jump labels before
>> > >> setup_machine_fdt()").
>> > >>
>> > >> Upon cursory inspection, the same basic issue appears to apply to
>> > >> arm32
>> > >> as well. In this case, we reorder setup_arch() to do things in the
>> > >> same
>> > >> order as is now the case on arm64.
>> > >>
>> > >> Reported-by: Stephen Boyd <swboyd@chromium.org>
>> > >> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> > >> Cc: Ard Biesheuvel <ardb@kernel.org>
>> > >> Cc: stable@vger.kernel.org
>> > >> Fixes: f5bda35fba61 ("random: use static branch for crng_ready()")
>> > >
>> > > Wouldn't it be better to defer the
>> > > static_branch_enable(&crng_is_ready) call to later in the boot (e.g.,
>> > > using an initcall()), rather than going around 'fixing' fragile,
>> > > working early boot code across multiple architectures?
>> >
>> > Yes, maybe. It's just more book keeping that's potentially
>> > unnecessary, which would be nice to avoid. I wrote a patch for this
>> > before, but it wasn't beautiful. And Catalin got a pretty easy arm64
>> > patch queued up sufficiently fast that I figured this was better.
>> >
>>
>> The problem is that your original patch was already backported as far
>> back as 5.10, and so this fix will need to be as well.
>>
>> Playing with the code that runs before the call to setup_machine_fdt()
>> is risky because it implies that issues that are introduced are likely
>> to limit the ability of the system to generate diagnostic output of
>> any kind, given that the device tree is what describes the topology of
>> the system to the kernel. Before that, there is no serial or graphical
>> console, and the only way to figure out what goes on is to connect a
>> JTAG debugger and single step through the code or dump the contents of
>> __log_buf[].
>>
>> I like the /dev/random work you have been doing but as you know, I was
>> skeptical about the need to backport all of that work to -stable, and
>> it appears my skepticism may have been justified.
>>
>> The patch in question is an unquantified performance optimization,
>> which means it does not meet the stable-kernel-rules.rst criteria, but
>> it was backported nonetheless. Now, we are in a situation where we
>> must refactor very early boot code to address a regression introduced
>> by that backport.
>>
>> > >
>> > >> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
>> > >> ---
>> > >>  arch/arm/kernel/setup.c | 12 ++++++------
>> > >>  1 file changed, 6 insertions(+), 6 deletions(-)
>> > >>
>> > >> diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
>> > >> index 1e8a50a97edf..ef40d9f5d5a7 100644
>> > >> --- a/arch/arm/kernel/setup.c
>> > >> +++ b/arch/arm/kernel/setup.c
>> > >> @@ -1097,10 +1097,15 @@ void __init setup_arch(char **cmdline_p)
>> > >>         const struct machine_desc *mdesc = NULL;
>> > >>         void *atags_vaddr = NULL;
>> > >>
>> > >> +       setup_initial_init_mm(_text, _etext, _edata, _end);
>> > >> +       setup_processor();
>> > >> +       early_fixmap_init();
>> > >> +       early_ioremap_init();
>> > >> +       jump_label_init();
>> > >> +
>> > >
>> > > Is it really necessary to reorder all these calls? What does
>> > > jump_label_init() actually need?
>> >
>> > I'm not quite sure, but it matched how arm64 does things now. Was
>> > hoping somebody with deep arm32 knowledge (e.g. you or rmk) would be
>> > able to eyeball that to confirm.
>> >
>>
>> As far as I can tell, the early patching code on ARM does not rely on
>> the early fixmap code. Did you try just moving jump_label_init()
>> earlier in the function?
>>
>
> The below seems to work too:
>
> --- a/arch/arm/kernel/setup.c
> +++ b/arch/arm/kernel/setup.c
> @@ -1101,6 +1101,7 @@ void __init setup_arch(char **cmdline_p)
>                 atags_vaddr = FDT_VIRT_BASE(__atags_pointer);
>
>         setup_processor();
> +       jump_label_init();
>         if (atags_vaddr) {
>                 mdesc = setup_machine_fdt(atags_vaddr);
>                 if (mdesc)
>

Oh, awesome, that's exactly what I was about to try when I got to my
laptop in a few hours. Do you want to send the v2 with that, or should
I do so after also testing it in a bit?

Jason

Re: [PATCH] ARM: initialize jump labels before setup_machine_fdt()

[Jason A. Donenfeld]

Hi Ard,

On 6/3/22, Ard Biesheuvel <ardb@kernel.org> wrote:
> The problem is that your original patch

You remain extremely unpleasant to communicate with. Can we keep
things on topic please?

> As far as I can tell, the early patching code on ARM does not rely on
> the early fixmap code. Did you try just moving jump_label_init()
> earlier in the function?
>
> Also, how did you test this change?

Just booting a few configs in QEMU. I don't have access to real
hardware right now unfortunately.

Let me give a try to just moving the jump_label_init() function alone.
That'd certainly make this patch a lot more basic, which would be a
good thing, and might assuage your well justified concerns that too
much boot order churn will break something subtle. I was just afraid
of complicated intermingling with the other stuff after I saw that
arm64 did things in the other order. But maybe that's silly.

I'll send a v2 if that works, and send an update here if it doesn't.
Thanks for the suggestion.

Jason

Re: [PATCH] ARM: initialize jump labels before setup_machine_fdt()

[Ard Biesheuvel]

On Fri, 3 Jun 2022 at 10:12, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>
> Hi Ard,
>
> On 6/3/22, Ard Biesheuvel <ardb@kernel.org> wrote:
> > The problem is that your original patch
>
> You remain extremely unpleasant to communicate with.

Noted.

> Can we keep
> things on topic please?
>

Sure. Are you saying the original patch is off-topic? Isn't that the
patch that caused the regression to begin with?

> > As far as I can tell, the early patching code on ARM does not rely on
> > the early fixmap code. Did you try just moving jump_label_init()
> > earlier in the function?
> >
> > Also, how did you test this change?
>
> Just booting a few configs in QEMU. I don't have access to real
> hardware right now unfortunately.
>
> Let me give a try to just moving the jump_label_init() function alone.
> That'd certainly make this patch a lot more basic, which would be a
> good thing, and might assuage your well justified concerns that too
> much boot order churn will break something subtle. I was just afraid
> of complicated intermingling with the other stuff after I saw that
> arm64 did things in the other order. But maybe that's silly.
>
> I'll send a v2 if that works, and send an update here if it doesn't.
> Thanks for the suggestion.
>
> Jason