[PATCH] arm64: Allow packing uncompressed images into distro packages

[PATCH] arm64: Allow packing uncompressed images into distro packages

[Bjorn Andersson]

From: Bjorn Andersson <quic_bjorande@quicinc.com>

The distro packages (deb-pkg, pacman-pkg, rpm-pkg) are generated using
the compressed kernel image, which means that the kernel once installed
can not be booted with systemd-boot.

This differs from the packages generated by the distros themselves,
which uses the uncompressed image.

Expand the newly introduced CONFIG_COMPRESSED_INSTALL option to allow
selection of which version of the kernel image should be packaged into
the distro packages.

Signed-off-by: Bjorn Andersson <quic_bjorande@quicinc.com>
---
 arch/arm64/Makefile | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index f6bc3da1ef11..7bb9a0a5500a 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -166,9 +166,13 @@ BOOT_TARGETS	:= Image vmlinuz.efi image.fit
 PHONY += $(BOOT_TARGETS)
 
 ifeq ($(CONFIG_EFI_ZBOOT),)
-KBUILD_IMAGE	:= $(boot)/Image.gz
+  ifeq ($(CONFIG_COMPRESSED_INSTALL),y)
+    KBUILD_IMAGE := $(boot)/Image.gz
+  else
+    KBUILD_IMAGE := $(boot)/Image
+  endif
 else
-KBUILD_IMAGE	:= $(boot)/vmlinuz.efi
+  KBUILD_IMAGE := $(boot)/vmlinuz.efi
 endif
 
 all:	$(notdir $(KBUILD_IMAGE))
@@ -182,13 +186,6 @@ $(BOOT_TARGETS): vmlinux
 Image.%: Image
 	$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
 
-ifeq ($(CONFIG_COMPRESSED_INSTALL),y)
- DEFAULT_KBUILD_IMAGE = $(KBUILD_IMAGE)
-else
- DEFAULT_KBUILD_IMAGE = $(boot)/Image
-endif
-
-install: KBUILD_IMAGE := $(DEFAULT_KBUILD_IMAGE)
 install zinstall:
 	$(call cmd,install)
 

---
base-commit: 469f1bad3c1c6e268059f78c0eec7e9552b3894c
change-id: 20240819-uncompressed-distro-packages-8da6959ed698

Best regards,
-- 
Bjorn Andersson <quic_bjorande@quicinc.com>

Re: [PATCH] arm64: Allow packing uncompressed images into distro packages

[Konrad Dybcio]

On 20.08.2024 5:11 AM, Bjorn Andersson wrote:
> From: Bjorn Andersson <quic_bjorande@quicinc.com>
> 
> The distro packages (deb-pkg, pacman-pkg, rpm-pkg) are generated using
> the compressed kernel image, which means that the kernel once installed
> can not be booted with systemd-boot.
> 
> This differs from the packages generated by the distros themselves,
> which uses the uncompressed image.
> 
> Expand the newly introduced CONFIG_COMPRESSED_INSTALL option to allow
> selection of which version of the kernel image should be packaged into
> the distro packages.
> 
> Signed-off-by: Bjorn Andersson <quic_bjorande@quicinc.com>
> ---

Reviewed-by: Konrad Dybcio <konradybcio@kernel.org>

Konrad

Re: [PATCH] arm64: Allow packing uncompressed images into distro packages

[Will Deacon]

On Mon, Aug 19, 2024 at 08:11:58PM -0700, Bjorn Andersson wrote:
> From: Bjorn Andersson <quic_bjorande@quicinc.com>
> 
> The distro packages (deb-pkg, pacman-pkg, rpm-pkg) are generated using
> the compressed kernel image, which means that the kernel once installed
> can not be booted with systemd-boot.
> 
> This differs from the packages generated by the distros themselves,
> which uses the uncompressed image.
> 
> Expand the newly introduced CONFIG_COMPRESSED_INSTALL option to allow
> selection of which version of the kernel image should be packaged into
> the distro packages.
> 
> Signed-off-by: Bjorn Andersson <quic_bjorande@quicinc.com>
> ---
>  arch/arm64/Makefile | 15 ++++++---------
>  1 file changed, 6 insertions(+), 9 deletions(-)
> 
> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index f6bc3da1ef11..7bb9a0a5500a 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -166,9 +166,13 @@ BOOT_TARGETS	:= Image vmlinuz.efi image.fit
>  PHONY += $(BOOT_TARGETS)
>  
>  ifeq ($(CONFIG_EFI_ZBOOT),)
> -KBUILD_IMAGE	:= $(boot)/Image.gz
> +  ifeq ($(CONFIG_COMPRESSED_INSTALL),y)
> +    KBUILD_IMAGE := $(boot)/Image.gz
> +  else
> +    KBUILD_IMAGE := $(boot)/Image
> +  endif
>  else
> -KBUILD_IMAGE	:= $(boot)/vmlinuz.efi
> +  KBUILD_IMAGE := $(boot)/vmlinuz.efi
>  endif
>  
>  all:	$(notdir $(KBUILD_IMAGE))
> @@ -182,13 +186,6 @@ $(BOOT_TARGETS): vmlinux
>  Image.%: Image
>  	$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
>  
> -ifeq ($(CONFIG_COMPRESSED_INSTALL),y)
> - DEFAULT_KBUILD_IMAGE = $(KBUILD_IMAGE)
> -else
> - DEFAULT_KBUILD_IMAGE = $(boot)/Image
> -endif
> -
> -install: KBUILD_IMAGE := $(DEFAULT_KBUILD_IMAGE)

Hmm, doesn't this mean that we always install vmlinuz.efi if
CONFIG_EFI_ZBOOT=y?

Will

Re: [PATCH] arm64: Allow packing uncompressed images into distro packages

[Bjorn Andersson]

On Fri, Aug 23, 2024 at 11:58:54AM +0100, Will Deacon wrote:
> On Mon, Aug 19, 2024 at 08:11:58PM -0700, Bjorn Andersson wrote:
> > From: Bjorn Andersson <quic_bjorande@quicinc.com>
> > 
> > The distro packages (deb-pkg, pacman-pkg, rpm-pkg) are generated using
> > the compressed kernel image, which means that the kernel once installed
> > can not be booted with systemd-boot.
> > 
> > This differs from the packages generated by the distros themselves,
> > which uses the uncompressed image.
> > 
> > Expand the newly introduced CONFIG_COMPRESSED_INSTALL option to allow
> > selection of which version of the kernel image should be packaged into
> > the distro packages.
> > 
> > Signed-off-by: Bjorn Andersson <quic_bjorande@quicinc.com>
> > ---
> >  arch/arm64/Makefile | 15 ++++++---------
> >  1 file changed, 6 insertions(+), 9 deletions(-)
> > 
> > diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> > index f6bc3da1ef11..7bb9a0a5500a 100644
> > --- a/arch/arm64/Makefile
> > +++ b/arch/arm64/Makefile
> > @@ -166,9 +166,13 @@ BOOT_TARGETS	:= Image vmlinuz.efi image.fit
> >  PHONY += $(BOOT_TARGETS)
> >  
> >  ifeq ($(CONFIG_EFI_ZBOOT),)
> > -KBUILD_IMAGE	:= $(boot)/Image.gz
> > +  ifeq ($(CONFIG_COMPRESSED_INSTALL),y)
> > +    KBUILD_IMAGE := $(boot)/Image.gz
> > +  else
> > +    KBUILD_IMAGE := $(boot)/Image
> > +  endif
> >  else
> > -KBUILD_IMAGE	:= $(boot)/vmlinuz.efi
> > +  KBUILD_IMAGE := $(boot)/vmlinuz.efi
> >  endif
> >  
> >  all:	$(notdir $(KBUILD_IMAGE))
> > @@ -182,13 +186,6 @@ $(BOOT_TARGETS): vmlinux
> >  Image.%: Image
> >  	$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
> >  
> > -ifeq ($(CONFIG_COMPRESSED_INSTALL),y)
> > - DEFAULT_KBUILD_IMAGE = $(KBUILD_IMAGE)
> > -else
> > - DEFAULT_KBUILD_IMAGE = $(boot)/Image
> > -endif
> > -
> > -install: KBUILD_IMAGE := $(DEFAULT_KBUILD_IMAGE)
> 
> Hmm, doesn't this mean that we always install vmlinuz.efi if
> CONFIG_EFI_ZBOOT=y?
> 

Hmm, you're right, I failed to parse that part.

That said, prior to Linus' change we'd always install "Image" and I read
his commit message to allow installing "Image.gz".

But the change also made it possible to install "vmlinuz.efi", by
setting both options to =y. Was this intentional?

Can you confirm that this is what we want:

ZBOOT | COMPRESS | BUILD_IMAGE | install
------+----------+-------------+--------
  N   |    N     | Image       | Image
  N   |    Y     | Image.gz    | Image.gz
  Y   |    N     | vmlinuz.efi | Image (?)
  Y   |    Y     | vmlinuz.efi | vmlinuz.efi (was Image in v6.10)

It seems reasonable to keep the two last entries in the "install" column
either "Image/Image.gz" or "vmlinuz.efi/vmlinuz.efi" (if it should be
possible to pass vmlinuz.efi to installkernel).

Regards,
Bjorn

Re: [PATCH] arm64: Allow packing uncompressed images into distro packages

[Linus Torvalds]

On Sat, 24 Aug 2024 at 07:08, Bjorn Andersson <quic_bjorande@quicinc.com> wrote:
>
> But the change also made it possible to install "vmlinuz.efi", by
> setting both options to =y. Was this intentional?

Absolutely. My arm64 config in fact has EFI_ZBOOT enabled.

IOW, the intent of that CONFIG_COMPRESSED_INSTALL was simply to make
"make install" do the same thing that "make zinstall" used to do.

I in fact initially limited the whole COMPRESSED_INSTALL question to
be *only* for when EFI_ZBOOT is enabled (because that was my
situation), and privately asked Will if maybe non-EFI people want it.
So the patch originally had

+       depends on EFI && EFI_ZBOOT

and I asked Will

  Comments? Do the non-EFI_ZBOOT cases also perhaps want this (ie
  "Image.gz" as opposed to "vmlinuz.efi")?

  I intentionally tried to make it as limited as possible, but maybe the
  non-EFI people would want this too?

and he thought that it would be better to just make this compressed
install question be independent of anything else, and literally just
boil down to "do you want 'make install' to do the same thing as 'make
zinstall' does?"

I have *no* idea about what the actual package manager case wants, though.

              Linus

Re: [PATCH] arm64: Allow packing uncompressed images into distro packages

[Bjorn Andersson]

On Sat, Aug 24, 2024 at 10:22:54AM +0800, Linus Torvalds wrote:
> On Sat, 24 Aug 2024 at 07:08, Bjorn Andersson <quic_bjorande@quicinc.com> wrote:
> >
> > But the change also made it possible to install "vmlinuz.efi", by
> > setting both options to =y. Was this intentional?
> 
> Absolutely. My arm64 config in fact has EFI_ZBOOT enabled.
> 

Per CONFIG_EFI_ZBOOT help text, the ZBOOT is a compressed image wrapped
in an EFI container. Add to this that both GRUB and Android Boot Loader
happily loads the compressed Image (i.e. Image.gz).

So, in my mind there's: uncompressed Image, compressed Image.gz, and
compressed Image.gz packaged in an EFI application to uncompress itself.

But your change makes more sense than you just wanting the Image.gz,
which puzzled me...

> IOW, the intent of that CONFIG_COMPRESSED_INSTALL was simply to make
> "make install" do the same thing that "make zinstall" used to do.
> 

I was convinced that make zinstall do install the Image.gz, looking at
the Makefiles I am however not able to see how.

> I in fact initially limited the whole COMPRESSED_INSTALL question to
> be *only* for when EFI_ZBOOT is enabled (because that was my
> situation), and privately asked Will if maybe non-EFI people want it.
> So the patch originally had
> 
> +       depends on EFI && EFI_ZBOOT
> 
> and I asked Will
> 
>   Comments? Do the non-EFI_ZBOOT cases also perhaps want this (ie
>   "Image.gz" as opposed to "vmlinuz.efi")?
> 
>   I intentionally tried to make it as limited as possible, but maybe the
>   non-EFI people would want this too?
> 

Just to clarify this point, I only have EFI systems.

> and he thought that it would be better to just make this compressed
> install question be independent of anything else, and literally just
> boil down to "do you want 'make install' to do the same thing as 'make
> zinstall' does?"
> 
> I have *no* idea about what the actual package manager case wants, though.
> 

I looked at, and tried, Arch Linux, Debian, and Fedora. The package
managers effectively unpacks the files, invokes mkinitcpio and feeds the
result to the bootloader. So if you have a bootloader (like
systemd-boot) that doesn't decompress the Image it's asked to load, then
what's being put into the package needs to be uncompressed.

This is why I would like the option to enter the packaging steps with
KBUILD_IMAGE=Image...


I think it would make more sense to have CONFIG_COMPRESSED_INSTALL
represent Image.gz vs Image (compressed vs uncompressed). And if you ask
for an EFI-wrapped Image (ZBOOT) we compress and install that for you
regardless of CONFIG_COMPRESSED_INSTALL.

Regards,
Bjorn

Re: [PATCH] arm64: Allow packing uncompressed images into distro packages

[Will Deacon]

On Fri, Aug 23, 2024 at 04:08:44PM -0700, Bjorn Andersson wrote:
> On Fri, Aug 23, 2024 at 11:58:54AM +0100, Will Deacon wrote:
> > On Mon, Aug 19, 2024 at 08:11:58PM -0700, Bjorn Andersson wrote:
> > > From: Bjorn Andersson <quic_bjorande@quicinc.com>
> > > 
> > > The distro packages (deb-pkg, pacman-pkg, rpm-pkg) are generated using
> > > the compressed kernel image, which means that the kernel once installed
> > > can not be booted with systemd-boot.
> > > 
> > > This differs from the packages generated by the distros themselves,
> > > which uses the uncompressed image.
> > > 
> > > Expand the newly introduced CONFIG_COMPRESSED_INSTALL option to allow
> > > selection of which version of the kernel image should be packaged into
> > > the distro packages.
> > > 
> > > Signed-off-by: Bjorn Andersson <quic_bjorande@quicinc.com>
> > > ---
> > >  arch/arm64/Makefile | 15 ++++++---------
> > >  1 file changed, 6 insertions(+), 9 deletions(-)
> > > 
> > > diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> > > index f6bc3da1ef11..7bb9a0a5500a 100644
> > > --- a/arch/arm64/Makefile
> > > +++ b/arch/arm64/Makefile
> > > @@ -166,9 +166,13 @@ BOOT_TARGETS	:= Image vmlinuz.efi image.fit
> > >  PHONY += $(BOOT_TARGETS)
> > >  
> > >  ifeq ($(CONFIG_EFI_ZBOOT),)
> > > -KBUILD_IMAGE	:= $(boot)/Image.gz
> > > +  ifeq ($(CONFIG_COMPRESSED_INSTALL),y)
> > > +    KBUILD_IMAGE := $(boot)/Image.gz
> > > +  else
> > > +    KBUILD_IMAGE := $(boot)/Image
> > > +  endif
> > >  else
> > > -KBUILD_IMAGE	:= $(boot)/vmlinuz.efi
> > > +  KBUILD_IMAGE := $(boot)/vmlinuz.efi
> > >  endif
> > >  
> > >  all:	$(notdir $(KBUILD_IMAGE))
> > > @@ -182,13 +186,6 @@ $(BOOT_TARGETS): vmlinux
> > >  Image.%: Image
> > >  	$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
> > >  
> > > -ifeq ($(CONFIG_COMPRESSED_INSTALL),y)
> > > - DEFAULT_KBUILD_IMAGE = $(KBUILD_IMAGE)
> > > -else
> > > - DEFAULT_KBUILD_IMAGE = $(boot)/Image
> > > -endif
> > > -
> > > -install: KBUILD_IMAGE := $(DEFAULT_KBUILD_IMAGE)
> > 
> > Hmm, doesn't this mean that we always install vmlinuz.efi if
> > CONFIG_EFI_ZBOOT=y?
> > 
> 
> Hmm, you're right, I failed to parse that part.
> 
> That said, prior to Linus' change we'd always install "Image" and I read
> his commit message to allow installing "Image.gz".
> 
> But the change also made it possible to install "vmlinuz.efi", by
> setting both options to =y. Was this intentional?
> 
> Can you confirm that this is what we want:
> 
> ZBOOT | COMPRESS | BUILD_IMAGE | install
> ------+----------+-------------+--------
>   N   |    N     | Image       | Image
>   N   |    Y     | Image.gz    | Image.gz
>   Y   |    N     | vmlinuz.efi | Image (?)
>   Y   |    Y     | vmlinuz.efi | vmlinuz.efi (was Image in v6.10)

I think that's the current behaviour, and I don't see a problem with it.

The main thing is to avoid breaking somebody's system in the default
configuration (i.e. when CONFIG_COMPRESSED_INSTALL=n) by installing an
image via `make install` that isn't supported universally by bootloaders.

Will

Re: [PATCH] arm64: Allow packing uncompressed images into distro packages

[Bjorn Andersson]

On Tue, Aug 27, 2024 at 12:33:56PM +0100, Will Deacon wrote:
> On Fri, Aug 23, 2024 at 04:08:44PM -0700, Bjorn Andersson wrote:
> > On Fri, Aug 23, 2024 at 11:58:54AM +0100, Will Deacon wrote:
> > > On Mon, Aug 19, 2024 at 08:11:58PM -0700, Bjorn Andersson wrote:
> > > > From: Bjorn Andersson <quic_bjorande@quicinc.com>
> > > > 
> > > > The distro packages (deb-pkg, pacman-pkg, rpm-pkg) are generated using
> > > > the compressed kernel image, which means that the kernel once installed
> > > > can not be booted with systemd-boot.
> > > > 
> > > > This differs from the packages generated by the distros themselves,
> > > > which uses the uncompressed image.
> > > > 
> > > > Expand the newly introduced CONFIG_COMPRESSED_INSTALL option to allow
> > > > selection of which version of the kernel image should be packaged into
> > > > the distro packages.
> > > > 
> > > > Signed-off-by: Bjorn Andersson <quic_bjorande@quicinc.com>
> > > > ---
> > > >  arch/arm64/Makefile | 15 ++++++---------
> > > >  1 file changed, 6 insertions(+), 9 deletions(-)
> > > > 
> > > > diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> > > > index f6bc3da1ef11..7bb9a0a5500a 100644
> > > > --- a/arch/arm64/Makefile
> > > > +++ b/arch/arm64/Makefile
> > > > @@ -166,9 +166,13 @@ BOOT_TARGETS	:= Image vmlinuz.efi image.fit
> > > >  PHONY += $(BOOT_TARGETS)
> > > >  
> > > >  ifeq ($(CONFIG_EFI_ZBOOT),)
> > > > -KBUILD_IMAGE	:= $(boot)/Image.gz
> > > > +  ifeq ($(CONFIG_COMPRESSED_INSTALL),y)
> > > > +    KBUILD_IMAGE := $(boot)/Image.gz
> > > > +  else
> > > > +    KBUILD_IMAGE := $(boot)/Image
> > > > +  endif
> > > >  else
> > > > -KBUILD_IMAGE	:= $(boot)/vmlinuz.efi
> > > > +  KBUILD_IMAGE := $(boot)/vmlinuz.efi
> > > >  endif
> > > >  
> > > >  all:	$(notdir $(KBUILD_IMAGE))
> > > > @@ -182,13 +186,6 @@ $(BOOT_TARGETS): vmlinux
> > > >  Image.%: Image
> > > >  	$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
> > > >  
> > > > -ifeq ($(CONFIG_COMPRESSED_INSTALL),y)
> > > > - DEFAULT_KBUILD_IMAGE = $(KBUILD_IMAGE)
> > > > -else
> > > > - DEFAULT_KBUILD_IMAGE = $(boot)/Image
> > > > -endif
> > > > -
> > > > -install: KBUILD_IMAGE := $(DEFAULT_KBUILD_IMAGE)
> > > 
> > > Hmm, doesn't this mean that we always install vmlinuz.efi if
> > > CONFIG_EFI_ZBOOT=y?
> > > 
> > 
> > Hmm, you're right, I failed to parse that part.
> > 
> > That said, prior to Linus' change we'd always install "Image" and I read
> > his commit message to allow installing "Image.gz".
> > 
> > But the change also made it possible to install "vmlinuz.efi", by
> > setting both options to =y. Was this intentional?
> > 
> > Can you confirm that this is what we want:
> > 
> > ZBOOT | COMPRESS | BUILD_IMAGE | install
> > ------+----------+-------------+--------
> >   N   |    N     | Image       | Image
> >   N   |    Y     | Image.gz    | Image.gz
> >   Y   |    N     | vmlinuz.efi | Image (?)
> >   Y   |    Y     | vmlinuz.efi | vmlinuz.efi (was Image in v6.10)
> 
> I think that's the current behaviour, and I don't see a problem with it.
> 
> The main thing is to avoid breaking somebody's system in the default
> configuration (i.e. when CONFIG_COMPRESSED_INSTALL=n) by installing an
> image via `make install` that isn't supported universally by bootloaders.
> 

Okay, thanks for confirming the expectation. I'll wrangle my patch to
match this.

Thanks,
Bjorn