[PATCH] scripts: Add script to extract built-in firmware blobs

[PATCH] scripts: Add script to extract built-in firmware blobs

[Guilherme G. Piccoli]

Through the options CONFIG_EXTRA_FIRMWARE{_DIR} one is able to build
a kernel including firmware blobs in a built-in fashion. This is
usually the case of built-in drivers that require some blobs in
order to work properly, for example, like in non-initrd based systems.

Add hereby a script to extract these blobs from a non-stripped vmlinux,
similar to the idea of "extract-ikconfig". The firmware loader interface
saves such built-in blobs as rodata entries, having a field for the FW
name as "_fw_<module_name>_<firmware_name>_bin"; the tool extracts files
named "<module_name>_<firmware_name>" for each rodata firmware entry
detected. It makes use of awk, bash, dd and readelf, pretty standard
tooling for Linux development.

Suggested-and-reviewed-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com>
---
 scripts/extract-fwblobs | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
 create mode 100755 scripts/extract-fwblobs

diff --git a/scripts/extract-fwblobs b/scripts/extract-fwblobs
new file mode 100755
index 000000000000..15e836e47368
--- /dev/null
+++ b/scripts/extract-fwblobs
@@ -0,0 +1,31 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# -----------------------------------------------------------------------------
+# Extracts the vmlinux built-in firmware blobs - requires a non-stripped image
+# -----------------------------------------------------------------------------
+
+if [ -z "$1" ]; then
+	echo "Must provide a non-stripped vmlinux as argument"
+	exit 1
+fi
+
+read -r RD_ADDR_HEX RD_OFF_HEX <<< "$( readelf -SW "$1" |\
+grep -w rodata | awk '{print "0x"$5" 0x"$6}' )"
+
+FW_SYMS="$(readelf -sW "$1" |\
+awk -n '/fw_end/ { end=$2 ; print name " 0x" start " 0x" end; } { start=$2; name=$8; }')"
+
+while IFS= read -r entry; do
+	read -r FW_NAME FW_ADDR_ST_HEX FW_ADDR_END_HEX <<< "$entry"
+
+	# Notice kernel prepends _fw_ and appends _bin to the FW name
+	# in rodata; hence we hereby filter that out.
+	FW_NAME=${FW_NAME:4:-4}
+
+	FW_OFFSET="$(printf "%d" $((FW_ADDR_ST_HEX - RD_ADDR_HEX + RD_OFF_HEX)))"
+	FW_SIZE="$(printf "%d" $((FW_ADDR_END_HEX - FW_ADDR_ST_HEX)))"
+
+	dd if="$1" of="./${FW_NAME}" bs="${FW_SIZE}" count=1 iflag=skip_bytes skip="${FW_OFFSET}"
+done <<< "${FW_SYMS}"
+
-- 
2.47.1

Re: [PATCH] scripts: Add script to extract built-in firmware blobs

[Andrew Morton]

On Mon, 20 Jan 2025 16:04:26 -0300 "Guilherme G. Piccoli" <gpiccoli@igalia.com> wrote:

> Through the options CONFIG_EXTRA_FIRMWARE{_DIR} one is able to build
> a kernel including firmware blobs in a built-in fashion. This is
> usually the case of built-in drivers that require some blobs in
> order to work properly, for example, like in non-initrd based systems.
> 
> Add hereby a script to extract these blobs from a non-stripped vmlinux,
> similar to the idea of "extract-ikconfig". The firmware loader interface
> saves such built-in blobs as rodata entries, having a field for the FW
> name as "_fw_<module_name>_<firmware_name>_bin"; the tool extracts files
> named "<module_name>_<firmware_name>" for each rodata firmware entry
> detected. It makes use of awk, bash, dd and readelf, pretty standard
> tooling for Linux development.

Please describe the use case(s) for this.  ie, why are you suggesting
this be added to Linux?

Re: [PATCH] scripts: Add script to extract built-in firmware blobs

[Guilherme G. Piccoli]

On 24/01/2025 03:38, Andrew Morton wrote:
> On Mon, 20 Jan 2025 16:04:26 -0300 "Guilherme G. Piccoli" <gpiccoli@igalia.com> wrote:
> 
>> Through the options CONFIG_EXTRA_FIRMWARE{_DIR} one is able to build
>> a kernel including firmware blobs in a built-in fashion. This is
>> usually the case of built-in drivers that require some blobs in
>> order to work properly, for example, like in non-initrd based systems.
>>
>> Add hereby a script to extract these blobs from a non-stripped vmlinux,
>> similar to the idea of "extract-ikconfig". The firmware loader interface
>> saves such built-in blobs as rodata entries, having a field for the FW
>> name as "_fw_<module_name>_<firmware_name>_bin"; the tool extracts files
>> named "<module_name>_<firmware_name>" for each rodata firmware entry
>> detected. It makes use of awk, bash, dd and readelf, pretty standard
>> tooling for Linux development.
> 
> Please describe the use case(s) for this.  ie, why are you suggesting
> this be added to Linux?
> 

Hi Andrew, there is currently no tool to extract a firmware blob that is
built-in on vmlinux to the best of my knowledge. So if we have a kernel
image containing the blobs, and we want to rebuild the kernel with some
debug patches for example (and given that the image also has
IKCONFIG=y), we currently can't do that for the same versions for all
the firmware blobs, _unless_ we have exact commits of linux-firmware for
the specific versions for each firmware included.

With this tool, we can blindly extract the FWs and easily re-add them in
the new debug kernel build, allowing a more deterministic testing
without the burden of "hunting down" the proper version of each firmware
binary.

Makes sense to you?
Cheers,


Guilherme

Re: [PATCH] scripts: Add script to extract built-in firmware blobs

[Andrew Morton]

On Fri, 24 Jan 2025 09:31:07 -0300 "Guilherme G. Piccoli" <gpiccoli@igalia.com> wrote:

> On 24/01/2025 03:38, Andrew Morton wrote:
> > On Mon, 20 Jan 2025 16:04:26 -0300 "Guilherme G. Piccoli" <gpiccoli@igalia.com> wrote:
> > 
> >> Through the options CONFIG_EXTRA_FIRMWARE{_DIR} one is able to build
> >> a kernel including firmware blobs in a built-in fashion. This is
> >> usually the case of built-in drivers that require some blobs in
> >> order to work properly, for example, like in non-initrd based systems.
> >>
> >> Add hereby a script to extract these blobs from a non-stripped vmlinux,
> >> similar to the idea of "extract-ikconfig". The firmware loader interface
> >> saves such built-in blobs as rodata entries, having a field for the FW
> >> name as "_fw_<module_name>_<firmware_name>_bin"; the tool extracts files
> >> named "<module_name>_<firmware_name>" for each rodata firmware entry
> >> detected. It makes use of awk, bash, dd and readelf, pretty standard
> >> tooling for Linux development.
> > 
> > Please describe the use case(s) for this.  ie, why are you suggesting
> > this be added to Linux?
> > 
> 
> Hi Andrew, there is currently no tool to extract a firmware blob that is
> built-in on vmlinux to the best of my knowledge. So if we have a kernel
> image containing the blobs, and we want to rebuild the kernel with some
> debug patches for example (and given that the image also has
> IKCONFIG=y), we currently can't do that for the same versions for all
> the firmware blobs, _unless_ we have exact commits of linux-firmware for
> the specific versions for each firmware included.
> 
> With this tool, we can blindly extract the FWs and easily re-add them in
> the new debug kernel build, allowing a more deterministic testing
> without the burden of "hunting down" the proper version of each firmware
> binary.
> 
> Makes sense to you?

OK, thanks.  I (badly) pasted the above into the changelog.