From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from metis.whiteo.stw.pengutronix.de (metis.whiteo.stw.pengutronix.de [185.203.201.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3F1E2EC0B4 for ; Wed, 26 Nov 2025 11:09:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.203.201.7 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764155368; cv=none; b=MMKuavirHR7cjbodD8zy97T6ghR+UXYWv7QUOB8dvDwAQMd/7aeY1HtH8Yv3zz+5UZtYDrWZeJFgiuO6DvBX39ub87FerXR3L30q86nuyssVpFsrKluyZGMRyNySycIu2Ya7xkuKP05oqAOz4JfeA/kAQPw/ozqYWnzsZMPsTqM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764155368; c=relaxed/simple; bh=zMLpnj1YKvJzqZF95IlGp+QunhE8V+boe3fyUyDFNSg=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=cYbpiLue3x1Zt65aKupCSmK9kxq5BIv1ZKdrW/aBj4ppT1vkU4EwjhOKhSNVeGYdd+H/YDlLuGCzTggpssPq+/nDo3shAdCbsZxWHoY3HllrwgK4+TaO0+UPJ9lA961UStfK5DDtKciBbg09uSpZSlfXLQqwtiFNezZM0uBulC0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pengutronix.de; spf=pass smtp.mailfrom=pengutronix.de; arc=none smtp.client-ip=185.203.201.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pengutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pengutronix.de Received: from ptz.office.stw.pengutronix.de ([2a0a:edc0:0:900:1d::77] helo=[127.0.0.1]) by metis.whiteo.stw.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1vODOf-0007RU-Pa; Wed, 26 Nov 2025 12:09:17 +0100 Message-ID: Date: Wed, 26 Nov 2025 12:09:16 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v6 4/8] scripts/make_fit: Provide a way to add built modules To: Simon Glass , linux-arm-kernel@lists.infradead.org Cc: =?UTF-8?Q?Thomas_Wei=C3=9Fschuh?= , Masahiro Yamada , Tom Rini , =?UTF-8?B?SiAuIE5ldXNjaMOkZmVy?= , Nicolas Schier , Chen-Yu Tsai , linux-kernel@vger.kernel.org References: <20251119181333.991099-1-sjg@chromium.org> <20251119181333.991099-5-sjg@chromium.org> From: Ahmad Fatoum Content-Language: en-US, de-DE, de-BE In-Reply-To: <20251119181333.991099-5-sjg@chromium.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SA-Exim-Connect-IP: 2a0a:edc0:0:900:1d::77 X-SA-Exim-Mail-From: a.fatoum@pengutronix.de X-SA-Exim-Scanned: No (on metis.whiteo.stw.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: linux-kernel@vger.kernel.org Hello Simon, On 11/19/25 7:13 PM, Simon Glass wrote: > Provide arguments to support building a ramdisk from a directory tree of > modules. This is a convenient way to try out a kernel with its modules. > > This makes use usr/gen_initramfs.sh where it can be found. Failing that > it uses the cpio tool rather than attempting to use a python module or > our own code. The list of modules is provided in a file. I don't think the fallback is a good idea. This script is part of the kernel tree and usr/gen_initramfs.sh is as well, so we shouldn't duplicate the logic. With a dependency on gen_init_cpio (See my modules-cpio-pkg patch), we should always have the script and the executable available and it's fine to throw an error if it's not. Thanks for putting in the work! Cheers, Ahmad > > Signed-off-by: Simon Glass > Suggested-by: Ahmad Fatoum > --- > Note: The approach of depending on modules_install caused build problems > as mentioned in the cover letter. > > I dropped the review tag from Nicolas Schier since there are quite a few > changes in this version. > > Changes in v6: > - Using the modules.order file instead of 'find' > - Use gen_initramfs.sh where available > > Changes in v4: > - Provide the list of modules from the Makefile > - Reduce verbosity (don't print every module filename) > > Changes in v3: > - Add a way to add built modules into the FIT > > scripts/make_fit.py | 127 +++++++++++++++++++++++++++++++++++++++++--- > 1 file changed, 119 insertions(+), 8 deletions(-) > > diff --git a/scripts/make_fit.py b/scripts/make_fit.py > index 1a74a9dcd85e..9dfef11fc4b3 100755 > --- a/scripts/make_fit.py > +++ b/scripts/make_fit.py > @@ -13,11 +13,17 @@ Usage: > -r /boot/initrd.img-6.14.0-27-generic @arch/arm64/boot/dts/dtbs-list > -E -c gzip > > + # Build with modules ramdisk instead of external ramdisk: > + make_fit.py -A arm64 -n 'Linux-6.17' -O linux > + -o arch/arm64/boot/image.fit -k /tmp/kern/arch/arm64/boot/image.itk > + -m module1.ko module2.ko module3.ko @arch/arm64/boot/dts/dtbs-list > + > Creates a FIT containing the supplied kernel, an optional ramdisk, and a set of > devicetree files, either specified individually or listed in a file (with an > '@' prefix). > > Use -r to specify an existing ramdisk/initrd file. > +Use -m to build a ramdisk from specified kernel module files. > > Use -E to generate an external FIT (where the data is placed after the > FIT data structure). This allows parsing of the data without loading > @@ -38,6 +44,7 @@ as U-Boot, Linuxboot, Tianocore, etc. > import argparse > import collections > import os > +import shutil > import subprocess > import sys > import tempfile > @@ -83,8 +90,14 @@ def parse_args(): > help='Specifies the operating system') > parser.add_argument('-k', '--kernel', type=str, required=True, > help='Specifies the (uncompressed) kernel input file (.itk)') > - parser.add_argument('-r', '--ramdisk', type=str, > + > + # Create mutually exclusive group for ramdisk options > + rd_group = parser.add_mutually_exclusive_group() > + rd_group.add_argument('-r', '--ramdisk', type=str, > help='Specifies the ramdisk/initrd input file') > + rd_group.add_argument('-m', '--modules', type=str, nargs='+', > + help='List of module filenames to include in ramdisk') > + > parser.add_argument('-v', '--verbose', action='store_true', > help='Enable verbose output') > parser.add_argument('dtbs', type=str, nargs='*', > @@ -240,6 +253,89 @@ def output_dtb(fsw, seq, fname, arch, compress): > fsw.property('data', compressed) > > > +def find_gen_initramfs(): > + """Find the gen_initramfs.sh script > + > + Looks for usr/gen_initramfs.sh relative to the current directory, > + or in the source directory if running from a build directory. > + > + Returns: > + str: Path to gen_initramfs.sh if found, None otherwise > + """ > + path = 'usr/gen_initramfs.sh' > + if not os.path.exists(path): > + srctree = os.environ.get('srctree') > + if not srctree: > + return None > + path = os.path.join(srctree, path) > + if not os.path.exists(path): > + return None > + > + return path > + > + > +def build_ramdisk(args, tmpdir): > + """Build a cpio ramdisk containing kernel modules > + > + Similar to mkinitramfs, this creates a compressed cpio-archive containing > + the kernel modules for the current kernel version. > + > + Args: > + args (Namespace): Program arguments > + tmpdir (str): Temporary directory to use for modules installation > + > + Returns: > + tuple: > + bytes: Compressed cpio data containing modules > + int: total uncompressed size > + """ > + suppress = None if args.verbose else subprocess.DEVNULL > + > + if args.verbose: > + print(f'Copying {len(args.modules)} modules to ramdisk') > + > + # Create output-directory structure > + outdir = os.path.join(tmpdir, 'initramfs') > + modules_dir = os.path.join(outdir, 'usr', 'lib', 'modules') > + os.makedirs(modules_dir, exist_ok=True) > + > + # Copy in the specified modules, renaming .o to .ko > + for module in args.modules: > + name, ext = os.path.splitext(module) > + dest = name + '.ko' if ext == '.o' else module > + > + # Preserve directory structure under modules_dir > + dest_path = os.path.join(modules_dir, dest) > + os.makedirs(os.path.dirname(dest_path), exist_ok=True) > + shutil.copy2(module, dest_path) > + > + if args.verbose: > + print(f'Creating cpio archive from {outdir}') > + > + with tempfile.NamedTemporaryFile() as cpio_file: > + # Use gen_initramfs.sh to create the cpio archive if possible > + gen_initramfs = find_gen_initramfs() > + if gen_initramfs: > + subprocess.run([gen_initramfs, '-o', cpio_file.name, outdir], > + check=True, stdout=suppress, stderr=suppress) > + else: > + # Change to initramfs directory and create cpio archive > + with subprocess.Popen(['find', '.', '-print0'], cwd=outdir, > + stdout=subprocess.PIPE) as find: > + with subprocess.Popen(['cpio', '-o', '-0', '-H', 'newc'], > + stdin=find.stdout, stdout=cpio_file, > + stderr=suppress, cwd=outdir) as cpio: > + find.stdout.close() > + cpio.wait() > + find.wait() > + > + cpio_file.seek(0) # Reset to beginning for reading > + if args.verbose: > + print('Reading ramdisk...' if args.compress == 'none' else > + f'Compressing ramdisk with {args.compress}...') > + return compress_data(cpio_file, args.compress), cpio_file.tell() I don't think the fallback is a good idea as it duplicates logic we have elsewhere. Just t > + > + > def process_dtb(fname, args): > """Process an input DTB, decomposing it if requested and is possible > > @@ -318,11 +414,12 @@ def _process_dtbs(args, fsw, entries, fdts): > return seq, size > > > -def build_fit(args): > +def build_fit(args, tmpdir): > """Build the FIT from the provided files and arguments > > Args: > args (Namespace): Program arguments > + tmpdir (str): Temporary directory for any temporary files > > Returns: > tuple: > @@ -344,20 +441,29 @@ def build_fit(args): > > # Handle the ramdisk if provided. Compression is not supported as it is > # already compressed. > + ramdisk_data = None > if args.ramdisk: > with open(args.ramdisk, 'rb') as inf: > - data = inf.read() > - size += len(data) > - write_ramdisk(fsw, data, args) > + ramdisk_data = inf.read() > + size += len(ramdisk_data) > + elif args.modules: > + if args.verbose: > + print('Building modules ramdisk...') > + ramdisk_data, uncomp_size = build_ramdisk(args, tmpdir) > + size += uncomp_size > + > + if ramdisk_data: > + write_ramdisk(fsw, ramdisk_data, args) > > count, fdt_size = _process_dtbs(args, fsw, entries, fdts) > size += fdt_size > > - finish_fit(fsw, entries, bool(args.ramdisk)) > + finish_fit(fsw, entries, has_ramdisk=bool(ramdisk_data)) > > - # Include the kernel itself in the returned file count > fdt = fsw.as_fdt() > fdt.pack() > + > + # Count FDT files, kernel, plus ramdisk if present > return fdt.as_bytearray(), count + 1 + bool(args.ramdisk), size > > > @@ -365,7 +471,12 @@ def run_make_fit(): > """Run the tool's main logic""" > args = parse_args() > > - out_data, count, size = build_fit(args) > + tmpdir = tempfile.mkdtemp(prefix='make_fit_') > + try: > + out_data, count, size = build_fit(args, tmpdir) > + finally: > + shutil.rmtree(tmpdir) > + > with open(args.output, 'wb') as outf: > outf.write(out_data) > -- Pengutronix e.K. | | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |