Re: Executable loading issues with erofs on arm?

[Jan Kiszka <jan.kiszka@siemens.com>]

On 08.07.25 17:12, Gao Xiang wrote:
> Hi Jan,
> 
> On 2025/7/8 20:43, Jan Kiszka wrote:
>> On 08.07.25 14:41, Jan Kiszka wrote:
>>> Hi all,
>>>
>>> for some days, I'm trying to understand if we have an integration issue
>>> with erofs or rather some upstream bug. After playing with various
>>> parameters, it rather looks like the latter:
>>>
>>> $ ls -l erofs-dir/
>>> total 132
>>> -rwxr-xr-x 1 1000 users 132868 Jul  8 10:50 dash
>>> (from Debian bookworm)
>>> $ mkfs.erofs -z lz4hc erofs.img erofs-dir/
>>> mkfs.erofs 1.8.6 (trixie version, but same happens with bookworm 1.5)
>>> Build completed.
>>> ------
>>> Filesystem UUID: aae0b2f0-4ee4-4850-af49-3c1aad7fa30c
>>> Filesystem total blocks: 17 (of 4096-byte blocks)
>>> Filesystem total inodes: 2
>>> Filesystem total metadata blocks: 1
>>> Filesystem total deduplicated bytes (of source files): 0
>>>
>>> Now I have 6.15-rc5 and a defconfig-close setting for the 32-bit ARM
>>> target BeagleBone Black. When booting into init=/bin/sh, then running
>>>
>>> # mount -t erofs /dev/mmcblk0p1 /mnt
>>> erofs (device mmcblk0p1): mounted with root inode @ nid 36.
>>> # /mnt/dash
>>> Segmentation fault
>>>
>>> I once also got this:
>>>
>>> Alignment trap: not handling instruction 2b00 at [<004debc0>]
>>> 8<--- cut here ---
>>> Unhandled fault: alignment exception (0x001) at 0x000004d9
>>> [000004d9] *pgd=00000000
>>> Bus error
>>>
>>> All is fine if I
>>>   - run the command once more
>>>   - dump the file first (cat /mnt/dash > /dev/null; /mnt/dash)
>>
>> Forgot to mention: That first dump when done via md5sum or so actually
>> gives the right checksum. So pure reading of the binary is also ok, just
>> trying to load it for execution fails on the first attempt.
> 
> Thanks for your report.  I rarely take care arm32 platform
> because I don't have such setup.
> 
> but could you share a reproducible rootfs image and
> I wonder if qemu could reproduce this?

The image can be generated from isar-cip-core
(https://gitlab.com/cip-project/cip-core/isar-cip-core), bbb image with
swupdate extension and erofs as immutable rootfs. As I wrote, those will
be 6.12 or 6.1 based, but I also injected a mainline kernel into that
with the same result. But all that only helps if you have some
beaglebone black in reach right now.

The same configuration, just for qemuarm as target, unfortunately does
not reproduce the issue.

> 
> Otherwise it's hard for me to debug this issue...

If you tell me how I could do that, I'm happy to instrument and analyze.
I just have no understanding of erofs yet, specifically how reading
files might be different from loading executables.

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center