Re: [PATCH RFC 0/4] erofs-utils: Add --meta_fix and --meta_only format options

[Hongbo Li <lihongbo22@huawei.com>]

On 2025/4/22 21:50, Gao Xiang wrote:
> Hi Hongbo,
> 
> On 2025/4/22 20:36, Hongbo Li wrote:
>> In this patchset, we have added two formatting options --meta_fix and
>> --meta_only to extend the ability of EROFS. In the case of using OBS,
>> we can convert the directory tree structure from OBS into the erofs
>> image and implement on-demand loading logic based on this. Since OBS
>> objects are often large, we need to separate the metadata area from
>> the data area, which is the reason we introduce the --meta_fix option.
>> To accelerate the formatting process, we can skip the formatting of
>> the raw data by adding --meta_only option.
> 
> Thanks for the patches!
> 
> I wonder if it's possible to reuse blobchunk.c codebase for
> such usage in the short term.
> 
Our initial plan was to reuse the blobchunk.c logic, but we found that 
the chunk-based layout has some minor issues when handling contiguous 
blocks—it would result in multiple elements in the chunk index array 
(whereas blobraw expects only oversized chunks).

Thanks,
Hongbo

> Also I hope there could be some better option knobs for these
> new features.
> 
>>
>> A simple usage example is as follows:
>>    1. Build one xattr with OBS key in s3fs.
>>    2. mkfs.erofs --meta_fix --meta_only data.img /mnt/s3fs to format
>>    3. Implement the loading logic in kernel or userspace.
>>
>> Based on the above logic, we can easily expose the directory tree
>> from OBS to users in the form of the EROFS file system and implement
>> on-demand data loading for large OBS objects.
> 
> BTW, It's possible to upstream OBS implementation to erofs-utils
> if any chance?
> 
> Thanks,
> Gao Xiang