Re: [RFC] Another take at restarting FUSE servers

[Luis Henriques <luis@igalia.com>]

On Mon, Aug 11 2025, Darrick J. Wong wrote:

> On Fri, Aug 01, 2025 at 11:15:26AM +0100, Luis Henriques wrote:
>> On Thu, Jul 31 2025, Darrick J. Wong wrote:
>> 
>> > On Thu, Jul 31, 2025 at 09:04:58AM -0400, Theodore Ts'o wrote:
>> >> On Tue, Jul 29, 2025 at 04:38:54PM -0700, Darrick J. Wong wrote:
>> >> > 
>> >> > Just speaking for fuse2fs here -- that would be kinda nifty if libfuse
>> >> > could restart itself.  It's unclear if doing so will actually enable us
>> >> > to clear the condition that caused the failure in the first place, but I
>> >> > suppose fuse2fs /does/ have e2fsck -fy at hand.  So maybe restarts
>> >> > aren't totally crazy.
>> >> 
>> >> I'm trying to understand what the failure scenario is here.  Is this
>> >> if the userspace fuse server (i.e., fuse2fs) has crashed?  If so, what
>> >> is supposed to happen with respect to open files, metadata and data
>> >> modifications which were in transit, etc.?  Sure, fuse2fs could run
>> >> e2fsck -fy, but if there are dirty inode on the system, that's going
>> >> potentally to be out of sync, right?
>> >> 
>> >> What are the recovery semantics that we hope to be able to provide?
>> >
>> > <echoing what we said on the ext4 call this morning>
>> >
>> > With iomap, most of the dirty state is in the kernel, so I think the new
>> > fuse2fs instance would poke the kernel with FUSE_NOTIFY_RESTARTED, which
>> > would initiate GETATTR requests on all the cached inodes to validate
>> > that they still exist; and then resend all the unacknowledged requests
>> > that were pending at the time.  It might be the case that you have to
>> > that in the reverse order; I only know enough about the design of fuse
>> > to suspect that to be true.
>> >
>> > Anyhow once those are complete, I think we can resume operations with
>> > the surviving inodes.  The ones that fail the GETATTR revalidation are
>> > fuse_make_bad'd, which effectively revokes them.
>> 
>> Ah! Interesting, I have been playing a bit with sending LOOKUP requests,
>> but probably GETATTR is a better option.
>> 
>> So, are you currently working on any of this?  Are you implementing this
>> new NOTIFY_RESTARTED request?  I guess it's time for me to have a closer
>> look at fuse2fs too.
>
> Nope, right now I'm concentrating on making sure the fuse/iomap IO path
> works reliably; and converting fuse2fs to be a lowlevel fuse server.

Great, thanks for clarifying.

> Eliminating all the path walking stuff that the highlevel fuse library
> does reduces the fstests runtime from 7.9 to 3.5h, and turning on iomap
> cuts that to 2.2h.

Wow! those are quite impressive numbers.  Looking forward to look into
those fuse2fs improvements!

Cheers,
-- 
Luís