From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8805E15278E for ; Sat, 18 Jul 2026 00:36:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784335002; cv=none; b=CZRSY58U8e0KLeojCCGPSP52btLSDTH0+uWfTrBDBkuWlGc4PsdPJoenxM0J+hRjT65Dof9XzE4qE00r756yp2ZLXmEmK65q1J0r/hgOVE+GYwe0H2CkrnRrNniEgDjAl+wzJJ0icOtRpj4ZN/2KkgvyAsFBfk12bNvj1dmimuA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784335002; c=relaxed/simple; bh=XCP9jvmDsfUIRNlp4XBmfZNhfw7LsC3rNmRbwFPPUrI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=bMakW/3tydQkvfXKftlBSYddr8md3F9Jag1LQSodNfK1qlfzCJLXu7sj3qlKAENKgJ14+kiUyGpslW0/FsBt7ZxVfcC7mCRBxP3uYlbXcX3wjPbcvNWYW6z1oQLXYolFrrn60tr8vZ7gWtLh+Iofm/6HsT+XFlB+n3JrEh5nkYk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=KJbbXrJm; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="KJbbXrJm" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-2ccdf36f63dso234705ad.0 for ; Fri, 17 Jul 2026 17:36:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1784335000; x=1784939800; darn=vger.kernel.org; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to:content-type; bh=Op9mpgY0d1zSc0ldTUhNOXDVKeO17oxSB/UFy9SPinU=; b=KJbbXrJmOsDbHm+VefARxMXFYHo5DYmaTlklk9yEtWHD/s0676jwvyE5aS7G8k9Vtj s/3MNAvOxXXMOYylzxQTw+818QZbop9VnweTsT076u1iDQk8pa+4bG15oILrdxouSwin 45DReJTFGWbvuVHEssSJCYOkYe16cTCxzHtQlWZPK8NjRr82+rdUog+m7E2EPHBrLgF+ FnKgnUY3zOiRcKtWyJMtOxaVI6qKn6VOPuClvxb4/RzyeK2R/HEmbq5kdf2ENsk2yIqG miNidDEZLA6+UQvRtppV898wfiFnU75/537LVzL0tYnpGFYsCZ62TY9CbRMMrHdjfJnx FdFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784335000; x=1784939800; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=Op9mpgY0d1zSc0ldTUhNOXDVKeO17oxSB/UFy9SPinU=; b=cCN2NXBr1mh++6RUffjNe0L88T9qIr0CjDv1QKRQTT6P2+Yhk+dDPigzJQ+CeE3BRt fJyBktZn09A8LDe/DwKuzECuYiaQvVbpUqsTYgUZFa5XelmsRBqcviKvYyqQv7bvNqXu YTOIrM7PDBHeFVQvdmlZFVxjf0bfn0Rt9PaJYZ7hV1bWy/qO8hqU4O9kz4RIAWgxHX// HZSlNuCD3DHGLfxnrq/cmIodFTarGpD+1NRVdrb6+evAcnZpxJ3QMfBAbLvE9rjzcmUQ 7Hd5Uo4tiNQSUY9vkIdjCNI2ynf0RQ+UkGTb4SlRiWmM/sCaTk7HN1+wtRnFC4fDj0A3 fyFg== X-Forwarded-Encrypted: i=1; AHgh+RqDz2S2yFr3t7UeSufj1ndwIhsYb+7ujHtuPV80psyAVW+2B6iLr1oxYXKgB5oTCR/ccq0clFXjJgu34GI=@vger.kernel.org X-Gm-Message-State: AOJu0YzaPXHrSzRckadXQVu6GqZ/4bQpgtyFvRQ/OSr7qqF7Q3DIk+xI l7rn/eP7JXZEushJdZaxy62VVKKbwsUjnuqVqaAaWp2p/Gv8EHtulNlx5NAPH3a4bw== X-Gm-Gg: AfdE7cmNqTDY7jSByDdSQxJSylSAEKg0Y2D01FIxJsfQpKw+Fm8zWp8J5dKjADjsNNO 9U/Y6zScsQPmMt0goIu/t06xvbOYLKHbZjjCgDtV8aPM8KCMRHzwLOxw+ZJqz+gt96tOe85JGzN 1tH93qUjSwHWioVpLqbxy0FxUpij4uVrO+pLruNaIaenIOgQC/AHjnlLa9yYjmtSMhgz4daH9y0 gZJWoKniI2nRY9pgBmdE31wp9tMW2KsPZL8bpP8E9u73NkAdyvwZQsrN/B1NEdUqMh86YrIuEIe sj+YYcJsPeQzjRWu/AQ1wE6bdUHeFqmpmp5x/fjjnT4iherQ7xqKfnFjjChtpmdExlueCE/VLld eB/Tqvjl3hJPST48WvMKaF1vcK1PRLB7/1OH9k2xJKq2BhJaM/xIqGoyn52nQD7tCVHsNl7CbL7 25pM5FcDWncIZywmTWot4TZchz86qxuTm+6bXNTJq+Rzti99S3agejGFBMxVv3MpVPvYmlj5nv X-Received: by 2002:a17:902:f295:b0:2c9:d89f:fd98 with SMTP id d9443c01a7336-2cf4a34a2f6mr629975ad.1.1784334999258; Fri, 17 Jul 2026 17:36:39 -0700 (PDT) Received: from google.com (168.136.83.34.bc.googleusercontent.com. [34.83.136.168]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2cf344f61ffsm19120875ad.33.2026.07.17.17.36.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Jul 2026 17:36:38 -0700 (PDT) Date: Sat, 18 Jul 2026 00:36:34 +0000 From: Samiullah Khawaja To: Pratyush Yadav Cc: Pasha Tatashin , Mike Rapoport , Alexander Graf , David Matlack , tarunsahu@google.com, open list , "open list:KEXEC HANDOVER (KHO)" , "open list:KEXEC HANDOVER (KHO)" Subject: Re: [PATCH 1/1] liveupdate: luo_file: Add internal APIs for file preservation Message-ID: References: <20260613012521.835490-1-skhawaja@google.com> <20260613012521.835490-2-skhawaja@google.com> <2vxzwlvljyzs.fsf@kernel.org> <2vxzse65k938.fsf@kernel.org> <2vxz7bnhjnpa.fsf@kernel.org> <2vxz4ihxil6m.fsf@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <2vxz4ihxil6m.fsf@kernel.org> On Fri, Jul 17, 2026 at 07:24:17PM +0200, Pratyush Yadav wrote: >On Mon, Jul 06 2026, Samiullah Khawaja wrote: > >> Hi, >> >> Sorry I was out of office, so couldn't contribute to this discussion >> early. >> >> On Mon, Jun 29, 2026 at 06:50:09PM +0200, Pratyush Yadav wrote: >>>On Mon, Jun 29 2026, Pratyush Yadav wrote: >>> >>>> On Mon, Jun 29 2026, Pasha Tatashin wrote: >>>> >>>>> On 06-26 13:57, Pratyush Yadav wrote: >>>>>> Hi Sami, >>>>>> >>>>> [snip] >>>>> Actually, preservation can also be performed in an order-independent manner. >>>>> While a handler can call liveupdate_get_token_outgoing() during .preserve(), >>>>> it can also defer this query until the .freeze() callback. Because .freeze() >>>>> is invoked after all files in the session have completed their .preserve() phase, >>>>> all dependency tokens are guaranteed to be available, completely eliminating any >>>>> topological ordering requirements during the initial preservation calls. It is >>>>> up to individual file handler implementations to decide whether they wish to >>>>> enforce ordering at .preserve() time or defer it to .freeze(). Quoting this text from pasha below. >>>> >>>> That is the worst of both worlds. I get your point that LUO doesn't want >>>> to enforce dependency ordering. My arguments against that are somewhat >>>> subjective so I can live with this. >> >> Pasha replied with some interesting points already, but I want to add >> some clarification here. >> >> During preservation, enforcing order gives the following functionality: >> >> - The token of the dependency (can be retrieved during freeze as you >> suggested). >> - The dependency is preserved and that means it has bound with the LUO >> session lifecycle. This guarantees that it is not going to go away >> until the session is closed. >> - Once preserved, the dependency is in some kind of "immutable" state. >> This might not be required by all dependent FDs, but it is critical >> for some. >> >> These ordering requirements should be clearly documented by the >> associated filehandler so the VMM knows how to do the preservation of a >> specific FD properly. This is actually similar to the memfd seal rule >> that iommufd preservation will enforce. >>>> >>>> But then you can't let file handlers enforce it as they wish. The >>>> dependency ordering is uAPI because it directly affects how VMMs >>>> preserve files. If the VMM has to keep track of dependencies for some >>>> file types and doesn't have to do so for others, that is a terrible and >>>> inconsistent API. >> >> But as you have already pointed out that the VFIO/IOMMUFD circular >> dependency is resolved, I am okay with enforcing dependency ordering >> during restore as well. However, establishing an ordered >> preserve/restore mandate in LUO at this point will force all future file >> handlers into complicated and buggy design choices if they have circular >> dependency or different lifecycle requirements. > >Not really. You can always _relax_ the ordering requirements if a need >does come up. Because all the programs following the ordering will still >continue to work. The other way round won't work though. Yes, this is a fair point. > >>>> >>>> Ideally, LUO should handle the dependencies on its own. preserve() can >>>> give LUO a list of files the preserved file depends on, and LUO makes >>>> sure all the dependencies are present in the session at freeze. We would >> >> This again assumes the lifecycle of various FDs and that dependencies >> can be resolved during freeze(). Some file handlers do want order of >> preservation, and this scheme only guarantees preservation dependency >> without order, introducing hacky dependency state management until >> freeze(). >>>> also need a way of getting the dependent files back from LUO on >>>> retrieve(). That would make sure the dependencies are properly enforced >>>> both on freeze and finish, and the enforcement isn't left up to the file >>>> handlers. >>>> >>>> Unfortunately all that sounds fairly complicated so I am not sure if we >>>> want to do that just yet, although I would like to hear your thoughts on >>>> this. >>> >>>We had discussion about this in the live update bi-weekly today. The >>>conclusion we arrived at is to keep the current functionality. That is, >>>we don't enforce preservation dependency. >>> >>>But that also means file handlers can't try to get their dependent file >>>in their preserve() callback, since that would implicitly enforce >>>ordering. They always _have_ to do it from their freeze() callback. >> >> This has problems at multiple levels: >> >> - This indirectly sets up a precedent in uAPI that the VMM is allowed to >> preserve the FDs in any order and the file handler will be able to >> handle this. >> - This will be very tricky and will force filehandler to have >> complicated/hacky state management to handle dependency. Specifically >> I do need iommufd to be preserved before preserving the VFIO cdev that >> has an iommufd dependency. Not sure, but I think this will also be >> tricky when we start preservation of VFs and SRIOV PFs. >> >> I don't think LUO should enforce an arbitrary rule like this unless we >> have a strong reasoning behind it. > >It looks like you, me, and Pasha are saying slightly different things. >We all first need to take a step back and lay out our proposals clearly. >I'll write my points down, and I'll try to reproduce what I understood >from discussions with you and Pasha. Thanks for laying down your points clearly. I'll reply to point 1 and 3 together as those are entangled I think. > >I think we should focus discussions around the below 3 high level topics, >and we can figure out the implementation details later. > >1. Do dependent files need to be preserved before their main files? >2. Do we apply the same rules to preserve() and retrieve()? >3. Do the rules apply to all file handler or can file handlers choose > their rules? > >Here's my take for 1: Say A depends on B. I think it would be a good >idea for LUO to enforce that B gets preserved before A. You already >mentioned a few reasons for why. Similarly, LUO should also enforce that >B gets restored before A. We can figure out where the actual enforcement >happens, but that would be the principle of the LUO API. > >Pasha suggests the opposite. In my discussions, and this email thread, >he doesn't want to enforce _any_ ordering. So you can preserve A and B >at any time and you check at freeze() that everything is present. Now I Just to correct/clarify, quoting Pasha from his earlier email in this thread: "It is up to individual file handler implementations to decide whether they wish to enforce ordering at .preserve() time or defer it to .freeze()." Basically, a FH can choose to enforce ordering during preservation or not enforce any ordering and check it during freeze(). I agree with Pasha on this point, as it gives FHs the flexibility they need. However, I see the value in maintaining consistency and simplicity for the VMMs, we can enforce that dependencies are preserved in order as a principle of the LUO API. I am ok with this. It might become tricky for any future circular dependencies down the road, but we can relax this if they come up, as you suggested above. >see the value in this idea. It removes the need of tracking ordering >from VMMs. But I think it is also somewhat dangerous, because what >happens when freeze() fails? VMMs need to recover and restart the VMs or >retry live update. I don't know if they would be able to do so. This >path will likely be less tested. > >But still, I can live with that. At least for memfd, guest_memfd, etc. >this shouldn't be very problematic. I am not sure about iommufd. > >So in conclusion, I would prefer to have ordering, but I can live with >no ordering as well. As elaborated in my previous email, I disagree with enforcing a no-ordering policy. > >For 2, I think we should apply the same rules on both sides. Mainly for >consistency. The VMMs should follow the right order for preservation and >restoration. Now again, I am fine with both enforcing the ordering or >not enforcing it, but I really think the rules should be consistent >across both sides. > >>From my reading, I think Pasha does as well, but from talking with you, >IIUC you think retrieve is a fundamentally different operation and it >should not enforce any ordering. > >I do see your point but I still think there is value in keeping things >symmetric. Unfortunately this is fairly subjective, so I don't have much >better arguments. I agree with you on this. For consistency and simplicity, we can apply the same rules on preserve/retrieve both. > >For 3, I _strongly_ think it should be a LUO wide policy. Leaving this >up to file handlers will become messy over time because each would come >up with its own rules and for VMMs there would be no consistency in how >they should use LUO. > >I hope this helps focus our discussion to the major points. And again, >we can figure out the implementation details as we go along, so I didn't >reply to some of the points you raised in this email, to not distract >too much. > >[...] > >-- >Regards, >Pratyush Yadav Also I think with the next revision of this series, if Pasha also agrees with this, I will add documentation about FD dependency in general. Thanks, Sami