From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qv1-f51.google.com (mail-qv1-f51.google.com [209.85.219.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79892371066 for ; Wed, 28 Jan 2026 19:59:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.51 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769630387; cv=none; b=l0eQShhQmkjSWNNPfbnmSOHDCLKgYz+qq8K1XpDdVKy/lu7JBmCV/YAj8u59aksqc5xZ/vYzUrCmULsDT1t94MaFISswQovCAERS5tLHcqHh8VlM4V2ZWru5PkKCSrD9DlLOO8i82ffRLD+gI4n6PMd4lexfkrWDDpcCn5tm5Iw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769630387; c=relaxed/simple; bh=INVF0NKpfeA9Zqk17ZaQUkV6i2PcpJndfTdw11nI7f4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=kwrsyPe1ze7ma6BvniyEDSkT7l9jCsGs6R1Sn0DhXv4TArKAFBzin9s1am0RiPZajJNdzVrOUWiVY3+ybp5TefsBDnzE4e/2YG4r9zaNCPEwxE5hrjhYfQtZ0AXIVjduZcMtPMilfuk416PeTg8kpbVy90WVxzUmj5alApUzyP0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca; spf=pass smtp.mailfrom=ziepe.ca; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b=CcabCQeT; arc=none smtp.client-ip=209.85.219.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="CcabCQeT" Received: by mail-qv1-f51.google.com with SMTP id 6a1803df08f44-8946e32e534so2568006d6.3 for ; Wed, 28 Jan 2026 11:59:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1769630384; x=1770235184; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=JYo9G/VcbU/Bswn8IpEHmUA4exilaj26AXaG+5ltVBw=; b=CcabCQeTgtloJsg9M9XqwWr5RM77JidblLiwWBDY3wgCX+GVBhbg1aEsvWtbHEiddF ODvR1bsjnNuoeqsBSdiu5C2wlYbgRMfqUBDqRILyZnuDkP53W4OpROdqpoSzy9U4PiGk 2CTorsgWdawe0CZLrSipbpDUyAwDUmU558JwrH/NsseXhCxxadmkwbHm3Lp35GrFHhJm /O+4rtGdfFckU/TgYmmtXkj2mdCYlc1dNGZRLxT2RaTk8LAIuTDWnDXAE9SklDTiVBs8 ZfEhUTWvRWGUJFVvowRafGhzji/+5nsq3n5Wtru648gIJlm9GBbXzn4uVPpyYtgoDkCe oWvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769630384; x=1770235184; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JYo9G/VcbU/Bswn8IpEHmUA4exilaj26AXaG+5ltVBw=; b=KzM9t7QprZwnbztaNypkmgwDq7g6Vbm5jadMCVEuODbBi1pLpE4nPq97wmSQfL6q0e b20HvJJPv82Z8DNzDSD8Zpvs/6lHdF3aUSpHiFBACSubecqVE1UJiXTNLXZ1IAGVYpKw 0aSW317uZAE5N/t2Q6ZrHxFLwnGBTBM0UFVZbTZfVG64myq3pSRa2e3/PQlV3zaG9nzS 2vLo43l2ya+ALUHFYG4ToaR6NuyhvUp+U9sroEZOitpx6oEEvY4z+DseHs3I9yiClEZl Uv4MT952I2IdPe6OCrqx3iS/JpKSI08QF2FfWaW0zSIfcmd4XUdWIXtWYdIdUtCSKNii jkow== X-Forwarded-Encrypted: i=1; AJvYcCW6rBddLsFjCCT1E2P9FcSwxTvaXbt5oQ7usGe7SC2jELtiwh/JfmBx5boJQR29OQc9sGe1nQkRIW9q1J8=@vger.kernel.org X-Gm-Message-State: AOJu0YxC9jS5EH1iPHuj9JErq+92UnlSJhkyQoFeMt7xllfjoKoLOajH AJiwqnoFZGTasptDdB+CtutEWRbC4IjthYwnIColKvNSqV21QigGV9HrSig1p70DmfA= X-Gm-Gg: AZuq6aKLxzFV9XFzHmbErQBQv0uJy2PAW2i4CQOikoBRXpYZv8QklCH+bszDWjGcyDx OTc7P0xxwGl6M8J5pAeYm/pWqGKHUIipmG1MC4M8Vk0Fb5TUc+er4olbx2bxD/V9L3PxYO6QQM4 JauJEBSHNuMKDjQX6cwQSYHHuJ3WTX8QASZeyoNI0vHLiMkfrBZBLXOXI8PxpaEgRsDLaMo227H v3A5ET28k/GFbe/sVAlGTPh0mo3akaGEy3h59zdZJy/CB0hUGzEzEktvU94WrCowPqi0lMhLIqp f2F74NKWMUikjYwLanoqiqdBf8qFkh9rHA2MOUIlBE6NT1Ns8xr1HkngzT/9vfiGX1Ex9m1QPg6 PDL55UuwzQz9umaHOwVkv7nM+9qnk+m9jWWaE2MKFkP6f7DIcw7Yf8/IufF+SVDBm5JfaMeCvRw 3dk06oP2/nwUG6JlrA+3x1jJ1mAPGr5kDp8A5g74N8qurSGY18xyFjzSVGRauXLU3llp8= X-Received: by 2002:ad4:5ce9:0:b0:880:5883:4d23 with SMTP id 6a1803df08f44-894cc7f5461mr96138006d6.16.1769630384364; Wed, 28 Jan 2026 11:59:44 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-162-112-119.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.162.112.119]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-894d375f3casm23658566d6.47.2026.01.28.11.59.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Jan 2026 11:59:43 -0800 (PST) Received: from jgg by wakko with local (Exim 4.97) (envelope-from ) id 1vlBhX-00000009es5-18WU; Wed, 28 Jan 2026 15:59:43 -0400 Date: Wed, 28 Jan 2026 15:59:43 -0400 From: Jason Gunthorpe To: Samiullah Khawaja Cc: David Woodhouse , Lu Baolu , Joerg Roedel , Will Deacon , Pasha Tatashin , iommu@lists.linux.dev, Robin Murphy , Pratyush Yadav , Kevin Tian , Alex Williamson , linux-kernel@vger.kernel.org, Saeed Mahameed , Adithya Jayachandran , Parav Pandit , Leon Romanovsky , William Tu , Vipin Sharma , dmatlack@google.com, YiFei Zhu , Chris Li , praan@google.com Subject: Re: [RFC PATCH v2 00/32] Add live update state preservation Message-ID: <20260128195943.GY1641016@ziepe.ca> References: <20251202230303.1017519-1-skhawaja@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20251202230303.1017519-1-skhawaja@google.com> On Tue, Dec 02, 2025 at 11:02:30PM +0000, Samiullah Khawaja wrote: > > Samiullah Khawaja (26): > iommu: Add liveupdate FLB for IOMMU state preservation > iommu: Register IOMMU FLB with iommufd file handler > iommu: Implement IOMMU LU FLB preserve/unpreserve callbacks > iommu: Add iommu_domain ops to preserve, unpreserve and restore > iommu/pages: Add APIs to preserve/unpreserve/restore iommu pages > iommupt: Implement preserve/unpreserve/restore callbacks > iommu: Add APIs to preserve/unpreserve iommu domains > iommufd: Use the iommu_domain_preserve/unpreserve APIs > iommu: Add API to keep track of iommu domain attachments > iommu: Add API to preserve/unpreserve a device > iommu/vt-d: Implement device and iommu preserve/unpreserve ops > iommufd: Add APIs to preserve/unpreserve a vfio cdev > vfio/pci: Preserve the iommufd state of the vfio cdev > iommu: Add APIs to get preserved state of a device > iommu/vt-d: Clean the context entries of unpreserved devices > iommu: Implement IOMMU FLB retrieve and finish ops > iommu: Add an API get the preserved state of an IOMMU > iommu/vt-d: restore state of the preserved IOMMU > iommu: Add helper APIs to fetch preserved device state > iommu/vt-d: reclaim domain ids of the preserved devices > iommu: restore preserved domain and reattach > iommu/vt-d: reuse the preserved domain id for preserved devices > iommufd: Handle the iommufd can_finish properly > iommu: Transfer device ownership after liveupdate > iommu: Allow replacing restored domain > iommufd/selftest: Add test to verify iommufd preservation > > YiFei Zhu (6): > iommufd: Allow HWPTs to have a NULL IOAS > iommufd: split alloc and domain assign from iommufd_hwpt_paging_alloc > iommufd: Add basic skeleton based on liveupdate_file_handler > iommufd-lu: Implement basic prepare/cancel/finish/retrieve using > folios > iommufd-lu: Implement ioctl to let userspace mark an HWPT to be > preserved > iommufd-lu: Persist iommu hardware pagetables for live update This really needs to be made smaller and more focused to be reviewable and mergable. Try to stick to 15-ish patches. There is a lot giong on here, I suggest focusing this only on the main iommu core, a vt-d driver implementation of the bare minimum, and an iommufd function to trigger preservation of a domain. Start off by just making the successor kernel fail to accept any drivers at all because the iommu_domain was preserved. ie restore the domain and set it as the default_domain and then fail in iommu_device_use_default_domain() and related functions. All it should do is convey an iommu_domain and the pasid table entry unchanged without hit from prior to successor kernel. That's the bare minimum. Then a second series which picks up from there and feeds the sucessor path through iommufd/vfio. Jason