From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A2FC3D5642 for ; Fri, 14 Aug 2026 19:46:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786736767; cv=none; b=cJPKi5w/GTonWXVYcWH6hAtGv9IcV8gzg1Ssb5L/f8UWXCWFXrZAzWed1VSsmIDJFpK5Xn2daos7HOI7vq3hgxcQwZJvI9nEng7vOECEhxqCjuDG1zrnGQggQlpqHLL1QFXUn+j1WoDT6qq7oYoE8nSiPCiGrpmkKRD6jC8wypw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786736767; c=relaxed/simple; bh=baOJZPd7nJlBagSvu4EY/e0qB3OgSpSkb6F80UmuE5I=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=YqXD306USfxpTY7u/24YkaK4+Q5wgnEhdpPZnGzdqFuEYjwyqRWZsDgI25iP40u+rf83DSqc7TNQ7WFjSJoRbPNwA9VJONNkJerPqEl/m7ZfR6RMgP94UUnF9zxFDrKAcGhwM69r5ppIJNAM0JunoojcmuMfjnAlOvc3dkK9Qyc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=mfMkq1dV; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="mfMkq1dV" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-2d3b445a84fso126895ad.1 for ; Fri, 14 Aug 2026 12:46:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786736765; x=1787341565; darn=vger.kernel.org; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to:content-type; bh=S64AgIWO7QxYO2SGcf+4LpPDoMjsF/PwjvLqa2d2Geo=; b=mfMkq1dVA4S4J8SNbiRKDV3R241tM+IHBOMv9IU/o1WDIm7Iwv4lvlBZOrmin63rxY vM+Jeot8v8LWcvNdu6ybHSWlmvEfMXxpUX2NRKgawxBILFLxLZGdLLmE0mpNncZ//SQL +Kkrgzb+MDHoHySCMgqOIVP4jqwLXom1Hdo7NGreHRMBkRHMil8ClSuwBHBxQItUXeKJ EqtTmAhITKmZxAQ3XTlcwsBCkid0E/9OBDfD/fKTXMTG1yjfD6l4I2oG4FjiURYl84WR FXmZd5Z7DZnXoJvan7N7g+3mH+xRXk5me1jSseeGBwwUAbPKVfrPmmgE70OzyyymyW0J DERg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786736765; x=1787341565; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=S64AgIWO7QxYO2SGcf+4LpPDoMjsF/PwjvLqa2d2Geo=; b=ZM1fhGSGbXupHczjNj61O517zs7Q91/tdP+4nH8maxEf55n24zBJKjoku3qqtogYHI fUJFVnT+13N5cy8hMVKfsFplQTxIhDQCHN9uoLVhW26Zory1ouziOO3CsYXjBlwnUfPN do9gY5QK4tE5d2KZukXbrR7DtaPbmFx9Gpp4o66QVhxva47Qu/OHIZI5Wc7HxfsnALLK OYs9lkgj/HL6zYPfzHg469nruG4pMEwJlWCM6vZkE/Kw0OgXkPY+GBDm5RNW0/fEinrO Yha1Iao8O1HncH7HrKAMqsVo9nmz7BtyBIKJ7QbrcNHVBmmO45ZGhXXnQzMy2m0CWWfR O6LA== X-Forwarded-Encrypted: i=1; AHgh+Rqm8pa5uzOF1+szsokZMG6U+/QnsHko2yJloqKDakJ4c9WpvxEH8d9N7zrUxeTsq+XMoVSL0yH3RDjkSuI=@vger.kernel.org X-Gm-Message-State: AOJu0Yz6Gr7fKlyj1NRcBCZLJlvpQULvJEuaoGQ6p/99kVip7cewt0i6 YW8M3tz+SHmpCMkMd9mleJ/NlMEfrR1f8ZzXSlNL9aW2ZgKziErDZmG5C2UoTHrZtA== X-Gm-Gg: AR+sD12eQwtrGB84X10jJ2YARSsMTyTUNTQ5lZVzUb3bshz2I3OlfQdhl+1RrXaCB1+ +cSmvKzhGxtgi67ls9S0vwIuF354JHAPemAueLipotjMNpgo1M3i6d54qKjFOFc6ZGH1wM9mWh7 nlwjULo6n/NUHaoDf6SJLetUPmOkuvwQfYGb3tIbvLLiC/vXHQMrsjZy9TPplNs3qS25dr7EtOD xt0pjLl8T0qO2GXoLAJSr6zkKU3BZ7S9MbG2R0eIc2f9Go88ZgskcRDNupPaIUIKN3JLqC/eNVO w4nZ6iOzoA2/oOWlPGfNjaVsz8iEH/D5J6AluHokPc776jT37kk9ebgTlUJ1ovxpjfKipcmavCz fpdrKbaA09BHSKkfHp08D5Uaks9ggghsRIk7q8z+B3VXZw+0ST/0xdMjeIkCs1UZBOcUX8F4A8+ jDiWHUq1Oqw+vgUBjoGpVLdbhikhttkZPo3DcI5GKvTjhNo6xSoTAdmr73kbMeP7pnWxClOJI44 nzf2gn3wftOg/vsuXfak5y6dxAOdClkysKcqdYm7SsuyQwloScC9zw3JNCB97ue X-Received: by 2002:a17:903:1b0f:b0:2cf:41ba:96b8 with SMTP id d9443c01a7336-2d3af229eb1mr12458355ad.6.1786736764793; Fri, 14 Aug 2026 12:46:04 -0700 (PDT) Received: from google.com (210.87.127.34.bc.googleusercontent.com. [34.127.87.210]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-394ea99ab1dsm3742041a91.7.2026.08.14.12.46.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Aug 2026 12:46:04 -0700 (PDT) Date: Fri, 14 Aug 2026 19:46:01 +0000 From: Samiullah Khawaja To: Ankit Soni Cc: David Woodhouse , Lu Baolu , Joerg Roedel , Will Deacon , Jason Gunthorpe , Robin Murphy , Kevin Tian , Alex Williamson , Shuah Khan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Pratyush Yadav , Pasha Tatashin , David Matlack , Andrew Morton , Pranjal Shrivastava , Vipin Sharma Subject: Re: [PATCH v4 11/18] iommu: Restore and reattach preserved domains to devices Message-ID: References: <20260808022723.3893618-1-skhawaja@google.com> <20260808022723.3893618-12-skhawaja@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: On Fri, Aug 14, 2026 at 04:59:50PM +0000, Ankit Soni wrote: >On Sat, Aug 08, 2026 at 02:27:16AM +0000, Samiullah Khawaja wrote: >> During default domain setup, restore the preserved domains by restoring >> the page tables using restore() iommupt op. Associated the restored >> domain with the iommu group of the preserved device, and reattach the >> domain to the device. >> >> Signed-off-by: Samiullah Khawaja >> --- >> drivers/iommu/iommu.c | 76 ++++++++++++++++++ >> drivers/iommu/liveupdate.c | 130 +++++++++++++++++++++++++++++++ >> include/linux/iommu-liveupdate.h | 69 ++++++++++++++++ >> 3 files changed, 275 insertions(+) >> > >../.. > >> diff --git a/drivers/iommu/liveupdate.c b/drivers/iommu/liveupdate.c >> index 20acf123b47a..04c0212cd81b 100644 >> --- a/drivers/iommu/liveupdate.c >> +++ b/drivers/iommu/liveupdate.c >> @@ -708,3 +708,133 @@ void iommu_unpreserve_device(struct iommu_domain *domain, struct device *dev) >> liveupdate_flb_put_outgoing(&iommu_flb); >> } >> EXPORT_SYMBOL_GPL(iommu_unpreserve_device); >> + >> +static inline bool match_device_ser(struct iommu_device_ser *match, >> + struct pci_dev *pdev) >> +{ >> + return match->devid == pci_dev_id(pdev) && match->pci_domain_nr == pci_domain_nr(pdev->bus); >> +} >> + >> +/** >> + * iommu_init_device_preserved_data() - Initialize preserved state for device >> + * @dev: Target device >> + * >> + * Looks up incoming Live Update state for @dev and attaches it to the device if >> + * found. >> + */ >> +void iommu_init_device_preserved_data(struct device *dev) >> +{ >> + struct iommu_device_ser *device_ser = NULL; >> + struct iommu_device_array_ser *array; >> + struct iommu_flb_obj *flb_obj; >> + int ret, idx; >> + >> + if (!dev_is_pci(dev)) >> + return; >> + >> + ret = iommu_liveupdate_flb_get_incoming(&flb_obj); >> + if (ret) >> + return; >> + >> + mutex_lock(&flb_obj->lock); >> + array = phys_to_virt(flb_obj->ser->device_array_phys); >> + iommu_liveupdate_for_each_arr(array) { >> + iommu_liveupdate_for_each_obj(array, device_ser, idx) { >> + if (match_device_ser(device_ser, to_pci_dev(dev))) { >> + device_ser->hdr.flags |= IOMMU_SER_FLAG_INCOMING; >> + goto out; >> + } >> + } >> + } >> + >> + device_ser = NULL; >> +out: >> + WRITE_ONCE(dev->iommu->device_ser, device_ser); >> + mutex_unlock(&flb_obj->lock); >> + liveupdate_flb_put_incoming(&iommu_flb); >> +} >> +EXPORT_SYMBOL(iommu_init_device_preserved_data); >> + >> +/** >> + * iommu_release_restored_device() - Release a restored device >> + * @dev: Target device >> + */ >> +void iommu_release_restored_device(struct device *dev) >> +{ >> + /* >> + * We do not support releasing the restored devices that are not >> + * reclaimed by the device drivers as they can fallback to the default >> + * domain. >> + */ >> + BUG_ON(dev_iommu_restored_state(dev)); > >Hi, Hi, Thanks for looking at this. >After a successful live update this is one sysfs write away, and nothing in >the series disarms it. > >At PCI probe, iommu_init_device_preserved_data() matches the incoming FLB on >devid + pci_domain_nr and sets IOMMU_SER_FLAG_INCOMING. >Nothing clears the flag or device_ser afterwards. The group is meanwhile owned >on behalf of iommufd (iommu.c:3229-3231, "will be reclaimed later by the >entity (iommufd) that preserved them"), and iommufd_liveupdate_retrieve() is >-EOPNOTSUPP, so the reclaim that would end the restored state cannot happen >yet. The device is left with the state permanently set. Yes, these points are valid and this is intentional. The IOMMU persistence support is split into two phases as mentioned in the cover letter. The reclaim logic in iommufd will come later as a phase 2. The preserved devices go to normal state when these are reclaimed through iommufd. https://lore.kernel.org/all/20260128195943.GY1641016@ziepe.ca/ Regarding the handling of sysfs, my concern is about it coming back and going to default domain as mentioned in the comment. But I will evaluate if we can allow device tear down here and reattach it to the preserved domain. Also please see the patchset breakdown here: https://docs.google.com/document/d/1enDn-uPE9U77U-xHEnzn6HHGKiePSAtMIP8EDU3NO0M Btw since this phase 1 is relatively stable and I don't expect many changes in it. I have started reviving the Phase 2 patches and will be sending them out as RFC soon if you want to experiment with it. Sami