From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54BC6385D69 for ; Sun, 14 Jun 2026 23:37:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781480262; cv=none; b=Sp51aNfuJgbWEq13ZkQW6P8LC1AVt4be5boAR10RSBGolkv3/7uJnJQtfo+8hipP+GfQqfSXg9scZA/NAZaltXrUsDOa7zquXQCqymjdawomiRYxa54ipQO7VgB5keOH0Ms+gSuUNXFSAD3ZUSxjhF0XXwjBCMO2y6O0pJlmvzI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781480262; c=relaxed/simple; bh=gJcCWuSa2tPgXxQm52WqQaf0NsMtzV5lBuq1/jQnXAU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=YubUt+ltVL5/5icpJG4N2g5efc1/0sNFCvgYdNO2oEDIw78LMRdLw4Tuy/+p5Au7Ihf2kSxbZIUFTlI9pQ6lCt/1Y1W6WdK8GJUECBtIYWBszN/2oCZ9Yq/2dLsKNaR3B574CeFF4CGr3j3vFGT1lv0zaVgf72Y/nWxxnNhGom4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--skhawaja.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EzQjJi9S; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--skhawaja.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EzQjJi9S" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-36b808bedfaso3789265a91.1 for ; Sun, 14 Jun 2026 16:37:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1781480259; x=1782085059; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rTbSdIBLoQ3ymz5Oo8C6SttlitRISpaU0y1we7JmTLI=; b=EzQjJi9SJkbfjW20Pi81Ug+Uia/oJZ3NkOdU2+8wB/UOvk5vwXlZPkl89KNDr64GWq VdOu7o7U//A8ukklZbJoZlIG9sl+rk0iu2OCFNhvRgOFK/tR4VnC9PgyxIkWW1eROM5D mboN5ZOFXfIYfeLvAqChUJVMhA18/H3B75r+MtJ5jRhNfcafvb1qeyt8NgjCS6uh7D0p 0/KdinZyOsm9Wm7/OCUmYfcKfCtqPMhAW20+5bueJghVI4LfP5nGiruE9bUC6dzRIiro efk30xDg03HX5mqG0k2quTPTb07n+GtGxdCqGUjZUb0ueVUPSgf/UVyfHnl7SDOt6uy+ 0CHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781480259; x=1782085059; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rTbSdIBLoQ3ymz5Oo8C6SttlitRISpaU0y1we7JmTLI=; b=kuEw8fok7tX6aSfejn0DL8JRTVDYN0BjOgl94sw5UtiLvszSGDyveQJHYhphidzSkm 6lj+NZGJg/OeWL7+BM1lI2EnukfVYKQSKhHflo1gTdVljmXtqPvDDWu5jdOipCMOIiOy P5c3PmesA5ygNCbwqei9N7rxdJRW/z6NIMOXiwL4tOsn4xN6BlJKj000Mj0P6s24MSCn OcfGEdg3UYaBxJgTMogBz+fARLw3uh92HOAWbkB20GjNAFppILGp7Sqhsv9PXhj/Nhym paZjtK2JfnsbA8QEX0v352XNAKUFyETbhFgcK+/HJ0BHoMBgQFVFBFCRpPHrI0oofk0E e6Ww== X-Forwarded-Encrypted: i=1; AFNElJ+0ei7U8lh0SZuqCyi/V5/5aZhT/5Glj6gOYIhtPifnt4FIKCSMRmXn7B5EbK9xXV6JZyQIx4TBptVx2Lk=@vger.kernel.org X-Gm-Message-State: AOJu0YxnwYYNd3aEVf9VyI+wrWOQHupZp0zUCQzquJss9Bs5bJXRC8G+ tji8sFkCVvDkxy7DkicObWHu8vpangybHJPiui01QX/Jki9Gv0LIiv6/FZOD5XPFYx6i8G5D7SE RzBetG9gASOcjwg== X-Received: from plblv16.prod.google.com ([2002:a17:903:2a90:b0:2c0:bd65:1a98]) (user=skhawaja job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ebc3:b0:2c2:27be:39a8 with SMTP id d9443c01a7336-2c66428edefmr91597125ad.31.1781480258491; Sun, 14 Jun 2026 16:37:38 -0700 (PDT) Date: Sun, 14 Jun 2026 23:37:21 +0000 In-Reply-To: <20260614233728.2212104-1-skhawaja@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260614233728.2212104-1-skhawaja@google.com> X-Mailer: git-send-email 2.54.0.1136.gdb2ca164c4-goog Message-ID: <20260614233728.2212104-12-skhawaja@google.com> Subject: [PATCH v3 11/18] iommu: Restore and reattach preserved domains to devices From: Samiullah Khawaja To: David Woodhouse , Lu Baolu , Joerg Roedel , Will Deacon , Jason Gunthorpe Cc: Samiullah Khawaja , Robin Murphy , Kevin Tian , Alex Williamson , Shuah Khan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Pratyush Yadav , Pasha Tatashin , David Matlack , Andrew Morton , Pranjal Shrivastava , Vipin Sharma Content-Type: text/plain; charset="UTF-8" During default domain setup, restore the preserved domains by restoring the page tables using restore() iommupt op. Associated the restored domain with the iommu group of the preserved device, and reattach the domain to the device. Signed-off-by: Samiullah Khawaja --- drivers/iommu/iommu.c | 65 ++++++++++++++++++++- drivers/iommu/liveupdate.c | 97 ++++++++++++++++++++++++++++++++ include/linux/iommu-liveupdate.h | 50 ++++++++++++++++ 3 files changed, 210 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 41effee10e33..c8b09b650d24 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -505,6 +506,14 @@ static int iommu_init_device(struct device *dev) goto err_free; } +#ifdef CONFIG_IOMMU_LIVEUPDATE + dev->iommu->device_ser = iommu_get_device_preserved_data(dev); + if (IS_ERR(dev->iommu->device_ser)) { + ret = PTR_ERR(dev->iommu->device_ser); + goto err_module_put; + } +#endif + iommu_dev = ops->probe_device(dev); if (IS_ERR(iommu_dev)) { ret = PTR_ERR(iommu_dev); @@ -2169,6 +2178,13 @@ static int __iommu_attach_device(struct iommu_domain *domain, ret = domain->ops->attach_dev(domain, dev, old); if (ret) return ret; + +#ifdef CONFIG_IOMMU_LIVEUPDATE + /* The associated state can be unset once restored. */ + if (dev_iommu_restored_state(dev)) + WRITE_ONCE(dev->iommu->device_ser, NULL); +#endif + dev->iommu->attach_deferred = 0; trace_attach_device_to_domain(dev); return 0; @@ -3124,6 +3140,47 @@ int iommu_fwspec_add_ids(struct device *dev, const u32 *ids, int num_ids) } EXPORT_SYMBOL_GPL(iommu_fwspec_add_ids); +static inline void *__iommu_group_restored_state(struct iommu_group *group) +{ + struct device *dev; + + dev = iommu_group_first_dev(group); + if (!dev_is_pci(dev)) + return NULL; + + return dev_iommu_restored_state(dev); +} + +static struct iommu_domain *__iommu_group_restore_domain(struct iommu_group *group) +{ + struct iommu_device_ser *device_ser; + struct iommu_domain *domain; + struct device *dev; + void *owner; + + lockdep_assert_held(&group->mutex); + dev = iommu_group_first_dev(group); + if (!dev_is_pci(dev)) + return NULL; + + device_ser = dev_iommu_restored_state(dev); + if (!device_ser) + return NULL; + + domain = iommu_restore_domain(dev, device_ser, &owner); + if (WARN_ON(IS_ERR(domain))) + return NULL; + + /* + * Ownership of groups with preserved devices is set during boot. These + * will be reclaimed later by the entity (iommufd) that preserved them. + */ + WARN_ON(group->owner); + group->owner = owner; + group->owner_cnt = 1; + return domain; +} + /** * iommu_setup_default_domain - Set the default_domain for the group * @group: Group to change @@ -3138,8 +3195,8 @@ static int iommu_setup_default_domain(struct iommu_group *group, int target_type) { struct iommu_domain *old_dom = group->default_domain; + struct iommu_domain *dom, *restored_domain; struct group_device *gdev; - struct iommu_domain *dom; bool direct_failed; int req_type; int ret; @@ -3183,6 +3240,10 @@ static int iommu_setup_default_domain(struct iommu_group *group, /* We must set default_domain early for __iommu_device_set_domain */ group->default_domain = dom; if (!group->domain) { + if (__iommu_group_restored_state(group)) + restored_domain = __iommu_group_restore_domain(group); + else + restored_domain = dom; /* * Drivers are not allowed to fail the first domain attach. * The only way to recover from this is to fail attaching the @@ -3190,7 +3251,7 @@ static int iommu_setup_default_domain(struct iommu_group *group, * in group->default_domain so it is freed after. */ ret = __iommu_group_set_domain_internal( - group, dom, IOMMU_SET_DOMAIN_MUST_SUCCEED); + group, restored_domain, IOMMU_SET_DOMAIN_MUST_SUCCEED); if (WARN_ON(ret)) goto out_free_old; } else { diff --git a/drivers/iommu/liveupdate.c b/drivers/iommu/liveupdate.c index 90750e78cd2a..732c19e69e69 100644 --- a/drivers/iommu/liveupdate.c +++ b/drivers/iommu/liveupdate.c @@ -257,6 +257,47 @@ int iommu_for_each_preserved_device(iommu_preserved_device_iter_fn fn, } EXPORT_SYMBOL(iommu_for_each_preserved_device); +static inline bool match_device_ser(struct iommu_device_ser *match, + struct pci_dev *pdev) +{ + return match->devid == pci_dev_id(pdev) && match->pci_domain_nr == pci_domain_nr(pdev->bus); +} + +struct iommu_device_ser *iommu_get_device_preserved_data(struct device *dev) +{ + struct iommu_device_ser *device_ser = NULL; + struct iommu_device_array_ser *array; + struct iommu_flb_obj *flb_obj; + int ret, idx; + + if (!dev_is_pci(dev)) + return NULL; + + ret = liveupdate_flb_get_incoming(&iommu_flb, (void **)&flb_obj); + if (ret == -ENODATA || ret == -ENOENT) + return NULL; + + if (ret) + return ERR_PTR(ret); + + if (!flb_obj->ser->device_array_phys) + return NULL; + + array = phys_to_virt(flb_obj->ser->device_array_phys); + iommu_liveupdate_for_each_obj(array, device_ser, idx) { + if (match_device_ser(device_ser, to_pci_dev(dev))) { + device_ser->hdr.flags |= IOMMU_SER_FLAG_INCOMING; + goto out; + } + } + + device_ser = NULL; +out: + liveupdate_flb_put_incoming(&iommu_flb); + return device_ser; +} +EXPORT_SYMBOL(iommu_get_device_preserved_data); + struct iommu_hw_ser *iommu_get_preserved_data(u64 token, enum iommu_type_ser type) { struct iommu_hw_ser *iommu_ser = NULL; @@ -561,3 +602,59 @@ void iommu_unpreserve_device(struct iommu_domain *domain, struct device *dev) iommu_unpreserve_locked(iommu->iommu_dev, flb_obj); } EXPORT_SYMBOL_GPL(iommu_unpreserve_device); + +struct iommu_domain *iommu_restore_domain(struct device *dev, + struct iommu_device_ser *ser, + void **owner) +{ + struct iommu_domain_ser *domain_ser; + struct iommu_flb_obj *flb_obj; + struct iommu_domain *domain; + struct pt_iommu *pt; + int ret; + + ret = liveupdate_flb_get_incoming(&iommu_flb, (void **)&flb_obj); + if (ret) + return ERR_PTR(ret); + + guard(mutex)(&flb_obj->lock); + + /* Preserved device should have a preserved domain */ + if (!ser->domain_iommu_ser.domain_phys) + return ERR_PTR(-EINVAL); + + domain_ser = phys_to_virt(ser->domain_iommu_ser.domain_phys); + if (domain_ser->restored_domain) { + *owner = ser; + domain = domain_ser->restored_domain; + goto out; + } + + domain_ser->hdr.flags |= IOMMU_SER_FLAG_INCOMING; + domain = iommu_paging_domain_alloc(dev); + if (IS_ERR(domain)) + goto out; + + pt = iommupt_from_domain(domain); + if (!pt) { + iommu_domain_free(domain); + domain = ERR_PTR(-EOPNOTSUPP); + goto out; + } + + ret = pt->ops->restore(pt, domain_ser); + if (ret) { + iommu_domain_free(domain); + domain = ERR_PTR(ret); + goto out; + } + + /* The device is owned by the preserved state. */ + *owner = ser; + domain->preserved_state = domain_ser; + domain_ser->restored_domain = domain; + +out: + liveupdate_flb_put_incoming(&iommu_flb); + return domain; +} diff --git a/include/linux/iommu-liveupdate.h b/include/linux/iommu-liveupdate.h index 5ad006892cbd..84522f180254 100644 --- a/include/linux/iommu-liveupdate.h +++ b/include/linux/iommu-liveupdate.h @@ -30,6 +30,20 @@ static inline void *dev_iommu_preserved_state(struct device *dev) return NULL; } +static inline void *dev_iommu_restored_state(struct device *dev) +{ + struct iommu_device_ser *ser; + + if (!dev->iommu) + return NULL; + + ser = READ_ONCE(dev->iommu->device_ser); + if (ser && (ser->hdr.flags & IOMMU_SER_FLAG_INCOMING)) + return ser; + + return NULL; +} + static inline void *iommu_domain_restored_state(struct iommu_domain *domain) { struct iommu_domain_ser *ser; @@ -41,8 +55,22 @@ static inline void *iommu_domain_restored_state(struct iommu_domain *domain) return NULL; } +static inline int dev_iommu_restore_did(struct device *dev, struct iommu_domain *domain) +{ + struct iommu_device_ser *ser = dev_iommu_restored_state(dev); + + if (ser && iommu_domain_restored_state(domain)) + return ser->domain_iommu_ser.attachment_id; + + return -1; +} + +struct iommu_domain *iommu_restore_domain(struct device *dev, + struct iommu_device_ser *ser, + void **owner); int iommu_for_each_preserved_device(iommu_preserved_device_iter_fn fn, void *arg); +struct iommu_device_ser *iommu_get_device_preserved_data(struct device *dev); struct iommu_hw_ser *iommu_get_preserved_data(u64 token, enum iommu_type_ser type); int iommu_preserve_domain(struct iommu_domain *domain, struct iommu_domain_ser **ser); void iommu_unpreserve_domain(struct iommu_domain *domain); @@ -60,16 +88,38 @@ static inline void *dev_iommu_preserved_state(struct device *dev) return NULL; } +static inline void *dev_iommu_restored_state(struct device *dev) +{ + return NULL; +} + +static inline int dev_iommu_restore_did(struct device *dev, struct iommu_domain *domain) +{ + return -1; +} + static inline void *iommu_domain_restored_state(struct iommu_domain *domain) { return NULL; } +static inline struct iommu_domain *iommu_restore_domain(struct device *dev, + struct iommu_device_ser *ser, + void **owner) +{ + return NULL; +} + static inline int iommu_for_each_preserved_device(iommu_preserved_device_iter_fn fn, void *arg) { return -EOPNOTSUPP; } +static inline struct iommu_device_ser *iommu_get_device_preserved_data(struct device *dev) +{ + return NULL; +} + static inline struct iommu_hw_ser *iommu_get_preserved_data(u64 token, enum iommu_type_ser type) { return NULL; -- 2.54.0.1136.gdb2ca164c4-goog