From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=209.85.210.72 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794283716; b=akgSTzFqSHIyLrSA7sV0UspImD6qygY41AmJrefK2XSYXxnnyBNn3ntIawVSYu2rCsTu 67vGa69kAPi+z6jxrR5FQUfOhelnflIwsZSiUARRz/hkscDBaUHTDmCkdN5BszeGNAMAL l6Xjohcnwwm0GmlNO9lt3owX6+7ROYM99CJAP4eJ4Oqm0uZu1v814p6wSiDl6Hy95YYYi 4b8MCdXtIr5hnHkmo0E/9jHW00UV3RrzfuYEv5x907LRH0oc613tXeZwQZ7Nqt/MmAJMs rbUY1qvJGRjBZWDNOMb4r3mXPfIgTm1tAcn2pEjg4BEZJGCXttjaH70q1XKuLMqdGgw== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794283716; h=DMARC-Filter:Received:DKIM-Signature:X-Google-DKIM-Signature: X-Forwarded-Encrypted:X-Gm-Message-State:X-Received:Date:In-Reply-To: Mime-Version:References:X-Mailer:Message-ID:Subject:From:To:Cc: Content-Type; bh=uMd6uU1Ewixftd9E5ZXDo/AGJ0QrsvaDjKM9cvmsWw4=; b=f3nx6Jwd7HTGbN+Ggk/LqQI3CO3oQu24YG12IqMNs3OFR+LRyosZzNZcTZD9hTTlNaVZ lEGdma8hw0hPKftppYIpBoygTzRGtWcv+EdSZyaQa6d7UJl2jGN6d0XG3CdrTw8DeHbeR nTpXIOrSIsHyvF67+Lvd53kNWb20ag1g/AJsxpef/p0OjFYbX8Triklb4tg75hSFi3cHg Vyzctl06Wkwi8FPjl5T3f6oykK27Y6SYy51zBGeb15MkGMIzmjM5AXHu4A7gbdt4U7UQG 7qgtnk5QtWUGbIlbQf9oBDT19FCxl6lVxRjf72r3FtDpYwN/t+6mbF8ChlvXrStMNrA== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=google.com; spf=pass smtp.mailfrom=flex--skhawaja.bounces.google.com; dkim=pass header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20260707 header.b=d21wZNlZ; arc=none smtp.remote-ip=209.85.210.72 Received: from mail-ot1-f72.google.com (mail-ot1-f72.google.com [209.85.210.72]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D1B9606A33D for ; Tue, 10 Nov 2026 04:08:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org 9D1B9606A33D Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--skhawaja.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20260707 header.b=d21wZNlZ Received: by mail-ot1-f72.google.com with SMTP id 46e09a7af769-845156f593bso879976a34.0 for ; Mon, 09 Nov 2026 20:08:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20260707; t=1794283715; x=1794888515; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=uMd6uU1Ewixftd9E5ZXDo/AGJ0QrsvaDjKM9cvmsWw4=; b=d21wZNlZAISYxH4LUMdJe6U8ZKU1ADTj+jeMVUfBB0ljQuNP48ocWEpomV/l6GY+Mz QRYg7uh2UAVLwo3tdGjZNxLK5hPna64/rvW4qobVF3EIcbaaCfRss0m7CyM1pQ0G5OIT ODFEn+nhHdncFFEPI+lmVpLx9HAPlFAkfgjn6YlxvsKJCwbNMw5xpLktQrSXE8yLmnPW womGMWkFI42rMu78KZHPGQb/Glus79fO64x7cwUFRLKt6mXHdw0wtekr6nXiHH+GuqeF xljyd22GSCBSJ/jSWvObvla5cUNp9yENwvgd1GjcBm9Hbe8sTLvJ3dfJgFeYbL3uZ9j/ vidg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1794283715; x=1794888515; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=uMd6uU1Ewixftd9E5ZXDo/AGJ0QrsvaDjKM9cvmsWw4=; b=P9Qxbpakp2Y+O1WelUcR08YuZZ/Nkgpemy146xn7+JV/1cHSvu4DI4a5/GvL8OKC9z W0qPEYla+X2UVzIm2fO0sh0NltX6ZSacWg9qQHcXvZcNWFpfBxFBBcgKkAJ8j1Nyc6i9 f3acxsQfUrP4k2OUSdluReGOyvVb5fMsu2G0yENjZ781U1FBlR8YwL0Okp/2xPANTuo+ noFax3sUxlZgnLOEkLhNDbp7Q4ahtNd7/VEFnaGZjG2yZvh/8Z2k3ipi+rFcNPUXrGrU 4R7kpeRHbOlWb2lS3ahTP5vCkm8k0b7IauCYIJmt9EfILcMHxOtZ5xAfqgnejmdysT4/ MyOQ== X-Forwarded-Encrypted: i=1; AL+hErE46w1cdSJDSQpz+DK9rO95ldw90g1Bdjuf+tzCZoJ7rTzAl1bhpBbOSitR2+8vdo1i4w/T5sdsw/2lgDs=@vger.kernel.org X-Gm-Message-State: APuU4Z+Gu93lKTE5Sh0Zf+VXxTnlCVFK0q2pfN+c9elSssQI4skPPE0G EcgU8dJTbOa2X3KDse55SLi6x9oOGSKs94qKVY+ObGQ2iPZK2SarCPyjfsucX1H1ms/7mfGVjkn R7q1GAgqmLRgXCQ== X-Received: from iobi4-n1.prod.google.com ([2002:a05:6602:c684:10b0:9d3:b408:7991]) (user=skhawaja job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6808:bd6:b0:4ec:d580:2440 with SMTP id 5614622812f47-5683cb3907cmr1189129b6e.54.1794283714349; Mon, 09 Nov 2026 20:08:34 -0800 (PST) Date: Tue, 10 Nov 2026 04:08:08 +0000 In-Reply-To: <20261110040828.1774343-1-skhawaja@google.com> X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list Mime-Version: 1.0 References: <20261110040828.1774343-1-skhawaja@google.com> X-Mailer: git-send-email 2.56.0.506.g506a7eabc5-goog Message-ID: <20261110040828.1774343-3-skhawaja@google.com> Subject: [PATCH v7 02/18] iommu/pages: Add APIs to preserve/unpreserve/restore iommu pages From: Samiullah Khawaja To: Joerg Roedel , Will Deacon , Lu Baolu , Jason Gunthorpe , David Woodhouse Cc: Samiullah Khawaja , Pranjal Shrivastava , Robin Murphy , Kevin Tian , Alex Williamson , Shuah Khan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Pratyush Yadav , Pasha Tatashin , David Matlack , Andrew Morton , Vipin Sharma , Ankit Soni , Nicolin Chen , John Starks Content-Type: text/plain; charset="UTF-8" IOMMU pages are allocated/freed using APIs using struct ioptdesc. For the proper preservation and restoration of ioptdesc add helper functions. Signed-off-by: Samiullah Khawaja Reviewed-by: Pranjal Shrivastava Reviewed-by: Lu Baolu --- drivers/iommu/iommu-pages.c | 109 ++++++++++++++++++++++++++++++++++-- drivers/iommu/iommu-pages.h | 30 ++++++++++ 2 files changed, 135 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/iommu-pages.c b/drivers/iommu/iommu-pages.c index 3bab175d8557..0aa18c771baa 100644 --- a/drivers/iommu/iommu-pages.c +++ b/drivers/iommu/iommu-pages.c @@ -6,6 +6,7 @@ #include "iommu-pages.h" #include #include +#include #include #define IOPTDESC_MATCH(pg_elm, elm) \ @@ -28,6 +29,13 @@ static inline size_t ioptdesc_mem_size(struct ioptdesc *desc) return 1UL << (folio_order(ioptdesc_folio(desc)) + PAGE_SHIFT); } +static inline void iommu_folio_update_stats(struct folio *folio, + unsigned long nr_pages) +{ + mod_node_page_state(folio_pgdat(folio), NR_IOMMU_PAGES, nr_pages); + lruvec_stat_mod_folio(folio, NR_SECONDARY_PAGETABLE, nr_pages); +} + /** * iommu_alloc_pages_node_sz - Allocate a zeroed page of a given size from * specific NUMA node @@ -80,8 +88,7 @@ void *iommu_alloc_pages_node_sz(int nid, gfp_t gfp, size_t size) * rather large, i.e. multiple gigabytes in size. */ pgcnt = 1UL << order; - mod_node_page_state(folio_pgdat(folio), NR_IOMMU_PAGES, pgcnt); - lruvec_stat_mod_folio(folio, NR_SECONDARY_PAGETABLE, pgcnt); + iommu_folio_update_stats(folio, pgcnt); return folio_address(folio); } @@ -95,8 +102,7 @@ static void __iommu_free_desc(struct ioptdesc *iopt) if (IOMMU_PAGES_USE_DMA_API) WARN_ON_ONCE(iopt->incoherent); - mod_node_page_state(folio_pgdat(folio), NR_IOMMU_PAGES, -pgcnt); - lruvec_stat_mod_folio(folio, NR_SECONDARY_PAGETABLE, -pgcnt); + iommu_folio_update_stats(folio, -pgcnt); folio_put(folio); } @@ -131,6 +137,101 @@ void iommu_put_pages_list(struct iommu_pages_list *list) } EXPORT_SYMBOL_GPL(iommu_put_pages_list); +#if IS_ENABLED(CONFIG_IOMMU_LIVEUPDATE) +/** + * iommu_unpreserve_pages - Unpreserve pages that were preserved in KHO + * @virt: Virtual address of page to be unpreserved + */ +void iommu_unpreserve_pages(void *virt) +{ + kho_unpreserve_folio(ioptdesc_folio(virt_to_ioptdesc(virt))); +} +EXPORT_SYMBOL_GPL(iommu_unpreserve_pages); + +/** + * iommu_preserve_pages - Preserve pages using KHO + * @virt: Virtual address of page to preserve + * + * Returns 0 on success, negative error on failure + */ +int iommu_preserve_pages(void *virt) +{ + return kho_preserve_folio(ioptdesc_folio(virt_to_ioptdesc(virt))); +} +EXPORT_SYMBOL_GPL(iommu_preserve_pages); + +/** + * iommu_unpreserve_pages_list - Unpreserve a list of KHO preserved pages + * @list: List of pages to unpreserve + */ +void iommu_unpreserve_pages_list(struct iommu_pages_list *list) +{ + struct ioptdesc *iopt; + + list_for_each_entry(iopt, &list->pages, iopt_freelist_elm) + kho_unpreserve_folio(ioptdesc_folio(iopt)); +} +EXPORT_SYMBOL_GPL(iommu_unpreserve_pages_list); + +/** + * iommu_restore_pages - Restore pages that were preserved in KHO + * @phys: Physical address of page to restore + */ +void iommu_restore_pages(u64 phys) +{ + struct ioptdesc *iopt; + struct folio *folio; + unsigned long pgcnt; + unsigned int order; + + folio = kho_restore_folio(phys); + if (!folio) + panic("Failed to restore preserved IOMMU page %llx\n", phys); + + iopt = folio_ioptdesc(folio); + + /* + * For the restored pages incoherent is set to false as these are not + * mapped using the DMA_API. The remapping of these pages using DMA_API + * is not needed as these are not going to be written to by the new + * kernel. + */ + iopt->incoherent = false; + + order = folio_order(folio); + pgcnt = 1UL << order; + iommu_folio_update_stats(folio, pgcnt); +} +EXPORT_SYMBOL_GPL(iommu_restore_pages); + +/** + * iommu_preserve_pages_list - Preserve list of pages using KHO + * @list: List of pages to preserve + * + * Returns 0 on success, negative error on failure + */ +int iommu_preserve_pages_list(struct iommu_pages_list *list) +{ + struct ioptdesc *iopt; + int ret; + + list_for_each_entry(iopt, &list->pages, iopt_freelist_elm) { + ret = kho_preserve_folio(ioptdesc_folio(iopt)); + if (ret) + goto err; + } + + return 0; + +err: + list_for_each_entry_continue_reverse(iopt, &list->pages, iopt_freelist_elm) + kho_unpreserve_folio(ioptdesc_folio(iopt)); + + return ret; +} +EXPORT_SYMBOL_GPL(iommu_preserve_pages_list); +#endif + /** * iommu_pages_start_incoherent - Setup the page for cache incoherent operation * @virt: The page to setup diff --git a/drivers/iommu/iommu-pages.h b/drivers/iommu/iommu-pages.h index e9e605b5fa3a..3c409ec3d170 100644 --- a/drivers/iommu/iommu-pages.h +++ b/drivers/iommu/iommu-pages.h @@ -53,6 +53,36 @@ void *iommu_alloc_pages_node_sz(int nid, gfp_t gfp, size_t size); void iommu_free_pages(void *virt); void iommu_put_pages_list(struct iommu_pages_list *list); +#if IS_ENABLED(CONFIG_IOMMU_LIVEUPDATE) +int iommu_preserve_pages(void *virt); +void iommu_unpreserve_pages(void *virt); +int iommu_preserve_pages_list(struct iommu_pages_list *list); +void iommu_unpreserve_pages_list(struct iommu_pages_list *list); +void iommu_restore_pages(u64 phys); +#else +static inline int iommu_preserve_pages(void *virt) +{ + return -EOPNOTSUPP; +} + +static inline void iommu_unpreserve_pages(void *virt) +{ +} + +static inline int iommu_preserve_pages_list(struct iommu_pages_list *list) +{ + return -EOPNOTSUPP; +} + +static inline void iommu_unpreserve_pages_list(struct iommu_pages_list *list) +{ +} + +static inline void iommu_restore_pages(u64 phys) +{ +} +#endif + /** * iommu_pages_list_add - add the page to a iommu_pages_list * @list: List to add the page to -- 2.56.0.506.g506a7eabc5-goog