From: Qiliang Yuan <qiliang.yuan@linux.dev>
To: "Sumit Semwal" <sumit.semwal@linaro.org>,
"Christian König" <christian.koenig@amd.com>,
"Philipp Stanner" <phasta@kernel.org>,
"Gerd Hoffmann" <kraxel@redhat.com>,
"Vivek Kasireddy" <vivek.kasireddy@intel.com>,
"Shuah Khan" <shuah@kernel.org>
Cc: linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org,
linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org,
linux-kselftest@vger.kernel.org,
Qiliang Yuan <qiliang.yuan@linux.dev>
Subject: [PATCH 2/3] udmabuf: map huge page backed ranges with PMDs
Date: Sun, 22 Nov 2026 02:37:37 -0500 [thread overview]
Message-ID: <20261122-feat-udmabuf-huge-pfnmap-v1-2-dcba888940c2@linux.dev> (raw)
In-Reply-To: <20261122-feat-udmabuf-huge-pfnmap-v1-0-dcba888940c2@linux.dev>
QEMU and crosvm hand guest memory to the host renderer and display as
udmabufs for virtio-gpu blob resources. The guest memory is a memfd that
is often backed by 2 MiB huge pages, and the host process mmap()s the
dma-buf to access it with the CPU.
udmabuf maps every page with its own PTE: the first fault inserts a PFN
for each 4 KiB page of the whole VMA. Mapping and touching 4 GiB this way
takes over 200 ms and a million PTEs, although the memory behind it is
2048 huge pages.
Add a ->huge_fault() handler that maps a PMD-sized range with one PMD
when it is backed by an aligned run of contiguous PFNs, and falls back to
PTEs otherwise. Have the pre-mapping on the first fault skip the ranges
the handler will map, so that their PTEs don't take the place of the
PMDs, and keep pre-mapping everything when the VMA can't have PMDs.
Mapping a udmabuf over a memfd filled with data and reading a byte of
each 4 KiB page through the mapping, base and patched kernels alternating
over 3 rounds of 5 runs (7 for shmem 4K), median of the per-round medians
and range in parentheses:
backing size before (ms) after (ms) change
hugetlb 256M 13.3 (13.3-13.7) 1.7 (1.7-1.8) -87%
hugetlb 1G 53.6 (53.0-53.7) 6.5 (4.2-6.6) -88%
hugetlb 4G 211 (211-212) 23.6 (17.6-25.2) -89%
hugetlb 8G 423 (422-424) 45.8 (45.8-47.2) -89%
shmem THP 1G 52.8 (52.5-53.1) 6.3 (5.8-6.8) -88%
shmem THP 4G 210 (207-214) 20.7 (19.5-21.1) -90%
shmem 4K 1G 52.0 (51.6-52.7) 51.8 (51.5-52.9) 0%
shmem 4K 4G 209 (209-211) 208 (208-209) 0%
Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
drivers/dma-buf/udmabuf.c | 69 ++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 68 insertions(+), 1 deletion(-)
diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c
index 018937435356e..f9c908f8b53a2 100644
--- a/drivers/dma-buf/udmabuf.c
+++ b/drivers/dma-buf/udmabuf.c
@@ -44,12 +44,63 @@ struct udmabuf {
struct miscdevice *device;
};
+/*
+ * Return whether the PMD-sized range of @vma at @addr is backed by one aligned
+ * run of contiguous PFNs, which can then be mapped with a single PMD.
+ */
+static bool udmabuf_pmd_mappable(struct vm_area_struct *vma, unsigned long addr,
+ unsigned long *pfn)
+{
+ struct udmabuf *ubuf = vma->vm_private_data;
+ const pgoff_t nr = PMD_SIZE >> PAGE_SHIFT;
+ unsigned long first;
+ pgoff_t pgoff, i;
+
+ if (!IS_ENABLED(CONFIG_ARCH_SUPPORTS_PMD_PFNMAP))
+ return false;
+
+ if (!IS_ALIGNED(addr, PMD_SIZE) || addr < vma->vm_start ||
+ vma->vm_end - addr < PMD_SIZE)
+ return false;
+
+ pgoff = vma->vm_pgoff + ((addr - vma->vm_start) >> PAGE_SHIFT);
+ if (pgoff >= ubuf->pagecount || ubuf->pagecount - pgoff < nr)
+ return false;
+
+ first = page_to_pfn(ubuf->pages[pgoff]);
+ if (!IS_ALIGNED(first, nr))
+ return false;
+
+ for (i = 1; i < nr; i++) {
+ if (page_to_pfn(ubuf->pages[pgoff + i]) != first + i)
+ return false;
+ }
+
+ *pfn = first;
+ return true;
+}
+
+#ifdef CONFIG_ARCH_SUPPORTS_HUGE_PFNMAP
+static vm_fault_t udmabuf_vm_huge_fault(struct vm_fault *vmf,
+ unsigned int order)
+{
+ unsigned long pfn;
+
+ if (order != PMD_ORDER ||
+ !udmabuf_pmd_mappable(vmf->vma, vmf->address & PMD_MASK, &pfn))
+ return VM_FAULT_FALLBACK;
+
+ return vmf_insert_pfn_pmd(vmf, pfn, vmf->flags & FAULT_FLAG_WRITE);
+}
+#endif
+
static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
{
struct vm_area_struct *vma = vmf->vma;
struct udmabuf *ubuf = vma->vm_private_data;
pgoff_t pgoff = vmf->pgoff;
unsigned long addr, pfn;
+ bool pmd_ok;
vm_fault_t ret;
if (pgoff >= ubuf->pagecount)
@@ -61,7 +112,12 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
if (ret & VM_FAULT_ERROR)
return ret;
- /* pre fault */
+ /*
+ * Pre-fault the rest of the VMA, except for the ranges that
+ * udmabuf_vm_huge_fault() maps with a PMD once they are accessed.
+ */
+ pmd_ok = thp_vma_allowable_order(vma, vma->vm_flags, TVA_PAGEFAULT,
+ PMD_ORDER);
pgoff = vma->vm_pgoff;
addr = vma->vm_start;
@@ -69,6 +125,14 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
if (addr == vmf->address)
continue;
+ if (pmd_ok && IS_ALIGNED(addr, PMD_SIZE) &&
+ (vmf->address & PMD_MASK) != addr &&
+ udmabuf_pmd_mappable(vma, addr, &pfn)) {
+ pgoff += (PMD_SIZE >> PAGE_SHIFT) - 1;
+ addr += PMD_SIZE - PAGE_SIZE;
+ continue;
+ }
+
if (WARN_ON(pgoff >= ubuf->pagecount))
break;
@@ -89,6 +153,9 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
static const struct vm_operations_struct udmabuf_vm_ops = {
.fault = udmabuf_vm_fault,
+#ifdef CONFIG_ARCH_SUPPORTS_HUGE_PFNMAP
+ .huge_fault = udmabuf_vm_huge_fault,
+#endif
};
static int mmap_udmabuf(struct dma_buf *buf, struct vm_area_struct *vma)
--
2.43.0
next prev parent reply other threads:[~2026-11-22 7:37 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-22 7:37 [PATCH 0/3] udmabuf: speed up mmap() of huge page backed buffers Qiliang Yuan
2026-11-22 7:37 ` [PATCH 1/3] dma-buf: align large mappings to PMD boundaries Qiliang Yuan
2026-11-22 7:37 ` Qiliang Yuan [this message]
2026-11-22 7:37 ` [PATCH 3/3] selftests/udmabuf: test mapping a huge page backed udmabuf Qiliang Yuan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261122-feat-udmabuf-huge-pfnmap-v1-2-dcba888940c2@linux.dev \
--to=qiliang.yuan@linux.dev \
--cc=christian.koenig@amd.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=kraxel@redhat.com \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=phasta@kernel.org \
--cc=shuah@kernel.org \
--cc=sumit.semwal@linaro.org \
--cc=vivek.kasireddy@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®