mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH 0/3] udmabuf: speed up mmap() of huge page backed buffers
@ 2026-11-22  7:37 Qiliang Yuan
  2026-11-22  7:37 ` [PATCH 1/3] dma-buf: align large mappings to PMD boundaries Qiliang Yuan
                   ` (2 more replies)
  0 siblings, 3 replies; 4+ messages in thread
From: Qiliang Yuan @ 2026-11-22  7:37 UTC (permalink / raw)
  To: Sumit Semwal, Christian König, Philipp Stanner,
	Gerd Hoffmann, Vivek Kasireddy, Shuah Khan
  Cc: linux-media, dri-devel, linaro-mm-sig, linux-kernel,
	linux-kselftest, Qiliang Yuan

QEMU and crosvm export virtio-gpu blob resources from guest memory as
udmabufs, which the host renderer and display mmap(). The guest memory is
often backed by 2 MiB huge pages, yet udmabuf maps it 4 KiB at a time:
mapping and touching a 4 GiB buffer inserts a million PTEs and takes over
200 ms.

Patch 1 has mmap() of a dma-buf align large mappings to PMD boundaries,
which any exporter mapping contiguous memory with PMDs needs, such as the
proposed VFIO dma-buf mmap() [1]. Patch 2 maps the huge page backed
ranges of a udmabuf with PMDs, which cuts the time to map and touch it
by 87-90% for hugetlb and shmem THP backed buffers and leaves 4 KiB
backed ones as they are. Patch 3 adds selftests.

[1] https://lore.kernel.org/r/20260527102319.100128-1-mattev@meta.com

Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
Qiliang Yuan (3):
      dma-buf: align large mappings to PMD boundaries
      udmabuf: map huge page backed ranges with PMDs
      selftests/udmabuf: test mapping a huge page backed udmabuf

 drivers/dma-buf/dma-buf.c                         |  1 +
 drivers/dma-buf/udmabuf.c                         | 69 ++++++++++++++++-
 tools/testing/selftests/drivers/dma-buf/udmabuf.c | 90 ++++++++++++++++++++++-
 3 files changed, 158 insertions(+), 2 deletions(-)
---
base-commit: c408466a4dfaee4b12587e926c5b85e68c8b3c40
change-id: 20261121-feat-udmabuf-huge-pfnmap-3258272d077e

Best regards,
-- 
Qiliang Yuan <qiliang.yuan@linux.dev>


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 1/3] dma-buf: align large mappings to PMD boundaries
  2026-11-22  7:37 [PATCH 0/3] udmabuf: speed up mmap() of huge page backed buffers Qiliang Yuan
@ 2026-11-22  7:37 ` Qiliang Yuan
  2026-11-22  7:37 ` [PATCH 2/3] udmabuf: map huge page backed ranges with PMDs Qiliang Yuan
  2026-11-22  7:37 ` [PATCH 3/3] selftests/udmabuf: test mapping a huge page backed udmabuf Qiliang Yuan
  2 siblings, 0 replies; 4+ messages in thread
From: Qiliang Yuan @ 2026-11-22  7:37 UTC (permalink / raw)
  To: Sumit Semwal, Christian König, Philipp Stanner,
	Gerd Hoffmann, Vivek Kasireddy, Shuah Khan
  Cc: linux-media, dri-devel, linaro-mm-sig, linux-kernel,
	linux-kselftest, Qiliang Yuan

A dma-buf exporter whose buffer is backed by physically contiguous memory
can map it with PMDs from a ->huge_fault() handler, which only works where
the user address and the buffer offset are aligned alike to PMD_SIZE.

dma_buf_fops has no ->get_unmapped_area(), so mmap() places a dma-buf at
whatever page aligned address the default allocator returns. A 4 GiB
udmabuf, for one, lands 616 KiB past a 2 MiB boundary, and then none of
its PMD-sized ranges can be mapped with a PMD.

Use thp_get_unmapped_area(), as filesystems that support huge mappings
do. It aligns a mapping that covers at least one whole PMD to the buffer
offset and leaves smaller mappings where they would be placed anyway.

Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
 drivers/dma-buf/dma-buf.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index 5c6957f8839a8..015a03c8d04db 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -608,6 +608,7 @@ static void dma_buf_show_fdinfo(struct seq_file *m, struct file *file)
 static const struct file_operations dma_buf_fops = {
 	.release	= dma_buf_file_release,
 	.mmap		= dma_buf_mmap_internal,
+	.get_unmapped_area = thp_get_unmapped_area,
 	.llseek		= dma_buf_llseek,
 	.poll		= dma_buf_poll,
 	.unlocked_ioctl	= dma_buf_ioctl,

-- 
2.43.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 2/3] udmabuf: map huge page backed ranges with PMDs
  2026-11-22  7:37 [PATCH 0/3] udmabuf: speed up mmap() of huge page backed buffers Qiliang Yuan
  2026-11-22  7:37 ` [PATCH 1/3] dma-buf: align large mappings to PMD boundaries Qiliang Yuan
@ 2026-11-22  7:37 ` Qiliang Yuan
  2026-11-22  7:37 ` [PATCH 3/3] selftests/udmabuf: test mapping a huge page backed udmabuf Qiliang Yuan
  2 siblings, 0 replies; 4+ messages in thread
From: Qiliang Yuan @ 2026-11-22  7:37 UTC (permalink / raw)
  To: Sumit Semwal, Christian König, Philipp Stanner,
	Gerd Hoffmann, Vivek Kasireddy, Shuah Khan
  Cc: linux-media, dri-devel, linaro-mm-sig, linux-kernel,
	linux-kselftest, Qiliang Yuan

QEMU and crosvm hand guest memory to the host renderer and display as
udmabufs for virtio-gpu blob resources. The guest memory is a memfd that
is often backed by 2 MiB huge pages, and the host process mmap()s the
dma-buf to access it with the CPU.

udmabuf maps every page with its own PTE: the first fault inserts a PFN
for each 4 KiB page of the whole VMA. Mapping and touching 4 GiB this way
takes over 200 ms and a million PTEs, although the memory behind it is
2048 huge pages.

Add a ->huge_fault() handler that maps a PMD-sized range with one PMD
when it is backed by an aligned run of contiguous PFNs, and falls back to
PTEs otherwise. Have the pre-mapping on the first fault skip the ranges
the handler will map, so that their PTEs don't take the place of the
PMDs, and keep pre-mapping everything when the VMA can't have PMDs.

Mapping a udmabuf over a memfd filled with data and reading a byte of
each 4 KiB page through the mapping, base and patched kernels alternating
over 3 rounds of 5 runs (7 for shmem 4K), median of the per-round medians
and range in parentheses:

  backing    size        before (ms)         after (ms)   change
  hugetlb    256M   13.3 (13.3-13.7)      1.7 (1.7-1.8)     -87%
  hugetlb      1G   53.6 (53.0-53.7)      6.5 (4.2-6.6)     -88%
  hugetlb      4G      211 (211-212)   23.6 (17.6-25.2)     -89%
  hugetlb      8G      423 (422-424)   45.8 (45.8-47.2)     -89%
  shmem THP    1G   52.8 (52.5-53.1)      6.3 (5.8-6.8)     -88%
  shmem THP    4G      210 (207-214)   20.7 (19.5-21.1)     -90%
  shmem 4K     1G   52.0 (51.6-52.7)   51.8 (51.5-52.9)       0%
  shmem 4K     4G      209 (209-211)      208 (208-209)       0%

Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
 drivers/dma-buf/udmabuf.c | 69 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 68 insertions(+), 1 deletion(-)

diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c
index 018937435356e..f9c908f8b53a2 100644
--- a/drivers/dma-buf/udmabuf.c
+++ b/drivers/dma-buf/udmabuf.c
@@ -44,12 +44,63 @@ struct udmabuf {
 	struct miscdevice *device;
 };
 
+/*
+ * Return whether the PMD-sized range of @vma at @addr is backed by one aligned
+ * run of contiguous PFNs, which can then be mapped with a single PMD.
+ */
+static bool udmabuf_pmd_mappable(struct vm_area_struct *vma, unsigned long addr,
+				 unsigned long *pfn)
+{
+	struct udmabuf *ubuf = vma->vm_private_data;
+	const pgoff_t nr = PMD_SIZE >> PAGE_SHIFT;
+	unsigned long first;
+	pgoff_t pgoff, i;
+
+	if (!IS_ENABLED(CONFIG_ARCH_SUPPORTS_PMD_PFNMAP))
+		return false;
+
+	if (!IS_ALIGNED(addr, PMD_SIZE) || addr < vma->vm_start ||
+	    vma->vm_end - addr < PMD_SIZE)
+		return false;
+
+	pgoff = vma->vm_pgoff + ((addr - vma->vm_start) >> PAGE_SHIFT);
+	if (pgoff >= ubuf->pagecount || ubuf->pagecount - pgoff < nr)
+		return false;
+
+	first = page_to_pfn(ubuf->pages[pgoff]);
+	if (!IS_ALIGNED(first, nr))
+		return false;
+
+	for (i = 1; i < nr; i++) {
+		if (page_to_pfn(ubuf->pages[pgoff + i]) != first + i)
+			return false;
+	}
+
+	*pfn = first;
+	return true;
+}
+
+#ifdef CONFIG_ARCH_SUPPORTS_HUGE_PFNMAP
+static vm_fault_t udmabuf_vm_huge_fault(struct vm_fault *vmf,
+					unsigned int order)
+{
+	unsigned long pfn;
+
+	if (order != PMD_ORDER ||
+	    !udmabuf_pmd_mappable(vmf->vma, vmf->address & PMD_MASK, &pfn))
+		return VM_FAULT_FALLBACK;
+
+	return vmf_insert_pfn_pmd(vmf, pfn, vmf->flags & FAULT_FLAG_WRITE);
+}
+#endif
+
 static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
 	struct udmabuf *ubuf = vma->vm_private_data;
 	pgoff_t pgoff = vmf->pgoff;
 	unsigned long addr, pfn;
+	bool pmd_ok;
 	vm_fault_t ret;
 
 	if (pgoff >= ubuf->pagecount)
@@ -61,7 +112,12 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
 	if (ret & VM_FAULT_ERROR)
 		return ret;
 
-	/* pre fault */
+	/*
+	 * Pre-fault the rest of the VMA, except for the ranges that
+	 * udmabuf_vm_huge_fault() maps with a PMD once they are accessed.
+	 */
+	pmd_ok = thp_vma_allowable_order(vma, vma->vm_flags, TVA_PAGEFAULT,
+					 PMD_ORDER);
 	pgoff = vma->vm_pgoff;
 	addr = vma->vm_start;
 
@@ -69,6 +125,14 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
 		if (addr == vmf->address)
 			continue;
 
+		if (pmd_ok && IS_ALIGNED(addr, PMD_SIZE) &&
+		    (vmf->address & PMD_MASK) != addr &&
+		    udmabuf_pmd_mappable(vma, addr, &pfn)) {
+			pgoff += (PMD_SIZE >> PAGE_SHIFT) - 1;
+			addr += PMD_SIZE - PAGE_SIZE;
+			continue;
+		}
+
 		if (WARN_ON(pgoff >= ubuf->pagecount))
 			break;
 
@@ -89,6 +153,9 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
 
 static const struct vm_operations_struct udmabuf_vm_ops = {
 	.fault = udmabuf_vm_fault,
+#ifdef CONFIG_ARCH_SUPPORTS_HUGE_PFNMAP
+	.huge_fault = udmabuf_vm_huge_fault,
+#endif
 };
 
 static int mmap_udmabuf(struct dma_buf *buf, struct vm_area_struct *vma)

-- 
2.43.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH 3/3] selftests/udmabuf: test mapping a huge page backed udmabuf
  2026-11-22  7:37 [PATCH 0/3] udmabuf: speed up mmap() of huge page backed buffers Qiliang Yuan
  2026-11-22  7:37 ` [PATCH 1/3] dma-buf: align large mappings to PMD boundaries Qiliang Yuan
  2026-11-22  7:37 ` [PATCH 2/3] udmabuf: map huge page backed ranges with PMDs Qiliang Yuan
@ 2026-11-22  7:37 ` Qiliang Yuan
  2 siblings, 0 replies; 4+ messages in thread
From: Qiliang Yuan @ 2026-11-22  7:37 UTC (permalink / raw)
  To: Sumit Semwal, Christian König, Philipp Stanner,
	Gerd Hoffmann, Vivek Kasireddy, Shuah Khan
  Cc: linux-media, dri-devel, linaro-mm-sig, linux-kernel,
	linux-kselftest, Qiliang Yuan

udmabuf maps the ranges of a buffer that are backed by huge pages with
PMDs, provided the mapping covers whole PMDs at an aligned address.

The existing tests of huge page backed buffers only map a few pages, so
they never get a PMD mapping or check what is read through one.

Map a 64 MiB udmabuf backed by 2 MiB huge pages and read back what was
written to the memfd: through a mapping of the whole buffer, which must
be PMD aligned when THP support is built in, through a mapping at an
unaligned offset, and from a forked child.

Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
 tools/testing/selftests/drivers/dma-buf/udmabuf.c | 90 ++++++++++++++++++++++-
 1 file changed, 89 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/drivers/dma-buf/udmabuf.c b/tools/testing/selftests/drivers/dma-buf/udmabuf.c
index d78aec662586b..e09330fece6fb 100644
--- a/tools/testing/selftests/drivers/dma-buf/udmabuf.c
+++ b/tools/testing/selftests/drivers/dma-buf/udmabuf.c
@@ -14,6 +14,7 @@
 #include <sys/ioctl.h>
 #include <sys/syscall.h>
 #include <sys/mman.h>
+#include <sys/wait.h>
 #include <linux/memfd.h>
 #include <linux/udmabuf.h>
 #include "kselftest.h"
@@ -130,6 +131,91 @@ static int compare_chunks(void *addr1, void *addr2, off64_t memfd_size)
 	return ret;
 }
 
+#define HPAGE_SIZE	(2UL << 20)
+#define HUGE_BUF_SIZE	(32 * HPAGE_SIZE)
+
+/* Check that @len bytes of @buf at @off read back what was written to @mem */
+static bool check_huge_range(int buf, char *mem, off_t off, size_t len)
+{
+	char *addr;
+	size_t i;
+	bool ok = true;
+
+	addr = mmap(NULL, len, PROT_READ, MAP_SHARED, buf, off);
+	if (addr == MAP_FAILED)
+		return false;
+
+	for (i = 0; i < len; i += getpagesize()) {
+		if (addr[i] != mem[off + i]) {
+			ok = false;
+			break;
+		}
+	}
+
+	munmap(addr, len);
+	return ok;
+}
+
+/*
+ * Map a udmabuf backed by 2MB huge pages, which can be mapped with PMDs, and
+ * read it back from a PMD-aligned address, a sub-range at an unaligned offset
+ * and a forked child.
+ */
+static void test_huge_mapping(int devfd)
+{
+	struct udmabuf_create create = {};
+	int memfd, buf, status;
+	bool thp, aligned, ok;
+	char *mem, *addr;
+	pid_t pid;
+	size_t i;
+
+	memfd = create_memfd_with_seals(HUGE_BUF_SIZE, true);
+	mem = mmap_fd(memfd, HUGE_BUF_SIZE);
+	for (i = 0; i < HUGE_BUF_SIZE; i += getpagesize())
+		mem[i] = (char)(i / getpagesize());
+
+	create.memfd = memfd;
+	create.size = HUGE_BUF_SIZE;
+	buf = ioctl(devfd, UDMABUF_CREATE, &create);
+	if (buf < 0) {
+		ksft_test_result_fail("%s: [FAIL,test-8]\n", TEST_PREFIX);
+		ksft_test_result_skip("%s: [SKIP,test-9]\n", TEST_PREFIX);
+		goto out;
+	}
+
+	/* The address is only aligned when THP support is built in */
+	thp = !access("/sys/kernel/mm/transparent_hugepage", F_OK);
+	addr = mmap(NULL, HUGE_BUF_SIZE, PROT_READ, MAP_SHARED, buf, 0);
+	aligned = addr != MAP_FAILED &&
+		  (!thp || !((unsigned long)addr % HPAGE_SIZE));
+	if (addr != MAP_FAILED)
+		munmap(addr, HUGE_BUF_SIZE);
+	ok = aligned && check_huge_range(buf, mem, 0, HUGE_BUF_SIZE);
+	if (ok)
+		ksft_test_result_pass("%s: [PASS,test-8]\n", TEST_PREFIX);
+	else
+		ksft_test_result_fail("%s: [FAIL,test-8]\n", TEST_PREFIX);
+
+	ok = check_huge_range(buf, mem, HPAGE_SIZE + getpagesize(),
+			      5 * HPAGE_SIZE + 3 * getpagesize());
+	pid = fork();
+	if (!pid)
+		_exit(check_huge_range(buf, mem, 0, HUGE_BUF_SIZE) ? 0 : 1);
+	if (pid < 0 || waitpid(pid, &status, 0) != pid ||
+	    !WIFEXITED(status) || WEXITSTATUS(status))
+		ok = false;
+	if (ok)
+		ksft_test_result_pass("%s: [PASS,test-9]\n", TEST_PREFIX);
+	else
+		ksft_test_result_fail("%s: [FAIL,test-9]\n", TEST_PREFIX);
+
+	close(buf);
+out:
+	munmap(mem, HUGE_BUF_SIZE);
+	close(memfd);
+}
+
 int main(int argc, char *argv[])
 {
 	struct udmabuf_create create;
@@ -138,7 +224,7 @@ int main(int argc, char *argv[])
 	void *addr1, *addr2;
 
 	ksft_print_header();
-	ksft_set_plan(7);
+	ksft_set_plan(9);
 
 	devfd = open("/dev/udmabuf", O_RDWR);
 	if (devfd < 0) {
@@ -268,6 +354,8 @@ int main(int argc, char *argv[])
 
 	close(buf);
 	close(memfd);
+
+	test_huge_mapping(devfd);
 	close(devfd);
 
 	ksft_print_msg("%s: ok\n", TEST_PREFIX);

-- 
2.43.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2026-11-22  7:38 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-22  7:37 [PATCH 0/3] udmabuf: speed up mmap() of huge page backed buffers Qiliang Yuan
2026-11-22  7:37 ` [PATCH 1/3] dma-buf: align large mappings to PMD boundaries Qiliang Yuan
2026-11-22  7:37 ` [PATCH 2/3] udmabuf: map huge page backed ranges with PMDs Qiliang Yuan
2026-11-22  7:37 ` [PATCH 3/3] selftests/udmabuf: test mapping a huge page backed udmabuf Qiliang Yuan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®