* [PATCH 0/3] udmabuf: speed up mmap() of huge page backed buffers
@ 2026-11-22 7:37 Qiliang Yuan
2026-11-22 7:37 ` [PATCH 1/3] dma-buf: align large mappings to PMD boundaries Qiliang Yuan
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: Qiliang Yuan @ 2026-11-22 7:37 UTC (permalink / raw)
To: Sumit Semwal, Christian König, Philipp Stanner,
Gerd Hoffmann, Vivek Kasireddy, Shuah Khan
Cc: linux-media, dri-devel, linaro-mm-sig, linux-kernel,
linux-kselftest, Qiliang Yuan
QEMU and crosvm export virtio-gpu blob resources from guest memory as
udmabufs, which the host renderer and display mmap(). The guest memory is
often backed by 2 MiB huge pages, yet udmabuf maps it 4 KiB at a time:
mapping and touching a 4 GiB buffer inserts a million PTEs and takes over
200 ms.
Patch 1 has mmap() of a dma-buf align large mappings to PMD boundaries,
which any exporter mapping contiguous memory with PMDs needs, such as the
proposed VFIO dma-buf mmap() [1]. Patch 2 maps the huge page backed
ranges of a udmabuf with PMDs, which cuts the time to map and touch it
by 87-90% for hugetlb and shmem THP backed buffers and leaves 4 KiB
backed ones as they are. Patch 3 adds selftests.
[1] https://lore.kernel.org/r/20260527102319.100128-1-mattev@meta.com
Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
Qiliang Yuan (3):
dma-buf: align large mappings to PMD boundaries
udmabuf: map huge page backed ranges with PMDs
selftests/udmabuf: test mapping a huge page backed udmabuf
drivers/dma-buf/dma-buf.c | 1 +
drivers/dma-buf/udmabuf.c | 69 ++++++++++++++++-
tools/testing/selftests/drivers/dma-buf/udmabuf.c | 90 ++++++++++++++++++++++-
3 files changed, 158 insertions(+), 2 deletions(-)
---
base-commit: c408466a4dfaee4b12587e926c5b85e68c8b3c40
change-id: 20261121-feat-udmabuf-huge-pfnmap-3258272d077e
Best regards,
--
Qiliang Yuan <qiliang.yuan@linux.dev>
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH 1/3] dma-buf: align large mappings to PMD boundaries
2026-11-22 7:37 [PATCH 0/3] udmabuf: speed up mmap() of huge page backed buffers Qiliang Yuan
@ 2026-11-22 7:37 ` Qiliang Yuan
2026-11-22 7:37 ` [PATCH 2/3] udmabuf: map huge page backed ranges with PMDs Qiliang Yuan
2026-11-22 7:37 ` [PATCH 3/3] selftests/udmabuf: test mapping a huge page backed udmabuf Qiliang Yuan
2 siblings, 0 replies; 4+ messages in thread
From: Qiliang Yuan @ 2026-11-22 7:37 UTC (permalink / raw)
To: Sumit Semwal, Christian König, Philipp Stanner,
Gerd Hoffmann, Vivek Kasireddy, Shuah Khan
Cc: linux-media, dri-devel, linaro-mm-sig, linux-kernel,
linux-kselftest, Qiliang Yuan
A dma-buf exporter whose buffer is backed by physically contiguous memory
can map it with PMDs from a ->huge_fault() handler, which only works where
the user address and the buffer offset are aligned alike to PMD_SIZE.
dma_buf_fops has no ->get_unmapped_area(), so mmap() places a dma-buf at
whatever page aligned address the default allocator returns. A 4 GiB
udmabuf, for one, lands 616 KiB past a 2 MiB boundary, and then none of
its PMD-sized ranges can be mapped with a PMD.
Use thp_get_unmapped_area(), as filesystems that support huge mappings
do. It aligns a mapping that covers at least one whole PMD to the buffer
offset and leaves smaller mappings where they would be placed anyway.
Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
drivers/dma-buf/dma-buf.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index 5c6957f8839a8..015a03c8d04db 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -608,6 +608,7 @@ static void dma_buf_show_fdinfo(struct seq_file *m, struct file *file)
static const struct file_operations dma_buf_fops = {
.release = dma_buf_file_release,
.mmap = dma_buf_mmap_internal,
+ .get_unmapped_area = thp_get_unmapped_area,
.llseek = dma_buf_llseek,
.poll = dma_buf_poll,
.unlocked_ioctl = dma_buf_ioctl,
--
2.43.0
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH 2/3] udmabuf: map huge page backed ranges with PMDs
2026-11-22 7:37 [PATCH 0/3] udmabuf: speed up mmap() of huge page backed buffers Qiliang Yuan
2026-11-22 7:37 ` [PATCH 1/3] dma-buf: align large mappings to PMD boundaries Qiliang Yuan
@ 2026-11-22 7:37 ` Qiliang Yuan
2026-11-22 7:37 ` [PATCH 3/3] selftests/udmabuf: test mapping a huge page backed udmabuf Qiliang Yuan
2 siblings, 0 replies; 4+ messages in thread
From: Qiliang Yuan @ 2026-11-22 7:37 UTC (permalink / raw)
To: Sumit Semwal, Christian König, Philipp Stanner,
Gerd Hoffmann, Vivek Kasireddy, Shuah Khan
Cc: linux-media, dri-devel, linaro-mm-sig, linux-kernel,
linux-kselftest, Qiliang Yuan
QEMU and crosvm hand guest memory to the host renderer and display as
udmabufs for virtio-gpu blob resources. The guest memory is a memfd that
is often backed by 2 MiB huge pages, and the host process mmap()s the
dma-buf to access it with the CPU.
udmabuf maps every page with its own PTE: the first fault inserts a PFN
for each 4 KiB page of the whole VMA. Mapping and touching 4 GiB this way
takes over 200 ms and a million PTEs, although the memory behind it is
2048 huge pages.
Add a ->huge_fault() handler that maps a PMD-sized range with one PMD
when it is backed by an aligned run of contiguous PFNs, and falls back to
PTEs otherwise. Have the pre-mapping on the first fault skip the ranges
the handler will map, so that their PTEs don't take the place of the
PMDs, and keep pre-mapping everything when the VMA can't have PMDs.
Mapping a udmabuf over a memfd filled with data and reading a byte of
each 4 KiB page through the mapping, base and patched kernels alternating
over 3 rounds of 5 runs (7 for shmem 4K), median of the per-round medians
and range in parentheses:
backing size before (ms) after (ms) change
hugetlb 256M 13.3 (13.3-13.7) 1.7 (1.7-1.8) -87%
hugetlb 1G 53.6 (53.0-53.7) 6.5 (4.2-6.6) -88%
hugetlb 4G 211 (211-212) 23.6 (17.6-25.2) -89%
hugetlb 8G 423 (422-424) 45.8 (45.8-47.2) -89%
shmem THP 1G 52.8 (52.5-53.1) 6.3 (5.8-6.8) -88%
shmem THP 4G 210 (207-214) 20.7 (19.5-21.1) -90%
shmem 4K 1G 52.0 (51.6-52.7) 51.8 (51.5-52.9) 0%
shmem 4K 4G 209 (209-211) 208 (208-209) 0%
Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
drivers/dma-buf/udmabuf.c | 69 ++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 68 insertions(+), 1 deletion(-)
diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c
index 018937435356e..f9c908f8b53a2 100644
--- a/drivers/dma-buf/udmabuf.c
+++ b/drivers/dma-buf/udmabuf.c
@@ -44,12 +44,63 @@ struct udmabuf {
struct miscdevice *device;
};
+/*
+ * Return whether the PMD-sized range of @vma at @addr is backed by one aligned
+ * run of contiguous PFNs, which can then be mapped with a single PMD.
+ */
+static bool udmabuf_pmd_mappable(struct vm_area_struct *vma, unsigned long addr,
+ unsigned long *pfn)
+{
+ struct udmabuf *ubuf = vma->vm_private_data;
+ const pgoff_t nr = PMD_SIZE >> PAGE_SHIFT;
+ unsigned long first;
+ pgoff_t pgoff, i;
+
+ if (!IS_ENABLED(CONFIG_ARCH_SUPPORTS_PMD_PFNMAP))
+ return false;
+
+ if (!IS_ALIGNED(addr, PMD_SIZE) || addr < vma->vm_start ||
+ vma->vm_end - addr < PMD_SIZE)
+ return false;
+
+ pgoff = vma->vm_pgoff + ((addr - vma->vm_start) >> PAGE_SHIFT);
+ if (pgoff >= ubuf->pagecount || ubuf->pagecount - pgoff < nr)
+ return false;
+
+ first = page_to_pfn(ubuf->pages[pgoff]);
+ if (!IS_ALIGNED(first, nr))
+ return false;
+
+ for (i = 1; i < nr; i++) {
+ if (page_to_pfn(ubuf->pages[pgoff + i]) != first + i)
+ return false;
+ }
+
+ *pfn = first;
+ return true;
+}
+
+#ifdef CONFIG_ARCH_SUPPORTS_HUGE_PFNMAP
+static vm_fault_t udmabuf_vm_huge_fault(struct vm_fault *vmf,
+ unsigned int order)
+{
+ unsigned long pfn;
+
+ if (order != PMD_ORDER ||
+ !udmabuf_pmd_mappable(vmf->vma, vmf->address & PMD_MASK, &pfn))
+ return VM_FAULT_FALLBACK;
+
+ return vmf_insert_pfn_pmd(vmf, pfn, vmf->flags & FAULT_FLAG_WRITE);
+}
+#endif
+
static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
{
struct vm_area_struct *vma = vmf->vma;
struct udmabuf *ubuf = vma->vm_private_data;
pgoff_t pgoff = vmf->pgoff;
unsigned long addr, pfn;
+ bool pmd_ok;
vm_fault_t ret;
if (pgoff >= ubuf->pagecount)
@@ -61,7 +112,12 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
if (ret & VM_FAULT_ERROR)
return ret;
- /* pre fault */
+ /*
+ * Pre-fault the rest of the VMA, except for the ranges that
+ * udmabuf_vm_huge_fault() maps with a PMD once they are accessed.
+ */
+ pmd_ok = thp_vma_allowable_order(vma, vma->vm_flags, TVA_PAGEFAULT,
+ PMD_ORDER);
pgoff = vma->vm_pgoff;
addr = vma->vm_start;
@@ -69,6 +125,14 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
if (addr == vmf->address)
continue;
+ if (pmd_ok && IS_ALIGNED(addr, PMD_SIZE) &&
+ (vmf->address & PMD_MASK) != addr &&
+ udmabuf_pmd_mappable(vma, addr, &pfn)) {
+ pgoff += (PMD_SIZE >> PAGE_SHIFT) - 1;
+ addr += PMD_SIZE - PAGE_SIZE;
+ continue;
+ }
+
if (WARN_ON(pgoff >= ubuf->pagecount))
break;
@@ -89,6 +153,9 @@ static vm_fault_t udmabuf_vm_fault(struct vm_fault *vmf)
static const struct vm_operations_struct udmabuf_vm_ops = {
.fault = udmabuf_vm_fault,
+#ifdef CONFIG_ARCH_SUPPORTS_HUGE_PFNMAP
+ .huge_fault = udmabuf_vm_huge_fault,
+#endif
};
static int mmap_udmabuf(struct dma_buf *buf, struct vm_area_struct *vma)
--
2.43.0
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH 3/3] selftests/udmabuf: test mapping a huge page backed udmabuf
2026-11-22 7:37 [PATCH 0/3] udmabuf: speed up mmap() of huge page backed buffers Qiliang Yuan
2026-11-22 7:37 ` [PATCH 1/3] dma-buf: align large mappings to PMD boundaries Qiliang Yuan
2026-11-22 7:37 ` [PATCH 2/3] udmabuf: map huge page backed ranges with PMDs Qiliang Yuan
@ 2026-11-22 7:37 ` Qiliang Yuan
2 siblings, 0 replies; 4+ messages in thread
From: Qiliang Yuan @ 2026-11-22 7:37 UTC (permalink / raw)
To: Sumit Semwal, Christian König, Philipp Stanner,
Gerd Hoffmann, Vivek Kasireddy, Shuah Khan
Cc: linux-media, dri-devel, linaro-mm-sig, linux-kernel,
linux-kselftest, Qiliang Yuan
udmabuf maps the ranges of a buffer that are backed by huge pages with
PMDs, provided the mapping covers whole PMDs at an aligned address.
The existing tests of huge page backed buffers only map a few pages, so
they never get a PMD mapping or check what is read through one.
Map a 64 MiB udmabuf backed by 2 MiB huge pages and read back what was
written to the memfd: through a mapping of the whole buffer, which must
be PMD aligned when THP support is built in, through a mapping at an
unaligned offset, and from a forked child.
Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
tools/testing/selftests/drivers/dma-buf/udmabuf.c | 90 ++++++++++++++++++++++-
1 file changed, 89 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/drivers/dma-buf/udmabuf.c b/tools/testing/selftests/drivers/dma-buf/udmabuf.c
index d78aec662586b..e09330fece6fb 100644
--- a/tools/testing/selftests/drivers/dma-buf/udmabuf.c
+++ b/tools/testing/selftests/drivers/dma-buf/udmabuf.c
@@ -14,6 +14,7 @@
#include <sys/ioctl.h>
#include <sys/syscall.h>
#include <sys/mman.h>
+#include <sys/wait.h>
#include <linux/memfd.h>
#include <linux/udmabuf.h>
#include "kselftest.h"
@@ -130,6 +131,91 @@ static int compare_chunks(void *addr1, void *addr2, off64_t memfd_size)
return ret;
}
+#define HPAGE_SIZE (2UL << 20)
+#define HUGE_BUF_SIZE (32 * HPAGE_SIZE)
+
+/* Check that @len bytes of @buf at @off read back what was written to @mem */
+static bool check_huge_range(int buf, char *mem, off_t off, size_t len)
+{
+ char *addr;
+ size_t i;
+ bool ok = true;
+
+ addr = mmap(NULL, len, PROT_READ, MAP_SHARED, buf, off);
+ if (addr == MAP_FAILED)
+ return false;
+
+ for (i = 0; i < len; i += getpagesize()) {
+ if (addr[i] != mem[off + i]) {
+ ok = false;
+ break;
+ }
+ }
+
+ munmap(addr, len);
+ return ok;
+}
+
+/*
+ * Map a udmabuf backed by 2MB huge pages, which can be mapped with PMDs, and
+ * read it back from a PMD-aligned address, a sub-range at an unaligned offset
+ * and a forked child.
+ */
+static void test_huge_mapping(int devfd)
+{
+ struct udmabuf_create create = {};
+ int memfd, buf, status;
+ bool thp, aligned, ok;
+ char *mem, *addr;
+ pid_t pid;
+ size_t i;
+
+ memfd = create_memfd_with_seals(HUGE_BUF_SIZE, true);
+ mem = mmap_fd(memfd, HUGE_BUF_SIZE);
+ for (i = 0; i < HUGE_BUF_SIZE; i += getpagesize())
+ mem[i] = (char)(i / getpagesize());
+
+ create.memfd = memfd;
+ create.size = HUGE_BUF_SIZE;
+ buf = ioctl(devfd, UDMABUF_CREATE, &create);
+ if (buf < 0) {
+ ksft_test_result_fail("%s: [FAIL,test-8]\n", TEST_PREFIX);
+ ksft_test_result_skip("%s: [SKIP,test-9]\n", TEST_PREFIX);
+ goto out;
+ }
+
+ /* The address is only aligned when THP support is built in */
+ thp = !access("/sys/kernel/mm/transparent_hugepage", F_OK);
+ addr = mmap(NULL, HUGE_BUF_SIZE, PROT_READ, MAP_SHARED, buf, 0);
+ aligned = addr != MAP_FAILED &&
+ (!thp || !((unsigned long)addr % HPAGE_SIZE));
+ if (addr != MAP_FAILED)
+ munmap(addr, HUGE_BUF_SIZE);
+ ok = aligned && check_huge_range(buf, mem, 0, HUGE_BUF_SIZE);
+ if (ok)
+ ksft_test_result_pass("%s: [PASS,test-8]\n", TEST_PREFIX);
+ else
+ ksft_test_result_fail("%s: [FAIL,test-8]\n", TEST_PREFIX);
+
+ ok = check_huge_range(buf, mem, HPAGE_SIZE + getpagesize(),
+ 5 * HPAGE_SIZE + 3 * getpagesize());
+ pid = fork();
+ if (!pid)
+ _exit(check_huge_range(buf, mem, 0, HUGE_BUF_SIZE) ? 0 : 1);
+ if (pid < 0 || waitpid(pid, &status, 0) != pid ||
+ !WIFEXITED(status) || WEXITSTATUS(status))
+ ok = false;
+ if (ok)
+ ksft_test_result_pass("%s: [PASS,test-9]\n", TEST_PREFIX);
+ else
+ ksft_test_result_fail("%s: [FAIL,test-9]\n", TEST_PREFIX);
+
+ close(buf);
+out:
+ munmap(mem, HUGE_BUF_SIZE);
+ close(memfd);
+}
+
int main(int argc, char *argv[])
{
struct udmabuf_create create;
@@ -138,7 +224,7 @@ int main(int argc, char *argv[])
void *addr1, *addr2;
ksft_print_header();
- ksft_set_plan(7);
+ ksft_set_plan(9);
devfd = open("/dev/udmabuf", O_RDWR);
if (devfd < 0) {
@@ -268,6 +354,8 @@ int main(int argc, char *argv[])
close(buf);
close(memfd);
+
+ test_huge_mapping(devfd);
close(devfd);
ksft_print_msg("%s: ok\n", TEST_PREFIX);
--
2.43.0
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-11-22 7:38 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-22 7:37 [PATCH 0/3] udmabuf: speed up mmap() of huge page backed buffers Qiliang Yuan
2026-11-22 7:37 ` [PATCH 1/3] dma-buf: align large mappings to PMD boundaries Qiliang Yuan
2026-11-22 7:37 ` [PATCH 2/3] udmabuf: map huge page backed ranges with PMDs Qiliang Yuan
2026-11-22 7:37 ` [PATCH 3/3] selftests/udmabuf: test mapping a huge page backed udmabuf Qiliang Yuan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®