mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [RFC PATCH 0/5] gup: introduce folio-range interface for pinning user memory
@ 2026-11-18 21:34 Swarna Prabhu
  2026-11-18 21:34 ` [PATCH 1/5] mm/gup: add struct gup_out output descriptor Swarna Prabhu
                   ` (6 more replies)
  0 siblings, 7 replies; 8+ messages in thread
From: Swarna Prabhu @ 2026-11-18 21:34 UTC (permalink / raw)
  To: akpm, david, jgg, jhubbard, viro
  Cc: linux-mm, linux-block, linux-kernel, dave, riel, yeoreum.yun,
	hch, Swarna Prabhu

From: Swarna Prabhu <s.prabhu@samsung.com>

This series is an RFC to gather design feedback to introduce a folio-range 
interface for pinning user memory in the GUP fast path. Pinned memory is 
returned as folio ranges instead of a page array: a run of consecutive PTEs 
mapping consecutive pages of one folio is pinned once and handed back as a single 
range. Block direct I/O converts each range into a single bvec, hence reducing the 
per-page pin and merge work.

Base and dependencies
=====================

This series is rebased on the following two series applied:

  - "mm: gup: batch PTE-mapped large folios in gup_fast_pte_range()"
    v4 (Yeoreum Yun; originally posted by Yuan-Hao Hsu [3]), applied to
    mm/core.git gup-7.5.ptebatch [4]. It batches the reference grabs for
    PTE-mapped large folios in GUP-fast.

  - "batch lookups in follow_page_mask()" v5 (Rik van Riel) [5]. It makes
    follow_page_mask() return a page count and record pages through
    gup_fill_pages().

    Note: "[PATCH v5 1/8] mm/gup: break out gup_fill_pages() helper" was
    posted just as testing for this RFC was done. To avoid delaying
    feedback on the overall design, this RFC retains the tested base
    linked in [5] and will be rebased onto the latest revision for next
    version.

This series builds on both: the run detection in gup_fast_pte_range() and
the batched slow-path lookup are reused, and this series changes how the
result of a run is recorded and returned.

Background
==========

Kundan's LPC '24 talk "Transitioning get_user_pages to folio based mapping"
[1] proposed a struct folio_vec (based on bcachefs) passed in place of a
pages array. Building on that and on suggestions from the community [2], 
this series exposes folio + index + nr_pages, where each range spans a 
single folio:

        struct folio_range {
                struct folio *folio;
                unsigned int  idx;
                unsigned int  nr_pages;
        };

Design
======

A "struct gup_out" descriptor in mm/gup.c holds either the page array or
the folio-range array. The fast and slow page-table walkers append to the
descriptor instead of advancing a pages[] pointer, so a single walk
serves both interfaces without duplicating the walkers.

The PTE batching in gup_fast_pte_range() is provided by Yeoreum Yun's
series [4], which this RFC builds on. This series reuses that batching
and its page-table revalidation logic. For the folio-range interface,
each pinned run is recorded directly as a folio range instead of being
expanded into individual page pointers in the output array.

pin_user_folio_ranges_fast() stops once the caller's range array is
full, so callers size the array in ranges, independently of the number
of pages requested. A single GUP-fast walk is additionally capped at
EXTRACT_MAX_PAGES (1024 pages) to bound the time spent with interrupts
disabled when the buffer is backed by small pages.

unpin_user_folio_ranges() walks the range array and calls gup_put_folio()
once per range, releasing the references pin_user_folio_ranges_fast()
took for it.

In iov_iter_extract_bvecs(), the pinned folio ranges are stored into the 
free slot of the bvec array and converted in place, one bvec per range. 
Block direct I/O picks this up through bio_iov_iter_get_pages() without 
changes to the block layer.

Non-folio mappings
==================

Memory that is not backed by folios, such as pages a driver inserted
into a VM_MIXEDMAP VMA, makes pin_user_folio_ranges_fast() fail with
-EOPNOTSUPP when no pages were pinned; such iterators fall back to the
page-array path.

For folio-range output, gup_fast_folio_allowed() additionally requires
a non-NULL folio->mapping, except for hugetlb folios. If the mapping is
NULL, it falls back to the slow path, where the VMA can be checked.
The existing file-backed and secretmem restrictions remain unchanged.

FOLL_LONGTERM is not supported yet and is rejected.

Testing
=======

Measured the time spent in bio_iov_iter_get_pages() per I/O with
function_graph, on the baseline and the baseline plus this series.
Both kernels ran with the performance governor, boost disabled, all
C-states except POLL disabled, and fio pinned to one CPU:

  trace-cmd record -p function_graph -g bio_iov_iter_get_pages \
    --max-graph-depth 8 -O funcgraph-overhead -O nofuncgraph-irqs -F -c \
    fio --name=io_uring_test --filename=/dev/nvme1n1 --ioengine=io_uring \
        --rw=randwrite --direct=1 --bs=<bs> --iodepth=8 --iomem=mmap \
        --iomem_align=2M --numjobs=1 --size=64M --cpus_allowed=23

Median of calls not interrupted during the walk:

  bs    buffer          baseline        patched          result
  128K  64 kB mTHP      13.08 us        13.09 us         neutral
  1M    64 kB mTHP      63.0 us         63.6 us          neutral
  4M    2 MB THP (PMD)  25.9 us         8.65 us          ~3.0x faster
  8M    2 MB THP (PMD)  49.4 us        ~17.0 us          ~2.9x faster

Note: the baseline is the upstream kernel with Yeoreum Yun's [4] and
Rik van Riel's [5] changes applied.

For larger I/O, the baseline's bio extraction path requests at max
256 pages per GUP call, limited by the available bvec slots. This
series instead uses those slots for folio ranges, each of which can 
describe many pages, thereby allowing the series to request up to 1024 
pages per call which accounts for the speed up observed in 
'bio_iov_iter_get_pages' for 4MB and 8MB DIRECT I/Os.

Testing: Ran selftests gup_test, gup_longterm and cow on the patched kernel
and no regressions were observed.

Link [1]: https://lpc.events/event/18/contributions/1773/
Link [2]: https://lore.kernel.org/all/f2b5189f-10de-4685-97f3-6ee08d159743@kernel.org/
Link [3]: https://lore.kernel.org/all/20260917083747.786-1-aa9736195201@gmail.com/
Link [4]: https://lore.kernel.org/all/20261110-refactor_gup_fast-v4-0-e779a7674274@arm.com/
Link [5]: https://lore.kernel.org/all/20261102193026.3860700-1-riel@surriel.com/

Swarna Prabhu (5):
  mm/gup: add struct gup_out output descriptor
  mm/gup: make the slow path walkers record into struct gup_out
  mm/gup: make the GUP-fast walkers record into struct gup_out
  mm/gup: return pinned memory as folio ranges
  iov_iter: extract bvecs of user-backed iters from folio ranges

 include/linux/mm.h       |  37 +++
 include/linux/mm_types.h |  23 ++
 lib/iov_iter.c           | 100 ++++++
 mm/gup.c                 | 695 +++++++++++++++++++++++++++++++--------
 4 files changed, 717 insertions(+), 138 deletions(-)

-- 
2.43.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2026-11-19  0:19 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-18 21:34 [RFC PATCH 0/5] gup: introduce folio-range interface for pinning user memory Swarna Prabhu
2026-11-18 21:34 ` [PATCH 1/5] mm/gup: add struct gup_out output descriptor Swarna Prabhu
2026-11-18 21:34 ` [PATCH 2/5] mm/gup: make the slow path walkers record into struct gup_out Swarna Prabhu
2026-11-18 21:34 ` [PATCH 3/5] mm/gup: make the GUP-fast " Swarna Prabhu
2026-11-18 21:34 ` [PATCH 4/5] mm/gup: return pinned memory as folio ranges Swarna Prabhu
2026-11-18 21:34 ` [PATCH 5/5] iov_iter: extract bvecs of user-backed iters from " Swarna Prabhu
2026-11-18 23:28 ` [RFC PATCH 0/5] gup: introduce folio-range interface for pinning user memory David Hildenbrand (Arm)
2026-11-19  0:19 ` John Hubbard

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®