mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: Qiliang Yuan <odys.yuan@gmail.com>
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>,
	Jan Kara <jack@suse.cz>, Joanne Koong <joannelkoong@gmail.com>,
	linux-fsdevel@vger.kernel.org, linux-mm@kvack.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH] mm/readahead: look two windows ahead for the hole on a marker hit
Date: Sun, 15 Nov 2026 20:33:28 -0800	[thread overview]
Message-ID: <20261115203328.f98ce9a068849c2f2fb32a41@linux-foundation.org> (raw)
In-Reply-To: <20261022-bug-mm-readahead-async-overtaken-v1-1-c05b8d8bd2da@gmail.com>

On Thu, 22 Oct 2026 23:24:33 +0800 Qiliang Yuan <odys.yuan@gmail.com> wrote:

> When a reader hits a folio marked for readahead that the readahead
> state does not expect, page_cache_async_ra() starts the next readahead
> at the first hole within one window after the folio. If that window is
> fully cached, it reads nothing.
> 
> memcpy() in glibc loads the end of a large source before copying it
> forward, so a loader that copies a file mapping out in pieces of a few
> megabytes faults at the end of each piece first. mmap read-around then
> reads a window centered on that fault, which reaches more than one
> window past the readahead marker the copy has yet to hit. When the copy
> reaches the marker, there is no hole within a window, no readahead is
> issued, and the next piece faults on its end again. Every piece then
> gets a synchronous read-around of order-0 folios, and threads copying
> parts of one file also contend on the i_pages lock to insert them. A
> copy from offset 0 avoids this as its first read-around is clamped to
> the start of the file.
> 
> Look up to two windows ahead for the hole, which reaches past such a
> read-around, so the reader keeps an asynchronous window and its folio
> order keeps ramping up.
> 
> Copying a 16 GiB file out of a private mapping with memcpy() in 8 MiB
> pieces, read_ahead_kb 8192, cold page cache, 32 vCPU VM, mean of two
> runs:
> 
>                                   before    after
>   1 thread from offset 2 GiB     7.62 s    2.00 s
>   8 threads, one range each      3.26 s    1.08 s
>   32 threads, one range each     4.05 s    1.34 s
> 
> Random touches read the same amount, and pread() copies and copies that
> interleave 7 MiB chunks across threads take the same time within
> run-to-run noise.

OK, thanks, nice numbers but there are complications.

Readahead is all about contiguously batching physical IO.  So it
matters whether the device is SSD, NVME, spinning FeO2, SATA, IDE, etc.

And you have QEMU in the way, so reads can get served from host
pagecache (depends on QEMU setup), which also matters.

So I just don't know what to conclude from your numbers.  I *suspect*
the gains are from reduced CPU & memory-access times which is of course
good.  But was there any actual physical device access involved?  This
would probably make the before-and-after numbers even more impressive!


I tested the patch on my LLM-driven test setup, which is similar to
yours.  The results are...  ambivalent.  The things which you want
faster are indeed faster, but other things slowed down (wall of words
appended).

The slowdowns happened because the present code misses the gap beyond
mmap read-around, so each 8 MiB memcpy hunk can need another
synchronous read.  The post-patch code finds that gap and keeps
readahead going.  I measured both costs: a full second window added
about 8 us to the access, while a gap there caused 512 MiB of unused
reads across 64 accesses.


: I tested the patch against the unpatched kernel with matching
: configurations. Both booted a 32-vCPU, two-node, 24 GiB VM. The test
: used a fully written 16 GiB ext4 file on the same raw scratch disk,
: with guest read_ahead_kb set to 8192.
: 
: I ran the author's private-mmap test, copying the file with memcpy()
: in 8 MiB pieces. Before each timed run, the previous process had
: exited, I dropped the guest file cache, and mincore() confirmed that
: none of the tested file pages were resident. Medians of three runs:
: 
:                          unpatched    patched
:   1 reader, from 2 GiB      6.87 s      2.12 s
:   8 readers                 2.18 s      0.70 s
:   32 readers                1.98 s      0.76 s
: 
: Every paired run improved. Buffered pread() and 7 MiB interleaved
: copies changed little. A shuffled mmap copy was about 1.3 times
: faster with the patch in two paired runs.
: 
: I also tested the cost of looking farther ahead. The kernel marks a
: cached page so that accessing it can start the next readahead. I
: timed the test process accessing that page at 64 separate locations.
: When both upcoming 8 MiB windows were already cached, the median
: access time rose from 20 to 28 microseconds. When the first window
: was cached and the second was empty, the patch read about 512 MiB
: across those locations that the test deliberately never used. The
: median access time rose from 17 to 470 microseconds.
: 
: A memory-limited run had more memory-limit events with the patch,
: but did not establish a repeatable slowdown for a competing reader.
: 
: The guest file cache was cold, but QEMU's host cache was warm; host
: NVMe read sectors did not increase during the main runs. These
: measurements show the effect on guest readahead, not cold-drive
: throughput.


      reply	other threads:[~2026-11-16  4:33 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-22 15:24 Qiliang Yuan
2026-11-16  4:33 ` Andrew Morton [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261115203328.f98ce9a068849c2f2fb32a41@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=jack@suse.cz \
    --cc=joannelkoong@gmail.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=odys.yuan@gmail.com \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®