[REGRESSION] mm/filemap: mmap read-around stays off for an open file after sparse faults, since 7.2

[REGRESSION] mm/filemap: mmap read-around stays off for an open file after sparse faults, since 7.2

[Trace]

Hello,

since 7.2, once an open file has taken more than MMAP_LOTSAMISS mmap
misses, it gets no more mmap read-around, even if the access then turns
sequential. For a reader that does not run into pages an earlier
read-around brought in, it stays that way until the file is opened
again. Each later page is a synchronous 4 KiB read, which on NFS is a
round trip.

I think it comes from this series:

  0b9c0aeba938 ("mm/filemap: count only the faulting address as a mmap hit")
  9b0fcac3cfe7 ("mm/filemap: do not count FAULT_FLAG_TRIED retries as mmap hits")

I have not bisected or built a revert. By reading mm/filemap.c, v7.1
and the current 6.18.y have the old accounting and v7.2 onwards the new.
I measured 7.3.1 and 6.18.60, and the program below also gives 16384
on 7.2.14 (with the zen patches) on a SATA SSD. do_sync_mmap_readahead()
and do_async_mmap_readahead() are unchanged from v7.3 in next-20261103
and in mm-unstable, mm-hotfixes-unstable and mm-new of 2026-11-03.

As I read it: above MMAP_LOTSAMISS, do_sync_mmap_readahead() returns
without reading, and mmap_miss only goes down on a hit. With no
read-around the next page is not in the cache, so the next fault is
another miss, and nothing lowers the counter again. Before the series,
the retry after each read-around counted the pages it had just brought
in, so a sparse reader did not reach the limit. The series' tests run
random, stride and sequential access each on its own, not one after
another on the same open file.

The program below writes a 2600 MiB file, drops it with
POSIX_FADV_DONTNEED, touches 2000 pages 1 MiB apart through a private
mapping, then reads 64 MiB sequentially and counts major faults. x86_64,
4 KiB pages, read_ahead_kb 128, ext4, plenty of free memory (7.3.1 on
NVMe, 6.18.60 on a SATA SSD):

                                          7.3.1   6.18.60
  sequential only                             1         1
  sparse pass, then sequential            16384         1
  sparse, MADV_SEQUENTIAL, sequential       512       512
  sparse, close and reopen, sequential        1         1

The same pattern in a Python version, on 7.3.1, on NFS v4.2 over 1 GbE
through WireGuard: 80-90 MiB/s with no sparse pass, 5.7 MiB/s after it.

How I met it: PyTorch loading a model from safetensors on NFS, on a
Radeon AI PRO R9700 with ROCm, where the driver faults in the mapping:

  folio_wait_bit_common
  nfs_do_read_folio
  nfs_read_folio
  filemap_read_folio
  filemap_fault
  __do_fault
  do_fault
  __handle_mm_fault
  handle_mm_fault
  hmm_range_fault_locked
  amdgpu_hmm_range_get_pages
  svm_range_validate_and_map
  svm_range_set_attr
  kfd_ioctl

safetensors maps the whole file (torch.UntypedStorage.from_file,
MAP_PRIVATE) and builds one tensor view per entry, which takes one
read fault per tensor, mostly on its last page. A model file holds
hundreds of tensors spread over the file, so mmap_miss passes
MMAP_LOTSAMISS within a few seconds of the mmap, before any weight is
read (bpftrace on filemap_fault, reading file->f_ra.mmap_miss). The
GPU driver then faults the weights in through the same mapping, every
page a synchronous 4 KiB nfs_read_folio, about 1800 a second.
Read-around came back only when the GPU's walk happened to continue
through a window that one of the first hundred faults had read around:
async readahead restarted there, and about 900 hits later mmap_miss was
under the limit again. With the page cache cold, one model's first
picture took 700 s on 7.3.1, 512 s of it below 10 MB/s, against 239 s
on 6.18; a larger model's first run took 54 minutes against 13.

Is read-around meant to stay off for the life of the open file once the
limit is passed? I have no patch to offer.

#regzbot introduced: v7.1..v7.2

Thanks,
Trace

---8<--- mmap-ra.c
/*
 * mmap-ra.c: does a sparse first pass turn off mmap read-around for the
 * rest of an open file?
 *
 *   gcc -O2 -o mmap-ra mmap-ra.c && ./mmap-ra /path/on/a/disk/testfile
 *
 * Writes a 2600 MiB file (removed at the end), drops it from the page
 * cache with POSIX_FADV_DONTNEED, and counts major faults for a 64 MiB
 * sequential read through a private mapping.
 */
#define _GNU_SOURCE
#include <err.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/resource.h>
#include <unistd.h>

#define MIB	(1UL << 20)
#define SIZE	(2600 * MIB)
#define SPARSE	2000		/* single pages, 1 MiB apart */
#define SEQ_OFF	(2100 * MIB)
#define SEQ_LEN	(64 * MIB)

static long majflt(void)
{
	struct rusage ru;

	getrusage(RUSAGE_SELF, &ru);
	return ru.ru_majflt;
}

static char *open_map(const char *path, int *fd)
{
	char *m;

	*fd = open(path, O_RDONLY);
	if (*fd < 0)
		err(1, "open");
	m = mmap(NULL, SIZE, PROT_READ, MAP_PRIVATE, *fd, 0);
	if (m == MAP_FAILED)
		err(1, "mmap");
	return m;
}

static void run(const char *path, const char *name, int sparse, int advice,
		int reopen)
{
	long page = sysconf(_SC_PAGESIZE), before;
	volatile char sink;
	unsigned long off;
	char *m;
	int fd;

	fd = open(path, O_RDONLY);
	if (fd < 0)
		err(1, "open");
	if (posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED))
		errx(1, "posix_fadvise");
	close(fd);

	m = open_map(path, &fd);
	if (sparse)
		for (off = 0; off < SPARSE; off++)
			sink = m[off * MIB + 37 * page];
	if (advice && madvise(m, SIZE, advice))
		err(1, "madvise");
	if (reopen) {
		munmap(m, SIZE);
		close(fd);
		m = open_map(path, &fd);
	}

	before = majflt();
	for (off = SEQ_OFF; off < SEQ_OFF + SEQ_LEN; off += page)
		sink = m[off];
	printf("%-40s %6ld major faults\n", name, majflt() - before);

	(void)sink;
	munmap(m, SIZE);
	close(fd);
}

int main(int argc, char **argv)
{
	static char buf[MIB];
	unsigned long i;
	int fd;

	if (argc != 2)
		errx(2, "usage: %s FILE", argv[0]);

	fd = open(argv[1], O_WRONLY | O_CREAT | O_EXCL, 0600);
	if (fd < 0)
		err(1, "create");
	memset(buf, 0x5a, sizeof(buf));
	for (i = 0; i < SIZE / MIB; i++)
		if (write(fd, buf, MIB) != (ssize_t)MIB)
			err(1, "write");
	if (fsync(fd))
		err(1, "fsync");
	close(fd);

	setvbuf(stdout, NULL, _IOLBF, 0);
	run(argv[1], "sequential only", 0, 0, 0);
	run(argv[1], "sparse pass, then sequential", 1, 0, 0);
	run(argv[1], "sparse, MADV_SEQUENTIAL, sequential", 1, MADV_SEQUENTIAL, 0);
	run(argv[1], "sparse, close and reopen, sequential", 1, 0, 1);

	unlink(argv[1]);
	return 0;
}