khugepaged selftest: intermittent collapse_full(madvise:anon) failure and low-memory false failures on arm64 64K

khugepaged selftest: intermittent collapse_full(madvise:anon) failure and low-memory false failures on arm64 64K

[Zhijian Han]

Hi,

I ran the khugepaged selftest (as of linux-next 20261019, tip 3d229ce89862,
which includes Kiryl's "run every supported collapse order by default"
series) across several QEMU configurations while setting up a test
environment for mTHP collapse work. Two observations that may be worth
acting on; full logs available on request.

Environment: QEMU virt, cortex-a76, 2 CPUs, khugepaged test run from a
statically linked binary inside a busybox initramfs (no distro userspace).

1. Intermittent collapse_full(madvise:anon) failure on arm64 64K pages
---------------------------------------------------------------------
With 6G of guest memory (so order-13 PMD folios are allocatable), over 7
complete runs I saw one failure:

  # Run test: collapse_full (madvise:anon)
  # Collapse multiple fully populated PTE table... Fail: Bad return value
  not ok 80 collapse_full

The other 6 runs were fully green (102 pass / 0 fail / 3 skip each). The
failing run did not report an allocation failure or OOM in the kernel log.
A minimal probe (mmap 512M, memset, single MADV_COLLAPSE) succeeds
deterministically on the same setup, so the failure seems specific to the
test's sequence rather than a plain allocation shortfall.

This is the same symptom family as bddffa2f9cbf ("kselftest: mm: fix
intermittent failure khugepaged test") — there may be a residual flaky
spot on 64K-page configurations. I was not able to reproduce it
deterministically; happy to run more instrumented iterations if that
would help.

2. Low-memory runs fail without a skip guard
--------------------------------------------
With 2G of guest memory, six cases fail outright (starting with
"Allocate huge page on fault... Fail") because an order-13 (512M) folio
cannot be allocated. collapse_compound_extreme() already skips itself
with "PMD too large for fault-time THP construction" on this
configuration; the PMD-order cases have no equivalent protection, so a
low-memory or fragmentation-heavy environment reports what looks like
kernel bugs rather than resource limitations.

A skip guard along the lines of "cannot allocate a PMD-order folio even
before the test proper — skipping" (or a documented minimum memory
requirement) would make the suite's results readable on such setups.

3. Confirmed skips on 64K (for the coverage ledger)
---------------------------------------------------
  ok 9   # SKIP collapse_order_mixed_sources: no supported source order below target
  ok 91/92 # SKIP collapse_compound_extreme: PMD too large for fault-time THP construction

These match the gaps noted in the v6 cover letter.

Baseline for comparison: x86-64 4K with a swap device attached is fully
green (76 pass / 0 fail / 1 skip — the mixed_sources skip).

Assisted-by: LLM (QEMU test harness)

-- 
Zhijian Han <hanzhijian1991@gmail.com>

Re: khugepaged selftest: intermittent collapse_full(madvise:anon) failure and low-memory false failures on arm64 64K

[Kiryl Shutsemau]

On Tue, Oct 20, 2026 at 10:41:16AM +0800, Zhijian Han wrote:
> Hi,
> 
> I ran the khugepaged selftest (as of linux-next 20261019, tip 3d229ce89862,
> which includes Kiryl's "run every supported collapse order by default"
> series) across several QEMU configurations while setting up a test
> environment for mTHP collapse work. Two observations that may be worth
> acting on; full logs available on request.
> 
> Environment: QEMU virt, cortex-a76, 2 CPUs, khugepaged test run from a
> statically linked binary inside a busybox initramfs (no distro userspace).
> 
> 1. Intermittent collapse_full(madvise:anon) failure on arm64 64K pages
> ---------------------------------------------------------------------
> With 6G of guest memory (so order-13 PMD folios are allocatable), over 7
> complete runs I saw one failure:
> 
>   # Run test: collapse_full (madvise:anon)
>   # Collapse multiple fully populated PTE table... Fail: Bad return value
>   not ok 80 collapse_full
> 
> The other 6 runs were fully green (102 pass / 0 fail / 3 skip each). The
> failing run did not report an allocation failure or OOM in the kernel log.
> A minimal probe (mmap 512M, memset, single MADV_COLLAPSE) succeeds
> deterministically on the same setup, so the failure seems specific to the
> test's sequence rather than a plain allocation shortfall.

A failed allocation would not show in the log: MADV_COLLAPSE allocates
with GFP_TRANSHUGE, which has __GFP_NOWARN, and returns ENOMEM.
collapse_full asks for four 512M folios after faulting in 2G of base
pages; your probe asks for one.

Could you check whether thp_collapse_alloc_failed in /proc/vmstat moves
in a failing run, or print errno?

> 
> This is the same symptom family as bddffa2f9cbf ("kselftest: mm: fix
> intermittent failure khugepaged test") — there may be a residual flaky
> spot on 64K-page configurations. I was not able to reproduce it
> deterministically; happy to run more instrumented iterations if that
> would help.
> 
> 2. Low-memory runs fail without a skip guard
> --------------------------------------------
> With 2G of guest memory, six cases fail outright (starting with
> "Allocate huge page on fault... Fail") because an order-13 (512M) folio
> cannot be allocated. collapse_compound_extreme() already skips itself
> with "PMD too large for fault-time THP construction" on this
> configuration; the PMD-order cases have no equivalent protection, so a
> low-memory or fragmentation-heavy environment reports what looks like
> kernel bugs rather than resource limitations.
> 
> A skip guard along the lines of "cannot allocate a PMD-order folio even
> before the test proper — skipping" (or a documented minimum memory
> requirement) would make the suite's results readable on such setups.

Generally, I don't think it is useful to run this selftest in a
memory-constrained environment. It is not the focus of the selftest.

Getting a reliable guard can be tricky, as we need not only free memory,
but memory that can be allocated for a PMD-size THP. Fragmentation or the
migratetype of the memory can also limit it.

Let's just keep it out of scope for the selftest.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: khugepaged selftest: intermittent collapse_full(madvise:anon) failure and low-memory false failures on arm64 64K

[Zhijian Han]

Hi Kiryl,

Thanks — instrumented as suggested: errno + thp_collapse_alloc_failed
printed on failure, then 30 full runs on the same setup (QEMU arm64 64K,
6G, linux-next 20261019). The failure reproduced 11/30 times and the
data does not point at allocation:

  6x EAGAIN, 5x EINVAL, 0x ENOMEM
  thp_collapse_alloc_failed=0 in every failing run

Per madvise_collapse_errno(), EAGAIN maps to transient page states
(SCAN_PAGE_LOCK/PAGE_LRU/PAGE_DIRTY_OR_WRITEBACK/...) and EINVAL to
"intrinsic" scan results — neither is the allocation-failure path,
which would have shown ENOMEM and a bump in the counter.

One thing that stands out: madvise_collapse_retry() retries EAGAIN only
once, so two consecutive EAGAINs fail the case even though the errno
table documents EAGAIN as "trying again might succeed". Could the test
retry harder on EAGAIN (loop with a short sleep), or is the single retry
deliberate to expose genuine kernel-side transients?

For the EINVAL cases I would need kernel-side detail on which scan
result is being hit; happy to run instrumented (tracefs or a debug
counter) if you point me at the right knob.

Full 30-run logs and the instrumentation diff available on request.

Understood and agreed on the low-memory guard — dropped.

Assisted-by: LLM (QEMU test harness and instrumentation)

-- 
Zhijian Han