khugepaged selftest: intermittent collapse_full(madvise:anon) failure and low-memory false failures on arm64 64K

khugepaged selftest: intermittent collapse_full(madvise:anon) failure and low-memory false failures on arm64 64K

[Zhijian Han]

Hi,

I ran the khugepaged selftest (as of linux-next 20261019, tip 3d229ce89862,
which includes Kiryl's "run every supported collapse order by default"
series) across several QEMU configurations while setting up a test
environment for mTHP collapse work. Two observations that may be worth
acting on; full logs available on request.

Environment: QEMU virt, cortex-a76, 2 CPUs, khugepaged test run from a
statically linked binary inside a busybox initramfs (no distro userspace).

1. Intermittent collapse_full(madvise:anon) failure on arm64 64K pages
---------------------------------------------------------------------
With 6G of guest memory (so order-13 PMD folios are allocatable), over 7
complete runs I saw one failure:

  # Run test: collapse_full (madvise:anon)
  # Collapse multiple fully populated PTE table... Fail: Bad return value
  not ok 80 collapse_full

The other 6 runs were fully green (102 pass / 0 fail / 3 skip each). The
failing run did not report an allocation failure or OOM in the kernel log.
A minimal probe (mmap 512M, memset, single MADV_COLLAPSE) succeeds
deterministically on the same setup, so the failure seems specific to the
test's sequence rather than a plain allocation shortfall.

This is the same symptom family as bddffa2f9cbf ("kselftest: mm: fix
intermittent failure khugepaged test") — there may be a residual flaky
spot on 64K-page configurations. I was not able to reproduce it
deterministically; happy to run more instrumented iterations if that
would help.

2. Low-memory runs fail without a skip guard
--------------------------------------------
With 2G of guest memory, six cases fail outright (starting with
"Allocate huge page on fault... Fail") because an order-13 (512M) folio
cannot be allocated. collapse_compound_extreme() already skips itself
with "PMD too large for fault-time THP construction" on this
configuration; the PMD-order cases have no equivalent protection, so a
low-memory or fragmentation-heavy environment reports what looks like
kernel bugs rather than resource limitations.

A skip guard along the lines of "cannot allocate a PMD-order folio even
before the test proper — skipping" (or a documented minimum memory
requirement) would make the suite's results readable on such setups.

3. Confirmed skips on 64K (for the coverage ledger)
---------------------------------------------------
  ok 9   # SKIP collapse_order_mixed_sources: no supported source order below target
  ok 91/92 # SKIP collapse_compound_extreme: PMD too large for fault-time THP construction

These match the gaps noted in the v6 cover letter.

Baseline for comparison: x86-64 4K with a swap device attached is fully
green (76 pass / 0 fail / 1 skip — the mixed_sources skip).

Assisted-by: LLM (QEMU test harness)

-- 
Zhijian Han <hanzhijian1991@gmail.com>

Re: khugepaged selftest: intermittent collapse_full(madvise:anon) failure and low-memory false failures on arm64 64K

[Kiryl Shutsemau]

On Tue, Oct 20, 2026 at 10:41:16AM +0800, Zhijian Han wrote:
> Hi,
> 
> I ran the khugepaged selftest (as of linux-next 20261019, tip 3d229ce89862,
> which includes Kiryl's "run every supported collapse order by default"
> series) across several QEMU configurations while setting up a test
> environment for mTHP collapse work. Two observations that may be worth
> acting on; full logs available on request.
> 
> Environment: QEMU virt, cortex-a76, 2 CPUs, khugepaged test run from a
> statically linked binary inside a busybox initramfs (no distro userspace).
> 
> 1. Intermittent collapse_full(madvise:anon) failure on arm64 64K pages
> ---------------------------------------------------------------------
> With 6G of guest memory (so order-13 PMD folios are allocatable), over 7
> complete runs I saw one failure:
> 
>   # Run test: collapse_full (madvise:anon)
>   # Collapse multiple fully populated PTE table... Fail: Bad return value
>   not ok 80 collapse_full
> 
> The other 6 runs were fully green (102 pass / 0 fail / 3 skip each). The
> failing run did not report an allocation failure or OOM in the kernel log.
> A minimal probe (mmap 512M, memset, single MADV_COLLAPSE) succeeds
> deterministically on the same setup, so the failure seems specific to the
> test's sequence rather than a plain allocation shortfall.

A failed allocation would not show in the log: MADV_COLLAPSE allocates
with GFP_TRANSHUGE, which has __GFP_NOWARN, and returns ENOMEM.
collapse_full asks for four 512M folios after faulting in 2G of base
pages; your probe asks for one.

Could you check whether thp_collapse_alloc_failed in /proc/vmstat moves
in a failing run, or print errno?

> 
> This is the same symptom family as bddffa2f9cbf ("kselftest: mm: fix
> intermittent failure khugepaged test") — there may be a residual flaky
> spot on 64K-page configurations. I was not able to reproduce it
> deterministically; happy to run more instrumented iterations if that
> would help.
> 
> 2. Low-memory runs fail without a skip guard
> --------------------------------------------
> With 2G of guest memory, six cases fail outright (starting with
> "Allocate huge page on fault... Fail") because an order-13 (512M) folio
> cannot be allocated. collapse_compound_extreme() already skips itself
> with "PMD too large for fault-time THP construction" on this
> configuration; the PMD-order cases have no equivalent protection, so a
> low-memory or fragmentation-heavy environment reports what looks like
> kernel bugs rather than resource limitations.
> 
> A skip guard along the lines of "cannot allocate a PMD-order folio even
> before the test proper — skipping" (or a documented minimum memory
> requirement) would make the suite's results readable on such setups.

Generally, I don't think it is useful to run this selftest in a
memory-constrained environment. It is not the focus of the selftest.

Getting a reliable guard can be tricky, as we need not only free memory,
but memory that can be allocated for a PMD-size THP. Fragmentation or the
migratetype of the memory can also limit it.

Let's just keep it out of scope for the selftest.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov