Re: [Patch v4 17/22] sched/cache: Avoid cache-aware scheduling for memory-heavy processes

[Tim Chen <tim.c.chen@linux.intel.com>]

On Fri, 2026-11-13 at 17:29 +0100, Jirka Hladky wrote:
> Hi Peter, Chen Yu, Tim,
> 
> 
> 

[snip]

> THE FOOTPRINT ESTIMATE NEVER LEAVES ZERO (MG class D, 64 threads)
>   Traced live on the stock kernel with bpftrace (comm-filtered),
>   across three CAS-on runs:
> 
>     - grp->footprint: 0 in 44,670 and 40,638 task_cache_work()
>       samples over two full runs (min = max = 0), while ~7M pages
>       (26.4 GiB) were resident. NUMA balancing was active: 8k-20k
>       hint faults and up to 3.3M pages migrated per run.
>     - In those two runs the footprint write never executed: all
>       ~8k-13k task_numa_placement() calls per run (one per hint
>       fault, via task_numa_fault()) returned at the scan_seq gate.
> 
>   WHY THE SIGNAL NEVER ACCUMULATES
>   1. grp->footprint is only written in task_numa_placement(),
>      which runs only when mm->numa_scan_seq changes. That counter
>      increments only in reset_ptenuma_scan(): when the NUMA scanner
>      has wrapped the whole address space.
>   2. The scanner walks the 26.4 GiB of mappings at ~0.3-1 GiB/s
>      (traced via mm->numa_scan_offset and change_prot_numa()
>      return values), so one sweep takes 30-40s or more. The 43s
>      benchmark completed 0 sweeps in two runs, 1 sweep (at t=33s)
>      in a third. The accounting runs at most once per task per
>      run.
>   3. The signal is also small: a run accumulates only 8k-20k
>      hint-fault events, 0.1-0.3% of the working set, below the
>      108 MiB threshold even when a sweep does complete.
>   4. And it is a rate signal, not a size: the increment is
>      diff = numa_faults[MEMBUF] - numa_faults[MEM] / 2, which is
>      zero at any steady fault rate.
> 
>   Cadence, coverage and formula each independently hold the
>   footprint orders of magnitude below the resident set.
> 
>   exceed_llc_capacity() is evaluated on every tick and every scan
>   epoch, but with footprint at 0 the comparison
> 
>       llc_bytes * scale  <  footprint * PAGE_SIZE
> 
>   is never true, so grp->cpu is never reset. After a 1.8s warm-up it
>   pointed at the same LLC for all 44,661 remaining samples.
> 
>   Without NUMA balancing the check never triggers, as sched-cache.rst
>   documents. With CAS on and numa_balancing=0 the same run takes
>   42.2s, the same as with NUMA balancing on (42.0s).
> 
> WHAT THE AGGREGATION COSTS (MG class D, 64 threads)
>   - With CAS on, 97% of thread-samples are on one node. The 26.4 GiB
>     working set starts ~50/50 across the nodes and NUMA balancing
>     then pulls it to the busy node (numa_maps 3.3M/2.7M pages ->
>     6.7M/0.2M; 1.42M pages migrated during the run).
>   - The cost is not on the CPU side: average CPU utilization is
>     essentially unchanged (6164% vs 6029% - about 61 of the 64
>     threads on-CPU in both configurations), and wakeup latency is
>     49us on average, 0.73ms max (perf sched). What grows is
>     UserTime, +48%: the threads keep running, each operation just
>     stalls longer on memory.
>   - The cost is the placement itself, not migration or remote
>     accesses:
>       a) 32 threads: memory fully converges to the busy node well
>          before the run ends, and the run is still 46% slower;
>       b) CAS on with numa_balancing=0: no migrations, pages stay
>          65/35, and the runtime is unchanged (42.2s);
>       c) CAS off with the process simply taskset-pinned to one
>          socket (memory 100% local from first touch, no
>          migrations): 40.3s, within the CAS-on range (40.6-42.9s).
>     64 memory-streaming threads on one socket get about two thirds
>     of the throughput they get spread across both sockets (Mop/s
>     -34%), wherever the pages live. So the only effective fix is to
>     not aggregate such a process in the first place.
> 

Thanks for your detailed analysis of the regression on NAS MG
and its root cause. Much appreciated.

> QUESTIONS
>   1. Was the sweep-boundary gating of this accounting known or
>      intended? For a workload with a large address space it means
>      the footprint sees at most one update per run; for a shorter
>      one, none at all.

Not intended. Your reading of the code is right. The footprint is
only written from task_numa_placement(), and that function returns
early unless mm->numa_scan_seq has moved, which happens once per full
sweep of the address space. The cadence is fine for NUMA placement.
But it may be too long for large footprints. The tests we
validated against had a small enough RSS that the first sweep finished
within the first seconds of the run.

One correction to your point 3. numa_faults counts pages, not fault
events. A THP hint fault adds HPAGE_PMD_NR (512) pages per fault. Your
3.3M migrated pages from 8k-20k faults suggests THP was in use. If so,
one completed sweep would push the footprint to millions of pages, far
above the 27.6k pages that equal a 108 MiB LLC. Two things would help
confirm this:

  - Was THP enabled for the MG run?
  - In the third run, did grp->footprint become non-zero after the
    sweep completed at t=33s?

If both are yes, then sweep cadence is the problem and the size
estimate is not.

>   2. Could the check fail closed when there is no usable estimate
>      (warm-up, footprint still 0, NUMA balancing off)? RSS is a poor
>      estimate of the working set, as Peter said, but it is an upper
>      bound: if RSS fits in the LLC, so does the working set. Using it
>      only on the permit side - aggregate without a footprint estimate
>      only if RSS fits - would keep the gains for small processes and
>      not aggregate the ones we cannot judge. The costs are
>      asymmetric: a missed aggregation forfeits a gain, a wrong one
>      cost 65% here.

I think you have a point. I see two ways to fix this issue,
and they are complementary.

Fix 1:

  - When footprint is 0, or NUMA balancing is off, compare the RSS of
    the mm against llc_bytes * scale. Aggregate only if RSS fits.
  - Once a footprint estimate exists, use it as today.

RSS is not used as the estimate here, only as an upper bound while
there is no footprint from the sweep. That is consistent with Peter's
objection in April. It also closes the NUMA-balancing-off gap that
sched-cache.rst documents.

The downside: a process with a large RSS but a small hot set will not
be aggregated until an estimate exists, and with a very large RSS that
can be the whole run. A process that runs for less than one sweep
never gets an estimate at all.

Fix 2:

On the cadence side: accumulate the pages reported by
task_numa_fault() into the footprint as they arrive within the
current sweep, instead of waiting for task_numa_placement(). The
footprint then grows during the first sweep and the check trips
quickly for your MG run. At the sweep boundary task_numa_placement()
only applies the decay of the previous window, so the steady state
value is unchanged.

The downside: for a workload like yours there is still a short window
where we aggregate before the footprint crosses the LLC size. With
THP each hint fault adds 512 pages, so the 27.6k pages of a 108 MiB
LLC are reached after about 54 faults. At the few hundred faults per
second you measured that is under a second once scanning starts, and
scanning starts after the 1s scan delay. That is about the 1.8s
warm-up you saw before grp->cpu settled.

Fix 2 depends on NUMA balancing. Fix 1 also covers
your numa_balancing=0 run.

I lean toward doing both: Fix 2 for the cadence, Fix 1 for NUMA
balancing off and for the first sweep. Peter, is the RSS bound
acceptable when it is used only as a gate and never as the estimate?


>   3. Our data supports Peter's point that LLC hit/miss information
>      would be the better signal: the damage is purely memory-side
>      (UserTime +59% at flat CPU% in the 112-thread run), and a
>      memory-streaming process would show a low hit rate regardless
>      of how well its footprint is estimated. Is anyone looking at
>      that?

I am not aware of patches posted along these lines. 

The two possible routes are RDT (CMT llc_occupancy and MBM bandwidth per RMID)
and PMU miss counters. RDT needs an RMID per cache group, which
resctrl owns.  Mixing RMID management with scheduler will be tricky.

The scheduler does not own the PMU, so the PMU route is
unlikely to be accepted. Near term the usable signals are the RSS
bound above and user hints through prctl to enable/disable
cache aware scheduling or modify tuning knobs per sched group.


>   4. Chen Yu mentioned that you are reworking the footprint
>      accounting for the prctl v2 series [2]. Could the detection fix
>      be part of that rework?
> 

The v2 work on footprint is about correctness, not cadence. 

Fix 2 I mentioned above touches the same footprint code that the 
v2 series reworks, so the two are not independent. I think we
would consider whether to add the proposed fixes in the same
series or leave it as a follow on.

Would you consider testing the patches when we post them?
We will CC you.

Tim

> [1] <50a8055e38958cdc39e07435aaad56e4719b2bd3.1794507151.git.tim.c.chen@linux.intel.com>
> [2] <5c912bd3-8701-4395-a118-4ee6e09233e2@intel.com>
> [3] NAS Parallel Benchmarks: https://www.nas.nasa.gov/software/npb.html
>     MG is the multigrid kernel: four V-cycles of 27-point stencil
>     sweeps over a 3D grid hierarchy (finest grid 512^3 for class C,
>     1024^3 for class D) solving a Poisson equation. Each sweep
>     streams through a few double-precision arrays - a 3.3 GiB working
>     set for class C, 26.4 GiB for class D - far beyond any LLC, with
>     little reuse between sweeps: the kernel is memory-bandwidth-bound
>     by construction.
> 
> The per-run logs, the placement/migration/sweep traces, and the
> bpftrace scripts behind them are available if useful.
> 
> Regards,
> Jirka Hladky
> Red Hat
>