Re: [PATCH v8 0/8] mm: Hot page tracking and promotion infrastructure

[Bharata B Rao <bharata@amd.com>]

On 10-Aug-26 9:46 AM, Matthew Wilcox wrote:
> On Mon, Aug 10, 2026 at 11:38:14AM +0800, Yongting Lin wrote:
>> On Tue, Jul 28, 2026 at 07:24:46PM +0100, Matthew Wilcox wrote:
>>> I don't think we should do this.  I don't think it's a use-case that
>>> customers actually want; I think it's something CPU vendors think their
>>> customers want.  I've said this repeatedly over the months this patchset
>>> has been in development.  I say no.
>>
>> From our perspective, this is not merely a hypothetical use case. We
>> are actively developing and evaluating CXL tiered-memory systems, and
>> expect promoting CXL-resident pages that become hot back to DRAM to be
>> a practical requirement.
> 
> But what *is* your use case?  Truly hot data ends up in L1/L2/L3 no
> matter whether it came across the ridiculously high latency CXL link or
> from regular DRAM.  So we're talking about "warm" data that's presumably
> measured in gigabytes (since current server CPUs have about half a
> gigabyte of LLC)
> 
> I see the basis for saying "this task is low priority, it gets its memory
> allocated on CXL" and "this task is high priority, it gets its memory
> allocated on DRAM".  I don't see the use case where we're trying to figure
> out that region A of this task is sufficiently warmer than region B, and so
> we want to demote region B back to CXL and promote region A back to DRAM.
> 
> I particularly don't see a case for trying to do it on single-page
> granularity.  There's just too much damn information to track.  Maybe at
> a 2MB or 1GB boundary, but not per page.

Replying to the observation about per-page tracking information...

Since pghot has seen several versions and implementations over time, things have
changed multiple times and it is also easy to miss the information about what
gets tracked per-page and how it gets used from my long cover note. Hence
pasting some info here if it helps.

How pghot works
===============
- Tracks frequency and last access time.
- Additionally, the accessing NUMA node ID (NID) for each recorded
  access is tracked in precision mode.
- These hotness parameters are maintained in a per-PFN hotness record
  within a per-section map hanging off the existing mem_section data
  structure. The map is RCU-protected and also carries a section-level
  "hot" flag used to gate scanning by kmigrated.
  - In default mode, one byte (u8) is used for the hotness record. 5 bits
    store time using a bucketing scheme representing up to 4s with
    HZ=1000. The default toptier NID (0) is used as the promotion target
    and can be changed via the vm.pghot_target_nid sysctl.
  - In precision mode, 4 bytes (u32) are used for each hotness record.
    14 bits store time, representing around 16s with HZ=1000, and the
    accessing NID is stored per-PFN as the promotion target.
- Classifies pages as hot based on configurable thresholds.
- Pages classified as hot are marked ready for migration using the ready
  bit (MSB of the hotness record in both modes).
- Per-lower-tier-node kmigrated threads periodically scan the PFNs of
  lower-tier nodes, checking the migration-ready bit to perform batched
  migrations. The scan interval and batch size are configurable via
  debugfs tunables.

Memory overhead
===============
Default mode: 1 byte per lower-tier PFN. For 1TB of lower-tier memory
this amounts to 256MB overhead (assuming 4K pages).

Precision mode: 4 bytes per lower-tier PFN. For 1TB of lower-tier memory
this amounts to 1GB overhead.

Regards,
Bharata.