mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Zi Yan" <ziy@nvidia.com>
To: "Gregory Price" <gourry@gourry.net>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"kernel-team@meta.com" <kernel-team@meta.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"david@kernel.org" <david@kernel.org>,
	"ljs@kernel.org" <ljs@kernel.org>,
	"liam@infradead.org" <liam@infradead.org>,
	"vbabka@kernel.org" <vbabka@kernel.org>,
	"rppt@kernel.org" <rppt@kernel.org>,
	"surenb@google.com" <surenb@google.com>,
	"mhocko@suse.com" <mhocko@suse.com>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"juri.lelli@redhat.com" <juri.lelli@redhat.com>,
	"vincent.guittot@linaro.org" <vincent.guittot@linaro.org>,
	"dietmar.eggemann@arm.com" <dietmar.eggemann@arm.com>,
	"rostedt@goodmis.org" <rostedt@goodmis.org>,
	"bsegall@google.com" <bsegall@google.com>,
	"mgorman@suse.de" <mgorman@suse.de>,
	"vschneid@redhat.com" <vschneid@redhat.com>,
	"kprateek.nayak@amd.com" <kprateek.nayak@amd.com>,
	"baolin.wang@linux.alibaba.com" <baolin.wang@linux.alibaba.com>,
	"nico.pache@linux.dev" <nico.pache@linux.dev>,
	"ryan.roberts@arm.com" <ryan.roberts@arm.com>,
	"dev.jain@arm.com" <dev.jain@arm.com>,
	"baohua@kernel.org" <baohua@kernel.org>,
	"lance.yang@linux.dev" <lance.yang@linux.dev>,
	"usama.arif@linux.dev" <usama.arif@linux.dev>,
	"kas@kernel.org" <kas@kernel.org>,
	"matthew.brost@intel.com" <matthew.brost@intel.com>,
	"joshua.hahnjy@gmail.com" <joshua.hahnjy@gmail.com>,
	"rakie.kim@sk.com" <rakie.kim@sk.com>,
	"byungchul@sk.com" <byungchul@sk.com>,
	"ying.huang@linux.alibaba.com" <ying.huang@linux.alibaba.com>,
	"Alistair Popple" <apopple@nvidia.com>,
	"jannh@google.com" <jannh@google.com>,
	"pfalcato@suse.de" <pfalcato@suse.de>,
	"osalvador@suse.de" <osalvador@suse.de>,
	"hannes@cmpxchg.org" <hannes@cmpxchg.org>,
	"raghavendra.kt@amd.com" <raghavendra.kt@amd.com>
Subject: Re: [PATCH v2 0/4] sched/numa: stop VMA scan filters from gating promotion
Date: Fri, 18 Sep 2026 16:56:34 -0400	[thread overview]
Message-ID: <DLIQQNUL2RTD.IGN8MYHYMW1E@nvidia.com> (raw)
In-Reply-To: <20260911001826.2109390-1-gourry@gourry.net>

On Thu Sep 10, 2026 at 8:18 PM EDT, Gregory Price wrote:
> NUMA balancing uses hinting faults for both task placement and memory-tier
> promotion. Several filters designed to avoid unproductive socket-placement
> faults can also prevent promotion.
>
> Read-only file mappings and VMAs without recent PID activity may never be
> scanned, while lower MM layers reject some shared folios. Hot memory in
> these mappings can therefore remain on a slow tier indefinitely.
>
> Separate promotion-only scans from socket-placement scans.
>
> MM_CP_PROT_NUMA_PROMO_ONLY carries that choice for one protection walk, so
> the PTE and PMD paths can restrict hinting faults to promotion candidates.
> Keeping this transient state in the protection flags avoids per-mm state and
> its associated lifetime, concurrency, and VMA identity problems.
>
>   - Allow eligible shared folios to be promoted to a fast tier.
>   - Scan read-only file mappings and PID-inactive VMAs for promotion
>     without re-enabling placement sampling.
>   - Track the last placement scan separately so promotion-only scans
>     cannot postpone the existing placement-starvation fallback.
>
> The series is ordered as follows:
>
>   1. Add promotion-only NUMA protection walks without changing behavior.
>   2. Permit eligible shared folios to be promoted to a fast tier.
>   3. Scan read-only file mappings using promotion-only scans.
>   4. Scan PID-inactive VMAs for promotion and account for placement scans
>      separately.
>
> Tested on a host with 768GB/256GB DRAM/CXL.
> Ran 2 ~430GB database workloads with large (>300GB) shmem VMAs.

Do you just launch them without any additional NUMA control or NUMA
policy configuration, like using numactl? Just want to understand the
scope of the issue and how well the fixes are tested. It might be good
to have a list of expected behaviors for people/AI to check against.

>
> Before change:
>   - 150-200GB/s DRAM bandwidth usage
>   - 40-45GB/s CXL bandwidth usage (maxed out)
>   - request latencies over 5ms (longer tails)
>
> After chage:
>   - 250GB/s+ sustained DRAM bandwidth usage
>   - ~10GB/s sustained CXL bandwidth usage
>   - request latencies 800us-2ms.
>
> Functional observation:
>   A 20 GB hash table VMA that previously remained entirely on CXL was
>   split evenly between DRAM and CXL after the changes - and tier
>   residency tracked hotness.  This was previously affected by the
>   stavation issue caused by the "unaccessed VMA" filter.
>
> Gregory Price (Meta) (4):
>   mm: support promotion-only NUMA hinting scans
>   mm: allow shared folios to be promoted to a fast tier
>   sched/numa: scan read-only file mappings in tiering mode
>   sched/numa: do not let VMA PID activity gate promotion
>
>  include/linux/mm.h       |  4 ++-
>  include/linux/mm_types.h |  7 ++++
>  kernel/sched/fair.c      | 72 +++++++++++++++++++++++++++++-----------
>  mm/huge_memory.c         |  3 +-
>  mm/internal.h            |  5 +--
>  mm/mempolicy.c           | 30 +++++++++++------
>  mm/migrate.c             |  6 ++--
>  mm/mprotect.c            |  4 ++-
>  8 files changed, 93 insertions(+), 38 deletions(-)




-- 
Best Regards,
Yan, Zi


  parent reply	other threads:[~2026-09-18 20:56 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-11  0:18 Gregory Price
2026-09-11  0:18 ` [PATCH v2 1/4] mm: support promotion-only NUMA hinting scans Gregory Price
2026-09-17 16:03   ` Peter Zijlstra
2026-09-17 16:14     ` Gregory Price
2026-09-18 12:26       ` David Hildenbrand (Arm)
2026-09-18 12:37   ` David Hildenbrand (Arm)
2026-09-18 13:46     ` Gregory Price
2026-09-18 13:56       ` David Hildenbrand (Arm)
2026-09-11  0:18 ` [PATCH v2 2/4] mm: allow shared folios to be promoted to a fast tier Gregory Price
2026-09-17 16:08   ` Peter Zijlstra
2026-09-17 16:18     ` Gregory Price
2026-09-17 16:23       ` Peter Zijlstra
2026-09-17 16:39         ` Gregory Price
2026-09-18  4:14         ` Bharata B Rao
2026-09-17 17:49   ` Zi Yan
2026-09-18 12:54     ` David Hildenbrand (Arm)
2026-09-18 12:53   ` David Hildenbrand (Arm)
2026-09-18 13:54     ` Gregory Price
2026-09-18 13:57       ` David Hildenbrand (Arm)
2026-09-11  0:18 ` [PATCH v2 3/4] sched/numa: scan read-only file mappings in tiering mode Gregory Price
2026-09-18 12:58   ` David Hildenbrand (Arm)
2026-09-18 13:57     ` Gregory Price
2026-09-18 13:59       ` David Hildenbrand (Arm)
2026-09-18 14:53         ` Lorenzo Stoakes (ARM)
2026-09-18 15:48           ` Gregory Price
2026-09-18 16:19             ` Lorenzo Stoakes (ARM)
2026-09-18 16:38               ` Gregory Price
2026-09-11  0:18 ` [PATCH v2 4/4] sched/numa: do not let VMA PID activity gate promotion Gregory Price
2026-09-17 16:19   ` Peter Zijlstra
2026-09-18 13:01   ` David Hildenbrand (Arm)
2026-09-18 13:59     ` Gregory Price
2026-09-11  5:38 ` [PATCH v2 0/4] sched/numa: stop VMA scan filters from gating promotion Gregory Price
2026-09-17  5:35 ` Andrew Morton
2026-09-17  6:59   ` Gregory Price
2026-09-17 15:53     ` David Hildenbrand (Arm)
2026-09-18 20:56 ` Zi Yan [this message]
2026-09-18 21:42   ` Gregory Price

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DLIQQNUL2RTD.IGN8MYHYMW1E@nvidia.com \
    --to=ziy@nvidia.com \
    --cc=akpm@linux-foundation.org \
    --cc=apopple@nvidia.com \
    --cc=baohua@kernel.org \
    --cc=baolin.wang@linux.alibaba.com \
    --cc=bsegall@google.com \
    --cc=byungchul@sk.com \
    --cc=david@kernel.org \
    --cc=dev.jain@arm.com \
    --cc=dietmar.eggemann@arm.com \
    --cc=gourry@gourry.net \
    --cc=hannes@cmpxchg.org \
    --cc=jannh@google.com \
    --cc=joshua.hahnjy@gmail.com \
    --cc=juri.lelli@redhat.com \
    --cc=kas@kernel.org \
    --cc=kernel-team@meta.com \
    --cc=kprateek.nayak@amd.com \
    --cc=lance.yang@linux.dev \
    --cc=liam@infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=ljs@kernel.org \
    --cc=matthew.brost@intel.com \
    --cc=mgorman@suse.de \
    --cc=mhocko@suse.com \
    --cc=mingo@redhat.com \
    --cc=nico.pache@linux.dev \
    --cc=osalvador@suse.de \
    --cc=peterz@infradead.org \
    --cc=pfalcato@suse.de \
    --cc=raghavendra.kt@amd.com \
    --cc=rakie.kim@sk.com \
    --cc=rostedt@goodmis.org \
    --cc=rppt@kernel.org \
    --cc=ryan.roberts@arm.com \
    --cc=surenb@google.com \
    --cc=usama.arif@linux.dev \
    --cc=vbabka@kernel.org \
    --cc=vincent.guittot@linaro.org \
    --cc=vschneid@redhat.com \
    --cc=ying.huang@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®