Re: [RFC PATCH v0 0/2] Batch migration for NUMA balancing

[Bharata B Rao <bharata@amd.com>]

Hi SJ,

On 22-May-25 12:15 AM, SeongJae Park wrote:
> Hi Bharata,
> 
> On Wed, 21 May 2025 13:32:36 +0530 Bharata B Rao <bharata@amd.com> wrote:
> 
>> Hi,
>>
>> This is an attempt to convert the NUMA balancing to do batched
>> migration instead of migrating one folio at a time. The basic
>> idea is to collect (from hint fault handler) the folios to be
>> migrated in a list and batch-migrate them from task_work context.
>> More details about the specifics are present in patch 2/2.
>>
>> During LSFMM[1] and subsequent discussions in MM alignment calls[2],
>> it was suggested that separate migration threads to handle migration
>> or promotion request may be desirable. Existing NUMA balancing, hot
>> page promotion and other future promotion techniques could off-load
>> migration part to these threads. Or if we manage to have a single
>> source of hotness truth like kpromoted[3], then that too can hand
>> over migration requests to the migration threads. I am envisaging
>> that different hotness sources like kmmscand[4], MGLRU[5], IBS[6]
>> and CXL HMU would push hot page info to kpromoted, which would
>> then isolate and push the folios to be promoted to the migrator
>> thread.
> 
> I think (or, hope) it would also be not very worthless or rude to mention other
> existing and ongoing works that have potentials to serve for similar purpose or
> collaborate in future, here.
> 
> DAMON is designed for a sort of multi-source access information handling.  In
> LSFMM, I proposed[1] damon_report_access() interface for making it easier to be
> extended for more types of access information.  Currenlty damon_report_access()
> is under early development.  I think this has a potential to serve something
> similar to your single source goal.
> 
> Also in LSFMM, I proposed damos_add_folio() for a case that callers want to
> utilize DAMON worker thread (kdamond) as an asynchronous memory
> management operations execution thread while using its other features such as
> [auto-tuned] quotas.  I think this has a potential to serve something similar
> to your migration threads.  I haven't started damos_add_folio() development
> yet, though.
> 
> I remember we discussed about DAMON on mailing list and in LSFMM a bit, on your
> session.  IIRC, you were also looking for a time to see if there is a chance to
> use DAMON in some way.  Due to the technical issue, we were unable to discuss
> on the two new proposals on my LSFMM session, and it has been a bit while since
> our last discussion.  So if you don't mind, I'd like to ask if you have some
> opinions or comments about these.
> 
> [1] https://lwn.net/Articles/1016525/

Since this patchset was just about making the migration batched and 
async for NUMAB, I didn't mention DAMON as an alternative here.

One of the concerns I always had about DAMON when it is considered as 
replacement for existing hot page migration is its current inability to 
gather and maintain hot page info at per-folio granularity. How much 
that eventually matters to the workloads has to be really seen.

Regards,
Bharata.

Regards,
Bharata.