Re: [RFC PATCH v0 2/2] mm: sched: Batch-migrate misplaced pages

[Zi Yan <ziy@nvidia.com>]

On 26 May 2025, at 21:18, Huang, Ying wrote:

> Zi Yan <ziy@nvidia.com> writes:
>
>> On 26 May 2025, at 5:29, David Hildenbrand wrote:
>>
>>> On 22.05.25 19:30, Zi Yan wrote:
>>>> On 22 May 2025, at 13:21, David Hildenbrand wrote:
>>>>
>>>>> On 22.05.25 18:38, Zi Yan wrote:
>>>>>> On 22 May 2025, at 12:26, David Hildenbrand wrote:
>>>>>>
>>>>>>> On 22.05.25 18:24, Zi Yan wrote:
>>>>>>>> On 22 May 2025, at 12:11, David Hildenbrand wrote:
>>>>>>>>
>>>>>>>>> On 21.05.25 10:02, Bharata B Rao wrote:
>>>>>>>>>> Currently the folios identified as misplaced by the NUMA
>>>>>>>>>> balancing sub-system are migrated one by one from the NUMA
>>>>>>>>>> hint fault handler as and when they are identified as
>>>>>>>>>> misplaced.
>>>>>>>>>>
>>>>>>>>>> Instead of such singe folio migrations, batch them and
>>>>>>>>>> migrate them at once.
>>>>>>>>>>
>>>>>>>>>> Identified misplaced folios are isolated and stored in
>>>>>>>>>> a per-task list. A new task_work is queued from task tick
>>>>>>>>>> handler to migrate them in batches. Migration is done
>>>>>>>>>> periodically or if pending number of isolated foios exceeds
>>>>>>>>>> a threshold.
>>>>>>>>>
>>>>>>>>> That means that these pages are effectively unmovable for
>>>>>>>>> other purposes (CMA, compaction, long-term pinning, whatever)
>>>>>>>>> until that list was drained.
>>>>>>>>>
>>>>>>>>> Bad.
>>>>>>>>
>>>>>>>> Probably we can mark these pages and when others want to migrate the page,
>>>>>>>> get_new_page() just looks at the page's target node and get a new page from
>>>>>>>> the target node.
>>>>>>>
>>>>>>> How do you envision that working when CMA needs to migrate this exact page to a different location?
>>>>>>>
>>>>>>> It cannot isolate it for migration because ... it's already isolated ... so it will give up.
>>>>>>>
>>>>>>> Marking might not be easy I assume ...
>>>>>>
>>>>>> I guess you mean we do not have any extra bit to indicate this page is isolated,
>>>>>> but it can be migrated. My point is that if this page is going to be migrated
>>>>>> due to other reasons, like CMA, compaction, why not migrate it to the target
>>>>>> node instead of moving it around within the same node.
>>>>>
>>>>> I think we'd have to identify that
>>>>>
>>>>> a) This page is isolate for migration (could be isolated for other
>>>>>     reasons)
>>>>>
>>>>> b) The one responsible for the isolation is numa code (could be someone
>>>>>     else)
>>>>>
>>>>> c) We're allowed to grab that page from that list (IOW sync against
>>>>>     others, and especially also against), to essentially "steal" the
>>>>>     isolated page.
>>>>
>>>> Right. c) sounds like adding more contention to the candidate list.
>>>> I wonder if we can just mark the page as migration candidate (using
>>>> a page flag or something else), then migrate it whenever CMA,
>>>> compaction, long-term pinning and more look at the page.
>>>
>>> I mean, all these will migrate the page either way, no need to add another flag for that.
>>>
>>> I guess what you mean, indicating that the migration destination
>>> should be on a different node than the current one.
>>
>> Yes.
>>
>>>
>>> Well, and for the NUMA scanner (below) to find which pages to migrate.
>>>
>>> ... to be this raises some questions: like, if we don't migrate
>>> immediately, could that information ("migrate this page") actually
>>> now be wrong? I guess a way to
>>
>> Could be. So it is better to evaluate the page before the actual migration, in
>> case the page is no longer needed in a remote node.
>>
>>> obtain the destination node would suffice: if the destination node
>>> matches, no need to migrate from that NUMA scanner.
>>
>> Right. The destination node could be calculated by certain metric like most recent
>> accesses or last remote node access time.
>
> Do we have the necessary information available?  last_cpupid have either
> last accessing CPU or last scanning timestamp, not both.  Any other
> information source?

Not at the moment. A unified page access information framework
is probably needed. The recent LSFMM has a related discussion[1]. We
also have a biweekly discussion on it[2].

[1] https://lwn.net/Articles/1016722/
[2] https://lore.kernel.org/linux-mm/ae6e7b19-f221-9a5d-a3eb-799ed271de11@google.com/

Best Regards,
Yan, Zi