Re: [PATCH v17 00/12] migrate on fault for device pages

["Mika Penttilä" <mpenttil@redhat.com>]

On 11/12/26 16:00, Mika Penttilä wrote:
> Hi Alexandru!
>
> On 11/12/26 13:56, Alexandru Elisei wrote:
>> Hi Mika,
>>
>> I want to help getting this series merged, and I've been familiarizing myself
>> with hmm and device migration, please bare with me if the questions I'm asking
>> are obvious or downright silly.
> Thanks for the interest and willing to help! I am preparing a v18 of the
> series and plan to post it early next week.
>
> I try to answer your questions, there are quite a bunch of them!
>
>> On Tue, Nov 03, 2026 at 08:16:17AM +0200, mpenttil@redhat.com wrote:
>>> From: Mika Penttilä <mpenttil@redhat.com>
>>>
>>> Currently, the way device page faulting and migration works
>>> is not optimal, if you want to do both fault handling and
>>> migration at once.
>>>
>>> Being able to migrate not present pages (or pages mapped with incorrect
>>> permissions, eg. COW) to the GPU requires doing either of the
>>> following sequences:
>>>
>>> 1. hmm_range_fault() - fault in non-present pages with correct permissions, etc.
>>> 2. migrate_vma_*() - migrate the pages
>>>
>>> Or:
>>>
>>> 1. migrate_vma_*() - migrate present pages
>>> 2. If non-present pages detected by migrate_vma_*():
>>>    a) call hmm_range_fault() to fault pages in
>>>    b) call migrate_vma_*() again to migrate now present pages
>> Regarding both sequences, I'm trying to get a better understanding of what
>> the issue is. I assume the caller wants to fault in system memory, and then
>> migrate it to the device. Is that correct?
> Yes that's the common thing to do.
>
>
>> In my analysis I'm also not going to refer to THPs, since that would make
>> things even more complicated than they are. I think we can come back to
>> THPs later if we think it would be useful.
>>
>> For the !pte_present() case:
>>
>> Sequence 1: hmm_range_fault() -> .. -> hmm_vma_handle_pte() handles
>> !pte_present() pages in two ways: faults in swap pages and device private
>> pages belonging to a different device, and waits on migration entries. Then
>> the migrate_vma_*() flow is used to migrate the range.
>>
>> Sequence 2: migrate_vma_setup() ignores !pte_present() page table entries
>> After this, the caller follows the same steps as in sequence 1 -
>> hmm_range_fault() faults in !pte_present() pages, and migrate_vma_*() is
>> called again to migrate the pages that were faulted in/waited on by
>> hmm_range_fault().
>>
>> Now, for the CoW case that you mentioned, I'm going to assume that the
>> caller wants write permissions otherwise nothing needs to be done:
>>
>> Sequence 1: hmm_range_fault() -> hmm_vma_handle_pte() records a fault
>> (hmm_pte_need_fault() returns HMM_NEED_FAULT | HMM_NEED_WRITE_FAULT),
>> handle_mm_fault() -> .. -> do_wp_page() breaks CoW and maps an exclusive
>> folio with write permissions, then the migrate_vma_* flow will take care of
>>
>> Sequence 2, migrate_vma_setup() installs migration entries for the page,
>> then migrate_vma_finalize() replaces the migration entries with device
>> private entries.
>>
>> Looks to me that for CoW pages, all that's needed is migrate_vma_*(), and
>> the subsequent calls to hmm_range_fault() and migrate_vma_*() are not
>> necessary. Is that your conclusion too?
> For  a device fault: the driver want the data in device memory with the
> permissions the device asked for. Your !pte_present() analysis matches;
> note that pte_none() in anonymous vmas and the zero page don't need
> hmm_range_fault() at all, migrate_vma_*() populates them directly.
> That's why the common case is cheap with sequence 2, and only swap
> entries, other devices private pages and migration entries need the
> extra walks.
> For cow with write intent migrate_vma_*() alone is enough.

Is *not* enough meant to say


--Mika