From: "Mika Penttilä" <mpenttil@redhat.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Aveline Noir <jm5905938@gmail.com>,
airlied@gmail.com, apopple@nvidia.com, balbirs@nvidia.com,
christian.koenig@amd.com, dri-devel@lists.freedesktop.org,
Francois Dugast <francois.dugast@intel.com>,
intel-xe@lists.freedesktop.org, jgg@nvidia.com, jgg@ziepe.ca,
leon@kernel.org, leonro@nvidia.com, linux-kernel@vger.kernel.org,
linux-mm@kvack.org, Lorenzo Stoakes <lorenzo.stoakes@oracle.com>,
Matthew Brost <matthew.brost@intel.com>,
mm-commits@vger.kernel.org,
Patchwork <patchwork@emeril.freedesktop.org>,
Shuah Khan <shuah@kernel.org>, SeongJae Park <sj@kernel.org>,
thomas.hellstrom@linux.intel.com,
Vlastimil Babka <vbabka@suse.cz>,
ziy@nvidia.com
Subject: Re: [PATCH] mm/hmm: account for subpage offsets in device-private PMDs
Date: Sun, 22 Nov 2026 19:37:09 +0200 [thread overview]
Message-ID: <2f60e148-f9ea-4b48-b8b9-3c524b6a8a2d@redhat.com> (raw)
In-Reply-To: <20261122091419.f82e88761dbd12771fb6c976@linux-foundation.org>
On 11/22/26 19:14, Andrew Morton wrote:
> On Sun, 22 Nov 2026 18:41:39 +0200 Mika Penttilä <mpenttil@redhat.com> wrote:
>
>> On 11/22/26 16:45, Aveline Noir wrote:
>>
>>> Hi Andrew,
>>>
>>> On Sun, 22 Nov 2026 01:52:46 -0800 Andrew Morton wrote:
>>>
>>>> Store the corresponding RAM backing page instead.
>>> This avoids the oops on the first read through the second descriptor,
>>> but leaves a stale backing page pointer after CPU faultback.
>>>
>>> Since the mirror XArray entries don't take a page reference, they rely
>>> on invalidation. When the CPU later faults the page back to RAM,
>>> dmirror_interval_invalidate() skips MMU_NOTIFY_MIGRATE for the same
>>> mdevice, and dmirror_devmem_fault_alloc_and_copy() only clears the
>>> migrating descriptor's entry before the backing folio is freed. The
>>> second descriptor still points to that freed backing page.
>>>
>>> Reading the same page again through the second descriptor in QEMU
>>> triggers a KASAN use-after-free in dmirror_do_read(), and the read
>>> returns the stale data instead of what the CPU wrote.
>> I think this is right, but a fairly artificial scenario: it needs two
>> descriptors open on the same test device in one test, a migration through one,
>> a read through the other, a CPU fault back, and then another read through
>> the second descriptor..
>>
>> The fix lets the device private page content be read without an oops
>> in the context of one fd which is an improvement.
> May as well fix it. I'll test this updated version.
>
> Mika, please feel free to take this over, perhaps integrate it (or
> something like it) into whatever you've got going on? Or not, let me
> know ;)
I can handle it in the context where it made difference for me.
>
> From: Andrew Morton <akpm@linux-foundation.org>
> Subject: lib/test_hmm: invalidate all mirrors on device-private migration
> Date: Sun, 22 Nov 2026 08:53:12 -0800
>
> When one open of the fake HMM device migrates a folio to device-private
> memory, a second open can fault the same address into its mirror page
> table. The second mirror then stores the inaccessible device page, which
> can oops the kernel when dmirror_do_read() copies from it. Store the
> corresponding RAM backing subpage instead.
>
> That conversion also makes the backing page's lifetime significant to
> the second mirror. migrate_vma_setup() notifies all mirrors before CPU
> faultback, but dmirror_interval_invalidate() skips the notification for
> every open on the same device. The faulting open clears only its own
> XArray entry. The other open can read or write the backing page after
> migrate_vma_finalize() frees it. Aveline reported a KASAN use-after-free
> and stale data through the second open.
>
> Handle migration invalidation for every mirror under its mutex. This
> clears pointers before faultback can free the backing pages and waits
> for an active mirror read or write. During system-to-device migration,
> the migrating open installs new entries after migration; other opens
> refault on their next access. Remove the now redundant XArray erase
> from the faultback copy loop.
>
> Extend the PMD migration selftest to read its last device-private
> subpage through a second open, then fault it back to RAM, change it,
> and read the new value through that open. The trigger is available to
> a process permitted to open /dev/hmm_dmirror* when CONFIG_TEST_HMM is
> enabled; it does not depend on physical GPU hardware.
>
> Link: https://lore.kernel.org/r/20261122015246.d8b6b98ff5a8157168156339@linux-foundation.org
> Link: https://lore.kernel.org/r/179535872126.34.4789080961796247940.b4-reply@b4
> Fixes: b2ef9f5a5cb3 ("mm/hmm/test: add selftest driver for HMM")
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> Reported-by: Aveline Noir <jm5905938@gmail.com>
> Cc: Ralph Campbell <rcampbell@nvidia.com>
> Cc: Jason Gunthorpe <jgg@ziepe.ca>
> Cc: Leon Romanovsky <leon@kernel.org>
> Cc: Mika Penttilä <mpenttil@redhat.com>
> Cc: Shuah Khan <shuah@kernel.org>
> Cc: David Hildenbrand <david@kernel.org>
> Cc: Liam R. Howlett <liam@infradead.org>
> Cc: Vlastimil Babka <vbabka@kernel.org>
> Cc: Mike Rapoport <rppt@kernel.org>
> Cc: Michal Hocko <mhocko@suse.com>
> Cc: <stable@vger.kernel.org>
>
> lib/test_hmm.c | 20 +++++++-----------
> tools/testing/selftests/mm/hmm-tests.c | 25 +++++++++++++++++++++++
> 2 files changed, 33 insertions(+), 12 deletions(-)
>
> --- a/lib/test_hmm.c~lib-test_hmm-invalidate-all-mirrors-on-device-private-migration
> +++ a/lib/test_hmm.c
> @@ -292,6 +292,12 @@ static int dmirror_do_fault(struct dmirr
> page = hmm_pfn_to_page(*pfns);
> WARN_ON(!page);
>
> + if (is_device_private_page(page)) {
> + struct folio *folio = page_folio(page);
> +
> + page = BACKING_PAGE(&folio->page) +
> + folio_page_idx(folio, page);
> + }
> entry = page;
> if (*pfns & HMM_PFN_WRITE)
> entry = xa_tag_pointer(entry, DPT_XA_TAG_WRITE);
> @@ -327,14 +333,6 @@ static bool dmirror_interval_invalidate(
> {
> struct dmirror *dmirror = container_of(mni, struct dmirror, notifier);
>
> - /*
> - * Ignore invalidation callbacks for device private pages since
> - * the invalidation is handled as part of the migration process.
> - */
> - if (range->event == MMU_NOTIFY_MIGRATE &&
> - range->owner == dmirror->mdevice)
> - return true;
> -
> if (mmu_notifier_range_blockable(range))
> mutex_lock(&dmirror->mutex);
> else if (!mutex_trylock(&dmirror->mutex))
> @@ -1198,7 +1196,6 @@ static vm_fault_t dmirror_devmem_fault_a
>
> src_page = pfn_to_page(page_to_pfn(spage) + i);
>
> - xa_erase(&dmirror->pt, addr >> PAGE_SHIFT);
> addr += PAGE_SIZE;
> copy_highpage(dst_page, src_page);
> }
> @@ -1822,9 +1819,8 @@ static vm_fault_t dmirror_devmem_fault(s
> }
> migrate_vma_pages(&args);
> /*
> - * No device finalize step is needed since
> - * dmirror_devmem_fault_alloc_and_copy() will have already
> - * invalidated the device page table.
> + * The migration notifier has invalidated all mirror page tables
> + * before the backing pages can be freed.
> */
> migrate_vma_finalize(&args);
> err:
> --- a/tools/testing/selftests/mm/hmm-tests.c~lib-test_hmm-invalidate-all-mirrors-on-device-private-migration
> +++ a/tools/testing/selftests/mm/hmm-tests.c
> @@ -2087,6 +2087,7 @@ TEST_F(hmm, migrate_anon_huge_fault)
> void *old_ptr;
> void *map;
> int pagemap_fd;
> + int fd2 = -1;
> int *ptr;
> int ret;
>
> @@ -2143,12 +2144,36 @@ TEST_F(hmm, migrate_anon_huge_fault)
> }
>
> close(pagemap_fd);
> +
> + /* Read a device-private subpage through another open. */
> + fd2 = hmm_open(variant->device_number);
> + ASSERT_GE(fd2, 0);
> + buffer->ptr = (char *)map + (npages - 1) * self->page_size;
> + ret = hmm_dmirror_cmd(fd2, HMM_DMIRROR_READ, buffer, 1);
> + buffer->ptr = map;
> + ASSERT_EQ(ret, 0);
> + ASSERT_EQ(buffer->cpages, 1);
> + ASSERT_EQ(*(int *)buffer->mirror,
> + (npages - 1) * self->page_size / sizeof(int));
> }
>
> /* Fault pages back to system memory and check them. */
> for (i = 0, ptr = buffer->ptr; i < size / sizeof(*ptr); ++i)
> ASSERT_EQ(ptr[i], i);
>
> + if (fd2 >= 0) {
> + /* This open must not retain the freed device backing page. */
> + ptr[(npages - 1) * self->page_size / sizeof(*ptr)] = -1;
> + buffer->ptr = (char *)map + (npages - 1) * self->page_size;
> + ret = hmm_dmirror_cmd(fd2, HMM_DMIRROR_READ, buffer, 1);
> + buffer->ptr = map;
> + ASSERT_EQ(ret, 0);
> + ASSERT_EQ(buffer->cpages, 1);
> + ASSERT_EQ(buffer->faults, 1);
> + ASSERT_EQ(*(int *)buffer->mirror, -1);
> + ASSERT_EQ(close(fd2), 0);
> + }
> +
> buffer->ptr = old_ptr;
> hmm_buffer_free(buffer);
> }
> _
--Mika
prev parent reply other threads:[~2026-11-22 17:37 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-31 4:07 Aveline Noir
2026-11-22 0:00 ` Andrew Morton
2026-11-22 5:55 ` Mika Penttilä
2026-11-22 9:52 ` Andrew Morton
2026-11-22 12:04 ` Mika Penttilä
2026-11-22 14:45 ` Aveline Noir
2026-11-22 16:32 ` Mika Penttilä
2026-11-22 16:41 ` Mika Penttilä
2026-11-22 17:14 ` Andrew Morton
2026-11-22 17:37 ` Mika Penttilä [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2f60e148-f9ea-4b48-b8b9-3c524b6a8a2d@redhat.com \
--to=mpenttil@redhat.com \
--cc=airlied@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=apopple@nvidia.com \
--cc=balbirs@nvidia.com \
--cc=christian.koenig@amd.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=francois.dugast@intel.com \
--cc=intel-xe@lists.freedesktop.org \
--cc=jgg@nvidia.com \
--cc=jgg@ziepe.ca \
--cc=jm5905938@gmail.com \
--cc=leon@kernel.org \
--cc=leonro@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lorenzo.stoakes@oracle.com \
--cc=matthew.brost@intel.com \
--cc=mm-commits@vger.kernel.org \
--cc=patchwork@emeril.freedesktop.org \
--cc=shuah@kernel.org \
--cc=sj@kernel.org \
--cc=thomas.hellstrom@linux.intel.com \
--cc=vbabka@suse.cz \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®