Re: [PATCH v3 28/33] drm/xe: Use IOVA-based DMA mapping for eligible tt BOs

[Matthew Auld <matthew.auld@intel.com>]

On 03/11/2026 20:21, Matthew Brost wrote:
> On Tue, Nov 03, 2026 at 07:00:24PM +0000, Matthew Auld wrote:
>> On 11/07/2026 03:56, Matthew Brost wrote:
>>> Add support for mapping a ttm_tt's pages using the IOVA-based DMA API
>>> (dma_iova_try_alloc()/link()/sync()/unlink()/free()) as an alternative
>>> to the scatter-gather list. An IOVA mapping is a single contiguous DMA
>>> range, which is cheaper to walk and program than an sg list.
>>>
>>> Whether the IOVA API can be used depends on the IOMMU mode, so the
>>> decision is made once per tt at xe_ttm_tt_create() time and recorded in
>>> struct xe_ttm_tt::use_iova. To keep the scope limited, only device BOs
>>> that are not GGTT-mapped are eligible: dma-buf imports manage their own
>>> mapping and GGTT bindings are programmed directly from the sg list.
>>> dma_iova_try_alloc() may still fail, in which case the tt transparently
>>> falls back to the sg list, which remains the default.
>>>
>>> The IOVA reservation is allocated at tt creation and freed at tt
>>> destroy. xe_tt_map_iova() links the tt pages (coalescing physically
>>> contiguous runs) and syncs the IOTLB; xe_tt_unmap_iova() unlinks them
>>> but keeps the reservation so the tt can be remapped without
>>> re-allocating IOVA space. xe_tt_map()/xe_tt_unmap() dispatch between the
>>> IOVA and sg backends, and xe_bo_dma_unmap_pinned() learns about the IOVA
>>> case.
>>>
>>> Consumers that walked the sg list via xe_res_first_sg(xe_bo_sg()) now go
>>> through the new xe_res_first_tt() helper, which initializes either an
>>> IOVA cursor (xe_res_first_iova()) or an sg cursor depending on the tt's
>>> mapping mode.
>>>
>>> Cc: Carlos Santa <carlos.santa@intel.com>
>>> Cc: Ryan Neph <ryanneph@google.com>
>>> Cc: Christian Koenig <christian.koenig@amd.com>
>>> Cc: Huang Rui <ray.huang@amd.com>
>>> Cc: Matthew Auld <matthew.auld@intel.com>
>>> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
>>> Cc: Maxime Ripard <mripard@kernel.org>
>>> Cc: Thomas Zimmermann <tzimmermann@suse.de>
>>> Cc: David Airlie <airlied@gmail.com>
>>> Cc: Simona Vetter <simona@ffwll.ch>
>>> Cc: dri-devel@lists.freedesktop.org
>>> Cc: linux-kernel@vger.kernel.org
>>> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
>>> Assisted-by: GitHub_Copilot:claude-opus-4.8
>>> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
>>
>>
>> <snip>
>>
>>>    static void xe_ttm_tt_destroy(struct ttm_device *ttm_dev, struct ttm_tt *tt)
>>>    {
>>> +	struct xe_ttm_tt *xe_tt = container_of(tt, struct xe_ttm_tt, ttm);
>>> +
>>> +	/*
>>> +	 * Release the IOVA reservation made in xe_ttm_tt_create(). The pages
>>> +	 * must already have been unlinked by xe_tt_unmap().
>>> +	 */
>>> +	if (xe_tt->use_iova) {
>>> +		struct xe_device *xe = ttm_to_xe_device(ttm_dev);
>>> +
>>> +		xe_assert(xe, !xe_tt->iova_linked);
>>> +		dma_iova_free(xe->drm.dev, &xe_tt->iova);
>>
>> Do we know if this is allowed? I think there is xe_bo_pci_dev_remove_all()
>> which requires that evict_all will correctly tear down iommu mappings on
>> stuff like pci unplug and I don't think tt_destroy is called in that flow?
> 
> But unpopulate is called which unlinks / syncs (TLB flush for the IOVA).
> 
>> Does having the iova here cause issues? Also, for VRAM flow I think there is
>> a tt_create() via pipeline gutting which will now create a new iova?
>>
> 
> Maybe we shuffle this around a bit so the iova alloc is tied to
> populate/unpopulate. It was been a while I forget exactly why I did this
> way.
> 
>> Aside from that, maybe we can land 27/28 as standalone?
> 
> Yes, let me extract these two into standalone patches today and
> reworking if needed.

Quickly double checked this, indeed looks to be needed:

[  808.058892] [IGT] core_hotunplug: finished subtest 
hotreplug-lateclose-with-load, SUCCESS
[  808.058928] [IGT] core_hotunplug: exiting, ret=0
[  808.059673] BUG: kernel NULL pointer dereference, address:
[  808.061068] RIP: 0010:iommu_get_dma_domain+0x10/0x20
[  808.061141] PKRU: 55555554
[  808.061144] Call Trace:
[  808.061146]  <TASK>
[  808.061149]  dma_iova_free+0x18/0xf0
[  808.061153]  xe_ttm_tt_destroy+0x23/0x40 [xe]
[  808.061231]  ttm_bo_put+0x242/0x3d0 [ttm]
[  808.061412]  ? drm_gpuvm_bo_put+0x199/0x200 [drm_gpuvm]
[  808.061418]  xe_vma_destroy_late+0x89/0x90 [xe]
[  808.061476]  xe_vm_close_and_put+0x5de/0x920 [xe]
[  808.061534]  xe_file_close+0x105/0x190 [xe]
[  808.061712]  drm_file_free+0x218/0x260
[  808.061717]  drm_release_noglobal+0x6d/0xd0
[  808.061721]  __fput+0xf9/0x270
[  808.061725]  task_work_run+0x8b/0xb0
[  808.061730]  do_exit+0x229/0x8b0
[  808.061735]  do_group_exit+0x8b/0x90
[  808.061739]  __x64_sys_exit_group+0x17/0x20
[  808.061744]  x64_sys_call+0x18c6/0x18d0
[  808.061749]  do_syscall_64+0xee/0x350
[  808.061753]  ? __pte_offset_map+0x1d/0xd0
[  808.061757]  ? next_uptodate_folio+0xbb/0x3a0
[  808.061762]  ? set_pte_range+0x20d/0x290
[  808.061766]  ? filemap_map_pages+0x8e1/0x9c0
[  808.061772]  ? count_memcg_events+0x7c/0x1b0
[  808.061776]  ? handle_mm_fault+0xbde/0x14f0
[  808.061781]  ? __se_sys_close+0x7a/0xe0
[  808.061785]  ? __se_sys_close+0x7a/0xe0
[  808.061790]  ? do_user_addr_fault+0x272/0x6c0
[  808.061794]  ? irqentry_exit+0x3f/0x640
[  808.061799]  ? do_syscall_64+0x42/0x350
[  808.061803]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[  808.061808] RIP: 0033:0x7f95428c71b8

I assume this is dev->iommu_group = NULL post unplug.

> 
> Matt