Re: [PATCH net-next 08/10] net/mlx5e: Recreate netdev channels on data direct device unbind

[Mina Almasry <almasrymina@google.com>]

On Thu, Oct 8, 2026 at 6:28 AM Tariq Toukan <tariqt@nvidia.com> wrote:
> From: Dragos Tatulea <dtatulea@nvidia.com>
>
> Note that this does not tear down dmabuf bindings attached to the data
> direct device; the recreated channels will still pick up the binding via
> rxq->mp_params and post DMA addresses from the data direct IOMMU domain
> to the PF, causing IOMMU faults. A devmem revoke hook is needed to close
> those bindings before the switch.

(Note: LLM-assisted review comment below.)

Rather than shipping this known IOMMU fault window on MLX5_DATA_DIRECT_UNBIND,
could we add the core detach helper in patch 09/10 and invoke it here under
netdev_lock(priv->netdev) before mlx5e_safe_switch_params()?

Specifically, reopening channels on the PF without detaching active bindings
leaves three issues:
1. RX: reopened queues pick up stale rxq->mp_params and post data_direct IOVAs
   onto ch->pdev.
2. TX: active TX bindings (net_devmem_dmabuf_bindings) and in-flight or
   retransmitted skbs still hold data_direct IOVAs while mlx5e_xmit() switches
   to sq->mkey_be (ch->pdev).
3. DMA unmap ordering: binding->sgt / binding->attachment stay mapped to
   &dd_dev->pdev->dev until userspace closes all sockets, after
   mlx5_data_direct_remove() has already returned.

If mlx5e_data_direct_event() calls a core helper such as
netdev_unbind_dmabuf_dma_dev(priv->netdev, &dd_dev->pdev->dev) before
mlx5e_safe_switch_params():
- netif_mp_close_rxq() clears rxq->mp_params so reopened channels use host
  page_pool pages on ch->pdev.
- Clearing WRITE_ONCE(binding->dev, NULL) + synchronize_net() causes existing
  validate_xmit_unreadable_skb() (READ_ONCE(binding->dev) != dev) to drop any
  in-flight/retransmitted TX skbs automatically with zero driver fast-path
  changes.
- dma_buf_unmap_attachment_unlocked() + dma_buf_detach() complete synchronously
  before mlx5_data_direct_remove() finishes.

-- 
Thanks,
Mina