[PATCH v1 0/5] iommu: Keep a device blocked after a failed PCI reset

[Nicolin Chen <nicolinc@nvidia.com>]

This series is split from a bigger SMMUv3 series, so that the PCI and the
IOMMU core changes can be reviewed and merged on their own:
https://lore.kernel.org/all/cover.1790188510.git.nicolinc@nvidia.com/
The SMMUv3 ATC invalidation timeout handling in that series will depend on
this series.

pci_dev_reset_iommu_prepare() blocks the IOMMU translation and ATS of a PCI
device before a reset. pci_dev_reset_iommu_done() restores them after the
reset. However, done() is called unconditionally, even when the reset has
failed. A failed reset gives no guarantee that the device cache is clean.
For an ATS-capable device, its ATC might still have stale entries pointing
to the memory pages that the OS has reclaimed. Re-attaching such a device
to its translation domains could corrupt the system memory.

Thus, pass in the reset result to pci_dev_reset_iommu_done(). On a failure,
keep the device blocked until a subsequent reset succeeds. This new policy
applies to every ATS-capable PCI device behind an IOMMU, so no IOMMU driver
needs to opt in.

First, fix two PCI reset issues found along the way. A probe of the reset
capability must not block the IOMMU, as it is only a state query. And the
cxl_reset_bus_function() must not report a "cxl_bus" method that can never
succeed.

Then, convert gdev->blocked from a bool into an enum in the IOMMU core, so
that a failed reset can be told apart from an ongoing reset. Also, pass the
blocked state of a device being removed into iommu_deinit_device(), which
lets the release_domain get the correct old domain.

Note that the PCI reset functions do overload -ENOTTY. It might mean either
"the reset was not attempted" or "this method failed, so try the next one".
The IOMMU core cannot tell these two apart. So, it keeps the device blocked
on -ENOTTY as well, with a FIXME and a warning.

This is on Github:
https://github.com/nicolinc/iommufd/commits/iommu_reset_failure-v1

Changelog
v1:
 * Drop the Assisted-by tags
 * Split from the SMMUv3 series above
 * Carry over the Reviewed-by tags from Baolu and Jason

Nicolin Chen (5):
  PCI: Don't suspend IOMMU when probing reset capability
  PCI/CXL: Probe the underlying bus reset in cxl_reset_bus_function()
  iommu: Convert gdev->blocked from bool to enum blocked_reason
  iommu: Pass in gdev's blocked state to iommu_deinit_device()
  iommu: Pass in reset result to pci_dev_reset_iommu_done()

 include/linux/iommu.h  |  5 ++-
 drivers/iommu/iommu.c  | 87 ++++++++++++++++++++++++++++++++++++------
 drivers/pci/pci-acpi.c |  2 +-
 drivers/pci/pci.c      | 37 +++++++++++-------
 drivers/pci/quirks.c   | 13 ++++---
 5 files changed, 111 insertions(+), 33 deletions(-)


base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.43.0