[PATCH v6 0/9] iommu/amd: Use 128-bit cmpxchg operation to update DTE

[Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>]

This series modifies current implementation to use 128-bit cmpxchg to
update DTE when needed as specified in the AMD I/O Virtualization
Techonology (IOMMU) Specification.

Please note that I have verified with the hardware designer, and they have
confirmed that the IOMMU hardware has always been implemented with 256-bit
read. The next revision of the IOMMU spec will be updated to correctly
describe this part.  Therefore, I have updated the implementation to avoid
unnecessary flushing.

Changes in v6:

* Patch 2, 4, 7: Newly add

* Patch 3, 5, 6, 7, 9: Add READ_ONCE() per Uros.

* Patch 3:
  - Modify write_dte_[higher|lower]128() to avoid copying old DTE in the loop.

* Patch 5:
  - Use dev_data->dte_cache to restore persistent DTE bits in set_dte_entry().
  - Simplify make_clear_dte():
    - Remove bit preservation logic.
    - Remove non-SNP check for setting TV since it should not be needed.

* Patch 6:
  - Use find_dev_data(..., alias) since the dev_data might not have been allocated.
  - Move dev_iommu_priv_set() to before setup_aliases().

v5: https://lore.kernel.org/lkml/20241007041353.4756-1-suravee.suthikulpanit@amd.com/
v4: https://lore.kernel.org/lkml/20240916171805.324292-1-suravee.suthikulpanit@amd.com/
v3: https://lore.kernel.org/lkml/20240906121308.5013-1-suravee.suthikulpanit@amd.com/
v2: https://lore.kernel.org/lkml/20240829180726.5022-1-suravee.suthikulpanit@amd.com/
v1: https://lore.kernel.org/lkml/20240819161839.4657-1-suravee.suthikulpanit@amd.com/

Thanks,
Suravee

Suravee Suthikulpanit (8):
  iommu/amd: Disable AMD IOMMU if CMPXCHG16B feature is not supported
  iommu/amd: Introduce helper function to update 256-bit DTE
  iommu/amd: Introduce per-device DTE cache to store persistent bits
  iommu/amd: Modify set_dte_entry() to use 256-bit DTE helpers
  iommu/amd: Introduce helper function get_dte256()
  iommu/amd: Move erratum 63 logic to write_dte_lower128()
  iommu/amd: Modify clear_dte_entry() to avoid in-place update
  iommu/amd: Lock DTE before updating the entry with WRITE_ONCE()

Uros Bizjak (1):
  asm/rwonce: Introduce [READ|WRITE]_ONCE() support for __int128

 drivers/iommu/amd/amd_iommu.h       |   4 +-
 drivers/iommu/amd/amd_iommu_types.h |  25 +-
 drivers/iommu/amd/init.c            |  79 ++----
 drivers/iommu/amd/iommu.c           | 364 ++++++++++++++++++++--------
 include/asm-generic/rwonce.h        |   2 +-
 include/linux/compiler_types.h      |   8 +-
 6 files changed, 322 insertions(+), 160 deletions(-)

-- 
2.34.1