[PATCH v2] iommu/amd: Enhance "Completion-wait Time-out" error message

[PATCH v2] iommu/amd: Enhance "Completion-wait Time-out" error message

[Dheeraj Kumar Srivastava]

Current IOMMU driver prints "Completion-wait Time-out" error message with
insufficient information to further debug the issue.

Enhancing the error message as following:
1. Log IOMMU PCI device ID in the error message.
2. Also dump the command immediately preceding the COMPLETION_WAIT
   command to identify what the IOMMU is waiting for.

Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com>
---
Changes since v1:
-> Remove the code that dumps the entire command buffer on each
   completion wait timeout when amd_iommu_dump=1. [Suravee and Vasant]

 drivers/iommu/amd/amd_iommu_types.h |  2 ++
 drivers/iommu/amd/iommu.c           | 18 +++++++++++++++++-
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_iommu_types.h
index 95f63c5f6159..d495e5e4a9a2 100644
--- a/drivers/iommu/amd/amd_iommu_types.h
+++ b/drivers/iommu/amd/amd_iommu_types.h
@@ -247,6 +247,8 @@
 #define CMD_BUFFER_ENTRIES 512
 #define MMIO_CMD_SIZE_SHIFT 56
 #define MMIO_CMD_SIZE_512 (0x9ULL << MMIO_CMD_SIZE_SHIFT)
+#define MMIO_CMD_TAIL_MASK	GENMASK_ULL(18, 4)
+#define MMIO_CMD_BUFFER_TAIL(x)	FIELD_GET(MMIO_CMD_TAIL_MASK, (x))
 
 /* constants for event buffer handling */
 #define EVT_BUFFER_SIZE		8192 /* 512 entries */
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index eb348c63a8d0..89f7f6b01cad 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -1158,6 +1158,7 @@ irqreturn_t amd_iommu_int_handler(int irq, void *data)
 
 static int wait_on_sem(struct amd_iommu *iommu, u64 data)
 {
+	struct iommu_cmd *cmd;
 	int i = 0;
 
 	while (*iommu->cmd_sem != data && i < LOOP_TIMEOUT) {
@@ -1166,7 +1167,22 @@ static int wait_on_sem(struct amd_iommu *iommu, u64 data)
 	}
 
 	if (i == LOOP_TIMEOUT) {
-		pr_alert("Completion-Wait loop timed out\n");
+		int head, tail;
+
+		head = readl(iommu->mmio_base + MMIO_CMD_HEAD_OFFSET);
+		tail = readl(iommu->mmio_base + MMIO_CMD_TAIL_OFFSET);
+
+		pr_alert("IOMMU %04x:%02x:%02x.%01x: Completion-Wait loop timed out\n",
+			 iommu->pci_seg->id, PCI_BUS_NUM(iommu->devid),
+			 PCI_SLOT(iommu->devid), PCI_FUNC(iommu->devid));
+		/*
+		 * On command buffer completion timeout, step back by 2 commands
+		 * to locate the actual command that is causing the issue.
+		 */
+		tail = (MMIO_CMD_BUFFER_TAIL(tail) - 2) & (CMD_BUFFER_ENTRIES - 1);
+		cmd = (struct iommu_cmd *)(iommu->cmd_buf + tail * sizeof(*cmd));
+		dump_command(iommu_virt_to_phys(cmd));
+
 		return -EIO;
 	}
 
-- 
2.25.1

Re: [PATCH v2] iommu/amd: Enhance "Completion-wait Time-out" error message

[kernel test robot]

Hi Dheeraj,

kernel test robot noticed the following build warnings:

[auto build test WARNING on linus/master]
[also build test WARNING on v6.18-rc2 next-20251024]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Dheeraj-Kumar-Srivastava/iommu-amd-Enhance-Completion-wait-Time-out-error-message/20251024-141312
base:   linus/master
patch link:    https://lore.kernel.org/r/20251024061041.707-1-dheerajkumar.srivastava%40amd.com
patch subject: [PATCH v2] iommu/amd: Enhance "Completion-wait Time-out" error message
config: x86_64-buildonly-randconfig-003-20251025 (https://download.01.org/0day-ci/archive/20251025/202510250855.7wiC1wSz-lkp@intel.com/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251025/202510250855.7wiC1wSz-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202510250855.7wiC1wSz-lkp@intel.com/

All warnings (new ones prefixed by >>):

   drivers/iommu/amd/iommu.c: In function 'wait_on_sem':
>> drivers/iommu/amd/iommu.c:1171:21: warning: variable 'head' set but not used [-Wunused-but-set-variable]
    1171 |                 int head, tail;
         |                     ^~~~


vim +/head +1171 drivers/iommu/amd/iommu.c

  1153	
  1154	/****************************************************************************
  1155	 *
  1156	 * IOMMU command queuing functions
  1157	 *
  1158	 ****************************************************************************/
  1159	
  1160	static int wait_on_sem(struct amd_iommu *iommu, u64 data)
  1161	{
  1162		struct iommu_cmd *cmd;
  1163		int i = 0;
  1164	
  1165		while (*iommu->cmd_sem != data && i < LOOP_TIMEOUT) {
  1166			udelay(1);
  1167			i += 1;
  1168		}
  1169	
  1170		if (i == LOOP_TIMEOUT) {
> 1171			int head, tail;
  1172	
  1173			head = readl(iommu->mmio_base + MMIO_CMD_HEAD_OFFSET);
  1174			tail = readl(iommu->mmio_base + MMIO_CMD_TAIL_OFFSET);
  1175	
  1176			pr_alert("IOMMU %04x:%02x:%02x.%01x: Completion-Wait loop timed out\n",
  1177				 iommu->pci_seg->id, PCI_BUS_NUM(iommu->devid),
  1178				 PCI_SLOT(iommu->devid), PCI_FUNC(iommu->devid));
  1179			/*
  1180			 * On command buffer completion timeout, step back by 2 commands
  1181			 * to locate the actual command that is causing the issue.
  1182			 */
  1183			tail = (MMIO_CMD_BUFFER_TAIL(tail) - 2) & (CMD_BUFFER_ENTRIES - 1);
  1184			cmd = (struct iommu_cmd *)(iommu->cmd_buf + tail * sizeof(*cmd));
  1185			dump_command(iommu_virt_to_phys(cmd));
  1186	
  1187			return -EIO;
  1188		}
  1189	
  1190		return 0;
  1191	}
  1192	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki