Re: [EXTERNAL] [tip: x86/apic] x86/io_apic: Cleanup trigger/polarity helpers

[Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>]

David

On 11/17/20 9:00 AM, Suravee Suthikulpanit wrote:
> David,
> 
> On 11/13/20 10:14 PM, David Woodhouse wrote:
>> On Wed, 2020-11-11 at 14:30 -0600, Tom Lendacky wrote:
>>> I had trouble cloning your tree for some reason, so just took the top
>>> three patches and applied them to the tip tree. This all appears to be
>>> working. I'll let the IOMMU experts take a closer look (adding Suravee).
>>
>> Thanks. I see Thomas has taken the first two into the tip.git x86/apic
>> branch already, so we're just looking for an ack on the third. Which is
>> this one...
>>
>>  From 49ee4fa51b8c06d14b7c4c74d15a7d76f865a8ea Mon Sep 17 00:00:00 2001
>> From: David Woodhouse <dwmw@amazon.co.uk>
>> Date: Wed, 11 Nov 2020 12:09:01 +0000
>> Subject: [PATCH] iommu/amd: Fix IOMMU interrupt generation in X2APIC mode
>>
>> The AMD IOMMU has two modes for generating its own interrupts.
>>
>> The first is very much based on PCI MSI, and can be configured by Linux
>> precisely that way. But like legacy unmapped PCI MSI it's limited to
>> 8 bits of APIC ID.
>>
>> The second method does not use PCI MSI at all in hardawre, and instead
>> configures the INTCAPXT registers in the IOMMU directly with the APIC ID
>> and vector.
>>
>> In the latter case, the IOMMU driver would still use pci_enable_msi(),
>> read back (through MMIO) the MSI message that Linux wrote to the PCI MSI
>> table, then swizzle those bits into the appropriate register.
>>
>> Historically, this worked because__irq_compose_msi_msg() would silently
>> generate an invalid MSI message with the high bits of the APIC ID in the
>> high bits of the MSI address. That hack was intended only for the Intel
>> IOMMU, and I recently enforced that, introducing a warning in
>> __irq_msi_compose_msg() if it was invoked with an APIC ID above 255.
>>
>> Fix the AMD IOMMU not to depend on that hack any more, by having its own
>> irqdomain and directly putting the bits from the irq_cfg into the right
>> place in its ->activate() method.
>>
>> Fixes: 47bea873cf80 "x86/msi: Only use high bits of MSI address for DMAR unit")
>> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> 
> I'm still working on testing this series using IO_PAGE_FAULT injection to trigger the IOMMU interrupts. I am still 
> debugging some issues, and I'll keep you updated on the findings.
> 
> Thanks,
> Suravee

I might need your help debugging this issue. I'm seeing the following error:

[   14.005937] irq 29, desc: 00000000d200500b, depth: 0, count: 0, unhandled: 0
[   14.006234] ->handle_irq():  00000000eab4b6eb, handle_bad_irq+0x0/0x230
[   14.006234] ->irq_data.chip(): 000000001cce6d6b, intcapxt_controller+0x0/0x120
[   14.006234] ->action(): 0000000083bfd734
[   14.006234] ->action->handler(): 0000000094806345, amd_iommu_int_handler+0x0/0x10
[   14.006234] unexpected IRQ trap at vector 1d

Do you have any idea what might have gone wrong here?

Thanks,
Suravee