Dear Linux kernel developers,
  We met problems with the AMD IOMMU on Gigabyte F2A88X-UP4 motherboard. With
IOMMU option enabled in BIOS, we fail to get on-board VIA xHCI controller
initialized and worked. We tried to contact the motherboard vendor Gigabyte
China office. But they did not provide useful support. So, we hope you can get
us out of this trouble.

  Now please allow me to explain the situation in detail.
1, With IOMMU option enabled in BIOS, we received the following complaints from
"dmesg" and the VIA xHCI controller cannot work.
--------------------------part of information taken from demsg-----------------
[    1.214464] xhci_hcd 0000:03:00.0: xHCI Host Controller
[    1.214471] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 10
[    1.214576] AMD-Vi: Event logged [IO_PAGE_FAULT device=03:00.0 domain=0x001b address=0x00000000bcd55880 flags=0x0010]
[    1.214683] AMD-Vi: Event logged [IO_PAGE_FAULT device=03:00.0 domain=0x001b address=0x00000000bcd55880 flags=0x0010]
[    1.214783] AMD-Vi: Event logged [IO_PAGE_FAULT device=03:00.0 domain=0x001b address=0x00000000bcd55880 flags=0x0010]
...
repeated above message
...
[    1.920436] tsc: Refined TSC clocksource calibration: 4092.556 MHz
[   19.512395] xhci_hcd 0000:03:00.0: can't setup: -110 
[   19.512488] xhci_hcd 0000:03:00.0: USB bus 10 deregistered
[   19.512507] Switched to clocksource tsc
[   19.512591] xhci_hcd 0000:03:00.0: init 0000:03:00.0 fail, -110
[   19.512652] xhci_hcd: probe of 0000:03:00.0 failed with error -110
-----------------------------end of the information----------------------------

2, If we disabled the IOMMU option in BIOS, there will be no such error message.
And the VIA xHCI controller worked as it should be.

3, The problem first appeared on Ubuntu13.04 64bit OS. Later we confirmed
Ubuntu13.10 also had such problem. We even tried the latest vanilla kernel
and still same problem occurred. And by the way, the BIOS used is already
the latest version F5c on Gigabyte website.

4, After some digging, we found this problem happens under the circumstance
below:
1). When to do xHCI reset, VIA xHCI controller accesses PHY address
0x00000000bcd55880.
2). With IOMMU enabled, IOMMU captures this address, regards it as a virtual
address and tries to translate it. 
3). IOMMU found this address is out of its page table and reports IO_PAGE_FAULT.

In fact, from dmesg, we can know PHY address 0x00000000bcd55880 belongs to BIOS
reserved area.
-----------------------------BIOS RAM map info----------------------------------
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009e7ff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009e800-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bcd44fff] usable
[    0.000000] BIOS-e820: [mem 0x00000000bcd45000-0x00000000bcd85fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000bcd86000-0x00000000bd046fff] usable
[    0.000000] BIOS-e820: [mem 0x00000000bd047000-0x00000000bd112fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x00000000bd113000-0x00000000be1b0fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000be1b1000-0x00000000be1b1fff] usable
[    0.000000] BIOS-e820: [mem 0x00000000be1b2000-0x00000000be3b7fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x00000000be3b8000-0x00000000be84bfff] usable
[    0.000000] BIOS-e820: [mem 0x00000000be84c000-0x00000000befdcfff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000befdd000-0x00000000beffffff] usable
[    0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed00000-0x00000000fed00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed40000-0x00000000fed44fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed80000-0x00000000fed8ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000043effffff] usable
-----------------------------end of BIOS RAM map info---------------------------
So, we think if xHCI accesses a PHY address, this PHY address should not be
translated by IOMMU. It should be excluded by IVMD table.

5, So, we created a patch to exclude the address by IVMD talbe(please check the
attched patch 0001-Create-fake-ivmd-table-to-make-via-xhci-work-on-GA-F.patch).
After applying the patch, the VIA xHCI controller worked with IOMMU enabled.
However, it's only a temporary solution.

This problem really troubles us a lot. We hope gigabyte engineers can pay
attention to this issue and update BIOS if possible. If someone in this mail
list can setup a bridge for us to contact gigabyte, we are very appreciated.
Of course, any other comments or suggestions are also welcomed. Thank you!

P.S. We collected as much information as possible. Please check the attachments
appended. All of them were collected on Ubuntu13.10 64bit OS with latest vanilla
kernel installed.

Best Regards,
Liangliang Quan