[PATCH v3 0/2] PCI: Wait after FLR for devices that don't honor Immediate Readiness

[Alexander Gruhlke <gruhlke@mailbox.org>]

Since v7.2, VFIO passthrough of a Samsung 990 PRO NVMe [144d:a80c] fails
on the second VM start, and the device is inaccessible until reboot.  The
drive advertises Immediate Readiness but returns ~0 for up to 7 ms after
FLR, so since 10baa9b4df40 ("PCI: Drop unnecessary retries when restoring
BARs") the config restore after FLR is lost.

Patch 1 ignores Immediate Readiness on that drive.  Firmware 8B2QJXD7
fixes it, but the Revision ID and Subsystem ID don't change, so the
quirk can't tell firmware versions apart.  A 990 EVO Plus [144d:a80d]
reports the same errors [1], but it isn't confirmed to advertise
Immediate Readiness, so it's not included.

Patch 2 is independent and keeps pci_dev_wait() from treating ~0 as
ready with RRS Software Visibility, which also affects the Intel
[8086:0a54] in the open d591f6804e7e regression.  Hui, Linux-Fan: testing
patch 2 on that drive would be appreciated.

Tested on v7.2.9 (Ryzen 9 7950X, X670E): with patch 1 and drive firmware
1B2QJXD7, three VM starts in a row work, and every FLR reports "ready
0ms after FLR".  With firmware 8B2QJXD7, plain v7.2.9 works.

Patch 2 is only tested for regressions, as the drive no longer shows the
problem: below a Root Port with RRS SV, three VM starts work and every
FLR reports "ready 0ms after FLR".  The VF path is untested with the
patch, but on another machine, an X550 VF below a Root Port with RRS SV
reads ~0 from the Vendor ID and valid data from the Command register
100 ms after FLR.  v1's variant, which read the Command register, was
tested against the old firmware, and the drive's Vendor ID read ~0
while it wasn't ready.  Applies to pci/next.

Changes since v2:
- Patch 1: add Lukas's Reviewed-by
- Patch 2: read the Command register on VFs whose Vendor ID reads ~0
  instead of treating them as ready (Lukas, sashiko)

v2: https://lore.kernel.org/r/20261011163249.18479-1-gruhlke@mailbox.org
v1: https://lore.kernel.org/r/20261010160529.27974-1-gruhlke@mailbox.org

[1] https://bbs.archlinux.org/viewtopic.php?id=314833

Alexander Gruhlke (2):
  PCI: Ignore Immediate Readiness on Samsung 990 PRO NVMe
  PCI: Don't treat ~0 as ready in pci_dev_wait() with RRS SV

 drivers/pci/pci.c    | 14 ++++++++++----
 drivers/pci/quirks.c | 11 +++++++++++
 2 files changed, 21 insertions(+), 4 deletions(-)

-- 
2.56.0