[PATCH v2 0/2] PCI: Wait after FLR for devices that don't honor Immediate Readiness

[PATCH v2 0/2] PCI: Wait after FLR for devices that don't honor Immediate Readiness

[Alexander Gruhlke]

Since v7.2, VFIO passthrough of a Samsung 990 PRO NVMe [144d:a80c] fails
on the second VM start, and the device is inaccessible until reboot.  The
drive advertises Immediate Readiness but returns ~0 for up to 7 ms after
FLR, so since 10baa9b4df40 ("PCI: Drop unnecessary retries when restoring
BARs") the config restore after FLR is lost.

Patch 1 ignores Immediate Readiness on that drive.  A 990 EVO Plus
[144d:a80d] reports the same errors [1], but it isn't confirmed to
advertise Immediate Readiness, so it's not included.

Patch 2 is independent and keeps pci_dev_wait() from treating ~0 as
ready with RRS Software Visibility, which also affects the Intel
[8086:0a54] in the open d591f6804e7e regression.  Hui, Linux-Fan: testing
patch 2 on that drive would be appreciated.

Tested on v7.2.9 (Ryzen 9 7950X, X670E, drive firmware 1B2QJXD7): with
patch 1, three VM starts in a row work, and every FLR reports "ready 0ms
after FLR".  After updating the drive to firmware 8B2QJXD7, it still
advertises Immediate Readiness but responds right after FLR, and plain
v7.2.9 works, so only older firmware is affected.

Patch 2 in this form is only tested for regressions, as the drive no
longer shows the problem.  v1's variant, which read the Command register,
was tested against it, and the drive's Vendor ID read ~0 while it wasn't
ready.  Applies to pci/next.

Changes since v1:
- Replace the pcie_flr() change with a quirk for the 990 PRO (Lukas)
- pci_dev_wait(): check the Vendor ID for ~0 except on VFs instead of
  reading the Command register, and "may return" in the comment (Lukas)
- Swap the patch order, as the quirk alone fixes the regression

v1: https://lore.kernel.org/r/20261010160529.27974-1-gruhlke@mailbox.org

[1] https://bbs.archlinux.org/viewtopic.php?id=314833

Alexander Gruhlke (2):
  PCI: Ignore Immediate Readiness on Samsung 990 PRO NVMe
  PCI: Don't treat ~0 as ready in pci_dev_wait() with RRS SV

 drivers/pci/pci.c    |  8 +++++---
 drivers/pci/quirks.c | 11 +++++++++++
 2 files changed, 16 insertions(+), 3 deletions(-)

-- 
2.56.0

[PATCH v2 1/2] PCI: Ignore Immediate Readiness on Samsung 990 PRO NVMe

[Alexander Gruhlke]

The Samsung 990 PRO [144d:a80c] advertises Immediate Readiness, so
pcie_flr() doesn't wait after FLR, but the drive returns ~0 for up to
7 ms.  Since commit 10baa9b4df40 ("PCI: Drop unnecessary retries when
restoring BARs"), the config restore after FLR is lost, which breaks
VFIO passthrough of the drive.  Firmware 8B2QJXD7 responds right after
FLR, but older firmware such as 1B2QJXD7 doesn't, and the firmware
version isn't visible here.

Ignore Immediate Readiness on this device, so pcie_flr() waits.

Fixes: 10baa9b4df40 ("PCI: Drop unnecessary retries when restoring BARs")
Link: https://bbs.archlinux.org/viewtopic.php?id=314833
Link: https://lore.kernel.org/r/20261010160529.27974-1-gruhlke@mailbox.org
Suggested-by: Lukas Wunner <lukas@wunner.de>
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Alexander Gruhlke <gruhlke@mailbox.org>
---
 drivers/pci/quirks.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 0500d5fd4f..e8597d1209 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -5623,6 +5623,17 @@ static void quirk_no_flr_snet(struct pci_dev *dev)
 }
 DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_SOLIDRUN, 0x1000, quirk_no_flr_snet);
 
+/* Samsung 990 PRO advertises Immediate Readiness, but isn't ready after FLR */
+static void quirk_no_imm_ready(struct pci_dev *dev)
+{
+	if (!dev->imm_ready)
+		return;
+
+	pci_info(dev, "Ignoring incorrectly advertised Immediate Readiness support\n");
+	dev->imm_ready = 0;
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_SAMSUNG, 0xa80c, quirk_no_imm_ready);
+
 static void quirk_no_ext_tags(struct pci_dev *pdev)
 {
 	struct pci_host_bridge *bridge = pci_find_host_bridge(pdev->bus);
-- 
2.56.0

[PATCH v2 2/2] PCI: Don't treat ~0 as ready in pci_dev_wait() with RRS SV

[Alexander Gruhlke]

With RRS Software Visibility, pci_dev_wait() considers a device ready as
soon as reading the Vendor ID doesn't return the RRS value.  Devices
aren't required to respond with RRS while they aren't ready, so the read
may return ~0 and pci_dev_wait() stops waiting too early.  This was seen
with Intel [8086:0a54] and Samsung [144d:a80c] NVMe SSDs.

Also wait for a Vendor ID other than ~0, except on VFs, whose Vendor ID
normally reads ~0.

Fixes: d591f6804e7e ("PCI: Wait for device readiness with Configuration RRS")
Link: https://lore.kernel.org/r/20250611101442.387378-1-hui.wang@canonical.com
Suggested-by: Lukas Wunner <lukas@wunner.de>
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Alexander Gruhlke <gruhlke@mailbox.org>
---
 drivers/pci/pci.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 350bae907e..fef8a799c1 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -1213,9 +1213,10 @@ static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout)
 	 * initialize.
 	 *
 	 * If the device is below a Root Port with Configuration RRS
-	 * Software Visibility enabled, reading the Vendor ID returns a
+	 * Software Visibility enabled, reading the Vendor ID may return a
 	 * special data value if the device responded with RRS.  Read the
-	 * Vendor ID until we get non-RRS status.
+	 * Vendor ID until we get non-RRS status and, unless it's a VF,
+	 * something other than ~0.
 	 *
 	 * If there's no Root Port or Configuration RRS Software Visibility
 	 * is not enabled, the device may still respond with RRS, but
@@ -1235,7 +1236,8 @@ static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout)
 
 		if (root && root->config_rrs_sv) {
 			pci_read_config_dword(dev, PCI_VENDOR_ID, &id);
-			if (!pci_bus_rrs_vendor_id(id))
+			if (!pci_bus_rrs_vendor_id(id) &&
+			    (!PCI_POSSIBLE_ERROR(id) || dev->is_virtfn))
 				break;
 		} else {
 			pci_read_config_dword(dev, PCI_COMMAND, &id);
-- 
2.56.0

Re: [PATCH v2 1/2] PCI: Ignore Immediate Readiness on Samsung 990 PRO NVMe

[Lukas Wunner]

On Sun, Oct 11, 2026 at 06:32:48PM +0200, Alexander Gruhlke wrote:
> The Samsung 990 PRO [144d:a80c] advertises Immediate Readiness, so
> pcie_flr() doesn't wait after FLR, but the drive returns ~0 for up to
> 7 ms.  Since commit 10baa9b4df40 ("PCI: Drop unnecessary retries when
> restoring BARs"), the config restore after FLR is lost, which breaks
> VFIO passthrough of the drive.  Firmware 8B2QJXD7 responds right after
> FLR, but older firmware such as 1B2QJXD7 doesn't, and the firmware
> version isn't visible here.

If a firmware update is available from the vendor, we generally
refrain from adding a quirk and instead tell users to update.

OTOH, the offending patch breaks setups that worked before,
which is annoying.

It's also a bit of a bummer that the quirk can't check the
firmware version.  Has the upgrade changed the Revision ID
in the Config Space header?  If it did, that could be used
to identiy non-broken devices, but usually it's only incremented
for new silicon revisions.

Up to Bjorn what to do here.

FWIW,
Reviewed-by: Lukas Wunner <lukas@wunner.de>

Re: [PATCH v2 1/2] PCI: Ignore Immediate Readiness on Samsung 990 PRO NVMe

[Alexander Gruhlke]

On Sun, Oct 11, 2026 at 08:00:32PM +0200, Lukas Wunner wrote:
> It's also a bit of a bummer that the quirk can't check the
> firmware version.  Has the upgrade changed the Revision ID
> in the Config Space header?

No, it's 0x00 with both 1B2QJXD7 and 8B2QJXD7, and the Subsystem ID
(144d:a801) didn't change either.  On fixed firmware the quirk only
costs the 100 ms wait per FLR.

Fine with me either way, thanks for the review.

Alexander