mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Ziyou Wang (Lenovo)" <ziyou.dev@foxmail.com>
To: helgaas@kernel.org
Cc: bhelgaas@google.com, lukas@wunner.de,
	sathyanarayanan.kuppuswamy@linux.intel.com,
	linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
	stable@vger.kernel.org, ziyou.dev@foxmail.com
Subject: Re: [PATCH v2] PCI/DPC: Increase recovery timeout to 60 seconds
Date: Thu, 19 Nov 2026 11:30:36 +0800	[thread overview]
Message-ID: <tencent_F14FD4212049828EDB98B55B6E0FE116CF07@qq.com> (raw)
In-Reply-To: <20261106152143.GA1621130@bhelgaas>

Hi Bjorn,

Thanks for your review!

On Fri, Nov 06, 2026 at 09:21:43AM -0600, Bjorn Helgaas wrote:
> Was there a "not ready ... after DPC; giving up" message here?
> pci_dev_wait() should log *something* about how it resolved this
> situation.

No.  Here is the complete log from the DPC event to the re-enumeration:

  [121.693046] pcieport 0000:00:03.1: EDR: EDR event received
  [121.695743] pcieport 0000:00:03.1: DPC: containment event, status:0x1f01: unmasked uncorrectable error detected
  [121.695825] pcieport 0000:00:03.1: PCIe Bus Error: severity=Uncorrectable (Fatal)
  [121.695829] pcieport 0000:00:03.1:   device [1022:1108] error status/mask=00200000/00000000
  [121.695831] pcieport 0000:00:03.1:   [21] ACSViol           | Receiver    | Transaction Layer (First)
  [121.695834] pcieport 0000:00:03.1: AER:   TLP Header: 0x60002080 0x010042ff 0x00000017 0x7e325c00
  [121.695841] nvme nvme0: frozen state error detected, reset controller
  [121.721168] nvme0n1: I/O Cmd(0x2) @ LBA 67600640, 256 blocks, I/O Error (sct 0x3 / sc 0x71)
  [121.721182] I/O error, dev nvme0n1, sector 67600640 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 2
  [121.721193] nvme0n1: I/O Cmd(0x2) @ LBA 67600384, 256 blocks, I/O Error (sct 0x3 / sc 0x71)
  [121.721196] I/O error, dev nvme0n1, sector 67600384 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 2
  [123.018691] nvme 0000:01:00.0: not ready 1023ms after DPC; waiting
  [124.106690] nvme 0000:01:00.0: not ready 2047ms after DPC; waiting
  [125.642703] pcieport 0000:00:03.1: pciehp: Slot(64): Link Down
  [125.647840] Buffer I/O error on dev nvme0n1, logical block 8450048, async page read
  [125.647846] Buffer I/O error on dev nvme0n1, logical block 8450049, async page read
  [125.647847] Buffer I/O error on dev nvme0n1, logical block 8450050, async page read
  [125.647848] Buffer I/O error on dev nvme0n1, logical block 8450051, async page read
  [125.647850] Buffer I/O error on dev nvme0n1, logical block 8450052, async page read
  [125.647850] Buffer I/O error on dev nvme0n1, logical block 8450053, async page read
  [125.647851] Buffer I/O error on dev nvme0n1, logical block 8450054, async page read
  [125.647852] Buffer I/O error on dev nvme0n1, logical block 8450055, async page read
  [125.647853] Buffer I/O error on dev nvme0n1, logical block 8450056, async page read
  [125.647854] Buffer I/O error on dev nvme0n1, logical block 8450057, async page read
  [125.849608] pcieport 0000:00:03.1: pciehp: Slot(64): Card present
  [125.849614] pcieport 0000:00:03.1: pciehp: Slot(64): Link Up
  [126.218696] pcieport 0000:00:03.1: AER: subordinate device reset failed
  [126.218700] pcieport 0000:00:03.1: AER: device recovery failed
  [126.705745] pci 0000:01:00.0: [1dbe:5668] type 00 class 0x010802 PCIe Endpoint
  [126.705786] pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x00003fff 64bit]
  [126.705793] pci 0000:01:00.0: ROM [mem 0x00000000-0x0000ffff pref]
  [126.705801] pci 0000:01:00.0: Max Payload Size set to 512 (was 128, max 512)
  [126.705987] pci 0000:01:00.0: _HPX attempts Link Control setting (AND 0xffff OR 0x0040)
  [126.706073] pci 0000:01:00.0: supports D1 D2
  [126.706074] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
  [126.706432] pci 0000:01:00.0: Adding to iommu group 16
  [126.706481] pci 0000:01:00.0: ROM [mem 0xde300000-0xde30ffff pref]: assigned
  [126.706483] pci 0000:01:00.0: BAR 0 [mem 0xde310000-0xde313fff 64bit]: assigned
  [126.706491] pcieport 0000:00:03.1: PCI bridge to [bus 01]
  [126.706494] pcieport 0000:00:03.1:   bridge window [io  0x1000-0x1fff]
  [126.706498] pcieport 0000:00:03.1:   bridge window [mem 0xde300000-0xde3fffff]
  [126.706500] pcieport 0000:00:03.1:   bridge window [mem 0x10021600000-0x100217fffff 64bit pref]
  [126.709685] nvme nvme1: pci function 0000:01:00.0
  [126.709692] nvme 0000:01:00.0: enabling device (0140 -> 0142)
  [126.712068] nvme nvme1: D3 entry latency set to 10 seconds
  [126.720936] nvme nvme1: 129/0/0 default/read/poll queues
  [126.727879]  nvme0n2: p1 p2 p3

pci_dev_wait() never reached its timeout.  The 4s pci_dpc_recovered()
timeout expired first, so pciehp removed the device and marked it
disconnected.  When pci_dev_wait() woke up, it took the
pci_dev_is_disconnected() path, which only logs at debug level and
returns -ENOTTY.  That is why the -ENOTTY surfaced as "AER:
subordinate device reset failed" instead.

The device needs a little over 4s to become ready ("ready 4095ms
after DPC" with this patch), so the 4s timeout is losing that race.

One thing worth noting: before e8df1d194c39b ("PCI: Don't treat ~0 as
ready in pci_dev_wait() with RRS SV"), this failure was masked on this
system.  The RRS SV path only checked for the 0x0001 RRS Vendor ID, so
a ~0 read from the not-yet-ready device was treated as "ready",
pci_bridge_wait_for_secondary_bus() returned early, and the DPC
"recovery" appeared to succeed within the 4s window, even though the
device was not actually usable (nvme then failed with -19 after the
"successful" recovery).  With e8df1d194c39b, pci_dev_wait() correctly
waits, which re-exposed this 4s race.

Best regards,
Ziyou Wang


  reply	other threads:[~2026-11-19  3:31 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20261106082612.1840817-1-ziyou.dev@foxmail.com>
2026-11-06  8:56 ` Ziyou Wang (Lenovo)
2026-11-06 15:21   ` Bjorn Helgaas
2026-11-19  3:30     ` Ziyou Wang (Lenovo) [this message]
2026-11-19  9:01       ` Lukas Wunner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=tencent_F14FD4212049828EDB98B55B6E0FE116CF07@qq.com \
    --to=ziyou.dev@foxmail.com \
    --cc=bhelgaas@google.com \
    --cc=helgaas@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=lukas@wunner.de \
    --cc=sathyanarayanan.kuppuswamy@linux.intel.com \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®