PCI: failed link retraining quirk costs 1 s per empty AMD Promontory 21 port

PCI: failed link retraining quirk costs 1 s per empty AMD Promontory 21 port

[Tomasz Bojanowski]

Hi,

On an AMD AM5 desktop the pcie_failed_link_retrain() quirk fires on
two empty downstream ports of the chipset switch and adds 2 s to every
boot.  This seems to be a different case from the one addressed by
"PCI: Skip Target Speed quirk on clamped ports with no link" [1]: here
the Target Link Speed is not clamped, so that patch does not return
early and the LBMS branch is taken instead.

Hardware: MSI MAG B850M MORTAR WIFI (MS-7E61), BIOS 1.A68 09/07/2026,
AMD Promontory 21 switch, downstream ports [1022:43f5].  Nothing is
connected behind 06:00.0 or 06:08.0.

Seen on Fedora kernels 7.2.5, 7.2.6 and 7.2.7 (33 boots, every time).
The same two messages were already present on openSUSE 7.1.x kernels
on this machine in July, so this is not new in 7.2.

  [    0.248171] pci 0000:06:00.0: broken device, retraining non-functional downstream link at 2.5GT/s
  [    1.248035] pci 0000:06:00.0: retraining failed
  [    1.248972] pci 0000:06:08.0: broken device, retraining non-functional downstream link at 2.5GT/s
  [    2.248036] pci 0000:06:08.0: retraining failed

Port state after boot (06:08.0 is the same except Width x2 in LnkCap):

  LnkCap:  Port #0, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <32us
  LnkSta:  Speed 2.5GT/s, Width x1
           TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
  SltCap:  AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
  LnkCtl2: Target Link Speed: 16GT/s

Since the message is printed, DLLLA was clear and pcie_lbms_seen() was
true when the quirk ran from pci_device_add(), before bwctrl is bound,
so the LBMS bit itself must have been set on a port with nothing behind
it.  The kerneldoc above the quirk expects both bits to be clear on an
unconnected port, which this chipset does not seem to honour.
BWMgmt- after boot is expected, as bwctrl clears LBMS when it binds.

Each port then waits PCIE_LINK_RETRAIN_TIMEOUT_MS before the error path
restores the original 16GT/s target.  With this chipset used on many
AM5 boards, other systems may well lose 1 s per empty chipset port too.

Let me know if any other information would be useful.

[1] https://lore.kernel.org/all/20260801201244.4421-1-andiwild@gmail.com/

Thanks,
Tomasz Bojanowski

Re: PCI: failed link retraining quirk costs 1 s per empty AMD Promontory 21 port

[Thorsten Leemhuis]

On 9/23/26 21:13, Tomasz Bojanowski wrote:
> 
> On an AMD AM5 desktop the pcie_failed_link_retrain() quirk fires on
> two empty downstream ports of the chipset switch and adds 2 s to every
> boot.  This seems to be a different case from the one addressed by
> "PCI: Skip Target Speed quirk on clamped ports with no link" [1]: here
> the Target Link Speed is not clamped, so that patch does not return
> early and the LBMS branch is taken instead.

Hmm, nobody replied afaics (or did I miss something), so it seems nobody
has an idea what's wrong here. So I guess you need to perform a git
bisection.

Ciao, Thorsten
> Hardware: MSI MAG B850M MORTAR WIFI (MS-7E61), BIOS 1.A68 09/07/2026,
> AMD Promontory 21 switch, downstream ports [1022:43f5].  Nothing is
> connected behind 06:00.0 or 06:08.0.
> 
> Seen on Fedora kernels 7.2.5, 7.2.6 and 7.2.7 (33 boots, every time).
> The same two messages were already present on openSUSE 7.1.x kernels
> on this machine in July, so this is not new in 7.2.
> 
>   [    0.248171] pci 0000:06:00.0: broken device, retraining non-functional downstream link at 2.5GT/s
>   [    1.248035] pci 0000:06:00.0: retraining failed
>   [    1.248972] pci 0000:06:08.0: broken device, retraining non-functional downstream link at 2.5GT/s
>   [    2.248036] pci 0000:06:08.0: retraining failed
> 
> Port state after boot (06:08.0 is the same except Width x2 in LnkCap):
> 
>   LnkCap:  Port #0, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <32us
>   LnkSta:  Speed 2.5GT/s, Width x1
>            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
>   SltCap:  AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
>   LnkCtl2: Target Link Speed: 16GT/s
> 
> Since the message is printed, DLLLA was clear and pcie_lbms_seen() was
> true when the quirk ran from pci_device_add(), before bwctrl is bound,
> so the LBMS bit itself must have been set on a port with nothing behind
> it.  The kerneldoc above the quirk expects both bits to be clear on an
> unconnected port, which this chipset does not seem to honour.
> BWMgmt- after boot is expected, as bwctrl clears LBMS when it binds.
> 
> Each port then waits PCIE_LINK_RETRAIN_TIMEOUT_MS before the error path
> restores the original 16GT/s target.  With this chipset used on many
> AM5 boards, other systems may well lose 1 s per empty chipset port too.
> 
> Let me know if any other information would be useful.
> 
> [1] https://lore.kernel.org/all/20260801201244.4421-1-andiwild@gmail.com/
> 
> Thanks,
> Tomasz Bojanowski

Re: PCI: failed link retraining quirk costs 1 s per empty AMD Promontory 21 port

[Tomasz Bojanowski]

> 04.10.2026 17:29 CEST Thorsten Leemhuis <regressions@leemhuis.info> napisał(a):
> Hmm, nobody replied afaics (or did I miss something), so it seems nobody
> has an idea what's wrong here. So I guess you need to perform a git
> bisection.

Thanks for following up.  You did not miss anything, there were no
replies so far.

I am not sure a bisection is possible though: I do not know of any
kernel where this does not happen.  I see the same 2 s delay on 7.1.x
(openSUSE) and on 7.2.x (Fedora), and I believe it was already there
on 6.19 and 7.0 as well, so I did not mean to report it as a
regression, sorry if the report read that way.  As far as I can tell
the quirk simply behaves as written: LBMS is set on a port with no
link, so the "broken device" branch is taken and the retrain times out.

I would be grateful for a hint from the PCI folks on whether this is
expected on empty ports.

Thanks,
Tomasz