Re: [PATCH v2] PCI: Mask Replay Timer Timeout for Realtek RTS525A

[Max Lee <max.lee@canonical.com>]

On Thu, Jul 02, 2026 at 04:42:43PM +0200, Lukas Wunner wrote:
> There were Advisory Non-Fatal Errors on both ends of the links.
> The upstream kernel does not support ANFE so far, but this
> development branch contains three tentative patches to add it:
>
> https://github.com/l1k/linux/commits/anfe_v1/
>
> So far this is compile-tested only. You may want to give these
> patches a spin to see which Non-Fatal Errors are signaled. The
> kernel should also dump the TLP Prefix Log for those errors and
> you can use this tool to decode it:
>
> https://github.com/mmpg-x86/tlp-tool
>
> See the example usage in: Documentation/PCI/pcieaer-howto.rst
>
> The TLP Prefix Log might give a hint as to the root cause.
>
> [...]
>
> Bjorn mentioned commit eeee3b5e6d0b, which states that the Replay Timer
> Timeout errors (only) occur when ASPM is enabled. That may be the
> actual root cause, so you may want to play with ASPM settings (disable
> L1 substates etc) to see if it makes the issue go away. Disabling
> non-working ASPM settings in a quirk would be better than silencing
> the ensuing errors.

Thanks Lukas for the suggestions

I tested Lukas' anfe_v1 branch on the affected system:

https://github.com/l1k/linux.git anfe_v1
ca3701e1b037 ("PCI/AER: Support Advisory Non-Fatal Errors")

The tested kernel was:

Linux localhost 7.2.0-rc1+ #2 SMP PREEMPT_DYNAMIC Fri Jul 3 08:00:24 UTC 2026

With that kernel, the storm still reproduced, but the reported error
remained a Correctable Replay Timer Timeout. I did not see Advisory
Non-Fatal Error or TLP Prefix/Header Log output for this storm.

The relevant dmesg lines were still:

pcieport 0000:00:1c.6: AER: Correctable error message received from 0000:58:00.0
pci 0000:58:00.0: PCIe Bus Error: severity=Correctable
pci 0000:58:00.0: device [10ec:525a] error status/mask=00001000/00004000
pci 0000:58:00.0: [12] Timeout | Transmitter | Data Link Layer

In that boot, dmesg contained 767 printed Correctable/Timeout messages
and 76 aer_ratelimit lines. The actual interrupt/counter volume was much
higher. Before stopping the storm, /proc/interrupts showed:

IR-PCI-MSI-0000:00:1c.6 ... 37981589 ... PCIe PME, aerdrv, PCIe bwctrl

The AER sysfs counters showed:

Endpoint 0000:58:00.0:
Timeout 12018352
TOTAL_ERR_COR 12018352
TOTAL_ERR_FATAL 0
TOTAL_ERR_NONFATAL 0

Root Port 0000:00:1c.6:
aer_rootport_total_err_cor 12062319
aer_rootport_total_err_fatal 0
aer_rootport_total_err_nonfatal 0

I then tested the ASPM settings directly. With OS-native AER active,
disabling only L0s on the RTS525A link stopped new AER interrupt and
counter growth while leaving L1 enabled:

echo 0 > /sys/bus/pci/devices/0000:58:00.0/link/l0s_aspm

After that, over a 10 second interval:

irq_delta=0
endpoint_cor_delta=0
rootport_cor_delta=0
link_l0s=0
link_l1=1

Disabling L1, L1 substates, or Clock PM alone did not stop the storm.
The storm also did not require the rtsx_pci driver to bind: it was
reproducible while rtsx_pci/rtsx_pci_sdmmc were blacklisted and the
endpoint was not enabled by a driver.

Based on this, I agree that disabling the non-working ASPM state is a
better direction than masking the resulting AER reports.

I tested a v3 patch that disables L0s for the RTS525A by removing L0s
from the device's ASPM link capability:

DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_REALTEK, 0x525a,
quirk_disable_aspm_l0s);

I tested that patch on top of Lukas' anfe_v1 branch. The patched kernel
booted as:

Linux localhost 7.2.0-rc1-rts525a-l0s+ #4 SMP PREEMPT_DYNAMIC Mon Jul
6 02:08:30 UTC 2026

The quirk ran during enumeration:

pci 0000:58:00.0: ASPM: Link Capabilities L0s treated as unsupported
to avoid device defect

With the temporary rtsx_pci blacklist removed and the initramfs rebuilt,
the card reader driver bound normally:

  rtsx_pci 0000:58:00.0: enabling device (0000 -> 0002)

  rtsx_pci_sdmmc         36864  0
  rtsx_pci              143360  1 rtsx_pci_sdmmc

The endpoint was enabled and bound to rtsx_pci:

  0000:58:00.0 enable=1
  0000:58:00.0 power_state=D3hot
  driver=/sys/bus/pci/drivers/rtsx_pci

L0s was no longer enabled, while L1 remained enabled on both ends of the
link:

  0000:00:1c.6:
    LnkCtl: ASPM L1 Enabled

  0000:58:00.0:
    LnkCtl: ASPM L1 Enabled

The endpoint link sysfs state also showed L1 and L1 substates still
enabled:

  clkpm=1
  l1_1_aspm=1
  l1_1_pcipm=1
  l1_2_aspm=1
  l1_2_pcipm=1
  l1_aspm=1

The l0s_aspm sysfs file was no longer present for the endpoint link,
consistent with L0s being removed from the ASPM capability.

After booting the patched kernel with rtsx_pci bound, there were no AER
Correctable Replay Timer Timeout messages and no aer_ratelimit lines in
dmesg:

  aer_correctable=0
  timeout_lines=0
  ratelimit_lines=0

Over a 10 second interval after boot, IRQ and AER counters did not
increase:

  irq_delta=0
  endpoint_cor_delta=0
  rootport_cor_delta=0

The counters stayed at zero:

  Endpoint 0000:58:00.0:
    Timeout 0
    TOTAL_ERR_COR 0

  Root Port 0000:00:1c.6:
    aer_rootport_total_err_cor 0

So for this system, the evidence points specifically at L0s on the
RTS525A link: enabling L0s triggers the Correctable Replay Timer Timeout
storm, and disabling L0s stops it while leaving L1 enabled.

I will send v3 changing the quirk from masking AER Replay Timer Timeout
to disabling ASPM L0s for the RTS525A.

Thanks,
Max