[PATCH] PCI: Avoid D3 on PLX PEX 8532/8533/8548 downstream ports

[PATCH] PCI: Avoid D3 on PLX PEX 8532/8533/8548 downstream ports

[Imre Kaloz]

Runtime PM suspends an idle downstream port of a PLX/Broadcom PEX
8532/8533/8548 switch to D3hot, but that takes the whole switch down
with it: any device behind another port of the same switch then
faults with a fatal bus error on its next access, e.g. mptsas probe
on a Sun Ultra 45.

This surfaced on sparc64 once commit a5fb3ff63287 ("PCI: Allow PCI
bridges to go to D3Hot on all non-x86") let bridges enter D3hot. Keep
these downstream ports out of D3 by setting PCI_DEV_FLAGS_NO_D3.

Fixes: a5fb3ff63287 ("PCI: Allow PCI bridges to go to D3Hot on all non-x86")
Cc: stable@vger.kernel.org
Signed-off-by: Imre Kaloz <kaloz@kernel.org>
---
 drivers/pci/quirks.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index de9bbccda21f..a3c9250182a6 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -2381,6 +2381,24 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_PLX, PCI_DEVICE_ID_PLX_9050,
 DECLARE_PCI_FIXUP_HEADER(0x1402, 0x2000, quirk_plx_pci9050);
 DECLARE_PCI_FIXUP_HEADER(0x1402, 0x2600, quirk_plx_pci9050);
 
+/*
+ * An idle downstream port of a PLX/Broadcom PEX 8532/8533/8548 switch
+ * autosuspends into D3hot, and that takes the whole switch down with
+ * it -- any device behind another port of the same switch then sees a
+ * fatal, non-recoverable bus fault on its next access. Keep these
+ * ports out of D3 entirely.
+ */
+static void quirk_plx_pex_no_d3(struct pci_dev *dev)
+{
+	if (pci_pcie_type(dev) == PCI_EXP_TYPE_DOWNSTREAM) {
+		dev->dev_flags |= PCI_DEV_FLAGS_NO_D3;
+		pci_info(dev, "avoiding D3 on this downstream port\n");
+	}
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PLX, 0x8532, quirk_plx_pex_no_d3);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PLX, 0x8533, quirk_plx_pex_no_d3);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PLX, 0x8548, quirk_plx_pex_no_d3);
+
 static void quirk_netmos(struct pci_dev *dev)
 {
 	unsigned int num_parallel = (dev->subsystem_device & 0xf0) >> 4;

base-commit: fe2ec83746e501645709761605c2464a44fd2929
-- 
2.47.3

Re: [PATCH] PCI: Avoid D3 on PLX PEX 8532/8533/8548 downstream ports

[Magnus Lindholm]

Hi Imre,

Tested this on a Sun Ultra 45 with PLX PEX8532 downstream ports and an
LSI SAS1064 controller.

Without the patch, enabling CONFIG_PCIEPORTBUS reproducibly causes a
Cheetah bus-error panic when mptbase probes the controller. The kernel
boots with pcie_port_pm=off.

With this patch, the same kernel and config boot successfully without
pcie_port_pm=off or pcie_ports=compat. The quirk is applied to all five
PEX8532 downstream ports, the LSISAS1064 initializes, and the modular
mptsas/ext4 setup boots through the initramfs normally.

Tested-by: Magnus Lindholm <linmag7@gmail.com>

Re: [PATCH] PCI: Avoid D3 on PLX PEX 8532/8533/8548 downstream ports

[Bjorn Helgaas]

[+cc several Broadcom folks, help with possible switch erratum]

On Sun, Sep 27, 2026 at 12:56:09PM +0200, Imre Kaloz wrote:
> Runtime PM suspends an idle downstream port of a PLX/Broadcom PEX
> 8532/8533/8548 switch to D3hot, but that takes the whole switch down
> with it: any device behind another port of the same switch then
> faults with a fatal bus error on its next access, e.g. mptsas probe
> on a Sun Ultra 45.
> 
> This surfaced on sparc64 once commit a5fb3ff63287 ("PCI: Allow PCI
> bridges to go to D3Hot on all non-x86") let bridges enter D3hot. Keep
> these downstream ports out of D3 by setting PCI_DEV_FLAGS_NO_D3.

I don't really have any objection to this, but I am concerned that
this quirk only applies to a few specific devices, but
https://admin.pci-ids.ucw.cz/read/PC/10b5 shows a TON of
similar-looking switches that I assume have the same behavior, so
this may be just the tip of the iceberg.

> Fixes: a5fb3ff63287 ("PCI: Allow PCI bridges to go to D3Hot on all non-x86")
> Cc: stable@vger.kernel.org
> Signed-off-by: Imre Kaloz <kaloz@kernel.org>
> ---
>  drivers/pci/quirks.c | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
> 
> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
> index de9bbccda21f..a3c9250182a6 100644
> --- a/drivers/pci/quirks.c
> +++ b/drivers/pci/quirks.c
> @@ -2381,6 +2381,24 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_PLX, PCI_DEVICE_ID_PLX_9050,
>  DECLARE_PCI_FIXUP_HEADER(0x1402, 0x2000, quirk_plx_pci9050);
>  DECLARE_PCI_FIXUP_HEADER(0x1402, 0x2600, quirk_plx_pci9050);
>  
> +/*
> + * An idle downstream port of a PLX/Broadcom PEX 8532/8533/8548 switch
> + * autosuspends into D3hot, and that takes the whole switch down with
> + * it -- any device behind another port of the same switch then sees a
> + * fatal, non-recoverable bus fault on its next access. Keep these
> + * ports out of D3 entirely.
> + */
> +static void quirk_plx_pex_no_d3(struct pci_dev *dev)
> +{
> +	if (pci_pcie_type(dev) == PCI_EXP_TYPE_DOWNSTREAM) {
> +		dev->dev_flags |= PCI_DEV_FLAGS_NO_D3;
> +		pci_info(dev, "avoiding D3 on this downstream port\n");
> +	}
> +}
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PLX, 0x8532, quirk_plx_pex_no_d3);
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PLX, 0x8533, quirk_plx_pex_no_d3);
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PLX, 0x8548, quirk_plx_pex_no_d3);
> +
>  static void quirk_netmos(struct pci_dev *dev)
>  {
>  	unsigned int num_parallel = (dev->subsystem_device & 0xf0) >> 4;
> 
> base-commit: fe2ec83746e501645709761605c2464a44fd2929
> -- 
> 2.47.3
> 

Re: [PATCH] PCI: Avoid D3 on PLX PEX 8532/8533/8548 downstream ports

[Bjorn Helgaas]

On Sun, Sep 27, 2026 at 12:56:09PM +0200, Imre Kaloz wrote:
> Runtime PM suspends an idle downstream port of a PLX/Broadcom PEX
> 8532/8533/8548 switch to D3hot, but that takes the whole switch down
> with it: any device behind another port of the same switch then
> faults with a fatal bus error on its next access, e.g. mptsas probe
> on a Sun Ultra 45.
> 
> This surfaced on sparc64 once commit a5fb3ff63287 ("PCI: Allow PCI
> bridges to go to D3Hot on all non-x86") let bridges enter D3hot. Keep
> these downstream ports out of D3 by setting PCI_DEV_FLAGS_NO_D3.
> 
> Fixes: a5fb3ff63287 ("PCI: Allow PCI bridges to go to D3Hot on all non-x86")
> Cc: stable@vger.kernel.org
> Signed-off-by: Imre Kaloz <kaloz@kernel.org>

Applied to pci/pm for v7.4, thanks!

> ---
>  drivers/pci/quirks.c | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
> 
> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
> index de9bbccda21f..a3c9250182a6 100644
> --- a/drivers/pci/quirks.c
> +++ b/drivers/pci/quirks.c
> @@ -2381,6 +2381,24 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_PLX, PCI_DEVICE_ID_PLX_9050,
>  DECLARE_PCI_FIXUP_HEADER(0x1402, 0x2000, quirk_plx_pci9050);
>  DECLARE_PCI_FIXUP_HEADER(0x1402, 0x2600, quirk_plx_pci9050);
>  
> +/*
> + * An idle downstream port of a PLX/Broadcom PEX 8532/8533/8548 switch
> + * autosuspends into D3hot, and that takes the whole switch down with
> + * it -- any device behind another port of the same switch then sees a
> + * fatal, non-recoverable bus fault on its next access. Keep these
> + * ports out of D3 entirely.
> + */
> +static void quirk_plx_pex_no_d3(struct pci_dev *dev)
> +{
> +	if (pci_pcie_type(dev) == PCI_EXP_TYPE_DOWNSTREAM) {
> +		dev->dev_flags |= PCI_DEV_FLAGS_NO_D3;
> +		pci_info(dev, "avoiding D3 on this downstream port\n");
> +	}
> +}
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PLX, 0x8532, quirk_plx_pex_no_d3);
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PLX, 0x8533, quirk_plx_pex_no_d3);
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PLX, 0x8548, quirk_plx_pex_no_d3);
> +
>  static void quirk_netmos(struct pci_dev *dev)
>  {
>  	unsigned int num_parallel = (dev->subsystem_device & 0xf0) >> 4;
> 
> base-commit: fe2ec83746e501645709761605c2464a44fd2929
> -- 
> 2.47.3
> 

Re: [PATCH] PCI: Avoid D3 on PLX PEX 8532/8533/8548 downstream ports

[Florian Fainelli]

On 10/12/26 14:34, Bjorn Helgaas wrote:
> [+cc several Broadcom folks, help with possible switch erratum]
> 
> On Sun, Sep 27, 2026 at 12:56:09PM +0200, Imre Kaloz wrote:
>> Runtime PM suspends an idle downstream port of a PLX/Broadcom PEX
>> 8532/8533/8548 switch to D3hot, but that takes the whole switch down
>> with it: any device behind another port of the same switch then
>> faults with a fatal bus error on its next access, e.g. mptsas probe
>> on a Sun Ultra 45.
>>
>> This surfaced on sparc64 once commit a5fb3ff63287 ("PCI: Allow PCI
>> bridges to go to D3Hot on all non-x86") let bridges enter D3hot. Keep
>> these downstream ports out of D3 by setting PCI_DEV_FLAGS_NO_D3.
> 
> I don't really have any objection to this, but I am concerned that
> this quirk only applies to a few specific devices, but
> https://admin.pci-ids.ucw.cz/read/PC/10b5 shows a TON of
> similar-looking switches that I assume have the same behavior, so
> this may be just the tip of the iceberg.
> 
>> Fixes: a5fb3ff63287 ("PCI: Allow PCI bridges to go to D3Hot on all non-x86")
>> Cc: stable@vger.kernel.org
>> Signed-off-by: Imre Kaloz <kaloz@kernel.org>

Our internal portal does not have any document for the 8532/8533/8548 so 
unfortunately I cannot help here with identifying an errata document.
-- 
Florian