mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3
@ 2026-11-15 17:18 Aviv Bakal
  2026-11-15 17:18 ` [PATCH 1/3] PCI/PM: Honor PCI_DEV_FLAGS_NO_D3 in pci_bridge_d3_possible() Aviv Bakal
                   ` (3 more replies)
  0 siblings, 4 replies; 7+ messages in thread
From: Aviv Bakal @ 2026-11-15 17:18 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Manivannan Sadhasivam, linux-pci, linux-kernel, Zeev Zilberman

d4c79b63d82d ("PCI: Allow D3 for native hotplug-capable Root Ports on
non-x86 platforms") lets native pciehp ports runtime suspend to D3hot
on non-x86. The PCIe switch ports on AWS instances advertise PME from
D3hot but do not send it, so hotplug events such as an ENI attach are
lost on Graviton. x86 keeps the old logic, so the problem does not show
up there.

Patches 1-2 handle existing switch devices, which advertise PME:
patch 1 makes pci_bridge_d3_possible() honor PCI_DEV_FLAGS_NO_D3, and
patch 2 sets that flag on the Amazon switch ports.

Patch 3 handles switch devices that advertise no PME from D3hot: such
hotplug ports are kept out of D3.

Aviv Bakal (3):
  PCI/PM: Honor PCI_DEV_FLAGS_NO_D3 in pci_bridge_d3_possible()
  PCI: Avoid runtime D3 on Amazon PCIe switch ports
  PCI/PM: Keep hotplug ports out of D3 if they can't signal PME from
    D3hot

 drivers/pci/pci.c    | 16 +++++++++++++++-
 drivers/pci/quirks.c | 12 ++++++++++++
 2 files changed, 27 insertions(+), 1 deletion(-)


base-commit: 23c62304dde29abe92013d85910d03a77dada35d
-- 
2.50.1


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH 1/3] PCI/PM: Honor PCI_DEV_FLAGS_NO_D3 in pci_bridge_d3_possible()
  2026-11-15 17:18 [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3 Aviv Bakal
@ 2026-11-15 17:18 ` Aviv Bakal
  2026-11-15 17:18 ` [PATCH 2/3] PCI: Avoid runtime D3 on Amazon PCIe switch ports Aviv Bakal
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 7+ messages in thread
From: Aviv Bakal @ 2026-11-15 17:18 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Manivannan Sadhasivam, linux-pci, linux-kernel, Zeev Zilberman, stable

pci_set_power_state() does not put a device with PCI_DEV_FLAGS_NO_D3
into D3, but pci_bridge_d3_possible() ignores the flag.
pcie_portdrv_probe() therefore enables runtime PM on such a port, and
on runtime suspend pciehp masks the hotplug interrupt while the port
stays in D0.

Return false from pci_bridge_d3_possible() when the flag is set.

Fixes: d4c79b63d82d ("PCI: Allow D3 for native hotplug-capable Root Ports on non-x86 platforms")
Cc: stable@vger.kernel.org # 7.3.x
Co-developed-by: Zeev Zilberman <zeev@amazon.com>
Signed-off-by: Zeev Zilberman <zeev@amazon.com>
Signed-off-by: Aviv Bakal <avivb@amazon.com>
---
 drivers/pci/pci.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 7c6f2aba8144..1a23be4b6a07 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3055,6 +3055,10 @@ bool pci_bridge_d3_possible(struct pci_dev *bridge)
 		if (pci_bridge_d3_disable)
 			return false;
 
+		/* The port is known not to work correctly in D3 */
+		if (bridge->dev_flags & PCI_DEV_FLAGS_NO_D3)
+			return false;
+
 		/*
 		 * Hotplug ports handled by platform firmware may not be put
 		 * into D3 by the OS, e.g. ACPI slots ...
-- 
2.50.1


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH 2/3] PCI: Avoid runtime D3 on Amazon PCIe switch ports
  2026-11-15 17:18 [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3 Aviv Bakal
  2026-11-15 17:18 ` [PATCH 1/3] PCI/PM: Honor PCI_DEV_FLAGS_NO_D3 in pci_bridge_d3_possible() Aviv Bakal
@ 2026-11-15 17:18 ` Aviv Bakal
  2026-11-15 17:18 ` [PATCH 3/3] PCI/PM: Keep hotplug ports out of D3 if they can't signal PME from D3hot Aviv Bakal
  2026-11-15 17:22 ` [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3 Lukas Wunner
  3 siblings, 0 replies; 7+ messages in thread
From: Aviv Bakal @ 2026-11-15 17:18 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Manivannan Sadhasivam, linux-pci, linux-kernel, Zeev Zilberman, stable

The PCIe switch ports on AWS instances (vendor 0x1d0f) advertise PME
from D3hot but do not send it. Since commit d4c79b63d82d ("PCI: Allow
D3 for native hotplug-capable Root Ports on non-x86 platforms") these
ports runtime suspend to D3hot on non-x86 platforms, and hotplug events
such as an ENI attach are lost.

Mark them PCI_DEV_FLAGS_NO_D3.

Fixes: d4c79b63d82d ("PCI: Allow D3 for native hotplug-capable Root Ports on non-x86 platforms")
Cc: stable@vger.kernel.org # 7.3.x
Co-developed-by: Zeev Zilberman <zeev@amazon.com>
Signed-off-by: Zeev Zilberman <zeev@amazon.com>
Signed-off-by: Aviv Bakal <avivb@amazon.com>
---
 drivers/pci/quirks.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 7f1f47ea2ee4..292047804b09 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -1476,6 +1476,18 @@ DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AL, PCI_ANY_ID,
 DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_VIA, PCI_ANY_ID,
 				PCI_CLASS_STORAGE_IDE, 8, quirk_no_ata_d3);
 
+/*
+ * Amazon 1d0f PCIe switch ports (e.g. 1d0f:cec2) on AWS instances
+ * advertise PME# from D3hot but never assert it, so hotplug events on
+ * a port in D3 are lost.
+ */
+static void quirk_amazon_bridge_no_d3(struct pci_dev *pdev)
+{
+	pdev->dev_flags |= PCI_DEV_FLAGS_NO_D3;
+}
+DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_AMAZON, PCI_ANY_ID,
+			      PCI_CLASS_BRIDGE_PCI, 8, quirk_amazon_bridge_no_d3);
+
 /*
  * This was originally an Alpha-specific thing, but it really fits here.
  * The i82375 PCI/EISA bridge appears as non-classified. Fix that.
-- 
2.50.1


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH 3/3] PCI/PM: Keep hotplug ports out of D3 if they can't signal PME from D3hot
  2026-11-15 17:18 [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3 Aviv Bakal
  2026-11-15 17:18 ` [PATCH 1/3] PCI/PM: Honor PCI_DEV_FLAGS_NO_D3 in pci_bridge_d3_possible() Aviv Bakal
  2026-11-15 17:18 ` [PATCH 2/3] PCI: Avoid runtime D3 on Amazon PCIe switch ports Aviv Bakal
@ 2026-11-15 17:18 ` Aviv Bakal
  2026-11-15 17:22 ` [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3 Lukas Wunner
  3 siblings, 0 replies; 7+ messages in thread
From: Aviv Bakal @ 2026-11-15 17:18 UTC (permalink / raw)
  To: Bjorn Helgaas
  Cc: Manivannan Sadhasivam, linux-pci, linux-kernel, Zeev Zilberman

On runtime suspend pciehp masks the hotplug interrupt and relies on
PME to resume the port on a slot event. pci_bridge_d3_possible() does
not check whether the port can signal PME from D3hot, so a port that
cannot is not resumed by slot events.

Return false from pci_bridge_d3_possible() for such native hotplug
ports. pci_pm_init() computes bridge_d3 before it reads PME_Support,
so move that assignment after dev->pme_support is set.

Co-developed-by: Zeev Zilberman <zeev@amazon.com>
Signed-off-by: Zeev Zilberman <zeev@amazon.com>
Signed-off-by: Aviv Bakal <avivb@amazon.com>
---
 drivers/pci/pci.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 1a23be4b6a07..0b2e20e781e6 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3081,6 +3081,14 @@ bool pci_bridge_d3_possible(struct pci_dev *bridge)
 		if (platform_pci_bridge_d3(bridge))
 			return true;
 
+		/*
+		 * pciehp masks the hotplug interrupt on runtime suspend and
+		 * relies on PME to resume the port on a slot event. A port
+		 * that cannot signal PME from D3hot is not resumed.
+		 */
+		if (bridge->is_pciehp && !pci_pme_capable(bridge, PCI_D3hot))
+			return false;
+
 		/*
 		 * Hotplug ports handled natively by the OS on x86 platforms
 		 * were not validated by vendors for runtime D3 at least until
@@ -3247,7 +3255,6 @@ void pci_pm_init(struct pci_dev *dev)
 	dev->pm_cap = pm;
 	dev->d3hot_delay = PCI_PM_D3HOT_WAIT;
 	dev->d3cold_delay = PCI_PM_D3COLD_WAIT;
-	dev->bridge_d3 = pci_bridge_d3_possible(dev);
 	dev->d3cold_allowed = true;
 
 	dev->d1_support = false;
@@ -3283,6 +3290,9 @@ void pci_pm_init(struct pci_dev *dev)
 		pci_pme_active(dev, false);
 	}
 
+	/* pci_bridge_d3_possible() looks at pme_support */
+	dev->bridge_d3 = pci_bridge_d3_possible(dev);
+
 poweron:
 	pci_update_current_state(dev, PCI_UNKNOWN);
 	if (dev->current_state == PCI_D0) {
-- 
2.50.1


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3
  2026-11-15 17:18 [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3 Aviv Bakal
                   ` (2 preceding siblings ...)
  2026-11-15 17:18 ` [PATCH 3/3] PCI/PM: Keep hotplug ports out of D3 if they can't signal PME from D3hot Aviv Bakal
@ 2026-11-15 17:22 ` Lukas Wunner
  2026-11-16 16:55   ` Aviv Bakal
  3 siblings, 1 reply; 7+ messages in thread
From: Lukas Wunner @ 2026-11-15 17:22 UTC (permalink / raw)
  To: Aviv Bakal
  Cc: Bjorn Helgaas, Manivannan Sadhasivam, linux-pci, linux-kernel,
	Zeev Zilberman, Kuppuswamy Sathyanarayanan

[cc += Sathya]

On Sun, Nov 15, 2026 at 07:18:28PM +0200, Aviv Bakal wrote:
> d4c79b63d82d ("PCI: Allow D3 for native hotplug-capable Root Ports on
> non-x86 platforms") lets native pciehp ports runtime suspend to D3hot
> on non-x86. The PCIe switch ports on AWS instances advertise PME from
> D3hot but do not send it, so hotplug events such as an ENI attach are
> lost on Graviton. x86 keeps the old logic, so the problem does not show
> up there.

Intel Catlow Lake suffers from a similar issue:

https://lore.kernel.org/all/20261103193136.2410148-1-sathyanarayanan.kuppuswamy@linux.intel.com/

Please try to follow the approach taken there.

Thanks,

Lukas

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3
  2026-11-15 17:22 ` [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3 Lukas Wunner
@ 2026-11-16 16:55   ` Aviv Bakal
  2026-11-16 17:03     ` Lukas Wunner
  0 siblings, 1 reply; 7+ messages in thread
From: Aviv Bakal @ 2026-11-16 16:55 UTC (permalink / raw)
  To: Lukas Wunner
  Cc: Bjorn Helgaas, Manivannan Sadhasivam, Kuppuswamy Sathyanarayanan,
	Zeev Zilberman, linux-pci, linux-kernel

On Sun, Nov 15, 2026 at 07:22:33PM +0100, Lukas Wunner wrote:
> Intel Catlow Lake suffers from a similar issue:
>
> https://lore.kernel.org/all/20261103193136.2410148-1-sathyanarayanan.kuppuswamy@linux.intel.com/
>
> Please try to follow the approach taken there.

Thanks Lukas. In the Catlow Lake case the Root Ports still signal the
hot-add with a PME interrupt, and only PME Status is unreliable. Our
ports are switch downstream ports that advertise PME from D3hot but
never send a PM_PME message, so the Root Port gets no interrupt and a
pcie_pme_irq() hook would never run. Keeping these ports out of D3
looks like the only option for the existing devices.

I sent a v2 that fixes a runtime PM imbalance in patch 1 when NO_D3 is
set by a FINAL fixup.

Thanks,
Aviv

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3
  2026-11-16 16:55   ` Aviv Bakal
@ 2026-11-16 17:03     ` Lukas Wunner
  0 siblings, 0 replies; 7+ messages in thread
From: Lukas Wunner @ 2026-11-16 17:03 UTC (permalink / raw)
  To: Aviv Bakal
  Cc: Bjorn Helgaas, Manivannan Sadhasivam, Kuppuswamy Sathyanarayanan,
	Zeev Zilberman, linux-pci, linux-kernel

On Mon, Nov 16, 2026 at 06:55:22PM +0200, Aviv Bakal wrote:
> On Sun, Nov 15, 2026 at 07:22:33PM +0100, Lukas Wunner wrote:
> > Intel Catlow Lake suffers from a similar issue:
> >
> > https://lore.kernel.org/all/20261103193136.2410148-1-sathyanarayanan.kuppuswamy@linux.intel.com/
> >
> > Please try to follow the approach taken there.
> 
> Thanks Lukas. In the Catlow Lake case the Root Ports still signal the
> hot-add with a PME interrupt, and only PME Status is unreliable. Our
> ports are switch downstream ports that advertise PME from D3hot but
> never send a PM_PME message, so the Root Port gets no interrupt and a
> pcie_pme_irq() hook would never run. Keeping these ports out of D3
> looks like the only option for the existing devices.

We cache the power states which support PME in struct pci_dev::pme_support.
Have you considered just modifying that in a DECLARE_PCI_FIXUP_FINAL()
callback?  See e.g. pci_fixup_no_d0_pme(), pci_fixup_no_msi_no_pme(),
pci_fixup_amd_ehci_pme(), pci_fixup_amd_fch_xhci_pme().

> I sent a v2 that fixes a runtime PM imbalance in patch 1 when NO_D3 is
> set by a FINAL fixup.

Please wait one or two weeks before respinning to allow time for the
community to review your proposed changes and provide feedback.

Thanks,

Lukas

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2026-11-16 17:03 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-15 17:18 [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3 Aviv Bakal
2026-11-15 17:18 ` [PATCH 1/3] PCI/PM: Honor PCI_DEV_FLAGS_NO_D3 in pci_bridge_d3_possible() Aviv Bakal
2026-11-15 17:18 ` [PATCH 2/3] PCI: Avoid runtime D3 on Amazon PCIe switch ports Aviv Bakal
2026-11-15 17:18 ` [PATCH 3/3] PCI/PM: Keep hotplug ports out of D3 if they can't signal PME from D3hot Aviv Bakal
2026-11-15 17:22 ` [PATCH 0/3] PCI/PM: Keep hotplug ports that can't wake from D3hot out of D3 Lukas Wunner
2026-11-16 16:55   ` Aviv Bakal
2026-11-16 17:03     ` Lukas Wunner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®