[PATCH v2 0/2] Wait for device readiness after D3hot -> D0uninitialized transition

[PATCH v2 0/2] Wait for device readiness after D3hot -> D0uninitialized transition

[Bjorn Helgaas]

In pci_power_up(), wait for device to become ready after a soft reset
(D3hot->D0uninitialized transition).  We already wait 10 ms, but even after
that, the device is permitted to respond to config requests with Request
Retry Status (RRS) completion status.  Use pci_dev_wait() to wait for
successful completion.

Bjorn Helgaas (2):
  PCI: Log device readiness timeouts as errors
  PCI: Wait for device readiness after D3hot -> D0uninitialized
    transition

 drivers/pci/pci.c | 28 ++++++++++++++++++++++++----
 1 file changed, 24 insertions(+), 4 deletions(-)


Changes since v1:
  - Log device readiness timeouts as errors, not warnings
  - On timeout, set dev->current_state to D3cold and return -EIO

v1: https://lore.kernel.org/all/20260514153124.404060-1-bhelgaas@google.com/

[PATCH v2 1/2] PCI: Log device readiness timeouts as errors

[Bjorn Helgaas]

pci_dev_wait() waits for a device to be Configuration-Ready after a reset,
such as a Function-Level Reset (FLR), a soft reset during a D3hot->
D0uninitialized transition when No_Soft_Reset == 0), or a power-up sequence
from D3cold->D0uninitialized.

If pci_dev_wait() returns success, the device is guaranteed to respond to
configuration requests with Successful Completion status.  If it times out,
device is completely non-responsive.

Upgrade the log level from pci_warn() to pci_err() to reflect this failure
state.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/pci.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 8f7cfcc00090..5a9af0bb2c71 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -1253,8 +1253,8 @@ static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout)
 		}
 
 		if (delay > timeout) {
-			pci_warn(dev, "not ready %dms after %s; giving up\n",
-				 delay - 1, reset_type);
+			pci_err(dev, "not ready %dms after %s; giving up\n",
+				delay - 1, reset_type);
 			return -ENOTTY;
 		}
 
-- 
2.51.0

[PATCH v2 2/2] PCI: Wait for device readiness after D3hot -> D0uninitialized transition

[Bjorn Helgaas]

From: Bjorn Helgaas <helgaas@kernel.org>

For a device that advertises No_Soft_Reset == 0, a transition from D3hot to
D0uninitialized is a soft reset, and the resulting internal device state is
undefined.

Per PCIe r7.0, sec 2.3.1, a transition from D3hot to D0uninitialized
mandates a minimum 10 ms delay before accessing the device. Following this
delay, the device is permitted to respond to initial configuration requests
with a Request Retry Status (RRS) completion status if it needs more time
to initialize.

Call pci_dev_wait() after pci_power_up() performs a D3hot->D0uninitialized
transition to ensure the device is ready to accept config accesses, as is
done after the similar transition in pci_pm_reset().

If the device is already ready, this is essentially a no-op except for one
additional config read.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 drivers/pci/pci.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 5a9af0bb2c71..8228d2782f95 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -1300,7 +1300,18 @@ int pci_power_up(struct pci_dev *dev)
 	bool need_restore;
 	pci_power_t state;
 	u16 pmcsr;
+	int ret;
 
+	/*
+	 * When setting power state to D0, platform_pci_set_power_state()
+	 * ensures main power is on.  If it puts the device in D0, it also
+	 * completes any required delays after the transition; if it leaves
+	 * the device in D1, D2, or D3hot, we use the PM Capability to
+	 * transition to D0.
+	 *
+	 * In all cases, the device is either Configuration-Ready or
+	 * inaccessible upon return.
+	 */
 	platform_pci_set_power_state(dev, PCI_D0);
 
 	if (!dev->pm_cap) {
@@ -1341,10 +1352,19 @@ int pci_power_up(struct pci_dev *dev)
 	pci_write_config_word(dev, dev->pm_cap + PCI_PM_CTRL, 0);
 
 	/* Mandatory transition delays; see PCI PM 1.2. */
-	if (state == PCI_D3hot)
+	if (state == PCI_D3hot) {
 		pci_dev_d3_sleep(dev);
-	else if (state == PCI_D2)
+		if (!(pmcsr & PCI_PM_CTRL_NO_SOFT_RESET)) {
+			ret = pci_dev_wait(dev, "power up D3hot->D0uninitialized",
+					   PCIE_RESET_READY_POLL_MS);
+			if (ret) {
+				dev->current_state = PCI_D3cold;
+				return -EIO;
+			}
+		}
+	} else if (state == PCI_D2) {
 		udelay(PCI_PM_D2_DELAY);
+	}
 
 end:
 	dev->current_state = PCI_D0;
-- 
2.51.0

Re: [PATCH v2 2/2] PCI: Wait for device readiness after D3hot -> D0uninitialized transition

[Rafael J. Wysocki]

On Mon, May 18, 2026 at 9:12 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> From: Bjorn Helgaas <helgaas@kernel.org>
>
> For a device that advertises No_Soft_Reset == 0, a transition from D3hot to
> D0uninitialized is a soft reset, and the resulting internal device state is
> undefined.
>
> Per PCIe r7.0, sec 2.3.1, a transition from D3hot to D0uninitialized
> mandates a minimum 10 ms delay before accessing the device. Following this
> delay, the device is permitted to respond to initial configuration requests
> with a Request Retry Status (RRS) completion status if it needs more time
> to initialize.
>
> Call pci_dev_wait() after pci_power_up() performs a D3hot->D0uninitialized
> transition to ensure the device is ready to accept config accesses, as is
> done after the similar transition in pci_pm_reset().
>
> If the device is already ready, this is essentially a no-op except for one
> additional config read.
>
> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> ---
>  drivers/pci/pci.c | 24 ++++++++++++++++++++++--
>  1 file changed, 22 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 5a9af0bb2c71..8228d2782f95 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -1300,7 +1300,18 @@ int pci_power_up(struct pci_dev *dev)
>         bool need_restore;
>         pci_power_t state;
>         u16 pmcsr;
> +       int ret;
>
> +       /*
> +        * When setting power state to D0, platform_pci_set_power_state()
> +        * ensures main power is on.  If it puts the device in D0, it also
> +        * completes any required delays after the transition; if it leaves
> +        * the device in D1, D2, or D3hot, we use the PM Capability to
> +        * transition to D0.
> +        *
> +        * In all cases, the device is either Configuration-Ready or
> +        * inaccessible upon return.
> +        */
>         platform_pci_set_power_state(dev, PCI_D0);
>
>         if (!dev->pm_cap) {
> @@ -1341,10 +1352,19 @@ int pci_power_up(struct pci_dev *dev)
>         pci_write_config_word(dev, dev->pm_cap + PCI_PM_CTRL, 0);
>
>         /* Mandatory transition delays; see PCI PM 1.2. */
> -       if (state == PCI_D3hot)
> +       if (state == PCI_D3hot) {
>                 pci_dev_d3_sleep(dev);
> -       else if (state == PCI_D2)
> +               if (!(pmcsr & PCI_PM_CTRL_NO_SOFT_RESET)) {

Hmm.  The pmcsr value here is from before the pci_write_config_word(),
so AFAICS it should be updated after the write.

This means an additional config space access and I wonder if it can be
added to pci_dev_wait()?

> +                       ret = pci_dev_wait(dev, "power up D3hot->D0uninitialized",
> +                                          PCIE_RESET_READY_POLL_MS);
> +                       if (ret) {
> +                               dev->current_state = PCI_D3cold;
> +                               return -EIO;
> +                       }
> +               }
> +       } else if (state == PCI_D2) {
>                 udelay(PCI_PM_D2_DELAY);
> +       }
>
>  end:
>         dev->current_state = PCI_D0;
> --

Re: [PATCH v2 2/2] PCI: Wait for device readiness after D3hot -> D0uninitialized transition

[Lukas Wunner]

On Mon, May 18, 2026 at 09:38:12PM +0200, Rafael J. Wysocki wrote:
> On Mon, May 18, 2026 at 9:12 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > +++ b/drivers/pci/pci.c
> > @@ -1341,10 +1352,19 @@ int pci_power_up(struct pci_dev *dev)
> >         pci_write_config_word(dev, dev->pm_cap + PCI_PM_CTRL, 0);
> >
> >         /* Mandatory transition delays; see PCI PM 1.2. */
> > -       if (state == PCI_D3hot)
> > +       if (state == PCI_D3hot) {
> >                 pci_dev_d3_sleep(dev);
> > -       else if (state == PCI_D2)
> > +               if (!(pmcsr & PCI_PM_CTRL_NO_SOFT_RESET)) {
> 
> Hmm.  The pmcsr value here is from before the pci_write_config_word(),
> so AFAICS it should be updated after the write.

The No_Soft_Reset bit is RO (PCIe r7.0 sec 7.5.2.2), so its value should
never change when writing zero to the Power Management Control/Status
Register.

Thanks,

Lukas

Re: [PATCH v2 2/2] PCI: Wait for device readiness after D3hot -> D0uninitialized transition

[Rafael J. Wysocki]

On Mon, May 18, 2026 at 9:46 PM Lukas Wunner <lukas@wunner.de> wrote:
>
> On Mon, May 18, 2026 at 09:38:12PM +0200, Rafael J. Wysocki wrote:
> > On Mon, May 18, 2026 at 9:12 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > > +++ b/drivers/pci/pci.c
> > > @@ -1341,10 +1352,19 @@ int pci_power_up(struct pci_dev *dev)
> > >         pci_write_config_word(dev, dev->pm_cap + PCI_PM_CTRL, 0);
> > >
> > >         /* Mandatory transition delays; see PCI PM 1.2. */
> > > -       if (state == PCI_D3hot)
> > > +       if (state == PCI_D3hot) {
> > >                 pci_dev_d3_sleep(dev);
> > > -       else if (state == PCI_D2)
> > > +               if (!(pmcsr & PCI_PM_CTRL_NO_SOFT_RESET)) {
> >
> > Hmm.  The pmcsr value here is from before the pci_write_config_word(),
> > so AFAICS it should be updated after the write.
>
> The No_Soft_Reset bit is RO (PCIe r7.0 sec 7.5.2.2), so its value should
> never change when writing zero to the Power Management Control/Status
> Register.

Oh right, I confused it with something else.

Re: [PATCH v2 0/2] Wait for device readiness after D3hot -> D0uninitialized transition

[Rafael J. Wysocki]

On Mon, May 18, 2026 at 9:12 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> In pci_power_up(), wait for device to become ready after a soft reset
> (D3hot->D0uninitialized transition).  We already wait 10 ms, but even after
> that, the device is permitted to respond to config requests with Request
> Retry Status (RRS) completion status.  Use pci_dev_wait() to wait for
> successful completion.
>
> Bjorn Helgaas (2):
>   PCI: Log device readiness timeouts as errors
>   PCI: Wait for device readiness after D3hot -> D0uninitialized
>     transition
>
>  drivers/pci/pci.c | 28 ++++++++++++++++++++++++----
>  1 file changed, 24 insertions(+), 4 deletions(-)
>
>
> Changes since v1:
>   - Log device readiness timeouts as errors, not warnings
>   - On timeout, set dev->current_state to D3cold and return -EIO
>
> v1: https://lore.kernel.org/all/20260514153124.404060-1-bhelgaas@google.com/

For the series:

Reviewed-by: Rafael J. Wysocki (Intel) <rafael@kernel.org>

Re: [PATCH v2 0/2] Wait for device readiness after D3hot -> D0uninitialized transition

[Bjorn Helgaas]

On Mon, May 18, 2026 at 02:12:17PM -0500, Bjorn Helgaas wrote:
> In pci_power_up(), wait for device to become ready after a soft reset
> (D3hot->D0uninitialized transition).  We already wait 10 ms, but even after
> that, the device is permitted to respond to config requests with Request
> Retry Status (RRS) completion status.  Use pci_dev_wait() to wait for
> successful completion.
> 
> Bjorn Helgaas (2):
>   PCI: Log device readiness timeouts as errors
>   PCI: Wait for device readiness after D3hot -> D0uninitialized
>     transition
> 
>  drivers/pci/pci.c | 28 ++++++++++++++++++++++++----
>  1 file changed, 24 insertions(+), 4 deletions(-)
> 
> 
> Changes since v1:
>   - Log device readiness timeouts as errors, not warnings
>   - On timeout, set dev->current_state to D3cold and return -EIO
> 
> v1: https://lore.kernel.org/all/20260514153124.404060-1-bhelgaas@google.com/

I applied these to pci/reset for v7.2.