Re: [PATCH v2] PCI: qcom: Block accesses to downstream devices on link down

[Marc Zyngier <maz@kernel.org>]

On Wed, 16 Sep 2026 00:14:21 +0100,
Bjorn Helgaas <helgaas@kernel.org> wrote:
> 
> [+cc James, Andrew, Marc for guidance on generic arm64 PCIe error
> recovery; beginning of thread:
> https://lore.kernel.org/all/20260819-ecam_blocker-v2-1-e7a8fdc1c5cb@oss.qualcomm.com]
> 
> On Tue, Sep 15, 2026 at 07:16:11PM +0200, Manivannan Sadhasivam wrote:
> > On Fri, Sep 11, 2026 at 12:18:07PM -0500, Bjorn Helgaas wrote:
> > > On Fri, Sep 11, 2026 at 08:17:29AM +0200, Manivannan Sadhasivam wrote:
> > > > On Tue, Sep 08, 2026 at 06:00:30PM -0500, Bjorn Helgaas wrote:
> > > > > On Wed, Aug 19, 2026 at 11:36:54PM -0700, Qiang Yu wrote:
> > > > > > After a PCIe link goes down, software may still access the
> > > > > > BAR (MMIO) space or configuration space of devices behind
> > > > > > that link before recovery has run. As the link is down,
> > > > > > these accesses never complete, resulting in a storm of
> > > > > > Completion Timeout AERs.
> > > > > 
> > > > > What is special about qcom here?  It seems like the Completion
> > > > > Timeouts and AER interrupts should happen with every PCIe
> > > > > controller.
> > > > 
> > > > The special behavior which is common across many (not all) ARM
> > > > SoCs is that they don't synthesize all-one response for
> > > > completion timeouts, unlike RCs in x86 machines. Rather, they
> > > > return AXI error response, resulting in CPU treating them as
> > > > SError, in-addition to AER storm.
> > > > 
> > > > Commit message missed mentioning SError though.
> > > 
> > > I don't know how SError works, but this sounds like a pretty big
> > > open issue with respect to RAS.  I don't think we want a kernel
> > > panic because a device failed to respond to a config or MMIO
> > > access, e.g., if a card or Thunderbolt cable got unplugged.
> 
> I still don't know anything about arm64 or SError, but I see these KVM
> commits about using ESB to manage SError in some cases:
> 
>   0e5b9c085dce ("KVM: arm64: Consume pending SError as early as possible")
>   472fc011ccd3 ("KVM: arm64: nVHE: Don't consume host SErrors with ESB")
> 
> I assume it's not possible or practical to use ESB in the native host
> case to deal with issues like this in a generic way instead of the
> vendor-specific feature Qcom is using here?

In general, an SError is fatal. No ifs, no buts. It is asynchronous,
and has no syndrome information. Once you find out about it, you're
already dead.

ESB is a way to paper over this sorry state of affair by making sure
that the error is observed at a known point. You can think of it as a
barrier synchronising potential errors for transactions in flight. KVM
uses that to attribute SErrors triggered by a guest, on an exception
boundary.

That's not applicable to the kernel itself, unless you want to place
an ESB after each and every load/store in the kernel (/s).

Modern versions of the architecture allow these errors to be reported
as *synchronous* exceptions, with all the syndrome information you
want. I guess this HW doesn't implement it, which is a pretty big
mistake for systems that allow surprise removal of devices...

	M.

-- 
Without deviation from the norm, progress is not possible.