From: Mayank Rana <mayank.rana@oss.qualcomm.com>
To: Qiang Yu <qiang.yu@oss.qualcomm.com>, Bjorn Helgaas <helgaas@kernel.org>
Cc: "Manivannan Sadhasivam" <mani@kernel.org>,
"Lorenzo Pieralisi" <lpieralisi@kernel.org>,
"Krzysztof Wilczyński" <kwilczynski@kernel.org>,
"Rob Herring" <robh@kernel.org>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Konrad Dybcio" <konrad.dybcio@oss.qualcomm.com>,
linux-pci@vger.kernel.org, linux-arm-msm@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2] PCI: qcom: Block accesses to downstream devices on link down
Date: Wed, 30 Sep 2026 09:51:57 -0700 [thread overview]
Message-ID: <c1213157-076a-496c-870e-cf5469193600@oss.qualcomm.com> (raw)
In-Reply-To: <aqeTyRnmsEcChW8G@hu-qianyu-lv.qualcomm.com>
Hi Qiang
On 9/13/2026 11:27 PM, Qiang Yu wrote:
> On Fri, Sep 11, 2026 at 12:18:07PM -0500, Bjorn Helgaas wrote:
>> On Fri, Sep 11, 2026 at 08:17:29AM +0200, Manivannan Sadhasivam wrote:
>>> On Tue, Sep 08, 2026 at 06:00:30PM -0500, Bjorn Helgaas wrote:
>>>> On Wed, Aug 19, 2026 at 11:36:54PM -0700, Qiang Yu wrote:
>>>>> After a PCIe link goes down, software may still access the BAR
>>>>> (MMIO) space or configuration space of devices behind that link
>>>>> before recovery has run. As the link is down, these accesses
>>>>> never complete, resulting in a storm of Completion Timeout AERs.
>>>>
>>>> What is special about qcom here? It seems like the Completion
>>>> Timeouts and AER interrupts should happen with every PCIe
>>>> controller.
>>>
>>> The special behavior which is common across many (not all) ARM SoCs
>>> is that they don't synthesize all-one response for completion
>>> timeouts, unlike RCs in x86 machines. Rather, they return AXI error
>>> response, resulting in CPU treating them as SError, in-addition to
>>> AER storm.
>>>
>>> Commit message missed mentioning SError though.
>>
>> I don't know how SError works, but this sounds like a pretty big open
>> issue with respect to RAS. I don't think we want a kernel panic
>> because a device failed to respond to a config or MMIO access, e.g.,
>> if a card or Thunderbolt cable got unplugged.
>>
>>>> Is this mitigating an issue that will still happen on other
>>>> controllers and should be solved elsewhere, e.g., by changing the
>>>> software that accesses the BAR to look for the error responses it
>>>> gets when the Completion Timeout happens?
>>>
>>> That would be too late as we don't get a proper error response.
>>> That's why this patch is used.
>>>
>>>> The patch refers to the ECAM blocker (which I assume affects
>>>> config accesses) and doesn't mention MMIO. Is the SLV_AXI stuff
>>>> for MMIO?
>>>
>>> ECAM blocker is a Qcom's custom implementation which prevents the
>>> config access to go out of the link and terminate the request
>>> properly. And yes, it doesn't affect MMIO like BAR.
>>
>> The commit log implies that the patch blocks both config and MMIO
>> accesses: "software may access MMIO or config space ... Use ECAM
>> blocker to drop these accesses." If the ECAM blocker only affects
>> config accesses, we need to reword that description.
>>
>
> This patch can block both config space and MMIO accesses.
To avoid confusion, can you rename APIs by replacing ecam by outbound
i.e.
qcom_pcie_init_ecam_blocker() as qcom_pcie_init_outboud_blocker() and
qcom_pcie_enable_ecam_blocker() as qcom_pcie_enable_outbound_blocker().
>> If this patch doesn't affect MMIO accesses, I assume they will still
>> fail and lead to SError? I'm a little dubious about the utility of
>> this because I assume most runtime driver accesses will be to MMIO
>> space, not config space.
>>
>> I assume this is also racy: any config access that happens in the
>> window between the link going down and the ECAM blocker being enabled
>> will still fail and result in SError, right?
>>
>
> Yes, there is a window between link going down and the ECAM blocker being
> enabled, this is why I mentioned this patch can *reduce* the completion
> timeout errors in commit msg.
>
> - Qiang Yu
>>>> Assume the root port leads to a switch, and the switch has two
>>>> links to downstream devices. If one of the switch downstream
>>>> links goes down, is this going to kill the device on the other
>>>> link that is still up?
>>>
>>> ECAM blocker is only enabled when the PCIe link to the Root Port
>>> goes down. We don't get LDn interrupt when the switch downstream
>>> link goes down. In that case, the switch should synthesize the error
>>> response if the CPU tries to access the unavailable device.
>>
>> For config or MMIO reads targeting a device below a switch, if the
>> link to the device is down, I think the switch will return an
>> Unsupported Request completion without data (PCIe r7.0, sec 2.3.1.1).
>>
>> The PCIe spec doesn't prescribe whether RC synthesizes data to
>> complete the read, but x86 RCs typically synthesize ~0 data. I guess
>> some arm64 RCs assert SError instead of completing the read.
>>
>> IIUC, if the link from the RP goes down, we'll take the LDn interrupt,
>> enable ECAM blocker, and future config reads would return ~0. But if
>> a link below a switch goes down, there's no LDn interrupt and it
>> sounds like future config reads would result in a UR Cpl and SError.
>
Regards,
Mayank
next prev parent reply other threads:[~2026-09-30 16:52 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-20 6:36 Qiang Yu
2026-08-20 11:29 ` Konrad Dybcio
2026-09-01 12:35 ` Manivannan Sadhasivam
2026-09-01 12:39 ` Konrad Dybcio
2026-09-01 14:06 ` Xilin Wu
2026-09-03 7:50 ` Krishna Chaitanya Chundru
2026-09-08 23:00 ` Bjorn Helgaas
2026-09-11 6:17 ` Manivannan Sadhasivam
2026-09-11 17:18 ` Bjorn Helgaas
2026-09-14 6:27 ` Qiang Yu
2026-09-30 16:51 ` Mayank Rana [this message]
2026-10-01 14:03 ` Qiang Yu
2026-09-15 17:16 ` Manivannan Sadhasivam
2026-09-15 23:14 ` Bjorn Helgaas
2026-09-16 8:03 ` Marc Zyngier
2026-09-16 16:36 ` Bjorn Helgaas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c1213157-076a-496c-870e-cf5469193600@oss.qualcomm.com \
--to=mayank.rana@oss.qualcomm.com \
--cc=bhelgaas@google.com \
--cc=helgaas@kernel.org \
--cc=konrad.dybcio@oss.qualcomm.com \
--cc=kwilczynski@kernel.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=lpieralisi@kernel.org \
--cc=mani@kernel.org \
--cc=qiang.yu@oss.qualcomm.com \
--cc=robh@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®