From: Nicolin Chen <nicolinc@nvidia.com>
To: Pranjal Shrivastava <praan@google.com>
Cc: <iommu@lists.linux.dev>, Will Deacon <will@kernel.org>,
Joerg Roedel <joro@8bytes.org>,
Robin Murphy <robin.murphy@arm.com>,
Jason Gunthorpe <jgg@ziepe.ca>,
Mostafa Saleh <smostafa@google.com>,
Daniel Mentz <danielmentz@google.com>,
Ashish Mhetre <amhetre@nvidia.com>,
<linux-arm-kernel@lists.infradead.org>,
Thomas Gleixner <tglx@kernel.org>, Radu Rendec <radu@rendec.net>,
Bjorn Helgaas <bhelgaas@google.com>, <linux-pci@vger.kernel.org>,
<linux-kernel@vger.kernel.org>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
<rafael@kernel.org>, "Danilo Krummrich" <dakr@kernel.org>,
<driver-core@lists.linux.dev>
Subject: Re: [PATCH v11 11/16] iommu/arm-smmu-v3: Add CMDQ_PROD_STOP_FLAG to gate CMDQ submissions
Date: Thu, 1 Oct 2026 11:03:15 -0700 [thread overview]
Message-ID: <ar6gY9grmdMZcN/M@nvidia.com> (raw)
In-Reply-To: <ar3yWUmi8E0k37Ml@google.com>
On Thu, Oct 01, 2026 at 05:40:41AM +0000, Pranjal Shrivastava wrote:
> On Wed, Sep 30, 2026 at 01:33:58PM -0700, Nicolin Chen wrote:
> > We may get away from the TLB maintenance. But ATC can be the case
> > broken by the stop flag?
> >
>
> Not really, I guess Sashiko missed the comment in a later patch. This
> was discussed with Jason in v9 [1].
>
> The key thing is that ATC_INVs can't be issued by the time we set this
> flag anyway.. if we're in the suspend callback, the PCIe EP is already
> suspended, so none of the ATC invalidation TLPs would be responded to.
Okay.
> I'd addressed this in the RPM patch with a comment right above this check:
>
> * Note that eliding ATC invalidations (CMDQ_OP_ATC_INV) is safe
> * because client PCIe endpoints are guaranteed to be suspended
> * (via device links) before the SMMU is suspended. With the PCIe
> * links in a low-power state, no new TLPs can be transmitted.
> * It is strictly the responsibility of the client/endpoint driver
> * to quiesce DMA and ensure that the ATC state is cleared across
> * power state transitions.
>
> i.e. we only start eliding once *all* consumers are RPM suspended, so
> the "endpoint in D0" scenario can't happen.. and a suspended PCIe
> function shouldn't be issuing translated requests anyway.
Maybe move it to this PATCH-11?
> So we can't issue ATC_INVs during suspend (the EP is already down), nor
> during resume (the SMMU resumes *before* the EP is made active). The EP
> can't use its ATC while suspended, and if it loses power/resets on the
> way back to D0 (from D3cold, or D3hot with No_Soft_Reset=0), it comes
> back with an empty ATC.. same assumption the PCI reset path makes today
> (pci_dev_reset_iommu_prepare()).
In that case, would the STOP flag be too late? It's only set in
the middle of the SMMU suspend. So, an ATC command (via doamin
invalidation) might be issued prior to the Point of Commitment,
which will be timed out due to the unresponding EP?
Nicolin
next prev parent reply other threads:[~2026-10-01 18:03 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-29 3:44 [PATCH v11 00/16] iommu/arm-smmu-v3: Implement Runtime/System Sleep ops Pranjal Shrivastava
2026-09-29 3:44 ` [PATCH v11 01/16] iommu/arm-smmu-v3: Refactor arm_smmu_setup_irqs Pranjal Shrivastava
2026-09-29 3:44 ` [PATCH v11 02/16] iommu/arm-smmu-v3: Add Q_POS() macro Pranjal Shrivastava
2026-09-29 3:44 ` [PATCH v11 03/16] iommu/arm-smmu-v3: Add arm_smmu_drain_queue() helper Pranjal Shrivastava
2026-09-30 18:24 ` Nicolin Chen
2026-09-30 20:17 ` Pranjal Shrivastava
2026-09-29 3:44 ` [PATCH v11 04/16] iommu/tegra241-cmdqv: Add a helper to drain VCMDQs Pranjal Shrivastava
2026-09-29 3:44 ` [PATCH v11 05/16] iommu/arm-smmu-v3: Add a helper to drain cmd queues Pranjal Shrivastava
2026-09-29 3:45 ` [PATCH v11 06/16] iommu/tegra241-cmdqv: Restore PROD and CONS after resume Pranjal Shrivastava
2026-09-29 3:45 ` [PATCH v11 07/16] genirq/msi: Cache MSI message in irq_chip_write_msi_msg() Pranjal Shrivastava
2026-09-29 3:45 ` [PATCH v11 08/16] genirq/msi: Provide msi_device_domain_restore_msi_msgs() Pranjal Shrivastava
2026-09-29 3:45 ` [PATCH v11 09/16] iommu/arm-smmu-v3: Restore MSI config on resume Pranjal Shrivastava
2026-09-29 3:45 ` [PATCH v11 10/16] iommu/arm-smmu-v3: Factor out arm_smmu_handle_gerror() Pranjal Shrivastava
2026-09-30 18:34 ` Nicolin Chen
2026-09-30 20:00 ` Pranjal Shrivastava
2026-09-30 20:12 ` Nicolin Chen
2026-09-30 20:03 ` Pranjal Shrivastava
2026-09-29 3:45 ` [PATCH v11 11/16] iommu/arm-smmu-v3: Add CMDQ_PROD_STOP_FLAG to gate CMDQ submissions Pranjal Shrivastava
2026-09-30 20:33 ` Nicolin Chen
2026-10-01 5:40 ` Pranjal Shrivastava
2026-10-01 18:03 ` Nicolin Chen [this message]
2026-09-29 3:45 ` [PATCH v11 12/16] iommu/tegra241-cmdqv: Add a helper to quiesce VCMDQs Pranjal Shrivastava
2026-09-30 19:02 ` Nicolin Chen
2026-09-30 19:57 ` Pranjal Shrivastava
2026-09-30 20:03 ` Nicolin Chen
2026-09-29 3:45 ` [PATCH v11 13/16] iommu/arm-smmu-v3: Implement pm_runtime & system sleep ops Pranjal Shrivastava
2026-10-01 20:22 ` Nicolin Chen
2026-09-29 3:45 ` [PATCH v11 14/16] iommu/arm-smmu-v3: Enable pm_runtime and setup devlinks Pranjal Shrivastava
2026-09-29 3:45 ` [PATCH v11 15/16] iommu/arm-smmu-v3: Invoke pm_runtime before hw access Pranjal Shrivastava
2026-09-29 3:45 ` [PATCH v11 16/16] iommu/arm-smmu-v3: Add KUnit unit tests for Runtime PM Pranjal Shrivastava
2026-10-01 19:12 ` Nicolin Chen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ar6gY9grmdMZcN/M@nvidia.com \
--to=nicolinc@nvidia.com \
--cc=amhetre@nvidia.com \
--cc=bhelgaas@google.com \
--cc=dakr@kernel.org \
--cc=danielmentz@google.com \
--cc=driver-core@lists.linux.dev \
--cc=gregkh@linuxfoundation.org \
--cc=iommu@lists.linux.dev \
--cc=jgg@ziepe.ca \
--cc=joro@8bytes.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=praan@google.com \
--cc=radu@rendec.net \
--cc=rafael@kernel.org \
--cc=robin.murphy@arm.com \
--cc=smostafa@google.com \
--cc=tglx@kernel.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®