From: "Alexandre Courbot" <acourbot@nvidia.com>
To: "John Hubbard" <jhubbard@nvidia.com>
Cc: "Danilo Krummrich" <dakr@kernel.org>,
"Timur Tabi" <ttabi@nvidia.com>,
"Alistair Popple" <apopple@nvidia.com>,
"Eliot Courtney" <ecourtney@nvidia.com>,
"Zhi Wang" <zhiw@nvidia.com>, "David Airlie" <airlied@gmail.com>,
"Simona Vetter" <simona@ffwll.ch>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Miguel Ojeda" <ojeda@kernel.org>,
"Alex Gaynor" <alex.gaynor@gmail.com>,
"Boqun Feng" <boqun.feng@gmail.com>,
"Gary Guo" <gary@garyguo.net>,
"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
"Benno Lossin" <lossin@kernel.org>,
"Andreas Hindborg" <a.hindborg@kernel.org>,
"Alice Ryhl" <aliceryhl@google.com>,
"Trevor Gross" <tmgross@umich.edu>,
nova-gpu@lists.linux.dev, LKML <linux-kernel@vger.kernel.org>,
"Will Pierce" <wpierce@nvidia.com>
Subject: Re: [PATCH v3 12/14] gpu: nova-core: drive GSP events with the SWGEN0 interrupt
Date: Mon, 07 Sep 2026 15:59:23 +0900 [thread overview]
Message-ID: <DL8W1O8BW36T.2HVKESOX2KAWX@nvidia.com> (raw)
In-Reply-To: <20260903031514.1515905-13-jhubbard@nvidia.com>
On Thu Sep 3, 2026 at 12:15 PM JST, John Hubbard wrote:
> The GSP posts events, logs and error records to the GSP-to-CPU queue and
> raises the falcon SWGEN0 output. A falcon signals the interrupt tree
> only on a transition of the causes it routes to the host, and IRQSTAT
> also reports the causes the falcon keeps for its own RISC-V core. GSP
> boot polls for its own notifications, so it leaves the SWGEN0 latch set
> and leaves pending bits behind in the tree.
>
> nova-core drained the queue only while polling for a command reply, so
> an event sat unread until the next command was sent.
>
> Service the queue from a threaded handler on the GSP notification
> vector. The top half runs in hard interrupt context and touches only
> registers: it clears the GIN leaf, takes the causes pending for the
> host, writes INTR_RETRIGGER so that a cause arriving while the top half
> runs still signals the tree, and rearms PCI delivery. Draining the queue
> takes the command-queue mutex, which can sleep, so the top half wakes
> the IRQ thread to do it.
>
> Intersect IRQSTAT with the RISC-V routing registers the way Open RM
> does, so the firmware's own causes are left alone. Clear the latch of a
> host cause that is not a posted message, since nova-core has no recovery
> path for one and the retrigger would raise it again.
>
> Put the interrupt setup on the GPU rather than in the driver's probe.
> The handler is then torn down before the queue it drains is freed, and
> before the GSP is unloaded. Quiesce the tree and clear the latch before
> registering, so no boot state reaches the handler, and keep the subtree
> enabled at TOP for as long as the handler is registered. Quiescing
> disables the subtree, and under pre-Hopper MSI the rearm is a
> configuration-space write that never enables it again.
So this is a mishmash of many different things which makes it very
tedious to review. The falcon HAL stuff belongs in patch 11, the new
`SubtreeSet` method in patch 3, `irq.rs` changes where relevant, the
Cmdq::drain should be its own patch, and this patch should really just
add the handler and wire things together.
> Assisted-by: Cursor:claude-opus-5
On this revision the AI assistance showed mostly in the tedious comments
restating what the code does and the unneeded churn. These really take a
toll in terms of time and energy (and dare I say motivation). We need a
more thorough human pre-submit pass because otherwise the net effect is
a shift of labor onto reviewers, whose bandwidth is very limited.
<...>
> diff --git a/drivers/gpu/nova-core/falcon/hal.rs b/drivers/gpu/nova-core/falcon/hal.rs
> index 7e532889a1f4..5272b3b63ae4 100644
> --- a/drivers/gpu/nova-core/falcon/hal.rs
> +++ b/drivers/gpu/nova-core/falcon/hal.rs
> @@ -1,8 +1,15 @@
> // SPDX-License-Identifier: GPL-2.0
>
> -use kernel::prelude::*;
> +use kernel::{
> + io::{
> + register::WithBase,
> + Io, //
> + },
> + prelude::*, //
> +};
>
> use crate::{
> + driver::Bar0,
> falcon::{
> Falcon,
> FalconBromParams,
> @@ -12,6 +19,7 @@
> Architecture,
> Chipset, //
> },
> + regs,
> };
>
> mod ga102;
> @@ -72,6 +80,45 @@ fn signature_reg_fuse_version(
> fn load_method(&self) -> LoadMethod;
> }
>
> +/// Returns whether `chipset`'s falcons implement `NV_PFALCON_FALCON_INTR_RETRIGGER`.
> +///
> +/// Turing falcons do not. Ampere and later do, including GA100, whose falcon otherwise uses the
> +/// Turing HAL, so this is keyed on the architecture rather than provided through [`FalconHal`].
> +pub(crate) fn has_intr_retrigger(chipset: Chipset) -> bool {
> + !matches!(chipset.arch(), Architecture::Turing)
> +}
> +
> +/// Returns whether `chipset` carries the RISC-V interrupt routing registers at the Turing
> +/// offsets.
> +///
> +/// GA102 moved `NV_PRISCV_RISCV_IRQMASK` and `NV_PRISCV_RISCV_IRQDEST`, and GA100 kept the Turing
> +/// offsets, which is also why [`falcon_hal`] gives GA100 the Turing HAL.
> +fn has_turing_riscv_routing(chipset: Chipset) -> bool {
> + matches!(chipset.arch(), Architecture::Turing) || chipset == Chipset::GA100
> +}
> +
> +/// Returns the interrupt causes a RISC-V falcon on `chipset` routes to the host, in the layout of
> +/// `NV_PFALCON_FALCON_IRQSTAT`.
> +///
> +/// A cause reaches the host only if the RISC-V core both enables it and directs it there, which
> +/// `NV_PRISCV_RISCV_IRQMASK` and `NV_PRISCV_RISCV_IRQDEST` say. Every other latched cause belongs
> +/// to the firmware running on the core.
> +pub(crate) fn host_intr_routing<E: FalconEngine>(bar: Bar0<'_>, chipset: Chipset) -> u32 {
> + if has_turing_riscv_routing(chipset) {
> + bar.read(regs::tu102::NV_PRISCV_RISCV_IRQMASK::of::<E>())
> + .value()
> + & bar
> + .read(regs::tu102::NV_PRISCV_RISCV_IRQDEST::of::<E>())
> + .value()
> + } else {
> + bar.read(regs::ga102::NV_PRISCV_RISCV_IRQMASK::of::<E>())
> + .value()
> + & bar
> + .read(regs::ga102::NV_PRISCV_RISCV_IRQDEST::of::<E>())
> + .value()
> + }
> +}
> +
Why not use regular HAL methods here? This completely breaks the pattern
we introduced for HALs. If the current HALs don't fit the routing you
need, then we should introduce a new one.
next prev parent reply other threads:[~2026-09-07 6:59 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-03 3:14 [PATCH v3 00/14] nova-core: GPU interrupt support and GSP event delivery John Hubbard
2026-09-03 3:15 ` [PATCH v3 01/14] rust: pci: declare IrqType and IrqTypes with impl_flags John Hubbard
2026-09-03 3:15 ` [PATCH v3 02/14] rust: sync: completion: add wait_for_completion_timeout() John Hubbard
2026-09-03 3:15 ` [PATCH v3 03/14] gpu: nova-core: add the GIN vector and subtree newtypes John Hubbard
2026-09-05 1:39 ` Alexandre Courbot
2026-09-03 3:15 ` [PATCH v3 04/14] gpu: nova-core: add the GIN CPU interrupt tree and MSI EOI registers John Hubbard
2026-09-03 3:15 ` [PATCH v3 05/14] gpu: nova-core: add the per-architecture GIN CPU interrupt HAL John Hubbard
2026-09-05 6:11 ` Alexandre Courbot
2026-09-03 3:15 ` [PATCH v3 06/14] gpu: nova-core: add the GIN interrupt tree and allocate its vectors John Hubbard
2026-09-05 13:55 ` Alexandre Courbot
2026-09-06 23:10 ` John Hubbard
2026-09-07 0:24 ` Alexandre Courbot
2026-09-03 3:15 ` [PATCH v3 07/14] gpu: nova-core: add an interrupt delivery self-test John Hubbard
2026-09-03 3:29 ` sashiko-bot
2026-09-03 3:57 ` John Hubbard
2026-09-07 5:26 ` Alexandre Courbot
2026-09-03 3:15 ` [PATCH v3 08/14] gpu: nova-core: log GSP events instead of discarding them John Hubbard
2026-09-03 3:15 ` [PATCH v3 09/14] gpu: nova-core: recover the GSP receive path from corrupt framing John Hubbard
2026-09-04 10:53 ` Alexandre Courbot
2026-09-04 11:17 ` Gary Guo
2026-09-04 13:45 ` Alexandre Courbot
2026-09-03 3:15 ` [PATCH v3 10/14] gpu: nova-core: bound a GSP wait by a single deadline John Hubbard
2026-09-04 11:13 ` Alexandre Courbot
2026-09-04 11:26 ` Gary Guo
2026-09-04 13:32 ` Alexandre Courbot
2026-09-04 13:41 ` Gary Guo
2026-09-03 3:15 ` [PATCH v3 11/14] gpu: nova-core: add the falcon interrupt status and routing registers John Hubbard
2026-09-03 3:15 ` [PATCH v3 12/14] gpu: nova-core: drive GSP events with the SWGEN0 interrupt John Hubbard
2026-09-03 3:28 ` sashiko-bot
2026-09-03 3:55 ` John Hubbard
2026-09-04 1:53 ` John Hubbard
2026-09-07 6:59 ` Alexandre Courbot [this message]
2026-09-07 18:17 ` John Hubbard
2026-09-03 3:15 ` [PATCH v3 13/14] gpu: nova-core: add KUnit tests for the interrupt tree and HALs John Hubbard
2026-09-03 3:15 ` [PATCH v3 14/14] gpu: nova-core: document the GIN interrupt controller and GSP events John Hubbard
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DL8W1O8BW36T.2HVKESOX2KAWX@nvidia.com \
--to=acourbot@nvidia.com \
--cc=a.hindborg@kernel.org \
--cc=airlied@gmail.com \
--cc=alex.gaynor@gmail.com \
--cc=aliceryhl@google.com \
--cc=apopple@nvidia.com \
--cc=bhelgaas@google.com \
--cc=bjorn3_gh@protonmail.com \
--cc=boqun.feng@gmail.com \
--cc=dakr@kernel.org \
--cc=ecourtney@nvidia.com \
--cc=gary@garyguo.net \
--cc=jhubbard@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lossin@kernel.org \
--cc=nova-gpu@lists.linux.dev \
--cc=ojeda@kernel.org \
--cc=simona@ffwll.ch \
--cc=tmgross@umich.edu \
--cc=ttabi@nvidia.com \
--cc=wpierce@nvidia.com \
--cc=zhiw@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®