Re: [PATCH v2 00/15] nova-core: GPU interrupt support and GSP event delivery

[John Hubbard <jhubbard@nvidia.com>]

On 8/28/26 6:22 PM, John Hubbard wrote:
> I'm posting a v2 because there have been some significant changes, as a
> result of the v1 review, plus Danilo's new IRQ commits that I've rebased
> onto.

There was a failure in git-send-email part way through. I'll attempt to
resend, once I figure out what went wrong.

Only the cover letter and the first 4 patches got sent.

thanks,
-- 
John Hubbard
> 
> This series adds support for GIN, the GPU Interrupt and Notification
> unit, which is the GPU's interrupt controller, so that GSP events reach
> the driver as interrupts instead of only when the driver polls.
> 
> The design uses a threaded IRQ handler. The top half touches only
> GPU registers, while the threaded bottom half drains the message queue.
> 
> Fine-grained locking is left for a follow-up patchset, I'm working on
> that next. For now, there is just a big ugly lock around anything that
> even gets close to the GSP message queue. :)
> 
> This is based on drm-rust-next, plus the six commits of Danilo
> Krummrich's PCI interrupt-vector series [1], which I've cherry-picked
> from mainline for now.
> 
> There is a git branch with the patches as applied to drm-rust-next:
> 
>     https://github.com/johnhubbard/linux/tree/nova-core-gin-interrupt-tree-v2/
> 
> Changes in v2:
> 
> * Rebased onto current drm-rust-next. The PCI interrupt-vector rework
>   that v1 patches 2 and 3 proposed is in mainline now as Danilo's series
>   [1], so both are dropped. This series carries those six commits as
>   prerequisites until drm-rust-next picks them up.
> 
> * nova-core uses the merged API: the driver's device data owns the
>   vector allocation under an explicit lifetime rather than through
>   devres, and each handler takes an IrqRequest for its own subtree's
>   vector.
> 
> * Dropped "allocate PCI MSI vector during probe" (v1 patch 4). What it
>   added is replaced by patch 6, and its commit message justified MSI
>   with a VFIO claim that does not hold. nova-core still allocates MSI-X
>   or MSI, and no longer falls back to INTx. (Danilo)
> 
> * Dropped the type-invariant documentation and the SAFETY rewrites from
>   the wait_for_completion_timeout() patch, leaving only the new method
>   itself. (Alexandre)
> 
> * New: declare pci::IrqType and IrqTypes with impl_flags, so a call site
>   reads IrqType::MsiX | IrqType::Msi. (Gary)
> 
> * New: the GIN vector and subtree newtypes. A vector, a leaf index, a
>   set of vectors within one leaf, one subtree, a set of subtrees, and a
>   leaf count are separate types now, so a leaf mask cannot be passed
>   where a TOP bit belongs. The HAL returns a LeafCount rather than a
>   usize. GSP_LEAF and GSP_BIT are gone, along with the
>   LeafIndex::new::<GSP_LEAF>() calls. (Danilo)
> 
> * Merged the tree API patch into the vector allocation patch, and moved
>   both after the HAL. Tree owns the BAR mapping, so no tree method takes
>   a bar argument. The Leaf<Idle>/Leaf<Pending> type state gives way to a
>   LeafPending newtype that only Tree::read_pending hands out, and enable
>   and disable are Tree methods. (Danilo)
> 
> * Added LeafEnableGuard and TopEnableGuard. The self-test's teardown
>   guard and GspIrq's open-coded destructor are both gone, and probe no
>   longer needs a separate interrupt-enable step. (Danilo)
> 
> * Moved the GIN and MSI EOI register definitions to irq/regs.rs.
>   (Danilo)
> 
> * 13 KUnit tests rather than 14. The tree tests now cover the newtypes,
>   and testing those needs no BAR mapping. One test went away because a
>   leaf count derives its subtree set by construction.
> 
> Tested on Turing, Ampere, Blackwell GPUs.
> 
> One known gap: driver_read_area still reads the GSP producer pointer
> with no acquire barrier. Gary Guo's barrier series puts dma_mb(Read) at
> exactly that point [2], so let's just wait for his fix to land.
> 
> [1] https://lore.kernel.org/all/20260813165234.620555-1-dakr@kernel.org/
> [2] https://lore.kernel.org/all/20260609-rust-barrier-v2-4-30fcc48e1cd0@garyguo.net/
> 
> 
> Joel Fernandes (2):
>   rust: sync: completion: add wait_for_completion_timeout()
>   gpu: nova-core: add the GIN interrupt tree and allocate its vectors
> 
> John Hubbard (13):
>   rust: pci: declare IrqType and IrqTypes with impl_flags
>   gpu: nova-core: add the GIN CPU interrupt tree and MSI EOI registers
>   gpu: nova-core: add the GIN vector and subtree newtypes
>   gpu: nova-core: add the per-architecture GIN CPU interrupt HAL
>   gpu: nova-core: add an interrupt delivery self-test
>   gpu: nova-core: dispatch GSP events instead of discarding them
>   gpu: nova-core: match GSP RPC replies by sequence, not just function
>   gpu: nova-core: recover the GSP receive path from corrupt framing
>   gpu: nova-core: bound a GSP wait by a single deadline
>   gpu: nova-core: drive GSP events with the SWGEN0 interrupt
>   gpu: nova-core: retrigger the GSP falcon and clear every latched cause
>   gpu: nova-core: add KUnit tests for the interrupt tree and HALs
>   gpu: nova-core: document the GIN interrupt controller and GSP events
> 
>  Documentation/gpu/nova/core/interrupts.rst  | 686 ++++++++++++++++++++
>  Documentation/gpu/nova/index.rst            |   1 +
>  drivers/gpu/nova-core/Kconfig               |  15 +
>  drivers/gpu/nova-core/driver.rs             |  55 +-
>  drivers/gpu/nova-core/falcon/gsp.rs         |  71 +-
>  drivers/gpu/nova-core/falcon/hal.rs         |  32 +
>  drivers/gpu/nova-core/gpu.rs                |  28 +-
>  drivers/gpu/nova-core/gsp.rs                |  17 +-
>  drivers/gpu/nova-core/gsp/cmdq.rs           | 286 ++++++--
>  drivers/gpu/nova-core/gsp/commands.rs       |   8 +-
>  drivers/gpu/nova-core/gsp/fw.rs             |  13 +-
>  drivers/gpu/nova-core/gsp/sequencer.rs      |   8 +-
>  drivers/gpu/nova-core/irq.rs                | 105 +++
>  drivers/gpu/nova-core/irq/doorbell_test.rs  | 298 +++++++++
>  drivers/gpu/nova-core/irq/gsp.rs            | 232 +++++++
>  drivers/gpu/nova-core/irq/hal.rs            | 192 ++++++
>  drivers/gpu/nova-core/irq/hal/gh100.rs      |  31 +
>  drivers/gpu/nova-core/irq/hal/tu102.rs      |  30 +
>  drivers/gpu/nova-core/irq/interrupt_tree.rs | 615 ++++++++++++++++++
>  drivers/gpu/nova-core/irq/regs.rs           |  71 ++
>  drivers/gpu/nova-core/nova_core.rs          |   1 +
>  drivers/gpu/nova-core/regs.rs               |  24 +
>  rust/kernel/pci/irq.rs                      |  68 +-
>  rust/kernel/sync/completion.rs              |  23 +-
>  24 files changed, 2767 insertions(+), 143 deletions(-)
>  create mode 100644 Documentation/gpu/nova/core/interrupts.rst
>  create mode 100644 drivers/gpu/nova-core/irq.rs
>  create mode 100644 drivers/gpu/nova-core/irq/doorbell_test.rs
>  create mode 100644 drivers/gpu/nova-core/irq/gsp.rs
>  create mode 100644 drivers/gpu/nova-core/irq/hal.rs
>  create mode 100644 drivers/gpu/nova-core/irq/hal/gh100.rs
>  create mode 100644 drivers/gpu/nova-core/irq/hal/tu102.rs
>  create mode 100644 drivers/gpu/nova-core/irq/interrupt_tree.rs
>  create mode 100644 drivers/gpu/nova-core/irq/regs.rs
>