* [PATCH v3 01/33] rust: pci: add domain_nr() accessor
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:06 ` [PATCH v3 02/33] gpu: nova-core: set MCTP transport header version to 1 John Hubbard
` (31 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The nova-core driver has to send the GPU's PCI location to the GPU's
firmware as one word that holds the domain, the bus and the device
number.
The PCI abstraction has dev_id(), which packs the bus, the device and
the function into one value. It has no accessor that reports the domain.
Add an accessor for the domain number. pci_domain_nr() is a static
inline, so it needs a C helper. The bus of a bound device always has a
domain assigned, so a negative value would be a kernel bug rather than a
condition to report. A debug assertion checks for it before the cast to
u32, and the accessor returns no error.
Suggested-by: Miguel Ojeda <ojeda@kernel.org>
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
rust/helpers/pci.c | 5 +++++
rust/kernel/pci.rs | 12 ++++++++++++
2 files changed, 17 insertions(+)
diff --git a/rust/helpers/pci.c b/rust/helpers/pci.c
index 3686e405160d..53cdc7f36fef 100644
--- a/rust/helpers/pci.c
+++ b/rust/helpers/pci.c
@@ -7,6 +7,11 @@ __rust_helper u16 rust_helper_pci_dev_id(struct pci_dev *dev)
return PCI_DEVID(dev->bus->number, dev->devfn);
}
+__rust_helper int rust_helper_pci_domain_nr(struct pci_dev *dev)
+{
+ return pci_domain_nr(dev->bus);
+}
+
__rust_helper resource_size_t
rust_helper_pci_resource_start(struct pci_dev *pdev, int bar)
{
diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
index 19a219847c17..8d379a2d7c71 100644
--- a/rust/kernel/pci.rs
+++ b/rust/kernel/pci.rs
@@ -400,6 +400,18 @@ pub fn dev_id(&self) -> u16 {
unsafe { bindings::pci_dev_id(self.as_raw()) }
}
+ /// Returns the PCI domain number of the bus that this device is on.
+ #[inline]
+ pub fn domain_nr(&self) -> u32 {
+ // SAFETY: By its type invariant `self.as_raw` is always a valid pointer to a
+ // `struct pci_dev`.
+ let domain_nr = unsafe { bindings::pci_domain_nr(self.as_raw()) };
+ debug_assert!(domain_nr >= 0);
+
+ // CAST: `pci_domain_nr` returns a non-negative domain number in an `int`.
+ domain_nr as u32
+ }
+
/// Returns the PCI subsystem vendor ID.
#[inline]
pub fn subsystem_vendor_id(&self) -> u16 {
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 02/33] gpu: nova-core: set MCTP transport header version to 1
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
2026-09-18 1:06 ` [PATCH v3 01/33] rust: pci: add domain_nr() accessor John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:06 ` [PATCH v3 03/33] gpu: nova-core: gsp: give the command queue its own BAR0 mapping John Hubbard
` (30 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
FSP, the GPU's Foundation Security Processor, receives the driver's
messages as packets of MCTP, the Management Component Transport
Protocol. It requires header version 1 in bits 3:0 of the MCTP transport
header.
Nova-core left those bits at zero, so every Chain of Trust request and
every Product Reconfiguration Control request that it sent carried an
invalid transport header.
Declare the version field and set it. Declare the destination endpoint
ID field as well. The driver leaves that field at zero.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
Documentation/gpu/nova/core/fsp.rst | 2 ++
drivers/gpu/nova-core/mctp.rs | 15 +++++++++++++--
2 files changed, 15 insertions(+), 2 deletions(-)
diff --git a/Documentation/gpu/nova/core/fsp.rst b/Documentation/gpu/nova/core/fsp.rst
index 52d618d22bb8..60c647181da0 100644
--- a/Documentation/gpu/nova/core/fsp.rst
+++ b/Documentation/gpu/nova/core/fsp.rst
@@ -106,6 +106,8 @@ All FSP messages share a common header format consisting of two 32-bit words:
- Bit 30: EOM (End of Message)
- Bits 29:28: Packet sequence number
- Bits 23:16: Source Endpoint ID
+- Bits 15:8: Destination Endpoint ID
+- Bits 3:0: MCTP header version (1)
**NVDM header** (NVIDIA Vendor Defined Message):
diff --git a/drivers/gpu/nova-core/mctp.rs b/drivers/gpu/nova-core/mctp.rs
index 90c642c91a72..a3872a740233 100644
--- a/drivers/gpu/nova-core/mctp.rs
+++ b/drivers/gpu/nova-core/mctp.rs
@@ -42,13 +42,24 @@ pub(crate) struct MctpHeader(u32) {
29:28 seq;
/// Source endpoint ID.
23:16 seid;
+ /// Destination endpoint ID.
+ 15:8 deid;
+ /// MCTP header version.
+ 3:0 version;
}
}
impl MctpHeader {
- /// Builds a single-packet MCTP header (`SOM=1`, `EOM=1`, `SEQ=0`, `SEID=0`).
+ /// The MCTP header version that this driver uses.
+ const VERSION: u32 = 1;
+
+ /// Builds the MCTP header of a message that fits in one packet: `SOM` and `EOM` set, the
+ /// version set, and every other field zero.
pub(crate) fn single_packet() -> Self {
- Self::zeroed().with_som(true).with_eom(true)
+ Self::zeroed()
+ .with_const_version::<{ Self::VERSION }>()
+ .with_som(true)
+ .with_eom(true)
}
/// Returns whether this is a complete single-packet message (`SOM=1` and `EOM=1`).
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 03/33] gpu: nova-core: gsp: give the command queue its own BAR0 mapping
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
2026-09-18 1:06 ` [PATCH v3 01/33] rust: pci: add domain_nr() accessor John Hubbard
2026-09-18 1:06 ` [PATCH v3 02/33] gpu: nova-core: set MCTP transport header version to 1 John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:06 ` [PATCH v3 04/33] gpu: nova-core: firmware: add r000 bindings John Hubbard
` (29 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
Sending a command to the GSP ends by writing the GSP doorbell, a BAR0
register, so every send path through the command queue needs the BAR0
mapping. A command too large for one queue element goes out as a first
element followed by continuation records, so the send path has four
functions.
Nova-core passed the mapping down from the caller as an argument, which
was threaded through all four functions, and every caller of a send had
to carry it as well.
Store the mapping in the command queue and drop the argument. The queue
lives exactly as long as the mapping does, since both belong to the
bound device.
Suggested-by: Alexandre Courbot <acourbot@nvidia.com>
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gpu.rs | 4 ++--
drivers/gpu/nova-core/gsp.rs | 11 +++++++----
drivers/gpu/nova-core/gsp/boot.rs | 10 +++-------
drivers/gpu/nova-core/gsp/cmdq.rs | 24 ++++++++++++++----------
4 files changed, 26 insertions(+), 23 deletions(-)
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index d1e0da7b8682..9e0570e97cfe 100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -414,7 +414,7 @@ pub(crate) fn new<'a>(
vgpu: VgpuManager::new(pdev, spec.chipset, fsp.as_mut()),
- gsp <- Gsp::new(pdev),
+ gsp <- Gsp::new(pdev, bar),
// This member must be initialized last, so the `UnloadBundle` can never be dropped
// from outside of the constructed `GspResources`, ensuring that the unload sequence
@@ -456,7 +456,7 @@ pub(crate) fn new<'a>(
gsp_static_info: {
// Obtain and display basic GPU information.
- let info = gsp_resources.gsp.get_static_info(bar)?;
+ let info = gsp_resources.gsp.get_static_info()?;
match info.gpu_name() {
Ok(name) => dev_info!(dev, "GPU name: {}\n", name),
Err(e) => dev_warn!(dev, "GPU name unavailable: {:?}\n", e),
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index b85e0b6475f0..c9b7498639b7 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -162,7 +162,10 @@ pub(crate) struct Gsp<'gsp> {
impl<'gsp> Gsp<'gsp> {
// Creates an in-place initializer for a `Gsp` manager for `pdev`.
- pub(crate) fn new(pdev: &'gsp pci::Device<device::Bound>) -> impl PinInit<Self, Error> + 'gsp {
+ pub(crate) fn new(
+ pdev: &'gsp pci::Device<device::Bound>,
+ bar: Bar0<'gsp>,
+ ) -> impl PinInit<Self, Error> + 'gsp {
pin_init::pin_init_scope(move || {
let dev = pdev.as_ref();
@@ -174,7 +177,7 @@ pub(crate) fn new(pdev: &'gsp pci::Device<device::Bound>) -> impl PinInit<Self,
// _kgspInitLibosLoggingStructures (allocates memory for buffers)
// kgspSetupLibosInitArgs_IMPL (creates pLibosInitArgs[] array)
Ok(try_pin_init!(Self {
- cmdq <- Cmdq::new(dev),
+ cmdq <- Cmdq::new(dev, bar),
rmargs: Coherent::init(dev, GFP_KERNEL, GspArgumentsPadded::new(&cmdq))?,
libos: {
let mut libos = CoherentBox::zeroed_slice(
@@ -218,8 +221,8 @@ pub(crate) fn new(pdev: &'gsp pci::Device<device::Bound>) -> impl PinInit<Self,
}
/// Query the GSP for the static GPU information.
- pub(crate) fn get_static_info(&self, bar: Bar0<'_>) -> Result<commands::GetGspStaticInfoReply> {
- self.cmdq.send_command(bar, commands::GetGspStaticInfo)
+ pub(crate) fn get_static_info(&self) -> Result<commands::GetGspStaticInfoReply> {
+ self.cmdq.send_command(commands::GetGspStaticInfo)
}
}
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index 60bed3dc2f5a..4fb1b69ac9d5 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -10,7 +10,6 @@
};
use crate::{
- driver::Bar0,
falcon::{
gsp::Gsp,
Falcon, //
@@ -36,7 +35,6 @@ pub(crate) fn boot(
mut ctx: super::GspBootContext<'_, 'gsp>,
) -> Result<Option<super::UnloadBundle<'gsp>>> {
let pdev = ctx.pdev;
- let bar = ctx.bar;
let chipset = ctx.chipset;
let gsp_falcon = ctx.gsp_falcon;
let dev = pdev.as_ref();
@@ -45,9 +43,9 @@ pub(crate) fn boot(
let gsp_fw = KBox::pin_init(GspFirmware::new(dev, chipset), GFP_KERNEL)?;
self.cmdq
- .send_command_no_wait(bar, commands::SetSystemInfo::new(pdev, chipset))?;
+ .send_command_no_wait(commands::SetSystemInfo::new(pdev, chipset))?;
self.cmdq
- .send_command_no_wait(bar, commands::SetRegistry::new(ctx.vgpu.state())?)?;
+ .send_command_no_wait(commands::SetRegistry::new(ctx.vgpu.state())?)?;
// Perform the chipset-specific boot sequence, and retrieve the unload bundle.
let unload_bundle = hal.boot(&self, &mut ctx, &gsp_fw)?.or_else(|| {
@@ -89,12 +87,11 @@ pub(crate) fn boot(
/// Shut down the GSP and wait until it is offline.
fn shutdown_gsp(
cmdq: &Cmdq<'_>,
- bar: Bar0<'_>,
gsp_falcon: &Falcon<'_, Gsp>,
mode: commands::PowerStateLevel,
) -> Result {
// Command to shut the GSP down.
- cmdq.send_command(bar, commands::UnloadingGuestDriver::new(mode))?;
+ cmdq.send_command(commands::UnloadingGuestDriver::new(mode))?;
// Wait until GSP signals it is suspended.
const LIBOS_INTERRUPT_PROCESSOR_SUSPENDED: u32 = bits::bit_u32(31);
@@ -120,7 +117,6 @@ pub(crate) fn unload(
// Shut down the GSP. Keep going even in case of error.
let mut res = Self::shutdown_gsp(
&self.cmdq,
- ctx.bar,
ctx.gsp_falcon,
commands::PowerStateLevel::Level0,
)
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index f1231569aa33..9250d596a3e4 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -532,6 +532,7 @@ impl<'cmdq> Cmdq<'cmdq> {
/// Creates a new command queue for `dev`.
pub(crate) fn new(
dev: &'cmdq device::Device<device::Bound>,
+ bar: Bar0<'cmdq>,
) -> impl PinInit<Self, Error> + 'cmdq {
pin_init_scope(move || {
let gsp_mem = DmaGspMem::new(dev)?;
@@ -540,6 +541,7 @@ pub(crate) fn new(
dma_addr: gsp_mem.0.dma_address(),
inner <- new_mutex!(CmdqInner {
dev,
+ bar,
gsp_mem,
seq: 0,
poisoned: Cell::new(false),
@@ -582,7 +584,7 @@ fn notify_gsp(bar: Bar0<'_>) {
/// written to by its [`CommandToGsp::init_variable_payload`] method.
///
/// Error codes returned by the command and reply initializers are propagated as-is.
- pub(crate) fn send_command<M>(&self, bar: Bar0<'_>, command: M) -> Result<M::Reply>
+ pub(crate) fn send_command<M>(&self, command: M) -> Result<M::Reply>
where
M: CommandToGsp,
M::Reply: MessageFromGsp,
@@ -590,7 +592,7 @@ pub(crate) fn send_command<M>(&self, bar: Bar0<'_>, command: M) -> Result<M::Rep
Error: From<<M::Reply as MessageFromGsp>::InitError>,
{
let mut inner = self.inner.lock();
- inner.send_command(bar, command)?;
+ inner.send_command(command)?;
inner.await_msg()
}
@@ -604,12 +606,12 @@ pub(crate) fn send_command<M>(&self, bar: Bar0<'_>, command: M) -> Result<M::Rep
/// written to by its [`CommandToGsp::init_variable_payload`] method.
///
/// Error codes returned by the command initializers are propagated as-is.
- pub(crate) fn send_command_no_wait<M>(&self, bar: Bar0<'_>, command: M) -> Result
+ pub(crate) fn send_command_no_wait<M>(&self, command: M) -> Result
where
M: CommandToGsp<Reply = NoReply>,
Error: From<M::InitError>,
{
- self.inner.lock().send_command(bar, command)
+ self.inner.lock().send_command(command)
}
/// Waits for an unsolicited GSP event of type `M`. Events that arrive before it are logged and
@@ -652,6 +654,8 @@ pub(crate) fn drain(&self) -> Result {
struct CmdqInner<'a> {
/// Device this command queue belongs to.
dev: &'a device::Device,
+ /// MMIO mapping of PCI BAR0, for writing the GSP doorbell.
+ bar: Bar0<'a>,
/// Current command sequence number.
seq: u32,
/// Set once a message fails framing or checksum validation. Every later receive fails, since
@@ -678,7 +682,7 @@ impl CmdqInner<'_> {
/// written to by its [`CommandToGsp::init_variable_payload`] method.
///
/// Error codes returned by the command initializers are propagated as-is.
- fn send_single_command<M>(&mut self, bar: Bar0<'_>, command: M) -> Result
+ fn send_single_command<M>(&mut self, command: M) -> Result
where
M: CommandToGsp,
// This allows all error types, including `Infallible`, to be used for `M::InitError`.
@@ -732,7 +736,7 @@ fn send_single_command<M>(&mut self, bar: Bar0<'_>, command: M) -> Result
let elem_count = dst.header.element_count();
self.seq += 1;
self.gsp_mem.advance_cpu_write_ptr(elem_count);
- Cmdq::notify_gsp(bar);
+ Cmdq::notify_gsp(self.bar);
Ok(())
}
@@ -748,19 +752,19 @@ fn send_single_command<M>(&mut self, bar: Bar0<'_>, command: M) -> Result
/// written to by its [`CommandToGsp::init_variable_payload`] method.
///
/// Error codes returned by the command initializers are propagated as-is.
- fn send_command<M>(&mut self, bar: Bar0<'_>, command: M) -> Result
+ fn send_command<M>(&mut self, command: M) -> Result
where
M: CommandToGsp,
Error: From<M::InitError>,
{
match SplitState::new(command)? {
- SplitState::Single(command) => self.send_single_command(bar, command),
+ SplitState::Single(command) => self.send_single_command(command),
SplitState::Split(command, mut continuations) => {
- self.send_single_command(bar, command)?;
+ self.send_single_command(command)?;
while let Some(continuation) = continuations.next() {
// Turbofish needed because the compiler cannot infer M here.
- self.send_single_command::<ContinuationRecord<'_>>(bar, continuation)?;
+ self.send_single_command::<ContinuationRecord<'_>>(continuation)?;
}
Ok(())
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 04/33] gpu: nova-core: firmware: add r000 bindings
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (2 preceding siblings ...)
2026-09-18 1:06 ` [PATCH v3 03/33] gpu: nova-core: gsp: give the command queue its own BAR0 mapping John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:06 ` [PATCH v3 05/33] gpu: nova-core: regs: add msgq v2 BAR0 register declarations John Hubbard
` (28 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
Nova-core boots the r570 GSP firmware, release 570.144. The GSP firmware
releases from r615 onward share one ABI, and their firmware files carry
no version number in their names, because each image records its version
internally. The driver calls that ABI "r000", so that its name does not
change with each release.
Add the bindings that bindgen generates from the r000 headers of Open
RM, the open-source NVIDIA kernel driver. They have no user yet. The
following patches build the r000 queue code, commands and boot event
handlers on them, and the driver keeps compiling against the r570
bindings until the patch that switches firmware.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/fw.rs | 1 +
drivers/gpu/nova-core/gsp/fw/r000_00.rs | 29 +
.../gpu/nova-core/gsp/fw/r000_00/bindings.rs | 851 ++++++++++++++++++
3 files changed, 881 insertions(+)
create mode 100644 drivers/gpu/nova-core/gsp/fw/r000_00.rs
create mode 100644 drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index 918a7ae809eb..285c23cea771 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -2,6 +2,7 @@
// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
pub(crate) mod commands;
+mod r000_00;
mod r570_144;
// Alias to avoid repeating the version number with every use.
diff --git a/drivers/gpu/nova-core/gsp/fw/r000_00.rs b/drivers/gpu/nova-core/gsp/fw/r000_00.rs
new file mode 100644
index 000000000000..0c32f4ec1e27
--- /dev/null
+++ b/drivers/gpu/nova-core/gsp/fw/r000_00.rs
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Bindings generated by `bindgen` from the r000 GSP firmware headers.
+//!
+//! Use them only through the parent module, which abstracts or re-exports the symbols that the
+//! driver needs.
+
+#![allow(
+ dead_code,
+ clippy::all,
+ clippy::undocumented_unsafe_blocks,
+ clippy::ptr_as_ptr,
+ clippy::ref_as_ptr,
+ missing_docs,
+ non_camel_case_types,
+ non_upper_case_globals,
+ non_snake_case,
+ improper_ctypes,
+ unreachable_pub,
+ unsafe_op_in_unsafe_fn
+)]
+use kernel::ffi;
+use pin_init::MaybeZeroable;
+
+include!("r000_00/bindings.rs");
+
+// SAFETY: This type has a size of zero, so its inclusion into another type should not affect their
+// ability to implement `Zeroable`.
+unsafe impl<T> kernel::prelude::Zeroable for __IncompleteArrayField<T> {}
diff --git a/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs b/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs
new file mode 100644
index 000000000000..454d0ea9360f
--- /dev/null
+++ b/drivers/gpu/nova-core/gsp/fw/r000_00/bindings.rs
@@ -0,0 +1,851 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#[repr(C)]
+#[derive(Default)]
+pub struct __IncompleteArrayField<T>(::core::marker::PhantomData<T>, [T; 0]);
+impl<T> __IncompleteArrayField<T> {
+ #[inline]
+ pub const fn new() -> Self {
+ __IncompleteArrayField(::core::marker::PhantomData, [])
+ }
+ #[inline]
+ pub fn as_ptr(&self) -> *const T {
+ self as *const _ as *const T
+ }
+ #[inline]
+ pub fn as_mut_ptr(&mut self) -> *mut T {
+ self as *mut _ as *mut T
+ }
+ #[inline]
+ pub unsafe fn as_slice(&self, len: usize) -> &[T] {
+ ::core::slice::from_raw_parts(self.as_ptr(), len)
+ }
+ #[inline]
+ pub unsafe fn as_mut_slice(&mut self, len: usize) -> &mut [T] {
+ ::core::slice::from_raw_parts_mut(self.as_mut_ptr(), len)
+ }
+}
+impl<T> ::core::fmt::Debug for __IncompleteArrayField<T> {
+ fn fmt(&self, fmt: &mut ::core::fmt::Formatter<'_>) -> ::core::fmt::Result {
+ fmt.write_str("__IncompleteArrayField")
+ }
+}
+#[repr(C)]
+pub struct __BindgenUnionField<T>(::core::marker::PhantomData<T>);
+impl<T> __BindgenUnionField<T> {
+ #[inline]
+ pub const fn new() -> Self {
+ __BindgenUnionField(::core::marker::PhantomData)
+ }
+ #[inline]
+ pub unsafe fn as_ref(&self) -> &T {
+ ::core::mem::transmute(self)
+ }
+ #[inline]
+ pub unsafe fn as_mut(&mut self) -> &mut T {
+ ::core::mem::transmute(self)
+ }
+}
+impl<T> ::core::default::Default for __BindgenUnionField<T> {
+ #[inline]
+ fn default() -> Self {
+ Self::new()
+ }
+}
+impl<T> ::core::clone::Clone for __BindgenUnionField<T> {
+ #[inline]
+ fn clone(&self) -> Self {
+ *self
+ }
+}
+impl<T> ::core::marker::Copy for __BindgenUnionField<T> {}
+impl<T> ::core::fmt::Debug for __BindgenUnionField<T> {
+ fn fmt(&self, fmt: &mut ::core::fmt::Formatter<'_>) -> ::core::fmt::Result {
+ fmt.write_str("__BindgenUnionField")
+ }
+}
+impl<T> ::core::hash::Hash for __BindgenUnionField<T> {
+ fn hash<H: ::core::hash::Hasher>(&self, _state: &mut H) {}
+}
+impl<T> ::core::cmp::PartialEq for __BindgenUnionField<T> {
+ fn eq(&self, _other: &__BindgenUnionField<T>) -> bool {
+ true
+ }
+}
+impl<T> ::core::cmp::Eq for __BindgenUnionField<T> {}
+pub const NV2080_CTRL_GPU_SET_POWER_STATE_GPU_LEVEL_0: u32 = 0;
+pub const NV2080_CTRL_GPU_SET_POWER_STATE_GPU_LEVEL_3: u32 = 3;
+pub const NV2080_CTRL_GPU_SET_POWER_STATE_GPU_LEVEL_7: u32 = 7;
+pub const NV_VGPU_MSG_SIGNATURE_VALID: u32 = 1129337430;
+pub const GSP_FW_HEAP_PARAM_OS_SIZE_LIBOS2: u32 = 0;
+pub const GSP_FW_HEAP_PARAM_OS_SIZE_LIBOS3_BAREMETAL: u32 = 23068672;
+pub const GSP_FW_HEAP_PARAM_BASE_RM_SIZE_TU10X: u32 = 8388608;
+pub const GSP_FW_HEAP_PARAM_BASE_RM_SIZE_GH100: u32 = 18874368;
+pub const GSP_FW_HEAP_PARAM_SIZE_PER_GB: u32 = 98304;
+pub const GSP_FW_HEAP_PARAM_CLIENT_ALLOC_SIZE: u32 = 100663296;
+pub const GSP_FW_HEAP_SIZE_VGPU_DEFAULT: u32 = 609222656;
+pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MIN_MB: u32 = 64;
+pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MAX_MB: u32 = 256;
+pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MIN_MB: u32 = 88;
+pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MAX_MB: u32 = 280;
+pub const GSP_FW_WPR_META_REVISION: u32 = 1;
+pub const GSP_FW_WPR_META_MAGIC: i64 = -2577556379034558285;
+pub type __u8 = ffi::c_uchar;
+pub type __u16 = ffi::c_ushort;
+pub type __u32 = ffi::c_uint;
+pub type __u64 = ffi::c_ulonglong;
+pub type u8_ = __u8;
+pub type u16_ = __u16;
+pub type u32_ = __u32;
+pub type u64_ = __u64;
+pub const NV_VGPU_MSG_FUNCTION_NOP: _bindgen_ty_2 = 0;
+pub const NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO: _bindgen_ty_2 = 1;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_ROOT: _bindgen_ty_2 = 2;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_DEVICE: _bindgen_ty_2 = 3;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_MEMORY: _bindgen_ty_2 = 4;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_CTX_DMA: _bindgen_ty_2 = 5;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_CHANNEL_DMA: _bindgen_ty_2 = 6;
+pub const NV_VGPU_MSG_FUNCTION_MAP_MEMORY: _bindgen_ty_2 = 7;
+pub const NV_VGPU_MSG_FUNCTION_BIND_CTX_DMA: _bindgen_ty_2 = 8;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_OBJECT: _bindgen_ty_2 = 9;
+pub const NV_VGPU_MSG_FUNCTION_FREE: _bindgen_ty_2 = 10;
+pub const NV_VGPU_MSG_FUNCTION_LOG: _bindgen_ty_2 = 11;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_VIDMEM: _bindgen_ty_2 = 12;
+pub const NV_VGPU_MSG_FUNCTION_UNMAP_MEMORY: _bindgen_ty_2 = 13;
+pub const NV_VGPU_MSG_FUNCTION_MAP_MEMORY_DMA: _bindgen_ty_2 = 14;
+pub const NV_VGPU_MSG_FUNCTION_UNMAP_MEMORY_DMA: _bindgen_ty_2 = 15;
+pub const NV_VGPU_MSG_FUNCTION_GET_EDID: _bindgen_ty_2 = 16;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_DISP_CHANNEL: _bindgen_ty_2 = 17;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_DISP_OBJECT: _bindgen_ty_2 = 18;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_SUBDEVICE: _bindgen_ty_2 = 19;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_DYNAMIC_MEMORY: _bindgen_ty_2 = 20;
+pub const NV_VGPU_MSG_FUNCTION_DUP_OBJECT: _bindgen_ty_2 = 21;
+pub const NV_VGPU_MSG_FUNCTION_IDLE_CHANNELS: _bindgen_ty_2 = 22;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_EVENT: _bindgen_ty_2 = 23;
+pub const NV_VGPU_MSG_FUNCTION_SEND_EVENT: _bindgen_ty_2 = 24;
+pub const NV_VGPU_MSG_FUNCTION_REMAPPER_CONTROL: _bindgen_ty_2 = 25;
+pub const NV_VGPU_MSG_FUNCTION_DMA_CONTROL: _bindgen_ty_2 = 26;
+pub const NV_VGPU_MSG_FUNCTION_DMA_FILL_PTE_MEM: _bindgen_ty_2 = 27;
+pub const NV_VGPU_MSG_FUNCTION_MANAGE_HW_RESOURCE: _bindgen_ty_2 = 28;
+pub const NV_VGPU_MSG_FUNCTION_BIND_ARBITRARY_CTX_DMA: _bindgen_ty_2 = 29;
+pub const NV_VGPU_MSG_FUNCTION_CREATE_FB_SEGMENT: _bindgen_ty_2 = 30;
+pub const NV_VGPU_MSG_FUNCTION_DESTROY_FB_SEGMENT: _bindgen_ty_2 = 31;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_SHARE_DEVICE: _bindgen_ty_2 = 32;
+pub const NV_VGPU_MSG_FUNCTION_DEFERRED_API_CONTROL: _bindgen_ty_2 = 33;
+pub const NV_VGPU_MSG_FUNCTION_REMOVE_DEFERRED_API: _bindgen_ty_2 = 34;
+pub const NV_VGPU_MSG_FUNCTION_SIM_ESCAPE_READ: _bindgen_ty_2 = 35;
+pub const NV_VGPU_MSG_FUNCTION_SIM_ESCAPE_WRITE: _bindgen_ty_2 = 36;
+pub const NV_VGPU_MSG_FUNCTION_SIM_MANAGE_DISPLAY_CONTEXT_DMA: _bindgen_ty_2 = 37;
+pub const NV_VGPU_MSG_FUNCTION_FREE_VIDMEM_VIRT: _bindgen_ty_2 = 38;
+pub const NV_VGPU_MSG_FUNCTION_PERF_GET_PSTATE_INFO: _bindgen_ty_2 = 39;
+pub const NV_VGPU_MSG_FUNCTION_PERF_GET_PERFMON_SAMPLE: _bindgen_ty_2 = 40;
+pub const NV_VGPU_MSG_FUNCTION_PERF_GET_VIRTUAL_PSTATE_INFO: _bindgen_ty_2 = 41;
+pub const NV_VGPU_MSG_FUNCTION_PERF_GET_LEVEL_INFO: _bindgen_ty_2 = 42;
+pub const NV_VGPU_MSG_FUNCTION_MAP_SEMA_MEMORY: _bindgen_ty_2 = 43;
+pub const NV_VGPU_MSG_FUNCTION_UNMAP_SEMA_MEMORY: _bindgen_ty_2 = 44;
+pub const NV_VGPU_MSG_FUNCTION_SET_SURFACE_PROPERTIES: _bindgen_ty_2 = 45;
+pub const NV_VGPU_MSG_FUNCTION_CLEANUP_SURFACE: _bindgen_ty_2 = 46;
+pub const NV_VGPU_MSG_FUNCTION_UNLOADING_GUEST_DRIVER: _bindgen_ty_2 = 47;
+pub const NV_VGPU_MSG_FUNCTION_TDR_SET_TIMEOUT_STATE: _bindgen_ty_2 = 48;
+pub const NV_VGPU_MSG_FUNCTION_SWITCH_TO_VGA: _bindgen_ty_2 = 49;
+pub const NV_VGPU_MSG_FUNCTION_GPU_EXEC_REG_OPS: _bindgen_ty_2 = 50;
+pub const NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO: _bindgen_ty_2 = 51;
+pub const NV_VGPU_MSG_FUNCTION_ALLOC_VIRTMEM: _bindgen_ty_2 = 52;
+pub const NV_VGPU_MSG_FUNCTION_UPDATE_PDE_2: _bindgen_ty_2 = 53;
+pub const NV_VGPU_MSG_FUNCTION_SET_PAGE_DIRECTORY: _bindgen_ty_2 = 54;
+pub const NV_VGPU_MSG_FUNCTION_GET_STATIC_PSTATE_INFO: _bindgen_ty_2 = 55;
+pub const NV_VGPU_MSG_FUNCTION_TRANSLATE_GUEST_GPU_PTES: _bindgen_ty_2 = 56;
+pub const NV_VGPU_MSG_FUNCTION_RESERVED_57: _bindgen_ty_2 = 57;
+pub const NV_VGPU_MSG_FUNCTION_RESET_CURRENT_GR_CONTEXT: _bindgen_ty_2 = 58;
+pub const NV_VGPU_MSG_FUNCTION_SET_SEMA_MEM_VALIDATION_STATE: _bindgen_ty_2 = 59;
+pub const NV_VGPU_MSG_FUNCTION_GET_ENGINE_UTILIZATION: _bindgen_ty_2 = 60;
+pub const NV_VGPU_MSG_FUNCTION_UPDATE_GPU_PDES: _bindgen_ty_2 = 61;
+pub const NV_VGPU_MSG_FUNCTION_GET_ENCODER_CAPACITY: _bindgen_ty_2 = 62;
+pub const NV_VGPU_MSG_FUNCTION_VGPU_PF_REG_READ32: _bindgen_ty_2 = 63;
+pub const NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO_EXT: _bindgen_ty_2 = 64;
+pub const NV_VGPU_MSG_FUNCTION_GET_GSP_STATIC_INFO: _bindgen_ty_2 = 65;
+pub const NV_VGPU_MSG_FUNCTION_RMFS_INIT: _bindgen_ty_2 = 66;
+pub const NV_VGPU_MSG_FUNCTION_RMFS_CLOSE_QUEUE: _bindgen_ty_2 = 67;
+pub const NV_VGPU_MSG_FUNCTION_RMFS_CLEANUP: _bindgen_ty_2 = 68;
+pub const NV_VGPU_MSG_FUNCTION_RMFS_TEST: _bindgen_ty_2 = 69;
+pub const NV_VGPU_MSG_FUNCTION_UPDATE_BAR_PDE: _bindgen_ty_2 = 70;
+pub const NV_VGPU_MSG_FUNCTION_CONTINUATION_RECORD: _bindgen_ty_2 = 71;
+pub const NV_VGPU_MSG_FUNCTION_GSP_SET_SYSTEM_INFO: _bindgen_ty_2 = 72;
+pub const NV_VGPU_MSG_FUNCTION_SET_REGISTRY: _bindgen_ty_2 = 73;
+pub const NV_VGPU_MSG_FUNCTION_GSP_INIT_POST_OBJGPU: _bindgen_ty_2 = 74;
+pub const NV_VGPU_MSG_FUNCTION_SUBDEV_EVENT_SET_NOTIFICATION: _bindgen_ty_2 = 75;
+pub const NV_VGPU_MSG_FUNCTION_GSP_RM_CONTROL: _bindgen_ty_2 = 76;
+pub const NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO2: _bindgen_ty_2 = 77;
+pub const NV_VGPU_MSG_FUNCTION_DUMP_PROTOBUF_COMPONENT: _bindgen_ty_2 = 78;
+pub const NV_VGPU_MSG_FUNCTION_UNSET_PAGE_DIRECTORY: _bindgen_ty_2 = 79;
+pub const NV_VGPU_MSG_FUNCTION_GET_CONSOLIDATED_STATIC_INFO: _bindgen_ty_2 = 80;
+pub const NV_VGPU_MSG_FUNCTION_GMMU_REGISTER_FAULT_BUFFER: _bindgen_ty_2 = 81;
+pub const NV_VGPU_MSG_FUNCTION_GMMU_UNREGISTER_FAULT_BUFFER: _bindgen_ty_2 = 82;
+pub const NV_VGPU_MSG_FUNCTION_GMMU_REGISTER_CLIENT_SHADOW_FAULT_BUFFER: _bindgen_ty_2 = 83;
+pub const NV_VGPU_MSG_FUNCTION_GMMU_UNREGISTER_CLIENT_SHADOW_FAULT_BUFFER: _bindgen_ty_2 = 84;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_VGPU_FB_USAGE: _bindgen_ty_2 = 85;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_NVFBC_SW_SESSION_UPDATE_INFO: _bindgen_ty_2 = 86;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_NVENC_SW_SESSION_UPDATE_INFO: _bindgen_ty_2 = 87;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_RESET_CHANNEL: _bindgen_ty_2 = 88;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_RESET_ISOLATED_CHANNEL: _bindgen_ty_2 = 89;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_HANDLE_VF_PRI_FAULT: _bindgen_ty_2 = 90;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_CLK_GET_EXTENDED_INFO: _bindgen_ty_2 = 91;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_PERF_BOOST: _bindgen_ty_2 = 92;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_PERF_VPSTATES_GET_CONTROL: _bindgen_ty_2 = 93;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_ZBC_CLEAR_TABLE: _bindgen_ty_2 = 94;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_ZBC_COLOR_CLEAR: _bindgen_ty_2 = 95;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_ZBC_DEPTH_CLEAR: _bindgen_ty_2 = 96;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GPFIFO_SCHEDULE: _bindgen_ty_2 = 97;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_TIMESLICE: _bindgen_ty_2 = 98;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_PREEMPT: _bindgen_ty_2 = 99;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FIFO_DISABLE_CHANNELS: _bindgen_ty_2 = 100;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_TSG_INTERLEAVE_LEVEL: _bindgen_ty_2 = 101;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_CHANNEL_INTERLEAVE_LEVEL: _bindgen_ty_2 = 102;
+pub const NV_VGPU_MSG_FUNCTION_GSP_RM_ALLOC: _bindgen_ty_2 = 103;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_P2P_CAPS_V2: _bindgen_ty_2 = 104;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_CIPHER_AES_ENCRYPT: _bindgen_ty_2 = 105;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_CIPHER_SESSION_KEY: _bindgen_ty_2 = 106;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_CIPHER_SESSION_KEY_STATUS: _bindgen_ty_2 = 107;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_CLEAR_ALL_SM_ERROR_STATES: _bindgen_ty_2 = 108;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_READ_ALL_SM_ERROR_STATES: _bindgen_ty_2 = 109;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SET_EXCEPTION_MASK: _bindgen_ty_2 = 110;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_PROMOTE_CTX: _bindgen_ty_2 = 111;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_CTXSW_PREEMPTION_BIND: _bindgen_ty_2 = 112;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_SET_CTXSW_PREEMPTION_MODE: _bindgen_ty_2 = 113;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_CTXSW_ZCULL_BIND: _bindgen_ty_2 = 114;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_INITIALIZE_CTX: _bindgen_ty_2 = 115;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_VASPACE_COPY_SERVER_RESERVED_PDES: _bindgen_ty_2 = 116;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FIFO_CLEAR_FAULTED_BIT: _bindgen_ty_2 = 117;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_LATEST_ECC_ADDRESSES: _bindgen_ty_2 = 118;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_MC_SERVICE_INTERRUPTS: _bindgen_ty_2 = 119;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DMA_SET_DEFAULT_VASPACE: _bindgen_ty_2 = 120;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_CE_PCE_MASK: _bindgen_ty_2 = 121;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_ZBC_CLEAR_TABLE_ENTRY: _bindgen_ty_2 = 122;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_NVLINK_PEER_ID_MASK: _bindgen_ty_2 = 123;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_NVLINK_STATUS: _bindgen_ty_2 = 124;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_P2P_CAPS: _bindgen_ty_2 = 125;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_P2P_CAPS_MATRIX: _bindgen_ty_2 = 126;
+pub const NV_VGPU_MSG_FUNCTION_RESERVED_0: _bindgen_ty_2 = 127;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_RESERVE_PM_AREA_SMPC: _bindgen_ty_2 = 128;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_RESERVE_HWPM_LEGACY: _bindgen_ty_2 = 129;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_B0CC_EXEC_REG_OPS: _bindgen_ty_2 = 130;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_BIND_PM_RESOURCES: _bindgen_ty_2 = 131;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SUSPEND_CONTEXT: _bindgen_ty_2 = 132;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_RESUME_CONTEXT: _bindgen_ty_2 = 133;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_EXEC_REG_OPS: _bindgen_ty_2 = 134;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SET_MODE_MMU_DEBUG: _bindgen_ty_2 = 135;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_READ_SINGLE_SM_ERROR_STATE: _bindgen_ty_2 = 136;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_CLEAR_SINGLE_SM_ERROR_STATE: _bindgen_ty_2 = 137;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SET_MODE_ERRBAR_DEBUG: _bindgen_ty_2 = 138;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SET_NEXT_STOP_TRIGGER_TYPE: _bindgen_ty_2 = 139;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_ALLOC_PMA_STREAM: _bindgen_ty_2 = 140;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_PMA_STREAM_UPDATE_GET_PUT: _bindgen_ty_2 = 141;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FB_GET_INFO_V2: _bindgen_ty_2 = 142;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FIFO_SET_CHANNEL_PROPERTIES: _bindgen_ty_2 = 143;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_GET_CTX_BUFFER_INFO: _bindgen_ty_2 = 144;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_KGR_GET_CTX_BUFFER_PTES: _bindgen_ty_2 = 145;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_EVICT_CTX: _bindgen_ty_2 = 146;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FB_GET_FS_INFO: _bindgen_ty_2 = 147;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GRMGR_GET_GR_FS_INFO: _bindgen_ty_2 = 148;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_STOP_CHANNEL: _bindgen_ty_2 = 149;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_PC_SAMPLING_MODE: _bindgen_ty_2 = 150;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_PERF_RATED_TDP_GET_STATUS: _bindgen_ty_2 = 151;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_PERF_RATED_TDP_SET_CONTROL: _bindgen_ty_2 = 152;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FREE_PMA_STREAM: _bindgen_ty_2 = 153;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_TIMER_SET_GR_TICK_FREQ: _bindgen_ty_2 = 154;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FIFO_SETUP_VF_ZOMBIE_SUBCTX_PDB: _bindgen_ty_2 = 155;
+pub const NV_VGPU_MSG_FUNCTION_GET_CONSOLIDATED_GR_STATIC_INFO: _bindgen_ty_2 = 156;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SET_SINGLE_SM_SINGLE_STEP: _bindgen_ty_2 = 157;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_GET_TPC_PARTITION_MODE: _bindgen_ty_2 = 158;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_SET_TPC_PARTITION_MODE: _bindgen_ty_2 = 159;
+pub const NV_VGPU_MSG_FUNCTION_UVM_PAGING_CHANNEL_ALLOCATE: _bindgen_ty_2 = 160;
+pub const NV_VGPU_MSG_FUNCTION_UVM_PAGING_CHANNEL_DESTROY: _bindgen_ty_2 = 161;
+pub const NV_VGPU_MSG_FUNCTION_UVM_PAGING_CHANNEL_MAP: _bindgen_ty_2 = 162;
+pub const NV_VGPU_MSG_FUNCTION_UVM_PAGING_CHANNEL_UNMAP: _bindgen_ty_2 = 163;
+pub const NV_VGPU_MSG_FUNCTION_UVM_PAGING_CHANNEL_PUSH_STREAM: _bindgen_ty_2 = 164;
+pub const NV_VGPU_MSG_FUNCTION_UVM_PAGING_CHANNEL_SET_HANDLES: _bindgen_ty_2 = 165;
+pub const NV_VGPU_MSG_FUNCTION_UVM_METHOD_STREAM_GUEST_PAGES_OPERATION: _bindgen_ty_2 = 166;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_INTERNAL_QUIESCE_PMA_CHANNEL: _bindgen_ty_2 = 167;
+pub const NV_VGPU_MSG_FUNCTION_DCE_RM_INIT: _bindgen_ty_2 = 168;
+pub const NV_VGPU_MSG_FUNCTION_REGISTER_VIRTUAL_EVENT_BUFFER: _bindgen_ty_2 = 169;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_EVENT_BUFFER_UPDATE_GET: _bindgen_ty_2 = 170;
+pub const NV_VGPU_MSG_FUNCTION_GET_PLCABLE_ADDRESS_KIND: _bindgen_ty_2 = 171;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_PERF_LIMITS_SET_STATUS_V2: _bindgen_ty_2 = 172;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_INTERNAL_SRIOV_PROMOTE_PMA_STREAM: _bindgen_ty_2 = 173;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_MMU_DEBUG_MODE: _bindgen_ty_2 = 174;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_INTERNAL_PROMOTE_FAULT_METHOD_BUFFERS: _bindgen_ty_2 = 175;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FLCN_GET_CTX_BUFFER_SIZE: _bindgen_ty_2 = 176;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FLCN_GET_CTX_BUFFER_INFO: _bindgen_ty_2 = 177;
+pub const NV_VGPU_MSG_FUNCTION_DISABLE_CHANNELS: _bindgen_ty_2 = 178;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FABRIC_MEMORY_DESCRIBE: _bindgen_ty_2 = 179;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FABRIC_MEM_STATS: _bindgen_ty_2 = 180;
+pub const NV_VGPU_MSG_FUNCTION_SAVE_HIBERNATION_DATA: _bindgen_ty_2 = 181;
+pub const NV_VGPU_MSG_FUNCTION_RESTORE_HIBERNATION_DATA: _bindgen_ty_2 = 182;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_INTERNAL_MEMSYS_SET_ZBC_REFERENCED: _bindgen_ty_2 = 183;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_EXEC_PARTITIONS_CREATE: _bindgen_ty_2 = 184;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_EXEC_PARTITIONS_DELETE: _bindgen_ty_2 = 185;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GPFIFO_GET_WORK_SUBMIT_TOKEN: _bindgen_ty_2 = 186;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GPFIFO_SET_WORK_SUBMIT_TOKEN_NOTIF_INDEX: _bindgen_ty_2 = 187;
+pub const NV_VGPU_MSG_FUNCTION_PMA_SCRUBBER_SHARED_BUFFER_GUEST_PAGES_OPERATION: _bindgen_ty_2 =
+ 188;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_MASTER_GET_VIRTUAL_FUNCTION_ERROR_CONT_INTR_MASK:
+ _bindgen_ty_2 = 189;
+pub const NV_VGPU_MSG_FUNCTION_RESERVED_190: _bindgen_ty_2 = 190;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_SUBDEVICE_GET_P2P_CAPS: _bindgen_ty_2 = 191;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_BUS_SET_P2P_MAPPING: _bindgen_ty_2 = 192;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_BUS_UNSET_P2P_MAPPING: _bindgen_ty_2 = 193;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FLA_SETUP_INSTANCE_MEM_BLOCK: _bindgen_ty_2 = 194;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_MIGRATABLE_OPS: _bindgen_ty_2 = 195;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_TOTAL_HS_CREDITS: _bindgen_ty_2 = 196;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_HS_CREDITS: _bindgen_ty_2 = 197;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_HS_CREDITS: _bindgen_ty_2 = 198;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_PM_AREA_PC_SAMPLER: _bindgen_ty_2 = 199;
+pub const NV_VGPU_MSG_FUNCTION_INVALIDATE_TLB: _bindgen_ty_2 = 200;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_QUERY_ECC_STATUS: _bindgen_ty_2 = 201;
+pub const NV_VGPU_MSG_FUNCTION_ECC_NOTIFIER_WRITE_ACK: _bindgen_ty_2 = 202;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_GET_MODE_MMU_DEBUG: _bindgen_ty_2 = 203;
+pub const NV_VGPU_MSG_FUNCTION_RM_API_CONTROL: _bindgen_ty_2 = 204;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_CMD_INTERNAL_GPU_START_FABRIC_PROBE: _bindgen_ty_2 = 205;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_NVLINK_GET_INBAND_RECEIVED_DATA: _bindgen_ty_2 = 206;
+pub const NV_VGPU_MSG_FUNCTION_GET_STATIC_DATA: _bindgen_ty_2 = 207;
+pub const NV_VGPU_MSG_FUNCTION_RESERVED_208: _bindgen_ty_2 = 208;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_GET_INFO_V2: _bindgen_ty_2 = 209;
+pub const NV_VGPU_MSG_FUNCTION_GET_BRAND_CAPS: _bindgen_ty_2 = 210;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_CMD_NVLINK_INBAND_SEND_DATA: _bindgen_ty_2 = 211;
+pub const NV_VGPU_MSG_FUNCTION_UPDATE_GPM_GUEST_BUFFER_INFO: _bindgen_ty_2 = 212;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_CMD_INTERNAL_CONTROL_GSP_TRACE: _bindgen_ty_2 = 213;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_ZBC_STENCIL_CLEAR: _bindgen_ty_2 = 214;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_SUBDEVICE_GET_VGPU_HEAP_STATS: _bindgen_ty_2 = 215;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_SUBDEVICE_GET_LIBOS_HEAP_STATS: _bindgen_ty_2 = 216;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SET_MODE_MMU_GCC_DEBUG: _bindgen_ty_2 = 217;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_GET_MODE_MMU_GCC_DEBUG: _bindgen_ty_2 = 218;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_RESERVE_HES: _bindgen_ty_2 = 219;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_RELEASE_HES: _bindgen_ty_2 = 220;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_RESERVE_CCU_PROF: _bindgen_ty_2 = 221;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_RELEASE_CCU_PROF: _bindgen_ty_2 = 222;
+pub const NV_VGPU_MSG_FUNCTION_SETUP_HIBERNATION_BUFFER: _bindgen_ty_2 = 223;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_CMD_GET_CHIPLET_HS_CREDIT_POOL: _bindgen_ty_2 = 224;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_CMD_GET_HS_CREDITS_MAPPING: _bindgen_ty_2 = 225;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_EXEC_PARTITIONS_EXPORT: _bindgen_ty_2 = 226;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_CMD_INTERNAL_GPU_CHECK_CTS_ID_VALID: _bindgen_ty_2 = 227;
+pub const NV_VGPU_MSG_FUNCTION_INIT_GSP_TRACE_CRASH_BUFFER: _bindgen_ty_2 = 228;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_SET_MIGRATION_BLOCK: _bindgen_ty_2 = 229;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FIFO_VGPU_SWRUNLIST_SUBMIT: _bindgen_ty_2 = 230;
+pub const NV_VGPU_MSG_FUNCTION_CTRL_FIFO_VGPU_SWRUNLIST_CHANNEL_UPDATE: _bindgen_ty_2 = 231;
+pub const NV_VGPU_MSG_FUNCTION_NUM_FUNCTIONS: _bindgen_ty_2 = 232;
+pub type _bindgen_ty_2 = ffi::c_uint;
+pub const NV_VGPU_MSG_EVENT_FIRST_EVENT: _bindgen_ty_3 = 4096;
+pub const NV_VGPU_MSG_EVENT_GSP_INIT_DONE: _bindgen_ty_3 = 4097;
+pub const NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER: _bindgen_ty_3 = 4098;
+pub const NV_VGPU_MSG_EVENT_POST_EVENT: _bindgen_ty_3 = 4099;
+pub const NV_VGPU_MSG_EVENT_RC_TRIGGERED: _bindgen_ty_3 = 4100;
+pub const NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED: _bindgen_ty_3 = 4101;
+pub const NV_VGPU_MSG_EVENT_OS_ERROR_LOG: _bindgen_ty_3 = 4102;
+pub const NV_VGPU_MSG_EVENT_RG_LINE_INTR: _bindgen_ty_3 = 4103;
+pub const NV_VGPU_MSG_EVENT_GPUACCT_PERFMON_UTIL_SAMPLES: _bindgen_ty_3 = 4104;
+pub const NV_VGPU_MSG_EVENT_SIM_READ: _bindgen_ty_3 = 4105;
+pub const NV_VGPU_MSG_EVENT_SIM_WRITE: _bindgen_ty_3 = 4106;
+pub const NV_VGPU_MSG_EVENT_SEMAPHORE_SCHEDULE_CALLBACK: _bindgen_ty_3 = 4107;
+pub const NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT: _bindgen_ty_3 = 4108;
+pub const NV_VGPU_MSG_EVENT_VGPU_GSP_PLUGIN_TRIGGERED: _bindgen_ty_3 = 4109;
+pub const NV_VGPU_MSG_EVENT_PERF_GPU_BOOST_SYNC_LIMITS_CALLBACK: _bindgen_ty_3 = 4110;
+pub const NV_VGPU_MSG_EVENT_PERF_BRIDGELESS_INFO_UPDATE: _bindgen_ty_3 = 4111;
+pub const NV_VGPU_MSG_EVENT_VGPU_CONFIG: _bindgen_ty_3 = 4112;
+pub const NV_VGPU_MSG_EVENT_DISPLAY_MODESET: _bindgen_ty_3 = 4113;
+pub const NV_VGPU_MSG_EVENT_EXTDEV_INTR_SERVICE: _bindgen_ty_3 = 4114;
+pub const NV_VGPU_MSG_EVENT_NVLINK_INBAND_RECEIVED_DATA_256: _bindgen_ty_3 = 4115;
+pub const NV_VGPU_MSG_EVENT_NVLINK_INBAND_RECEIVED_DATA_512: _bindgen_ty_3 = 4116;
+pub const NV_VGPU_MSG_EVENT_NVLINK_INBAND_RECEIVED_DATA_1024: _bindgen_ty_3 = 4117;
+pub const NV_VGPU_MSG_EVENT_NVLINK_INBAND_RECEIVED_DATA_2048: _bindgen_ty_3 = 4118;
+pub const NV_VGPU_MSG_EVENT_NVLINK_INBAND_RECEIVED_DATA_4096: _bindgen_ty_3 = 4119;
+pub const NV_VGPU_MSG_EVENT_TIMED_SEMAPHORE_RELEASE: _bindgen_ty_3 = 4120;
+pub const NV_VGPU_MSG_EVENT_NVLINK_IS_GPU_DEGRADED: _bindgen_ty_3 = 4121;
+pub const NV_VGPU_MSG_EVENT_PFM_REQ_HNDLR_STATE_SYNC_CALLBACK: _bindgen_ty_3 = 4122;
+pub const NV_VGPU_MSG_EVENT_NVLINK_FAULT_UP: _bindgen_ty_3 = 4123;
+pub const NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE: _bindgen_ty_3 = 4124;
+pub const NV_VGPU_MSG_EVENT_MIG_CI_CONFIG_UPDATE: _bindgen_ty_3 = 4125;
+pub const NV_VGPU_MSG_EVENT_UPDATE_GSP_TRACE: _bindgen_ty_3 = 4126;
+pub const NV_VGPU_MSG_EVENT_NVLINK_FATAL_ERROR_RECOVERY: _bindgen_ty_3 = 4127;
+pub const NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD: _bindgen_ty_3 = 4128;
+pub const NV_VGPU_MSG_EVENT_FECS_ERROR: _bindgen_ty_3 = 4129;
+pub const NV_VGPU_MSG_EVENT_RECOVERY_ACTION: _bindgen_ty_3 = 4130;
+pub const NV_VGPU_MSG_EVENT_TRIGGER_BUGCHECK: _bindgen_ty_3 = 4131;
+pub const NV_VGPU_MSG_EVENT_BIND_BAR2: _bindgen_ty_3 = 4132;
+pub const NV_VGPU_MSG_EVENT_FORCED_DRIVER_SHUTDOWN: _bindgen_ty_3 = 4133;
+pub const NV_VGPU_MSG_EVENT_GSP_LOAD_EXEC_GENERIC_BOOTLOADER: _bindgen_ty_3 = 4134;
+pub const NV_VGPU_MSG_EVENT_GSP_LOAD_EXEC_HS_BINARY: _bindgen_ty_3 = 4135;
+pub const NV_VGPU_MSG_EVENT_UPDATE_GRID_DISPLAYLESS_PARAMS: _bindgen_ty_3 = 4136;
+pub const NV_VGPU_MSG_EVENT_PMU_HALTED: _bindgen_ty_3 = 4137;
+pub const NV_VGPU_MSG_EVENT_ECC_POSSIBLE_ERR_FAST_PATH: _bindgen_ty_3 = 4138;
+pub const NV_VGPU_MSG_EVENT_NVLINK_TRAFFIC_QUIESCE_NOTIFY: _bindgen_ty_3 = 4139;
+pub const NV_VGPU_MSG_EVENT_GPU_GFM_STATE_CHANGE: _bindgen_ty_3 = 4140;
+pub const NV_VGPU_MSG_EVENT_NVLINK_ABM_FABRIC_HEALTH_MASK_UPDATE: _bindgen_ty_3 = 4141;
+pub const NV_VGPU_MSG_EVENT_NUM_EVENTS: _bindgen_ty_3 = 4142;
+pub type _bindgen_ty_3 = ffi::c_uint;
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct MESSAGE_QUEUE_INIT_ARGUMENTS {
+ pub flags: u64_,
+ pub sharedMemPhysAddr: u64_,
+ pub cmdQueueOffset: u64_,
+ pub statQueueOffset: u64_,
+ pub queueElementHdrSize: u32_,
+ pub queueElementSizeMin: u32_,
+ pub queueElementSizeMax: u32_,
+ pub queueHeaderAlign: u32_,
+ pub queueElementAlign: u32_,
+ pub pageTableEntryCount: u32_,
+ pub reserved: [u8_; 8usize],
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GSP_SR_INIT_ARGUMENTS {
+ pub oldLevel: u32_,
+ pub flags: u32_,
+ pub reserved: [u8_; 8usize],
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GSP_ARGUMENTS_CACHED {
+ pub magic: u32_,
+ pub size: u16_,
+ pub reserved: [u8_; 2usize],
+ pub flags: u64_,
+ pub messageQueueInitArguments: MESSAGE_QUEUE_INIT_ARGUMENTS,
+ pub srInitArguments: GSP_SR_INIT_ARGUMENTS,
+ pub reserved2: [u8_; 4usize],
+ pub gpuInstance: u32_,
+ pub profilerArgs: GSP_ARGUMENTS_CACHED__bindgen_ty_1,
+ pub sysmemHeapArgs: GSP_ARGUMENTS_CACHED__bindgen_ty_2,
+ pub rmStateMonitorBufferArgs: GSP_ARGUMENTS_CACHED__bindgen_ty_3,
+ pub bindataArgs: GSP_ARGUMENTS_CACHED__bindgen_ty_4,
+ pub vbiosOverrideArgs: GSP_ARGUMENTS_CACHED__bindgen_ty_5,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GSP_ARGUMENTS_CACHED__bindgen_ty_1 {
+ pub pa: u64_,
+ pub size: u64_,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GSP_ARGUMENTS_CACHED__bindgen_ty_2 {
+ pub pa: u64_,
+ pub size: u64_,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GSP_ARGUMENTS_CACHED__bindgen_ty_3 {
+ pub pa: u64_,
+ pub size: u64_,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GSP_ARGUMENTS_CACHED__bindgen_ty_4 {
+ pub radix3: u64_,
+ pub size: u64_,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GSP_ARGUMENTS_CACHED__bindgen_ty_5 {
+ pub pa: u64_,
+ pub size: u64_,
+}
+#[repr(C)]
+#[derive(Copy, Clone, MaybeZeroable)]
+pub union rpc_message_rpc_union_field_v03_00 {
+ pub spare: u32_,
+ pub cpuRmGfid: u32_,
+}
+impl Default for rpc_message_rpc_union_field_v03_00 {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+pub type rpc_message_rpc_union_field_v = rpc_message_rpc_union_field_v03_00;
+#[repr(C)]
+#[derive(MaybeZeroable)]
+pub struct rpc_message_header_v03_00 {
+ pub header_version: u32_,
+ pub signature: u32_,
+ pub length: u32_,
+ pub function: u32_,
+ pub rpc_result: u32_,
+ pub rpc_result_private: u32_,
+ pub sequence: u32_,
+ pub u: rpc_message_rpc_union_field_v,
+ pub rpc_message_data: __IncompleteArrayField<u8_>,
+}
+impl Default for rpc_message_header_v03_00 {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+pub type rpc_message_header_v = rpc_message_header_v03_00;
+#[repr(C)]
+#[derive(Copy, Clone, MaybeZeroable)]
+pub struct GspFwWprMeta {
+ pub magic: u64_,
+ pub revision: u64_,
+ pub sysmemAddrOfRadix3Elf: u64_,
+ pub sizeOfRadix3Elf: u64_,
+ pub sysmemAddrOfBootloader: u64_,
+ pub sizeOfBootloader: u64_,
+ pub bootloaderCodeOffset: u64_,
+ pub bootloaderDataOffset: u64_,
+ pub bootloaderManifestOffset: u64_,
+ pub __bindgen_anon_1: GspFwWprMeta__bindgen_ty_1,
+ pub gspFwRsvdStart: u64_,
+ pub nonWprHeapOffset: u64_,
+ pub nonWprHeapSize: u64_,
+ pub gspFwWprStart: u64_,
+ pub gspFwHeapOffset: u64_,
+ pub gspFwHeapSize: u64_,
+ pub gspFwOffset: u64_,
+ pub bootBinOffset: u64_,
+ pub frtsOffset: u64_,
+ pub frtsSize: u64_,
+ pub gspFwWprEnd: u64_,
+ pub fbSize: u64_,
+ pub vgaWorkspaceOffset: u64_,
+ pub vgaWorkspaceSize: u64_,
+ pub bootCount: u64_,
+ pub __bindgen_anon_2: GspFwWprMeta__bindgen_ty_2,
+ pub gspFwHeapVfPartitionCount: u8_,
+ pub flags: u8_,
+ pub pagingConfig: u16_,
+ pub pmuReservedSize: u32_,
+ pub verified: u64_,
+}
+#[repr(C)]
+#[derive(Copy, Clone, MaybeZeroable)]
+pub union GspFwWprMeta__bindgen_ty_1 {
+ pub __bindgen_anon_1: GspFwWprMeta__bindgen_ty_1__bindgen_ty_1,
+ pub __bindgen_anon_2: GspFwWprMeta__bindgen_ty_1__bindgen_ty_2,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GspFwWprMeta__bindgen_ty_1__bindgen_ty_1 {
+ pub sysmemAddrOfSignature: u64_,
+ pub sizeOfSignature: u64_,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GspFwWprMeta__bindgen_ty_1__bindgen_ty_2 {
+ pub gspFwHeapFreeListWprOffset: u32_,
+ pub unused0: u32_,
+ pub unused1: u64_,
+}
+impl Default for GspFwWprMeta__bindgen_ty_1 {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+#[repr(C)]
+#[derive(Copy, Clone, MaybeZeroable)]
+pub union GspFwWprMeta__bindgen_ty_2 {
+ pub __bindgen_anon_1: GspFwWprMeta__bindgen_ty_2__bindgen_ty_1,
+ pub __bindgen_anon_2: GspFwWprMeta__bindgen_ty_2__bindgen_ty_2,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GspFwWprMeta__bindgen_ty_2__bindgen_ty_1 {
+ pub partitionRpcAddr: u64_,
+ pub partitionRpcRequestOffset: u16_,
+ pub partitionRpcReplyOffset: u16_,
+ pub elfCodeOffset: u32_,
+ pub elfDataOffset: u32_,
+ pub elfCodeSize: u32_,
+ pub elfDataSize: u32_,
+ pub lsUcodeVersion: u32_,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GspFwWprMeta__bindgen_ty_2__bindgen_ty_2 {
+ pub partitionRpcPadding: [u32_; 4usize],
+ pub sysmemAddrOfCrashReportQueue: u64_,
+ pub sizeOfCrashReportQueue: u32_,
+ pub lsUcodeVersionPadding: [u32_; 1usize],
+}
+impl Default for GspFwWprMeta__bindgen_ty_2 {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+impl Default for GspFwWprMeta {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+pub type LibosAddress = u64_;
+pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_NONE: LibosMemoryRegionKind = 0;
+pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_CONTIGUOUS: LibosMemoryRegionKind = 1;
+pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_RADIX3: LibosMemoryRegionKind = 2;
+pub type LibosMemoryRegionKind = ffi::c_uint;
+pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_NONE: LibosMemoryRegionLoc = 0;
+pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_SYSMEM: LibosMemoryRegionLoc = 1;
+pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_FB: LibosMemoryRegionLoc = 2;
+pub type LibosMemoryRegionLoc = ffi::c_uint;
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct LibosMemoryRegionInitArgument {
+ pub id8: LibosAddress,
+ pub pa: LibosAddress,
+ pub size: LibosAddress,
+ pub kind: u8_,
+ pub loc: u8_,
+ pub __bindgen_padding_0: [u8; 6usize],
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct msgqTxHeader {
+ pub versionMajor: u16_,
+ pub versionMinor: u16_,
+ pub size: u32_,
+ pub msgSize: u32_,
+ pub msgCount: u32_,
+ pub entryOff: u32_,
+ pub reserved: [u32_; 3usize],
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GSP_MSG_QUEUE_ENCRYPTION_TAG {
+ pub authTagBuffer: [u8_; 16usize],
+}
+#[repr(C)]
+#[derive(MaybeZeroable)]
+pub struct GSP_MSG_QUEUE_ELEMENT {
+ pub mctpMagic: u32_,
+ pub mctpPayloadSize: u32_,
+ pub mctpHeader: u32_,
+ pub nvdmHeader: u32_,
+ pub __bindgen_anon_1: GSP_MSG_QUEUE_ELEMENT__bindgen_ty_1,
+}
+#[repr(C)]
+#[derive(MaybeZeroable)]
+pub struct GSP_MSG_QUEUE_ELEMENT__bindgen_ty_1 {
+ pub withEncryption: __BindgenUnionField<GSP_MSG_QUEUE_ELEMENT__bindgen_ty_1__bindgen_ty_1>,
+ pub noEncryption: __BindgenUnionField<GSP_MSG_QUEUE_ELEMENT__bindgen_ty_1__bindgen_ty_2>,
+ pub bindgen_union_field: [u32; 6usize],
+}
+#[repr(C)]
+#[derive(Debug, Default, MaybeZeroable)]
+pub struct GSP_MSG_QUEUE_ELEMENT__bindgen_ty_1__bindgen_ty_1 {
+ pub encryptionTag: GSP_MSG_QUEUE_ENCRYPTION_TAG,
+ pub nvdmPayloadSize: u32_,
+ pub reserved: u32_,
+ pub payload: __IncompleteArrayField<u8_>,
+}
+#[repr(C)]
+#[derive(Debug, Default, MaybeZeroable)]
+pub struct GSP_MSG_QUEUE_ELEMENT__bindgen_ty_1__bindgen_ty_2 {
+ pub nvdmPayloadSize: u32_,
+ pub reserved: u32_,
+ pub payload: __IncompleteArrayField<u8_>,
+}
+impl Default for GSP_MSG_QUEUE_ELEMENT__bindgen_ty_1 {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+impl Default for GSP_MSG_QUEUE_ELEMENT {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+#[repr(C)]
+#[derive(MaybeZeroable)]
+pub struct GMCAPI_HEADER {
+ pub command: u32_,
+ pub size: u32_,
+ pub sequence: u64_,
+ pub __bindgen_anon_1: GMCAPI_HEADER__bindgen_ty_1,
+ pub reserved: [u32_; 5usize],
+ pub data: __IncompleteArrayField<u8_>,
+}
+#[repr(C)]
+#[derive(Copy, Clone, MaybeZeroable)]
+pub union GMCAPI_HEADER__bindgen_ty_1 {
+ pub request: GMCAPI_HEADER__bindgen_ty_1__bindgen_ty_1,
+ pub response: GMCAPI_HEADER__bindgen_ty_1__bindgen_ty_2,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GMCAPI_HEADER__bindgen_ty_1__bindgen_ty_1 {
+ pub max_response_size: u32_,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GMCAPI_HEADER__bindgen_ty_1__bindgen_ty_2 {
+ pub status: u32_,
+}
+impl Default for GMCAPI_HEADER__bindgen_ty_1 {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+impl Default for GMCAPI_HEADER {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_RESERVED_LAST: GMCAPI_COMMANDS = 65535;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_GSP_INIT: GMCAPI_COMMANDS = 65537;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER: GMCAPI_COMMANDS = 65538;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_EXEC_HS_BINARY: GMCAPI_COMMANDS = 65539;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_GSP_SUSPEND: GMCAPI_COMMANDS = 65540;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_GSP_RESUME_DONE: GMCAPI_COMMANDS = 65541;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_ADD_VGPU_TYPE: GMCAPI_COMMANDS = 131073;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_SUPPORTED_VGPU_TYPES: GMCAPI_COMMANDS = 131074;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_CREATABLE_VGPU_TYPES: GMCAPI_COMMANDS = 131075;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_ASSIGN_VGPU_TYPE: GMCAPI_COMMANDS = 131076;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_DEASSIGN_VGPU_TYPE: GMCAPI_COMMANDS = 131077;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_VGPU_PROPERTIES: GMCAPI_COMMANDS = 131078;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_ASSIGNED_VF_VGPU_TYPE: GMCAPI_COMMANDS = 131079;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_BOOTLOAD_GSP_VGPU_PLUGIN_TASK: GMCAPI_COMMANDS = 131104;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_SHUTDOWN_GSP_VGPU_PLUGIN_TASK: GMCAPI_COMMANDS = 131105;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_SHUTDOWN_GSP_VGPU_PLUGIN_TASK_COMPLETE: GMCAPI_COMMANDS =
+ 131106;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_CLEANUP_GSP_VGPU_PLUGIN_RESOURCES: GMCAPI_COMMANDS = 131107;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_VGPU_PLUGIN_TRIGGERED_EVENT: GMCAPI_COMMANDS = 131108;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_VGPU_MGR_SCRUB_GUEST_FB: GMCAPI_COMMANDS = 131109;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_VGPU_MGR_ALLOC_GSP_CEUTILS: GMCAPI_COMMANDS = 131110;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_VGPU_MGR_FREE_GSP_CEUTILS: GMCAPI_COMMANDS = 131111;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_SCHED_CONTROL: GMCAPI_COMMANDS = 196609;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_MMU_FAULT_QUEUED: GMCAPI_COMMANDS = 262145;
+pub const GMCAPI_COMMANDS_GMCAPI_CMD_INVALID: GMCAPI_COMMANDS = 4294967295;
+pub type GMCAPI_COMMANDS = ffi::c_uint;
+pub const GSP_DMA_TARGET_GSP_DMA_TARGET_LOCAL_FB: GSP_DMA_TARGET = 0;
+pub const GSP_DMA_TARGET_GSP_DMA_TARGET_COHERENT_SYSTEM: GSP_DMA_TARGET = 1;
+pub const GSP_DMA_TARGET_GSP_DMA_TARGET_NONCOHERENT_SYSTEM: GSP_DMA_TARGET = 2;
+pub const GSP_DMA_TARGET_GSP_DMA_TARGET_COUNT: GSP_DMA_TARGET = 3;
+pub type GSP_DMA_TARGET = ffi::c_uint;
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GSP_FMC_INIT_PARAMS {
+ pub regkeys: u32_,
+ pub reserved: u32_,
+}
+#[repr(C)]
+#[derive(Debug, Copy, Clone, MaybeZeroable)]
+pub struct GSP_ACR_BOOT_GSP_RM_PARAMS {
+ pub target: GSP_DMA_TARGET,
+ pub gspRmDescSize: u32_,
+ pub gspRmDescOffset: u64_,
+ pub wprCarveoutOffset: u64_,
+ pub wprCarveoutSize: u32_,
+ pub bIsGspRmBoot: u8_,
+ pub bInstInSysMode: u8_,
+ pub bIcuEnabled: u8_,
+ pub bScrubCbcSr: u8_,
+}
+impl Default for GSP_ACR_BOOT_GSP_RM_PARAMS {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+#[repr(C)]
+#[derive(Debug, Copy, Clone, MaybeZeroable)]
+pub struct GSP_RM_PARAMS {
+ pub target: GSP_DMA_TARGET,
+ pub reserved: u32_,
+ pub bootArgsOffset: u64_,
+}
+impl Default for GSP_RM_PARAMS {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+#[repr(C)]
+#[derive(Debug, Copy, Clone, MaybeZeroable)]
+pub struct GSP_SPDM_PARAMS {
+ pub target: GSP_DMA_TARGET,
+ pub payloadBufferSize: u32_,
+ pub payloadBufferOffset: u64_,
+}
+impl Default for GSP_SPDM_PARAMS {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
+pub struct GSP_RM_MEM_PARAMS {
+ pub flushSysmemAddrValLo: u32_,
+ pub flushSysmemAddrValHi: u32_,
+}
+#[repr(C)]
+#[derive(Debug, Copy, Clone, MaybeZeroable)]
+pub struct GSP_FMC_BOOT_PARAMS {
+ pub magic: u32_,
+ pub size: u16_,
+ pub reserved0: u16_,
+ pub reserved1: [u64_; 3usize],
+ pub initParams: GSP_FMC_INIT_PARAMS,
+ pub reserved2: [u64_; 3usize],
+ pub bootGspRmParams: GSP_ACR_BOOT_GSP_RM_PARAMS,
+ pub reserved3: [u64_; 4usize],
+ pub gspRmParams: GSP_RM_PARAMS,
+ pub reserved4: [u64_; 2usize],
+ pub gspSpdmParams: GSP_SPDM_PARAMS,
+ pub reserved5: [u64_; 2usize],
+ pub gspRmMemParams: GSP_RM_MEM_PARAMS,
+ pub reserved6: [u64_; 7usize],
+}
+impl Default for GSP_FMC_BOOT_PARAMS {
+ fn default() -> Self {
+ let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
+ unsafe {
+ ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
+ s.assume_init()
+ }
+ }
+}
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 05/33] gpu: nova-core: regs: add msgq v2 BAR0 register declarations
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (3 preceding siblings ...)
2026-09-18 1:06 ` [PATCH v3 04/33] gpu: nova-core: firmware: add r000 bindings John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:06 ` [PATCH v3 06/33] gpu: nova-core: gsp: ring the GSP doorbell from the queue memory John Hubbard
` (27 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
Msgq is the pair of ring buffers in shared memory through which the
driver and GSP-RM communicate. The command queue carries commands from
the driver to GSP-RM, and the message queue carries messages from GSP-RM
to the driver. Each ring buffer has a write pointer and a read pointer.
Msgq v0, which the r570 firmware uses, keeps both pointers of each ring
buffer in that ring buffer's header, in shared memory. Msgq v2, which
the r000 firmware uses, keeps the four pointers, two per ring buffer, in
four BAR0 registers instead:
NV_PGSP_QUEUE_HEAD command queue write pointer
NV_PGSP_QUEUE_TAIL command queue read pointer
NV_PGSP_MSGQ_HEAD message queue write pointer
NV_PGSP_MSGQ_TAIL message queue read pointer
The GSP numbers its queues 0 through 7, and each numbered queue is one
such pair of ring buffers with its own four pointer registers, so the
hardware has 32 pointer registers in all. Nova-core uses queue 0 only.
A write to NV_PGSP_QUEUE_HEAD also interrupts the GSP. Msgq v0 uses that
write as the doorbell, so nova-core already declares NV_PGSP_QUEUE_HEAD.
Declare the other three, NV_PGSP_QUEUE_TAIL, NV_PGSP_MSGQ_HEAD and
NV_PGSP_MSGQ_TAIL, each as a scalar at its queue 0 offset rather than as
an array of eight.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/regs.rs | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/drivers/gpu/nova-core/gsp/regs.rs b/drivers/gpu/nova-core/gsp/regs.rs
index 3c410d65e8e4..fae7825f0846 100644
--- a/drivers/gpu/nova-core/gsp/regs.rs
+++ b/drivers/gpu/nova-core/gsp/regs.rs
@@ -8,13 +8,32 @@
};
// PGSP
+// The GSP has eight numbered queues, each with its own four msgq pointer registers. nova-core
+// uses queue 0 only, so each register is declared as a scalar at its queue 0 offset.
register! {
base: NovaRegisters;
+ /// Write pointer of the CPU-to-GSP command queue, which the driver advances. A write to this
+ /// register also interrupts the GSP, so this register is also the doorbell.
pub(super) NV_PGSP_QUEUE_HEAD(u32) @ 0x00110c00 {
31:0 address;
}
+
+ /// Read pointer of the CPU-to-GSP command queue, which the GSP advances.
+ pub(super) NV_PGSP_QUEUE_TAIL(u32) @ 0x00110c04 {
+ 31:0 address;
+ }
+
+ /// Write pointer of the GSP-to-CPU message queue, which the GSP advances.
+ pub(super) NV_PGSP_MSGQ_HEAD(u32) @ 0x00110c80 {
+ 31:0 address;
+ }
+
+ /// Read pointer of the GSP-to-CPU message queue, which the driver advances.
+ pub(super) NV_PGSP_MSGQ_TAIL(u32) @ 0x00110c84 {
+ 31:0 address;
+ }
}
// PBUS
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 06/33] gpu: nova-core: gsp: ring the GSP doorbell from the queue memory
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (4 preceding siblings ...)
2026-09-18 1:06 ` [PATCH v3 05/33] gpu: nova-core: regs: add msgq v2 BAR0 register declarations John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:06 ` [PATCH v3 07/33] gpu: nova-core: gsp: make command allocation generic over the header John Hubbard
` (26 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
A command reaches GSP-RM in two steps. The driver advances the command
queue's write pointer, which publishes the command, and then writes
NV_PGSP_QUEUE_HEAD, which interrupts the GSP so that it reads the queue.
That register write is the doorbell. In msgq v0, the write pointer is in
the queue's shared-memory header, so the two steps are two writes. In
msgq v2, the write pointer is NV_PGSP_QUEUE_HEAD itself, so one register
write does both.
Nova-core split the two steps across two types. The queue memory type
advanced the write pointer in shared memory, and the command queue type
then rang the doorbell through a BAR0 mapping that it held for that one
purpose. The type that advanced the pointer had no BAR0 mapping, so it
could not have made the msgq v2 pointer advance.
Move the BAR0 mapping into the queue memory type, and ring the doorbell
at the end of the operation that advances the write pointer. The switch
to msgq v2 then replaces that operation's two writes with the one
register write. The switch also moves the command queue's read pointer
and the message queue's two pointers into BAR0 registers, and the queue
memory type is where those register reads and writes belong as well.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/cmdq.rs | 47 ++++++++++++++++---------------
1 file changed, 24 insertions(+), 23 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index 9250d596a3e4..f4545e3b52b3 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -239,11 +239,16 @@ unsafe impl FromBytes for GspMem {}
/// pointer and the GSP read pointer. This region is returned by [`Self::driver_write_area`].
/// * The driver owns (i.e. can read from) the part of the GSP message queue between the CPU read
/// pointer and the GSP write pointer. This region is returned by [`Self::driver_read_area`].
-struct DmaGspMem<'a>(Coherent<'a, GspMem>);
+struct DmaGspMem<'a> {
+ /// The queues, mapped for the GSP.
+ mem: Coherent<'a, GspMem>,
+ /// MMIO mapping of PCI BAR0, for the doorbell register.
+ bar: Bar0<'a>,
+}
impl<'a> DmaGspMem<'a> {
/// Allocate a new instance and map it for `dev`.
- fn new(dev: &'a device::Device<device::Bound>) -> Result<Self> {
+ fn new(dev: &'a device::Device<device::Bound>, bar: Bar0<'a>) -> Result<Self> {
const MSGQ_SIZE: u32 = num::usize_into_u32::<{ size_of::<Msgq>() }>();
const RX_HDR_OFF: u32 = num::usize_into_u32::<{ mem::offset_of!(Msgq, rx) }>();
@@ -254,7 +259,7 @@ fn new(dev: &'a device::Device<device::Bound>) -> Result<Self> {
let gsp_mem: Coherent<'_, _> = gsp_mem.into();
PteArray::init(io_project!(gsp_mem, .ptes), gsp_mem.dma_address())?;
- Ok(Self(gsp_mem))
+ Ok(Self { mem: gsp_mem, bar })
}
/// Returns the region of the CPU message queue that the driver is currently allowed to write
@@ -267,7 +272,7 @@ fn new(dev: &'a device::Device<device::Bound>) -> Result<Self> {
let rx = self.gsp_read_ptr();
// Pointer to the first entry of the CPU message queue.
- let data = ptr::project!(mut self.0.as_mut_ptr(), .cpuq.msgq.data[build: 0]);
+ let data = ptr::project!(mut self.mem.as_mut_ptr(), .cpuq.msgq.data[build: 0]);
let (tail_end, wrap_end) = if rx == 0 {
// The write area is non-wrapping, and stops at the second-to-last entry of the command
@@ -329,7 +334,7 @@ fn driver_write_area_size(&self) -> usize {
let rx = self.cpu_read_ptr();
// Pointer to the first entry of the GSP message queue.
- let data = ptr::project!(self.0.as_ptr(), .gspq.msgq.data[build: 0]);
+ let data = ptr::project!(self.mem.as_ptr(), .gspq.msgq.data[build: 0]);
let (tail_end, wrap_end) = if rx <= tx {
// Read area is non-wrapping and stops right before `tx`.
@@ -413,7 +418,7 @@ fn allocate_command(&mut self, size: usize, timeout: Delta) -> Result<GspCommand
//
// - The returned value is within `0..MSGQ_NUM_PAGES`.
fn gsp_write_ptr(&self) -> u32 {
- let ptr = MsgqTxHeader::write_ptr(io_project!(self.0, .gspq.tx)) % MSGQ_NUM_PAGES;
+ let ptr = MsgqTxHeader::write_ptr(io_project!(self.mem, .gspq.tx)) % MSGQ_NUM_PAGES;
// ORDERING: LOAD->LOAD ordering needed to order `gsp_write_ptr` read before data read.
dma_mb(Read);
@@ -427,7 +432,7 @@ fn gsp_write_ptr(&self) -> u32 {
//
// - The returned value is within `0..MSGQ_NUM_PAGES`.
fn gsp_read_ptr(&self) -> u32 {
- let ptr = MsgqRxHeader::read_ptr(io_project!(self.0, .gspq.rx)) % MSGQ_NUM_PAGES;
+ let ptr = MsgqRxHeader::read_ptr(io_project!(self.mem, .gspq.rx)) % MSGQ_NUM_PAGES;
// ORDERING: LOAD->STORE ordering needed to order `gsp_read_ptr` read before data write.
dma_mb(Full);
@@ -441,7 +446,7 @@ fn gsp_read_ptr(&self) -> u32 {
//
// - The returned value is within `0..MSGQ_NUM_PAGES`.
fn cpu_read_ptr(&self) -> u32 {
- MsgqRxHeader::read_ptr(io_project!(self.0, .cpuq.rx)) % MSGQ_NUM_PAGES
+ MsgqRxHeader::read_ptr(io_project!(self.mem, .cpuq.rx)) % MSGQ_NUM_PAGES
}
// Informs the GSP that it can send `elem_count` new pages into the message queue.
@@ -449,7 +454,7 @@ fn advance_cpu_read_ptr(&mut self, elem_count: u32) {
// ORDERING: LOAD->STORE ordering needed to order `cpu_read_ptr` write after data read.
dma_mb(Full);
- let rx = io_project!(self.0, .cpuq.rx);
+ let rx = io_project!(self.mem, .cpuq.rx);
let rptr = MsgqRxHeader::read_ptr(rx).wrapping_add(elem_count) % MSGQ_NUM_PAGES;
MsgqRxHeader::set_read_ptr(rx, rptr)
}
@@ -460,17 +465,22 @@ fn advance_cpu_read_ptr(&mut self, elem_count: u32) {
//
// - The returned value is within `0..MSGQ_NUM_PAGES`.
fn cpu_write_ptr(&self) -> u32 {
- MsgqTxHeader::write_ptr(io_project!(self.0, .cpuq.tx)) % MSGQ_NUM_PAGES
+ MsgqTxHeader::write_ptr(io_project!(self.mem, .cpuq.tx)) % MSGQ_NUM_PAGES
}
- // Informs the GSP that it can process `elem_count` new pages from the command queue.
+ // Publishes `elem_count` more pages of the command queue to the GSP and rings the doorbell.
fn advance_cpu_write_ptr(&mut self, elem_count: u32) {
// ORDERING: STORE->STORE ordering needed to order `cpu_write_ptr` write after data write.
dma_mb(Write);
- let tx = io_project!(self.0, .cpuq.tx);
+ let tx = io_project!(self.mem, .cpuq.tx);
let wptr = MsgqTxHeader::write_ptr(tx).wrapping_add(elem_count) % MSGQ_NUM_PAGES;
MsgqTxHeader::set_write_ptr(tx, wptr);
+
+ // A write to the head register interrupts the GSP. The pointer itself is in the
+ // shared-memory header, so the value written does not matter.
+ self.bar
+ .write_reg(regs::NV_PGSP_QUEUE_HEAD::zeroed().with_address(0u32));
}
}
@@ -535,13 +545,12 @@ pub(crate) fn new(
bar: Bar0<'cmdq>,
) -> impl PinInit<Self, Error> + 'cmdq {
pin_init_scope(move || {
- let gsp_mem = DmaGspMem::new(dev)?;
+ let gsp_mem = DmaGspMem::new(dev, bar)?;
Ok(try_pin_init!(Self {
- dma_addr: gsp_mem.0.dma_address(),
+ dma_addr: gsp_mem.mem.dma_address(),
inner <- new_mutex!(CmdqInner {
dev,
- bar,
gsp_mem,
seq: 0,
poisoned: Cell::new(false),
@@ -563,11 +572,6 @@ fn calculate_checksum<T: Iterator<Item = u8>>(it: T) -> u32 {
((sum64 >> 32) as u32) ^ (sum64 as u32)
}
- /// Notifies the GSP that we have updated the command queue pointers.
- fn notify_gsp(bar: Bar0<'_>) {
- bar.write_reg(regs::NV_PGSP_QUEUE_HEAD::zeroed().with_address(0u32));
- }
-
/// Sends `command` to the GSP and waits for the reply.
///
/// Events that arrive before the reply are logged and consumed.
@@ -654,8 +658,6 @@ pub(crate) fn drain(&self) -> Result {
struct CmdqInner<'a> {
/// Device this command queue belongs to.
dev: &'a device::Device,
- /// MMIO mapping of PCI BAR0, for writing the GSP doorbell.
- bar: Bar0<'a>,
/// Current command sequence number.
seq: u32,
/// Set once a message fails framing or checksum validation. Every later receive fails, since
@@ -736,7 +738,6 @@ fn send_single_command<M>(&mut self, command: M) -> Result
let elem_count = dst.header.element_count();
self.seq += 1;
self.gsp_mem.advance_cpu_write_ptr(elem_count);
- Cmdq::notify_gsp(self.bar);
Ok(())
}
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 07/33] gpu: nova-core: gsp: make command allocation generic over the header
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (5 preceding siblings ...)
2026-09-18 1:06 ` [PATCH v3 06/33] gpu: nova-core: gsp: ring the GSP doorbell from the queue memory John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:06 ` [PATCH v3 08/33] gpu: nova-core: gsp: compute the queue regions from a count and a slot John Hubbard
` (25 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The r000 firmware accepts GMC (GPU Management Controller) API commands,
which open with a different element header from RPC commands. The two
kinds of command go into the same command queue, so the wait for free
space, the maximum element size and the split of a command across the
end of the ring buffer are the same for both.
Nova-core's queue allocation named the RPC element header directly, so
there was no way to reserve queue space for a command with another
header.
Make the allocation generic over the header type, along with the handle
that it returns for the command. The one existing caller passes an RPC
element header and is unchanged.
No functional changes.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/cmdq.rs | 26 ++++++++++++++++----------
1 file changed, 16 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index f4545e3b52b3..80e6e79c5f3c 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -129,8 +129,8 @@ fn init_variable_payload(
Ok(())
}
- /// Total size of the command (including its variable-length payload) without the
- /// [`GspMsgElement`] header.
+ /// Total size of the command, its variable-length payload included, without the headers that
+ /// open the queue element.
fn size(&self) -> usize {
size_of::<Self::Command>() + self.variable_payload_len()
}
@@ -374,14 +374,19 @@ fn driver_write_area_size(&self) -> usize {
///
/// - `EMSGSIZE` if the command is larger than [`GSP_MSG_QUEUE_ELEMENT_SIZE_MAX`].
/// - `ETIMEDOUT` if space does not become available within the timeout.
- /// - `EIO` if the command header is not properly aligned.
- fn allocate_command(&mut self, size: usize, timeout: Delta) -> Result<GspCommand<'_>> {
- if size_of::<GspMsgElement>() + size > GSP_MSG_QUEUE_ELEMENT_SIZE_MAX {
+ /// - `EIO` if the first free slot is too short for the headers of type `H`, or misaligned for
+ /// them.
+ fn allocate_command<H: FromBytes + AsBytes>(
+ &mut self,
+ size: usize,
+ timeout: Delta,
+ ) -> Result<GspCommand<'_, H>> {
+ if size_of::<H>() + size > GSP_MSG_QUEUE_ELEMENT_SIZE_MAX {
return Err(EMSGSIZE);
}
read_poll_timeout(
|| Ok(self.driver_write_area_size()),
- |available_bytes| *available_bytes >= size_of::<GspMsgElement>() + size,
+ |available_bytes| *available_bytes >= size_of::<H>() + size,
Delta::from_micros(1),
timeout,
)?;
@@ -393,8 +398,7 @@ fn allocate_command(&mut self, size: usize, timeout: Delta) -> Result<GspCommand
(slice_1.as_flattened_mut(), slice_2.as_flattened_mut())
};
- // Extract area for the `GspMsgElement`.
- let (header, slice_1) = GspMsgElement::from_bytes_mut_prefix(slice_1).ok_or(EIO)?;
+ let (header, slice_1) = H::from_bytes_mut_prefix(slice_1).ok_or(EIO)?;
// Create the contents area.
let (slice_1, slice_2) = if slice_1.len() > size {
@@ -486,10 +490,12 @@ fn advance_cpu_write_ptr(&mut self, elem_count: u32) {
/// A command ready to be sent on the command queue.
///
+/// `H` is the type of the headers that open the element, such as [`GspMsgElement`] for an RM RPC.
+///
/// This is the type returned by [`DmaGspMem::allocate_command`].
-struct GspCommand<'a> {
+struct GspCommand<'a, H> {
// Writable reference to the header of the command.
- header: &'a mut GspMsgElement,
+ header: &'a mut H,
// Writable slices to the contents of the command. The second slice is zero unless the command
// loops over the command queue.
contents: (&'a mut [u8], &'a mut [u8]),
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 08/33] gpu: nova-core: gsp: compute the queue regions from a count and a slot
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (6 preceding siblings ...)
2026-09-18 1:06 ` [PATCH v3 07/33] gpu: nova-core: gsp: make command allocation generic over the header John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:06 ` [PATCH v3 09/33] gpu: nova-core: add GMC API message types John Hubbard
` (24 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The r000 firmware uses msgq v2, the queue layout that keeps the queue
pointers in BAR0 registers as counts that do not wrap at the ring size.
The r570 firmware's layout keeps the pointers in shared memory as
indices into the ring. The two layouts differ in where a pointer is
read and in how the size of the region that the driver may write, and
of the region that it may read, follows from a queue's write pointer
and read pointer. Splitting a region across the end of the ring is the
same in both, and whether a region wraps follows from the order of the
two pointers.
The functions for the writable region and for the readable region each
branched on the order of the write pointer and the read pointer. Each
branch chose where the two slices ended, and the function then built
the slices with open-coded pointer arithmetic. The SAFETY comments
argued the slice bounds branch by branch, so the switch to msgq v2
would have had to rewrite the branches and the argument along with the
pointer rules.
Compute the number of slots in a region and its start slot once, and
split the ring at the start slot. The first slice ends at the end of
the region or at the end of the ring, whichever comes first, and the
second slice carries the rest.
No functional changes.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/cmdq.rs | 120 ++++++++++--------------------
1 file changed, 41 insertions(+), 79 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index 80e6e79c5f3c..a1c9b7cce255 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -262,107 +262,69 @@ fn new(dev: &'a device::Device<device::Bound>, bar: Bar0<'a>) -> Result<Self> {
Ok(Self { mem: gsp_mem, bar })
}
- /// Returns the region of the CPU message queue that the driver is currently allowed to write
- /// to.
+ /// Returns the region of the CPU message queue that the driver may write to.
///
- /// As the message queue is a circular buffer, the region may be discontiguous in memory. In
- /// that case the second slice will have a non-zero length.
+ /// The ring wraps, so the region comes as two slices, and the second is empty unless the
+ /// region crosses the end of the ring.
fn driver_write_area(&mut self) -> (&mut [[u8; GSP_PAGE_SIZE]], &mut [[u8; GSP_PAGE_SIZE]]) {
- let tx = self.cpu_write_ptr();
- let rx = self.gsp_read_ptr();
+ let avail = num::u32_as_usize(self.free_slots());
+ let w_slot = num::u32_as_usize(self.cpu_write_ptr());
// Pointer to the first entry of the CPU message queue.
let data = ptr::project!(mut self.mem.as_mut_ptr(), .cpuq.msgq.data[build: 0]);
- let (tail_end, wrap_end) = if rx == 0 {
- // The write area is non-wrapping, and stops at the second-to-last entry of the command
- // queue (to leave the last one empty).
- (MSGQ_NUM_PAGES - 1, 0)
- } else if rx <= tx {
- // The write area wraps and continues until `rx - 1`.
- (MSGQ_NUM_PAGES, rx - 1)
- } else {
- // The write area doesn't wrap and stops at `rx - 1`.
- (rx - 1, 0)
- };
-
// SAFETY:
- // - `data` was created from a valid pointer, and `rx` and `tx` are in the
- // `0..MSGQ_NUM_PAGES` range per the invariants of `cpu_write_ptr` and `gsp_read_ptr`,
- // thus the created slices are valid.
- // - The area starting at `tx` and ending at `rx - 2` modulo `MSGQ_NUM_PAGES`,
- // inclusive, belongs to the driver for writing and is not accessed concurrently by
- // the GSP.
- // - The caller holds a reference to `self` for as long as the returned slices are live,
- // meaning the CPU write pointer cannot be advanced and thus that the returned area
- // remains exclusive to the CPU for the duration of the slices.
- // - The created slices point to non-overlapping sub-ranges of `data` in all
- // branches (in the `rx <= tx` case, the second slice ends at `rx - 1` which is strictly
- // less than `tx` where the first slice starts; in the other cases the second slice is
- // empty), so creating two `&mut` references from them does not violate aliasing rules.
- unsafe {
- (
- core::slice::from_raw_parts_mut(
- data.add(num::u32_as_usize(tx)),
- num::u32_as_usize(tail_end - tx),
- ),
- core::slice::from_raw_parts_mut(data, num::u32_as_usize(wrap_end)),
- )
- }
+ // - `data` points to the `MSGQ_NUM_PAGES` initialized entries of the CPU message queue.
+ // - The returned slices cover the `avail` free slots from the write pointer on, which the
+ // GSP does not read until `advance_cpu_write_ptr` publishes them.
+ // - `split_at_mut` gives two non-overlapping halves, and the `&mut self` borrow lasts as
+ // long as the returned slices, so that no other call hands out the same region while
+ // they live.
+ let data =
+ unsafe { core::slice::from_raw_parts_mut(data, num::u32_as_usize(MSGQ_NUM_PAGES)) };
+ let (before_w, after_w) = data.split_at_mut(w_slot);
+
+ let in_after = avail.min(after_w.len());
+ let in_before = avail - in_after;
+ (&mut after_w[..in_after], &mut before_w[..in_before])
}
- /// Returns the size of the region of the CPU message queue that the driver is currently allowed
- /// to write to, in bytes.
- fn driver_write_area_size(&self) -> usize {
+ /// Returns the number of command queue slots that the driver may still write.
+ fn free_slots(&self) -> u32 {
let tx = self.cpu_write_ptr();
let rx = self.gsp_read_ptr();
- // `rx` and `tx` are both in `0..MSGQ_NUM_PAGES` per the invariants of `gsp_read_ptr` and
- // `cpu_write_ptr`. The minimum value case is where `rx == 0` and `tx == MSGQ_NUM_PAGES -
- // 1`, which gives `0 + MSGQ_NUM_PAGES - (MSGQ_NUM_PAGES - 1) - 1 == 0`.
- let slots = (rx + MSGQ_NUM_PAGES - tx - 1) % MSGQ_NUM_PAGES;
- num::u32_as_usize(slots) * GSP_PAGE_SIZE
+ // One slot always stays empty, so that a full ring and an empty ring differ in their
+ // pointers. `tx` is below `MSGQ_NUM_PAGES`, so the subtraction does not underflow.
+ (rx + MSGQ_NUM_PAGES - tx - 1) % MSGQ_NUM_PAGES
}
- /// Returns the region of the GSP message queue that the driver is currently allowed to read
- /// from.
- ///
- /// As the message queue is a circular buffer, the region may be discontiguous in memory. In
- /// that case the second slice will have a non-zero length.
+ /// Returns the number of bytes that the driver can still write to the command queue.
+ fn driver_write_area_size(&self) -> usize {
+ num::u32_as_usize(self.free_slots()) * GSP_PAGE_SIZE
+ }
+
+ /// Returns the region of the GSP message queue that the driver may read, as two slices
+ /// because the ring wraps.
fn driver_read_area(&self) -> (&[[u8; GSP_PAGE_SIZE]], &[[u8; GSP_PAGE_SIZE]]) {
let tx = self.gsp_write_ptr();
let rx = self.cpu_read_ptr();
+ let avail = num::u32_as_usize((tx + MSGQ_NUM_PAGES - rx) % MSGQ_NUM_PAGES);
+ let r_slot = num::u32_as_usize(rx);
// Pointer to the first entry of the GSP message queue.
let data = ptr::project!(self.mem.as_ptr(), .gspq.msgq.data[build: 0]);
- let (tail_end, wrap_end) = if rx <= tx {
- // Read area is non-wrapping and stops right before `tx`.
- (tx, 0)
- } else {
- // Read area is wrapping and stops right before `tx`.
- (MSGQ_NUM_PAGES, tx)
- };
-
// SAFETY:
- // - `data` was created from a valid pointer, and `rx` and `tx` are in the
- // `0..MSGQ_NUM_PAGES` range per the invariants of `gsp_write_ptr` and `cpu_read_ptr`,
- // thus the created slices are valid.
- // - The area starting at `rx` and ending at `tx - 1` modulo `MSGQ_NUM_PAGES`,
- // inclusive, belongs to the driver for reading and is not accessed concurrently by
- // the GSP.
- // - The caller holds a reference to `self` for as long as the returned slices are live,
- // meaning the CPU read pointer cannot be advanced and thus that the returned area
- // remains exclusive to the CPU for the duration of the slices.
- unsafe {
- (
- core::slice::from_raw_parts(
- data.add(num::u32_as_usize(rx)),
- num::u32_as_usize(tail_end - rx),
- ),
- core::slice::from_raw_parts(data, num::u32_as_usize(wrap_end)),
- )
- }
+ // - `data` points to the `MSGQ_NUM_PAGES` initialized entries of the GSP message queue.
+ // - The returned slices cover the `avail` slots that the GSP has already written. The GSP
+ // does not write them again until `advance_cpu_read_ptr` releases them.
+ let data = unsafe { core::slice::from_raw_parts(data, num::u32_as_usize(MSGQ_NUM_PAGES)) };
+ let (before_r, after_r) = data.split_at(r_slot);
+
+ let in_after = avail.min(after_r.len());
+ let in_before = avail - in_after;
+ (&after_r[..in_after], &before_r[..in_before])
}
/// Allocates a region on the command queue that is large enough to send a command of `size`
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 09/33] gpu: nova-core: add GMC API message types
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (7 preceding siblings ...)
2026-09-18 1:06 ` [PATCH v3 08/33] gpu: nova-core: gsp: compute the queue regions from a count and a slot John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:06 ` [PATCH v3 10/33] gpu: nova-core: add GMC send path John Hubbard
` (23 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
With the r570 firmware, every element on the GSP queues is an RM RPC
message, and the driver builds and decodes only that form. The r000
firmware adds a second kind of element on the same queues, the GMC API
message. GMC is the GPU Management Controller, and its API carries
GSP-RM's boot and management commands. GSP-RM's newer interfaces use the
GMC API rather than RPC messages, its ABI is stable, and on r000 the
boot protocol itself runs over it.
Every element opens with the same two transport headers. The first is
an MCTP header. MCTP is the Management Component Transport Protocol,
and this is its packet header. The second is an NVDM header, the NVIDIA
vendor-defined message header, whose type field selects the header that
follows: the RPC header or the GMC API header. A GMC API header has a
fixed size and carries the command id, the payload size and a sequence
number. Its last word is the largest response that the sender accepts,
and in a response the same word carries the status.
Nova-core's queue types named the RPC header directly, so there was no
way to build a GMC element or to tell one from an RPC element.
Add the types that a GMC element needs:
* The queue element header, which holds the two transport headers and
the element's lengths, as a type of its own.
* The two NVDM type values that the GSP queues use, one for RPC and one
for GMC.
* The GMC API header, and the element header that opens a GMC element
with it.
The GMC API header is written out field by field rather than wrapped
around the struct from the generated bindings. The bindings put the
request and response fields of that header in a union, so reading the
status of a response through them requires unsafe code. A struct
written out with the same layout reads the status without unsafe code,
and static assertions check that its layout matches the bindings.
The types have no user yet. The following patches add the send and
receive paths for GMC elements, and the driver keeps booting r570 over
RPC until the patch that switches firmware.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/fw.rs | 220 ++++++++++++++++++++++++++++++++
drivers/gpu/nova-core/mctp.rs | 4 +
2 files changed, 224 insertions(+)
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index 285c23cea771..1aca5ca82764 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -50,6 +50,11 @@
cmdq::Cmdq, //
GSP_PAGE_SIZE,
},
+ mctp::{
+ MctpHeader,
+ NvdmHeader,
+ NvdmType, //
+ },
num::{
self,
FromSafeCast, //
@@ -889,6 +894,221 @@ unsafe impl AsBytes for GspMsgElement {}
// are valid.
unsafe impl FromBytes for GspMsgElement {}
+/// First word of every queue element: `"MCTP"` in ASCII.
+const MCTP_MAGIC: u32 = 0x4D43_5450;
+
+/// The queue element header that opens every queue element, whatever kind of message follows.
+///
+/// It holds an MCTP (Management Component Transport Protocol) header and an NVDM (NVIDIA
+/// vendor-defined message) header. The NVDM type selects the message header that follows: the RPC
+/// header or the GMC (GPU Management Controller) API header.
+///
+/// ```text
+/// +------------------------------------+
+/// | queue element header | QueueElementHeader: magic, element length, MCTP
+/// | | header, NVDM header, message length
+/// +------------------------------------+
+/// | message header | the RPC header or the GMC API header. The NVDM
+/// +------------------------------------+ type selects between the two.
+/// | payload | command-specific data
+/// +------------------------------------+
+/// ```
+#[repr(C)]
+pub(crate) struct QueueElementHeader {
+ magic: u32,
+ /// Length of the whole element: the queue element header, the message header and the
+ /// payload. Open RM calls it `mctpPayloadSize`.
+ element_len: u32,
+ mctp: MctpHeader,
+ nvdm: NvdmHeader,
+ /// Length of the message header and the payload, the queue element header excluded. Open RM
+ /// calls it `nvdmPayloadSize`.
+ message_len: u32,
+ reserved: u32,
+}
+
+static_assert!(
+ core::mem::offset_of!(QueueElementHeader, magic)
+ == core::mem::offset_of!(r000_00::GSP_MSG_QUEUE_ELEMENT, mctpMagic)
+);
+static_assert!(
+ core::mem::offset_of!(QueueElementHeader, element_len)
+ == core::mem::offset_of!(r000_00::GSP_MSG_QUEUE_ELEMENT, mctpPayloadSize)
+);
+static_assert!(
+ core::mem::offset_of!(QueueElementHeader, mctp)
+ == core::mem::offset_of!(r000_00::GSP_MSG_QUEUE_ELEMENT, mctpHeader)
+);
+static_assert!(
+ core::mem::offset_of!(QueueElementHeader, nvdm)
+ == core::mem::offset_of!(r000_00::GSP_MSG_QUEUE_ELEMENT, nvdmHeader)
+);
+
+#[expect(dead_code)]
+impl QueueElementHeader {
+ /// Builds the queue element header of an element whose message header and payload together
+ /// take `message_len` bytes.
+ ///
+ /// # Errors
+ ///
+ /// - `EOVERFLOW` if a length does not fit its 32-bit field.
+ fn new(nvdm: NvdmType, message_len: usize) -> Result<Self> {
+ Ok(Self {
+ magic: MCTP_MAGIC,
+ element_len: size_of::<Self>()
+ .checked_add(message_len)
+ .ok_or(EOVERFLOW)?
+ .try_into()
+ .map_err(|_| EOVERFLOW)?,
+ mctp: MctpHeader::single_packet(),
+ nvdm: NvdmHeader::new(nvdm),
+ message_len: message_len.try_into().map_err(|_| EOVERFLOW)?,
+ reserved: 0,
+ })
+ }
+
+ /// Returns the length of the whole element, the queue element header included.
+ fn element_len(&self) -> usize {
+ num::u32_as_usize(self.element_len)
+ }
+
+ /// Returns the length of the payload that follows a message header of `message_header_len`
+ /// bytes.
+ fn payload_len(&self, message_header_len: usize) -> usize {
+ num::u32_as_usize(self.message_len).saturating_sub(message_header_len)
+ }
+
+ /// Returns the number of queue slots that this element occupies.
+ fn element_count(&self) -> u32 {
+ self.element_len
+ .div_ceil(num::usize_into_u32::<GSP_PAGE_SIZE>())
+ }
+}
+
+// SAFETY: All fields are integer types or transparent wrappers over one, with no padding.
+unsafe impl AsBytes for QueueElementHeader {}
+
+// SAFETY: All fields are integer types for which all bit patterns are valid.
+unsafe impl FromBytes for QueueElementHeader {}
+
+/// Header of a GMC API message.
+#[repr(C)]
+#[derive(Zeroable)]
+pub(crate) struct GmcApiHeader {
+ /// Command id in the low three bytes, flags in the high byte.
+ pub(crate) command: u32,
+ /// Payload size in bytes.
+ pub(crate) size: u32,
+ /// Sequence number that GSP-RM copies from a request into its response.
+ pub(crate) sequence: u64,
+ /// In a request, the largest response that the sender accepts. In a response, the `NV_STATUS`.
+ pub(crate) max_resp_or_status: u32,
+ reserved: [u32; 5],
+}
+
+static_assert!(size_of::<GmcApiHeader>() == size_of::<r000_00::GMCAPI_HEADER>());
+static_assert!(
+ core::mem::offset_of!(GmcApiHeader, command)
+ == core::mem::offset_of!(r000_00::GMCAPI_HEADER, command)
+);
+static_assert!(
+ core::mem::offset_of!(GmcApiHeader, size)
+ == core::mem::offset_of!(r000_00::GMCAPI_HEADER, size)
+);
+static_assert!(
+ core::mem::offset_of!(GmcApiHeader, sequence)
+ == core::mem::offset_of!(r000_00::GMCAPI_HEADER, sequence)
+);
+static_assert!(
+ core::mem::offset_of!(GmcApiHeader, max_resp_or_status)
+ == core::mem::offset_of!(r000_00::GMCAPI_HEADER, __bindgen_anon_1)
+);
+static_assert!(
+ core::mem::offset_of!(GmcApiHeader, reserved)
+ == core::mem::offset_of!(r000_00::GMCAPI_HEADER, reserved)
+);
+
+impl GmcApiHeader {
+ /// Returns the `NV_STATUS` that a response carries.
+ ///
+ /// The value is meaningful only on a response, which GSP-RM marks with a flag in the command
+ /// word. In a request, the same word holds the largest response that the sender accepts.
+ #[expect(dead_code)]
+ pub(crate) fn status(&self) -> u32 {
+ self.max_resp_or_status
+ }
+}
+
+// SAFETY: All fields are integer types with no uninitialized padding bytes.
+unsafe impl AsBytes for GmcApiHeader {}
+
+// SAFETY: All fields are integer types for which all bit patterns are valid.
+unsafe impl FromBytes for GmcApiHeader {}
+
+/// The headers that open a GMC API queue element: the queue element header and the GMC API
+/// header.
+#[repr(C)]
+pub(crate) struct GspGmcMsgElement {
+ element_header: QueueElementHeader,
+ pub(crate) gmc: GmcApiHeader,
+}
+
+// `AsBytes` below requires that no padding separates the two headers.
+static_assert!(
+ size_of::<GspGmcMsgElement>() == size_of::<QueueElementHeader>() + size_of::<GmcApiHeader>()
+);
+
+#[expect(dead_code)]
+impl GspGmcMsgElement {
+ /// Creates the queue element header and the GMC API header of a request that carries
+ /// `payload_size` bytes of payload.
+ ///
+ /// `max_response_size` is the largest response that the sender accepts, and zero for a request
+ /// that GSP-RM does not answer.
+ ///
+ /// # Errors
+ ///
+ /// - `EOVERFLOW` if a length does not fit its 32-bit field.
+ pub(crate) fn init(
+ command_id: u32,
+ sequence: u64,
+ payload_size: usize,
+ max_response_size: u32,
+ ) -> impl Init<Self, Error> {
+ try_init!(GspGmcMsgElement {
+ element_header: QueueElementHeader::new(
+ NvdmType::GmcApi,
+ size_of::<GmcApiHeader>()
+ .checked_add(payload_size)
+ .ok_or(EOVERFLOW)?,
+ )?,
+ gmc: GmcApiHeader {
+ command: command_id,
+ size: payload_size.try_into().map_err(|_| EOVERFLOW)?,
+ sequence,
+ max_resp_or_status: max_response_size,
+ reserved: [0; 5],
+ },
+ })
+ }
+
+ /// Returns the length of the whole element, both headers included.
+ pub(crate) fn length(&self) -> usize {
+ self.element_header.element_len()
+ }
+
+ /// Returns the number of queue slots that this element occupies.
+ pub(crate) fn element_count(&self) -> u32 {
+ self.element_header.element_count()
+ }
+}
+
+// SAFETY: All fields are integer types with no uninitialized padding bytes.
+unsafe impl AsBytes for GspGmcMsgElement {}
+
+// SAFETY: All fields are integer types for which all bit patterns are valid.
+unsafe impl FromBytes for GspGmcMsgElement {}
+
/// Arguments for GSP startup.
#[repr(transparent)]
#[derive(Zeroable)]
diff --git a/drivers/gpu/nova-core/mctp.rs b/drivers/gpu/nova-core/mctp.rs
index a3872a740233..0eb964c74e32 100644
--- a/drivers/gpu/nova-core/mctp.rs
+++ b/drivers/gpu/nova-core/mctp.rs
@@ -28,6 +28,10 @@ pub(crate) enum NvdmType with TryFrom<Bounded<u32, 8>> {
Cot = 0x14,
/// FSP command response.
FspResponse = 0x15,
+ /// RPC message to or from GSP-RM.
+ RmRpc = 0x25,
+ /// GMC (GPU Management Controller) API message to or from GSP-RM.
+ GmcApi = 0x26,
}
}
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 10/33] gpu: nova-core: add GMC send path
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (8 preceding siblings ...)
2026-09-18 1:06 ` [PATCH v3 09/33] gpu: nova-core: add GMC API message types John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:06 ` [PATCH v3 11/33] gpu: nova-core: add GMC transport receive path John Hubbard
` (22 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The r000 boot protocol has the driver send GSP_INIT, the request that
carries the driver's configuration to GSP-RM, as a GMC command. GMC and
RPC commands share the command queue and the RPC sequence counter, and
a GMC element, unlike an RPC element, carries no checksum.
Nova-core's send path filled in RPC elements only, so there was no way
to send a GMC command.
Add a send path for GMC commands. It has no caller yet. A following
patch adds the GSP_INIT sender.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/cmdq.rs | 43 +++++++++++++++++++++++++++++++
drivers/gpu/nova-core/gsp/fw.rs | 1 -
2 files changed, 43 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index a1c9b7cce255..c0c3d8596b30 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -54,6 +54,7 @@
driver::Bar0,
gsp::{
fw::{
+ GspGmcMsgElement,
GspMsgElement,
MsgFunction,
MsgqRxHeader,
@@ -749,6 +750,48 @@ fn poison(&self, reason: fmt::Arguments<'_>) -> Error {
EIO
}
+ /// Sends a GMC API request to the GSP.
+ ///
+ /// `payload` follows the GMC API header in the element, and `max_response_size` is the largest
+ /// response that the caller accepts. The request carries the next sequence number, which GSP-RM
+ /// copies into its response. The number is consumed even if the send fails.
+ ///
+ /// # Errors
+ ///
+ /// Errors from [`DmaGspMem::allocate_command`] are propagated as-is.
+ #[expect(dead_code)]
+ fn send_gmc(&mut self, command_id: u32, payload: &[u8], max_response_size: u32) -> Result {
+ let seq = self.seq;
+ self.seq = self.seq.wrapping_add(1);
+
+ let dst = self
+ .gsp_mem
+ .allocate_command::<GspGmcMsgElement>(payload.len(), Self::ALLOCATE_TIMEOUT)?;
+
+ let msg_element =
+ GspGmcMsgElement::init(command_id, u64::from(seq), payload.len(), max_response_size);
+ // SAFETY: `dst.header` is a valid reference, and not written if the initializer fails.
+ unsafe {
+ pin_init::raw_try_init(core::ptr::from_mut(dst.header), msg_element)?;
+ }
+
+ SBufferIter::new_writer([&mut dst.contents.0[..], &mut dst.contents.1[..]])
+ .write_all(payload)?;
+
+ dev_dbg!(
+ &self.dev,
+ "GSP GMC: send: seq# {}, command_id=0x{:x}, length=0x{:x}\n",
+ seq,
+ command_id,
+ dst.header.length(),
+ );
+
+ let elem_count = dst.header.element_count();
+ self.gsp_mem.advance_cpu_write_ptr(elem_count);
+
+ Ok(())
+ }
+
/// Wait for a message to become available on the message queue.
///
/// This works purely at the transport layer and does not interpret or validate the message
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index 1aca5ca82764..75bc3d66f71f 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -1058,7 +1058,6 @@ pub(crate) struct GspGmcMsgElement {
size_of::<GspGmcMsgElement>() == size_of::<QueueElementHeader>() + size_of::<GmcApiHeader>()
);
-#[expect(dead_code)]
impl GspGmcMsgElement {
/// Creates the queue element header and the GMC API header of a request that carries
/// `payload_size` bytes of payload.
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 11/33] gpu: nova-core: add GMC transport receive path
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (9 preceding siblings ...)
2026-09-18 1:06 ` [PATCH v3 10/33] gpu: nova-core: add GMC send path John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:06 ` [PATCH v3 12/33] gpu: nova-core: gsp: add GMC dispatch on receive John Hubbard
` (21 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The r000 firmware posts GMC and RPC messages on the same queue. Behind
the transport headers, a GMC element carries a GMC header instead of an
RPC header.
Nova-core's receive path decoded RPC elements only.
Add a receive path for GMC elements. It validates the transport headers
before it trusts the length that they declare, as Open RM does. The read
pointer advances by that length, so a bad header leaves no way to find
the next element. The receive path poisons the queue on a bad header,
and every later receive fails until the device is reset. The GMC receive
path has no caller yet.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/cmdq.rs | 167 ++++++++++++++++++++++++++----
drivers/gpu/nova-core/gsp/fw.rs | 64 ++++++++++--
drivers/gpu/nova-core/mctp.rs | 14 ++-
3 files changed, 214 insertions(+), 31 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index c0c3d8596b30..a1bf536dc109 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -59,12 +59,14 @@
MsgFunction,
MsgqRxHeader,
MsgqTxHeader,
+ QueueElementHeader,
GSP_MSG_QUEUE_ELEMENT_SIZE_MAX, //
},
PteArray,
GSP_PAGE_SHIFT,
GSP_PAGE_SIZE, //
},
+ mctp::NvdmType,
num,
sbuffer::SBufferIter, //
};
@@ -475,6 +477,44 @@ struct GspMessage<'a> {
contents: (&'a [u8], &'a [u8]),
}
+/// A GMC (GPU Management Controller) API message ready to be processed from the message queue.
+///
+/// This is the message that [`QueueElement::Gmc`] carries.
+#[expect(dead_code)]
+struct GmcMessage<'a> {
+ // The queue element header and the GMC API header that open the element.
+ header: &'a GspGmcMsgElement,
+ // Slices to the payload that follows the GMC API header. The second slice is empty unless the
+ // payload wraps around the end of the message queue.
+ contents: (&'a [u8], &'a [u8]),
+}
+
+/// A queue element that has passed validation, decoded as an RPC message or as a GMC API message.
+///
+/// The queue element header holds an MCTP (Management Component Transport Protocol) header and an
+/// NVDM (NVIDIA vendor-defined message) header. The NVDM type selects between the two kinds of
+/// message.
+///
+/// This is the type returned by [`CmdqInner::wait_for_element`].
+enum QueueElement<'a> {
+ /// A GMC API message.
+ Gmc(GmcMessage<'a>),
+ /// An element whose NVDM type names another kind of message, such as an RM RPC. Only its
+ /// queue element header is decoded.
+ Other(&'a QueueElementHeader),
+}
+
+impl QueueElement<'_> {
+ /// Returns the number of queue slots that the element occupies.
+ #[expect(dead_code)]
+ fn element_count(&self) -> u32 {
+ match self {
+ Self::Gmc(message) => message.header.element_count(),
+ Self::Other(element_header) => element_header.element_count(),
+ }
+ }
+}
+
/// GSP command queue.
///
/// Provides the ability to send commands and receive messages from the GSP using a shared memory
@@ -838,29 +878,7 @@ fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {
header.length(),
);
- let payload_length = header.payload_length();
-
- // Check that the driver read area is large enough for the message.
- if slice_1.len() + slice_2.len() < payload_length {
- return Err(self.poison(fmt!(
- "message advertises {} payload bytes but only {} are readable",
- payload_length,
- slice_1.len() + slice_2.len()
- )));
- }
-
- // Cut the message slices down to the actual length of the message.
- let (slice_1, slice_2) = if slice_1.len() > payload_length {
- // PANIC: we checked above that `slice_1` is at least as long as `payload_length`.
- (slice_1.split_at(payload_length).0, &slice_2[0..0])
- } else {
- (
- slice_1,
- // PANIC: we checked above that `slice_1.len() + slice_2.len()` is at least as
- // large as `payload_length`.
- slice_2.split_at(payload_length - slice_1.len()).0,
- )
- };
+ let (slice_1, slice_2) = self.payload_slices(slice_1, slice_2, header.payload_length())?;
// Validate checksum.
if Cmdq::calculate_checksum(SBufferIter::new_reader([
@@ -1029,4 +1047,107 @@ fn drain(&mut self) -> Result {
Ok(())
}
+
+ /// Truncates the read area that follows the queue element header and the message header to
+ /// the `payload_length` bytes of payload.
+ ///
+ /// # Errors
+ ///
+ /// - `EIO` if fewer bytes than that are readable, which poisons the queue.
+ fn payload_slices<'a>(
+ &self,
+ slice_1: &'a [u8],
+ slice_2: &'a [u8],
+ payload_length: usize,
+ ) -> Result<(&'a [u8], &'a [u8])> {
+ if slice_1.len() + slice_2.len() < payload_length {
+ return Err(self.poison(fmt!(
+ "message advertises {} payload bytes but only {} are readable",
+ payload_length,
+ slice_1.len() + slice_2.len()
+ )));
+ }
+
+ Ok(if slice_1.len() > payload_length {
+ // PANIC: we checked above that `slice_1` is at least as long as `payload_length`.
+ (slice_1.split_at(payload_length).0, &slice_2[0..0])
+ } else {
+ (
+ slice_1,
+ // PANIC: we checked above that `slice_1.len() + slice_2.len()` is at least as
+ // large as `payload_length`.
+ slice_2.split_at(payload_length - slice_1.len()).0,
+ )
+ })
+ }
+
+ /// Waits for the next queue element and decodes it as an RPC message or as a GMC API message.
+ ///
+ /// ```text
+ /// +------------------------------------+
+ /// | queue element header: magic, MCTP | validated
+ /// | header, NVDM header, lengths |
+ /// +------------------------------------+
+ /// | message header | decoded as a GMC API header when the NVDM type
+ /// +------------------------------------+ is GmcApi, and left undecoded otherwise
+ /// | payload | truncated to the length that the queue
+ /// +------------------------------------+ element header declares
+ /// ```
+ ///
+ /// # Errors
+ ///
+ /// - `ETIMEDOUT` if no element arrives within `timeout`.
+ /// - `EIO` if the queue is already poisoned, or if the framing is invalid, which poisons it
+ /// (see [`Self::poisoned`]).
+ #[expect(dead_code)]
+ fn wait_for_element(&self, timeout: Delta) -> Result<QueueElement<'_>> {
+ if self.poisoned.get() {
+ return Err(EIO);
+ }
+
+ let (slice_1, slice_2) = read_poll_timeout(
+ || Ok(self.gsp_mem.driver_read_area()),
+ |driver_area| !driver_area.0.is_empty(),
+ Delta::from_millis(1),
+ timeout,
+ )
+ .map(|(slice_1, slice_2)| (slice_1.as_flattened(), slice_2.as_flattened()))?;
+
+ let Some((element_header, _)) = QueueElementHeader::from_bytes_prefix(slice_1) else {
+ return Err(self.poison(fmt!(
+ "read area of {} bytes is shorter than a queue element header",
+ slice_1.len()
+ )));
+ };
+
+ if let Err(error) = element_header.validate() {
+ return Err(self.poison(fmt!(
+ "element has a bad queue element header ({:?}), declared length {}",
+ error,
+ element_header.element_len()
+ )));
+ }
+
+ if !element_header.is_nvdm_type(NvdmType::GmcApi) {
+ return Ok(QueueElement::Other(element_header));
+ }
+
+ let Some((header, slice_1)) = GspGmcMsgElement::from_bytes_prefix(slice_1) else {
+ return Err(self.poison(fmt!(
+ "read area of {} bytes is shorter than a GMC element header",
+ slice_1.len()
+ )));
+ };
+
+ let Some(payload_length) = header.payload_length() else {
+ return Err(self.poison(fmt!(
+ "GMC message seq# {} declares a message shorter than the GMC API header",
+ header.gmc.sequence
+ )));
+ };
+
+ let contents = self.payload_slices(slice_1, slice_2, payload_length)?;
+
+ Ok(QueueElement::Gmc(GmcMessage { header, contents }))
+ }
}
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index 75bc3d66f71f..5548ca77f49b 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -944,7 +944,6 @@ pub(crate) struct QueueElementHeader {
== core::mem::offset_of!(r000_00::GSP_MSG_QUEUE_ELEMENT, nvdmHeader)
);
-#[expect(dead_code)]
impl QueueElementHeader {
/// Builds the queue element header of an element whose message header and payload together
/// take `message_len` bytes.
@@ -968,21 +967,68 @@ fn new(nvdm: NvdmType, message_len: usize) -> Result<Self> {
}
/// Returns the length of the whole element, the queue element header included.
- fn element_len(&self) -> usize {
+ pub(crate) fn element_len(&self) -> usize {
num::u32_as_usize(self.element_len)
}
/// Returns the length of the payload that follows a message header of `message_header_len`
- /// bytes.
- fn payload_len(&self, message_header_len: usize) -> usize {
- num::u32_as_usize(self.message_len).saturating_sub(message_header_len)
+ /// bytes, or `None` if the declared message is shorter than that header.
+ fn payload_len(&self, message_header_len: usize) -> Option<usize> {
+ num::u32_as_usize(self.message_len).checked_sub(message_header_len)
}
/// Returns the number of queue slots that this element occupies.
- fn element_count(&self) -> u32 {
+ pub(crate) fn element_count(&self) -> u32 {
self.element_len
.div_ceil(num::usize_into_u32::<GSP_PAGE_SIZE>())
}
+
+ /// Validates the queue element header.
+ ///
+ /// Returns the first check that fails as a [`QueueElementHeaderError`].
+ pub(crate) fn validate(&self) -> Result<(), QueueElementHeaderError> {
+ if self.magic != MCTP_MAGIC {
+ return Err(QueueElementHeaderError::BadMagic);
+ }
+ // The MCTP start-of-message and end-of-message bits are not checked. Every element carries
+ // one whole message, because a large RPC is split into continuation records, not packets.
+ if !self.mctp.has_expected_version() {
+ return Err(QueueElementHeaderError::BadMctpVersion);
+ }
+ if !self.nvdm.has_nvidia_vendor() {
+ return Err(QueueElementHeaderError::BadNvdmVendor);
+ }
+
+ // Under confidential compute, GSP-RM pads the element out to whole queue slots, so the
+ // element may be longer than its queue element header and message together, but never
+ // shorter.
+ let length = self.element_len();
+ let min_length = size_of::<Self>().saturating_add(num::u32_as_usize(self.message_len));
+ if length < min_length || length > GSP_MSG_QUEUE_ELEMENT_SIZE_MAX {
+ return Err(QueueElementHeaderError::BadLength);
+ }
+
+ Ok(())
+ }
+
+ pub(crate) fn is_nvdm_type(&self, nvdm_type: NvdmType) -> bool {
+ self.nvdm.validate(nvdm_type)
+ }
+}
+
+/// The check of [`QueueElementHeader::validate`] that a queue element header fails.
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum QueueElementHeaderError {
+ /// The first word is not `"MCTP"`.
+ BadMagic,
+ /// The MCTP header carries a version other than the one that this driver uses.
+ BadMctpVersion,
+ /// The NVDM header names a vendor other than NVIDIA, or a message type other than
+ /// vendor-defined.
+ BadNvdmVendor,
+ /// The element length is shorter than the queue element header and the message together, or
+ /// above the maximum element size.
+ BadLength,
}
// SAFETY: All fields are integer types or transparent wrappers over one, with no padding.
@@ -1091,6 +1137,12 @@ pub(crate) fn init(
})
}
+ /// Returns the length of the payload that follows the GMC API header, or `None` if the queue
+ /// element header declares a message shorter than the GMC API header.
+ pub(crate) fn payload_length(&self) -> Option<usize> {
+ self.element_header.payload_len(size_of::<GmcApiHeader>())
+ }
+
/// Returns the length of the whole element, both headers included.
pub(crate) fn length(&self) -> usize {
self.element_header.element_len()
diff --git a/drivers/gpu/nova-core/mctp.rs b/drivers/gpu/nova-core/mctp.rs
index 0eb964c74e32..d2a7c6c02c35 100644
--- a/drivers/gpu/nova-core/mctp.rs
+++ b/drivers/gpu/nova-core/mctp.rs
@@ -70,6 +70,11 @@ pub(crate) fn single_packet() -> Self {
pub(crate) fn is_single_packet(self) -> bool {
self.som().into_bool() && self.eom().into_bool()
}
+
+ /// Returns `true` if this MCTP header carries [`Self::VERSION`].
+ pub(crate) fn has_expected_version(self) -> bool {
+ u32::from(self.version()) == Self::VERSION
+ }
}
/// MCTP message type for PCI vendor-defined messages.
@@ -96,10 +101,15 @@ pub(crate) fn new(nvdm_type: NvdmType) -> Self {
.with_nvdm_type(nvdm_type)
}
- /// Validates this header against the expected NVIDIA NVDM format and type.
- pub(crate) fn validate(self, expected_type: NvdmType) -> bool {
+ pub(crate) fn has_nvidia_vendor(self) -> bool {
u8::from(self.msg_type()) == MSG_TYPE_VENDOR_PCI
&& u16::from(self.vendor_id()) == Vendor::NVIDIA.as_raw()
+ }
+
+ /// Returns `true` if this NVDM header names the NVIDIA vendor and the NVDM type
+ /// `expected_type`.
+ pub(crate) fn validate(self, expected_type: NvdmType) -> bool {
+ self.has_nvidia_vendor()
&& matches!(self.nvdm_type(), Ok(nvdm_type) if nvdm_type == expected_type)
}
}
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 12/33] gpu: nova-core: gsp: add GMC dispatch on receive
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (10 preceding siblings ...)
2026-09-18 1:06 ` [PATCH v3 11/33] gpu: nova-core: add GMC transport receive path John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:06 ` [PATCH v3 13/33] gpu: nova-core: separate the generic falcon bootloader from FWSEC John Hubbard
` (20 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The r000 boot protocol delivers its load-and-execute steps as GMC
events, each named by a command id. A GMC element states its payload
size twice, in the GMC header and in the queue element header, and
GSP-RM writes both from the same payload.
Add a receive that passes a GMC element's command id and payload to a
handler that the caller supplies, and that logs and drops an element
that is not a GMC element. The read pointer advances past the element
whether or not the handler accepts it, so that a handler receives each
element once. An element whose two payload sizes differ has a corrupt
header, and the driver cannot know which one is correct, so the receive
poisons the queue rather than trust either size. This receive has no
caller yet.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/cmdq.rs | 104 ++++++++++++++++++++++++++++--
drivers/gpu/nova-core/gsp/fw.rs | 8 +++
2 files changed, 107 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index a1bf536dc109..b202bd8185ba 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -480,7 +480,6 @@ struct GspMessage<'a> {
/// A GMC (GPU Management Controller) API message ready to be processed from the message queue.
///
/// This is the message that [`QueueElement::Gmc`] carries.
-#[expect(dead_code)]
struct GmcMessage<'a> {
// The queue element header and the GMC API header that open the element.
header: &'a GspGmcMsgElement,
@@ -506,7 +505,6 @@ enum QueueElement<'a> {
impl QueueElement<'_> {
/// Returns the number of queue slots that the element occupies.
- #[expect(dead_code)]
fn element_count(&self) -> u32 {
match self {
Self::Gmc(message) => message.header.element_count(),
@@ -627,6 +625,21 @@ pub(crate) fn send_command_no_wait<M>(&self, command: M) -> Result
self.inner.lock().send_command(command)
}
+ /// Receives one GMC event and passes its command id and payload slices to `handler`.
+ ///
+ /// This method may sleep while waiting. The queue mutex stays locked across the wait and the
+ /// `handler` call, so `handler` must not call back into this [`Cmdq`].
+ ///
+ /// See [`CmdqInner::receive_gmc_and_dispatch`] for the return value and the errors.
+ #[expect(dead_code)]
+ pub(crate) fn receive_gmc_and_dispatch<R>(
+ &self,
+ timeout: Delta,
+ handler: impl FnOnce(u32, &[u8], &[u8]) -> Result<Option<R>>,
+ ) -> Result<Option<R>> {
+ self.inner.lock().receive_gmc_and_dispatch(timeout, handler)
+ }
+
/// Waits for an unsolicited GSP event of type `M`. Events that arrive before it are logged and
/// consumed.
///
@@ -1097,9 +1110,9 @@ fn payload_slices<'a>(
/// # Errors
///
/// - `ETIMEDOUT` if no element arrives within `timeout`.
- /// - `EIO` if the queue is already poisoned, or if the framing is invalid, which poisons it
- /// (see [`Self::poisoned`]).
- #[expect(dead_code)]
+ /// - `EIO` if the queue is already poisoned, or if the framing is invalid, or if the GMC API
+ /// header and the queue element header declare different payload sizes. Each of these
+ /// poisons the queue (see [`Self::poisoned`]).
fn wait_for_element(&self, timeout: Delta) -> Result<QueueElement<'_>> {
if self.poisoned.get() {
return Err(EIO);
@@ -1146,8 +1159,89 @@ fn wait_for_element(&self, timeout: Delta) -> Result<QueueElement<'_>> {
)));
};
+ // GSP-RM writes both sizes from the same payload, so a difference means that one of the
+ // two headers is corrupt, and the driver cannot know which.
+ if payload_length != num::u32_as_usize(header.gmc.size) {
+ return Err(self.poison(fmt!(
+ "GMC seq# {}: GMC API header declares {} payload bytes, element header {}",
+ header.gmc.sequence,
+ header.gmc.size,
+ payload_length
+ )));
+ }
+
let contents = self.payload_slices(slice_1, slice_2, payload_length)?;
Ok(QueueElement::Gmc(GmcMessage { header, contents }))
}
+
+ /// Waits for the next queue element, passes it to `f`, and advances the read pointer past it.
+ ///
+ /// The read pointer advances whether `f` succeeds or fails, so that `f` is called once per
+ /// element. The element and its payload slices are valid only inside `f`.
+ ///
+ /// # Errors
+ ///
+ /// - `ETIMEDOUT` if `timeout` has elapsed before any element becomes available.
+ /// - `EIO` if the queue is poisoned or the element is invalid, as [`Self::wait_for_element`]
+ /// describes.
+ ///
+ /// Errors from `f` are propagated as-is.
+ fn consume_element<R>(
+ &mut self,
+ timeout: Delta,
+ f: impl FnOnce(&Self, QueueElement<'_>) -> Result<R>,
+ ) -> Result<R> {
+ let element = self.wait_for_element(timeout)?;
+ let element_count = element.element_count();
+
+ let result = f(self, element);
+
+ self.gsp_mem.advance_cpu_read_ptr(element_count);
+
+ result
+ }
+
+ /// Receives the next queue element and, if it is a GMC element, passes it to `handler`.
+ ///
+ /// `handler` receives the command id and the payload that follows the GMC API header, as two
+ /// slices because the ring may wrap, and returns `None` for an element that it declines.
+ ///
+ /// Returns `Ok(None)` when `handler` declines the element or when the element is not a GMC
+ /// element.
+ ///
+ /// # Errors
+ ///
+ /// - `ETIMEDOUT` if no element arrives within `timeout`.
+ /// - `EIO` if the queue is poisoned or the queue element header is invalid, as
+ /// [`Self::wait_for_element`] describes.
+ ///
+ /// Errors from `handler` are propagated as-is.
+ fn receive_gmc_and_dispatch<R>(
+ &mut self,
+ timeout: Delta,
+ handler: impl FnOnce(u32, &[u8], &[u8]) -> Result<Option<R>>,
+ ) -> Result<Option<R>> {
+ self.consume_element(timeout, |this, element| match element {
+ QueueElement::Other(_) => {
+ dev_warn!(&this.dev, "GSP GMC: dropping non-GMC queue element\n");
+
+ Ok(None)
+ }
+ QueueElement::Gmc(message) => {
+ let header = message.header;
+ let command_id = header.gmc.command_id();
+
+ dev_dbg!(
+ &this.dev,
+ "GSP GMC: event: seq# {}, command_id=0x{:x}, length=0x{:x}\n",
+ header.gmc.sequence,
+ command_id,
+ header.length(),
+ );
+
+ handler(command_id, message.contents.0, message.contents.1)
+ }
+ })
+ }
}
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index 5548ca77f49b..f23d071f0e16 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -1052,6 +1052,9 @@ pub(crate) struct GmcApiHeader {
reserved: [u32; 5],
}
+/// Bits of [`GmcApiHeader::command`] that hold the command id. The high byte holds flags.
+const GMCAPI_COMMAND_ID_MASK: u32 = 0x00ff_ffff;
+
static_assert!(size_of::<GmcApiHeader>() == size_of::<r000_00::GMCAPI_HEADER>());
static_assert!(
core::mem::offset_of!(GmcApiHeader, command)
@@ -1075,6 +1078,11 @@ pub(crate) struct GmcApiHeader {
);
impl GmcApiHeader {
+ /// Returns the command id, without the flag byte.
+ pub(crate) fn command_id(&self) -> u32 {
+ self.command & GMCAPI_COMMAND_ID_MASK
+ }
+
/// Returns the `NV_STATUS` that a response carries.
///
/// The value is meaningful only on a response, which GSP-RM marks with a flag in the command
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 13/33] gpu: nova-core: separate the generic falcon bootloader from FWSEC
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (11 preceding siblings ...)
2026-09-18 1:06 ` [PATCH v3 12/33] gpu: nova-core: gsp: add GMC dispatch on receive John Hubbard
@ 2026-09-18 1:06 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 14/33] gpu: nova-core: add the falcon DMA and suspend helpers for r000 boot John Hubbard
` (19 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:06 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The generic falcon bootloader is a small program that loads a larger
image into a falcon. The r000 boot protocol needs it to run an image
that GSP-RM names in a load-and-execute event during boot.
Nova-core used the bootloader only to load FWSEC, so the bootloader code
was part of the FWSEC code, and there was no way to run it on another
image.
Move the bootloader into its own type, which FWSEC then uses.
Place the bootloader at the top of the falcon's IMEM, using the IMEM
size that the HWCFG register reports, instead of assuming a 64 KiB IMEM.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/falcon.rs | 8 ++
drivers/gpu/nova-core/firmware.rs | 1 +
.../nova-core/firmware/fwsec/bootloader.rs | 63 ++----------
.../gpu/nova-core/firmware/gen_bootloader.rs | 99 +++++++++++++++++++
drivers/gpu/nova-core/gsp/hal/tu102.rs | 6 +-
drivers/gpu/nova-core/regs.rs | 5 +
6 files changed, 124 insertions(+), 58 deletions(-)
create mode 100644 drivers/gpu/nova-core/firmware/gen_bootloader.rs
diff --git a/drivers/gpu/nova-core/falcon.rs b/drivers/gpu/nova-core/falcon.rs
index 9015de965a53..04e35cbcb6f0 100644
--- a/drivers/gpu/nova-core/falcon.rs
+++ b/drivers/gpu/nova-core/falcon.rs
@@ -385,6 +385,14 @@ pub(crate) fn new(
})
}
+ /// Returns the size of this falcon's IMEM in bytes.
+ pub(crate) fn imem_size(&self) -> usize {
+ let blocks =
+ usize::from_safe_cast(*self.pfalcon.read(regs::NV_PFALCON_FALCON_HWCFG).imem_size());
+
+ blocks * MEM_BLOCK_ALIGNMENT
+ }
+
/// Resets DMA-related registers.
pub(crate) fn dma_reset(&self) {
self.pfalcon.update(regs::NV_PFALCON_FBIF_CTL, |v| {
diff --git a/drivers/gpu/nova-core/firmware.rs b/drivers/gpu/nova-core/firmware.rs
index d8f6509a35d1..358c9b8db0b8 100644
--- a/drivers/gpu/nova-core/firmware.rs
+++ b/drivers/gpu/nova-core/firmware.rs
@@ -24,6 +24,7 @@
pub(crate) mod booter;
pub(crate) mod fwsec;
+pub(crate) mod gen_bootloader;
pub(crate) mod gsp;
pub(crate) mod gsp_fmc;
pub(crate) mod radix3;
diff --git a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
index a87878fe2aec..1584a4b814d2 100644
--- a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
@@ -17,17 +17,11 @@
Io, //
},
prelude::*,
- ptr::{
- Alignable,
- Alignment, //
- },
- sizes,
transmute::AsBytes,
};
use crate::{
falcon::{
- self,
gsp::Gsp,
Falcon,
FalconBromParams,
@@ -41,10 +35,7 @@
},
firmware::{
fwsec::FwsecFirmware,
- tlv::{
- request_tlv, //
- Tlv,
- },
+ gen_bootloader::GenericBootloader, //
},
gpu::Chipset,
num::FromSafeCast, //
@@ -103,16 +94,11 @@ unsafe impl AsBytes for BootloaderDmemDescV2 {}
pub(crate) struct FwsecFirmwareWithBl<'a> {
/// DMA object the bootloader will copy the firmware from.
_firmware_dma: Coherent<'a, [u8]>,
- /// Code of the bootloader to be loaded into non-secure IMEM.
- ucode: KVec<u8>,
+ bootloader: GenericBootloader,
/// Descriptor to be loaded into DMEM for the bootloader to read.
dmem_desc: BootloaderDmemDescV2,
- /// Range-validated start offset of the firmware code in IMEM.
- imem_dst_start: u16,
/// BROM parameters of the loaded firmware.
brom_params: FalconBromParams,
- /// Range-validated `desc.start_tag`.
- start_tag: u16,
}
impl<'a> FwsecFirmwareWithBl<'a> {
@@ -122,29 +108,9 @@ pub(crate) fn new(
firmware: FwsecFirmware,
dev: &'a Device<device::Bound>,
chipset: Chipset,
+ falcon: &Falcon<'_, Gsp>,
) -> Result<Self> {
- let fw = request_tlv(dev, chipset, "gen_bootloader")?;
- let tlv = Tlv::new(fw.data())?;
- dev_dbg!(
- dev,
- "loaded generic bootloader firmware v{}\n",
- tlv.get_string(b"VERS")?
- );
-
- let ucode = {
- let blob = tlv.get_bytes(b"BLOB")?;
- let code_size = usize::from_safe_cast(tlv.get_u32(b"CDSZ")?);
- let code = blob.get(..code_size).ok_or(EINVAL)?;
- let aligned_code_size = code_size
- .align_up(Alignment::new::<{ falcon::MEM_BLOCK_ALIGNMENT }>())
- .ok_or(EINVAL)?;
-
- let mut ucode = KVec::with_capacity(aligned_code_size, GFP_KERNEL)?;
- ucode.extend_from_slice(code, GFP_KERNEL)?;
- ucode.resize(aligned_code_size, 0, GFP_KERNEL)?;
-
- ucode
- };
+ let bootloader = GenericBootloader::new(dev, chipset, falcon.imem_size())?;
// `BootloaderDmemDescV2` expects the source to be a mirror image of the destination and
// uses the same offset parameter for both.
@@ -215,21 +181,11 @@ pub(crate) fn new(
}
};
- // The bootloader's code must be loaded in the area right below the first 64K of IMEM.
- const BOOTLOADER_LOAD_CEILING: usize = sizes::SZ_64K;
- let imem_dst_start = BOOTLOADER_LOAD_CEILING
- .checked_sub(ucode.len())
- .ok_or(EOVERFLOW)?;
-
- let start_tag = u16::try_from(tlv.get_u32(b"STRT")?)?;
-
Ok(Self {
_firmware_dma: firmware_dma,
- ucode,
+ bootloader,
dmem_desc,
brom_params: firmware.brom_params(),
- imem_dst_start: u16::try_from(imem_dst_start)?,
- start_tag,
})
}
@@ -278,7 +234,7 @@ fn brom_params(&self) -> FalconBromParams {
fn boot_addr(&self) -> u32 {
// On V2 platforms, the boot address is extracted from the generic bootloader, because the
// gbl is what actually copies FWSEC into memory, so that is what needs to be booted.
- u32::from(self.start_tag) << 8
+ self.bootloader.boot_addr()
}
}
@@ -288,12 +244,7 @@ fn imem_sec_load_params(&self) -> Option<FalconPioImemLoadTarget<'_>> {
}
fn imem_ns_load_params(&self) -> Option<FalconPioImemLoadTarget<'_>> {
- Some(FalconPioImemLoadTarget {
- data: self.ucode.as_ref(),
- dst_start: self.imem_dst_start,
- secure: false,
- start_tag: self.start_tag,
- })
+ Some(self.bootloader.imem_load_params())
}
fn dmem_load_params(&self) -> FalconPioDmemLoadTarget<'_> {
diff --git a/drivers/gpu/nova-core/firmware/gen_bootloader.rs b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
new file mode 100644
index 000000000000..65af57a34940
--- /dev/null
+++ b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
@@ -0,0 +1,99 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+//! The generic falcon bootloader: a small program that the driver loads into a falcon's IMEM by
+//! PIO. It reads a descriptor that the driver writes at DMEM offset 0, and loads the image that
+//! the descriptor names into IMEM and DMEM by DMA.
+
+use kernel::{
+ device,
+ prelude::*,
+ ptr::{
+ Alignable,
+ Alignment, //
+ },
+};
+
+use crate::{
+ falcon::{
+ self,
+ FalconPioImemLoadTarget, //
+ },
+ firmware::tlv::{
+ request_tlv, //
+ Tlv,
+ },
+ gpu::Chipset,
+ num::FromSafeCast, //
+};
+
+/// The generic falcon bootloader image and its IMEM load parameters.
+pub(crate) struct GenericBootloader {
+ /// Bootloader code, zero-padded to a whole number of falcon memory blocks.
+ ucode: KVec<u8>,
+ /// Byte offset in IMEM at which the code is loaded.
+ imem_dst_start: u16,
+ /// Tag under which the first code block is loaded.
+ start_tag: u16,
+}
+
+impl GenericBootloader {
+ /// Loads the generic bootloader image for `chipset`, placed in the last blocks of an IMEM of
+ /// `imem_size` bytes.
+ ///
+ /// # Errors
+ ///
+ /// - `EINVAL` if a required TLV field is absent or the image does not fit in IMEM.
+ /// - `ENOMEM` if the padded copy of the code cannot be allocated.
+ pub(crate) fn new(
+ dev: &device::Device<device::Bound>,
+ chipset: Chipset,
+ imem_size: usize,
+ ) -> Result<Self> {
+ let fw = request_tlv(dev, chipset, "gen_bootloader")?;
+ let tlv = Tlv::new(fw.data())?;
+ dev_dbg!(
+ dev,
+ "loaded generic bootloader firmware v{}\n",
+ tlv.get_string(b"VERS")?
+ );
+
+ let ucode = {
+ let blob = tlv.get_bytes(b"BLOB")?;
+ let code_size = usize::from_safe_cast(tlv.get_u32(b"CDSZ")?);
+ let code = blob.get(..code_size).ok_or(EINVAL)?;
+ let aligned_code_size = code_size
+ .align_up(Alignment::new::<{ falcon::MEM_BLOCK_ALIGNMENT }>())
+ .ok_or(EINVAL)?;
+
+ let mut ucode = KVec::with_capacity(aligned_code_size, GFP_KERNEL)?;
+ ucode.extend_from_slice(code, GFP_KERNEL)?;
+ ucode.resize(aligned_code_size, 0, GFP_KERNEL)?;
+
+ ucode
+ };
+
+ // The top of IMEM, above the blocks that the bootloader loads the image into.
+ let imem_dst_start = imem_size.checked_sub(ucode.len()).ok_or(EINVAL)?;
+
+ Ok(Self {
+ ucode,
+ imem_dst_start: u16::try_from(imem_dst_start)?,
+ start_tag: u16::try_from(tlv.get_u32(b"STRT")?)?,
+ })
+ }
+
+ pub(crate) fn boot_addr(&self) -> u32 {
+ u32::from(self.start_tag) << 8
+ }
+
+ /// Returns the PIO parameters that place this bootloader in non-secure IMEM.
+ pub(crate) fn imem_load_params(&self) -> FalconPioImemLoadTarget<'_> {
+ FalconPioImemLoadTarget {
+ data: self.ucode.as_ref(),
+ dst_start: self.imem_dst_start,
+ secure: false,
+ start_tag: self.start_tag,
+ }
+ }
+}
diff --git a/drivers/gpu/nova-core/gsp/hal/tu102.rs b/drivers/gpu/nova-core/gsp/hal/tu102.rs
index e90db1a23032..1b17ece53bd6 100644
--- a/drivers/gpu/nova-core/gsp/hal/tu102.rs
+++ b/drivers/gpu/nova-core/gsp/hal/tu102.rs
@@ -165,7 +165,7 @@ fn run_fwsec_frts(
)?;
if self.needs_fwsec_bootloader {
- let fwsec_frts_bl = FwsecFirmwareWithBl::new(fwsec_frts, dev, chipset)?;
+ let fwsec_frts_bl = FwsecFirmwareWithBl::new(fwsec_frts, dev, chipset, falcon)?;
// Load and run the bootloader, which will load FWSEC-FRTS and run it.
fwsec_frts_bl.run(dev, falcon)?;
} else {
@@ -223,7 +223,9 @@ fn build_unload_bundle<'gpu>(
// Load the FWSEC SB firmware, as well as its bootloader if required.
let fwsec_sb = FwsecFirmware::new(dev, gsp_falcon, bios, FwsecCommand::Sb)?;
let fwsec_sb = if self.needs_fwsec_bootloader {
- FwsecUnloadFirmware::WithBl(FwsecFirmwareWithBl::new(fwsec_sb, dev, chipset)?)
+ FwsecUnloadFirmware::WithBl(FwsecFirmwareWithBl::new(
+ fwsec_sb, dev, chipset, gsp_falcon,
+ )?)
} else {
FwsecUnloadFirmware::WithoutBl(fwsec_sb)
};
diff --git a/drivers/gpu/nova-core/regs.rs b/drivers/gpu/nova-core/regs.rs
index c6ba226dcfe3..69dd6526e469 100644
--- a/drivers/gpu/nova-core/regs.rs
+++ b/drivers/gpu/nova-core/regs.rs
@@ -181,6 +181,11 @@ pub(crate) fn usable_fb_size(self) -> u64 {
31:0 value => u32;
}
+ pub(crate) NV_PFALCON_FALCON_HWCFG(u32) @ 0x00000108 {
+ /// Number of 256-byte blocks in this falcon's IMEM.
+ 8:0 imem_size;
+ }
+
pub(crate) NV_PFALCON_FALCON_DMACTL(u32) @ 0x0000010c {
7:7 secure_stat => bool;
6:3 dmaq_num;
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 14/33] gpu: nova-core: add the falcon DMA and suspend helpers for r000 boot
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (12 preceding siblings ...)
2026-09-18 1:06 ` [PATCH v3 13/33] gpu: nova-core: separate the generic falcon bootloader from FWSEC John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 15/33] gpu: nova-core: add the r000 load-and-execute HS binary handler John Hubbard
` (18 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The r000 boot protocol has two load-and-execute events. In each, GSP-RM
names an image that it has placed in the framebuffer, and the driver
copies the image into the GSP falcon's memory by DMA and runs it. The
event supplies the image's framebuffer address, and the copy goes
through a context DMA slot of the falcon's framebuffer interface. GSP-RM
also reports that the GSP's RISC-V core has suspended through a bit in
the falcon's MAILBOX0 register, rather than through the halted bit in
CPUCTL, and the driver has to wait for that bit before it resets the
falcon.
Nova-core's falcon DMA copied only from a DMA object that the driver had
allocated, so there was no way to copy an image from an address that an
event supplied. The shutdown path read the suspend bit inline, so there
was no wait for it that a boot event handler could share.
Add a DMA transfer that copies from a caller-supplied source address
through a caller-supplied context DMA slot. Add a check of the suspend
bit and a wait built on it, and make the shutdown path use the check in
place of its inline read.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/falcon.rs | 150 +++++++++++++++++++++++++++++-
drivers/gpu/nova-core/gsp/boot.rs | 6 +-
drivers/gpu/nova-core/regs.rs | 2 +-
3 files changed, 151 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/nova-core/falcon.rs b/drivers/gpu/nova-core/falcon.rs
index 04e35cbcb6f0..2a0fe86153aa 100644
--- a/drivers/gpu/nova-core/falcon.rs
+++ b/drivers/gpu/nova-core/falcon.rs
@@ -5,6 +5,7 @@
use hal::FalconHal;
use kernel::{
+ bits,
device,
dma::{
Coherent,
@@ -20,7 +21,7 @@
},
prelude::*,
sizes::SZ_4K,
- time::Delta,
+ time::Delta, //
};
use crate::{
@@ -43,6 +44,9 @@
/// Alignment (in bytes) of falcon memory blocks.
pub(crate) const MEM_BLOCK_ALIGNMENT: usize = 256;
+/// DMEM virtual address value that means "no virtual address assigned".
+const FLCN_DMEM_VA_INVALID: u32 = 0xffff_ffff;
+
bounded_enum! {
/// Revision number of a falcon core, used in the [`crate::regs::NV_PFALCON_FALCON_HWCFG1`]
/// register.
@@ -132,12 +136,36 @@ pub(crate) enum FalconMem {
/// Secure Instruction Memory.
ImemSecure,
/// Non-Secure Instruction Memory.
- #[expect(unused)]
+ #[expect(dead_code)]
ImemNonSecure,
/// Data Memory.
Dmem,
}
+/// Where a raw falcon DMA transfer reads its image from, relative to the DMA base address.
+#[derive(Copy, Clone)]
+pub(crate) enum FalconDmaSrcOffset {
+ /// The image starts this many bytes past the DMA base address.
+ Offset(u32),
+ /// Virtual address of the image in the target memory. The DMA engine tags each loaded block
+ /// with this value and adds the tag to the base address, so the base address must be the
+ /// source address minus this value.
+ Va(u32),
+}
+
+impl FalconDmaSrcOffset {
+ /// Returns the source offset of a DMEM image at virtual address `dmem_va`, or the start of the
+ /// source when `dmem_va` is `FLCN_DMEM_VA_INVALID`.
+ #[expect(dead_code)]
+ pub(crate) fn from_dmem_va(dmem_va: u32) -> Self {
+ if dmem_va == FLCN_DMEM_VA_INVALID {
+ Self::Offset(0)
+ } else {
+ Self::Va(dmem_va)
+ }
+ }
+}
+
bounded_enum! {
/// Defines the Framebuffer Interface (FBIF) aperture type.
/// This determines the memory type for external memory access during a DMA transfer, which is
@@ -600,6 +628,98 @@ fn dma_wr(
Ok(())
}
+ /// Transfers `len` bytes from `src_addr` into this falcon's `target_mem`.
+ ///
+ /// `src_addr` is a GPU physical address reached through the FBIF aperture, so the caller must
+ /// program `NV_PFALCON_FBIF_TRANSCFG` for `ctx_dma` before calling this.
+ ///
+ /// # Errors
+ ///
+ /// - `EINVAL` if `ctx_dma` is not a context DMA slot that the falcon has, or if `src_addr` is
+ /// not 256-byte aligned.
+ /// - `ERANGE` if `src_addr` does not fit the `DMATRFBASE` register pair.
+ /// - `EOVERFLOW` if a per-block source or destination offset exceeds `u32`.
+ #[expect(dead_code)]
+ pub(crate) fn raw_dma_transfer(
+ &self,
+ ctx_dma: u32,
+ src_addr: u64,
+ target_mem: FalconMem,
+ src: FalconDmaSrcOffset,
+ dst_offset: u32,
+ len: u32,
+ ) -> Result {
+ const DMA_LEN: u32 = num::usize_into_u32::<{ MEM_BLOCK_ALIGNMENT }>();
+
+ if src_addr % u64::from(DMA_LEN) > 0 {
+ dev_err!(
+ self.dev,
+ "raw DMA: source address {:#x} not 256B-aligned\n",
+ src_addr
+ );
+ return Err(EINVAL);
+ }
+
+ if src_addr >> 40 > u64::from(regs::NV_PFALCON_FALCON_DMATRFBASE1::BASE_MASK) {
+ dev_err!(
+ self.dev,
+ "raw DMA: source address {:#x} does not fit DMATRFBASE\n",
+ src_addr
+ );
+ return Err(ERANGE);
+ }
+
+ // An IMEM block is always tagged with its source offset. A DMEM block is tagged only when
+ // `SET_DMTAG` is set, so a DMEM virtual address sets `SET_DMTAG`.
+ let (src_offset, set_dmtag) = match src {
+ FalconDmaSrcOffset::Offset(offset) => (offset, false),
+ FalconDmaSrcOffset::Va(va) => (va, target_mem == FalconMem::Dmem),
+ };
+
+ let num_transfers = len.div_ceil(DMA_LEN);
+
+ self.pfalcon
+ .write_reg(regs::NV_PFALCON_FALCON_DMATRFBASE::zeroed().with_base(
+ // CAST: this drops the upper bits on purpose. They are written to
+ // `NV_PFALCON_FALCON_DMATRFBASE1` next.
+ (src_addr >> 8) as u32,
+ ));
+ self.pfalcon.write_reg(
+ regs::NV_PFALCON_FALCON_DMATRFBASE1::zeroed().try_with_base(src_addr >> 40)?,
+ );
+
+ // The `CTXDMA` field holds exactly the indices of the falcon's `TRANSCFG` slots, so the
+ // field's range is the bound on the slot.
+ let cmd = regs::NV_PFALCON_FALCON_DMATRFCMD::zeroed()
+ .with_size(DmaTrfCmdSize::Size256B)
+ .try_with_ctxdma(ctx_dma)
+ .map_err(|_| EINVAL)?
+ .with_falcon_mem(target_mem)
+ .with_set_dmtag(set_dmtag);
+
+ for pos in (0..num_transfers).map(|i| i * DMA_LEN) {
+ self.pfalcon.write_reg(
+ regs::NV_PFALCON_FALCON_DMATRFMOFFS::zeroed()
+ .try_with_offs(dst_offset.checked_add(pos).ok_or(EOVERFLOW)?)?,
+ );
+ self.pfalcon.write_reg(
+ regs::NV_PFALCON_FALCON_DMATRFFBOFFS::zeroed()
+ .with_offs(src_offset.checked_add(pos).ok_or(EOVERFLOW)?),
+ );
+
+ self.pfalcon.write_reg(cmd);
+
+ read_poll_timeout(
+ || Ok(self.pfalcon.read(regs::NV_PFALCON_FALCON_DMATRFCMD)),
+ |r| r.idle(),
+ Delta::ZERO,
+ Delta::from_secs(2),
+ )?;
+ }
+
+ Ok(())
+ }
+
/// Perform a DMA load into `IMEM` and `DMEM` of `fw`, and prepare the falcon to run it.
fn dma_load<F: FalconFirmware<Target = E> + FalconDmaLoadable>(&self, fw: &F) -> Result {
// DMA object with firmware content as the source of the DMA engine.
@@ -653,6 +773,32 @@ pub(crate) fn wait_till_halted(&self) -> Result<()> {
Ok(())
}
+ /// Returns `true` if the RISC-V core has suspended.
+ pub(crate) fn is_processor_suspended(&self) -> bool {
+ const INTERRUPT_PROCESSOR_SUSPENDED: u32 = bits::bit_u32(31);
+
+ self.read_mailbox0() & INTERRUPT_PROCESSOR_SUSPENDED != 0
+ }
+
+ /// Waits until the RISC-V core has suspended.
+ ///
+ /// The caller must write `MAILBOX0` before starting the core, or this returns as soon as it
+ /// reads the previous suspend.
+ ///
+ /// # Errors
+ ///
+ /// - `ETIMEDOUT` if the core has not suspended within two seconds.
+ #[expect(dead_code)]
+ pub(crate) fn wait_for_processor_suspend(&self) -> Result {
+ read_poll_timeout(
+ || Ok(self.is_processor_suspended()),
+ |suspended| *suspended,
+ Delta::ZERO,
+ Delta::from_secs(2),
+ )
+ .map(|_| ())
+ }
+
/// Start the falcon CPU.
pub(crate) fn start(&self) -> Result<()> {
match self.pfalcon.read(regs::NV_PFALCON_FALCON_CPUCTL).alias_en() {
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index 4fb1b69ac9d5..8518248c9732 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -2,7 +2,6 @@
// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
use kernel::{
- bits,
io::poll::read_poll_timeout,
prelude::*,
time::Delta,
@@ -94,10 +93,9 @@ fn shutdown_gsp(
cmdq.send_command(commands::UnloadingGuestDriver::new(mode))?;
// Wait until GSP signals it is suspended.
- const LIBOS_INTERRUPT_PROCESSOR_SUSPENDED: u32 = bits::bit_u32(31);
read_poll_timeout(
- || Ok(gsp_falcon.read_mailbox0()),
- |&mb0| mb0 & LIBOS_INTERRUPT_PROCESSOR_SUSPENDED != 0,
+ || Ok(gsp_falcon.is_processor_suspended()),
+ |suspended| *suspended,
Delta::from_millis(10),
Delta::from_secs(5),
)
diff --git a/drivers/gpu/nova-core/regs.rs b/drivers/gpu/nova-core/regs.rs
index 69dd6526e469..feb37de69be5 100644
--- a/drivers/gpu/nova-core/regs.rs
+++ b/drivers/gpu/nova-core/regs.rs
@@ -203,7 +203,7 @@ pub(crate) fn usable_fb_size(self) -> u64 {
}
pub(crate) NV_PFALCON_FALCON_DMATRFCMD(u32) @ 0x00000118 {
- 16:16 set_dmtag;
+ 16:16 set_dmtag => bool;
14:12 ctxdma;
10:8 size ?=> DmaTrfCmdSize;
5:5 is_write => bool;
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 15/33] gpu: nova-core: add the r000 load-and-execute HS binary handler
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (13 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 14/33] gpu: nova-core: add the falcon DMA and suspend helpers for r000 boot John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 16/33] gpu: nova-core: move the bootloader DMEM descriptor out of FWSEC John Hubbard
` (17 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
On GA102 and later, GSP-RM sends the driver a load-and-execute event
during boot that requests the driver to run a Heavy-Secured (HS) binary
on the GSP falcon. GSP-RM has placed the binary in the framebuffer, and
the event carries the binary's addresses. Once the binary has halted,
SEC2 restarts GSP-RM, in what Open RM calls the core resume.
Add the handler and the core resume. The handler copies the binary into
the falcon through an FBIF (framebuffer interface) aperture that it
programs for the load, starts the binary and waits for it to halt.
Put the falcons, the device and the boot parameters that the handler and
the core resume share into one context struct.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/falcon.rs | 51 +++++-
drivers/gpu/nova-core/falcon/gsp.rs | 10 +-
drivers/gpu/nova-core/gsp/boot.rs | 257 +++++++++++++++++++++++++++-
drivers/gpu/nova-core/regs.rs | 2 +
drivers/gpu/nova-core/sbuffer.rs | 1 -
5 files changed, 311 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/nova-core/falcon.rs b/drivers/gpu/nova-core/falcon.rs
index 2a0fe86153aa..8ad28dce955e 100644
--- a/drivers/gpu/nova-core/falcon.rs
+++ b/drivers/gpu/nova-core/falcon.rs
@@ -44,6 +44,10 @@
/// Alignment (in bytes) of falcon memory blocks.
pub(crate) const MEM_BLOCK_ALIGNMENT: usize = 256;
+/// `MAILBOX0` value that means "the falcon binary has not started". A binary that runs replaces
+/// it with its own status.
+pub(crate) const FLCN_ERR_BINARY_NOT_STARTED: u32 = 0xfe;
+
/// DMEM virtual address value that means "no virtual address assigned".
const FLCN_DMEM_VA_INVALID: u32 = 0xffff_ffff;
@@ -156,7 +160,6 @@ pub(crate) enum FalconDmaSrcOffset {
impl FalconDmaSrcOffset {
/// Returns the source offset of a DMEM image at virtual address `dmem_va`, or the start of the
/// source when `dmem_va` is `FLCN_DMEM_VA_INVALID`.
- #[expect(dead_code)]
pub(crate) fn from_dmem_va(dmem_va: u32) -> Self {
if dmem_va == FLCN_DMEM_VA_INVALID {
Self::Offset(0)
@@ -192,6 +195,17 @@ pub(crate) enum FalconFbifMemType with From<Bounded<u32, 1>> {
}
}
+bounded_enum! {
+ /// Engine ID that the falcon's framebuffer interface (FBIF) tags a DMA transfer with.
+ #[derive(Debug, Copy, Clone)]
+ pub(crate) enum FalconFbifEngineIdFlag with From<Bounded<u32, 1>> {
+ /// The BAR2 engine ID of PCI function 0.
+ Bar2Fn0 = 0,
+ /// The falcon's own engine ID.
+ Own = 1,
+ }
+}
+
const PFALCON_REGION_SIZE: usize = SZ_4K;
const PFALCON2_REGION_SIZE: usize = SZ_4K;
@@ -394,7 +408,7 @@ pub(crate) struct Falcon<'a, E: FalconEngine> {
bar: Bar0<'a>,
// TODO: make private
pub(crate) pfalcon: Mmio<'a, PFalconRegisters>,
- pfalcon2: Mmio<'a, PFalcon2Registers>,
+ pub(crate) pfalcon2: Mmio<'a, PFalcon2Registers>,
}
impl<'a, E: FalconEngine + 'static> Falcon<'a, E> {
@@ -628,6 +642,37 @@ fn dma_wr(
Ok(())
}
+ /// Programs FBIF context DMA slot `ctx_dma` with the value that `configure` returns, runs `f`,
+ /// and restores the slot once `f` has returned `Ok`.
+ ///
+ /// The slot keeps the programmed value if `f` fails, since a falcon that `f` started and that
+ /// has not halted may still be reading through the aperture.
+ ///
+ /// # Errors
+ ///
+ /// - `EINVAL` if `ctx_dma` is not a context DMA slot that the falcon has.
+ ///
+ /// Errors from `f` are propagated as-is.
+ pub(crate) fn with_fbif_transcfg<R>(
+ &self,
+ ctx_dma: u32,
+ configure: impl FnOnce(regs::NV_PFALCON_FBIF_TRANSCFG) -> regs::NV_PFALCON_FBIF_TRANSCFG,
+ f: impl FnOnce() -> Result<R>,
+ ) -> Result<R> {
+ // The location type is not `Copy`, so each register access builds its own.
+ let transcfg =
+ || regs::NV_PFALCON_FBIF_TRANSCFG::try_at(usize::from_safe_cast(ctx_dma)).ok_or(EINVAL);
+
+ let saved = self.pfalcon.read(transcfg()?);
+ self.pfalcon.update(transcfg()?, configure);
+
+ let result = f()?;
+
+ self.pfalcon.update(transcfg()?, |_| saved);
+
+ Ok(result)
+ }
+
/// Transfers `len` bytes from `src_addr` into this falcon's `target_mem`.
///
/// `src_addr` is a GPU physical address reached through the FBIF aperture, so the caller must
@@ -639,7 +684,6 @@ fn dma_wr(
/// not 256-byte aligned.
/// - `ERANGE` if `src_addr` does not fit the `DMATRFBASE` register pair.
/// - `EOVERFLOW` if a per-block source or destination offset exceeds `u32`.
- #[expect(dead_code)]
pub(crate) fn raw_dma_transfer(
&self,
ctx_dma: u32,
@@ -788,7 +832,6 @@ pub(crate) fn is_processor_suspended(&self) -> bool {
/// # Errors
///
/// - `ETIMEDOUT` if the core has not suspended within two seconds.
- #[expect(dead_code)]
pub(crate) fn wait_for_processor_suspend(&self) -> Result {
read_poll_timeout(
|| Ok(self.is_processor_suspended()),
diff --git a/drivers/gpu/nova-core/falcon/gsp.rs b/drivers/gpu/nova-core/falcon/gsp.rs
index dfa08bc6867c..70f7d55f9b59 100644
--- a/drivers/gpu/nova-core/falcon/gsp.rs
+++ b/drivers/gpu/nova-core/falcon/gsp.rs
@@ -115,15 +115,19 @@ pub(crate) fn retrigger_intr(bar: Bar0<'_>, chipset: Chipset) {
}
impl<'a> Falcon<'a, Gsp> {
- /// Checks if GSP reload/resume has completed during the boot process.
- pub(crate) fn check_reload_completed(&self, timeout: Delta) -> Result<bool> {
+ /// Waits until the Boot Sequence Interface (BSI) reports that the GSP reload has completed.
+ ///
+ /// # Errors
+ ///
+ /// - `ETIMEDOUT` if the reload has not completed within `timeout`.
+ pub(crate) fn check_reload_completed(&self, timeout: Delta) -> Result {
read_poll_timeout(
|| Ok(self.bar.read(regs::NV_PGC6_BSI_SECURE_SCRATCH_14)),
|val| val.boot_stage_3_handoff(),
Delta::ZERO,
timeout,
)
- .map(|_| true)
+ .map(|_| ())
}
/// Returns whether the RISC-V branch privilege lockdown bit is set.
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index 8518248c9732..3cfb21964250 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -2,24 +2,230 @@
// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
use kernel::{
- io::poll::read_poll_timeout,
+ device,
+ io::{
+ poll::read_poll_timeout,
+ register::Array,
+ Io, //
+ },
prelude::*,
time::Delta,
+ transmute::{
+ AsBytes,
+ FromBytes, //
+ },
types::ScopeGuard, //
};
use crate::{
falcon::{
gsp::Gsp,
- Falcon, //
+ sec2::Sec2,
+ Falcon,
+ FalconDmaSrcOffset,
+ FalconFbifEngineIdFlag,
+ FalconFbifMemType,
+ FalconFbifTarget,
+ FalconMem,
+ FalconModSelAlgo,
+ FLCN_ERR_BINARY_NOT_STARTED, //
},
firmware::gsp::GspFirmware,
gsp::{
cmdq::Cmdq,
commands, //
},
+ regs,
+ sbuffer::SBufferIter, //
};
+/// The falcons, device and boot parameters that the load-and-execute event handlers share.
+struct LoadExecContext<'a, 'gpu> {
+ gsp_falcon: &'a Falcon<'gpu, Gsp>,
+ sec2_falcon: &'a Falcon<'gpu, Sec2>,
+ dev: &'a device::Device,
+ /// GSP bootloader application version.
+ bootloader_app_version: u32,
+ /// DMA address of the LIBOS init arguments.
+ libos_dma_handle: u64,
+}
+
+impl LoadExecContext<'_, '_> {
+ /// Waits until GSP-RM has suspended the RISC-V core, then resets the GSP falcon and its DMA
+ /// registers.
+ ///
+ /// # Errors
+ ///
+ /// - `ETIMEDOUT` if the core has not suspended within two seconds.
+ ///
+ /// Errors from the falcon reset are propagated as-is.
+ fn reset_gsp_falcon_after_suspend(&self) -> Result {
+ let Self {
+ gsp_falcon, dev, ..
+ } = *self;
+
+ gsp_falcon.wait_for_processor_suspend().inspect_err(|_| {
+ dev_err!(
+ dev,
+ "Timeout waiting for GSP suspend (mbox0={:#x})\n",
+ gsp_falcon.read_mailbox0()
+ );
+ })?;
+
+ gsp_falcon.reset()?;
+ gsp_falcon.dma_reset();
+
+ Ok(())
+ }
+
+ /// Runs the core resume, in which SEC2 restarts GSP-RM after a load-and-execute binary has
+ /// halted on the GSP falcon.
+ ///
+ /// # Errors
+ ///
+ /// - `EIO` if SEC2 reports a failure, or if the GSP is not running RISC-V afterwards.
+ /// - `ETIMEDOUT` if SEC2 does not complete the reload within two seconds.
+ fn core_resume(&self) -> Result {
+ let Self {
+ gsp_falcon,
+ sec2_falcon,
+ dev,
+ ..
+ } = *self;
+
+ gsp_falcon.reset()?;
+
+ gsp_falcon.write_mailboxes(
+ Some(self.libos_dma_handle as u32),
+ Some((self.libos_dma_handle >> 32) as u32),
+ );
+
+ sec2_falcon.start()?;
+
+ gsp_falcon
+ .check_reload_completed(Delta::from_secs(2))
+ .inspect_err(|_| {
+ let mbox0 = sec2_falcon.read_mailbox0();
+ dev_err!(
+ dev,
+ "Timeout waiting for SEC2 to resume GSP-RM (SEC2 mbox0={:#x})\n",
+ mbox0
+ );
+ })?;
+
+ let sec2_mbox0 = sec2_falcon.read_mailbox0();
+ if sec2_mbox0 != 0 {
+ dev_err!(
+ dev,
+ "SEC2 reported error during core resume: {:#x}\n",
+ sec2_mbox0
+ );
+ return Err(EIO);
+ }
+
+ gsp_falcon.write_os_version(self.bootloader_app_version);
+
+ if !gsp_falcon.is_riscv_active() {
+ dev_err!(dev, "GSP RISC-V not active after core resume\n");
+ return Err(EIO);
+ }
+
+ Ok(())
+ }
+
+ /// Runs a Heavy-Secured (HS) binary on the GSP falcon, as a `GMCAPI_CMD_EXEC_HS_BINARY` event
+ /// requests, and then restarts GSP-RM.
+ ///
+ /// GSP-RM has placed the binary in the framebuffer, and the falcon's boot ROM (BROM) verifies
+ /// the binary's signature before the binary runs.
+ ///
+ /// # Errors
+ ///
+ /// - `EINVAL` if the payload is shorter than the parameter block, or the ucode id does not
+ /// fit the BROM register field.
+ /// - `ETIMEDOUT` if the RISC-V core does not suspend within two seconds, or the GSP falcon does
+ /// not halt within two seconds of starting the binary.
+ ///
+ /// Errors from [`Self::core_resume`] are propagated as-is.
+ #[expect(dead_code)]
+ fn handle_load_exec_hs_binary(&self, payload_0: &[u8], payload_1: &[u8]) -> Result {
+ let Self {
+ gsp_falcon, dev, ..
+ } = *self;
+ let params = read_params::<HsBinaryParams>(payload_0, payload_1)?;
+
+ self.reset_gsp_falcon_after_suspend()?;
+
+ gsp_falcon.with_fbif_transcfg(
+ HsBinaryParams::CTX_DMA,
+ |v| {
+ v.with_target(FalconFbifTarget::LocalFb)
+ .with_mem_type(FalconFbifMemType::Physical)
+ .with_engine_id_flag(FalconFbifEngineIdFlag::Bar2Fn0)
+ },
+ || {
+ if params.ucode_imem_size > 0 {
+ gsp_falcon.raw_dma_transfer(
+ HsBinaryParams::CTX_DMA,
+ params.imem_phys_addr,
+ FalconMem::ImemSecure,
+ FalconDmaSrcOffset::Va(params.ucode_imem_va),
+ params.ucode_imem_pa,
+ params.ucode_imem_size,
+ )?;
+ }
+
+ if params.ucode_dmem_size > 0 {
+ gsp_falcon.raw_dma_transfer(
+ HsBinaryParams::CTX_DMA,
+ params.dmem_phys_addr,
+ FalconMem::Dmem,
+ FalconDmaSrcOffset::from_dmem_va(params.ucode_dmem_va),
+ params.ucode_dmem_pa,
+ params.ucode_dmem_size,
+ )?;
+ }
+
+ gsp_falcon.pfalcon2.write(
+ Array::at(0),
+ regs::NV_PFALCON2_FALCON_BROM_PARAADDR::zeroed()
+ .with_value(params.hs_sig_dmem_addr),
+ );
+ gsp_falcon.pfalcon2.write_reg(
+ regs::NV_PFALCON2_FALCON_BROM_ENGIDMASK::zeroed()
+ .with_value(params.engine_id_mask),
+ );
+ gsp_falcon.pfalcon2.write_reg(
+ regs::NV_PFALCON2_FALCON_BROM_CURR_UCODE_ID::zeroed()
+ .with_ucode_id(u8::try_from(params.ucode_id).map_err(|_| EINVAL)?),
+ );
+ gsp_falcon.pfalcon2.write_reg(
+ regs::NV_PFALCON2_FALCON_MOD_SEL::zeroed().with_algo(FalconModSelAlgo::Rsa3k),
+ );
+
+ gsp_falcon.pfalcon.write_reg(
+ regs::NV_PFALCON_FALCON_BOOTVEC::zeroed().with_value(params.ucode_imem_va),
+ );
+
+ let (mbox0, _) = gsp_falcon
+ .boot(Some(FLCN_ERR_BINARY_NOT_STARTED), None)
+ .inspect_err(|_| {
+ dev_err!(
+ dev,
+ "Timeout waiting for HS binary to halt (mbox0={:#x})\n",
+ gsp_falcon.read_mailbox0()
+ );
+ })?;
+ dev_dbg!(dev, "HS binary halted with mbox0={:#x}\n", mbox0);
+
+ Ok(())
+ },
+ )?;
+
+ self.core_resume()
+ }
+}
+
impl<'gsp> super::Gsp<'gsp> {
/// Attempt to boot the GSP.
///
@@ -140,3 +346,50 @@ pub(crate) fn unload(
res.inspect(|()| dev_info!(dev, "GSP successfully unloaded\n"))
}
}
+
+/// Reads the parameter block of type `T` from the start of an event payload, which the ring may
+/// have split in two.
+///
+/// # Errors
+///
+/// - `EINVAL` if the payload is shorter than `T`.
+fn read_params<T: FromBytes + AsBytes + Zeroable>(payload_0: &[u8], payload_1: &[u8]) -> Result<T> {
+ let mut params = T::zeroed();
+
+ SBufferIter::new_reader([payload_0, payload_1]).read_exact(params.as_bytes_mut())?;
+
+ Ok(params)
+}
+
+/// Payload of a `GMCAPI_CMD_EXEC_HS_BINARY` event.
+///
+/// GSP-RM has written the code to `imem_phys_addr` and the data to `dmem_phys_addr` in the
+/// framebuffer before it sends the event.
+#[repr(C)]
+#[derive(Debug, Copy, Clone, Zeroable)]
+struct HsBinaryParams {
+ imem_phys_addr: u64,
+ dmem_phys_addr: u64,
+ _reserved64: [u64; 2],
+ ucode_imem_va: u32,
+ ucode_imem_pa: u32,
+ ucode_imem_size: u32,
+ ucode_dmem_va: u32,
+ ucode_dmem_pa: u32,
+ ucode_dmem_size: u32,
+ hs_sig_dmem_addr: u32,
+ engine_id_mask: u32,
+ ucode_id: u32,
+ _reserved32: [u32; 3],
+}
+
+impl HsBinaryParams {
+ /// Context DMA slot through which the binary is loaded.
+ const CTX_DMA: u32 = 0;
+}
+
+// SAFETY: This struct only contains integer types for which all bit patterns are valid.
+unsafe impl FromBytes for HsBinaryParams {}
+
+// SAFETY: This struct only contains integer types, laid out without padding.
+unsafe impl AsBytes for HsBinaryParams {}
diff --git a/drivers/gpu/nova-core/regs.rs b/drivers/gpu/nova-core/regs.rs
index feb37de69be5..ec8e05dc3351 100644
--- a/drivers/gpu/nova-core/regs.rs
+++ b/drivers/gpu/nova-core/regs.rs
@@ -18,6 +18,7 @@
DmaTrfCmdSize,
FalconCoreRev,
FalconCoreRevSubversion,
+ FalconFbifEngineIdFlag,
FalconFbifMemType,
FalconFbifTarget,
FalconMem,
@@ -286,6 +287,7 @@ pub(crate) fn usable_fb_size(self) -> u64 {
}
pub(crate) NV_PFALCON_FBIF_TRANSCFG(u32)[8] @ 0x00000600 {
+ 16:16 engine_id_flag => FalconFbifEngineIdFlag;
2:2 mem_type => FalconFbifMemType;
1:0 target ?=> FalconFbifTarget;
}
diff --git a/drivers/gpu/nova-core/sbuffer.rs b/drivers/gpu/nova-core/sbuffer.rs
index 3a41d224c77a..b8c01104c255 100644
--- a/drivers/gpu/nova-core/sbuffer.rs
+++ b/drivers/gpu/nova-core/sbuffer.rs
@@ -146,7 +146,6 @@ fn get_slice(&mut self, len: usize) -> Option<&'a [u8]> {
/// Ideally we would implement `Read`, but it is not available in `core`.
/// So mimic `std::io::Read::read_exact`.
- #[expect(unused)]
pub(crate) fn read_exact(&mut self, mut dst: &mut [u8]) -> Result {
while !dst.is_empty() {
match self.get_slice(dst.len()) {
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 16/33] gpu: nova-core: move the bootloader DMEM descriptor out of FWSEC
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (14 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 15/33] gpu: nova-core: add the r000 load-and-execute HS binary handler John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 17/33] gpu: nova-core: add the r000 load-and-execute bootloader handler John Hubbard
` (16 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The generic falcon bootloader reads its load parameters from a
descriptor that the driver places in DMEM. The r000 load-and-execute
bootloader event carries such a descriptor, so its handler in the
following patch needs the descriptor type.
The descriptor was defined in the FWSEC code, which was its only user.
Move the definition, unchanged, into the generic bootloader module. Its
fields become pub(crate) so that the FWSEC code can still fill them in.
No functional changes.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
.../nova-core/firmware/fwsec/bootloader.rs | 52 ++-----------------
.../gpu/nova-core/firmware/gen_bootloader.rs | 48 +++++++++++++++++
2 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
index 1584a4b814d2..96ad223df16f 100644
--- a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
@@ -35,60 +35,16 @@
},
firmware::{
fwsec::FwsecFirmware,
- gen_bootloader::GenericBootloader, //
+ gen_bootloader::{
+ BootloaderDmemDescV2,
+ GenericBootloader, //
+ },
},
gpu::Chipset,
num::FromSafeCast, //
regs,
};
-/// Structure used by the boot-loader to load the rest of the code.
-///
-/// This has to be filled by the GPU driver and copied into DMEM at offset
-/// [`BootloaderDesc.dmem_load_off`].
-#[repr(C, packed)]
-#[derive(Debug, Clone)]
-struct BootloaderDmemDescV2 {
- /// Reserved, should always be first element.
- reserved: [u32; 4],
- /// 16B signature for secure code, 0s if no secure code.
- signature: [u32; 4],
- /// DMA context used by the bootloader while loading code/data.
- ctx_dma: u32,
- /// 256B-aligned physical FB address where code is located.
- code_dma_base: u64,
- /// Offset from `code_dma_base` where the non-secure code is located.
- ///
- /// Also used as destination IMEM offset of non-secure code as the DMA firmware object is
- /// expected to be a mirror image of its loaded state.
- ///
- /// Must be multiple of 256.
- non_sec_code_off: u32,
- /// Size of the non-secure code part.
- non_sec_code_size: u32,
- /// Offset from `code_dma_base` where the secure code is located (must be multiple of 256).
- ///
- /// Also used as destination IMEM offset of secure code as the DMA firmware object is expected
- /// to be a mirror image of its loaded state.
- ///
- /// Must be multiple of 256.
- sec_code_off: u32,
- /// Size of the secure code part.
- sec_code_size: u32,
- /// Code entry point invoked by the bootloader after code is loaded.
- code_entry_point: u32,
- /// 256B-aligned physical FB address where data is located.
- data_dma_base: u64,
- /// Size of data block (should be multiple of 256B).
- data_size: u32,
- /// Number of arguments to be passed to the target firmware being loaded.
- argc: u32,
- /// Arguments to be passed to the target firmware being loaded.
- argv: u32,
-}
-// SAFETY: This struct doesn't contain uninitialized bytes and doesn't have interior mutability.
-unsafe impl AsBytes for BootloaderDmemDescV2 {}
-
/// Wrapper for [`FwsecFirmware`] that includes the bootloader performing the actual load
/// operation.
pub(crate) struct FwsecFirmwareWithBl<'a> {
diff --git a/drivers/gpu/nova-core/firmware/gen_bootloader.rs b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
index 65af57a34940..943a5bc16e93 100644
--- a/drivers/gpu/nova-core/firmware/gen_bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
@@ -12,6 +12,7 @@
Alignable,
Alignment, //
},
+ transmute::AsBytes, //
};
use crate::{
@@ -27,6 +28,53 @@
num::FromSafeCast, //
};
+/// Structure used by the boot-loader to load the rest of the code.
+///
+/// This has to be filled by the GPU driver and copied into DMEM at offset
+/// [`BootloaderDesc.dmem_load_off`].
+#[repr(C, packed)]
+#[derive(Debug, Clone)]
+pub(crate) struct BootloaderDmemDescV2 {
+ /// Reserved, should always be first element.
+ pub(crate) reserved: [u32; 4],
+ /// 16B signature for secure code, 0s if no secure code.
+ pub(crate) signature: [u32; 4],
+ /// DMA context used by the bootloader while loading code/data.
+ pub(crate) ctx_dma: u32,
+ /// 256B-aligned physical FB address where code is located.
+ pub(crate) code_dma_base: u64,
+ /// Offset from `code_dma_base` where the non-secure code is located.
+ ///
+ /// Also used as destination IMEM offset of non-secure code as the DMA firmware object is
+ /// expected to be a mirror image of its loaded state.
+ ///
+ /// Must be multiple of 256.
+ pub(crate) non_sec_code_off: u32,
+ /// Size of the non-secure code part.
+ pub(crate) non_sec_code_size: u32,
+ /// Offset from `code_dma_base` where the secure code is located (must be multiple of 256).
+ ///
+ /// Also used as destination IMEM offset of secure code as the DMA firmware object is expected
+ /// to be a mirror image of its loaded state.
+ ///
+ /// Must be multiple of 256.
+ pub(crate) sec_code_off: u32,
+ /// Size of the secure code part.
+ pub(crate) sec_code_size: u32,
+ /// Code entry point invoked by the bootloader after code is loaded.
+ pub(crate) code_entry_point: u32,
+ /// 256B-aligned physical FB address where data is located.
+ pub(crate) data_dma_base: u64,
+ /// Size of data block (should be multiple of 256B).
+ pub(crate) data_size: u32,
+ /// Number of arguments to be passed to the target firmware being loaded.
+ pub(crate) argc: u32,
+ /// Arguments to be passed to the target firmware being loaded.
+ pub(crate) argv: u32,
+}
+// SAFETY: This struct doesn't contain uninitialized bytes and doesn't have interior mutability.
+unsafe impl AsBytes for BootloaderDmemDescV2 {}
+
/// The generic falcon bootloader image and its IMEM load parameters.
pub(crate) struct GenericBootloader {
/// Bootloader code, zero-padded to a whole number of falcon memory blocks.
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 17/33] gpu: nova-core: add the r000 load-and-execute bootloader handler
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (15 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 16/33] gpu: nova-core: move the bootloader DMEM descriptor out of FWSEC John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 3:32 ` Timur Tabi
2026-09-18 1:07 ` [PATCH v3 18/33] gpu: nova-core: gsp: add the GMC boot event dispatcher John Hubbard
` (15 subsequent siblings)
32 siblings, 1 reply; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
On Turing and GA100, whose GSP boots through the generic falcon
bootloader, GSP-RM sends a second kind of load-and-execute event. It
requests the driver to run an image on the GSP falcon, and it carries a
descriptor that holds the image's addresses. The driver does not copy
the image itself. It loads the generic bootloader and the descriptor
into the falcon, and the bootloader reads the descriptor from DMEM and
copies the image, as in Open RM.
Add the handler. It runs the bootloader through the FBIF aperture that
the descriptor's address space and cache attribute select, and it
rejects the event on a chipset that boots without the bootloader.
Pair the bootloader with a descriptor as the firmware that the falcon
loads by PIO, and load FWSEC through the same pairing in place of the
FWSEC wrapper's own load implementation.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
.../nova-core/firmware/fwsec/bootloader.rs | 53 ++------
.../gpu/nova-core/firmware/gen_bootloader.rs | 85 ++++++++++--
drivers/gpu/nova-core/gsp/boot.rs | 125 +++++++++++++++++-
3 files changed, 209 insertions(+), 54 deletions(-)
diff --git a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
index 96ad223df16f..48e0c9672412 100644
--- a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
@@ -16,22 +16,16 @@
register::Array,
Io, //
},
- prelude::*,
- transmute::AsBytes,
+ prelude::*, //
};
use crate::{
falcon::{
gsp::Gsp,
Falcon,
- FalconBromParams,
FalconDmaLoadable,
FalconFbifMemType,
- FalconFbifTarget,
- FalconFirmware,
- FalconPioDmemLoadTarget,
- FalconPioImemLoadTarget,
- FalconPioLoadable, //
+ FalconFbifTarget, //
},
firmware::{
fwsec::FwsecFirmware,
@@ -50,16 +44,17 @@
pub(crate) struct FwsecFirmwareWithBl<'a> {
/// DMA object the bootloader will copy the firmware from.
_firmware_dma: Coherent<'a, [u8]>,
+ /// The bootloader that copies the firmware.
bootloader: GenericBootloader,
/// Descriptor to be loaded into DMEM for the bootloader to read.
dmem_desc: BootloaderDmemDescV2,
- /// BROM parameters of the loaded firmware.
- brom_params: FalconBromParams,
}
impl<'a> FwsecFirmwareWithBl<'a> {
- /// Loads the bootloader firmware for `dev` and `chipset`, and wrap `firmware` so it can be
- /// loaded using it.
+ /// Wraps `firmware` so that the generic bootloader can load it.
+ ///
+ /// The bootloader image for `chipset` is loaded here, to run from the last blocks of the
+ /// IMEM of `falcon`. The firmware is mapped for `dev`.
pub(crate) fn new(
firmware: FwsecFirmware,
dev: &'a Device<device::Bound>,
@@ -141,7 +136,6 @@ pub(crate) fn new(
_firmware_dma: firmware_dma,
bootloader,
dmem_desc,
- brom_params: firmware.brom_params(),
})
}
@@ -155,7 +149,7 @@ pub(crate) fn run(&self, dev: &Device<device::Bound>, falcon: &Falcon<'_, Gsp>)
.reset()
.inspect_err(|e| dev_err!(dev, "Failed to reset GSP falcon: {:?}\n", e))?;
falcon
- .pio_load(self)
+ .pio_load(&self.bootloader.with_descriptor(&self.dmem_desc))
.inspect_err(|e| dev_err!(dev, "Failed to load FWSEC firmware: {:?}\n", e))?;
// Configure DMA index for the bootloader to fetch the FWSEC firmware from system memory.
@@ -179,34 +173,3 @@ pub(crate) fn run(&self, dev: &Device<device::Bound>, falcon: &Falcon<'_, Gsp>)
}
}
}
-
-impl FalconFirmware for FwsecFirmwareWithBl<'_> {
- type Target = Gsp;
-
- fn brom_params(&self) -> FalconBromParams {
- self.brom_params.clone()
- }
-
- fn boot_addr(&self) -> u32 {
- // On V2 platforms, the boot address is extracted from the generic bootloader, because the
- // gbl is what actually copies FWSEC into memory, so that is what needs to be booted.
- self.bootloader.boot_addr()
- }
-}
-
-impl FalconPioLoadable for FwsecFirmwareWithBl<'_> {
- fn imem_sec_load_params(&self) -> Option<FalconPioImemLoadTarget<'_>> {
- None
- }
-
- fn imem_ns_load_params(&self) -> Option<FalconPioImemLoadTarget<'_>> {
- Some(self.bootloader.imem_load_params())
- }
-
- fn dmem_load_params(&self) -> FalconPioDmemLoadTarget<'_> {
- FalconPioDmemLoadTarget {
- data: self.dmem_desc.as_bytes(),
- dst_start: 0,
- }
- }
-}
diff --git a/drivers/gpu/nova-core/firmware/gen_bootloader.rs b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
index 943a5bc16e93..7e826bd6497b 100644
--- a/drivers/gpu/nova-core/firmware/gen_bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
@@ -12,28 +12,36 @@
Alignable,
Alignment, //
},
- transmute::AsBytes, //
+ transmute::{
+ AsBytes,
+ FromBytes, //
+ },
};
use crate::{
falcon::{
self,
- FalconPioImemLoadTarget, //
+ gsp::Gsp,
+ FalconBromParams,
+ FalconFirmware,
+ FalconPioDmemLoadTarget,
+ FalconPioImemLoadTarget,
+ FalconPioLoadable, //
},
firmware::tlv::{
request_tlv, //
Tlv,
},
gpu::Chipset,
- num::FromSafeCast, //
+ num::{
+ self,
+ FromSafeCast, //
+ },
};
-/// Structure used by the boot-loader to load the rest of the code.
-///
-/// This has to be filled by the GPU driver and copied into DMEM at offset
-/// [`BootloaderDesc.dmem_load_off`].
+/// Descriptor that the generic bootloader reads from DMEM offset 0 to find the image to load.
#[repr(C, packed)]
-#[derive(Debug, Clone)]
+#[derive(Debug, Clone, Zeroable)]
pub(crate) struct BootloaderDmemDescV2 {
/// Reserved, should always be first element.
pub(crate) reserved: [u32; 4],
@@ -72,9 +80,18 @@ pub(crate) struct BootloaderDmemDescV2 {
/// Arguments to be passed to the target firmware being loaded.
pub(crate) argv: u32,
}
+
+impl BootloaderDmemDescV2 {
+ /// Size of the descriptor in bytes, as the load-and-execute event states it.
+ pub(crate) const SIZE: u32 = num::usize_into_u32::<{ size_of::<BootloaderDmemDescV2>() }>();
+}
+
// SAFETY: This struct doesn't contain uninitialized bytes and doesn't have interior mutability.
unsafe impl AsBytes for BootloaderDmemDescV2 {}
+// SAFETY: This struct only contains integer types for which all bit patterns are valid.
+unsafe impl FromBytes for BootloaderDmemDescV2 {}
+
/// The generic falcon bootloader image and its IMEM load parameters.
pub(crate) struct GenericBootloader {
/// Bootloader code, zero-padded to a whole number of falcon memory blocks.
@@ -144,4 +161,56 @@ pub(crate) fn imem_load_params(&self) -> FalconPioImemLoadTarget<'_> {
start_tag: self.start_tag,
}
}
+
+ /// Returns the bootloader and `descriptor` as one firmware that [`Falcon::pio_load`] accepts.
+ pub(crate) fn with_descriptor<'a>(
+ &'a self,
+ dmem_desc: &'a BootloaderDmemDescV2,
+ ) -> GenericBootloaderLoad<'a> {
+ GenericBootloaderLoad {
+ bootloader: self,
+ dmem_desc,
+ }
+ }
+}
+
+/// The generic bootloader together with the descriptor that it reads from DMEM offset 0.
+pub(crate) struct GenericBootloaderLoad<'a> {
+ bootloader: &'a GenericBootloader,
+ dmem_desc: &'a BootloaderDmemDescV2,
+}
+
+impl FalconFirmware for GenericBootloaderLoad<'_> {
+ type Target = Gsp;
+
+ fn brom_params(&self) -> FalconBromParams {
+ // The bootloader is unsigned, and the falcon HALs of the chipsets that load it this way
+ // write no BROM registers, so these values never reach hardware.
+ FalconBromParams {
+ pkc_data_offset: 0,
+ engine_id_mask: 0,
+ ucode_id: 0,
+ }
+ }
+
+ fn boot_addr(&self) -> u32 {
+ self.bootloader.boot_addr()
+ }
+}
+
+impl FalconPioLoadable for GenericBootloaderLoad<'_> {
+ fn imem_sec_load_params(&self) -> Option<FalconPioImemLoadTarget<'_>> {
+ None
+ }
+
+ fn imem_ns_load_params(&self) -> Option<FalconPioImemLoadTarget<'_>> {
+ Some(self.bootloader.imem_load_params())
+ }
+
+ fn dmem_load_params(&self) -> FalconPioDmemLoadTarget<'_> {
+ FalconPioDmemLoadTarget {
+ data: self.dmem_desc.as_bytes(),
+ dst_start: 0,
+ }
+ }
}
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index 3cfb21964250..ee366d782218 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -30,7 +30,13 @@
FalconModSelAlgo,
FLCN_ERR_BINARY_NOT_STARTED, //
},
- firmware::gsp::GspFirmware,
+ firmware::{
+ gen_bootloader::{
+ BootloaderDmemDescV2,
+ GenericBootloader, //
+ },
+ gsp::GspFirmware, //
+ },
gsp::{
cmdq::Cmdq,
commands, //
@@ -41,6 +47,8 @@
/// The falcons, device and boot parameters that the load-and-execute event handlers share.
struct LoadExecContext<'a, 'gpu> {
+ /// The generic falcon bootloader, on chipsets that boot through it.
+ bootloader: Option<&'a GenericBootloader>,
gsp_falcon: &'a Falcon<'gpu, Gsp>,
sec2_falcon: &'a Falcon<'gpu, Sec2>,
dev: &'a device::Device,
@@ -133,6 +141,75 @@ fn core_resume(&self) -> Result {
Ok(())
}
+ /// Runs the generic bootloader on the GSP falcon, as a `GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER`
+ /// event requests, and then restarts GSP-RM.
+ ///
+ /// The descriptor that the event carries names the image that the bootloader loads.
+ ///
+ /// # Errors
+ ///
+ /// - `EINVAL` if this chipset boots without the generic bootloader, if the payload is shorter
+ /// than the parameter block, if the descriptor is not the size that this driver defines for
+ /// it, or if the event names a context DMA slot or an aperture that does not exist.
+ /// - `ETIMEDOUT` if the RISC-V core does not suspend within two seconds, or the GSP falcon does
+ /// not halt within two seconds of starting the image.
+ ///
+ /// Errors from [`Self::core_resume`] are propagated as-is.
+ #[expect(dead_code)]
+ fn handle_load_exec_bootloader(&self, payload_0: &[u8], payload_1: &[u8]) -> Result {
+ let Self {
+ gsp_falcon, dev, ..
+ } = *self;
+ let Some(bootloader) = self.bootloader else {
+ dev_err!(
+ dev,
+ "GSP asked for the generic bootloader, which this chipset does not use\n"
+ );
+ return Err(EINVAL);
+ };
+ let params = read_params::<LoadExecGenericBootloaderParams>(payload_0, payload_1)?;
+
+ if params.dmem_desc_size != BootloaderDmemDescV2::SIZE {
+ dev_err!(
+ dev,
+ "Load-exec descriptor is {} bytes, expected {}\n",
+ params.dmem_desc_size,
+ BootloaderDmemDescV2::SIZE
+ );
+ return Err(EINVAL);
+ }
+
+ let fbif_target = params.fbif_target()?;
+
+ self.reset_gsp_falcon_after_suspend()?;
+
+ gsp_falcon.with_fbif_transcfg(
+ params.dmem_desc.ctx_dma,
+ |v| {
+ v.with_target(fbif_target)
+ .with_mem_type(FalconFbifMemType::Physical)
+ },
+ || {
+ gsp_falcon.pio_load(&bootloader.with_descriptor(¶ms.dmem_desc))?;
+
+ let (mbox0, _) = gsp_falcon
+ .boot(Some(FLCN_ERR_BINARY_NOT_STARTED), None)
+ .inspect_err(|_| {
+ dev_err!(
+ dev,
+ "Timeout waiting for the loaded image to halt (mbox0={:#x})\n",
+ gsp_falcon.read_mailbox0()
+ );
+ })?;
+ dev_dbg!(dev, "Loaded image halted with mbox0={:#x}\n", mbox0);
+
+ Ok(())
+ },
+ )?;
+
+ self.core_resume()
+ }
+
/// Runs a Heavy-Secured (HS) binary on the GSP falcon, as a `GMCAPI_CMD_EXEC_HS_BINARY` event
/// requests, and then restarts GSP-RM.
///
@@ -361,6 +438,52 @@ fn read_params<T: FromBytes + AsBytes + Zeroable>(payload_0: &[u8], payload_1: &
Ok(params)
}
+/// Payload of a `GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER` event.
+///
+/// The descriptor carries the code and data addresses, and `addr_space` and `cpu_cache_attrib`
+/// say which FBIF (framebuffer interface) aperture reaches them.
+#[repr(C)]
+#[derive(Zeroable)]
+struct LoadExecGenericBootloaderParams {
+ dmem_desc: BootloaderDmemDescV2,
+ dmem_desc_size: u32,
+ addr_space: u32,
+ cpu_cache_attrib: u32,
+ _reserved: [u32; 4],
+}
+
+impl LoadExecGenericBootloaderParams {
+ const ADDR_SYSMEM: u32 = 1;
+ const ADDR_FBMEM: u32 = 2;
+ const NV_MEMORY_CACHED: u32 = 0;
+ const NV_MEMORY_UNCACHED: u32 = 1;
+
+ /// Returns the FBIF aperture that reaches the image.
+ ///
+ /// # Errors
+ ///
+ /// - `EINVAL` if the pair of address space and cache attribute is not one that this driver
+ /// maps.
+ fn fbif_target(&self) -> Result<FalconFbifTarget> {
+ match (self.addr_space, self.cpu_cache_attrib) {
+ (Self::ADDR_FBMEM, _) => Ok(FalconFbifTarget::LocalFb),
+ (Self::ADDR_SYSMEM, Self::NV_MEMORY_CACHED) => Ok(FalconFbifTarget::CoherentSysmem),
+ (Self::ADDR_SYSMEM, Self::NV_MEMORY_UNCACHED) => {
+ Ok(FalconFbifTarget::NoncoherentSysmem)
+ }
+ _ => Err(EINVAL),
+ }
+ }
+}
+
+// SAFETY: The nested descriptor is `FromBytes`, and every other field is an integer type for
+// which all bit patterns are valid.
+unsafe impl FromBytes for LoadExecGenericBootloaderParams {}
+
+// SAFETY: The nested descriptor is `AsBytes`, every other field is an integer type, and the
+// layout has no padding.
+unsafe impl AsBytes for LoadExecGenericBootloaderParams {}
+
/// Payload of a `GMCAPI_CMD_EXEC_HS_BINARY` event.
///
/// GSP-RM has written the code to `imem_phys_addr` and the data to `dmem_phys_addr` in the
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [PATCH v3 17/33] gpu: nova-core: add the r000 load-and-execute bootloader handler
2026-09-18 1:07 ` [PATCH v3 17/33] gpu: nova-core: add the r000 load-and-execute bootloader handler John Hubbard
@ 2026-09-18 3:32 ` Timur Tabi
0 siblings, 0 replies; 35+ messages in thread
From: Timur Tabi @ 2026-09-18 3:32 UTC (permalink / raw)
To: Alexandre Courbot, dakr, John Hubbard
Cc: lossin, a.hindborg, boqun.feng, Zhi Wang, simona, tmgross,
alex.gaynor, nova-gpu, linux-kernel, ojeda, bjorn3_gh,
Eliot Courtney, airlied, aliceryhl, bhelgaas, gary,
Alistair Popple
On Thu, 2026-09-17 at 18:07 -0700, John Hubbard wrote:
> @@ -133,6 +141,75 @@ fn core_resume(&self) -> Result {
> Ok(())
> }
>
> + /// Runs the generic bootloader on the GSP falcon, as a
> `GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER`
> + /// event requests, and then restarts GSP-RM.
> + ///
> + /// The descriptor that the event carries names the image that the bootloader loads.
> + ///
> + /// # Errors
> + ///
> + /// - `EINVAL` if this chipset boots without the generic bootloader, if the payload is
> shorter
> + /// than the parameter block, if the descriptor is not the size that this driver
> defines for
> + /// it, or if the event names a context DMA slot or an aperture that does not exist.
> + /// - `ETIMEDOUT` if the RISC-V core does not suspend within two seconds, or the GSP
> falcon does
> + /// not halt within two seconds of starting the image.
> + ///
> + /// Errors from [`Self::core_resume`] are propagated as-is.
> + #[expect(dead_code)]
> + fn handle_load_exec_bootloader(&self, payload_0: &[u8], payload_1: &[u8]) -> Result {
> + let Self {
> + gsp_falcon, dev, ..
> + } = *self;
Why do you create local variables of two member of LoadExecContext? By doing this:
let Self {
gsp_falcon, dev, ..
} = *self;
you force every member of LoadExecContext to implement Copy, even the ones you are not copying
here.
> + if params.dmem_desc_size != BootloaderDmemDescV2::SIZE {
> + dev_err!(
> + dev,
> + "Load-exec descriptor is {} bytes, expected {}\n",
> + params.dmem_desc_size,
> + BootloaderDmemDescV2::SIZE
> + );
So here, why not just do self.dev?
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v3 18/33] gpu: nova-core: gsp: add the GMC boot event dispatcher
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (16 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 17/33] gpu: nova-core: add the r000 load-and-execute bootloader handler John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 19/33] gpu: nova-core: gsp: rename the static configuration type John Hubbard
` (14 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The two load-and-execute events of the r000 boot protocol arrive as GMC
events, each named by a command id.
Nova-core had a handler for each event, but no dispatcher that read the
id and picked the handler, and each handler ended with its own copy of
the core resume.
Add the dispatcher, and move the core resume out of the two handlers
into it. Each handler now ends with the falcon halted, and the
dispatcher runs the core resume once after whichever handler ran. The
dispatcher has no caller yet.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/boot.rs | 69 +++++++++++++++++++++++--------
drivers/gpu/nova-core/gsp/fw.rs | 10 +++++
2 files changed, 62 insertions(+), 17 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index ee366d782218..c805b42dd7bc 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -39,7 +39,11 @@
},
gsp::{
cmdq::Cmdq,
- commands, //
+ commands,
+ fw::{
+ GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER,
+ GMCAPI_CMD_EXEC_HS_BINARY, //
+ }, //
},
regs,
sbuffer::SBufferIter, //
@@ -141,10 +145,51 @@ fn core_resume(&self) -> Result {
Ok(())
}
+ /// Runs the load-and-execute handler that `command_id` names on the event payload, which the
+ /// ring may have split in two, and then runs the core resume.
+ ///
+ /// # Errors
+ ///
+ /// - `EINVAL` if `command_id` is not a load-and-execute command.
+ ///
+ /// Errors from the handlers and from [`Self::core_resume`] are propagated as-is.
+ #[expect(dead_code)]
+ fn dispatch_gmc_boot_event(
+ &self,
+ command_id: u32,
+ payload_0: &[u8],
+ payload_1: &[u8],
+ ) -> Result {
+ let handled = match command_id {
+ GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER => {
+ self.handle_load_exec_bootloader(payload_0, payload_1)
+ }
+ GMCAPI_CMD_EXEC_HS_BINARY => self.handle_load_exec_hs_binary(payload_0, payload_1),
+ _ => {
+ dev_err!(
+ self.dev,
+ "Unexpected GMC boot event: command_id={:#010x}\n",
+ command_id
+ );
+ return Err(EINVAL);
+ }
+ };
+
+ handled.and_then(|()| self.core_resume()).inspect_err(|e| {
+ dev_err!(
+ self.dev,
+ "GMC boot event {:#010x} failed: {:?}\n",
+ command_id,
+ e
+ );
+ })
+ }
+
/// Runs the generic bootloader on the GSP falcon, as a `GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER`
- /// event requests, and then restarts GSP-RM.
+ /// event requests.
///
- /// The descriptor that the event carries names the image that the bootloader loads.
+ /// The descriptor that the event carries names the image that the bootloader loads. The GSP
+ /// falcon is left halted.
///
/// # Errors
///
@@ -153,9 +198,6 @@ fn core_resume(&self) -> Result {
/// it, or if the event names a context DMA slot or an aperture that does not exist.
/// - `ETIMEDOUT` if the RISC-V core does not suspend within two seconds, or the GSP falcon does
/// not halt within two seconds of starting the image.
- ///
- /// Errors from [`Self::core_resume`] are propagated as-is.
- #[expect(dead_code)]
fn handle_load_exec_bootloader(&self, payload_0: &[u8], payload_1: &[u8]) -> Result {
let Self {
gsp_falcon, dev, ..
@@ -205,16 +247,14 @@ fn handle_load_exec_bootloader(&self, payload_0: &[u8], payload_1: &[u8]) -> Res
Ok(())
},
- )?;
-
- self.core_resume()
+ )
}
/// Runs a Heavy-Secured (HS) binary on the GSP falcon, as a `GMCAPI_CMD_EXEC_HS_BINARY` event
- /// requests, and then restarts GSP-RM.
+ /// requests.
///
/// GSP-RM has placed the binary in the framebuffer, and the falcon's boot ROM (BROM) verifies
- /// the binary's signature before the binary runs.
+ /// the binary's signature before the binary runs. The GSP falcon is left halted.
///
/// # Errors
///
@@ -222,9 +262,6 @@ fn handle_load_exec_bootloader(&self, payload_0: &[u8], payload_1: &[u8]) -> Res
/// fit the BROM register field.
/// - `ETIMEDOUT` if the RISC-V core does not suspend within two seconds, or the GSP falcon does
/// not halt within two seconds of starting the binary.
- ///
- /// Errors from [`Self::core_resume`] are propagated as-is.
- #[expect(dead_code)]
fn handle_load_exec_hs_binary(&self, payload_0: &[u8], payload_1: &[u8]) -> Result {
let Self {
gsp_falcon, dev, ..
@@ -297,9 +334,7 @@ fn handle_load_exec_hs_binary(&self, payload_0: &[u8], payload_1: &[u8]) -> Resu
Ok(())
},
- )?;
-
- self.core_resume()
+ )
}
}
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index f23d071f0e16..e86283f67358 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -1055,6 +1055,16 @@ pub(crate) struct GmcApiHeader {
/// Bits of [`GmcApiHeader::command`] that hold the command id. The high byte holds flags.
const GMCAPI_COMMAND_ID_MASK: u32 = 0x00ff_ffff;
+/// GMC event that requests the driver to run the generic falcon bootloader on the descriptor that
+/// the event carries.
+pub(crate) const GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER: u32 =
+ r000_00::GMCAPI_COMMANDS_GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER;
+
+/// GMC event that requests the driver to run a Heavy-Secured (HS) binary that GSP-RM has placed in
+/// the framebuffer.
+pub(crate) const GMCAPI_CMD_EXEC_HS_BINARY: u32 =
+ r000_00::GMCAPI_COMMANDS_GMCAPI_CMD_EXEC_HS_BINARY;
+
static_assert!(size_of::<GmcApiHeader>() == size_of::<r000_00::GMCAPI_HEADER>());
static_assert!(
core::mem::offset_of!(GmcApiHeader, command)
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 19/33] gpu: nova-core: gsp: rename the static configuration type
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (17 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 18/33] gpu: nova-core: gsp: add the GMC boot event dispatcher John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 20/33] gpu: nova-core: gsp: return the static GPU configuration from boot John Hubbard
` (13 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The following patches fill the type that holds GSP-RM's static GPU
configuration from the r000 GSP_INIT reply, where no command fetches
the configuration.
The type was named after the r570 command that fetched it.
Name the type after its contents.
No functional changes.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gpu.rs | 4 ++--
drivers/gpu/nova-core/gsp.rs | 2 +-
drivers/gpu/nova-core/gsp/commands.rs | 12 ++++++------
3 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index 9e0570e97cfe..e7f1458330b4 100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -34,7 +34,7 @@
fsp::Fsp,
gsp::{
self,
- commands::GetGspStaticInfoReply,
+ commands::GspStaticInfo,
Gsp,
GspBootContext, //
},
@@ -314,7 +314,7 @@ pub(crate) struct Gpu<'gpu> {
#[pin]
_gsp_irq: GspIrq<'gpu>,
/// Static GPU information as provided by the GSP.
- gsp_static_info: GetGspStaticInfoReply,
+ gsp_static_info: GspStaticInfo,
/// GPU memory manager owning memory management resources.
///
/// Must be kept declared *before* `gsp_resources`, so that its components are dropped while
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index c9b7498639b7..6e7a057188c5 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -221,7 +221,7 @@ pub(crate) fn new(
}
/// Query the GSP for the static GPU information.
- pub(crate) fn get_static_info(&self) -> Result<commands::GetGspStaticInfoReply> {
+ pub(crate) fn get_static_info(&self) -> Result<commands::GspStaticInfo> {
self.cmdq.send_command(commands::GetGspStaticInfo)
}
}
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index a01ab14299c6..b3b28e397ebd 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -197,7 +197,7 @@ pub(crate) fn wait_gsp_init_done(cmdq: &Cmdq<'_>) -> Result {
impl CommandToGsp for GetGspStaticInfo {
const FUNCTION: MsgFunction = MsgFunction::GetGspStaticInfo;
type Command = fw::commands::GspStaticConfigInfo;
- type Reply = GetGspStaticInfoReply;
+ type Reply = GspStaticInfo;
type InitError = Infallible;
fn init(&self) -> impl Init<Self::Command, Self::InitError> {
@@ -206,7 +206,7 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
}
/// The reply from the GSP to the [`GetGspStaticInfo`] command.
-pub(crate) struct GetGspStaticInfoReply {
+pub(crate) struct GspStaticInfo {
gpu_name: [u8; 64],
/// BAR1 Page Directory Entry base address.
pub(crate) bar1_pde_base: u64,
@@ -216,7 +216,7 @@ pub(crate) struct GetGspStaticInfoReply {
pub(crate) total_fb_end: u64,
}
-impl MessageFromGsp for GetGspStaticInfoReply {
+impl MessageFromGsp for GspStaticInfo {
const FUNCTION: MsgFunction = MsgFunction::GetGspStaticInfo;
type Message = fw::commands::GspStaticConfigInfo;
type InitError = Error;
@@ -231,7 +231,7 @@ fn read(
}
let total_fb_end = msg.total_fb_end().ok_or(EINVAL)?;
- Ok(GetGspStaticInfoReply {
+ Ok(GspStaticInfo {
gpu_name: msg.gpu_name_str(),
bar1_pde_base: msg.bar1_pde_base(),
usable_fb_regions,
@@ -240,7 +240,7 @@ fn read(
}
}
-/// Error type for [`GetGspStaticInfoReply::gpu_name`].
+/// Error type for [`GspStaticInfo::gpu_name`].
#[derive(Debug)]
pub(crate) enum GpuNameError {
/// The GPU name string does not contain a null terminator.
@@ -251,7 +251,7 @@ pub(crate) enum GpuNameError {
InvalidUtf8(Utf8Error),
}
-impl GetGspStaticInfoReply {
+impl GspStaticInfo {
/// Returns the name of the GPU as a string.
///
/// Returns an error if the string given by the GSP does not contain a null terminator or
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 20/33] gpu: nova-core: gsp: return the static GPU configuration from boot
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (18 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 19/33] gpu: nova-core: gsp: rename the static configuration type John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 21/33] gpu: nova-core: gsp: add the GSP_INIT request builder John Hubbard
` (12 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
Once GSP-RM has started, it reports the static GPU configuration: the
GPU's name, its usable framebuffer regions and its BAR1 page directory.
Everything that the driver builds on top of the GSP depends on that
configuration. In the r000 boot protocol the configuration arrives as
the reply to the request that ends boot, so the boot sequence holds it
when it returns.
Nova-core fetched the configuration in a separate step after boot, once
the interrupt handler was registered, so on r000 the boot sequence would
have had to keep the reply for that step to read.
Return the configuration from the boot sequence together with the unload
bundle, and drop the separate step.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gpu.rs | 44 ++++++++++++++-------------
drivers/gpu/nova-core/gsp.rs | 13 +++++---
drivers/gpu/nova-core/gsp/boot.rs | 15 ++++++---
drivers/gpu/nova-core/gsp/commands.rs | 2 +-
4 files changed, 42 insertions(+), 32 deletions(-)
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index e7f1458330b4..0ed0f4722dc5 100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -34,7 +34,6 @@
fsp::Fsp,
gsp::{
self,
- commands::GspStaticInfo,
Gsp,
GspBootContext, //
},
@@ -298,8 +297,15 @@ struct GspResources<'gpu> {
/// GSP runtime data.
#[pin]
gsp: Gsp<'gpu>,
- /// GSP unload firmware bundle, if any.
- unload_bundle: Option<gsp::UnloadBundle<'gpu>>,
+ /// The static GPU configuration and the unload bundle that the boot sequence returned.
+ boot_result: gsp::BootResult<'gpu>,
+}
+
+impl GspResources<'_> {
+ /// Returns the static GPU configuration that GSP-RM reported at boot.
+ fn static_info(&self) -> &gsp::commands::GspStaticInfo {
+ &self.boot_result.static_info
+ }
}
/// Structure holding the resources required to operate the GPU.
@@ -313,8 +319,6 @@ pub(crate) struct Gpu<'gpu> {
/// before the GSP is unloaded.
#[pin]
_gsp_irq: GspIrq<'gpu>,
- /// Static GPU information as provided by the GSP.
- gsp_static_info: GspStaticInfo,
/// GPU memory manager owning memory management resources.
///
/// Must be kept declared *before* `gsp_resources`, so that its components are dropped while
@@ -347,7 +351,7 @@ fn drop(self: Pin<&mut Self>) {
let this = self.project();
let device = *this.device;
let bar = *this.bar;
- let bundle = this.unload_bundle.take();
+ let bundle = this.boot_result.unload_bundle.take();
let _ = this
.gsp
@@ -416,10 +420,10 @@ pub(crate) fn new<'a>(
gsp <- Gsp::new(pdev, bar),
- // This member must be initialized last, so the `UnloadBundle` can never be dropped
- // from outside of the constructed `GspResources`, ensuring that the unload sequence
- // is properly run in case of failure.
- unload_bundle: gsp.boot(GspBootContext {
+ // This member must be initialized last, so that the unload bundle can never be
+ // dropped from outside the constructed `GspResources`, and the unload sequence runs
+ // on a failure.
+ boot_result: gsp.boot(GspBootContext {
pdev,
bar,
chipset: spec.chipset,
@@ -454,9 +458,8 @@ pub(crate) fn new<'a>(
gsp_resources.gsp.cmdq.drain()?;
},
- gsp_static_info: {
- // Obtain and display basic GPU information.
- let info = gsp_resources.gsp.get_static_info()?;
+ _: {
+ let info = gsp_resources.static_info();
match info.gpu_name() {
Ok(name) => dev_info!(dev, "GPU name: {}\n", name),
Err(e) => dev_warn!(dev, "GPU name unavailable: {:?}\n", e),
@@ -476,13 +479,12 @@ pub(crate) fn new<'a>(
/ u64::SZ_1M
);
}
-
- info
},
// Create GPU memory manager owning memory management resources.
mm: {
- let usable_vram = gsp_static_info.usable_fb_regions.first().ok_or(ENODEV)?;
+ let info = gsp_resources.static_info();
+ let usable_vram = info.usable_fb_regions.first().ok_or(ENODEV)?;
let buddy_params = GpuBuddyParams {
base_offset: usable_vram.start,
size: usable_vram.end - usable_vram.start,
@@ -493,13 +495,13 @@ pub(crate) fn new<'a>(
bar,
gsp_resources.spec.chipset,
buddy_params,
- VramAddress::from_raw(gsp_static_info.total_fb_end),
+ VramAddress::from_raw(info.total_fb_end),
)?
},
// Create BAR1 user interface for CPU access to GPU virtual memory.
bar_user: {
- let pdb_addr = VramAddress::from_raw(gsp_static_info.bar1_pde_base);
+ let pdb_addr = VramAddress::from_raw(gsp_resources.static_info().bar1_pde_base);
let bar1_idx = crate::driver::bar1_resource_index(pdev)?;
let bar1_size = pdev.resource_len(bar1_idx)?;
Arc::pin_init(
@@ -520,14 +522,14 @@ pub(crate) fn new<'a>(
pub(crate) fn run_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bound>) {
let this = self.project();
let dev = pdev.as_ref();
- let regions = &this.gsp_static_info.usable_fb_regions;
+ let info = this.gsp_resources.static_info();
if let Err(err) = crate::mm::selftest::run(
dev,
this.mm,
- regions,
+ &info.usable_fb_regions,
this.bar_user,
- this.gsp_static_info.bar1_pde_base,
+ info.bar1_pde_base,
this.spec.chipset,
) {
dev_err!(dev, "self-tests failed: {:?}\n", err);
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index 6e7a057188c5..90b4c3380f11 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -219,12 +219,15 @@ pub(crate) fn new(
}))
})
}
-
- /// Query the GSP for the static GPU information.
- pub(crate) fn get_static_info(&self) -> Result<commands::GspStaticInfo> {
- self.cmdq.send_command(commands::GetGspStaticInfo)
- }
}
/// Opaque bundle required to unload the GSP. Created by [`Gsp::boot`], consumed by [`Gsp::unload`].
pub(crate) struct UnloadBundle<'a>(KBox<dyn hal::UnloadBundle + 'a>);
+
+/// The results of [`Gsp::boot`]: the static GPU configuration and the unload bundle.
+pub(crate) struct BootResult<'a> {
+ /// The unload bundle for [`Gsp::unload`], if one could be built.
+ pub(crate) unload_bundle: Option<UnloadBundle<'a>>,
+ /// The static GPU configuration, as GSP-RM reported it at the end of boot.
+ pub(crate) static_info: commands::GspStaticInfo,
+}
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index c805b42dd7bc..86a122438d44 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -339,18 +339,18 @@ fn handle_load_exec_hs_binary(&self, payload_0: &[u8], payload_1: &[u8]) -> Resu
}
impl<'gsp> super::Gsp<'gsp> {
- /// Attempt to boot the GSP.
+ /// Boots the GSP.
///
/// This is a GPU-dependent and complex procedure that involves loading firmware files from
/// user-space, patching them with signatures, and building firmware-specific intricate data
/// structures that the GSP will use at runtime.
///
- /// Upon return, the GSP is up and running, and its unload bundle (to be given as argument to
- /// [`Self::unload`]) returned.
+ /// Returns, with the GSP running, the static configuration that GSP-RM reported and the
+ /// unload bundle for [`Self::unload`].
pub(crate) fn boot(
self: Pin<&mut Self>,
mut ctx: super::GspBootContext<'_, 'gsp>,
- ) -> Result<Option<super::UnloadBundle<'gsp>>> {
+ ) -> Result<super::BootResult<'gsp>> {
let pdev = ctx.pdev;
let chipset = ctx.chipset;
let gsp_falcon = ctx.gsp_falcon;
@@ -398,7 +398,12 @@ pub(crate) fn boot(
// Wait until GSP is fully initialized.
commands::wait_gsp_init_done(&self.cmdq)?;
- Ok(unload_guard.dismiss().1)
+ let static_info = self.cmdq.send_command(commands::GetGspStaticInfo)?;
+
+ Ok(super::BootResult {
+ unload_bundle: unload_guard.dismiss().1,
+ static_info,
+ })
}
/// Shut down the GSP and wait until it is offline.
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index b3b28e397ebd..d866297fa0a5 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -205,7 +205,7 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
}
}
-/// The reply from the GSP to the [`GetGspStaticInfo`] command.
+/// The static GPU configuration, which GSP-RM reports in reply to [`GetGspStaticInfo`].
pub(crate) struct GspStaticInfo {
gpu_name: [u8; 64],
/// BAR1 Page Directory Entry base address.
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 21/33] gpu: nova-core: gsp: add the GSP_INIT request builder
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (19 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 20/33] gpu: nova-core: gsp: return the static GPU configuration from boot John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 22/33] gpu: nova-core: gsp: send GSP_INIT and decode its reply John Hubbard
` (11 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The r000 boot protocol replaces the system-info, registry and
static-info commands with one GSP_INIT request, whose payload is an NVKV
stream of key-value pairs.
Nova-core had the NVKV codec and the GSP_INIT schema, but no builder
that filled the schema in from the PCI device.
Add the builder. It carries over the registry keys that the r570
registry command sent, and it reports the architecture that the kernel
is built for. GSP-RM reads the GPU's PCI location as one word holding
the domain, bus and device number. That word is not the bus, device and
function triple that PCI_DEVID packs, so the builder assembles it rather
than using PCI_DEVID.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/commands.rs | 23 +++++
drivers/gpu/nova-core/gsp/fw/commands.rs | 110 +++++++++++++++++++++--
2 files changed, 125 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index d866297fa0a5..2c7965174445 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -30,8 +30,14 @@
},
fw::{
self,
+ commands::GspInitRequest,
MsgFunction, //
},
+ nvkv::{
+ Encodable,
+ EncodedStream,
+ Encoder, //
+ },
},
sbuffer::SBufferIter,
vgpu::VgpuState, //
@@ -264,6 +270,23 @@ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> {
}
}
+/// Builds the NVKV-encoded payload of a `GSP_INIT` request for `pdev`.
+///
+/// # Errors
+///
+/// - `ENOMEM` if the request or the encoder buffer cannot be allocated.
+#[expect(dead_code)]
+pub(crate) fn build_gsp_init_payload(
+ pdev: &pci::Device<device::Bound>,
+ chipset: Chipset,
+ vgpu_state: VgpuState,
+) -> Result<EncodedStream> {
+ let mut encoder = Encoder::new();
+ GspInitRequest::new(pdev, chipset, vgpu_state)?.encode(&mut encoder)?;
+
+ Ok(encoder.finish())
+}
+
pub(crate) use fw::commands::PowerStateLevel;
/// The `UnloadingGuestDriver` command, used to shut down the GSP.
diff --git a/drivers/gpu/nova-core/gsp/fw/commands.rs b/drivers/gpu/nova-core/gsp/fw/commands.rs
index f0a2d023f560..4fdd6ff23dc1 100644
--- a/drivers/gpu/nova-core/gsp/fw/commands.rs
+++ b/drivers/gpu/nova-core/gsp/fw/commands.rs
@@ -18,7 +18,8 @@
use crate::{
gpu::Chipset,
gsp::GSP_PAGE_SIZE,
- num::IntoSafeCast, //
+ num::IntoSafeCast,
+ vgpu::VgpuState, //
};
use crate::gsp::nvkv::{
@@ -258,6 +259,25 @@ pub(crate) enum HostArch {
Riscv64 = 5,
}
+impl HostArch {
+ /// Returns the variant that names the architecture for which this kernel is built.
+ fn host() -> Self {
+ if cfg!(target_arch = "x86_64") {
+ Self::X86_64
+ } else if cfg!(target_arch = "aarch64") {
+ Self::Aarch64
+ } else if cfg!(target_arch = "powerpc64") {
+ Self::Ppc64le
+ } else if cfg!(target_arch = "arm") {
+ Self::Arm
+ } else if cfg!(target_arch = "riscv64") {
+ Self::Riscv64
+ } else {
+ Self::None
+ }
+ }
+}
+
// TODO[FPRI]: This is a temporary solution to be replaced with the corresponding derive macros once
// they land.
impl TryFrom<u32> for HostArch {
@@ -284,7 +304,7 @@ fn from(value: HostArch) -> Self {
nvkv_encode! {
/// A GSP registry entry.
- struct RegKey {
+ pub(crate) struct RegKey {
key_name: Key<&'static [u8], { Self::REGKEY_NAME_KEY }>,
key_value: Key<u32, { Self::REGKEY_VALUE_U32_KEY }>,
}
@@ -294,6 +314,17 @@ impl RegKey {
// Define the Key IDs read/written by GSP.
const REGKEY_NAME_KEY: KeyId = 0x3070;
const REGKEY_VALUE_U32_KEY: KeyId = 0x3071;
+
+ /// Creates a registry entry.
+ ///
+ /// `key_name` must include its NUL terminator, which GSP-RM counts in the encoded name
+ /// length.
+ pub(crate) fn new(key_name: &'static [u8], key_value: u32) -> Self {
+ Self {
+ key_name: key_name.into(),
+ key_value: key_value.into(),
+ }
+ }
}
impl Encodable for KVVec<RegKey> {
@@ -329,22 +360,40 @@ impl VfInfo {
nvkv_encode! {
/// Payload of the `GSP_INIT` command.
- // TODO: expect() doesn't work here due to Self:: reference, fixed in 1.97.0
- // https://github.com/rust-lang/rust/pull/154377
- #[cfg_attr(not(CONFIG_KUNIT), allow(dead_code))]
- struct GspInitRequest {
+ pub(crate) struct GspInitRequest {
pci_device_id: Key<u32, { Self::PCI_DEVICE_ID_KEY }>,
pci_sub_device_id: Key<u32, { Self::PCI_SUBDEVICE_ID_KEY }>,
pci_revision_id: Key<u32, { Self::PCI_REVISION_ID_KEY }>,
pci_config_mirror_base: Key<u32, { Self::PCI_CONFIG_MIRROR_BASE_KEY }>,
pci_config_mirror_size: Key<u32, { Self::PCI_CONFIG_MIRROR_SIZE_KEY }>,
host_arch: Key<HostArch, { Self::HOST_ARCH_KEY }, u32>,
- bus_device_func: Key<u64, { Self::NV_DOMAIN_BUS_DEVICE_FUNC_KEY }>,
+ domain_bus_device: Key<u64, { Self::NV_DOMAIN_BUS_DEVICE_FUNC_KEY }>,
regkeys: KVVec<RegKey>,
vf_info: Option<VfInfo>,
}
}
+bitfield! {
+ /// A GPU's PCI location, encoded as GSP-RM decodes it. Despite the name that GSP-RM gives
+ /// the key, the function number is not part of the value.
+ struct DomainBusDevice(u64) {
+ 63:32 domain;
+ 15:8 bus;
+ 7:0 device;
+ }
+}
+
+/// Registry entries that the driver sends to GSP-RM on every boot.
+///
+/// `RMSecBusResetEnable` enables PCI secondary bus reset. `RMForcePcieConfigSave` makes GSP-RM
+/// preserve PCI configuration registers across any PCI reset. `RMDevidCheckIgnore` lets GSP-RM
+/// boot when the PCI device id is absent from its product name database.
+const REGISTRY_ENTRIES: &[(&[u8], u32)] = &[
+ (b"RMSecBusResetEnable\0", 1),
+ (b"RMForcePcieConfigSave\0", 1),
+ (b"RMDevidCheckIgnore\0", 1),
+];
+
impl GspInitRequest {
// Define the Key IDs read/written by GSP.
const PCI_DEVICE_ID_KEY: KeyId = 0x0001;
@@ -354,6 +403,51 @@ impl GspInitRequest {
const PCI_CONFIG_MIRROR_SIZE_KEY: KeyId = 0x0011;
const HOST_ARCH_KEY: KeyId = 0x0070;
const NV_DOMAIN_BUS_DEVICE_FUNC_KEY: KeyId = 0x1020;
+
+ /// Creates the request for `dev`.
+ ///
+ /// The registry keys are [`REGISTRY_ENTRIES`], plus `RMSetSriovMode` when `vgpu_state` reports
+ /// that vGPU is enabled.
+ ///
+ /// # Errors
+ ///
+ /// - `ENOMEM` if the registry list cannot be allocated.
+ pub(crate) fn new(
+ dev: &pci::Device<device::Bound>,
+ chipset: Chipset,
+ vgpu_state: VgpuState,
+ ) -> Result<Self> {
+ let mut regkeys = KVVec::new();
+ for &(name, value) in REGISTRY_ENTRIES {
+ regkeys.push(RegKey::new(name, value), GFP_KERNEL)?;
+ }
+ if matches!(vgpu_state, VgpuState::Enabled { .. }) {
+ regkeys.push(RegKey::new(b"RMSetSriovMode\0", 1), GFP_KERNEL)?;
+ }
+
+ let mirror = chipset.pci_config_mirror_range();
+ // `PCI_DEVID` packs the bus, device and function as the low half of a `Dbdf` does.
+ let dev_id = Dbdf::from(u32::from(dev.dev_id()));
+ let domain_bus_device = DomainBusDevice::zeroed()
+ .with_domain(dev.domain_nr())
+ .with_bus(u8::from(dev_id.bus()))
+ .with_device(u8::from(dev_id.device()));
+ let device_id = (u32::from(dev.device_id()) << 16) | u32::from(dev.vendor_id().as_raw());
+ let sub_device_id =
+ (u32::from(dev.subsystem_device_id()) << 16) | u32::from(dev.subsystem_vendor_id());
+
+ Ok(Self {
+ pci_device_id: device_id.into(),
+ pci_sub_device_id: sub_device_id.into(),
+ pci_revision_id: u32::from(dev.revision_id()).into(),
+ pci_config_mirror_base: mirror.start.into(),
+ pci_config_mirror_size: (mirror.end - mirror.start).into(),
+ host_arch: HostArch::host().into(),
+ domain_bus_device: u64::from(domain_bus_device).into(),
+ regkeys,
+ vf_info: None,
+ })
+ }
}
// Decode:
@@ -753,7 +847,7 @@ fn gsp_init_request() -> Result {
pci_config_mirror_base: 0x1234_5678.into(),
pci_config_mirror_size: 0x1000.into(),
host_arch: HostArch::Aarch64.into(),
- bus_device_func: 0x0001_0203_0405_0607.into(),
+ domain_bus_device: 0x0001_0203_0405_0607.into(),
regkeys,
vf_info: Some(VfInfo {
total_vfs: 8.into(),
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 22/33] gpu: nova-core: gsp: send GSP_INIT and decode its reply
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (20 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 21/33] gpu: nova-core: gsp: add the GSP_INIT request builder John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 23/33] gpu: nova-core: add LIBOS3 log buffers and state monitor buffer John Hubbard
` (10 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
GSP-RM answers GSP_INIT only once it has finished starting. To finish
starting, GSP-RM first raises the load-and-execute events and waits for
the driver to service them. So the code that waits for the reply has to
service those events while waiting.
Add the GSP_INIT sender and the wait for its reply. The wait passes each
event that arrives before the reply to a handler that the caller
supplies, and it keeps one deadline from the send however many events
arrive, as the RPC reply wait does. Decode the reply into the static
configuration type that the boot sequence already returns, so that the
rest of the driver reads the same configuration whichever firmware
produced it.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/cmdq.rs | 108 +++++++++++++++++++----
drivers/gpu/nova-core/gsp/commands.rs | 99 ++++++++++++++++++++-
drivers/gpu/nova-core/gsp/fw.rs | 7 +-
drivers/gpu/nova-core/gsp/fw/commands.rs | 55 ++++++++++--
4 files changed, 241 insertions(+), 28 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index b202bd8185ba..f64a97736ed7 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -625,19 +625,38 @@ pub(crate) fn send_command_no_wait<M>(&self, command: M) -> Result
self.inner.lock().send_command(command)
}
- /// Receives one GMC event and passes its command id and payload slices to `handler`.
+ /// Waits for the response to the GMC request with command id `command_id`, and passes every
+ /// other GMC element that arrives first to `on_other`.
///
- /// This method may sleep while waiting. The queue mutex stays locked across the wait and the
- /// `handler` call, so `handler` must not call back into this [`Cmdq`].
+ /// This method may sleep while waiting. The queue mutex stays locked across the whole wait and
+ /// across the `on_other` and `decode` calls, so neither may call back into this [`Cmdq`].
///
- /// See [`CmdqInner::receive_gmc_and_dispatch`] for the return value and the errors.
- #[expect(dead_code)]
- pub(crate) fn receive_gmc_and_dispatch<R>(
+ /// See [`CmdqInner::await_gmc_response`] for the return value and the errors.
+ pub(crate) fn await_gmc_response<R>(
&self,
- timeout: Delta,
- handler: impl FnOnce(u32, &[u8], &[u8]) -> Result<Option<R>>,
- ) -> Result<Option<R>> {
- self.inner.lock().receive_gmc_and_dispatch(timeout, handler)
+ command_id: u32,
+ on_other: impl FnMut(&GspGmcMsgElement, &[u8], &[u8]) -> Result,
+ decode: impl FnMut(&[u8], &[u8]) -> Result<R>,
+ ) -> Result<R> {
+ self.inner
+ .lock()
+ .await_gmc_response(command_id, on_other, decode)
+ }
+
+ /// Sends a GMC API request to the GSP without waiting for the response.
+ ///
+ /// # Errors
+ ///
+ /// Errors from [`DmaGspMem::allocate_command`] are propagated as-is.
+ pub(crate) fn send_gmc_no_wait(
+ &self,
+ command_id: u32,
+ payload: &[u8],
+ max_response_size: u32,
+ ) -> Result {
+ self.inner
+ .lock()
+ .send_gmc(command_id, payload, max_response_size)
}
/// Waits for an unsolicited GSP event of type `M`. Events that arrive before it are logged and
@@ -812,7 +831,6 @@ fn poison(&self, reason: fmt::Arguments<'_>) -> Error {
/// # Errors
///
/// Errors from [`DmaGspMem::allocate_command`] are propagated as-is.
- #[expect(dead_code)]
fn send_gmc(&mut self, command_id: u32, payload: &[u8], max_response_size: u32) -> Result {
let seq = self.seq;
self.seq = self.seq.wrapping_add(1);
@@ -1204,8 +1222,9 @@ fn consume_element<R>(
/// Receives the next queue element and, if it is a GMC element, passes it to `handler`.
///
- /// `handler` receives the command id and the payload that follows the GMC API header, as two
- /// slices because the ring may wrap, and returns `None` for an element that it declines.
+ /// `handler` receives the headers that open the element and the payload that follows the GMC
+ /// API header, as two slices because the ring may wrap, and returns `None` for an element that
+ /// it declines.
///
/// Returns `Ok(None)` when `handler` declines the element or when the element is not a GMC
/// element.
@@ -1220,7 +1239,7 @@ fn consume_element<R>(
fn receive_gmc_and_dispatch<R>(
&mut self,
timeout: Delta,
- handler: impl FnOnce(u32, &[u8], &[u8]) -> Result<Option<R>>,
+ handler: impl FnOnce(&GspGmcMsgElement, &[u8], &[u8]) -> Result<Option<R>>,
) -> Result<Option<R>> {
self.consume_element(timeout, |this, element| match element {
QueueElement::Other(_) => {
@@ -1230,18 +1249,73 @@ fn receive_gmc_and_dispatch<R>(
}
QueueElement::Gmc(message) => {
let header = message.header;
- let command_id = header.gmc.command_id();
dev_dbg!(
&this.dev,
"GSP GMC: event: seq# {}, command_id=0x{:x}, length=0x{:x}\n",
header.gmc.sequence,
- command_id,
+ header.gmc.command_id(),
header.length(),
);
- handler(command_id, message.contents.0, message.contents.1)
+ handler(header, message.contents.0, message.contents.1)
}
})
}
+
+ /// Waits for the response to the GMC request with command id `command_id`, up to
+ /// [`Cmdq::RECEIVE_TIMEOUT`] from the call.
+ ///
+ /// The response's payload is passed to `decode`, as two slices because the ring may wrap.
+ /// Every other GMC element that arrives first is passed to `on_other` with the headers that
+ /// open it and its payload slices, and any other element is logged. Neither kind of element
+ /// extends the deadline.
+ ///
+ /// # Errors
+ ///
+ /// - `ETIMEDOUT` if the response does not arrive before the deadline, however many other
+ /// elements arrive while waiting.
+ /// - `EIO` if the queue is poisoned or an element fails framing validation (see
+ /// [`Self::wait_for_element`]), or if the response carries a failure status.
+ ///
+ /// Errors from `on_other` and `decode` are propagated as-is.
+ fn await_gmc_response<R>(
+ &mut self,
+ command_id: u32,
+ mut on_other: impl FnMut(&GspGmcMsgElement, &[u8], &[u8]) -> Result,
+ mut decode: impl FnMut(&[u8], &[u8]) -> Result<R>,
+ ) -> Result<R> {
+ let dev = self.dev;
+ let deadline = Instant::<Monotonic>::now() + Cmdq::RECEIVE_TIMEOUT;
+ loop {
+ let remaining = deadline - Instant::<Monotonic>::now();
+ if remaining.is_negative() {
+ break Err(ETIMEDOUT);
+ }
+
+ let response =
+ self.receive_gmc_and_dispatch(remaining, |header, payload_0, payload_1| {
+ if header.gmc.command_id() != command_id {
+ return on_other(header, payload_0, payload_1).map(|()| None);
+ }
+
+ let status = header.gmc.status();
+ if status != 0 {
+ dev_err!(
+ dev,
+ "GSP GMC: command 0x{:x} failed, status={:#x}\n",
+ command_id,
+ status
+ );
+ return Err(EIO);
+ }
+
+ decode(payload_0, payload_1).map(Some)
+ })?;
+
+ if let Some(response) = response {
+ break Ok(response);
+ }
+ }
+ }
}
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index 2c7965174445..74ae92428cad 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -30,13 +30,21 @@
},
fw::{
self,
- commands::GspInitRequest,
- MsgFunction, //
+ commands::{
+ GspInitRequest,
+ GspInitResponse,
+ GspInitResponseSchema, //
+ },
+ GspGmcMsgElement,
+ MsgFunction,
+ GMCAPI_CMD_GSP_INIT, //
},
nvkv::{
+ Decoder,
Encodable,
EncodedStream,
- Encoder, //
+ Encoder,
+ UnknownKeyPolicy, //
},
},
sbuffer::SBufferIter,
@@ -287,6 +295,91 @@ pub(crate) fn build_gsp_init_payload(
Ok(encoder.finish())
}
+/// Largest `GSP_INIT` response that the driver accepts.
+const GSP_INIT_MAX_RESPONSE_SIZE: u32 = 48 * 1024;
+
+/// Sends `GSP_INIT` and returns the static GPU configuration that its reply carries.
+///
+/// Every GMC (GPU Management Controller) element that arrives before the reply is passed to
+/// `on_unsolicited_element` with the headers that open it and its payload, as two slices because
+/// the ring may wrap. The load-and-execute events that GSP-RM raises while it starts arrive this
+/// way.
+///
+/// `payload` is the stream from [`build_gsp_init_payload`].
+///
+/// # Errors
+///
+/// - `EIO` if GSP-RM reports a failure status.
+/// - `ETIMEDOUT` if the reply does not arrive within [`Cmdq::RECEIVE_TIMEOUT`] of the send,
+/// however many events arrive while waiting.
+///
+/// Errors from `on_unsolicited_element` and from decoding the reply are propagated as-is.
+#[expect(dead_code)]
+pub(crate) fn gsp_init(
+ cmdq: &Cmdq<'_>,
+ payload: &[u64],
+ on_unsolicited_element: impl FnMut(&GspGmcMsgElement, &[u8], &[u8]) -> Result,
+) -> Result<GspStaticInfo> {
+ // Qualified because `zerocopy::IntoBytes` also gives `[T]` an `as_bytes`.
+ let payload = AsBytes::as_bytes(payload);
+
+ cmdq.send_gmc_no_wait(GMCAPI_CMD_GSP_INIT, payload, GSP_INIT_MAX_RESPONSE_SIZE)?;
+
+ cmdq.await_gmc_response(
+ GMCAPI_CMD_GSP_INIT,
+ on_unsolicited_element,
+ decode_gsp_init_reply,
+ )
+}
+
+/// Decodes the `GSP_INIT` reply from its payload, which the ring may have split in two, into the
+/// static configuration type that the boot sequence returns.
+///
+/// # Errors
+///
+/// - `EINVAL` if the payload is not a whole number of NVKV words, if the stream is malformed or
+/// omits a required key, or if GSP-RM reported no framebuffer region.
+/// - `ENOMEM` if the words or the decoded regions cannot be allocated.
+fn decode_gsp_init_reply(payload_0: &[u8], payload_1: &[u8]) -> Result<GspStaticInfo> {
+ const WORD_SIZE: usize = size_of::<u64>();
+
+ let len = payload_0.len() + payload_1.len();
+ if len % WORD_SIZE != 0 {
+ return Err(EINVAL);
+ }
+
+ let mut words = KVVec::with_capacity(len / WORD_SIZE, GFP_KERNEL)?;
+ let mut bytes = SBufferIter::new_reader([payload_0, payload_1]);
+ for _ in 0..len / WORD_SIZE {
+ let mut word = [0u8; WORD_SIZE];
+ bytes.read_exact(&mut word)?;
+ words.push(u64::from_le_bytes(word), GFP_KERNEL)?;
+ }
+
+ let decoder = Decoder::new(&words, UnknownKeyPolicy::Ignore);
+ let mut schema = GspInitResponseSchema::default();
+ let decoded = KBox::try_init(decoder.decode(&mut schema)?, GFP_KERNEL)?;
+
+ let mut gpu_name = [0u8; GspInitResponse::MAX_GPU_NAME_LEN];
+ let name = decoded.gpu_name();
+ gpu_name
+ .get_mut(..name.len())
+ .ok_or(EINVAL)?
+ .copy_from_slice(name);
+
+ let mut usable_fb_regions = KVec::new();
+ for region in decoded.usable_fb_regions() {
+ usable_fb_regions.push(region, GFP_KERNEL)?;
+ }
+
+ Ok(GspStaticInfo {
+ gpu_name,
+ bar1_pde_base: decoded.bar1_pde_base(),
+ usable_fb_regions,
+ total_fb_end: decoded.total_fb_end().ok_or(EINVAL)?,
+ })
+}
+
pub(crate) use fw::commands::PowerStateLevel;
/// The `UnloadingGuestDriver` command, used to shut down the GSP.
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index e86283f67358..cc0bc8f8ae87 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -1048,13 +1048,17 @@ pub(crate) struct GmcApiHeader {
/// Sequence number that GSP-RM copies from a request into its response.
pub(crate) sequence: u64,
/// In a request, the largest response that the sender accepts. In a response, the `NV_STATUS`.
- pub(crate) max_resp_or_status: u32,
+ max_resp_or_status: u32,
reserved: [u32; 5],
}
/// Bits of [`GmcApiHeader::command`] that hold the command id. The high byte holds flags.
const GMCAPI_COMMAND_ID_MASK: u32 = 0x00ff_ffff;
+/// GMC request that carries the system information and registry keys to GSP-RM. GSP-RM answers
+/// it with the static GPU configuration once it has finished starting.
+pub(crate) const GMCAPI_CMD_GSP_INIT: u32 = r000_00::GMCAPI_COMMANDS_GMCAPI_CMD_GSP_INIT;
+
/// GMC event that requests the driver to run the generic falcon bootloader on the descriptor that
/// the event carries.
pub(crate) const GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER: u32 =
@@ -1097,7 +1101,6 @@ pub(crate) fn command_id(&self) -> u32 {
///
/// The value is meaningful only on a response, which GSP-RM marks with a flag in the command
/// word. In a request, the same word holds the largest response that the sender accepts.
- #[expect(dead_code)]
pub(crate) fn status(&self) -> u32 {
self.max_resp_or_status
}
diff --git a/drivers/gpu/nova-core/gsp/fw/commands.rs b/drivers/gpu/nova-core/gsp/fw/commands.rs
index 4fdd6ff23dc1..748639942fe1 100644
--- a/drivers/gpu/nova-core/gsp/fw/commands.rs
+++ b/drivers/gpu/nova-core/gsp/fw/commands.rs
@@ -455,10 +455,7 @@ pub(crate) fn new(
// Should decode with UnknownKeyPolicy::Ignore.
nvkv_decode! {
/// Schema for the `GSP_INIT` response.
- // TODO: expect() doesn't work here due to Self:: reference, fixed in 1.97.0
- // https://github.com/rust-lang/rust/pull/154377
- #[cfg_attr(not(CONFIG_KUNIT), allow(dead_code))]
- struct GspInitResponseSchema => GspInitResponse {
+ pub(crate) struct GspInitResponseSchema => GspInitResponse {
gpu_name:
Array<u8, { GspInitResponse::MAX_GPU_NAME_LEN }, { Self::GPU_NAME_STRING_KEY }>,
fb_regions: Accumulated<FbRegionSchema>,
@@ -475,15 +472,61 @@ impl GspInitResponseSchema {
}
/// Payload of the `GSP_INIT` response.
-struct GspInitResponse {
+pub(crate) struct GspInitResponse {
gpu_name: ArrayVec<u8, { Self::MAX_GPU_NAME_LEN }>,
fb_regions: KVVec<FbRegion>,
bar1_pde_base: u64,
+ #[cfg_attr(not(CONFIG_KUNIT = "y"), expect(dead_code))]
vmmu_segment_size: u64,
}
impl GspInitResponse {
- const MAX_GPU_NAME_LEN: usize = 64;
+ pub(crate) const MAX_GPU_NAME_LEN: usize = 64;
+
+ /// Tag of a general-purpose region. Any other tag marks a region that GSP-RM reserves for the
+ /// use that the tag names.
+ const FB_REGION_TAG_NONE: u32 = 0;
+
+ /// Returns the GPU name, which GSP-RM sends with its NUL terminator.
+ pub(crate) fn gpu_name(&self) -> &[u8] {
+ self.gpu_name.as_slice()
+ }
+
+ /// Returns an iterator over the FB regions from which the driver may allocate: the
+ /// general-purpose regions that are not protected and that support both compression and
+ /// isochronous access.
+ pub(crate) fn usable_fb_regions(&self) -> impl Iterator<Item = Range<u64>> + '_ {
+ self.fb_regions.iter().filter_map(|region| {
+ if region.limit >= region.base
+ && region.tag == Self::FB_REGION_TAG_NONE
+ && !region.flags.protected()
+ && region.flags.support_compressed()
+ && region.flags.support_iso()
+ {
+ region.limit.checked_add(1).map(|end| region.base..end)
+ } else {
+ None
+ }
+ })
+ }
+
+ /// Returns the exclusive end of the FB physical address space, which spans every region
+ /// including the ones that [`Self::usable_fb_regions`] leaves out.
+ ///
+ /// Returns `None` if no region that GSP-RM reported has a limit at or above its base.
+ pub(crate) fn total_fb_end(&self) -> Option<u64> {
+ self.fb_regions
+ .iter()
+ .filter(|region| region.limit >= region.base)
+ .map(|region| region.limit)
+ .max()?
+ .checked_add(1)
+ }
+
+ /// Returns the BAR1 page directory entry base address.
+ pub(crate) fn bar1_pde_base(&self) -> u64 {
+ self.bar1_pde_base
+ }
}
nvkv_decode! {
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 23/33] gpu: nova-core: add LIBOS3 log buffers and state monitor buffer
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (21 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 22/33] gpu: nova-core: gsp: send GSP_INIT and decode its reply John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 24/33] gpu: nova-core: add the ucodes firmware loader John Hubbard
` (9 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
GSP-RM runs on LIBOS, and each LIBOS task logs into a buffer that the
driver provides and names in the init argument array. The r000 firmware
runs LIBOS3, which has six logging tasks, and LIBOS3 also expects one
more page, the state monitor buffer, in which GSP-RM reports its own
state.
Nova-core allocated log buffers for three of the six tasks, and no state
monitor buffer.
Allocate the three missing log buffers and the state monitor buffer,
name every log buffer in the init argument array, and give each new log
a debugfs file next to the existing ones. The switch to r000 passes the
state monitor buffer to GSP-RM.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp.rs | 129 ++++++++++++++++++++++++++---------
1 file changed, 98 insertions(+), 31 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index 90b4c3380f11..a8aec431b0b1 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -80,7 +80,6 @@ pub(crate) fn dev(&self) -> &'gpu device::Device<device::Bound> {
/// Number of GSP pages to use in a RM log buffer.
const RM_LOG_BUFFER_NUM_PAGES: usize = 0x10;
-const LOG_BUFFER_SIZE: usize = RM_LOG_BUFFER_NUM_PAGES * GSP_PAGE_SIZE;
/// Array of page table entries, as understood by the GSP bootloader.
#[repr(C)]
@@ -116,9 +115,19 @@ fn init(view: CoherentView<'_, Self>, start: DmaAddress) -> Result<()> {
/// then pp points to index into the buffer where the next logging entry will
/// be written. Therefore, the logging data is valid if:
/// 1 <= pp < sizeof(buffer)/sizeof(u64)
-struct LogBuffer<'a>(Coherent<'a, [u8; LOG_BUFFER_SIZE]>);
+struct LogBuffer<'a, const NUM_PAGES: usize>(Coherent<'a, [[u8; GSP_PAGE_SIZE]; NUM_PAGES]>);
-impl<'a> LogBuffer<'a> {
+/// A log buffer at the default size, [`RM_LOG_BUFFER_NUM_PAGES`] pages.
+///
+/// Matches the registry defaults for the init, interrupt, RM and MNOC tasks
+/// (`NV_REG_STR_RM_GSP_LOG_BUFFER_SIZE_TASK_*_DEFAULT`).
+type TaskLogBuffer<'a> = LogBuffer<'a, RM_LOG_BUFFER_NUM_PAGES>;
+
+/// A single-page log buffer, the size of the logs of the root task and of the RM state monitor
+/// task.
+type SmallLogBuffer<'a> = LogBuffer<'a, 1>;
+
+impl<'a, const NUM_PAGES: usize> LogBuffer<'a, NUM_PAGES> {
/// Creates a new `LogBuffer` mapped on `dev`.
fn new(dev: &'a device::Device<device::Bound>) -> Result<Self> {
let obj = Self(Coherent::zeroed(dev, GFP_KERNEL)?);
@@ -127,22 +136,90 @@ fn new(dev: &'a device::Device<device::Bound>) -> Result<Self> {
let pte_view = io_project!(
obj.0,
- [build: size_of::<u64>()..][build: ..RM_LOG_BUFFER_NUM_PAGES * size_of::<u64>()]
+ [build: 0][build: size_of::<u64>()..][build: ..NUM_PAGES * size_of::<u64>()]
)
- .try_cast::<PteArray<RM_LOG_BUFFER_NUM_PAGES>>()?;
+ .try_cast::<PteArray<NUM_PAGES>>()?;
PteArray::init(pte_view, start_addr)?;
Ok(obj)
}
}
+/// The log buffers to which GSP-RM writes its debug output, one per LIBOS3 task.
struct LogBuffers<'a> {
- /// Init log buffer.
- loginit: LogBuffer<'a>,
- /// Interrupts log buffer.
- logintr: LogBuffer<'a>,
- /// RM log buffer.
- logrm: LogBuffer<'a>,
+ /// Init task.
+ loginit: TaskLogBuffer<'a>,
+ /// Interrupt task.
+ logintr: TaskLogBuffer<'a>,
+ /// RM task.
+ logrm: TaskLogBuffer<'a>,
+ /// MNOC task.
+ logmnoc: TaskLogBuffer<'a>,
+ /// Root task.
+ logroot: SmallLogBuffer<'a>,
+ /// RM state monitor task.
+ logrmon: SmallLogBuffer<'a>,
+}
+
+impl<'a> LogBuffers<'a> {
+ /// Number of log buffers.
+ const COUNT: usize = 6;
+
+ /// Allocates the six log buffers, mapped on `dev`.
+ fn new(dev: &'a device::Device<device::Bound>) -> Result<Self> {
+ Ok(Self {
+ loginit: TaskLogBuffer::new(dev)?,
+ logintr: TaskLogBuffer::new(dev)?,
+ logrm: TaskLogBuffer::new(dev)?,
+ logmnoc: TaskLogBuffer::new(dev)?,
+ logroot: SmallLogBuffer::new(dev)?,
+ logrmon: SmallLogBuffer::new(dev)?,
+ })
+ }
+
+ /// Fills the first [`Self::COUNT`] entries of `libos` with the log buffers, under the names
+ /// that GSP-RM looks them up by.
+ fn init_arguments(
+ &self,
+ libos: &mut CoherentBox<'_, [LibosMemoryRegionInitArgument]>,
+ ) -> Result {
+ libos.init_at(
+ 0,
+ LibosMemoryRegionInitArgument::new("LOGINIT", &self.loginit.0),
+ )?;
+ libos.init_at(
+ 1,
+ LibosMemoryRegionInitArgument::new("LOGINTR", &self.logintr.0),
+ )?;
+ libos.init_at(
+ 2,
+ LibosMemoryRegionInitArgument::new("LOGRM", &self.logrm.0),
+ )?;
+ libos.init_at(
+ 3,
+ LibosMemoryRegionInitArgument::new("LOGMNOC", &self.logmnoc.0),
+ )?;
+ libos.init_at(
+ 4,
+ LibosMemoryRegionInitArgument::new("LOGROOT", &self.logroot.0),
+ )?;
+ libos.init_at(
+ 5,
+ LibosMemoryRegionInitArgument::new("LOGRMON", &self.logrmon.0),
+ )?;
+
+ Ok(())
+ }
+
+ /// Exposes each log buffer as a binary file in `dir`, under the lowercase form of its name.
+ fn register_debugfs<'data>(&'data self, dir: &debugfs::ScopedDir<'data, '_>) {
+ dir.read_binary_file(c"loginit", &self.loginit.0);
+ dir.read_binary_file(c"logintr", &self.logintr.0);
+ dir.read_binary_file(c"logrm", &self.logrm.0);
+ dir.read_binary_file(c"logmnoc", &self.logmnoc.0);
+ dir.read_binary_file(c"logroot", &self.logroot.0);
+ dir.read_binary_file(c"logrmon", &self.logrmon.0);
+ }
}
/// GSP runtime data.
@@ -158,6 +235,8 @@ pub(crate) struct Gsp<'gsp> {
pub(crate) cmdq: Cmdq<'gsp>,
/// RM arguments.
rmargs: Coherent<'gsp, GspArgumentsPadded>,
+ /// Buffer in which GSP-RM reports its own state.
+ rm_state_monitor: Coherent<'gsp, [u8; GSP_PAGE_SIZE]>,
}
impl<'gsp> Gsp<'gsp> {
@@ -168,17 +247,12 @@ pub(crate) fn new(
) -> impl PinInit<Self, Error> + 'gsp {
pin_init::pin_init_scope(move || {
let dev = pdev.as_ref();
+ let log_buffers = LogBuffers::new(dev)?;
- let loginit = LogBuffer::new(dev)?;
- let logintr = LogBuffer::new(dev)?;
- let logrm = LogBuffer::new(dev)?;
-
- // Initialise the logging structures. The OpenRM equivalents are in:
- // _kgspInitLibosLoggingStructures (allocates memory for buffers)
- // kgspSetupLibosInitArgs_IMPL (creates pLibosInitArgs[] array)
Ok(try_pin_init!(Self {
cmdq <- Cmdq::new(dev, bar),
rmargs: Coherent::init(dev, GFP_KERNEL, GspArgumentsPadded::new(&cmdq))?,
+ rm_state_monitor: Coherent::zeroed(dev, GFP_KERNEL)?,
libos: {
let mut libos = CoherentBox::zeroed_slice(
dev,
@@ -186,20 +260,15 @@ pub(crate) fn new(
GFP_KERNEL,
)?;
- libos.init_at(0, LibosMemoryRegionInitArgument::new("LOGINIT", &loginit.0))?;
- libos.init_at(1, LibosMemoryRegionInitArgument::new("LOGINTR", &logintr.0))?;
- libos.init_at(2, LibosMemoryRegionInitArgument::new("LOGRM", &logrm.0))?;
- libos.init_at(3, LibosMemoryRegionInitArgument::new("RMARGS", rmargs))?;
+ log_buffers.init_arguments(&mut libos)?;
+ libos.init_at(
+ LogBuffers::COUNT,
+ LibosMemoryRegionInitArgument::new("RMARGS", rmargs),
+ )?;
libos.into()
},
logs <- {
- let log_buffers = LogBuffers {
- loginit,
- logintr,
- logrm,
- };
-
#[allow(static_mut_refs)]
// SAFETY: `DEBUGFS_ROOT` is created before driver registration and cleared
// after driver unregistration, so no probe() can race with its modification.
@@ -211,9 +280,7 @@ pub(crate) fn new(
.expect("DEBUGFS_ROOT not initialized");
log_parent.scope(log_buffers, dev.name(), |logs, dir| {
- dir.read_binary_file(c"loginit", &logs.loginit.0);
- dir.read_binary_file(c"logintr", &logs.logintr.0);
- dir.read_binary_file(c"logrm", &logs.logrm.0);
+ logs.register_debugfs(dir)
})
},
}))
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 24/33] gpu: nova-core: add the ucodes firmware loader
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (22 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 23/33] gpu: nova-core: add LIBOS3 log buffers and state monitor buffer John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 25/33] gpu: nova-core: gsp: let the GSP HAL load the generic bootloader John Hubbard
` (8 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The r000 GSP firmware includes a ucodes image, microcode that GSP-RM
loads at run time from a buffer that the driver maps for it. The image
is described by a TLV whose FILE tag names a separate file and whose
SIZE tag gives that file's length, which is the convention that the GSP
firmware image already uses.
The GSP firmware wrapper open-coded the reading of the FILE and SIZE
tags, so the ucodes loader could not share it.
Move the reading of the two tags into the TLV parser, and add the ucodes
loader on top of the parser. The loader maps the image through a radix3
page table, as the GSP firmware image is mapped, and the image is
required on every chipset that nova-core supports. The loader has no
caller until the switch to r000.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/firmware.rs | 7 ++-
drivers/gpu/nova-core/firmware/bindata.rs | 59 +++++++++++++++++++++++
drivers/gpu/nova-core/firmware/gsp.rs | 15 ++----
drivers/gpu/nova-core/firmware/tlv.rs | 40 +++++++++++++--
4 files changed, 105 insertions(+), 16 deletions(-)
create mode 100644 drivers/gpu/nova-core/firmware/bindata.rs
diff --git a/drivers/gpu/nova-core/firmware.rs b/drivers/gpu/nova-core/firmware.rs
index 358c9b8db0b8..d7ba03184ad5 100644
--- a/drivers/gpu/nova-core/firmware.rs
+++ b/drivers/gpu/nova-core/firmware.rs
@@ -22,6 +22,7 @@
num::IntoSafeCast, //
};
+pub(crate) mod bindata;
pub(crate) mod booter;
pub(crate) mod fwsec;
pub(crate) mod gen_bootloader;
@@ -349,7 +350,11 @@ const fn make_entry_chipset(self, chipset: gpu::Chipset) -> Self {
let mut this = self
.make_entry_file(name, "gsp_bootloader.tlv")
.make_entry_file(name, "gsp.tlv")
- .make_entry_file(name, "gsp.bin");
+ .make_entry_file(name, "gsp.bin")
+ .make_entry_file(name, "ucodes.tlv")
+ // The metadata's FILE tag gives the image's real file name at run time. This static
+ // entry names the usual one.
+ .make_entry_file(name, "ucodes.bin");
// Add the firmware files specific to the GSP boot method of `chipset`.
let boot_files = boot_firmware_files(chipset);
diff --git a/drivers/gpu/nova-core/firmware/bindata.rs b/drivers/gpu/nova-core/firmware/bindata.rs
new file mode 100644
index 000000000000..410cb741273c
--- /dev/null
+++ b/drivers/gpu/nova-core/firmware/bindata.rs
@@ -0,0 +1,59 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+//! Loading of the ucodes image, the bindata blob of microcode that GSP-RM loads at run time.
+
+use kernel::{
+ device,
+ dma::DmaAddress,
+ prelude::*, //
+};
+
+use crate::{
+ firmware::{
+ radix3::Radix3,
+ tlv::{
+ request_tlv,
+ Tlv, //
+ },
+ },
+ gpu::Chipset,
+};
+
+/// The ucodes image, mapped for GSP-RM through a radix3 page table.
+pub(crate) struct UcodesImage<'a> {
+ /// The image and the page table that maps it.
+ radix3: Pin<KBox<Radix3<'a>>>,
+}
+
+#[expect(dead_code)]
+impl<'a> UcodesImage<'a> {
+ /// Loads the ucodes image that the `ucodes` metadata file names, and maps it for `dev`.
+ ///
+ /// # Errors
+ ///
+ /// - `ENOENT` if the metadata file is not installed.
+ /// - `EINVAL` if the metadata is malformed.
+ /// - `ENOMEM` if the page table cannot be allocated.
+ ///
+ /// Errors from [`Tlv::load_file`] are propagated as-is.
+ pub(crate) fn new(dev: &'a device::Device<device::Bound>, chipset: Chipset) -> Result<Self> {
+ let firmware = request_tlv(dev, chipset, "ucodes")?;
+ let tlv = Tlv::new(firmware.data())?;
+ let image = tlv.load_file(dev, chipset)?;
+
+ Ok(Self {
+ radix3: KBox::pin_init(Radix3::new(dev, image), GFP_KERNEL)?,
+ })
+ }
+
+ /// Returns the DMA address of the level 0 page of the page table that maps the image.
+ pub(crate) fn radix3_dma_address(&self) -> DmaAddress {
+ self.radix3.dma_address()
+ }
+
+ /// Returns the size of the image in bytes.
+ pub(crate) fn size(&self) -> usize {
+ self.radix3.size()
+ }
+}
diff --git a/drivers/gpu/nova-core/firmware/gsp.rs b/drivers/gpu/nova-core/firmware/gsp.rs
index 341a8b19aa38..a2db7b6ba131 100644
--- a/drivers/gpu/nova-core/firmware/gsp.rs
+++ b/drivers/gpu/nova-core/firmware/gsp.rs
@@ -6,9 +6,7 @@
Coherent,
DmaAddress, //
},
- firmware,
- prelude::*,
- str::CString,
+ prelude::*, //
};
use crate::{
@@ -20,8 +18,7 @@
Tlv,
},
},
- gpu::Chipset,
- num::FromSafeCast,
+ gpu::Chipset, //
};
/// The GSP firmware image, its signatures, and the GSP bootloader.
@@ -48,13 +45,7 @@ pub(crate) fn new(
let tlv = Tlv::new(firmware.data())?;
dev_dbg!(dev, "loaded gsp firmware v{}\n", tlv.get_string(b"VERS")?);
- let size = usize::from_safe_cast(tlv.get_u32(b"SIZE")?);
- let mut fw_vvec = VVec::zeroed(size, GFP_KERNEL).map_err(|_| ENOMEM)?;
-
- let chip_name = chipset.name();
- let file = tlv.get_string(b"FILE")?;
- let filename = CString::try_from_fmt(fmt!("nvidia/{chip_name}/gsp/{file}"))?;
- firmware::request_into_buf(&filename, dev, fw_vvec.as_mut_slice())?;
+ let fw_vvec = tlv.load_file(dev, chipset)?;
let signatures = Coherent::from_slice(dev, tlv.get_bytes(b"SIGN")?, GFP_KERNEL)?;
diff --git a/drivers/gpu/nova-core/firmware/tlv.rs b/drivers/gpu/nova-core/firmware/tlv.rs
index 7b879f13a61e..7f278903dc8b 100644
--- a/drivers/gpu/nova-core/firmware/tlv.rs
+++ b/drivers/gpu/nova-core/firmware/tlv.rs
@@ -4,6 +4,7 @@
use kernel::{
device,
firmware,
+ fmt,
prelude::*,
str::CString, //
};
@@ -13,15 +14,18 @@
num::*, //
};
+/// Returns the path of `file` in `chipset`'s GSP firmware directory.
+fn gsp_firmware_path(chipset: gpu::Chipset, file: fmt::Arguments<'_>) -> Result<CString> {
+ CString::try_from_fmt(fmt!("nvidia/{}/gsp/{}", chipset.name(), file))
+}
+
/// Requests the GPU firmware TLV `name` suitable for `chipset`.
pub(crate) fn request_tlv(
dev: &device::Device,
chipset: gpu::Chipset,
name: &str,
) -> Result<firmware::Firmware> {
- let chip_name = chipset.name();
-
- let filename = CString::try_from_fmt(fmt!("nvidia/{chip_name}/gsp/{name}.tlv"))?;
+ let filename = gsp_firmware_path(chipset, fmt!("{name}.tlv"))?;
dev_dbg!(dev, "loading firmware image {:?}\n", &filename);
@@ -198,6 +202,36 @@ fn iter(&self) -> TlvIter<'_, 'a> {
self.iter().find(|b| b.tag == *tag).ok_or(EINVAL)
}
+ /// Loads the file that the `FILE` tag names from `chipset`'s GSP firmware directory.
+ ///
+ /// The `SIZE` tag gives the file's length, and the returned buffer is that long.
+ ///
+ /// # Errors
+ ///
+ /// - `EINVAL` if `FILE` or `SIZE` is absent, or `FILE` does not hold a valid string.
+ /// - `ENODATA` if `SIZE` is zero.
+ /// - `ENOMEM` if the buffer cannot be allocated.
+ ///
+ /// Errors from the firmware request, `ENOENT` in particular, are propagated as-is.
+ pub(crate) fn load_file(
+ &self,
+ dev: &device::Device,
+ chipset: gpu::Chipset,
+ ) -> Result<VVec<u8>> {
+ let file = self.get_string(b"FILE")?;
+ let path = gsp_firmware_path(chipset, fmt!("{file}"))?;
+
+ let size = usize::from_safe_cast(self.get_u32(b"SIZE")?);
+ if size == 0 {
+ return Err(ENODATA);
+ }
+
+ let mut data = VVec::zeroed(size, GFP_KERNEL).map_err(|_| ENOMEM)?;
+ firmware::request_into_buf(&path, dev, data.as_mut_slice())?;
+
+ Ok(data)
+ }
+
/// Return a slice of bytes.
///
/// Returns `EINVAL` if the value is empty.
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 25/33] gpu: nova-core: gsp: let the GSP HAL load the generic bootloader
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (23 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 24/33] gpu: nova-core: add the ucodes firmware loader John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 26/33] gpu: nova-core: gsp: add the GSP_SUSPEND request John Hubbard
` (7 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
Turing and GA100 boot the GSP through the generic falcon bootloader. On
those chipsets the r000 GSP-RM raises a load-and-execute event that
requests a bootloader run, and the handler for that event runs the
bootloader that the boot sequence supplies. Whether a chipset boots
through the bootloader is a property of the TU102 HAL: a field of its
two instances, set for Turing and GA100 and clear for the other Ampere
chipsets and for Ada.
Only the TU102 HAL tested that field, to choose how to load FWSEC. The
boot sequence had no way to obtain the bootloader for the handler
without a second list of the chipsets.
Add a HAL method that loads the generic bootloader. The TU102 HAL loads
it when its field is set, and every other HAL returns no bootloader. The
method has no caller until the switch to r000.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/hal.rs | 28 ++++++++++++++++++++++++--
drivers/gpu/nova-core/gsp/hal/tu102.rs | 14 +++++++++++++
2 files changed, 40 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp/hal.rs b/drivers/gpu/nova-core/gsp/hal.rs
index d8329f6fcc65..8c2a8abcb187 100644
--- a/drivers/gpu/nova-core/gsp/hal.rs
+++ b/drivers/gpu/nova-core/gsp/hal.rs
@@ -5,10 +5,16 @@
mod gh100;
mod tu102;
-use kernel::prelude::*;
+use kernel::{
+ device,
+ prelude::*, //
+};
use crate::{
- firmware::gsp::GspFirmware,
+ firmware::{
+ gen_bootloader::GenericBootloader,
+ gsp::GspFirmware, //
+ },
gpu::{
Architecture,
Chipset, //
@@ -42,6 +48,24 @@ fn boot<'gpu>(
gsp_fw: &GspFirmware<'gpu>,
) -> Result<Option<super::UnloadBundle<'gpu>>>;
+ /// Loads the generic falcon bootloader for the chipsets whose GSP boots through it.
+ ///
+ /// The bootloader runs from the last blocks of an IMEM of `imem_size` bytes. On every other
+ /// chipset this returns `None`.
+ ///
+ /// # Errors
+ ///
+ /// Errors from loading the bootloader image are propagated as-is.
+ #[expect(dead_code)]
+ fn generic_bootloader(
+ &self,
+ _dev: &device::Device<device::Bound>,
+ _chipset: Chipset,
+ _imem_size: usize,
+ ) -> Result<Option<GenericBootloader>> {
+ Ok(None)
+ }
+
/// Performs HAL-specific post-GSP boot tasks.
///
/// This method is called by the GSP boot code after the GSP is confirmed to be running, and
diff --git a/drivers/gpu/nova-core/gsp/hal/tu102.rs b/drivers/gpu/nova-core/gsp/hal/tu102.rs
index 1b17ece53bd6..32931fd0b7cf 100644
--- a/drivers/gpu/nova-core/gsp/hal/tu102.rs
+++ b/drivers/gpu/nova-core/gsp/hal/tu102.rs
@@ -31,6 +31,7 @@
FwsecCommand,
FwsecFirmware, //
},
+ gen_bootloader::GenericBootloader,
gsp::GspFirmware, //
},
gpu::Chipset,
@@ -316,6 +317,19 @@ fn boot<'gpu>(
Ok(unload_guard.dismiss())
}
+ fn generic_bootloader(
+ &self,
+ dev: &device::Device<device::Bound>,
+ chipset: Chipset,
+ imem_size: usize,
+ ) -> Result<Option<GenericBootloader>> {
+ if !self.needs_fwsec_bootloader {
+ return Ok(None);
+ }
+
+ GenericBootloader::new(dev, chipset, imem_size).map(Some)
+ }
+
fn post_boot(
&self,
gsp: &Gsp<'_>,
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 26/33] gpu: nova-core: gsp: add the GSP_SUSPEND request
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (24 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 25/33] gpu: nova-core: gsp: let the GSP HAL load the generic bootloader John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 27/33] gpu: nova-core: switch to the r000 GSP firmware John Hubbard
` (6 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The r000 firmware suspends GSP-RM on a GSP_SUSPEND request, a GMC API
command whose payload is one word of flags. The one flag marks a
suspend that is a power management transition, S3 or hibernate, rather
than a full unload. GSP-RM sends no response on the message queue. It
reports the completed suspend in the GSP falcon's MAILBOX0 register
instead.
Add the request, and send it without waiting for a response. The
request has no caller until the following switch to r000 sends it in
place of the r570 unload RPC.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/commands.rs | 15 ++++++++++++-
drivers/gpu/nova-core/gsp/fw.rs | 4 ++++
drivers/gpu/nova-core/gsp/fw/commands.rs | 27 ++++++++++++++++++++++++
3 files changed, 45 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index 74ae92428cad..00ff3271a46e 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -37,7 +37,8 @@
},
GspGmcMsgElement,
MsgFunction,
- GMCAPI_CMD_GSP_INIT, //
+ GMCAPI_CMD_GSP_INIT,
+ GMCAPI_CMD_GSP_SUSPEND, //
},
nvkv::{
Decoder,
@@ -382,6 +383,18 @@ fn decode_gsp_init_reply(payload_0: &[u8], payload_1: &[u8]) -> Result<GspStatic
pub(crate) use fw::commands::PowerStateLevel;
+/// Sends `GSP_SUSPEND`, which GSP-RM does not answer (see [`GMCAPI_CMD_GSP_SUSPEND`]).
+///
+/// # Errors
+///
+/// Errors from [`Cmdq::send_gmc_no_wait`] are propagated as-is.
+#[expect(dead_code)]
+pub(crate) fn gsp_suspend(cmdq: &Cmdq<'_>, level: PowerStateLevel) -> Result {
+ let params = fw::commands::GspSuspend::new(level);
+
+ cmdq.send_gmc_no_wait(GMCAPI_CMD_GSP_SUSPEND, AsBytes::as_bytes(¶ms), 0)
+}
+
/// The `UnloadingGuestDriver` command, used to shut down the GSP.
///
/// Only used within the `gsp` module.
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index cc0bc8f8ae87..ee582cbde5f6 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -1069,6 +1069,10 @@ pub(crate) struct GmcApiHeader {
pub(crate) const GMCAPI_CMD_EXEC_HS_BINARY: u32 =
r000_00::GMCAPI_COMMANDS_GMCAPI_CMD_EXEC_HS_BINARY;
+/// GMC request for GSP-RM to suspend. GSP-RM sends no response, and reports the completed
+/// suspend in the GSP falcon's `MAILBOX0` instead.
+pub(crate) const GMCAPI_CMD_GSP_SUSPEND: u32 = r000_00::GMCAPI_COMMANDS_GMCAPI_CMD_GSP_SUSPEND;
+
static_assert!(size_of::<GmcApiHeader>() == size_of::<r000_00::GMCAPI_HEADER>());
static_assert!(
core::mem::offset_of!(GmcApiHeader, command)
diff --git a/drivers/gpu/nova-core/gsp/fw/commands.rs b/drivers/gpu/nova-core/gsp/fw/commands.rs
index 748639942fe1..4e6c712637b0 100644
--- a/drivers/gpu/nova-core/gsp/fw/commands.rs
+++ b/drivers/gpu/nova-core/gsp/fw/commands.rs
@@ -225,6 +225,33 @@ pub(crate) fn is_power_transition(self) -> bool {
}
}
+/// Set in [`GspSuspend::flags`] when the suspend is a power management transition (S3 or
+/// hibernate) rather than a full unload.
+const GMCAPI_GSP_SUSPEND_FLAGS_PM_TRANSITION: u64 = 1 << 0;
+
+/// Payload of the `GSP_SUSPEND` GMC command.
+#[repr(C)]
+#[derive(Clone, Copy, Debug, Zeroable)]
+pub(crate) struct GspSuspend {
+ flags: u64,
+}
+
+impl GspSuspend {
+ /// Creates a `GSP_SUSPEND` payload for the given [`PowerStateLevel`].
+ pub(crate) fn new(level: PowerStateLevel) -> Self {
+ Self {
+ flags: if level.is_power_transition() {
+ GMCAPI_GSP_SUSPEND_FLAGS_PM_TRANSITION
+ } else {
+ 0
+ },
+ }
+ }
+}
+
+// SAFETY: The single field is an integer type, and the struct has no padding.
+unsafe impl AsBytes for GspSuspend {}
+
/// Payload of the `UnloadingGuestDriver` command and message.
#[repr(transparent)]
#[derive(Clone, Copy, Debug, Zeroable)]
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 27/33] gpu: nova-core: switch to the r000 GSP firmware
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (25 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 26/33] gpu: nova-core: gsp: add the GSP_SUSPEND request John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 28/33] gpu: nova-core: gsp: make the GSP_INIT reply the static configuration John Hubbard
` (5 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The r000 firmware changes five things about how GSP-RM and the driver
exchange messages:
* A queue element opens with the MCTP and NVDM transport headers and
carries no checksum.
* Msgq v2 keeps the four queue pointers in BAR0 registers, and the write
to the command queue head register is also the doorbell.
* The load-and-execute boot steps arrive as GMC events rather than as
RPC events.
* One GSP_INIT request replaces the system-info, registry and
static-info commands, and its reply replaces the init-done event.
* GSP-RM suspends on a GSP_SUSPEND command and never answers it, where
r570 answered an UnloadingGuestDriver RPC.
It also requires two more buffers from the driver at boot, the ucodes
image and a page in which GSP-RM reports its own state, and it reserves
more framebuffer for itself: a larger region for the PMU, the GPU's
power management unit, on GB100, and a larger heap outside the WPR, the
write-protected region, on GB20x.
Only one firmware runs at a time, so a driver that had made some of
these changes but not the others would not boot. Switch all of them
together, and change the driver's bindings alias so that it names the
r000 module.
Give the queue its own copies of the four pointers, as Open RM's msgq
library does. A GSP reset zeroes the pointer registers, so the driver
writes its two pointers out from its copies, and it keeps the last value
that it read of each GSP pointer and uses that value until GSP-RM writes
the register again. The pointers count elements without wrapping at the
ring size. A ring is empty when its two pointers are equal and full when
they differ by the ring size, so the ring no longer keeps one slot
empty.
Load the ucodes image at boot and give GSP-RM its location in the boot
arguments. GSP-RM reads the image only while it starts, so the image is
freed when the boot sequence returns. The TU102 HAL loads the generic
falcon bootloader for the load-and-execute handler on the chipsets that
boot through it.
Suspend GSP-RM with the GSP_SUSPEND request in place of the unload RPC.
GSP-RM posts messages while it suspends, and the shutdown path runs
after the GSP event interrupt is freed, so drain the queue while polling
for the suspend.
Three things then have no caller:
* The CPU sequencer, which ran the register scripts that r570's GSP-RM
sent as RPC events.
* The wait for the init-done event.
* The four RPC commands that GSP_INIT and GSP_SUSPEND replace.
Remove all three. Mark the RPC send-and-wait paths dead rather than
removing them: no caller waits for an RPC reply any more, but the RPC
commands that the driver sends once GSP-RM is up, which later series
add, use them unchanged.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
Documentation/gpu/nova/core/interrupts.rst | 11 +-
drivers/gpu/nova-core/fb/hal/gb100.rs | 24 +-
drivers/gpu/nova-core/fb/hal/gb202.rs | 6 +-
drivers/gpu/nova-core/firmware/bindata.rs | 1 -
drivers/gpu/nova-core/gsp.rs | 9 +-
drivers/gpu/nova-core/gsp/boot.rs | 47 +-
drivers/gpu/nova-core/gsp/cmdq.rs | 247 ++++-----
drivers/gpu/nova-core/gsp/commands.rs | 253 +---------
drivers/gpu/nova-core/gsp/fw.rs | 559 ++++++---------------
drivers/gpu/nova-core/gsp/fw/commands.rs | 198 +-------
drivers/gpu/nova-core/gsp/hal.rs | 14 -
drivers/gpu/nova-core/gsp/hal/tu102.rs | 12 -
drivers/gpu/nova-core/gsp/sequencer.rs | 379 --------------
drivers/gpu/nova-core/sbuffer.rs | 1 +
14 files changed, 324 insertions(+), 1437 deletions(-)
delete mode 100644 drivers/gpu/nova-core/gsp/sequencer.rs
diff --git a/Documentation/gpu/nova/core/interrupts.rst b/Documentation/gpu/nova/core/interrupts.rst
index 280dcf97688a..fdcd789cf4f9 100644
--- a/Documentation/gpu/nova/core/interrupts.rst
+++ b/Documentation/gpu/nova/core/interrupts.rst
@@ -450,7 +450,7 @@ thread to drain the queue::
A halt and a posted message can be pending together, so the top half services
every cause that the status reports.
-A drain fails when a message's framing or checksum is bad, which poisons the
+A drain fails when an element's framing is bad, and the bad framing poisons the
queue (see "Draining the GSP-to-CPU queue"). Every later event would fail the
same way, so the IRQ thread disables vector 155 and logs the failure, which
leaves the queue unserviced until the device is reset.
@@ -582,11 +582,10 @@ The read pointer advances past every message, whether it matched, was an event,
or matched but failed to decode, so a message is never left at the queue head
for the next receive to parse again.
-Corrupt framing is the exception. A message's length is inside the region that
-the checksum covers, so once the framing or the checksum fails there is no
-trustworthy length with which to skip the message. Such a failure poisons the
-queue: nova-core logs it once, and every later receive fails with ``EIO`` until
-the device is reset.
+Corrupt framing is the exception. An element that fails framing validation has
+no trustworthy length, so the read pointer cannot advance past it. Such a
+failure poisons the queue: nova-core logs it once, and every later receive
+fails with ``EIO`` until the device is reset.
The polling path and the IRQ thread both read the queue under the command-queue
mutex. Replies and events share one queue and one read pointer, so one lock is
diff --git a/drivers/gpu/nova-core/fb/hal/gb100.rs b/drivers/gpu/nova-core/fb/hal/gb100.rs
index 9fa094939600..33a00fc12aaa 100644
--- a/drivers/gpu/nova-core/fb/hal/gb100.rs
+++ b/drivers/gpu/nova-core/fb/hal/gb100.rs
@@ -79,10 +79,28 @@ fn write_sysmem_flush_page_gb100(hshub0: Mmio<'_, regs::Hshub0Registers>, addr:
hshub0.write_reg(regs::NV_PFB_HSHUB_EG_PCIE_FLUSH_SYSMEM_ADDR_LO::zeroed().with_adr(addr_lo));
}
-// This PMU reservation size is r570-specific.
+/// PMU backing store.
+const PMU_BACKING_STORE_SIZE: usize = 9 * SZ_1M;
+
+/// PMU surfaces, `kpmuReservedMemorySurfacesSizeGet` in Open RM.
+const PMU_SURFACES_SIZE: usize = SZ_16M + SZ_256K;
+
+/// Miscellaneous PMU memory.
+const PMU_MISC_SIZE: usize = SZ_4K;
+
+/// Alignment of the PMU reserved region, `KPMU_RESERVED_MEMORY_ALIGNMENT` in Open RM.
+const PMU_RESERVED_MEMORY_ALIGNMENT: Alignment = Alignment::new::<SZ_128K>();
+
+/// PMU region above FRTS: the backing store, the surfaces and the miscellaneous memory, aligned to
+/// [`PMU_RESERVED_MEMORY_ALIGNMENT`].
+const PMU_RESERVED_SIZE: usize = const_align_up(
+ PMU_BACKING_STORE_SIZE + PMU_SURFACES_SIZE + PMU_MISC_SIZE,
+ PMU_RESERVED_MEMORY_ALIGNMENT,
+)
+.unwrap();
+
pub(super) const fn pmu_reserved_size_gb100() -> u32 {
- usize_into_u32::<{ const_align_up(SZ_8M + SZ_16M + SZ_4K, Alignment::new::<SZ_128K>()).unwrap() }>(
- )
+ usize_into_u32::<PMU_RESERVED_SIZE>()
}
impl FbHal for Gb100 {
diff --git a/drivers/gpu/nova-core/fb/hal/gb202.rs b/drivers/gpu/nova-core/fb/hal/gb202.rs
index 4341ecf36188..345179063b30 100644
--- a/drivers/gpu/nova-core/fb/hal/gb202.rs
+++ b/drivers/gpu/nova-core/fb/hal/gb202.rs
@@ -72,9 +72,9 @@ fn pmu_reserved_size(&self) -> u32 {
}
fn non_wpr_heap_size(&self) -> u64 {
- // Non-WPR heap for GB20x (see Open RM: kgspGetNonWprHeapSize, GB202+).
- // This size is r570-specific.
- u64::SZ_2M + u64::SZ_128K
+ // Non-WPR heap for GB20x (see Open RM: kgspGetNonWprHeapSize, GB202+). This size is
+ // r000-specific.
+ 3 * u64::SZ_1M
}
fn frts_size(&self) -> u64 {
diff --git a/drivers/gpu/nova-core/firmware/bindata.rs b/drivers/gpu/nova-core/firmware/bindata.rs
index 410cb741273c..12a8e74cc11c 100644
--- a/drivers/gpu/nova-core/firmware/bindata.rs
+++ b/drivers/gpu/nova-core/firmware/bindata.rs
@@ -26,7 +26,6 @@ pub(crate) struct UcodesImage<'a> {
radix3: Pin<KBox<Radix3<'a>>>,
}
-#[expect(dead_code)]
impl<'a> UcodesImage<'a> {
/// Loads the ucodes image that the `ucodes` metadata file names, and maps it for `dev`.
///
diff --git a/drivers/gpu/nova-core/gsp.rs b/drivers/gpu/nova-core/gsp.rs
index a8aec431b0b1..85967119cd7e 100644
--- a/drivers/gpu/nova-core/gsp.rs
+++ b/drivers/gpu/nova-core/gsp.rs
@@ -26,7 +26,6 @@
mod fw;
mod nvkv;
mod regs;
-mod sequencer;
pub(crate) use fw::{
GspFmcBootParams,
@@ -251,8 +250,12 @@ pub(crate) fn new(
Ok(try_pin_init!(Self {
cmdq <- Cmdq::new(dev, bar),
- rmargs: Coherent::init(dev, GFP_KERNEL, GspArgumentsPadded::new(&cmdq))?,
rm_state_monitor: Coherent::zeroed(dev, GFP_KERNEL)?,
+ rmargs: Coherent::init(
+ dev,
+ GFP_KERNEL,
+ GspArgumentsPadded::new(&cmdq, rm_state_monitor),
+ )?,
libos: {
let mut libos = CoherentBox::zeroed_slice(
dev,
@@ -295,6 +298,6 @@ pub(crate) fn new(
pub(crate) struct BootResult<'a> {
/// The unload bundle for [`Gsp::unload`], if one could be built.
pub(crate) unload_bundle: Option<UnloadBundle<'a>>,
- /// The static GPU configuration, as GSP-RM reported it at the end of boot.
+ /// The static GPU configuration, as decoded from the `GSP_INIT` reply.
pub(crate) static_info: commands::GspStaticInfo,
}
diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs
index 86a122438d44..5da3d134eec7 100644
--- a/drivers/gpu/nova-core/gsp/boot.rs
+++ b/drivers/gpu/nova-core/gsp/boot.rs
@@ -31,6 +31,7 @@
FLCN_ERR_BINARY_NOT_STARTED, //
},
firmware::{
+ bindata::UcodesImage,
gen_bootloader::{
BootloaderDmemDescV2,
GenericBootloader, //
@@ -41,6 +42,7 @@
cmdq::Cmdq,
commands,
fw::{
+ GspArgumentsPadded,
GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER,
GMCAPI_CMD_EXEC_HS_BINARY, //
}, //
@@ -153,7 +155,6 @@ fn core_resume(&self) -> Result {
/// - `EINVAL` if `command_id` is not a load-and-execute command.
///
/// Errors from the handlers and from [`Self::core_resume`] are propagated as-is.
- #[expect(dead_code)]
fn dispatch_gmc_boot_event(
&self,
command_id: u32,
@@ -347,6 +348,10 @@ impl<'gsp> super::Gsp<'gsp> {
///
/// Returns, with the GSP running, the static configuration that GSP-RM reported and the
/// unload bundle for [`Self::unload`].
+ ///
+ /// # Errors
+ ///
+ /// - `ENOENT` if the ucodes image is not installed.
pub(crate) fn boot(
self: Pin<&mut Self>,
mut ctx: super::GspBootContext<'_, 'gsp>,
@@ -359,10 +364,12 @@ pub(crate) fn boot(
let gsp_fw = KBox::pin_init(GspFirmware::new(dev, chipset), GFP_KERNEL)?;
- self.cmdq
- .send_command_no_wait(commands::SetSystemInfo::new(pdev, chipset))?;
- self.cmdq
- .send_command_no_wait(commands::SetRegistry::new(ctx.vgpu.state())?)?;
+ let generic_bootloader = hal.generic_bootloader(dev, chipset, gsp_falcon.imem_size())?;
+
+ // GSP-RM reads the ucodes image through the image's page table only while it starts up, so
+ // the image is freed when the boot sequence returns.
+ let ucodes = UcodesImage::new(dev, chipset)?;
+ GspArgumentsPadded::set_bindata(&self.rmargs, &ucodes);
// Perform the chipset-specific boot sequence, and retrieve the unload bundle.
let unload_bundle = hal.boot(&self, &mut ctx, &gsp_fw)?.or_else(|| {
@@ -393,12 +400,20 @@ pub(crate) fn boot(
dev_dbg!(pdev, "RISC-V active? {}\n", gsp_falcon.is_riscv_active(),);
- hal.post_boot(&self, ctx, &gsp_fw)?;
-
- // Wait until GSP is fully initialized.
- commands::wait_gsp_init_done(&self.cmdq)?;
+ let init_payload = commands::build_gsp_init_payload(pdev, chipset, ctx.vgpu.state())?;
+ let load_exec = LoadExecContext {
+ bootloader: generic_bootloader.as_ref(),
+ gsp_falcon,
+ sec2_falcon: ctx.sec2_falcon,
+ dev,
+ bootloader_app_version: gsp_fw.bootloader.app_version,
+ libos_dma_handle: self.libos.dma_address(),
+ };
- let static_info = self.cmdq.send_command(commands::GetGspStaticInfo)?;
+ let static_info =
+ commands::gsp_init(&self.cmdq, &init_payload, |header, payload_0, payload_1| {
+ load_exec.dispatch_gmc_boot_event(header.gmc.command_id(), payload_0, payload_1)
+ })?;
Ok(super::BootResult {
unload_bundle: unload_guard.dismiss().1,
@@ -412,12 +427,16 @@ fn shutdown_gsp(
gsp_falcon: &Falcon<'_, Gsp>,
mode: commands::PowerStateLevel,
) -> Result {
- // Command to shut the GSP down.
- cmdq.send_command(commands::UnloadingGuestDriver::new(mode))?;
+ commands::gsp_suspend(cmdq, mode)?;
- // Wait until GSP signals it is suspended.
+ // GSP-RM posts messages while it suspends, and the GSP event interrupt is already freed,
+ // so this poll drains them.
read_poll_timeout(
- || Ok(gsp_falcon.is_processor_suspended()),
+ || {
+ cmdq.drain()?;
+
+ Ok(gsp_falcon.is_processor_suspended())
+ },
|suspended| *suspended,
Delta::from_millis(10),
Delta::from_secs(5),
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index f64a97736ed7..93c31b49903a 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -26,9 +26,7 @@
sync::{
barrier::{
dma_mb,
- Full,
- Read,
- Write, //
+ Full, //
},
Mutex, //
},
@@ -57,7 +55,6 @@
GspGmcMsgElement,
GspMsgElement,
MsgFunction,
- MsgqRxHeader,
MsgqTxHeader,
QueueElementHeader,
GSP_MSG_QUEUE_ELEMENT_SIZE_MAX, //
@@ -185,17 +182,13 @@ struct MsgqData {
///
/// Contains the data for a message queue, that either the driver or GSP writes to.
///
-/// Note that while the write pointer of `tx` corresponds to the `msgq` of the same instance, the
-/// read pointer of `rx` actually refers to the `Msgq` owned by the other side.
-/// This design ensures that only the driver or GSP ever writes to a given instance of this struct.
+/// The queue pointers are in BAR0 registers, so `tx` carries the queue's geometry alone. `msgq`
+/// is aligned to [`GSP_PAGE_SIZE`], and the bytes between the two are zero padding.
#[repr(C)]
-// There is no struct defined for this in the open-gpu-kernel-source headers.
-// Instead it is defined by code in `GspMsgQueuesInit()`.
+// The firmware headers declare no struct for this layout.
struct Msgq {
- /// Header for sending messages, including the write pointer.
+ /// The msgq TX header, which describes the queue to the GSP.
tx: MsgqTxHeader,
- /// Header for receiving messages, including the read pointer.
- rx: MsgqRxHeader,
/// The message queue proper.
msgq: MsgqData,
}
@@ -205,15 +198,11 @@ struct Msgq {
struct GspMem {
/// Self-mapping page table entries.
ptes: PteArray<{ Self::PTE_ARRAY_SIZE }>,
- /// CPU queue: the driver writes commands here, and the GSP reads them. It also contains the
- /// write and read pointers that the CPU updates. This means that the read pointer here is an
- /// index into the GSP queue.
+ /// CPU queue: the driver writes commands here, and the GSP reads them.
///
/// This member is read-only for the GSP.
cpuq: Msgq,
- /// GSP queue: the GSP writes messages here, and the driver reads them. It also contains the
- /// write and read pointers that the GSP updates. This means that the read pointer here is an
- /// index into the CPU queue.
+ /// GSP queue: the GSP writes messages here, and the driver reads them.
///
/// This member is read-only for the driver.
gspq: Msgq,
@@ -242,27 +231,50 @@ unsafe impl FromBytes for GspMem {}
/// pointer and the GSP read pointer. This region is returned by [`Self::driver_write_area`].
/// * The driver owns (i.e. can read from) the part of the GSP message queue between the CPU read
/// pointer and the GSP write pointer. This region is returned by [`Self::driver_read_area`].
+///
+/// Each pointer counts elements without wrapping at the ring size: a ring is empty when its two
+/// pointers are equal and full when they differ by the ring size. A GSP reset zeroes all four
+/// pointer registers, so the driver keeps its own two pointers here and writes them out. It also
+/// keeps the last value that it read of each GSP pointer, and uses that value while the register
+/// reads as zero.
struct DmaGspMem<'a> {
/// The queues, mapped for the GSP.
mem: Coherent<'a, GspMem>,
- /// MMIO mapping of PCI BAR0, for the doorbell register.
+ /// MMIO mapping of PCI BAR0, for the queue pointer registers.
bar: Bar0<'a>,
+ /// Number of elements that the driver has written to the command queue, published to the GSP
+ /// through `NV_PGSP_QUEUE_HEAD`.
+ cpu_write_ptr: u32,
+ /// Number of elements that the driver has consumed from the message queue, published to the
+ /// GSP through `NV_PGSP_MSGQ_TAIL`.
+ cpu_read_ptr: u32,
+ /// The GSP's count of consumed commands, as last read from `NV_PGSP_QUEUE_TAIL`.
+ last_gsp_read_ptr: Cell<u32>,
+ /// The GSP's count of posted messages, as last read from `NV_PGSP_MSGQ_HEAD`.
+ last_gsp_write_ptr: Cell<u32>,
}
impl<'a> DmaGspMem<'a> {
/// Allocate a new instance and map it for `dev`.
fn new(dev: &'a device::Device<device::Bound>, bar: Bar0<'a>) -> Result<Self> {
const MSGQ_SIZE: u32 = num::usize_into_u32::<{ size_of::<Msgq>() }>();
- const RX_HDR_OFF: u32 = num::usize_into_u32::<{ mem::offset_of!(Msgq, rx) }>();
+ const MSG_SIZE: u32 = num::usize_into_u32::<GSP_PAGE_SIZE>();
+ const ENTRY_OFF: u32 = num::usize_into_u32::<{ mem::offset_of!(Msgq, msgq) }>();
let mut gsp_mem = CoherentBox::<'_, GspMem>::zeroed(dev, GFP_KERNEL)?;
- gsp_mem.cpuq.tx = MsgqTxHeader::new(MSGQ_SIZE, RX_HDR_OFF, MSGQ_NUM_PAGES);
- gsp_mem.cpuq.rx = MsgqRxHeader::new();
+ gsp_mem.cpuq.tx = MsgqTxHeader::new(MSGQ_SIZE, MSG_SIZE, MSGQ_NUM_PAGES, ENTRY_OFF);
let gsp_mem: Coherent<'_, _> = gsp_mem.into();
PteArray::init(io_project!(gsp_mem, .ptes), gsp_mem.dma_address())?;
- Ok(Self { mem: gsp_mem, bar })
+ Ok(Self {
+ mem: gsp_mem,
+ bar,
+ cpu_write_ptr: 0,
+ cpu_read_ptr: 0,
+ last_gsp_read_ptr: Cell::new(0),
+ last_gsp_write_ptr: Cell::new(0),
+ })
}
/// Returns the region of the CPU message queue that the driver may write to.
@@ -271,7 +283,7 @@ fn new(dev: &'a device::Device<device::Bound>, bar: Bar0<'a>) -> Result<Self> {
/// region crosses the end of the ring.
fn driver_write_area(&mut self) -> (&mut [[u8; GSP_PAGE_SIZE]], &mut [[u8; GSP_PAGE_SIZE]]) {
let avail = num::u32_as_usize(self.free_slots());
- let w_slot = num::u32_as_usize(self.cpu_write_ptr());
+ let w_slot = num::u32_as_usize(self.cpu_write_ptr % MSGQ_NUM_PAGES);
// Pointer to the first entry of the CPU message queue.
let data = ptr::project!(mut self.mem.as_mut_ptr(), .cpuq.msgq.data[build: 0]);
@@ -292,14 +304,12 @@ fn new(dev: &'a device::Device<device::Bound>, bar: Bar0<'a>) -> Result<Self> {
(&mut after_w[..in_after], &mut before_w[..in_before])
}
- /// Returns the number of command queue slots that the driver may still write.
+ /// Returns the number of command queue slots that the driver may still write: the ring size
+ /// minus the elements that the GSP has not consumed.
fn free_slots(&self) -> u32 {
- let tx = self.cpu_write_ptr();
- let rx = self.gsp_read_ptr();
+ let used = self.cpu_write_ptr.wrapping_sub(self.gsp_read_ptr());
- // One slot always stays empty, so that a full ring and an empty ring differ in their
- // pointers. `tx` is below `MSGQ_NUM_PAGES`, so the subtraction does not underflow.
- (rx + MSGQ_NUM_PAGES - tx - 1) % MSGQ_NUM_PAGES
+ MSGQ_NUM_PAGES.saturating_sub(used)
}
/// Returns the number of bytes that the driver can still write to the command queue.
@@ -310,10 +320,8 @@ fn driver_write_area_size(&self) -> usize {
/// Returns the region of the GSP message queue that the driver may read, as two slices
/// because the ring wraps.
fn driver_read_area(&self) -> (&[[u8; GSP_PAGE_SIZE]], &[[u8; GSP_PAGE_SIZE]]) {
- let tx = self.gsp_write_ptr();
- let rx = self.cpu_read_ptr();
- let avail = num::u32_as_usize((tx + MSGQ_NUM_PAGES - rx) % MSGQ_NUM_PAGES);
- let r_slot = num::u32_as_usize(rx);
+ let avail = num::u32_as_usize(self.gsp_write_ptr().wrapping_sub(self.cpu_read_ptr));
+ let r_slot = num::u32_as_usize(self.cpu_read_ptr % MSGQ_NUM_PAGES);
// Pointer to the first entry of the GSP message queue.
let data = ptr::project!(self.mem.as_ptr(), .gspq.msgq.data[build: 0]);
@@ -381,75 +389,63 @@ fn allocate_command<H: FromBytes + AsBytes>(
})
}
- // Returns the index of the memory page the GSP will write the next message to.
- //
- // # Invariants
- //
- // - The returned value is within `0..MSGQ_NUM_PAGES`.
+ /// Returns the GSP's count of posted messages.
fn gsp_write_ptr(&self) -> u32 {
- let ptr = MsgqTxHeader::write_ptr(io_project!(self.mem, .gspq.tx)) % MSGQ_NUM_PAGES;
-
- // ORDERING: LOAD->LOAD ordering needed to order `gsp_write_ptr` read before data read.
- dma_mb(Read);
+ // ORDERING: `readl` ends in a read barrier, which orders this read before the data reads
+ // that follow it.
+ let ptr = *self.bar.read(regs::NV_PGSP_MSGQ_HEAD).address();
+
+ // A zeroed register is a GSP reset, not a count. No new element is readable until GSP-RM
+ // posts a message and writes the register back.
+ if ptr.wrapping_sub(self.cpu_read_ptr) > MSGQ_NUM_PAGES {
+ return self.last_gsp_write_ptr.get();
+ }
+ self.last_gsp_write_ptr.set(ptr);
ptr
}
- // Returns the index of the memory page the GSP will read the next command from.
- //
- // # Invariants
- //
- // - The returned value is within `0..MSGQ_NUM_PAGES`.
+ /// Returns the GSP's count of consumed commands.
fn gsp_read_ptr(&self) -> u32 {
- let ptr = MsgqRxHeader::read_ptr(io_project!(self.mem, .gspq.rx)) % MSGQ_NUM_PAGES;
+ let ptr = *self.bar.read(regs::NV_PGSP_QUEUE_TAIL).address();
- // ORDERING: LOAD->STORE ordering needed to order `gsp_read_ptr` read before data write.
+ // ORDERING: the data writes that follow must not pass this read. `readl` orders only the
+ // reads that follow it, so the writes need a full barrier.
dma_mb(Full);
- ptr
- }
+ // A zeroed register is a GSP reset, not a count. This returns the last count that it read
+ // until GSP-RM consumes a command and writes the register back. That count undercounts the
+ // free space at worst.
+ if self.cpu_write_ptr.wrapping_sub(ptr) > MSGQ_NUM_PAGES {
+ return self.last_gsp_read_ptr.get();
+ }
+ self.last_gsp_read_ptr.set(ptr);
- // Returns the index of the memory page the CPU can read the next message from.
- //
- // # Invariants
- //
- // - The returned value is within `0..MSGQ_NUM_PAGES`.
- fn cpu_read_ptr(&self) -> u32 {
- MsgqRxHeader::read_ptr(io_project!(self.mem, .cpuq.rx)) % MSGQ_NUM_PAGES
+ ptr
}
- // Informs the GSP that it can send `elem_count` new pages into the message queue.
+ /// Releases `elem_count` more elements of the message queue to the GSP.
fn advance_cpu_read_ptr(&mut self, elem_count: u32) {
- // ORDERING: LOAD->STORE ordering needed to order `cpu_read_ptr` write after data read.
- dma_mb(Full);
+ self.cpu_read_ptr = self.cpu_read_ptr.wrapping_add(elem_count);
- let rx = io_project!(self.mem, .cpuq.rx);
- let rptr = MsgqRxHeader::read_ptr(rx).wrapping_add(elem_count) % MSGQ_NUM_PAGES;
- MsgqRxHeader::set_read_ptr(rx, rptr)
- }
+ // ORDERING: the data reads must complete before this write releases the slots to the GSP.
+ // `writel` orders only the writes before it, so the reads need a full barrier.
+ dma_mb(Full);
- // Returns the index of the memory page the CPU can write the next command to.
- //
- // # Invariants
- //
- // - The returned value is within `0..MSGQ_NUM_PAGES`.
- fn cpu_write_ptr(&self) -> u32 {
- MsgqTxHeader::write_ptr(io_project!(self.mem, .cpuq.tx)) % MSGQ_NUM_PAGES
+ self.bar
+ .write_reg(regs::NV_PGSP_MSGQ_TAIL::zeroed().with_address(self.cpu_read_ptr));
}
- // Publishes `elem_count` more pages of the command queue to the GSP and rings the doorbell.
+ /// Publishes `elem_count` more elements of the command queue to the GSP.
+ ///
+ /// The write to `NV_PGSP_QUEUE_HEAD` that publishes them also interrupts the GSP.
fn advance_cpu_write_ptr(&mut self, elem_count: u32) {
- // ORDERING: STORE->STORE ordering needed to order `cpu_write_ptr` write after data write.
- dma_mb(Write);
+ self.cpu_write_ptr = self.cpu_write_ptr.wrapping_add(elem_count);
- let tx = io_project!(self.mem, .cpuq.tx);
- let wptr = MsgqTxHeader::write_ptr(tx).wrapping_add(elem_count) % MSGQ_NUM_PAGES;
- MsgqTxHeader::set_write_ptr(tx, wptr);
-
- // A write to the head register interrupts the GSP. The pointer itself is in the
- // shared-memory header, so the value written does not matter.
+ // ORDERING: `writel` begins with a write barrier, which orders the data writes before this
+ // register write.
self.bar
- .write_reg(regs::NV_PGSP_QUEUE_HEAD::zeroed().with_address(0u32));
+ .write_reg(regs::NV_PGSP_QUEUE_HEAD::zeroed().with_address(self.cpu_write_ptr));
}
}
@@ -566,19 +562,6 @@ pub(crate) fn new(
})
}
- /// Computes the checksum for the message pointed to by `it`.
- ///
- /// A message is made of several parts, so `it` is an iterator over byte slices representing
- /// these parts.
- fn calculate_checksum<T: Iterator<Item = u8>>(it: T) -> u32 {
- let sum64 = it
- .enumerate()
- .map(|(idx, byte)| (((idx % 8) * 8) as u32, byte))
- .fold(0, |acc, (rol, byte)| acc ^ u64::from(byte).rotate_left(rol));
-
- ((sum64 >> 32) as u32) ^ (sum64 as u32)
- }
-
/// Sends `command` to the GSP and waits for the reply.
///
/// Events that arrive before the reply are logged and consumed.
@@ -595,6 +578,7 @@ fn calculate_checksum<T: Iterator<Item = u8>>(it: T) -> u32 {
/// written to by its [`CommandToGsp::init_variable_payload`] method.
///
/// Error codes returned by the command and reply initializers are propagated as-is.
+ #[expect(dead_code)]
pub(crate) fn send_command<M>(&self, command: M) -> Result<M::Reply>
where
M: CommandToGsp,
@@ -617,6 +601,7 @@ pub(crate) fn send_command<M>(&self, command: M) -> Result<M::Reply>
/// written to by its [`CommandToGsp::init_variable_payload`] method.
///
/// Error codes returned by the command initializers are propagated as-is.
+ #[expect(dead_code)]
pub(crate) fn send_command_no_wait<M>(&self, command: M) -> Result
where
M: CommandToGsp<Reply = NoReply>,
@@ -669,9 +654,10 @@ pub(crate) fn send_gmc_no_wait(
///
/// - `ETIMEDOUT` if the event does not arrive within [`Self::RECEIVE_TIMEOUT`] of the call,
/// however many other events arrive while waiting.
- /// - `EIO` if the queue is poisoned, or if a message fails framing or checksum validation.
+ /// - `EIO` if the queue is poisoned, or if a message fails framing validation.
///
/// Error codes returned by [`MessageFromGsp::read`] are propagated as-is.
+ #[expect(dead_code)]
pub(crate) fn await_msg<M: MessageFromGsp>(&self) -> Result<M>
where
// This allows all error types, including `Infallible`, to be used for `M::InitError`.
@@ -689,7 +675,7 @@ pub(crate) fn await_msg<M: MessageFromGsp>(&self) -> Result<M>
///
/// # Errors
///
- /// `EIO` if the queue is poisoned, or if a message fails framing or checksum validation.
+ /// `EIO` if the queue is poisoned, or if a message fails framing validation.
pub(crate) fn drain(&self) -> Result {
self.inner.lock().drain()
}
@@ -701,7 +687,7 @@ struct CmdqInner<'a> {
dev: &'a device::Device,
/// Current command sequence number.
seq: u32,
- /// Set once a message fails framing or checksum validation. Every later receive fails, since
+ /// Set once a message fails framing validation. Every later receive fails, since
/// the bad message cannot be skipped. See "Draining the GSP-to-CPU queue" in
/// `Documentation/gpu/nova/core/interrupts.rst`.
///
@@ -742,7 +728,7 @@ fn send_single_command<M>(&mut self, command: M) -> Result
let (cmd, payload_1) = M::Command::from_bytes_mut_prefix(dst.contents.0).ok_or(EIO)?;
// Fill the header and command in-place.
- let msg_element = GspMsgElement::init(self.seq, size_in_bytes, M::FUNCTION);
+ let msg_element = GspMsgElement::init(size_in_bytes, M::FUNCTION);
// SAFETY: `msg_header` and `cmd` are valid references, and not touched if the initializer
// fails.
unsafe {
@@ -759,14 +745,6 @@ fn send_single_command<M>(&mut self, command: M) -> Result
}
drop(sbuffer);
- // Compute checksum now that the whole message is ready.
- dst.header
- .set_checksum(Cmdq::calculate_checksum(SBufferIter::new_reader([
- dst.header.as_bytes(),
- dst.contents.0,
- dst.contents.1,
- ])));
-
dev_dbg!(
&self.dev,
"GSP RPC: send: seq# {}, function={:?}, length=0x{:x}\n",
@@ -863,22 +841,19 @@ fn send_gmc(&mut self, command_id: u32, payload: &[u8], max_response_size: u32)
Ok(())
}
- /// Wait for a message to become available on the message queue.
+ /// Waits for a message to become available on the message queue.
///
- /// This works purely at the transport layer and does not interpret or validate the message
- /// beyond the advertised length in its [`GspMsgElement`].
+ /// This validates the queue element header and the lengths that it declares, and does not
+ /// interpret the RPC header that follows it.
///
- /// This method returns:
- ///
- /// - A reference to the [`GspMsgElement`] of the message,
- /// - Two byte slices with the contents of the message. The second slice is empty unless the
- /// message loops across the message queue.
+ /// Returns the message's [`GspMsgElement`] and its contents as two byte slices, the second of
+ /// which is empty unless the message wraps around the end of the message queue.
///
/// # Errors
///
/// - `ETIMEDOUT` if `timeout` has elapsed before any message becomes available.
- /// - `EIO` if the queue is already poisoned, or if the framing or the checksum is invalid,
- /// which poisons it (see [`Self::poisoned`]).
+ /// - `EIO` if the queue is already poisoned, or if the framing is invalid, which poisons it
+ /// (see [`Self::poisoned`]).
fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {
if self.poisoned.get() {
return Err(EIO);
@@ -901,6 +876,13 @@ fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {
)));
};
+ if header.validate_framing().is_err() {
+ return Err(self.poison(fmt!(
+ "RPC element has a bad queue element header, declared length {}",
+ header.length()
+ )));
+ }
+
dev_dbg!(
&self.dev,
"GSP RPC: receive: seq# {}, function={:?}, length=0x{:x}\n",
@@ -909,28 +891,19 @@ fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {
header.length(),
);
- let (slice_1, slice_2) = self.payload_slices(slice_1, slice_2, header.payload_length())?;
-
- // Validate checksum.
- if Cmdq::calculate_checksum(SBufferIter::new_reader([
- header.as_bytes(),
- slice_1,
- slice_2,
- ])) != 0
- {
+ let Some(payload_length) = header.payload_length() else {
return Err(self.poison(fmt!(
- "message with sequence {} has a bad checksum",
+ "RPC message seq# {} declares a message shorter than the RPC header",
header.sequence()
)));
- }
+ };
- Ok(GspMessage {
- header,
- contents: (slice_1, slice_2),
- })
+ let contents = self.payload_slices(slice_1, slice_2, payload_length)?;
+
+ Ok(GspMessage { header, contents })
}
- /// Receive a message from the GSP.
+ /// Receives a message from the GSP.
///
/// A message whose function code is `M::FUNCTION` is decoded and returned. Any other message
/// is logged as an event.
@@ -940,7 +913,7 @@ fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {
/// # Errors
///
/// - `ETIMEDOUT` if `timeout` has elapsed before any message becomes available.
- /// - `EIO` if the queue is poisoned or the message fails framing or checksum validation (see
+ /// - `EIO` if the queue is poisoned or the message fails framing validation (see
/// [`Self::wait_for_msg`]), or if the matched message is too short for `M::Message`.
/// - `ENOMSG` if the message was not the awaited reply.
///
@@ -1000,7 +973,7 @@ fn receive_msg<M: MessageFromGsp>(&mut self, timeout: Delta) -> Result<M>
///
/// - `ETIMEDOUT` if no message of type `M` arrives before the deadline, however many other
/// messages arrive while waiting.
- /// - `EIO` if the queue is poisoned or a message fails framing or checksum validation (see
+ /// - `EIO` if the queue is poisoned or a message fails framing validation (see
/// [`Self::wait_for_msg`]).
///
/// Error codes returned by [`MessageFromGsp::read`] are propagated as-is.
@@ -1057,7 +1030,7 @@ fn log_event(&self, function: Result<MsgFunction, u32>, seq: u32) {
///
/// # Errors
///
- /// `EIO` if the queue is poisoned, a message fails framing or checksum validation, or a
+ /// `EIO` if the queue is poisoned, a message fails framing validation, or a
/// message's page count overflows a `u32`.
fn drain(&mut self) -> Result {
while !self.gsp_mem.driver_read_area().0.is_empty() {
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index 00ff3271a46e..24f80c449c13 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -2,8 +2,6 @@
// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
use core::{
- array,
- convert::Infallible,
ffi::FromBytesUntilNulError,
ops::Range,
str::Utf8Error, //
@@ -13,21 +11,13 @@
device,
pci,
prelude::*,
- transmute::{
- AsBytes,
- FromBytes, //
- }, //
+ transmute::AsBytes, //
};
use crate::{
gpu::Chipset,
gsp::{
- cmdq::{
- Cmdq,
- CommandToGsp,
- MessageFromGsp,
- NoReply, //
- },
+ cmdq::Cmdq,
fw::{
self,
commands::{
@@ -36,7 +26,6 @@
GspInitResponseSchema, //
},
GspGmcMsgElement,
- MsgFunction,
GMCAPI_CMD_GSP_INIT,
GMCAPI_CMD_GSP_SUSPEND, //
},
@@ -52,175 +41,7 @@
vgpu::VgpuState, //
};
-/// The `GspSetSystemInfo` command.
-pub(crate) struct SetSystemInfo<'a> {
- pdev: &'a pci::Device<device::Bound>,
- chipset: Chipset,
-}
-
-impl<'a> SetSystemInfo<'a> {
- /// Creates a new `GspSetSystemInfo` command using the parameters of `pdev`.
- pub(crate) fn new(pdev: &'a pci::Device<device::Bound>, chipset: Chipset) -> Self {
- Self { pdev, chipset }
- }
-}
-
-impl<'a> CommandToGsp for SetSystemInfo<'a> {
- const FUNCTION: MsgFunction = MsgFunction::GspSetSystemInfo;
- type Command = fw::commands::GspSetSystemInfo;
- type Reply = NoReply;
- type InitError = Error;
-
- fn init(&self) -> impl Init<Self::Command, Self::InitError> {
- Self::Command::init(self.pdev, self.chipset)
- }
-}
-
-struct RegistryEntry {
- key: &'static str,
- value: u32,
-}
-
-/// The `SetRegistry` command.
-pub(crate) struct SetRegistry {
- entries: KVec<RegistryEntry>,
-}
-
-impl SetRegistry {
- /// Creates a new `SetRegistry` command, using a set of hardcoded entries.
- pub(crate) fn new(vgpu_state: VgpuState) -> Result<Self> {
- let mut entries = KVec::new();
-
- // RMSecBusResetEnable - enables PCI secondary bus reset
- entries.push(
- RegistryEntry {
- key: "RMSecBusResetEnable",
- value: 1,
- },
- GFP_KERNEL,
- )?;
-
- // RMForcePcieConfigSave - forces GSP-RM to preserve PCI configuration registers on
- // any PCI reset.
- entries.push(
- RegistryEntry {
- key: "RMForcePcieConfigSave",
- value: 1,
- },
- GFP_KERNEL,
- )?;
-
- // RMDevidCheckIgnore - allows GSP-RM to boot even if the PCI dev ID is not found
- // in the internal product name database.
- entries.push(
- RegistryEntry {
- key: "RMDevidCheckIgnore",
- value: 1,
- },
- GFP_KERNEL,
- )?;
-
- if matches!(vgpu_state, VgpuState::Enabled { .. }) {
- // RMSetSriovMode - required when vGPU is enabled.
- entries.push(
- RegistryEntry {
- key: "RMSetSriovMode",
- value: 1,
- },
- GFP_KERNEL,
- )?;
- }
-
- Ok(Self { entries })
- }
-}
-
-impl CommandToGsp for SetRegistry {
- const FUNCTION: MsgFunction = MsgFunction::SetRegistry;
- type Command = fw::commands::PackedRegistryTable;
- type Reply = NoReply;
- type InitError = Infallible;
-
- fn init(&self) -> impl Init<Self::Command, Self::InitError> {
- Self::Command::init(self.entries.len() as u32, self.size() as u32)
- }
-
- fn variable_payload_len(&self) -> usize {
- let mut key_size = 0;
- for entry in self.entries.iter() {
- key_size += entry.key.len() + 1; // +1 for NULL terminator
- }
- self.entries.len() * size_of::<fw::commands::PackedRegistryEntry>() + key_size
- }
-
- fn init_variable_payload(
- &self,
- dst: &mut SBufferIter<core::array::IntoIter<&mut [u8], 2>>,
- ) -> Result {
- let string_data_start_offset = size_of::<Self::Command>()
- + self.entries.len() * size_of::<fw::commands::PackedRegistryEntry>();
-
- // Array for string data.
- let mut string_data = KVec::new();
-
- for entry in self.entries.iter() {
- dst.write_all(
- fw::commands::PackedRegistryEntry::new(
- (string_data_start_offset + string_data.len()) as u32,
- entry.value,
- )
- .as_bytes(),
- )?;
-
- let key_bytes = entry.key.as_bytes();
- string_data.extend_from_slice(key_bytes, GFP_KERNEL)?;
- string_data.push(0, GFP_KERNEL)?;
- }
-
- dst.write_all(string_data.as_slice())
- }
-}
-
-/// Message type for GSP initialization done notification.
-struct GspInitDone;
-
-// SAFETY: `GspInitDone` is a zero-sized type with no bytes, therefore it
-// trivially has no uninitialized bytes.
-unsafe impl FromBytes for GspInitDone {}
-
-impl MessageFromGsp for GspInitDone {
- const FUNCTION: MsgFunction = MsgFunction::GspInitDone;
- type InitError = Infallible;
- type Message = ();
-
- fn read(
- _msg: &Self::Message,
- _sbuffer: &mut SBufferIter<array::IntoIter<&[u8], 2>>,
- ) -> Result<Self, Self::InitError> {
- Ok(GspInitDone)
- }
-}
-
-/// Waits for GSP initialization to complete.
-pub(crate) fn wait_gsp_init_done(cmdq: &Cmdq<'_>) -> Result {
- cmdq.await_msg::<GspInitDone>().map(|_| ())
-}
-
-/// The `GetGspStaticInfo` command.
-pub(crate) struct GetGspStaticInfo;
-
-impl CommandToGsp for GetGspStaticInfo {
- const FUNCTION: MsgFunction = MsgFunction::GetGspStaticInfo;
- type Command = fw::commands::GspStaticConfigInfo;
- type Reply = GspStaticInfo;
- type InitError = Infallible;
-
- fn init(&self) -> impl Init<Self::Command, Self::InitError> {
- Self::Command::init_zeroed()
- }
-}
-
-/// The static GPU configuration, which GSP-RM reports in reply to [`GetGspStaticInfo`].
+/// The static GPU configuration, as decoded from the `GSP_INIT` reply.
pub(crate) struct GspStaticInfo {
gpu_name: [u8; 64],
/// BAR1 Page Directory Entry base address.
@@ -231,30 +52,6 @@ pub(crate) struct GspStaticInfo {
pub(crate) total_fb_end: u64,
}
-impl MessageFromGsp for GspStaticInfo {
- const FUNCTION: MsgFunction = MsgFunction::GetGspStaticInfo;
- type Message = fw::commands::GspStaticConfigInfo;
- type InitError = Error;
-
- fn read(
- msg: &Self::Message,
- _sbuffer: &mut SBufferIter<array::IntoIter<&[u8], 2>>,
- ) -> Result<Self, Self::InitError> {
- let mut usable_fb_regions = KVec::new();
- for region in msg.usable_fb_regions() {
- usable_fb_regions.push(region, GFP_KERNEL)?;
- }
- let total_fb_end = msg.total_fb_end().ok_or(EINVAL)?;
-
- Ok(GspStaticInfo {
- gpu_name: msg.gpu_name_str(),
- bar1_pde_base: msg.bar1_pde_base(),
- usable_fb_regions,
- total_fb_end,
- })
- }
-}
-
/// Error type for [`GspStaticInfo::gpu_name`].
#[derive(Debug)]
pub(crate) enum GpuNameError {
@@ -284,7 +81,6 @@ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> {
/// # Errors
///
/// - `ENOMEM` if the request or the encoder buffer cannot be allocated.
-#[expect(dead_code)]
pub(crate) fn build_gsp_init_payload(
pdev: &pci::Device<device::Bound>,
chipset: Chipset,
@@ -315,7 +111,6 @@ pub(crate) fn build_gsp_init_payload(
/// however many events arrive while waiting.
///
/// Errors from `on_unsolicited_element` and from decoding the reply are propagated as-is.
-#[expect(dead_code)]
pub(crate) fn gsp_init(
cmdq: &Cmdq<'_>,
payload: &[u64],
@@ -388,50 +183,8 @@ fn decode_gsp_init_reply(payload_0: &[u8], payload_1: &[u8]) -> Result<GspStatic
/// # Errors
///
/// Errors from [`Cmdq::send_gmc_no_wait`] are propagated as-is.
-#[expect(dead_code)]
pub(crate) fn gsp_suspend(cmdq: &Cmdq<'_>, level: PowerStateLevel) -> Result {
let params = fw::commands::GspSuspend::new(level);
cmdq.send_gmc_no_wait(GMCAPI_CMD_GSP_SUSPEND, AsBytes::as_bytes(¶ms), 0)
}
-
-/// The `UnloadingGuestDriver` command, used to shut down the GSP.
-///
-/// Only used within the `gsp` module.
-pub(super) struct UnloadingGuestDriver {
- level: PowerStateLevel,
-}
-
-impl UnloadingGuestDriver {
- /// Creates a new `UnloadingGuestDriver` command for the given [`PowerStateLevel`].
- pub(super) fn new(level: PowerStateLevel) -> Self {
- Self { level }
- }
-}
-
-impl CommandToGsp for UnloadingGuestDriver {
- const FUNCTION: MsgFunction = MsgFunction::UnloadingGuestDriver;
- type Command = fw::commands::UnloadingGuestDriver;
- type Reply = UnloadingGuestDriverReply;
- type InitError = Infallible;
-
- fn init(&self) -> impl Init<Self::Command, Self::InitError> {
- fw::commands::UnloadingGuestDriver::new(self.level)
- }
-}
-
-/// The reply from the GSP to the [`UnloadingGuestDriver`] command.
-pub(super) struct UnloadingGuestDriverReply;
-
-impl MessageFromGsp for UnloadingGuestDriverReply {
- const FUNCTION: MsgFunction = MsgFunction::UnloadingGuestDriver;
- type InitError = Infallible;
- type Message = ();
-
- fn read(
- _msg: &Self::Message,
- _sbuffer: &mut SBufferIter<array::IntoIter<&[u8], 2>>,
- ) -> Result<Self, Self::InitError> {
- Ok(UnloadingGuestDriverReply)
- }
-}
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index ee582cbde5f6..0876e2cb785d 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -6,20 +6,14 @@
mod r570_144;
// Alias to avoid repeating the version number with every use.
-use r570_144 as bindings;
+use r000_00 as bindings;
use core::ops::Range;
use kernel::{
bitfield,
- dma::{
- Coherent,
- CoherentView, //
- },
- io::{
- io_read,
- io_write, //
- },
+ dma::Coherent,
+ io::io_write,
prelude::*,
ptr::{
Alignable,
@@ -41,15 +35,15 @@
FbRanges,
FbSizes, //
},
- firmware::gsp::GspFirmware,
+ firmware::{
+ bindata::UcodesImage,
+ gsp::GspFirmware, //
+ },
gpu::{
Architecture,
Chipset, //
},
- gsp::{
- cmdq::Cmdq, //
- GSP_PAGE_SIZE,
- },
+ gsp::{cmdq::Cmdq, GSP_PAGE_SHIFT, GSP_PAGE_SIZE},
mctp::{
MctpHeader,
NvdmHeader,
@@ -62,8 +56,10 @@
};
/// Maximum size of a single GSP message queue element in bytes.
-pub(crate) const GSP_MSG_QUEUE_ELEMENT_SIZE_MAX: usize =
- num::u32_as_usize(bindings::GSP_MSG_QUEUE_ELEMENT_SIZE_MAX);
+///
+/// GSP-RM reads this value at run time from the message queue init arguments rather than from a
+/// build-time constant, so the driver chooses it, and this constant is the one copy.
+pub(crate) const GSP_MSG_QUEUE_ELEMENT_SIZE_MAX: usize = GSP_PAGE_SIZE * 16;
/// Empty type to group methods related to heap parameters for running the GSP firmware.
enum GspFwHeapParams {}
@@ -97,7 +93,7 @@ fn client_alloc_size() -> u64 {
fn management_overhead(fb_size: u64) -> Result<u64> {
let fb_size_gb = fb_size.div_ceil(u64::SZ_1G);
- u64::from(bindings::GSP_FW_HEAP_PARAM_SIZE_PER_GB_FB)
+ u64::from(bindings::GSP_FW_HEAP_PARAM_SIZE_PER_GB)
.checked_mul(fb_size_gb)
.ok_or(EINVAL)?
.align_up(GSP_HEAP_ALIGNMENT)
@@ -304,7 +300,6 @@ pub(crate) enum MsgFunction {
GspInitDone = bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE,
GspLockdownNotice = bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE,
GspPostNoCat = bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD,
- GspRunCpuSequencer = bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER,
MmuFaultQueued = bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED,
OsErrorLog = bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG,
PostEvent = bindings::NV_VGPU_MSG_EVENT_POST_EVENT,
@@ -351,9 +346,6 @@ fn try_from(value: u32) -> Result<MsgFunction> {
bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE => Ok(MsgFunction::GspInitDone),
bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE => Ok(MsgFunction::GspLockdownNotice),
bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD => Ok(MsgFunction::GspPostNoCat),
- bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER => {
- Ok(MsgFunction::GspRunCpuSequencer)
- }
bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED => Ok(MsgFunction::MmuFaultQueued),
bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG => Ok(MsgFunction::OsErrorLog),
bindings::NV_VGPU_MSG_EVENT_POST_EVENT => Ok(MsgFunction::PostEvent),
@@ -371,277 +363,6 @@ fn from(value: MsgFunction) -> Self {
}
}
-/// Sequencer buffer opcode for GSP sequencer commands.
-#[derive(Copy, Clone, Debug, PartialEq)]
-#[repr(u32)]
-pub(crate) enum SeqBufOpcode {
- // Core operation opcodes
- CoreReset = bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_CORE_RESET,
- CoreResume = bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_CORE_RESUME,
- CoreStart = bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_CORE_START,
- CoreWaitForHalt = bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_CORE_WAIT_FOR_HALT,
-
- // Delay opcode
- DelayUs = bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_DELAY_US,
-
- // Register operation opcodes
- RegModify = bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_REG_MODIFY,
- RegPoll = bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_REG_POLL,
- RegStore = bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_REG_STORE,
- RegWrite = bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_REG_WRITE,
-}
-
-impl TryFrom<u32> for SeqBufOpcode {
- type Error = kernel::error::Error;
-
- fn try_from(value: u32) -> Result<SeqBufOpcode> {
- match value {
- bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_CORE_RESET => {
- Ok(SeqBufOpcode::CoreReset)
- }
- bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_CORE_RESUME => {
- Ok(SeqBufOpcode::CoreResume)
- }
- bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_CORE_START => {
- Ok(SeqBufOpcode::CoreStart)
- }
- bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_CORE_WAIT_FOR_HALT => {
- Ok(SeqBufOpcode::CoreWaitForHalt)
- }
- bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_DELAY_US => Ok(SeqBufOpcode::DelayUs),
- bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_REG_MODIFY => {
- Ok(SeqBufOpcode::RegModify)
- }
- bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_REG_POLL => Ok(SeqBufOpcode::RegPoll),
- bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_REG_STORE => Ok(SeqBufOpcode::RegStore),
- bindings::GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_REG_WRITE => Ok(SeqBufOpcode::RegWrite),
- _ => Err(EINVAL),
- }
- }
-}
-
-impl From<SeqBufOpcode> for u32 {
- fn from(value: SeqBufOpcode) -> Self {
- // CAST: `SeqBufOpcode` is `repr(u32)` and can thus be cast losslessly.
- value as u32
- }
-}
-
-/// Wrapper for GSP sequencer register write payload.
-#[repr(transparent)]
-#[derive(Copy, Clone, Debug)]
-pub(crate) struct RegWritePayload(bindings::GSP_SEQ_BUF_PAYLOAD_REG_WRITE);
-
-impl RegWritePayload {
- /// Returns the register address.
- pub(crate) fn addr(&self) -> u32 {
- self.0.addr
- }
-
- /// Returns the value to write.
- pub(crate) fn val(&self) -> u32 {
- self.0.val
- }
-}
-
-// SAFETY: This struct only contains integer types for which all bit patterns are valid.
-unsafe impl FromBytes for RegWritePayload {}
-
-// SAFETY: Padding is explicit and will not contain uninitialized data.
-unsafe impl AsBytes for RegWritePayload {}
-
-/// Wrapper for GSP sequencer register modify payload.
-#[repr(transparent)]
-#[derive(Copy, Clone, Debug)]
-pub(crate) struct RegModifyPayload(bindings::GSP_SEQ_BUF_PAYLOAD_REG_MODIFY);
-
-impl RegModifyPayload {
- /// Returns the register address.
- pub(crate) fn addr(&self) -> u32 {
- self.0.addr
- }
-
- /// Returns the mask to apply.
- pub(crate) fn mask(&self) -> u32 {
- self.0.mask
- }
-
- /// Returns the value to write.
- pub(crate) fn val(&self) -> u32 {
- self.0.val
- }
-}
-
-// SAFETY: This struct only contains integer types for which all bit patterns are valid.
-unsafe impl FromBytes for RegModifyPayload {}
-
-// SAFETY: Padding is explicit and will not contain uninitialized data.
-unsafe impl AsBytes for RegModifyPayload {}
-
-/// Wrapper for GSP sequencer register poll payload.
-#[repr(transparent)]
-#[derive(Copy, Clone, Debug)]
-pub(crate) struct RegPollPayload(bindings::GSP_SEQ_BUF_PAYLOAD_REG_POLL);
-
-impl RegPollPayload {
- /// Returns the register address.
- pub(crate) fn addr(&self) -> u32 {
- self.0.addr
- }
-
- /// Returns the mask to apply.
- pub(crate) fn mask(&self) -> u32 {
- self.0.mask
- }
-
- /// Returns the expected value.
- pub(crate) fn val(&self) -> u32 {
- self.0.val
- }
-
- /// Returns the timeout in microseconds.
- pub(crate) fn timeout(&self) -> u32 {
- self.0.timeout
- }
-}
-
-// SAFETY: This struct only contains integer types for which all bit patterns are valid.
-unsafe impl FromBytes for RegPollPayload {}
-
-// SAFETY: Padding is explicit and will not contain uninitialized data.
-unsafe impl AsBytes for RegPollPayload {}
-
-/// Wrapper for GSP sequencer delay payload.
-#[repr(transparent)]
-#[derive(Copy, Clone, Debug)]
-pub(crate) struct DelayUsPayload(bindings::GSP_SEQ_BUF_PAYLOAD_DELAY_US);
-
-impl DelayUsPayload {
- /// Returns the delay value in microseconds.
- pub(crate) fn val(&self) -> u32 {
- self.0.val
- }
-}
-
-// SAFETY: This struct only contains integer types for which all bit patterns are valid.
-unsafe impl FromBytes for DelayUsPayload {}
-
-// SAFETY: Padding is explicit and will not contain uninitialized data.
-unsafe impl AsBytes for DelayUsPayload {}
-
-/// Wrapper for GSP sequencer register store payload.
-#[repr(transparent)]
-#[derive(Copy, Clone, Debug)]
-pub(crate) struct RegStorePayload(bindings::GSP_SEQ_BUF_PAYLOAD_REG_STORE);
-
-impl RegStorePayload {
- /// Returns the register address.
- pub(crate) fn addr(&self) -> u32 {
- self.0.addr
- }
-
- /// Returns the storage index.
- #[allow(unused)]
- pub(crate) fn index(&self) -> u32 {
- self.0.index
- }
-}
-
-// SAFETY: This struct only contains integer types for which all bit patterns are valid.
-unsafe impl FromBytes for RegStorePayload {}
-
-// SAFETY: Padding is explicit and will not contain uninitialized data.
-unsafe impl AsBytes for RegStorePayload {}
-
-/// Wrapper for GSP sequencer buffer command.
-#[repr(transparent)]
-pub(crate) struct SequencerBufferCmd(bindings::GSP_SEQUENCER_BUFFER_CMD);
-
-impl SequencerBufferCmd {
- /// Returns the opcode as a `SeqBufOpcode` enum, or error if invalid.
- pub(crate) fn opcode(&self) -> Result<SeqBufOpcode> {
- self.0.opCode.try_into()
- }
-
- /// Returns the register write payload by value.
- ///
- /// Returns an error if the opcode is not `SeqBufOpcode::RegWrite`.
- pub(crate) fn reg_write_payload(&self) -> Result<RegWritePayload> {
- if self.opcode()? != SeqBufOpcode::RegWrite {
- return Err(EINVAL);
- }
- // SAFETY: Opcode is verified to be `RegWrite`, so union contains valid `RegWritePayload`.
- Ok(RegWritePayload(unsafe { self.0.payload.regWrite }))
- }
-
- /// Returns the register modify payload by value.
- ///
- /// Returns an error if the opcode is not `SeqBufOpcode::RegModify`.
- pub(crate) fn reg_modify_payload(&self) -> Result<RegModifyPayload> {
- if self.opcode()? != SeqBufOpcode::RegModify {
- return Err(EINVAL);
- }
- // SAFETY: Opcode is verified to be `RegModify`, so union contains valid `RegModifyPayload`.
- Ok(RegModifyPayload(unsafe { self.0.payload.regModify }))
- }
-
- /// Returns the register poll payload by value.
- ///
- /// Returns an error if the opcode is not `SeqBufOpcode::RegPoll`.
- pub(crate) fn reg_poll_payload(&self) -> Result<RegPollPayload> {
- if self.opcode()? != SeqBufOpcode::RegPoll {
- return Err(EINVAL);
- }
- // SAFETY: Opcode is verified to be `RegPoll`, so union contains valid `RegPollPayload`.
- Ok(RegPollPayload(unsafe { self.0.payload.regPoll }))
- }
-
- /// Returns the delay payload by value.
- ///
- /// Returns an error if the opcode is not `SeqBufOpcode::DelayUs`.
- pub(crate) fn delay_us_payload(&self) -> Result<DelayUsPayload> {
- if self.opcode()? != SeqBufOpcode::DelayUs {
- return Err(EINVAL);
- }
- // SAFETY: Opcode is verified to be `DelayUs`, so union contains valid `DelayUsPayload`.
- Ok(DelayUsPayload(unsafe { self.0.payload.delayUs }))
- }
-
- /// Returns the register store payload by value.
- ///
- /// Returns an error if the opcode is not `SeqBufOpcode::RegStore`.
- pub(crate) fn reg_store_payload(&self) -> Result<RegStorePayload> {
- if self.opcode()? != SeqBufOpcode::RegStore {
- return Err(EINVAL);
- }
- // SAFETY: Opcode is verified to be `RegStore`, so union contains valid `RegStorePayload`.
- Ok(RegStorePayload(unsafe { self.0.payload.regStore }))
- }
-}
-
-// SAFETY: This struct only contains integer types for which all bit patterns are valid.
-unsafe impl FromBytes for SequencerBufferCmd {}
-
-// SAFETY: Padding is explicit and will not contain uninitialized data.
-unsafe impl AsBytes for SequencerBufferCmd {}
-
-/// Wrapper for GSP run CPU sequencer RPC.
-#[repr(transparent)]
-pub(crate) struct RunCpuSequencer(bindings::rpc_run_cpu_sequencer_v17_00);
-
-impl RunCpuSequencer {
- /// Returns the command index.
- pub(crate) fn cmd_index(&self) -> u32 {
- self.0.cmdIndex
- }
-}
-
-// SAFETY: This struct only contains integer types for which all bit patterns are valid.
-unsafe impl FromBytes for RunCpuSequencer {}
-
-// SAFETY: Padding is explicit and will not contain uninitialized data.
-unsafe impl AsBytes for RunCpuSequencer {}
-
/// Struct containing the arguments required to pass a memory buffer to the GSP
/// for use during initialisation.
///
@@ -703,72 +424,34 @@ fn id8(name: &str) -> u64 {
}
}
-/// TX header for setting up a message queue with the GSP.
+/// The msgq version that the driver uses.
+const MSGQ_VERSION_MAJOR: u16 = 2;
+const MSGQ_VERSION_MINOR: u16 = 0;
+
+/// The msgq TX header, which describes a queue to the GSP: the msgq version that the driver uses,
+/// and the queue's geometry. The queue pointers are in BAR0 registers rather than in this header.
#[repr(transparent)]
pub(crate) struct MsgqTxHeader(bindings::msgqTxHeader);
impl MsgqTxHeader {
- /// Create a new TX queue header.
- ///
- /// # Arguments
- ///
- /// * `msgq_size` - Total size of the message queue structure, in bytes.
- /// * `rx_hdr_offset` - Offset, in bytes, of the start of the RX header in the message queue
- /// structure.
- /// * `msg_count` - Number of messages that can be sent, i.e. the number of memory pages
- /// allocated for the message queue in the message queue structure.
- pub(crate) fn new(msgq_size: u32, rx_hdr_offset: u32, msg_count: u32) -> Self {
+ /// Creates the msgq TX header of a queue of `msg_count` elements of `msg_size` bytes each,
+ /// whose first element starts `entry_off` bytes into the `msgq_size`-byte queue.
+ pub(crate) fn new(msgq_size: u32, msg_size: u32, msg_count: u32, entry_off: u32) -> Self {
Self(bindings::msgqTxHeader {
- version: 0,
+ versionMajor: MSGQ_VERSION_MAJOR,
+ versionMinor: MSGQ_VERSION_MINOR,
size: msgq_size,
- msgSize: num::usize_into_u32::<GSP_PAGE_SIZE>(),
+ msgSize: msg_size,
msgCount: msg_count,
- writePtr: 0,
- flags: 1,
- rxHdrOff: rx_hdr_offset,
- entryOff: num::usize_into_u32::<GSP_PAGE_SIZE>(),
+ entryOff: entry_off,
+ reserved: [0; 3],
})
}
-
- /// Returns the value of the write pointer for this queue.
- pub(crate) fn write_ptr(this: CoherentView<'_, Self>) -> u32 {
- io_read!(this, .0.writePtr)
- }
-
- /// Sets the value of the write pointer for this queue.
- pub(crate) fn set_write_ptr(this: CoherentView<'_, Self>, val: u32) {
- io_write!(this, .0.writePtr, val)
- }
}
// SAFETY: Padding is explicit and does not contain uninitialized data.
unsafe impl AsBytes for MsgqTxHeader {}
-/// RX header for setting up a message queue with the GSP.
-#[repr(transparent)]
-pub(crate) struct MsgqRxHeader(bindings::msgqRxHeader);
-
-/// Header for the message RX queue.
-impl MsgqRxHeader {
- /// Creates a new RX queue header.
- pub(crate) fn new() -> Self {
- Self(Default::default())
- }
-
- /// Returns the value of the read pointer for this queue.
- pub(crate) fn read_ptr(this: CoherentView<'_, Self>) -> u32 {
- io_read!(this, .0.readPtr)
- }
-
- /// Sets the value of the read pointer for this queue.
- pub(crate) fn set_read_ptr(this: CoherentView<'_, Self>, val: u32) {
- io_write!(this, .0.readPtr, val)
- }
-}
-
-// SAFETY: Padding is explicit and does not contain uninitialized data.
-unsafe impl AsBytes for MsgqRxHeader {}
-
bitfield! {
struct MsgHeaderVersion(u32) {
31:24 major;
@@ -806,92 +489,87 @@ fn init(cmd_size: usize, function: MsgFunction) -> impl Init<Self, Error> {
}
}
-/// GSP Message Element.
-///
-/// This is essentially a message header expected to be followed by the message data.
-#[repr(transparent)]
+/// The headers that open an RPC queue element: the queue element header and the RPC header.
+#[repr(C)]
pub(crate) struct GspMsgElement {
- inner: bindings::GSP_MSG_QUEUE_ELEMENT,
+ element_header: QueueElementHeader,
+ rpc: bindings::rpc_message_header_v,
}
+// `AsBytes` below requires that no padding separates the two headers.
+static_assert!(
+ size_of::<GspMsgElement>()
+ == size_of::<QueueElementHeader>() + size_of::<bindings::rpc_message_header_v>()
+);
+
impl GspMsgElement {
- /// Creates a new message element.
- ///
- /// # Arguments
- ///
- /// * `sequence` - Sequence number of the message.
- /// * `cmd_size` - Size of the command (not including the message element), in bytes.
- /// * `function` - Function of the message.
- pub(crate) fn init(
- sequence: u32,
- cmd_size: usize,
- function: MsgFunction,
- ) -> impl Init<Self, Error> {
+ /// Creates the queue element header and the RPC header of a command with a `cmd_size`-byte
+ /// payload.
+ pub(crate) fn init(cmd_size: usize, function: MsgFunction) -> impl Init<Self, Error> {
type RpcMessageHeader = bindings::rpc_message_header_v;
- type InnerGspMsgElement = bindings::GSP_MSG_QUEUE_ELEMENT;
- let init_inner = try_init!(InnerGspMsgElement {
- seqNum: sequence,
- elemCount: size_of::<Self>()
- .checked_add(cmd_size)
- .ok_or(EOVERFLOW)?
- .div_ceil(GSP_PAGE_SIZE)
- .try_into()
- .map_err(|_| EOVERFLOW)?,
- rpc <- RpcMessageHeader::init(cmd_size, function),
- ..Zeroable::init_zeroed()
- });
try_init!(GspMsgElement {
- inner <- init_inner,
+ element_header: QueueElementHeader::new(
+ NvdmType::RmRpc,
+ size_of::<RpcMessageHeader>()
+ .checked_add(cmd_size)
+ .ok_or(EOVERFLOW)?,
+ )?,
+ rpc <- RpcMessageHeader::init(cmd_size, function),
})
}
- /// Sets the checksum of this message.
- ///
- /// Since the header is also part of the checksum, this is usually called after the whole
- /// message has been written to the shared memory area.
- pub(crate) fn set_checksum(&mut self, checksum: u32) {
- self.inner.checkSum = checksum;
+ /// Returns the length of the payload that follows the RPC header, or `None` if the queue
+ /// element header declares a message shorter than the RPC header.
+ pub(crate) fn payload_length(&self) -> Option<usize> {
+ self.element_header
+ .payload_len(size_of::<bindings::rpc_message_header_v>())
}
- /// Returns the length of the message's payload.
- pub(crate) fn payload_length(&self) -> usize {
- // `rpc.length` includes the length of the RPC message header.
- num::u32_as_usize(self.inner.rpc.length)
- .saturating_sub(size_of::<bindings::rpc_message_header_v>())
+ /// Returns the length of the whole element, both headers included.
+ pub(crate) fn length(&self) -> usize {
+ self.element_header.element_len()
}
- /// Returns the total length of the message, message and RPC headers included.
- pub(crate) fn length(&self) -> usize {
- size_of::<Self>() + self.payload_length()
+ /// Validates the queue element header and that the element is long enough to hold the RPC
+ /// header after it.
+ ///
+ /// # Errors
+ ///
+ /// - `EIO` if [`QueueElementHeader::validate`] fails, or if the declared element length is
+ /// shorter than the two headers together.
+ pub(crate) fn validate_framing(&self) -> Result {
+ self.element_header.validate().map_err(|_| EIO)?;
+
+ if self.length() < size_of::<Self>() {
+ return Err(EIO);
+ }
+
+ Ok(())
}
// Returns the sequence number of the message.
pub(crate) fn sequence(&self) -> u32 {
- self.inner.rpc.sequence
+ self.rpc.sequence
}
// Returns the function of the message, if it is valid, or the invalid function number as an
// error.
pub(crate) fn function(&self) -> Result<MsgFunction, u32> {
- self.inner
- .rpc
- .function
- .try_into()
- .map_err(|_| self.inner.rpc.function)
+ self.rpc.function.try_into().map_err(|_| self.rpc.function)
}
// Returns the number of elements (i.e. memory pages) used by this message.
pub(crate) fn element_count(&self) -> u32 {
- self.inner.elemCount
+ self.element_header.element_count()
}
}
-// SAFETY: Padding is explicit and does not contain uninitialized data.
+// SAFETY: All fields are integer types or contain only integer types, with no
+// uninitialized padding bytes.
unsafe impl AsBytes for GspMsgElement {}
-// SAFETY: This struct only contains integer types for which all bit patterns
-// are valid.
+// SAFETY: All fields are integer types for which all bit patterns are valid.
unsafe impl FromBytes for GspMsgElement {}
/// First word of every queue element: `"MCTP"` in ASCII.
@@ -929,19 +607,19 @@ pub(crate) struct QueueElementHeader {
static_assert!(
core::mem::offset_of!(QueueElementHeader, magic)
- == core::mem::offset_of!(r000_00::GSP_MSG_QUEUE_ELEMENT, mctpMagic)
+ == core::mem::offset_of!(bindings::GSP_MSG_QUEUE_ELEMENT, mctpMagic)
);
static_assert!(
core::mem::offset_of!(QueueElementHeader, element_len)
- == core::mem::offset_of!(r000_00::GSP_MSG_QUEUE_ELEMENT, mctpPayloadSize)
+ == core::mem::offset_of!(bindings::GSP_MSG_QUEUE_ELEMENT, mctpPayloadSize)
);
static_assert!(
core::mem::offset_of!(QueueElementHeader, mctp)
- == core::mem::offset_of!(r000_00::GSP_MSG_QUEUE_ELEMENT, mctpHeader)
+ == core::mem::offset_of!(bindings::GSP_MSG_QUEUE_ELEMENT, mctpHeader)
);
static_assert!(
core::mem::offset_of!(QueueElementHeader, nvdm)
- == core::mem::offset_of!(r000_00::GSP_MSG_QUEUE_ELEMENT, nvdmHeader)
+ == core::mem::offset_of!(bindings::GSP_MSG_QUEUE_ELEMENT, nvdmHeader)
);
impl QueueElementHeader {
@@ -1057,42 +735,42 @@ pub(crate) struct GmcApiHeader {
/// GMC request that carries the system information and registry keys to GSP-RM. GSP-RM answers
/// it with the static GPU configuration once it has finished starting.
-pub(crate) const GMCAPI_CMD_GSP_INIT: u32 = r000_00::GMCAPI_COMMANDS_GMCAPI_CMD_GSP_INIT;
+pub(crate) const GMCAPI_CMD_GSP_INIT: u32 = bindings::GMCAPI_COMMANDS_GMCAPI_CMD_GSP_INIT;
/// GMC event that requests the driver to run the generic falcon bootloader on the descriptor that
/// the event carries.
pub(crate) const GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER: u32 =
- r000_00::GMCAPI_COMMANDS_GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER;
+ bindings::GMCAPI_COMMANDS_GMCAPI_CMD_EXEC_GENERIC_BOOTLOADER;
/// GMC event that requests the driver to run a Heavy-Secured (HS) binary that GSP-RM has placed in
/// the framebuffer.
pub(crate) const GMCAPI_CMD_EXEC_HS_BINARY: u32 =
- r000_00::GMCAPI_COMMANDS_GMCAPI_CMD_EXEC_HS_BINARY;
+ bindings::GMCAPI_COMMANDS_GMCAPI_CMD_EXEC_HS_BINARY;
/// GMC request for GSP-RM to suspend. GSP-RM sends no response, and reports the completed
/// suspend in the GSP falcon's `MAILBOX0` instead.
-pub(crate) const GMCAPI_CMD_GSP_SUSPEND: u32 = r000_00::GMCAPI_COMMANDS_GMCAPI_CMD_GSP_SUSPEND;
+pub(crate) const GMCAPI_CMD_GSP_SUSPEND: u32 = bindings::GMCAPI_COMMANDS_GMCAPI_CMD_GSP_SUSPEND;
-static_assert!(size_of::<GmcApiHeader>() == size_of::<r000_00::GMCAPI_HEADER>());
+static_assert!(size_of::<GmcApiHeader>() == size_of::<bindings::GMCAPI_HEADER>());
static_assert!(
core::mem::offset_of!(GmcApiHeader, command)
- == core::mem::offset_of!(r000_00::GMCAPI_HEADER, command)
+ == core::mem::offset_of!(bindings::GMCAPI_HEADER, command)
);
static_assert!(
core::mem::offset_of!(GmcApiHeader, size)
- == core::mem::offset_of!(r000_00::GMCAPI_HEADER, size)
+ == core::mem::offset_of!(bindings::GMCAPI_HEADER, size)
);
static_assert!(
core::mem::offset_of!(GmcApiHeader, sequence)
- == core::mem::offset_of!(r000_00::GMCAPI_HEADER, sequence)
+ == core::mem::offset_of!(bindings::GMCAPI_HEADER, sequence)
);
static_assert!(
core::mem::offset_of!(GmcApiHeader, max_resp_or_status)
- == core::mem::offset_of!(r000_00::GMCAPI_HEADER, __bindgen_anon_1)
+ == core::mem::offset_of!(bindings::GMCAPI_HEADER, __bindgen_anon_1)
);
static_assert!(
core::mem::offset_of!(GmcApiHeader, reserved)
- == core::mem::offset_of!(r000_00::GMCAPI_HEADER, reserved)
+ == core::mem::offset_of!(bindings::GMCAPI_HEADER, reserved)
);
impl GmcApiHeader {
@@ -1185,6 +863,12 @@ unsafe impl AsBytes for GspGmcMsgElement {}
// SAFETY: All fields are integer types for which all bit patterns are valid.
unsafe impl FromBytes for GspGmcMsgElement {}
+/// First word of `GSP_ARGUMENTS_CACHED`: `"GSP "` in ASCII.
+const GSP_ARGUMENTS_MAGIC_VALUE: u32 = 0x2050_5347;
+
+/// Flag that requests GSP-RM to place its stack in DMEM.
+const GSP_ARGUMENTS_FLAG_STACK_IN_DMEM: u64 = 0x02;
+
/// Arguments for GSP startup.
#[repr(transparent)]
#[derive(Zeroable)]
@@ -1193,11 +877,21 @@ pub(crate) struct GspArgumentsCached {
}
impl GspArgumentsCached {
- /// Creates the arguments for starting the GSP up using `cmdq` as its command queue.
- pub(crate) fn new<'a, 'b>(cmdq: &'a Cmdq<'b>) -> impl Init<Self> + use<'a, 'b> {
+ /// Creates the arguments for starting the GSP, with `cmdq` as its command queue and
+ /// `state_monitor` as the buffer in which GSP-RM reports its own state.
+ pub(crate) fn new<'a, 'b>(
+ cmdq: &'a Cmdq<'b>,
+ state_monitor: &'a Coherent<'b, [u8; GSP_PAGE_SIZE]>,
+ ) -> impl Init<Self> + use<'a, 'b> {
let init_inner = init!(bindings::GSP_ARGUMENTS_CACHED {
+ magic: GSP_ARGUMENTS_MAGIC_VALUE,
+ size: num::usize_into_u16::<{ size_of::<bindings::GSP_ARGUMENTS_CACHED>() }>(),
+ flags: GSP_ARGUMENTS_FLAG_STACK_IN_DMEM,
messageQueueInitArguments <- MessageQueueInitArguments::new(cmdq),
- bDmemStack: 1,
+ rmStateMonitorBufferArgs: bindings::GSP_ARGUMENTS_CACHED__bindgen_ty_3 {
+ pa: state_monitor.dma_address(),
+ size: num::usize_as_u64(state_monitor.size()),
+ },
..Zeroable::init_zeroed()
});
@@ -1221,12 +915,23 @@ pub(crate) struct GspArgumentsPadded {
}
impl GspArgumentsPadded {
- pub(crate) fn new<'a, 'b>(cmdq: &'a Cmdq<'b>) -> impl Init<Self> + use<'a, 'b> {
+ pub(crate) fn new<'a, 'b>(
+ cmdq: &'a Cmdq<'b>,
+ state_monitor: &'a Coherent<'b, [u8; GSP_PAGE_SIZE]>,
+ ) -> impl Init<Self> + use<'a, 'b> {
init!(GspArgumentsPadded {
- inner <- GspArgumentsCached::new(cmdq),
+ inner <- GspArgumentsCached::new(cmdq, state_monitor),
..Zeroable::init_zeroed()
})
}
+
+ /// Records where `ucodes` is mapped.
+ ///
+ /// GSP-RM reads the arguments once, when it starts, so a write after that point has no effect.
+ pub(crate) fn set_bindata(this: &Coherent<'_, Self>, ucodes: &UcodesImage<'_>) {
+ io_write!(this, .inner.inner.bindataArgs.radix3, ucodes.radix3_dma_address());
+ io_write!(this, .inner.inner.bindataArgs.size, num::usize_as_u64(ucodes.size()));
+ }
}
// SAFETY: Padding is explicit and will not contain uninitialized data.
@@ -1247,6 +952,15 @@ fn new<'a, 'b>(cmdq: &'a Cmdq<'b>) -> impl Init<Self> + use<'a, 'b> {
pageTableEntryCount: num::usize_into_u32::<{ Cmdq::NUM_PTES }>(),
cmdQueueOffset: num::usize_as_u64(Cmdq::CMDQ_OFFSET),
statQueueOffset: num::usize_as_u64(Cmdq::STATQ_OFFSET),
+
+ queueElementHdrSize: num::usize_into_u32::<{ size_of::<QueueElementHeader>() }>(),
+ queueElementSizeMin: num::usize_into_u32::<GSP_PAGE_SIZE>(),
+ queueElementSizeMax: num::usize_into_u32::<GSP_MSG_QUEUE_ELEMENT_SIZE_MAX>(),
+
+ // Both alignments are log2 values, which GSP-RM applies as `1 << n`.
+ queueHeaderAlign: 4,
+ queueElementAlign: num::usize_into_u32::<GSP_PAGE_SHIFT>(),
+
..Zeroable::init_zeroed()
})
}
@@ -1271,7 +985,9 @@ fn new(target: GspDmaTarget, wpr_meta_addr: u64) -> impl Init<Self> {
bIsGspRmBoot: 1,
wprCarveoutOffset: 0,
wprCarveoutSize: 0,
- __bindgen_padding_0: Default::default(),
+ bInstInSysMode: 0,
+ bIcuEnabled: 0,
+ bScrubCbcSr: 0,
});
params
@@ -1284,8 +1000,8 @@ impl GspRmParams {
fn new(target: GspDmaTarget, libos_addr: u64) -> impl Init<Self> {
let params = init!(Self {
target: target as u32,
+ reserved: 0,
bootArgsOffset: libos_addr,
- __bindgen_padding_0: Default::default(),
});
params
@@ -1294,6 +1010,9 @@ fn new(target: GspDmaTarget, libos_addr: u64) -> impl Init<Self> {
pub(crate) type GspFmcBootParams = bindings::GSP_FMC_BOOT_PARAMS;
+/// Magic value opening the ABI-stable `GSP_FMC_BOOT_PARAMS` header: `"FMC "` in ASCII.
+const GSP_FMC_BOOT_PARAMS_MAGIC: u32 = 0x2043_4d46;
+
// SAFETY: Padding is explicit and will not contain uninitialized data.
unsafe impl AsBytes for GspFmcBootParams {}
// SAFETY: This struct only contains integer types for which all bit patterns are valid.
@@ -1302,6 +1021,8 @@ unsafe impl FromBytes for GspFmcBootParams {}
impl GspFmcBootParams {
pub(crate) fn new(wpr_meta_addr: u64, libos_addr: u64) -> impl Init<Self> {
let init = init!(Self {
+ magic: GSP_FMC_BOOT_PARAMS_MAGIC,
+ size: num::usize_into_u16::<{ size_of::<Self>() }>(),
// Blackwell FSP obtains WPR info from other sources, so
// wprCarveoutOffset and wprCarveoutSize are left zero.
bootGspRmParams <- GspAcrBootGspRmParams::new(GspDmaTarget::CoherentSystem,
diff --git a/drivers/gpu/nova-core/gsp/fw/commands.rs b/drivers/gpu/nova-core/gsp/fw/commands.rs
index 4e6c712637b0..9792cea36770 100644
--- a/drivers/gpu/nova-core/gsp/fw/commands.rs
+++ b/drivers/gpu/nova-core/gsp/fw/commands.rs
@@ -9,16 +9,11 @@
device,
pci,
prelude::*,
- transmute::{
- AsBytes,
- FromBytes, //
- }, //
+ transmute::AsBytes, //
};
use crate::{
gpu::Chipset,
- gsp::GSP_PAGE_SIZE,
- num::IntoSafeCast,
vgpu::VgpuState, //
};
@@ -38,173 +33,7 @@
use super::bindings;
-/// Payload of the `GspSetSystemInfo` command.
-#[repr(transparent)]
-pub(crate) struct GspSetSystemInfo {
- inner: bindings::GspSystemInfo,
-}
-static_assert!(size_of::<GspSetSystemInfo>() < GSP_PAGE_SIZE);
-
-impl GspSetSystemInfo {
- /// Returns an in-place initializer for the `GspSetSystemInfo` command.
- pub(crate) fn init<'a>(
- dev: &'a pci::Device<device::Bound>,
- chipset: Chipset,
- ) -> impl Init<Self, Error> + 'a {
- type InnerGspSystemInfo = bindings::GspSystemInfo;
- let pci_config_mirror_range = chipset.pci_config_mirror_range();
- let init_inner = try_init!(InnerGspSystemInfo {
- gpuPhysAddr: dev.resource_start(0)?,
- gpuPhysFbAddr: dev.resource_start(1)?,
- gpuPhysInstAddr: dev.resource_start(3)?,
- nvDomainBusDeviceFunc: u64::from(dev.dev_id()),
-
- // Using TASK_SIZE in r535_gsp_rpc_set_system_info() seems wrong because
- // TASK_SIZE is per-task. That's probably a design issue in GSP-RM though.
- maxUserVa: (1 << 47) - 4096,
- pciConfigMirrorBase: pci_config_mirror_range.start,
- pciConfigMirrorSize: pci_config_mirror_range.end - pci_config_mirror_range.start,
-
- PCIDeviceID: (u32::from(dev.device_id()) << 16) | u32::from(dev.vendor_id().as_raw()),
- PCISubDeviceID: (u32::from(dev.subsystem_device_id()) << 16)
- | u32::from(dev.subsystem_vendor_id()),
- PCIRevisionID: u32::from(dev.revision_id()),
- bIsPrimary: 0,
- bPreserveVideoMemoryAllocations: 0,
- ..Zeroable::init_zeroed()
- });
-
- try_init!(GspSetSystemInfo {
- inner <- init_inner,
- })
- }
-}
-
-// SAFETY: These structs don't meet the no-padding requirements of AsBytes but
-// that is not a problem because they are not used outside the kernel.
-unsafe impl AsBytes for GspSetSystemInfo {}
-
-// SAFETY: These structs don't meet the no-padding requirements of FromBytes but
-// that is not a problem because they are not used outside the kernel.
-unsafe impl FromBytes for GspSetSystemInfo {}
-
-#[repr(transparent)]
-pub(crate) struct PackedRegistryEntry(bindings::PACKED_REGISTRY_ENTRY);
-
-impl PackedRegistryEntry {
- pub(crate) fn new(offset: u32, value: u32) -> Self {
- Self({
- bindings::PACKED_REGISTRY_ENTRY {
- nameOffset: offset,
-
- // We only support DWORD types for now. Support for other types
- // will come later if required.
- type_: bindings::REGISTRY_TABLE_ENTRY_TYPE_DWORD as u8,
- __bindgen_padding_0: Default::default(),
- data: value,
- length: 0,
- }
- })
- }
-}
-
-// SAFETY: Padding is explicit and will not contain uninitialized data.
-unsafe impl AsBytes for PackedRegistryEntry {}
-
-/// Payload of the `SetRegistry` command.
-#[repr(transparent)]
-pub(crate) struct PackedRegistryTable {
- inner: bindings::PACKED_REGISTRY_TABLE,
-}
-
-impl PackedRegistryTable {
- pub(crate) fn init(num_entries: u32, size: u32) -> impl Init<Self> {
- type InnerPackedRegistryTable = bindings::PACKED_REGISTRY_TABLE;
- let init_inner = init!(InnerPackedRegistryTable {
- numEntries: num_entries,
- size,
- entries: Default::default()
- });
-
- init!(PackedRegistryTable { inner <- init_inner })
- }
-}
-
-// SAFETY: Padding is explicit and will not contain uninitialized data.
-unsafe impl AsBytes for PackedRegistryTable {}
-
-// SAFETY: This struct only contains integer types for which all bit patterns
-// are valid.
-unsafe impl FromBytes for PackedRegistryTable {}
-
-/// Payload of the `GetGspStaticInfo` command and message.
-#[repr(transparent)]
-#[derive(Zeroable)]
-pub(crate) struct GspStaticConfigInfo(bindings::GspStaticConfigInfo_t);
-
-impl GspStaticConfigInfo {
- /// Returns a bytes array containing the (hopefully) zero-terminated name of this GPU.
- pub(crate) fn gpu_name_str(&self) -> [u8; 64] {
- self.0.gpuNameString
- }
-
- /// Returns the BAR1 Page Directory Entry base address.
- ///
- /// This is the root page table address for BAR1 virtual memory,
- /// set up by GSP-RM firmware.
- pub(crate) fn bar1_pde_base(&self) -> u64 {
- self.0.bar1PdeBase
- }
-
- /// Returns an iterator over valid FB regions from GSP firmware data.
- fn fb_regions(
- &self,
- ) -> impl Iterator<Item = &bindings::NV2080_CTRL_CMD_FB_GET_FB_REGION_FB_REGION_INFO> {
- let fb_info = &self.0.fbRegionInfoParams;
- fb_info
- .fbRegion
- .iter()
- .take(fb_info.numFBRegions.into_safe_cast())
- .filter(|reg| reg.limit >= reg.base)
- }
-
- /// Iterates over usable FB regions from GSP firmware data.
- ///
- /// Each yielded region is a [`Range<u64>`] suitable for driver memory allocation.
- /// Usable regions are those that satisfy all the following properties:
- /// - Are not reserved for firmware internal use.
- /// - Are not protected (hardware-enforced access restrictions).
- /// - Support compression (can use GPU memory compression for bandwidth).
- /// - Support ISO (isochronous memory for display requiring guaranteed bandwidth).
- pub(crate) fn usable_fb_regions(&self) -> impl Iterator<Item = Range<u64>> + '_ {
- self.fb_regions().filter_map(|reg| {
- // Filter: not reserved, not protected, supports compression and ISO.
- if reg.reserved == 0
- && reg.bProtected == 0
- && reg.supportCompressed != 0
- && reg.supportISO != 0
- {
- reg.limit.checked_add(1).map(|end| reg.base..end)
- } else {
- None
- }
- })
- }
-
- /// Computes the exclusive end of the FB physical address space.
- pub(crate) fn total_fb_end(&self) -> Option<u64> {
- self.fb_regions().map(|reg| reg.limit).max()?.checked_add(1)
- }
-}
-
-// SAFETY: Padding is explicit and will not contain uninitialized data.
-unsafe impl AsBytes for GspStaticConfigInfo {}
-
-// SAFETY: This struct only contains integer types for which all bit patterns
-// are valid.
-unsafe impl FromBytes for GspStaticConfigInfo {}
-
-/// Power level requested to the [`UnloadingGuestDriver`] command.
+/// Power level that a `GSP_SUSPEND` request names.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
#[repr(u32)]
#[expect(unused)]
@@ -252,29 +81,6 @@ pub(crate) fn new(level: PowerStateLevel) -> Self {
// SAFETY: The single field is an integer type, and the struct has no padding.
unsafe impl AsBytes for GspSuspend {}
-/// Payload of the `UnloadingGuestDriver` command and message.
-#[repr(transparent)]
-#[derive(Clone, Copy, Debug, Zeroable)]
-pub(crate) struct UnloadingGuestDriver(bindings::rpc_unloading_guest_driver_v1F_07);
-
-impl UnloadingGuestDriver {
- pub(crate) fn new(level: PowerStateLevel) -> Self {
- Self(bindings::rpc_unloading_guest_driver_v1F_07 {
- bInPMTransition: u8::from(level.is_power_transition()),
- bGc6Entering: 0,
- newLevel: level as u32,
- ..Zeroable::zeroed()
- })
- }
-}
-
-// SAFETY: Padding is explicit and will not contain uninitialized data.
-unsafe impl AsBytes for UnloadingGuestDriver {}
-
-// SAFETY: This struct only contains integer types for which all bit patterns
-// are valid.
-unsafe impl FromBytes for UnloadingGuestDriver {}
-
/// The host CPU architecture.
#[derive(Clone, Copy)]
pub(crate) enum HostArch {
diff --git a/drivers/gpu/nova-core/gsp/hal.rs b/drivers/gpu/nova-core/gsp/hal.rs
index 8c2a8abcb187..b23beb6e2907 100644
--- a/drivers/gpu/nova-core/gsp/hal.rs
+++ b/drivers/gpu/nova-core/gsp/hal.rs
@@ -56,7 +56,6 @@ fn boot<'gpu>(
/// # Errors
///
/// Errors from loading the bootloader image are propagated as-is.
- #[expect(dead_code)]
fn generic_bootloader(
&self,
_dev: &device::Device<device::Bound>,
@@ -65,19 +64,6 @@ fn generic_bootloader(
) -> Result<Option<GenericBootloader>> {
Ok(None)
}
-
- /// Performs HAL-specific post-GSP boot tasks.
- ///
- /// This method is called by the GSP boot code after the GSP is confirmed to be running, and
- /// after the initialization commands have been pushed onto its queue.
- fn post_boot(
- &self,
- _gsp: &Gsp<'_>,
- _ctx: &mut GspBootContext<'_, '_>,
- _gsp_fw: &GspFirmware<'_>,
- ) -> Result {
- Ok(())
- }
}
/// Returns the names of the firmware files required to boot the GSP of `chipset`, in addition to
diff --git a/drivers/gpu/nova-core/gsp/hal/tu102.rs b/drivers/gpu/nova-core/gsp/hal/tu102.rs
index 32931fd0b7cf..265039b8ff55 100644
--- a/drivers/gpu/nova-core/gsp/hal/tu102.rs
+++ b/drivers/gpu/nova-core/gsp/hal/tu102.rs
@@ -41,7 +41,6 @@
UnloadBundle, //
},
regs,
- sequencer::GspSequencer,
Gsp,
GspBootContext,
GspFwWprMeta, //
@@ -329,17 +328,6 @@ fn generic_bootloader(
GenericBootloader::new(dev, chipset, imem_size).map(Some)
}
-
- fn post_boot(
- &self,
- gsp: &Gsp<'_>,
- ctx: &mut GspBootContext<'_, '_>,
- gsp_fw: &GspFirmware<'_>,
- ) -> Result {
- GspSequencer::run(&gsp.cmdq, ctx, &gsp.libos, gsp_fw.bootloader.app_version)?;
-
- Ok(())
- }
}
/// The TU102 HAL requires the use of the FWSEC bootloader.
diff --git a/drivers/gpu/nova-core/gsp/sequencer.rs b/drivers/gpu/nova-core/gsp/sequencer.rs
deleted file mode 100644
index 250adc9fe74f..000000000000
--- a/drivers/gpu/nova-core/gsp/sequencer.rs
+++ /dev/null
@@ -1,379 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-//! GSP Sequencer implementation for Pre-hopper GSP boot sequence.
-
-use core::array;
-
-use kernel::{
- device,
- dma::Coherent,
- io::{
- poll::read_poll_timeout,
- Io, //
- },
- prelude::*,
- time::{
- delay::fsleep,
- Delta, //
- },
- transmute::FromBytes, //
-};
-
-use crate::{
- driver::Bar0,
- falcon::{
- gsp::Gsp,
- sec2::Sec2,
- Falcon, //
- },
- gsp::{
- cmdq::{
- Cmdq,
- MessageFromGsp, //
- },
- fw,
- GspBootContext,
- LibosMemoryRegionInitArgument, //
- },
- num::FromSafeCast,
- sbuffer::SBufferIter,
-};
-
-/// GSP Sequencer information containing the command sequence and data.
-struct GspSequence {
- /// Current command index for error reporting.
- cmd_index: u32,
- /// Command data buffer containing the sequence of commands.
- cmd_data: KVec<u8>,
-}
-
-impl MessageFromGsp for GspSequence {
- const FUNCTION: fw::MsgFunction = fw::MsgFunction::GspRunCpuSequencer;
- type InitError = Error;
- type Message = fw::RunCpuSequencer;
-
- fn read(
- msg: &Self::Message,
- sbuffer: &mut SBufferIter<array::IntoIter<&[u8], 2>>,
- ) -> Result<Self, Self::InitError> {
- let cmd_data = sbuffer.flush_into_kvec(GFP_KERNEL)?;
- Ok(GspSequence {
- cmd_index: msg.cmd_index(),
- cmd_data,
- })
- }
-}
-
-const CMD_SIZE: usize = size_of::<fw::SequencerBufferCmd>();
-
-/// GSP Sequencer Command types with payload data.
-/// Commands have an opcode and an opcode-dependent struct.
-#[allow(clippy::enum_variant_names)]
-#[derive(Debug)]
-pub(crate) enum GspSeqCmd {
- RegWrite(fw::RegWritePayload),
- RegModify(fw::RegModifyPayload),
- RegPoll(fw::RegPollPayload),
- DelayUs(fw::DelayUsPayload),
- RegStore(fw::RegStorePayload),
- CoreReset,
- CoreStart,
- CoreWaitForHalt,
- CoreResume,
-}
-
-impl GspSeqCmd {
- /// Creates a new `GspSeqCmd` from raw data returning the command and its size in bytes.
- pub(crate) fn new(data: &[u8], dev: &device::Device) -> Result<(Self, usize)> {
- let fw_cmd = fw::SequencerBufferCmd::from_bytes(data).ok_or(EINVAL)?;
- let opcode_size = core::mem::size_of::<u32>();
-
- let (cmd, size) = match fw_cmd.opcode()? {
- fw::SeqBufOpcode::RegWrite => {
- let payload = fw_cmd.reg_write_payload()?;
- let size = opcode_size + size_of_val(&payload);
- (GspSeqCmd::RegWrite(payload), size)
- }
- fw::SeqBufOpcode::RegModify => {
- let payload = fw_cmd.reg_modify_payload()?;
- let size = opcode_size + size_of_val(&payload);
- (GspSeqCmd::RegModify(payload), size)
- }
- fw::SeqBufOpcode::RegPoll => {
- let payload = fw_cmd.reg_poll_payload()?;
- let size = opcode_size + size_of_val(&payload);
- (GspSeqCmd::RegPoll(payload), size)
- }
- fw::SeqBufOpcode::DelayUs => {
- let payload = fw_cmd.delay_us_payload()?;
- let size = opcode_size + size_of_val(&payload);
- (GspSeqCmd::DelayUs(payload), size)
- }
- fw::SeqBufOpcode::RegStore => {
- let payload = fw_cmd.reg_store_payload()?;
- let size = opcode_size + size_of_val(&payload);
- (GspSeqCmd::RegStore(payload), size)
- }
- fw::SeqBufOpcode::CoreReset => (GspSeqCmd::CoreReset, opcode_size),
- fw::SeqBufOpcode::CoreStart => (GspSeqCmd::CoreStart, opcode_size),
- fw::SeqBufOpcode::CoreWaitForHalt => (GspSeqCmd::CoreWaitForHalt, opcode_size),
- fw::SeqBufOpcode::CoreResume => (GspSeqCmd::CoreResume, opcode_size),
- };
-
- if data.len() < size {
- dev_err!(dev, "Data is not enough for command\n");
- return Err(EINVAL);
- }
-
- Ok((cmd, size))
- }
-}
-
-/// GSP Sequencer for executing firmware commands during boot.
-pub(crate) struct GspSequencer<'a> {
- /// `Bar0` for register access.
- bar: Bar0<'a>,
- /// SEC2 falcon for core operations.
- sec2_falcon: &'a Falcon<'a, Sec2>,
- /// GSP falcon for core operations.
- gsp_falcon: &'a Falcon<'a, Gsp>,
- /// LibOS memory region init arguments.
- libos: &'a Coherent<'a, [LibosMemoryRegionInitArgument]>,
- /// Bootloader application version.
- bootloader_app_version: u32,
- /// Device for logging.
- dev: &'a device::Device,
-}
-
-impl fw::RegWritePayload {
- fn run(&self, sequencer: &GspSequencer<'_>) -> Result {
- let addr = usize::from_safe_cast(self.addr());
-
- sequencer.bar.try_write32(self.val(), addr)
- }
-}
-
-impl fw::RegModifyPayload {
- fn run(&self, sequencer: &GspSequencer<'_>) -> Result {
- let addr = usize::from_safe_cast(self.addr());
-
- sequencer.bar.try_read32(addr).and_then(|val| {
- sequencer
- .bar
- .try_write32((val & !self.mask()) | self.val(), addr)
- })
- }
-}
-
-impl fw::RegPollPayload {
- fn run(&self, sequencer: &GspSequencer<'_>) -> Result {
- let addr = usize::from_safe_cast(self.addr());
-
- // Default timeout to 4 seconds.
- let timeout_us = if self.timeout() == 0 {
- 4_000_000
- } else {
- i64::from(self.timeout())
- };
-
- // First read.
- sequencer.bar.try_read32(addr)?;
-
- // Poll the requested register with requested timeout.
- read_poll_timeout(
- || sequencer.bar.try_read32(addr),
- |current| (current & self.mask()) == self.val(),
- Delta::ZERO,
- Delta::from_micros(timeout_us),
- )
- .map(|_| ())
- }
-}
-
-impl fw::DelayUsPayload {
- fn run(&self, _sequencer: &GspSequencer<'_>) -> Result {
- fsleep(Delta::from_micros(i64::from(self.val())));
- Ok(())
- }
-}
-
-impl fw::RegStorePayload {
- fn run(&self, sequencer: &GspSequencer<'_>) -> Result {
- let addr = usize::from_safe_cast(self.addr());
-
- sequencer.bar.try_read32(addr).map(|_| ())
- }
-}
-
-impl GspSeqCmd {
- fn run(&self, seq: &GspSequencer<'_>) -> Result {
- match self {
- GspSeqCmd::RegWrite(cmd) => cmd.run(seq),
- GspSeqCmd::RegModify(cmd) => cmd.run(seq),
- GspSeqCmd::RegPoll(cmd) => cmd.run(seq),
- GspSeqCmd::DelayUs(cmd) => cmd.run(seq),
- GspSeqCmd::RegStore(cmd) => cmd.run(seq),
- GspSeqCmd::CoreReset => {
- seq.gsp_falcon.reset()?;
- seq.gsp_falcon.dma_reset();
- Ok(())
- }
- GspSeqCmd::CoreStart => {
- seq.gsp_falcon.start()?;
- Ok(())
- }
- GspSeqCmd::CoreWaitForHalt => {
- seq.gsp_falcon.wait_till_halted()?;
- Ok(())
- }
- GspSeqCmd::CoreResume => {
- // At this point, 'SEC2-RTOS' has been loaded into SEC2 by the sequencer
- // but neither SEC2-RTOS nor GSP-RM is running yet. This part of the
- // sequencer will start both.
-
- // Reset the GSP to prepare it for resuming.
- seq.gsp_falcon.reset()?;
-
- let libos_dma_address = seq.libos.dma_address();
-
- // Write the libOS DMA address to GSP mailboxes.
- seq.gsp_falcon.write_mailboxes(
- Some(libos_dma_address as u32),
- Some((libos_dma_address >> 32) as u32),
- );
-
- // Start the SEC2 falcon which will trigger GSP-RM to resume on the GSP.
- seq.sec2_falcon.start()?;
-
- // Poll until GSP-RM reload/resume has completed (up to 2 seconds).
- seq.gsp_falcon.check_reload_completed(Delta::from_secs(2))?;
-
- // Verify SEC2 completed successfully by checking its mailbox for errors.
- let mbox0 = seq.sec2_falcon.read_mailbox0();
- if mbox0 != 0 {
- dev_err!(seq.dev, "Sequencer: sec2 errors: {:?}\n", mbox0);
- return Err(EIO);
- }
-
- // Configure GSP with the bootloader version.
- seq.gsp_falcon.write_os_version(seq.bootloader_app_version);
-
- // Verify the GSP's RISC-V core is active indicating successful GSP boot.
- if !seq.gsp_falcon.is_riscv_active() {
- dev_err!(seq.dev, "Sequencer: RISC-V core is not active\n");
- return Err(EIO);
- }
- Ok(())
- }
- }
- }
-}
-
-/// Iterator over GSP sequencer commands.
-struct GspSeqIter<'a> {
- /// Command data buffer.
- cmd_data: &'a [u8],
- /// Current position in the buffer.
- current_offset: usize,
- /// Total number of commands to process.
- total_cmds: u32,
- /// Number of commands processed so far.
- cmds_processed: u32,
- /// Device for logging.
- dev: &'a device::Device,
-}
-
-impl<'a> GspSeqIter<'a> {
- fn new(seq: &'a GspSequence, dev: &'a device::Device) -> Self {
- Self {
- cmd_data: &seq.cmd_data,
- current_offset: 0,
- total_cmds: seq.cmd_index,
- cmds_processed: 0,
- dev,
- }
- }
-}
-
-impl<'a> Iterator for GspSeqIter<'a> {
- type Item = Result<GspSeqCmd>;
-
- fn next(&mut self) -> Option<Self::Item> {
- // Stop if we've processed all commands or reached the end of data.
- if self.cmds_processed >= self.total_cmds || self.current_offset >= self.cmd_data.len() {
- return None;
- }
-
- // Check if we have enough data for opcode.
- if self.current_offset + core::mem::size_of::<u32>() > self.cmd_data.len() {
- return Some(Err(EIO));
- }
-
- let offset = self.current_offset;
-
- // Handle command creation based on available data,
- // zero-pad if necessary (since last command may not be full size).
- let mut buffer = [0u8; CMD_SIZE];
- let copy_len = if offset + CMD_SIZE <= self.cmd_data.len() {
- CMD_SIZE
- } else {
- self.cmd_data.len() - offset
- };
- buffer[..copy_len].copy_from_slice(&self.cmd_data[offset..offset + copy_len]);
- let cmd_result = GspSeqCmd::new(&buffer, self.dev);
-
- cmd_result.map_or_else(
- |_err| {
- dev_err!(self.dev, "Error parsing command at offset {}\n", offset);
- None
- },
- |(cmd, size)| {
- self.current_offset += size;
- self.cmds_processed += 1;
- Some(Ok(cmd))
- },
- )
- }
-}
-
-impl<'a> GspSequencer<'a> {
- pub(crate) fn run(
- cmdq: &Cmdq<'_>,
- ctx: &'a GspBootContext<'_, '_>,
- libos: &'a Coherent<'a, [LibosMemoryRegionInitArgument]>,
- bootloader_app_version: u32,
- ) -> Result {
- let seq_info = cmdq.await_msg::<GspSequence>()?;
-
- let sequencer = GspSequencer {
- bar: ctx.bar,
- sec2_falcon: ctx.sec2_falcon,
- gsp_falcon: ctx.gsp_falcon,
- libos,
- bootloader_app_version,
- dev: ctx.dev(),
- };
-
- dev_dbg!(sequencer.dev, "Running CPU Sequencer commands\n");
-
- for cmd_result in GspSeqIter::new(&seq_info, sequencer.dev) {
- match cmd_result {
- Ok(cmd) => cmd.run(&sequencer)?,
- Err(e) => {
- dev_err!(
- sequencer.dev,
- "Error running command at index {}\n",
- seq_info.cmd_index
- );
- return Err(e);
- }
- }
- }
-
- dev_dbg!(
- sequencer.dev,
- "CPU Sequencer commands completed successfully\n"
- );
- Ok(())
- }
-}
diff --git a/drivers/gpu/nova-core/sbuffer.rs b/drivers/gpu/nova-core/sbuffer.rs
index b8c01104c255..f2cb9276b6e8 100644
--- a/drivers/gpu/nova-core/sbuffer.rs
+++ b/drivers/gpu/nova-core/sbuffer.rs
@@ -164,6 +164,7 @@ pub(crate) fn read_exact(&mut self, mut dst: &mut [u8]) -> Result {
/// Read all the remaining data into a [`KVec`].
///
/// `self` will be empty after this operation.
+ #[expect(unused)]
pub(crate) fn flush_into_kvec(&mut self, flags: kernel::alloc::Flags) -> Result<KVec<u8>> {
let mut buf = KVec::<u8>::new();
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 28/33] gpu: nova-core: gsp: make the GSP_INIT reply the static configuration
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (26 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 27/33] gpu: nova-core: switch to the r000 GSP firmware John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 29/33] gpu: nova-core: firmware: delete the r570 bindings John Hubbard
` (4 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The boot sequence returns the static GPU configuration, the GPU's name,
its usable framebuffer regions and its BAR1 page directory base, to the
rest of the driver. On r000 that configuration is the decoded GSP_INIT
reply.
The configuration type came from the r570 boot protocol, where the
reader of the r570 static-info reply filled it from a C struct. The
GSP_INIT decoder filled the same type by copying the name and the
usable regions out of the decoded reply. The decoder failed the boot
with EINVAL when the reply reported no framebuffer region. With the
r570 reader gone, that copy was the type's only purpose.
Make the decoded reply the static configuration type, and read the
configuration through its accessors. The accessor for the usable
regions yields an iterator, so the regions are no longer copied into a
vector. A reply that reports no framebuffer region now fails with
ENODEV when the driver creates its memory manager.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gpu.rs | 16 +++---
drivers/gpu/nova-core/gsp/commands.rs | 70 ++----------------------
drivers/gpu/nova-core/gsp/fw/commands.rs | 41 +++++++++++---
drivers/gpu/nova-core/mm.rs | 4 +-
4 files changed, 48 insertions(+), 83 deletions(-)
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index 0ed0f4722dc5..fd1a74913d7c 100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -465,16 +465,16 @@ pub(crate) fn new<'a>(
Err(e) => dev_warn!(dev, "GPU name unavailable: {:?}\n", e),
}
- if !info.usable_fb_regions.is_empty() {
+ if info.usable_fb_regions().next().is_some() {
dev_dbg!(dev, "Usable FB regions:\n");
- for region in &info.usable_fb_regions {
+ for region in info.usable_fb_regions() {
dev_dbg!(dev, " - {:#x?}\n", region);
}
dev_dbg!(
dev,
"Total usable VRAM: {} MiB\n",
- info.usable_fb_regions.iter().fold(0u64, |res, region| res
+ info.usable_fb_regions().fold(0u64, |res, region| res
.saturating_add(region.end - region.start))
/ u64::SZ_1M
);
@@ -484,7 +484,7 @@ pub(crate) fn new<'a>(
// Create GPU memory manager owning memory management resources.
mm: {
let info = gsp_resources.static_info();
- let usable_vram = info.usable_fb_regions.first().ok_or(ENODEV)?;
+ let usable_vram = info.usable_fb_regions().next().ok_or(ENODEV)?;
let buddy_params = GpuBuddyParams {
base_offset: usable_vram.start,
size: usable_vram.end - usable_vram.start,
@@ -495,13 +495,13 @@ pub(crate) fn new<'a>(
bar,
gsp_resources.spec.chipset,
buddy_params,
- VramAddress::from_raw(info.total_fb_end),
+ VramAddress::from_raw(info.total_fb_end().ok_or(ENODEV)?),
)?
},
// Create BAR1 user interface for CPU access to GPU virtual memory.
bar_user: {
- let pdb_addr = VramAddress::from_raw(gsp_resources.static_info().bar1_pde_base);
+ let pdb_addr = VramAddress::from_raw(gsp_resources.static_info().bar1_pde_base());
let bar1_idx = crate::driver::bar1_resource_index(pdev)?;
let bar1_size = pdev.resource_len(bar1_idx)?;
Arc::pin_init(
@@ -527,9 +527,9 @@ pub(crate) fn run_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bou
if let Err(err) = crate::mm::selftest::run(
dev,
this.mm,
- &info.usable_fb_regions,
+ info.usable_fb_regions(),
this.bar_user,
- info.bar1_pde_base,
+ info.bar1_pde_base(),
this.spec.chipset,
) {
dev_err!(dev, "self-tests failed: {:?}\n", err);
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index 24f80c449c13..128d6f8dcb43 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -1,12 +1,6 @@
// SPDX-License-Identifier: GPL-2.0
// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
-use core::{
- ffi::FromBytesUntilNulError,
- ops::Range,
- str::Utf8Error, //
-};
-
use kernel::{
device,
pci,
@@ -22,7 +16,6 @@
self,
commands::{
GspInitRequest,
- GspInitResponse,
GspInitResponseSchema, //
},
GspGmcMsgElement,
@@ -41,40 +34,7 @@
vgpu::VgpuState, //
};
-/// The static GPU configuration, as decoded from the `GSP_INIT` reply.
-pub(crate) struct GspStaticInfo {
- gpu_name: [u8; 64],
- /// BAR1 Page Directory Entry base address.
- pub(crate) bar1_pde_base: u64,
- /// Usable FB (VRAM) regions for driver memory allocation.
- pub(crate) usable_fb_regions: KVec<Range<u64>>,
- /// Exclusive end of the FB physical address space.
- pub(crate) total_fb_end: u64,
-}
-
-/// Error type for [`GspStaticInfo::gpu_name`].
-#[derive(Debug)]
-pub(crate) enum GpuNameError {
- /// The GPU name string does not contain a null terminator.
- NoNullTerminator(FromBytesUntilNulError),
-
- /// The GPU name string contains invalid UTF-8.
- #[expect(dead_code)]
- InvalidUtf8(Utf8Error),
-}
-
-impl GspStaticInfo {
- /// Returns the name of the GPU as a string.
- ///
- /// Returns an error if the string given by the GSP does not contain a null terminator or
- /// contains invalid UTF-8.
- pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> {
- CStr::from_bytes_until_nul(&self.gpu_name)
- .map_err(GpuNameError::NoNullTerminator)?
- .to_str()
- .map_err(GpuNameError::InvalidUtf8)
- }
-}
+pub(crate) use fw::commands::GspStaticInfo;
/// Builds the NVKV-encoded payload of a `GSP_INIT` request for `pdev`.
///
@@ -128,13 +88,12 @@ pub(crate) fn gsp_init(
)
}
-/// Decodes the `GSP_INIT` reply from its payload, which the ring may have split in two, into the
-/// static configuration type that the boot sequence returns.
+/// Decodes the `GSP_INIT` reply from its payload, which the ring may have split in two.
///
/// # Errors
///
-/// - `EINVAL` if the payload is not a whole number of NVKV words, if the stream is malformed or
-/// omits a required key, or if GSP-RM reported no framebuffer region.
+/// - `EINVAL` if the payload is not a whole number of NVKV words, or if the stream is malformed
+/// or omits a required key.
/// - `ENOMEM` if the words or the decoded regions cannot be allocated.
fn decode_gsp_init_reply(payload_0: &[u8], payload_1: &[u8]) -> Result<GspStaticInfo> {
const WORD_SIZE: usize = size_of::<u64>();
@@ -154,26 +113,9 @@ fn decode_gsp_init_reply(payload_0: &[u8], payload_1: &[u8]) -> Result<GspStatic
let decoder = Decoder::new(&words, UnknownKeyPolicy::Ignore);
let mut schema = GspInitResponseSchema::default();
- let decoded = KBox::try_init(decoder.decode(&mut schema)?, GFP_KERNEL)?;
-
- let mut gpu_name = [0u8; GspInitResponse::MAX_GPU_NAME_LEN];
- let name = decoded.gpu_name();
- gpu_name
- .get_mut(..name.len())
- .ok_or(EINVAL)?
- .copy_from_slice(name);
-
- let mut usable_fb_regions = KVec::new();
- for region in decoded.usable_fb_regions() {
- usable_fb_regions.push(region, GFP_KERNEL)?;
- }
+ let info = KBox::try_init(decoder.decode(&mut schema)?, GFP_KERNEL)?;
- Ok(GspStaticInfo {
- gpu_name,
- bar1_pde_base: decoded.bar1_pde_base(),
- usable_fb_regions,
- total_fb_end: decoded.total_fb_end().ok_or(EINVAL)?,
- })
+ Ok(KBox::into_inner(info))
}
pub(crate) use fw::commands::PowerStateLevel;
diff --git a/drivers/gpu/nova-core/gsp/fw/commands.rs b/drivers/gpu/nova-core/gsp/fw/commands.rs
index 9792cea36770..60edbb12627f 100644
--- a/drivers/gpu/nova-core/gsp/fw/commands.rs
+++ b/drivers/gpu/nova-core/gsp/fw/commands.rs
@@ -1,7 +1,11 @@
// SPDX-License-Identifier: GPL-2.0
// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
-use core::ops::Range;
+use core::{
+ ffi::FromBytesUntilNulError,
+ ops::Range,
+ str::Utf8Error, //
+};
use kernel::{
alloc::ArrayVec,
@@ -288,9 +292,9 @@ pub(crate) fn new(
// Should decode with UnknownKeyPolicy::Ignore.
nvkv_decode! {
/// Schema for the `GSP_INIT` response.
- pub(crate) struct GspInitResponseSchema => GspInitResponse {
+ pub(crate) struct GspInitResponseSchema => GspStaticInfo {
gpu_name:
- Array<u8, { GspInitResponse::MAX_GPU_NAME_LEN }, { Self::GPU_NAME_STRING_KEY }>,
+ Array<u8, { GspStaticInfo::MAX_GPU_NAME_LEN }, { Self::GPU_NAME_STRING_KEY }>,
fb_regions: Accumulated<FbRegionSchema>,
bar1_pde_base: Required<u64, { Self::BAR1_PDE_BASE_KEY }>,
vmmu_segment_size: Key<u64, { Self::VMMU_SEGMENT_SIZE_KEY }>,
@@ -304,8 +308,8 @@ impl GspInitResponseSchema {
const VMMU_SEGMENT_SIZE_KEY: KeyId = 0x1050;
}
-/// Payload of the `GSP_INIT` response.
-pub(crate) struct GspInitResponse {
+/// The static GPU configuration, as decoded from the `GSP_INIT` reply.
+pub(crate) struct GspStaticInfo {
gpu_name: ArrayVec<u8, { Self::MAX_GPU_NAME_LEN }>,
fb_regions: KVVec<FbRegion>,
bar1_pde_base: u64,
@@ -313,16 +317,35 @@ pub(crate) struct GspInitResponse {
vmmu_segment_size: u64,
}
-impl GspInitResponse {
+/// Error type for [`GspStaticInfo::gpu_name`].
+#[derive(Debug)]
+pub(crate) enum GpuNameError {
+ /// The GPU name string does not contain a NUL terminator.
+ NoNullTerminator(FromBytesUntilNulError),
+
+ /// The GPU name string contains invalid UTF-8.
+ #[expect(dead_code)]
+ InvalidUtf8(Utf8Error),
+}
+
+impl GspStaticInfo {
pub(crate) const MAX_GPU_NAME_LEN: usize = 64;
/// Tag of a general-purpose region. Any other tag marks a region that GSP-RM reserves for the
/// use that the tag names.
const FB_REGION_TAG_NONE: u32 = 0;
- /// Returns the GPU name, which GSP-RM sends with its NUL terminator.
- pub(crate) fn gpu_name(&self) -> &[u8] {
- self.gpu_name.as_slice()
+ /// Returns the name of the GPU as a string.
+ ///
+ /// # Errors
+ ///
+ /// - [`GpuNameError::NoNullTerminator`] if the name that GSP-RM sent has no NUL terminator.
+ /// - [`GpuNameError::InvalidUtf8`] if the name is not valid UTF-8.
+ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> {
+ CStr::from_bytes_until_nul(self.gpu_name.as_slice())
+ .map_err(GpuNameError::NoNullTerminator)?
+ .to_str()
+ .map_err(GpuNameError::InvalidUtf8)
}
/// Returns an iterator over the FB regions from which the driver may allocate: the
diff --git a/drivers/gpu/nova-core/mm.rs b/drivers/gpu/nova-core/mm.rs
index a5bc4042577b..ea85c821f0e2 100644
--- a/drivers/gpu/nova-core/mm.rs
+++ b/drivers/gpu/nova-core/mm.rs
@@ -308,7 +308,7 @@ pub(crate) mod selftest {
pub(crate) fn run(
dev: &device::Device<device::Bound>,
mm: &mut GpuMm<'_>,
- usable_fb_regions: &[Range<u64>],
+ mut usable_fb_regions: impl Iterator<Item = Range<u64>>,
bar_user: &Arc<bar_user::BarUser<'_>>,
bar1_pdb: u64,
chipset: Chipset,
@@ -316,7 +316,7 @@ pub(crate) fn run(
// VRAM span the self-tests are free to overwrite, from the chosen test base.
const SELFTEST_SPAN: u64 = u64::SZ_64M;
- let base = usable_fb_regions.iter().find_map(|region| {
+ let base = usable_fb_regions.find_map(|region| {
// Tests rely on this being 8 byte aligned for checking misalignment handling.
let base = region.start.align_up(Alignment::new::<8>())?;
(base.checked_add(SELFTEST_SPAN)? <= region.end).then_some(base)
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 29/33] gpu: nova-core: firmware: delete the r570 bindings
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (27 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 28/33] gpu: nova-core: gsp: make the GSP_INIT reply the static configuration John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 30/33] gpu: nova-core: match GSP RPC replies by sequence, not just function John Hubbard
` (3 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
The switch to the r000 firmware left the r570 bindings with no reference
anywhere in the driver, but the r570 bindings module stayed declared and
its generated definitions were still compiled.
Delete the r570 bindings module, along with the last comment that named
r570.
Assisted-by: LLM
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/fsp.rs | 3 +-
drivers/gpu/nova-core/gsp/fw.rs | 1 -
drivers/gpu/nova-core/gsp/fw/r570_144.rs | 31 -
.../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 1060 -----------------
4 files changed, 2 insertions(+), 1093 deletions(-)
delete mode 100644 drivers/gpu/nova-core/gsp/fw/r570_144.rs
delete mode 100644 drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs
diff --git a/drivers/gpu/nova-core/fsp.rs b/drivers/gpu/nova-core/fsp.rs
index b738dcabcdef..02521db56e03 100644
--- a/drivers/gpu/nova-core/fsp.rs
+++ b/drivers/gpu/nova-core/fsp.rs
@@ -259,7 +259,8 @@ fn frts_vidmem_offset(hal: &dyn hal::FspHal, fb_info: &FbSizes) -> Result<u64> {
// As per OpenRM's `kfspPrepareBootCommands_GH100`.
if fb_info.pmu_reserved_size != 0 {
offset = (offset + u64::from(fb_info.pmu_reserved_size))
- // The 2 MiB alignment is r570-specific.
+ // This must be aligned to WPR alignment, which is 128 KiB, and must also be at
+ // least 128 KiB. 2 MiB satisfies both.
.align_up(Alignment::new::<SZ_2M>())
.ok_or(EINVAL)?;
}
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index 0876e2cb785d..14271fbe0c25 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -3,7 +3,6 @@
pub(crate) mod commands;
mod r000_00;
-mod r570_144;
// Alias to avoid repeating the version number with every use.
use r000_00 as bindings;
diff --git a/drivers/gpu/nova-core/gsp/fw/r570_144.rs b/drivers/gpu/nova-core/gsp/fw/r570_144.rs
deleted file mode 100644
index 2e6f0d298756..000000000000
--- a/drivers/gpu/nova-core/gsp/fw/r570_144.rs
+++ /dev/null
@@ -1,31 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-//! Firmware bindings.
-//!
-//! Imports the generated bindings by `bindgen`.
-//!
-//! This module may not be directly used. Please abstract or re-export the needed symbols in the
-//! parent module instead.
-
-#![allow(
- dead_code,
- clippy::all,
- clippy::undocumented_unsafe_blocks,
- clippy::ptr_as_ptr,
- clippy::ref_as_ptr,
- missing_docs,
- non_camel_case_types,
- non_upper_case_globals,
- non_snake_case,
- improper_ctypes,
- unreachable_pub,
- unsafe_op_in_unsafe_fn
-)]
-use kernel::ffi;
-use pin_init::MaybeZeroable;
-
-include!("r570_144/bindings.rs");
-
-// SAFETY: This type has a size of zero, so its inclusion into another type should not affect their
-// ability to implement `Zeroable`.
-unsafe impl<T> kernel::prelude::Zeroable for __IncompleteArrayField<T> {}
diff --git a/drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs b/drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs
deleted file mode 100644
index afe3e007f088..000000000000
--- a/drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs
+++ /dev/null
@@ -1,1060 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-#[repr(C)]
-#[derive(Default)]
-pub struct __IncompleteArrayField<T>(::core::marker::PhantomData<T>, [T; 0]);
-impl<T> __IncompleteArrayField<T> {
- #[inline]
- pub const fn new() -> Self {
- __IncompleteArrayField(::core::marker::PhantomData, [])
- }
- #[inline]
- pub fn as_ptr(&self) -> *const T {
- self as *const _ as *const T
- }
- #[inline]
- pub fn as_mut_ptr(&mut self) -> *mut T {
- self as *mut _ as *mut T
- }
- #[inline]
- pub unsafe fn as_slice(&self, len: usize) -> &[T] {
- ::core::slice::from_raw_parts(self.as_ptr(), len)
- }
- #[inline]
- pub unsafe fn as_mut_slice(&mut self, len: usize) -> &mut [T] {
- ::core::slice::from_raw_parts_mut(self.as_mut_ptr(), len)
- }
-}
-impl<T> ::core::fmt::Debug for __IncompleteArrayField<T> {
- fn fmt(&self, fmt: &mut ::core::fmt::Formatter<'_>) -> ::core::fmt::Result {
- fmt.write_str("__IncompleteArrayField")
- }
-}
-pub const NV2080_CTRL_GPU_SET_POWER_STATE_GPU_LEVEL_0: u32 = 0;
-pub const NV2080_CTRL_GPU_SET_POWER_STATE_GPU_LEVEL_3: u32 = 3;
-pub const NV2080_CTRL_GPU_SET_POWER_STATE_GPU_LEVEL_7: u32 = 7;
-pub const NV_VGPU_MSG_SIGNATURE_VALID: u32 = 1129337430;
-pub const GSP_FW_HEAP_PARAM_OS_SIZE_LIBOS2: u32 = 0;
-pub const GSP_FW_HEAP_PARAM_OS_SIZE_LIBOS3_BAREMETAL: u32 = 23068672;
-pub const GSP_FW_HEAP_PARAM_BASE_RM_SIZE_TU10X: u32 = 8388608;
-pub const GSP_FW_HEAP_PARAM_BASE_RM_SIZE_GH100: u32 = 14680064;
-pub const GSP_FW_HEAP_PARAM_SIZE_PER_GB_FB: u32 = 98304;
-pub const GSP_FW_HEAP_PARAM_CLIENT_ALLOC_SIZE: u32 = 100663296;
-pub const GSP_FW_HEAP_SIZE_VGPU_DEFAULT: u32 = 609222656;
-pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MIN_MB: u32 = 64;
-pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MAX_MB: u32 = 256;
-pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MIN_MB: u32 = 88;
-pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MAX_MB: u32 = 280;
-pub const GSP_FW_WPR_META_REVISION: u32 = 1;
-pub const GSP_FW_WPR_META_MAGIC: i64 = -2577556379034558285;
-pub const REGISTRY_TABLE_ENTRY_TYPE_DWORD: u32 = 1;
-pub const GSP_MSG_QUEUE_ELEMENT_SIZE_MAX: u32 = 65536;
-pub type __u8 = ffi::c_uchar;
-pub type __u16 = ffi::c_ushort;
-pub type __u32 = ffi::c_uint;
-pub type __u64 = ffi::c_ulonglong;
-pub type u8_ = __u8;
-pub type u16_ = __u16;
-pub type u32_ = __u32;
-pub type u64_ = __u64;
-pub const NV_VGPU_MSG_FUNCTION_NOP: _bindgen_ty_2 = 0;
-pub const NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO: _bindgen_ty_2 = 1;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_ROOT: _bindgen_ty_2 = 2;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_DEVICE: _bindgen_ty_2 = 3;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_MEMORY: _bindgen_ty_2 = 4;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_CTX_DMA: _bindgen_ty_2 = 5;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_CHANNEL_DMA: _bindgen_ty_2 = 6;
-pub const NV_VGPU_MSG_FUNCTION_MAP_MEMORY: _bindgen_ty_2 = 7;
-pub const NV_VGPU_MSG_FUNCTION_BIND_CTX_DMA: _bindgen_ty_2 = 8;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_OBJECT: _bindgen_ty_2 = 9;
-pub const NV_VGPU_MSG_FUNCTION_FREE: _bindgen_ty_2 = 10;
-pub const NV_VGPU_MSG_FUNCTION_LOG: _bindgen_ty_2 = 11;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_VIDMEM: _bindgen_ty_2 = 12;
-pub const NV_VGPU_MSG_FUNCTION_UNMAP_MEMORY: _bindgen_ty_2 = 13;
-pub const NV_VGPU_MSG_FUNCTION_MAP_MEMORY_DMA: _bindgen_ty_2 = 14;
-pub const NV_VGPU_MSG_FUNCTION_UNMAP_MEMORY_DMA: _bindgen_ty_2 = 15;
-pub const NV_VGPU_MSG_FUNCTION_GET_EDID: _bindgen_ty_2 = 16;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_DISP_CHANNEL: _bindgen_ty_2 = 17;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_DISP_OBJECT: _bindgen_ty_2 = 18;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_SUBDEVICE: _bindgen_ty_2 = 19;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_DYNAMIC_MEMORY: _bindgen_ty_2 = 20;
-pub const NV_VGPU_MSG_FUNCTION_DUP_OBJECT: _bindgen_ty_2 = 21;
-pub const NV_VGPU_MSG_FUNCTION_IDLE_CHANNELS: _bindgen_ty_2 = 22;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_EVENT: _bindgen_ty_2 = 23;
-pub const NV_VGPU_MSG_FUNCTION_SEND_EVENT: _bindgen_ty_2 = 24;
-pub const NV_VGPU_MSG_FUNCTION_REMAPPER_CONTROL: _bindgen_ty_2 = 25;
-pub const NV_VGPU_MSG_FUNCTION_DMA_CONTROL: _bindgen_ty_2 = 26;
-pub const NV_VGPU_MSG_FUNCTION_DMA_FILL_PTE_MEM: _bindgen_ty_2 = 27;
-pub const NV_VGPU_MSG_FUNCTION_MANAGE_HW_RESOURCE: _bindgen_ty_2 = 28;
-pub const NV_VGPU_MSG_FUNCTION_BIND_ARBITRARY_CTX_DMA: _bindgen_ty_2 = 29;
-pub const NV_VGPU_MSG_FUNCTION_CREATE_FB_SEGMENT: _bindgen_ty_2 = 30;
-pub const NV_VGPU_MSG_FUNCTION_DESTROY_FB_SEGMENT: _bindgen_ty_2 = 31;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_SHARE_DEVICE: _bindgen_ty_2 = 32;
-pub const NV_VGPU_MSG_FUNCTION_DEFERRED_API_CONTROL: _bindgen_ty_2 = 33;
-pub const NV_VGPU_MSG_FUNCTION_REMOVE_DEFERRED_API: _bindgen_ty_2 = 34;
-pub const NV_VGPU_MSG_FUNCTION_SIM_ESCAPE_READ: _bindgen_ty_2 = 35;
-pub const NV_VGPU_MSG_FUNCTION_SIM_ESCAPE_WRITE: _bindgen_ty_2 = 36;
-pub const NV_VGPU_MSG_FUNCTION_SIM_MANAGE_DISPLAY_CONTEXT_DMA: _bindgen_ty_2 = 37;
-pub const NV_VGPU_MSG_FUNCTION_FREE_VIDMEM_VIRT: _bindgen_ty_2 = 38;
-pub const NV_VGPU_MSG_FUNCTION_PERF_GET_PSTATE_INFO: _bindgen_ty_2 = 39;
-pub const NV_VGPU_MSG_FUNCTION_PERF_GET_PERFMON_SAMPLE: _bindgen_ty_2 = 40;
-pub const NV_VGPU_MSG_FUNCTION_PERF_GET_VIRTUAL_PSTATE_INFO: _bindgen_ty_2 = 41;
-pub const NV_VGPU_MSG_FUNCTION_PERF_GET_LEVEL_INFO: _bindgen_ty_2 = 42;
-pub const NV_VGPU_MSG_FUNCTION_MAP_SEMA_MEMORY: _bindgen_ty_2 = 43;
-pub const NV_VGPU_MSG_FUNCTION_UNMAP_SEMA_MEMORY: _bindgen_ty_2 = 44;
-pub const NV_VGPU_MSG_FUNCTION_SET_SURFACE_PROPERTIES: _bindgen_ty_2 = 45;
-pub const NV_VGPU_MSG_FUNCTION_CLEANUP_SURFACE: _bindgen_ty_2 = 46;
-pub const NV_VGPU_MSG_FUNCTION_UNLOADING_GUEST_DRIVER: _bindgen_ty_2 = 47;
-pub const NV_VGPU_MSG_FUNCTION_TDR_SET_TIMEOUT_STATE: _bindgen_ty_2 = 48;
-pub const NV_VGPU_MSG_FUNCTION_SWITCH_TO_VGA: _bindgen_ty_2 = 49;
-pub const NV_VGPU_MSG_FUNCTION_GPU_EXEC_REG_OPS: _bindgen_ty_2 = 50;
-pub const NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO: _bindgen_ty_2 = 51;
-pub const NV_VGPU_MSG_FUNCTION_ALLOC_VIRTMEM: _bindgen_ty_2 = 52;
-pub const NV_VGPU_MSG_FUNCTION_UPDATE_PDE_2: _bindgen_ty_2 = 53;
-pub const NV_VGPU_MSG_FUNCTION_SET_PAGE_DIRECTORY: _bindgen_ty_2 = 54;
-pub const NV_VGPU_MSG_FUNCTION_GET_STATIC_PSTATE_INFO: _bindgen_ty_2 = 55;
-pub const NV_VGPU_MSG_FUNCTION_TRANSLATE_GUEST_GPU_PTES: _bindgen_ty_2 = 56;
-pub const NV_VGPU_MSG_FUNCTION_RESERVED_57: _bindgen_ty_2 = 57;
-pub const NV_VGPU_MSG_FUNCTION_RESET_CURRENT_GR_CONTEXT: _bindgen_ty_2 = 58;
-pub const NV_VGPU_MSG_FUNCTION_SET_SEMA_MEM_VALIDATION_STATE: _bindgen_ty_2 = 59;
-pub const NV_VGPU_MSG_FUNCTION_GET_ENGINE_UTILIZATION: _bindgen_ty_2 = 60;
-pub const NV_VGPU_MSG_FUNCTION_UPDATE_GPU_PDES: _bindgen_ty_2 = 61;
-pub const NV_VGPU_MSG_FUNCTION_GET_ENCODER_CAPACITY: _bindgen_ty_2 = 62;
-pub const NV_VGPU_MSG_FUNCTION_VGPU_PF_REG_READ32: _bindgen_ty_2 = 63;
-pub const NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO_EXT: _bindgen_ty_2 = 64;
-pub const NV_VGPU_MSG_FUNCTION_GET_GSP_STATIC_INFO: _bindgen_ty_2 = 65;
-pub const NV_VGPU_MSG_FUNCTION_RMFS_INIT: _bindgen_ty_2 = 66;
-pub const NV_VGPU_MSG_FUNCTION_RMFS_CLOSE_QUEUE: _bindgen_ty_2 = 67;
-pub const NV_VGPU_MSG_FUNCTION_RMFS_CLEANUP: _bindgen_ty_2 = 68;
-pub const NV_VGPU_MSG_FUNCTION_RMFS_TEST: _bindgen_ty_2 = 69;
-pub const NV_VGPU_MSG_FUNCTION_UPDATE_BAR_PDE: _bindgen_ty_2 = 70;
-pub const NV_VGPU_MSG_FUNCTION_CONTINUATION_RECORD: _bindgen_ty_2 = 71;
-pub const NV_VGPU_MSG_FUNCTION_GSP_SET_SYSTEM_INFO: _bindgen_ty_2 = 72;
-pub const NV_VGPU_MSG_FUNCTION_SET_REGISTRY: _bindgen_ty_2 = 73;
-pub const NV_VGPU_MSG_FUNCTION_GSP_INIT_POST_OBJGPU: _bindgen_ty_2 = 74;
-pub const NV_VGPU_MSG_FUNCTION_SUBDEV_EVENT_SET_NOTIFICATION: _bindgen_ty_2 = 75;
-pub const NV_VGPU_MSG_FUNCTION_GSP_RM_CONTROL: _bindgen_ty_2 = 76;
-pub const NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO2: _bindgen_ty_2 = 77;
-pub const NV_VGPU_MSG_FUNCTION_DUMP_PROTOBUF_COMPONENT: _bindgen_ty_2 = 78;
-pub const NV_VGPU_MSG_FUNCTION_UNSET_PAGE_DIRECTORY: _bindgen_ty_2 = 79;
-pub const NV_VGPU_MSG_FUNCTION_GET_CONSOLIDATED_STATIC_INFO: _bindgen_ty_2 = 80;
-pub const NV_VGPU_MSG_FUNCTION_GMMU_REGISTER_FAULT_BUFFER: _bindgen_ty_2 = 81;
-pub const NV_VGPU_MSG_FUNCTION_GMMU_UNREGISTER_FAULT_BUFFER: _bindgen_ty_2 = 82;
-pub const NV_VGPU_MSG_FUNCTION_GMMU_REGISTER_CLIENT_SHADOW_FAULT_BUFFER: _bindgen_ty_2 = 83;
-pub const NV_VGPU_MSG_FUNCTION_GMMU_UNREGISTER_CLIENT_SHADOW_FAULT_BUFFER: _bindgen_ty_2 = 84;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_VGPU_FB_USAGE: _bindgen_ty_2 = 85;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_NVFBC_SW_SESSION_UPDATE_INFO: _bindgen_ty_2 = 86;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_NVENC_SW_SESSION_UPDATE_INFO: _bindgen_ty_2 = 87;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_RESET_CHANNEL: _bindgen_ty_2 = 88;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_RESET_ISOLATED_CHANNEL: _bindgen_ty_2 = 89;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_HANDLE_VF_PRI_FAULT: _bindgen_ty_2 = 90;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_CLK_GET_EXTENDED_INFO: _bindgen_ty_2 = 91;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_PERF_BOOST: _bindgen_ty_2 = 92;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_PERF_VPSTATES_GET_CONTROL: _bindgen_ty_2 = 93;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_ZBC_CLEAR_TABLE: _bindgen_ty_2 = 94;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_ZBC_COLOR_CLEAR: _bindgen_ty_2 = 95;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_ZBC_DEPTH_CLEAR: _bindgen_ty_2 = 96;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GPFIFO_SCHEDULE: _bindgen_ty_2 = 97;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_TIMESLICE: _bindgen_ty_2 = 98;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_PREEMPT: _bindgen_ty_2 = 99;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_FIFO_DISABLE_CHANNELS: _bindgen_ty_2 = 100;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_TSG_INTERLEAVE_LEVEL: _bindgen_ty_2 = 101;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_CHANNEL_INTERLEAVE_LEVEL: _bindgen_ty_2 = 102;
-pub const NV_VGPU_MSG_FUNCTION_GSP_RM_ALLOC: _bindgen_ty_2 = 103;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_P2P_CAPS_V2: _bindgen_ty_2 = 104;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_CIPHER_AES_ENCRYPT: _bindgen_ty_2 = 105;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_CIPHER_SESSION_KEY: _bindgen_ty_2 = 106;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_CIPHER_SESSION_KEY_STATUS: _bindgen_ty_2 = 107;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_CLEAR_ALL_SM_ERROR_STATES: _bindgen_ty_2 = 108;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_READ_ALL_SM_ERROR_STATES: _bindgen_ty_2 = 109;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SET_EXCEPTION_MASK: _bindgen_ty_2 = 110;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_PROMOTE_CTX: _bindgen_ty_2 = 111;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_CTXSW_PREEMPTION_BIND: _bindgen_ty_2 = 112;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_SET_CTXSW_PREEMPTION_MODE: _bindgen_ty_2 = 113;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_CTXSW_ZCULL_BIND: _bindgen_ty_2 = 114;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_INITIALIZE_CTX: _bindgen_ty_2 = 115;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_VASPACE_COPY_SERVER_RESERVED_PDES: _bindgen_ty_2 = 116;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_FIFO_CLEAR_FAULTED_BIT: _bindgen_ty_2 = 117;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_LATEST_ECC_ADDRESSES: _bindgen_ty_2 = 118;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_MC_SERVICE_INTERRUPTS: _bindgen_ty_2 = 119;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DMA_SET_DEFAULT_VASPACE: _bindgen_ty_2 = 120;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_CE_PCE_MASK: _bindgen_ty_2 = 121;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_ZBC_CLEAR_TABLE_ENTRY: _bindgen_ty_2 = 122;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_NVLINK_PEER_ID_MASK: _bindgen_ty_2 = 123;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_NVLINK_STATUS: _bindgen_ty_2 = 124;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_P2P_CAPS: _bindgen_ty_2 = 125;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_P2P_CAPS_MATRIX: _bindgen_ty_2 = 126;
-pub const NV_VGPU_MSG_FUNCTION_RESERVED_0: _bindgen_ty_2 = 127;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_RESERVE_PM_AREA_SMPC: _bindgen_ty_2 = 128;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_RESERVE_HWPM_LEGACY: _bindgen_ty_2 = 129;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_B0CC_EXEC_REG_OPS: _bindgen_ty_2 = 130;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_BIND_PM_RESOURCES: _bindgen_ty_2 = 131;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SUSPEND_CONTEXT: _bindgen_ty_2 = 132;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_RESUME_CONTEXT: _bindgen_ty_2 = 133;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_EXEC_REG_OPS: _bindgen_ty_2 = 134;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SET_MODE_MMU_DEBUG: _bindgen_ty_2 = 135;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_READ_SINGLE_SM_ERROR_STATE: _bindgen_ty_2 = 136;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_CLEAR_SINGLE_SM_ERROR_STATE: _bindgen_ty_2 = 137;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SET_MODE_ERRBAR_DEBUG: _bindgen_ty_2 = 138;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SET_NEXT_STOP_TRIGGER_TYPE: _bindgen_ty_2 = 139;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_ALLOC_PMA_STREAM: _bindgen_ty_2 = 140;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_PMA_STREAM_UPDATE_GET_PUT: _bindgen_ty_2 = 141;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_FB_GET_INFO_V2: _bindgen_ty_2 = 142;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_FIFO_SET_CHANNEL_PROPERTIES: _bindgen_ty_2 = 143;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_GET_CTX_BUFFER_INFO: _bindgen_ty_2 = 144;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_KGR_GET_CTX_BUFFER_PTES: _bindgen_ty_2 = 145;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_EVICT_CTX: _bindgen_ty_2 = 146;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_FB_GET_FS_INFO: _bindgen_ty_2 = 147;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GRMGR_GET_GR_FS_INFO: _bindgen_ty_2 = 148;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_STOP_CHANNEL: _bindgen_ty_2 = 149;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_PC_SAMPLING_MODE: _bindgen_ty_2 = 150;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_PERF_RATED_TDP_GET_STATUS: _bindgen_ty_2 = 151;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_PERF_RATED_TDP_SET_CONTROL: _bindgen_ty_2 = 152;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_FREE_PMA_STREAM: _bindgen_ty_2 = 153;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_TIMER_SET_GR_TICK_FREQ: _bindgen_ty_2 = 154;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_FIFO_SETUP_VF_ZOMBIE_SUBCTX_PDB: _bindgen_ty_2 = 155;
-pub const NV_VGPU_MSG_FUNCTION_GET_CONSOLIDATED_GR_STATIC_INFO: _bindgen_ty_2 = 156;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SET_SINGLE_SM_SINGLE_STEP: _bindgen_ty_2 = 157;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_GET_TPC_PARTITION_MODE: _bindgen_ty_2 = 158;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GR_SET_TPC_PARTITION_MODE: _bindgen_ty_2 = 159;
-pub const NV_VGPU_MSG_FUNCTION_UVM_PAGING_CHANNEL_ALLOCATE: _bindgen_ty_2 = 160;
-pub const NV_VGPU_MSG_FUNCTION_UVM_PAGING_CHANNEL_DESTROY: _bindgen_ty_2 = 161;
-pub const NV_VGPU_MSG_FUNCTION_UVM_PAGING_CHANNEL_MAP: _bindgen_ty_2 = 162;
-pub const NV_VGPU_MSG_FUNCTION_UVM_PAGING_CHANNEL_UNMAP: _bindgen_ty_2 = 163;
-pub const NV_VGPU_MSG_FUNCTION_UVM_PAGING_CHANNEL_PUSH_STREAM: _bindgen_ty_2 = 164;
-pub const NV_VGPU_MSG_FUNCTION_UVM_PAGING_CHANNEL_SET_HANDLES: _bindgen_ty_2 = 165;
-pub const NV_VGPU_MSG_FUNCTION_UVM_METHOD_STREAM_GUEST_PAGES_OPERATION: _bindgen_ty_2 = 166;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_INTERNAL_QUIESCE_PMA_CHANNEL: _bindgen_ty_2 = 167;
-pub const NV_VGPU_MSG_FUNCTION_DCE_RM_INIT: _bindgen_ty_2 = 168;
-pub const NV_VGPU_MSG_FUNCTION_REGISTER_VIRTUAL_EVENT_BUFFER: _bindgen_ty_2 = 169;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_EVENT_BUFFER_UPDATE_GET: _bindgen_ty_2 = 170;
-pub const NV_VGPU_MSG_FUNCTION_GET_PLCABLE_ADDRESS_KIND: _bindgen_ty_2 = 171;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_PERF_LIMITS_SET_STATUS_V2: _bindgen_ty_2 = 172;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_INTERNAL_SRIOV_PROMOTE_PMA_STREAM: _bindgen_ty_2 = 173;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_MMU_DEBUG_MODE: _bindgen_ty_2 = 174;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_INTERNAL_PROMOTE_FAULT_METHOD_BUFFERS: _bindgen_ty_2 = 175;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_FLCN_GET_CTX_BUFFER_SIZE: _bindgen_ty_2 = 176;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_FLCN_GET_CTX_BUFFER_INFO: _bindgen_ty_2 = 177;
-pub const NV_VGPU_MSG_FUNCTION_DISABLE_CHANNELS: _bindgen_ty_2 = 178;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_FABRIC_MEMORY_DESCRIBE: _bindgen_ty_2 = 179;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_FABRIC_MEM_STATS: _bindgen_ty_2 = 180;
-pub const NV_VGPU_MSG_FUNCTION_SAVE_HIBERNATION_DATA: _bindgen_ty_2 = 181;
-pub const NV_VGPU_MSG_FUNCTION_RESTORE_HIBERNATION_DATA: _bindgen_ty_2 = 182;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_INTERNAL_MEMSYS_SET_ZBC_REFERENCED: _bindgen_ty_2 = 183;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_EXEC_PARTITIONS_CREATE: _bindgen_ty_2 = 184;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_EXEC_PARTITIONS_DELETE: _bindgen_ty_2 = 185;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GPFIFO_GET_WORK_SUBMIT_TOKEN: _bindgen_ty_2 = 186;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GPFIFO_SET_WORK_SUBMIT_TOKEN_NOTIF_INDEX: _bindgen_ty_2 = 187;
-pub const NV_VGPU_MSG_FUNCTION_PMA_SCRUBBER_SHARED_BUFFER_GUEST_PAGES_OPERATION: _bindgen_ty_2 =
- 188;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_MASTER_GET_VIRTUAL_FUNCTION_ERROR_CONT_INTR_MASK:
- _bindgen_ty_2 = 189;
-pub const NV_VGPU_MSG_FUNCTION_SET_SYSMEM_DIRTY_PAGE_TRACKING_BUFFER: _bindgen_ty_2 = 190;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_SUBDEVICE_GET_P2P_CAPS: _bindgen_ty_2 = 191;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_BUS_SET_P2P_MAPPING: _bindgen_ty_2 = 192;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_BUS_UNSET_P2P_MAPPING: _bindgen_ty_2 = 193;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_FLA_SETUP_INSTANCE_MEM_BLOCK: _bindgen_ty_2 = 194;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_MIGRATABLE_OPS: _bindgen_ty_2 = 195;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_TOTAL_HS_CREDITS: _bindgen_ty_2 = 196;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GET_HS_CREDITS: _bindgen_ty_2 = 197;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_HS_CREDITS: _bindgen_ty_2 = 198;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_PM_AREA_PC_SAMPLER: _bindgen_ty_2 = 199;
-pub const NV_VGPU_MSG_FUNCTION_INVALIDATE_TLB: _bindgen_ty_2 = 200;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_QUERY_ECC_STATUS: _bindgen_ty_2 = 201;
-pub const NV_VGPU_MSG_FUNCTION_ECC_NOTIFIER_WRITE_ACK: _bindgen_ty_2 = 202;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_GET_MODE_MMU_DEBUG: _bindgen_ty_2 = 203;
-pub const NV_VGPU_MSG_FUNCTION_RM_API_CONTROL: _bindgen_ty_2 = 204;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_CMD_INTERNAL_GPU_START_FABRIC_PROBE: _bindgen_ty_2 = 205;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_NVLINK_GET_INBAND_RECEIVED_DATA: _bindgen_ty_2 = 206;
-pub const NV_VGPU_MSG_FUNCTION_GET_STATIC_DATA: _bindgen_ty_2 = 207;
-pub const NV_VGPU_MSG_FUNCTION_RESERVED_208: _bindgen_ty_2 = 208;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_GPU_GET_INFO_V2: _bindgen_ty_2 = 209;
-pub const NV_VGPU_MSG_FUNCTION_GET_BRAND_CAPS: _bindgen_ty_2 = 210;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_CMD_NVLINK_INBAND_SEND_DATA: _bindgen_ty_2 = 211;
-pub const NV_VGPU_MSG_FUNCTION_UPDATE_GPM_GUEST_BUFFER_INFO: _bindgen_ty_2 = 212;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_CMD_INTERNAL_CONTROL_GSP_TRACE: _bindgen_ty_2 = 213;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_SET_ZBC_STENCIL_CLEAR: _bindgen_ty_2 = 214;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_SUBDEVICE_GET_VGPU_HEAP_STATS: _bindgen_ty_2 = 215;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_SUBDEVICE_GET_LIBOS_HEAP_STATS: _bindgen_ty_2 = 216;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_SET_MODE_MMU_GCC_DEBUG: _bindgen_ty_2 = 217;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_DBG_GET_MODE_MMU_GCC_DEBUG: _bindgen_ty_2 = 218;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_RESERVE_HES: _bindgen_ty_2 = 219;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_RELEASE_HES: _bindgen_ty_2 = 220;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_RESERVE_CCU_PROF: _bindgen_ty_2 = 221;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_RELEASE_CCU_PROF: _bindgen_ty_2 = 222;
-pub const NV_VGPU_MSG_FUNCTION_RESERVED: _bindgen_ty_2 = 223;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_CMD_GET_CHIPLET_HS_CREDIT_POOL: _bindgen_ty_2 = 224;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_CMD_GET_HS_CREDITS_MAPPING: _bindgen_ty_2 = 225;
-pub const NV_VGPU_MSG_FUNCTION_CTRL_EXEC_PARTITIONS_EXPORT: _bindgen_ty_2 = 226;
-pub const NV_VGPU_MSG_FUNCTION_NUM_FUNCTIONS: _bindgen_ty_2 = 227;
-pub type _bindgen_ty_2 = ffi::c_uint;
-pub const NV_VGPU_MSG_EVENT_FIRST_EVENT: _bindgen_ty_3 = 4096;
-pub const NV_VGPU_MSG_EVENT_GSP_INIT_DONE: _bindgen_ty_3 = 4097;
-pub const NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER: _bindgen_ty_3 = 4098;
-pub const NV_VGPU_MSG_EVENT_POST_EVENT: _bindgen_ty_3 = 4099;
-pub const NV_VGPU_MSG_EVENT_RC_TRIGGERED: _bindgen_ty_3 = 4100;
-pub const NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED: _bindgen_ty_3 = 4101;
-pub const NV_VGPU_MSG_EVENT_OS_ERROR_LOG: _bindgen_ty_3 = 4102;
-pub const NV_VGPU_MSG_EVENT_RG_LINE_INTR: _bindgen_ty_3 = 4103;
-pub const NV_VGPU_MSG_EVENT_GPUACCT_PERFMON_UTIL_SAMPLES: _bindgen_ty_3 = 4104;
-pub const NV_VGPU_MSG_EVENT_SIM_READ: _bindgen_ty_3 = 4105;
-pub const NV_VGPU_MSG_EVENT_SIM_WRITE: _bindgen_ty_3 = 4106;
-pub const NV_VGPU_MSG_EVENT_SEMAPHORE_SCHEDULE_CALLBACK: _bindgen_ty_3 = 4107;
-pub const NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT: _bindgen_ty_3 = 4108;
-pub const NV_VGPU_MSG_EVENT_VGPU_GSP_PLUGIN_TRIGGERED: _bindgen_ty_3 = 4109;
-pub const NV_VGPU_MSG_EVENT_PERF_GPU_BOOST_SYNC_LIMITS_CALLBACK: _bindgen_ty_3 = 4110;
-pub const NV_VGPU_MSG_EVENT_PERF_BRIDGELESS_INFO_UPDATE: _bindgen_ty_3 = 4111;
-pub const NV_VGPU_MSG_EVENT_VGPU_CONFIG: _bindgen_ty_3 = 4112;
-pub const NV_VGPU_MSG_EVENT_DISPLAY_MODESET: _bindgen_ty_3 = 4113;
-pub const NV_VGPU_MSG_EVENT_EXTDEV_INTR_SERVICE: _bindgen_ty_3 = 4114;
-pub const NV_VGPU_MSG_EVENT_NVLINK_INBAND_RECEIVED_DATA_256: _bindgen_ty_3 = 4115;
-pub const NV_VGPU_MSG_EVENT_NVLINK_INBAND_RECEIVED_DATA_512: _bindgen_ty_3 = 4116;
-pub const NV_VGPU_MSG_EVENT_NVLINK_INBAND_RECEIVED_DATA_1024: _bindgen_ty_3 = 4117;
-pub const NV_VGPU_MSG_EVENT_NVLINK_INBAND_RECEIVED_DATA_2048: _bindgen_ty_3 = 4118;
-pub const NV_VGPU_MSG_EVENT_NVLINK_INBAND_RECEIVED_DATA_4096: _bindgen_ty_3 = 4119;
-pub const NV_VGPU_MSG_EVENT_TIMED_SEMAPHORE_RELEASE: _bindgen_ty_3 = 4120;
-pub const NV_VGPU_MSG_EVENT_NVLINK_IS_GPU_DEGRADED: _bindgen_ty_3 = 4121;
-pub const NV_VGPU_MSG_EVENT_PFM_REQ_HNDLR_STATE_SYNC_CALLBACK: _bindgen_ty_3 = 4122;
-pub const NV_VGPU_MSG_EVENT_NVLINK_FAULT_UP: _bindgen_ty_3 = 4123;
-pub const NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE: _bindgen_ty_3 = 4124;
-pub const NV_VGPU_MSG_EVENT_MIG_CI_CONFIG_UPDATE: _bindgen_ty_3 = 4125;
-pub const NV_VGPU_MSG_EVENT_UPDATE_GSP_TRACE: _bindgen_ty_3 = 4126;
-pub const NV_VGPU_MSG_EVENT_NVLINK_FATAL_ERROR_RECOVERY: _bindgen_ty_3 = 4127;
-pub const NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD: _bindgen_ty_3 = 4128;
-pub const NV_VGPU_MSG_EVENT_FECS_ERROR: _bindgen_ty_3 = 4129;
-pub const NV_VGPU_MSG_EVENT_RECOVERY_ACTION: _bindgen_ty_3 = 4130;
-pub const NV_VGPU_MSG_EVENT_NUM_EVENTS: _bindgen_ty_3 = 4131;
-pub type _bindgen_ty_3 = ffi::c_uint;
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct NV0080_CTRL_GPU_GET_SRIOV_CAPS_PARAMS {
- pub totalVFs: u32_,
- pub firstVfOffset: u32_,
- pub vfFeatureMask: u32_,
- pub __bindgen_padding_0: [u8; 4usize],
- pub FirstVFBar0Address: u64_,
- pub FirstVFBar1Address: u64_,
- pub FirstVFBar2Address: u64_,
- pub bar0Size: u64_,
- pub bar1Size: u64_,
- pub bar2Size: u64_,
- pub b64bitBar0: u8_,
- pub b64bitBar1: u8_,
- pub b64bitBar2: u8_,
- pub bSriovEnabled: u8_,
- pub bSriovHeavyEnabled: u8_,
- pub bEmulateVFBar0TlbInvalidationRegister: u8_,
- pub bClientRmAllocatedCtxBuffer: u8_,
- pub bNonPowerOf2ChannelCountSupported: u8_,
- pub bVfResizableBAR1Supported: u8_,
- pub __bindgen_padding_1: [u8; 7usize],
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct NV2080_CTRL_BIOS_GET_SKU_INFO_PARAMS {
- pub BoardID: u32_,
- pub chipSKU: [ffi::c_char; 9usize],
- pub chipSKUMod: [ffi::c_char; 5usize],
- pub __bindgen_padding_0: [u8; 2usize],
- pub skuConfigVersion: u32_,
- pub project: [ffi::c_char; 5usize],
- pub projectSKU: [ffi::c_char; 5usize],
- pub CDP: [ffi::c_char; 6usize],
- pub projectSKUMod: [ffi::c_char; 2usize],
- pub __bindgen_padding_1: [u8; 2usize],
- pub businessCycle: u32_,
-}
-pub type NV2080_CTRL_CMD_FB_GET_FB_REGION_SURFACE_MEM_TYPE_FLAG = [u8_; 17usize];
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct NV2080_CTRL_CMD_FB_GET_FB_REGION_FB_REGION_INFO {
- pub base: u64_,
- pub limit: u64_,
- pub reserved: u64_,
- pub performance: u32_,
- pub supportCompressed: u8_,
- pub supportISO: u8_,
- pub bProtected: u8_,
- pub blackList: NV2080_CTRL_CMD_FB_GET_FB_REGION_SURFACE_MEM_TYPE_FLAG,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct NV2080_CTRL_CMD_FB_GET_FB_REGION_INFO_PARAMS {
- pub numFBRegions: u32_,
- pub __bindgen_padding_0: [u8; 4usize],
- pub fbRegion: [NV2080_CTRL_CMD_FB_GET_FB_REGION_FB_REGION_INFO; 16usize],
-}
-#[repr(C)]
-#[derive(Debug, Copy, Clone, MaybeZeroable)]
-pub struct NV2080_CTRL_GPU_GET_GID_INFO_PARAMS {
- pub index: u32_,
- pub flags: u32_,
- pub length: u32_,
- pub data: [u8_; 256usize],
-}
-impl Default for NV2080_CTRL_GPU_GET_GID_INFO_PARAMS {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct DOD_METHOD_DATA {
- pub status: u32_,
- pub acpiIdListLen: u32_,
- pub acpiIdList: [u32_; 16usize],
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct JT_METHOD_DATA {
- pub status: u32_,
- pub jtCaps: u32_,
- pub jtRevId: u16_,
- pub bSBIOSCaps: u8_,
- pub __bindgen_padding_0: u8,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct MUX_METHOD_DATA_ELEMENT {
- pub acpiId: u32_,
- pub mode: u32_,
- pub status: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct MUX_METHOD_DATA {
- pub tableLen: u32_,
- pub acpiIdMuxModeTable: [MUX_METHOD_DATA_ELEMENT; 16usize],
- pub acpiIdMuxPartTable: [MUX_METHOD_DATA_ELEMENT; 16usize],
- pub acpiIdMuxStateTable: [MUX_METHOD_DATA_ELEMENT; 16usize],
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct CAPS_METHOD_DATA {
- pub status: u32_,
- pub optimusCaps: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct ACPI_METHOD_DATA {
- pub bValid: u8_,
- pub __bindgen_padding_0: [u8; 3usize],
- pub dodMethodData: DOD_METHOD_DATA,
- pub jtMethodData: JT_METHOD_DATA,
- pub muxMethodData: MUX_METHOD_DATA,
- pub capsMethodData: CAPS_METHOD_DATA,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct VIRTUAL_DISPLAY_GET_MAX_RESOLUTION_PARAMS {
- pub headIndex: u32_,
- pub maxHResolution: u32_,
- pub maxVResolution: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct VIRTUAL_DISPLAY_GET_NUM_HEADS_PARAMS {
- pub numHeads: u32_,
- pub maxNumHeads: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct BUSINFO {
- pub deviceID: u16_,
- pub vendorID: u16_,
- pub subdeviceID: u16_,
- pub subvendorID: u16_,
- pub revisionID: u8_,
- pub __bindgen_padding_0: u8,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GSP_VF_INFO {
- pub totalVFs: u32_,
- pub firstVFOffset: u32_,
- pub FirstVFBar0Address: u64_,
- pub FirstVFBar1Address: u64_,
- pub FirstVFBar2Address: u64_,
- pub b64bitBar0: u8_,
- pub b64bitBar1: u8_,
- pub b64bitBar2: u8_,
- pub __bindgen_padding_0: [u8; 5usize],
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GSP_PCIE_CONFIG_REG {
- pub linkCap: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct EcidManufacturingInfo {
- pub ecidLow: u32_,
- pub ecidHigh: u32_,
- pub ecidExtended: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct FW_WPR_LAYOUT_OFFSET {
- pub nonWprHeapOffset: u64_,
- pub frtsOffset: u64_,
-}
-#[repr(C)]
-#[derive(Debug, Copy, Clone, MaybeZeroable)]
-pub struct GspStaticConfigInfo_t {
- pub grCapsBits: [u8_; 23usize],
- pub __bindgen_padding_0: u8,
- pub gidInfo: NV2080_CTRL_GPU_GET_GID_INFO_PARAMS,
- pub SKUInfo: NV2080_CTRL_BIOS_GET_SKU_INFO_PARAMS,
- pub __bindgen_padding_1: [u8; 4usize],
- pub fbRegionInfoParams: NV2080_CTRL_CMD_FB_GET_FB_REGION_INFO_PARAMS,
- pub sriovCaps: NV0080_CTRL_GPU_GET_SRIOV_CAPS_PARAMS,
- pub sriovMaxGfid: u32_,
- pub engineCaps: [u32_; 3usize],
- pub poisonFuseEnabled: u8_,
- pub __bindgen_padding_2: [u8; 7usize],
- pub fb_length: u64_,
- pub fbio_mask: u64_,
- pub fb_bus_width: u32_,
- pub fb_ram_type: u32_,
- pub fbp_mask: u64_,
- pub l2_cache_size: u32_,
- pub gpuNameString: [u8_; 64usize],
- pub gpuShortNameString: [u8_; 64usize],
- pub gpuNameString_Unicode: [u16_; 64usize],
- pub bGpuInternalSku: u8_,
- pub bIsQuadroGeneric: u8_,
- pub bIsQuadroAd: u8_,
- pub bIsNvidiaNvs: u8_,
- pub bIsVgx: u8_,
- pub bGeforceSmb: u8_,
- pub bIsTitan: u8_,
- pub bIsTesla: u8_,
- pub bIsMobile: u8_,
- pub bIsGc6Rtd3Allowed: u8_,
- pub bIsGc8Rtd3Allowed: u8_,
- pub bIsGcOffRtd3Allowed: u8_,
- pub bIsGcoffLegacyAllowed: u8_,
- pub bIsMigSupported: u8_,
- pub RTD3GC6TotalBoardPower: u16_,
- pub RTD3GC6PerstDelay: u16_,
- pub __bindgen_padding_3: [u8; 2usize],
- pub bar1PdeBase: u64_,
- pub bar2PdeBase: u64_,
- pub bVbiosValid: u8_,
- pub __bindgen_padding_4: [u8; 3usize],
- pub vbiosSubVendor: u32_,
- pub vbiosSubDevice: u32_,
- pub bPageRetirementSupported: u8_,
- pub bSplitVasBetweenServerClientRm: u8_,
- pub bClRootportNeedsNosnoopWAR: u8_,
- pub __bindgen_padding_5: u8,
- pub displaylessMaxHeads: VIRTUAL_DISPLAY_GET_NUM_HEADS_PARAMS,
- pub displaylessMaxResolution: VIRTUAL_DISPLAY_GET_MAX_RESOLUTION_PARAMS,
- pub __bindgen_padding_6: [u8; 4usize],
- pub displaylessMaxPixels: u64_,
- pub hInternalClient: u32_,
- pub hInternalDevice: u32_,
- pub hInternalSubdevice: u32_,
- pub bSelfHostedMode: u8_,
- pub bAtsSupported: u8_,
- pub bIsGpuUefi: u8_,
- pub bIsEfiInit: u8_,
- pub ecidInfo: [EcidManufacturingInfo; 2usize],
- pub fwWprLayoutOffset: FW_WPR_LAYOUT_OFFSET,
-}
-impl Default for GspStaticConfigInfo_t {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GspSystemInfo {
- pub gpuPhysAddr: u64_,
- pub gpuPhysFbAddr: u64_,
- pub gpuPhysInstAddr: u64_,
- pub gpuPhysIoAddr: u64_,
- pub nvDomainBusDeviceFunc: u64_,
- pub simAccessBufPhysAddr: u64_,
- pub notifyOpSharedSurfacePhysAddr: u64_,
- pub pcieAtomicsOpMask: u64_,
- pub consoleMemSize: u64_,
- pub maxUserVa: u64_,
- pub pciConfigMirrorBase: u32_,
- pub pciConfigMirrorSize: u32_,
- pub PCIDeviceID: u32_,
- pub PCISubDeviceID: u32_,
- pub PCIRevisionID: u32_,
- pub pcieAtomicsCplDeviceCapMask: u32_,
- pub oorArch: u8_,
- pub __bindgen_padding_0: [u8; 7usize],
- pub clPdbProperties: u64_,
- pub Chipset: u32_,
- pub bGpuBehindBridge: u8_,
- pub bFlrSupported: u8_,
- pub b64bBar0Supported: u8_,
- pub bMnocAvailable: u8_,
- pub chipsetL1ssEnable: u32_,
- pub bUpstreamL0sUnsupported: u8_,
- pub bUpstreamL1Unsupported: u8_,
- pub bUpstreamL1PorSupported: u8_,
- pub bUpstreamL1PorMobileOnly: u8_,
- pub bSystemHasMux: u8_,
- pub upstreamAddressValid: u8_,
- pub FHBBusInfo: BUSINFO,
- pub chipsetIDInfo: BUSINFO,
- pub __bindgen_padding_1: [u8; 2usize],
- pub acpiMethodData: ACPI_METHOD_DATA,
- pub hypervisorType: u32_,
- pub bIsPassthru: u8_,
- pub __bindgen_padding_2: [u8; 7usize],
- pub sysTimerOffsetNs: u64_,
- pub gspVFInfo: GSP_VF_INFO,
- pub bIsPrimary: u8_,
- pub isGridBuild: u8_,
- pub __bindgen_padding_3: [u8; 2usize],
- pub pcieConfigReg: GSP_PCIE_CONFIG_REG,
- pub gridBuildCsp: u32_,
- pub bPreserveVideoMemoryAllocations: u8_,
- pub bTdrEventSupported: u8_,
- pub bFeatureStretchVblankCapable: u8_,
- pub bEnableDynamicGranularityPageArrays: u8_,
- pub bClockBoostSupported: u8_,
- pub bRouteDispIntrsToCPU: u8_,
- pub __bindgen_padding_4: [u8; 6usize],
- pub hostPageSize: u64_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct MESSAGE_QUEUE_INIT_ARGUMENTS {
- pub sharedMemPhysAddr: u64_,
- pub pageTableEntryCount: u32_,
- pub __bindgen_padding_0: [u8; 4usize],
- pub cmdQueueOffset: u64_,
- pub statQueueOffset: u64_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GSP_SR_INIT_ARGUMENTS {
- pub oldLevel: u32_,
- pub flags: u32_,
- pub bInPMTransition: u8_,
- pub __bindgen_padding_0: [u8; 3usize],
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GSP_ARGUMENTS_CACHED {
- pub messageQueueInitArguments: MESSAGE_QUEUE_INIT_ARGUMENTS,
- pub srInitArguments: GSP_SR_INIT_ARGUMENTS,
- pub gpuInstance: u32_,
- pub bDmemStack: u8_,
- pub __bindgen_padding_0: [u8; 7usize],
- pub profilerArgs: GSP_ARGUMENTS_CACHED__bindgen_ty_1,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GSP_ARGUMENTS_CACHED__bindgen_ty_1 {
- pub pa: u64_,
- pub size: u64_,
-}
-#[repr(C)]
-#[derive(Copy, Clone, MaybeZeroable)]
-pub union rpc_message_rpc_union_field_v03_00 {
- pub spare: u32_,
- pub cpuRmGfid: u32_,
-}
-impl Default for rpc_message_rpc_union_field_v03_00 {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-pub type rpc_message_rpc_union_field_v = rpc_message_rpc_union_field_v03_00;
-#[repr(C)]
-#[derive(MaybeZeroable)]
-pub struct rpc_message_header_v03_00 {
- pub header_version: u32_,
- pub signature: u32_,
- pub length: u32_,
- pub function: u32_,
- pub rpc_result: u32_,
- pub rpc_result_private: u32_,
- pub sequence: u32_,
- pub u: rpc_message_rpc_union_field_v,
- pub rpc_message_data: __IncompleteArrayField<u8_>,
-}
-impl Default for rpc_message_header_v03_00 {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-pub type rpc_message_header_v = rpc_message_header_v03_00;
-#[repr(C)]
-#[derive(Copy, Clone, MaybeZeroable)]
-pub struct GspFwWprMeta {
- pub magic: u64_,
- pub revision: u64_,
- pub sysmemAddrOfRadix3Elf: u64_,
- pub sizeOfRadix3Elf: u64_,
- pub sysmemAddrOfBootloader: u64_,
- pub sizeOfBootloader: u64_,
- pub bootloaderCodeOffset: u64_,
- pub bootloaderDataOffset: u64_,
- pub bootloaderManifestOffset: u64_,
- pub __bindgen_anon_1: GspFwWprMeta__bindgen_ty_1,
- pub gspFwRsvdStart: u64_,
- pub nonWprHeapOffset: u64_,
- pub nonWprHeapSize: u64_,
- pub gspFwWprStart: u64_,
- pub gspFwHeapOffset: u64_,
- pub gspFwHeapSize: u64_,
- pub gspFwOffset: u64_,
- pub bootBinOffset: u64_,
- pub frtsOffset: u64_,
- pub frtsSize: u64_,
- pub gspFwWprEnd: u64_,
- pub fbSize: u64_,
- pub vgaWorkspaceOffset: u64_,
- pub vgaWorkspaceSize: u64_,
- pub bootCount: u64_,
- pub __bindgen_anon_2: GspFwWprMeta__bindgen_ty_2,
- pub gspFwHeapVfPartitionCount: u8_,
- pub flags: u8_,
- pub padding: [u8_; 2usize],
- pub pmuReservedSize: u32_,
- pub verified: u64_,
-}
-#[repr(C)]
-#[derive(Copy, Clone, MaybeZeroable)]
-pub union GspFwWprMeta__bindgen_ty_1 {
- pub __bindgen_anon_1: GspFwWprMeta__bindgen_ty_1__bindgen_ty_1,
- pub __bindgen_anon_2: GspFwWprMeta__bindgen_ty_1__bindgen_ty_2,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GspFwWprMeta__bindgen_ty_1__bindgen_ty_1 {
- pub sysmemAddrOfSignature: u64_,
- pub sizeOfSignature: u64_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GspFwWprMeta__bindgen_ty_1__bindgen_ty_2 {
- pub gspFwHeapFreeListWprOffset: u32_,
- pub unused0: u32_,
- pub unused1: u64_,
-}
-impl Default for GspFwWprMeta__bindgen_ty_1 {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-#[repr(C)]
-#[derive(Copy, Clone, MaybeZeroable)]
-pub union GspFwWprMeta__bindgen_ty_2 {
- pub __bindgen_anon_1: GspFwWprMeta__bindgen_ty_2__bindgen_ty_1,
- pub __bindgen_anon_2: GspFwWprMeta__bindgen_ty_2__bindgen_ty_2,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GspFwWprMeta__bindgen_ty_2__bindgen_ty_1 {
- pub partitionRpcAddr: u64_,
- pub partitionRpcRequestOffset: u16_,
- pub partitionRpcReplyOffset: u16_,
- pub elfCodeOffset: u32_,
- pub elfDataOffset: u32_,
- pub elfCodeSize: u32_,
- pub elfDataSize: u32_,
- pub lsUcodeVersion: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GspFwWprMeta__bindgen_ty_2__bindgen_ty_2 {
- pub partitionRpcPadding: [u32_; 4usize],
- pub sysmemAddrOfCrashReportQueue: u64_,
- pub sizeOfCrashReportQueue: u32_,
- pub lsUcodeVersionPadding: [u32_; 1usize],
-}
-impl Default for GspFwWprMeta__bindgen_ty_2 {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-impl Default for GspFwWprMeta {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-pub type LibosAddress = u64_;
-pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_NONE: LibosMemoryRegionKind = 0;
-pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_CONTIGUOUS: LibosMemoryRegionKind = 1;
-pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_RADIX3: LibosMemoryRegionKind = 2;
-pub type LibosMemoryRegionKind = ffi::c_uint;
-pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_NONE: LibosMemoryRegionLoc = 0;
-pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_SYSMEM: LibosMemoryRegionLoc = 1;
-pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_FB: LibosMemoryRegionLoc = 2;
-pub type LibosMemoryRegionLoc = ffi::c_uint;
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct LibosMemoryRegionInitArgument {
- pub id8: LibosAddress,
- pub pa: LibosAddress,
- pub size: LibosAddress,
- pub kind: u8_,
- pub loc: u8_,
- pub __bindgen_padding_0: [u8; 6usize],
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct PACKED_REGISTRY_ENTRY {
- pub nameOffset: u32_,
- pub type_: u8_,
- pub __bindgen_padding_0: [u8; 3usize],
- pub data: u32_,
- pub length: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, MaybeZeroable)]
-pub struct PACKED_REGISTRY_TABLE {
- pub size: u32_,
- pub numEntries: u32_,
- pub entries: __IncompleteArrayField<PACKED_REGISTRY_ENTRY>,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct msgqTxHeader {
- pub version: u32_,
- pub size: u32_,
- pub msgSize: u32_,
- pub msgCount: u32_,
- pub writePtr: u32_,
- pub flags: u32_,
- pub rxHdrOff: u32_,
- pub entryOff: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct msgqRxHeader {
- pub readPtr: u32_,
-}
-#[repr(C)]
-#[repr(align(8))]
-#[derive(MaybeZeroable)]
-pub struct GSP_MSG_QUEUE_ELEMENT {
- pub authTagBuffer: [u8_; 16usize],
- pub aadBuffer: [u8_; 16usize],
- pub checkSum: u32_,
- pub seqNum: u32_,
- pub elemCount: u32_,
- pub __bindgen_padding_0: [u8; 4usize],
- pub rpc: rpc_message_header_v,
-}
-impl Default for GSP_MSG_QUEUE_ELEMENT {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-pub const GSP_DMA_TARGET_GSP_DMA_TARGET_LOCAL_FB: GSP_DMA_TARGET = 0;
-pub const GSP_DMA_TARGET_GSP_DMA_TARGET_COHERENT_SYSTEM: GSP_DMA_TARGET = 1;
-pub const GSP_DMA_TARGET_GSP_DMA_TARGET_NONCOHERENT_SYSTEM: GSP_DMA_TARGET = 2;
-pub const GSP_DMA_TARGET_GSP_DMA_TARGET_COUNT: GSP_DMA_TARGET = 3;
-pub type GSP_DMA_TARGET = ffi::c_uint;
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GSP_FMC_INIT_PARAMS {
- pub regkeys: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Copy, Clone, MaybeZeroable)]
-pub struct GSP_ACR_BOOT_GSP_RM_PARAMS {
- pub target: GSP_DMA_TARGET,
- pub gspRmDescSize: u32_,
- pub gspRmDescOffset: u64_,
- pub wprCarveoutOffset: u64_,
- pub wprCarveoutSize: u32_,
- pub bIsGspRmBoot: u8_,
- pub __bindgen_padding_0: [u8; 3usize],
-}
-impl Default for GSP_ACR_BOOT_GSP_RM_PARAMS {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-#[repr(C)]
-#[derive(Debug, Copy, Clone, MaybeZeroable)]
-pub struct GSP_RM_PARAMS {
- pub target: GSP_DMA_TARGET,
- pub __bindgen_padding_0: [u8; 4usize],
- pub bootArgsOffset: u64_,
-}
-impl Default for GSP_RM_PARAMS {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-#[repr(C)]
-#[derive(Debug, Copy, Clone, MaybeZeroable)]
-pub struct GSP_SPDM_PARAMS {
- pub target: GSP_DMA_TARGET,
- pub __bindgen_padding_0: [u8; 4usize],
- pub payloadBufferOffset: u64_,
- pub payloadBufferSize: u32_,
- pub __bindgen_padding_1: [u8; 4usize],
-}
-impl Default for GSP_SPDM_PARAMS {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-#[repr(C)]
-#[derive(Debug, Copy, Clone, MaybeZeroable)]
-pub struct GSP_FMC_BOOT_PARAMS {
- pub initParams: GSP_FMC_INIT_PARAMS,
- pub __bindgen_padding_0: [u8; 4usize],
- pub bootGspRmParams: GSP_ACR_BOOT_GSP_RM_PARAMS,
- pub gspRmParams: GSP_RM_PARAMS,
- pub gspSpdmParams: GSP_SPDM_PARAMS,
-}
-impl Default for GSP_FMC_BOOT_PARAMS {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct rpc_unloading_guest_driver_v1F_07 {
- pub bInPMTransition: u8_,
- pub bGc6Entering: u8_,
- pub __bindgen_padding_0: [u8; 2usize],
- pub newLevel: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, MaybeZeroable)]
-pub struct rpc_run_cpu_sequencer_v17_00 {
- pub bufferSizeDWord: u32_,
- pub cmdIndex: u32_,
- pub regSaveArea: [u32_; 8usize],
- pub commandBuffer: __IncompleteArrayField<u32_>,
-}
-pub const GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_REG_WRITE: GSP_SEQ_BUF_OPCODE = 0;
-pub const GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_REG_MODIFY: GSP_SEQ_BUF_OPCODE = 1;
-pub const GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_REG_POLL: GSP_SEQ_BUF_OPCODE = 2;
-pub const GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_DELAY_US: GSP_SEQ_BUF_OPCODE = 3;
-pub const GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_REG_STORE: GSP_SEQ_BUF_OPCODE = 4;
-pub const GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_CORE_RESET: GSP_SEQ_BUF_OPCODE = 5;
-pub const GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_CORE_START: GSP_SEQ_BUF_OPCODE = 6;
-pub const GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_CORE_WAIT_FOR_HALT: GSP_SEQ_BUF_OPCODE = 7;
-pub const GSP_SEQ_BUF_OPCODE_GSP_SEQ_BUF_OPCODE_CORE_RESUME: GSP_SEQ_BUF_OPCODE = 8;
-pub type GSP_SEQ_BUF_OPCODE = ffi::c_uint;
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GSP_SEQ_BUF_PAYLOAD_REG_WRITE {
- pub addr: u32_,
- pub val: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GSP_SEQ_BUF_PAYLOAD_REG_MODIFY {
- pub addr: u32_,
- pub mask: u32_,
- pub val: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GSP_SEQ_BUF_PAYLOAD_REG_POLL {
- pub addr: u32_,
- pub mask: u32_,
- pub val: u32_,
- pub timeout: u32_,
- pub error: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GSP_SEQ_BUF_PAYLOAD_DELAY_US {
- pub val: u32_,
-}
-#[repr(C)]
-#[derive(Debug, Default, Copy, Clone, MaybeZeroable)]
-pub struct GSP_SEQ_BUF_PAYLOAD_REG_STORE {
- pub addr: u32_,
- pub index: u32_,
-}
-#[repr(C)]
-#[derive(Copy, Clone, MaybeZeroable)]
-pub struct GSP_SEQUENCER_BUFFER_CMD {
- pub opCode: GSP_SEQ_BUF_OPCODE,
- pub payload: GSP_SEQUENCER_BUFFER_CMD__bindgen_ty_1,
-}
-#[repr(C)]
-#[derive(Copy, Clone, MaybeZeroable)]
-pub union GSP_SEQUENCER_BUFFER_CMD__bindgen_ty_1 {
- pub regWrite: GSP_SEQ_BUF_PAYLOAD_REG_WRITE,
- pub regModify: GSP_SEQ_BUF_PAYLOAD_REG_MODIFY,
- pub regPoll: GSP_SEQ_BUF_PAYLOAD_REG_POLL,
- pub delayUs: GSP_SEQ_BUF_PAYLOAD_DELAY_US,
- pub regStore: GSP_SEQ_BUF_PAYLOAD_REG_STORE,
-}
-impl Default for GSP_SEQUENCER_BUFFER_CMD__bindgen_ty_1 {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
-impl Default for GSP_SEQUENCER_BUFFER_CMD {
- fn default() -> Self {
- let mut s = ::core::mem::MaybeUninit::<Self>::uninit();
- unsafe {
- ::core::ptr::write_bytes(s.as_mut_ptr(), 0, 1);
- s.assume_init()
- }
- }
-}
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 30/33] gpu: nova-core: match GSP RPC replies by sequence, not just function
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (28 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 29/33] gpu: nova-core: firmware: delete the r570 bindings John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 31/33] gpu: nova-core: gsp: split the reply match out of the RPC receive path John Hubbard
` (2 subsequent siblings)
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
GSP-RM copies a command's RPC sequence number into its reply, alongside
the function code. The r570 firmware leaves that field zero, and the
r000 firmware fills it in.
Nova-core matched a reply on the function code alone and never set the
sequence in a command, so a reply to a command that had already timed
out could satisfy a later command with the same function code.
Write the sequence number into every RPC command, and require both the
function code and the sequence to match before accepting a reply. An
unsolicited event answers no command, so a caller that waits for an
event still matches on the function code alone. A message with the
expected function code and a stale sequence is a late reply rather than
an event, so it is logged and dropped.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
Documentation/gpu/nova/core/interrupts.rst | 9 ++-
drivers/gpu/nova-core/gsp/cmdq.rs | 94 ++++++++++++++--------
drivers/gpu/nova-core/gsp/fw.rs | 13 ++-
3 files changed, 77 insertions(+), 39 deletions(-)
diff --git a/Documentation/gpu/nova/core/interrupts.rst b/Documentation/gpu/nova/core/interrupts.rst
index fdcd789cf4f9..8519c73e9989 100644
--- a/Documentation/gpu/nova/core/interrupts.rst
+++ b/Documentation/gpu/nova/core/interrupts.rst
@@ -574,9 +574,12 @@ function code says which it is.
receive trace at debug level already records every message's arrival with
its sequence number, function code, and length.
-The sequence number takes no part in the match, because the GSP does not echo
-the command's sequence number on every reply. On r570 the reply to
-``UnloadingGuestDriver`` carries sequence 0.
+A command's reply must carry the RPC sequence number that nova-core wrote into
+the command, as well as its function code. A message with the awaited function
+code and a different sequence number is a stale reply to a command that already
+timed out, so it is logged at warning level and dropped rather than classified
+as an event. An unsolicited event answers no command, so a caller awaiting one
+matches on the function code alone.
The read pointer advances past every message, whether it matched, was an event,
or matched but failed to decode, so a message is never left at the queue head
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index 93c31b49903a..fc26c7d8aac0 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -555,7 +555,7 @@ pub(crate) fn new(
inner <- new_mutex!(CmdqInner {
dev,
gsp_mem,
- seq: 0,
+ rpc_seq: 0,
poisoned: Cell::new(false),
}),
}))
@@ -587,9 +587,9 @@ pub(crate) fn send_command<M>(&self, command: M) -> Result<M::Reply>
Error: From<<M::Reply as MessageFromGsp>::InitError>,
{
let mut inner = self.inner.lock();
- inner.send_command(command)?;
+ let expected_seq = inner.send_command(command)?;
- inner.await_msg()
+ inner.await_msg(Some(expected_seq))
}
/// Sends `command` to the GSP without waiting for a reply.
@@ -607,7 +607,7 @@ pub(crate) fn send_command_no_wait<M>(&self, command: M) -> Result
M: CommandToGsp<Reply = NoReply>,
Error: From<M::InitError>,
{
- self.inner.lock().send_command(command)
+ self.inner.lock().send_command(command).map(|_| ())
}
/// Waits for the response to the GMC request with command id `command_id`, and passes every
@@ -647,6 +647,8 @@ pub(crate) fn send_gmc_no_wait(
/// Waits for an unsolicited GSP event of type `M`. Events that arrive before it are logged and
/// consumed.
///
+ /// The event answers no command, so it is matched on its function code alone.
+ ///
/// The queue mutex is held for the whole wait, up to [`Self::RECEIVE_TIMEOUT`], so no other
/// caller can send a command or consume an event meanwhile.
///
@@ -663,7 +665,7 @@ pub(crate) fn await_msg<M: MessageFromGsp>(&self) -> Result<M>
// This allows all error types, including `Infallible`, to be used for `M::InitError`.
Error: From<M::InitError>,
{
- self.inner.lock().await_msg()
+ self.inner.lock().await_msg(None)
}
/// Logs and consumes every message the GSP has already posted, and returns without waiting for
@@ -685,8 +687,9 @@ pub(crate) fn drain(&self) -> Result {
struct CmdqInner<'a> {
/// Device this command queue belongs to.
dev: &'a device::Device,
- /// Current command sequence number.
- seq: u32,
+ /// Next RPC sequence number, advanced once per command, however many messages the command is
+ /// split into.
+ rpc_seq: u32,
/// Set once a message fails framing validation. Every later receive fails, since
/// the bad message cannot be skipped. See "Draining the GSP-to-CPU queue" in
/// `Documentation/gpu/nova/core/interrupts.rst`.
@@ -711,7 +714,7 @@ impl CmdqInner<'_> {
/// written to by its [`CommandToGsp::init_variable_payload`] method.
///
/// Error codes returned by the command initializers are propagated as-is.
- fn send_single_command<M>(&mut self, command: M) -> Result
+ fn send_single_command<M>(&mut self, command: M, rpc_seq: u32) -> Result
where
M: CommandToGsp,
// This allows all error types, including `Infallible`, to be used for `M::InitError`.
@@ -728,7 +731,7 @@ fn send_single_command<M>(&mut self, command: M) -> Result
let (cmd, payload_1) = M::Command::from_bytes_mut_prefix(dst.contents.0).ok_or(EIO)?;
// Fill the header and command in-place.
- let msg_element = GspMsgElement::init(size_in_bytes, M::FUNCTION);
+ let msg_element = GspMsgElement::init(rpc_seq, size_in_bytes, M::FUNCTION);
// SAFETY: `msg_header` and `cmd` are valid references, and not touched if the initializer
// fails.
unsafe {
@@ -748,22 +751,22 @@ fn send_single_command<M>(&mut self, command: M) -> Result
dev_dbg!(
&self.dev,
"GSP RPC: send: seq# {}, function={:?}, length=0x{:x}\n",
- self.seq,
+ rpc_seq,
M::FUNCTION,
dst.header.length(),
);
// All set - update the write pointer and inform the GSP of the new command.
let elem_count = dst.header.element_count();
- self.seq += 1;
self.gsp_mem.advance_cpu_write_ptr(elem_count);
Ok(())
}
- /// Sends `command` to the GSP.
+ /// Sends `command` to the GSP and returns the RPC sequence number assigned to it.
///
- /// The command may be split into multiple messages if it is large.
+ /// The command may be split into multiple messages if it is large. GSP-RM copies the
+ /// sequence number into its reply.
///
/// # Errors
///
@@ -772,24 +775,27 @@ fn send_single_command<M>(&mut self, command: M) -> Result
/// written to by its [`CommandToGsp::init_variable_payload`] method.
///
/// Error codes returned by the command initializers are propagated as-is.
- fn send_command<M>(&mut self, command: M) -> Result
+ fn send_command<M>(&mut self, command: M) -> Result<u32>
where
M: CommandToGsp,
Error: From<M::InitError>,
{
+ let rpc_seq = self.rpc_seq;
+ self.rpc_seq = self.rpc_seq.wrapping_add(1);
+
match SplitState::new(command)? {
- SplitState::Single(command) => self.send_single_command(command),
+ SplitState::Single(command) => self.send_single_command(command, rpc_seq)?,
SplitState::Split(command, mut continuations) => {
- self.send_single_command(command)?;
+ self.send_single_command(command, rpc_seq)?;
while let Some(continuation) = continuations.next() {
// Turbofish needed because the compiler cannot infer M here.
- self.send_single_command::<ContinuationRecord<'_>>(continuation)?;
+ self.send_single_command::<ContinuationRecord<'_>>(continuation, rpc_seq)?;
}
-
- Ok(())
}
}
+
+ Ok(rpc_seq)
}
/// Logs `reason`, poisons the queue, and returns `EIO` for the caller to propagate.
@@ -803,22 +809,26 @@ fn poison(&self, reason: fmt::Arguments<'_>) -> Error {
/// Sends a GMC API request to the GSP.
///
/// `payload` follows the GMC API header in the element, and `max_response_size` is the largest
- /// response that the caller accepts. The request carries the next sequence number, which GSP-RM
- /// copies into its response. The number is consumed even if the send fails.
+ /// response that the caller accepts. The request carries the next RPC sequence number, which
+ /// GSP-RM copies into its response. The number is consumed even if the send fails.
///
/// # Errors
///
/// Errors from [`DmaGspMem::allocate_command`] are propagated as-is.
fn send_gmc(&mut self, command_id: u32, payload: &[u8], max_response_size: u32) -> Result {
- let seq = self.seq;
- self.seq = self.seq.wrapping_add(1);
+ let rpc_seq = self.rpc_seq;
+ self.rpc_seq = self.rpc_seq.wrapping_add(1);
let dst = self
.gsp_mem
.allocate_command::<GspGmcMsgElement>(payload.len(), Self::ALLOCATE_TIMEOUT)?;
- let msg_element =
- GspGmcMsgElement::init(command_id, u64::from(seq), payload.len(), max_response_size);
+ let msg_element = GspGmcMsgElement::init(
+ command_id,
+ u64::from(rpc_seq),
+ payload.len(),
+ max_response_size,
+ );
// SAFETY: `dst.header` is a valid reference, and not written if the initializer fails.
unsafe {
pin_init::raw_try_init(core::ptr::from_mut(dst.header), msg_element)?;
@@ -830,7 +840,7 @@ fn send_gmc(&mut self, command_id: u32, payload: &[u8], max_response_size: u32)
dev_dbg!(
&self.dev,
"GSP GMC: send: seq# {}, command_id=0x{:x}, length=0x{:x}\n",
- seq,
+ rpc_seq,
command_id,
dst.header.length(),
);
@@ -908,6 +918,10 @@ fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {
/// A message whose function code is `M::FUNCTION` is decoded and returned. Any other message
/// is logged as an event.
///
+ /// With `expected_seq` set, the message must carry that RPC sequence number too. A message
+ /// with the expected function code and a different sequence is a stale reply to a command
+ /// that already timed out, so it is logged and dropped rather than classified as an event.
+ ///
/// The read pointer advances past the message in every case, including a decode failure.
///
/// # Errors
@@ -918,7 +932,11 @@ fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {
/// - `ENOMSG` if the message was not the awaited reply.
///
/// Error codes returned by [`MessageFromGsp::read`] are propagated as-is.
- fn receive_msg<M: MessageFromGsp>(&mut self, timeout: Delta) -> Result<M>
+ fn receive_msg<M: MessageFromGsp>(
+ &mut self,
+ timeout: Delta,
+ expected_seq: Option<u32>,
+ ) -> Result<M>
where
// This allows all error types, including `Infallible`, to be used for `M::InitError`.
Error: From<M::InitError>,
@@ -926,9 +944,11 @@ fn receive_msg<M: MessageFromGsp>(&mut self, timeout: Delta) -> Result<M>
let message = self.wait_for_msg(timeout)?;
let function = message.header.function();
let seq = message.header.sequence();
+ let func_matches = matches!(function, Ok(f) if f == M::FUNCTION);
+ let matched = func_matches && expected_seq.is_none_or(|expected| seq == expected);
// An early return here would leave the read pointer on this message.
- let result = if matches!(function, Ok(f) if f == M::FUNCTION) {
+ let result = if matched {
match M::Message::from_bytes_prefix(message.contents.0) {
Some((cmd, contents_1)) => {
let mut sbuffer = SBufferIter::new_reader([contents_1, message.contents.1]);
@@ -951,7 +971,17 @@ fn receive_msg<M: MessageFromGsp>(&mut self, timeout: Delta) -> Result<M>
}
}
} else {
- self.log_event(function, seq);
+ if func_matches {
+ dev_warn!(
+ &self.dev,
+ "GSP RPC: dropping stale {:?} reply (seq {}, awaiting {:?})\n",
+ M::FUNCTION,
+ seq,
+ expected_seq,
+ );
+ } else {
+ self.log_event(function, seq);
+ }
Err(ENOMSG)
};
@@ -967,7 +997,7 @@ fn receive_msg<M: MessageFromGsp>(&mut self, timeout: Delta) -> Result<M>
/// Receives a message of type `M`, waiting up to [`Cmdq::RECEIVE_TIMEOUT`] from the call.
///
/// Any other message that arrives first is logged as an event and does not extend the
- /// deadline.
+ /// deadline. `expected_seq` narrows the match as [`Self::receive_msg`] describes.
///
/// # Errors
///
@@ -977,7 +1007,7 @@ fn receive_msg<M: MessageFromGsp>(&mut self, timeout: Delta) -> Result<M>
/// [`Self::wait_for_msg`]).
///
/// Error codes returned by [`MessageFromGsp::read`] are propagated as-is.
- fn await_msg<M: MessageFromGsp>(&mut self) -> Result<M>
+ fn await_msg<M: MessageFromGsp>(&mut self, expected_seq: Option<u32>) -> Result<M>
where
// This allows all error types, including `Infallible`, to be used for `M::InitError`.
Error: From<M::InitError>,
@@ -988,7 +1018,7 @@ fn await_msg<M: MessageFromGsp>(&mut self) -> Result<M>
if remaining.is_negative() {
break Err(ETIMEDOUT);
}
- match self.receive_msg::<M>(remaining) {
+ match self.receive_msg::<M>(remaining, expected_seq) {
Ok(msg) => break Ok(msg),
Err(ENOMSG) => continue,
Err(e) => break Err(e),
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index 14271fbe0c25..ced7da14c0b2 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -470,13 +470,14 @@ fn new() -> Self {
}
impl bindings::rpc_message_header_v {
- fn init(cmd_size: usize, function: MsgFunction) -> impl Init<Self, Error> {
+ fn init(sequence: u32, cmd_size: usize, function: MsgFunction) -> impl Init<Self, Error> {
type RpcMessageHeader = bindings::rpc_message_header_v;
try_init!(RpcMessageHeader {
header_version: MsgHeaderVersion::new().into(),
signature: bindings::NV_VGPU_MSG_SIGNATURE_VALID,
function: function.into(),
+ sequence,
length: size_of::<Self>()
.checked_add(cmd_size)
.ok_or(EOVERFLOW)
@@ -503,8 +504,12 @@ pub(crate) struct GspMsgElement {
impl GspMsgElement {
/// Creates the queue element header and the RPC header of a command with a `cmd_size`-byte
- /// payload.
- pub(crate) fn init(cmd_size: usize, function: MsgFunction) -> impl Init<Self, Error> {
+ /// payload and the RPC sequence number `rpc_seq`.
+ pub(crate) fn init(
+ rpc_seq: u32,
+ cmd_size: usize,
+ function: MsgFunction,
+ ) -> impl Init<Self, Error> {
type RpcMessageHeader = bindings::rpc_message_header_v;
try_init!(GspMsgElement {
@@ -514,7 +519,7 @@ pub(crate) fn init(cmd_size: usize, function: MsgFunction) -> impl Init<Self, Er
.checked_add(cmd_size)
.ok_or(EOVERFLOW)?,
)?,
- rpc <- RpcMessageHeader::init(cmd_size, function),
+ rpc <- RpcMessageHeader::init(rpc_seq, cmd_size, function),
})
}
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 31/33] gpu: nova-core: gsp: split the reply match out of the RPC receive path
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (29 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 30/33] gpu: nova-core: match GSP RPC replies by sequence, not just function John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 32/33] gpu: nova-core: gsp: decode queue elements by their NVDM type John Hubbard
2026-09-18 1:07 ` [PATCH v3 33/33] gpu: nova-core: gsp: match a GMC response by flag, id and sequence John Hubbard
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
GSP-RM posts RPC and GMC messages on one queue, so a receive path has
to decode each element by its kind before it can match an RPC reply.
The RPC receive path matched the reply inline, in the function that
also waited for the element and advanced the read pointer past it. A
dispatch on the element kind in that function would have nested the
whole match one level deeper.
Move the match into a helper of its own, which decodes the awaited
reply or logs the message. The receive path keeps the wait and the
read pointer advance.
The following patch adds the dispatch.
No functional changes.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/cmdq.rs | 58 ++++++++++++++++++++++---------
1 file changed, 41 insertions(+), 17 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index fc26c7d8aac0..2258c4f2cfd1 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -915,12 +915,8 @@ fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {
/// Receives a message from the GSP.
///
- /// A message whose function code is `M::FUNCTION` is decoded and returned. Any other message
- /// is logged as an event.
- ///
- /// With `expected_seq` set, the message must carry that RPC sequence number too. A message
- /// with the expected function code and a different sequence is a stale reply to a command
- /// that already timed out, so it is logged and dropped rather than classified as an event.
+ /// [`Self::match_rpc_reply`] decodes the message as the awaited reply of type `M`, or logs it.
+ /// `expected_seq` narrows the match.
///
/// The read pointer advances past the message in every case, including a decode failure.
///
@@ -942,13 +938,48 @@ fn receive_msg<M: MessageFromGsp>(
Error: From<M::InitError>,
{
let message = self.wait_for_msg(timeout)?;
+
+ // An early return here would leave the read pointer on this message.
+ let result = self.match_rpc_reply::<M>(&message, expected_seq);
+
+ // Advance the read pointer past this message.
+ self.gsp_mem.advance_cpu_read_ptr(u32::try_from(
+ message.header.length().div_ceil(GSP_PAGE_SIZE),
+ )?);
+
+ result
+ }
+
+ /// Decodes `message` as the awaited reply of type `M`, or logs it.
+ ///
+ /// A message whose function code is `M::FUNCTION` is decoded and returned. Any other message
+ /// is logged as an event.
+ ///
+ /// With `expected_seq` set, the message must carry that RPC sequence number too. A message
+ /// with the expected function code and a different sequence is a stale reply to a command
+ /// that already timed out, so it is logged as stale rather than as an event.
+ ///
+ /// # Errors
+ ///
+ /// - `EIO` if the matched message is too short for `M::Message`.
+ /// - `ENOMSG` if the message is not the awaited reply.
+ ///
+ /// Error codes returned by [`MessageFromGsp::read`] are propagated as-is.
+ fn match_rpc_reply<M: MessageFromGsp>(
+ &self,
+ message: &GspMessage<'_>,
+ expected_seq: Option<u32>,
+ ) -> Result<M>
+ where
+ // This allows all error types, including `Infallible`, to be used for `M::InitError`.
+ Error: From<M::InitError>,
+ {
let function = message.header.function();
let seq = message.header.sequence();
let func_matches = matches!(function, Ok(f) if f == M::FUNCTION);
let matched = func_matches && expected_seq.is_none_or(|expected| seq == expected);
- // An early return here would leave the read pointer on this message.
- let result = if matched {
+ if matched {
match M::Message::from_bytes_prefix(message.contents.0) {
Some((cmd, contents_1)) => {
let mut sbuffer = SBufferIter::new_reader([contents_1, message.contents.1]);
@@ -984,20 +1015,13 @@ fn receive_msg<M: MessageFromGsp>(
}
Err(ENOMSG)
- };
-
- // Advance the read pointer past this message.
- self.gsp_mem.advance_cpu_read_ptr(u32::try_from(
- message.header.length().div_ceil(GSP_PAGE_SIZE),
- )?);
-
- result
+ }
}
/// Receives a message of type `M`, waiting up to [`Cmdq::RECEIVE_TIMEOUT`] from the call.
///
/// Any other message that arrives first is logged as an event and does not extend the
- /// deadline. `expected_seq` narrows the match as [`Self::receive_msg`] describes.
+ /// deadline. `expected_seq` narrows the match as [`Self::match_rpc_reply`] describes.
///
/// # Errors
///
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 32/33] gpu: nova-core: gsp: decode queue elements by their NVDM type
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (30 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 31/33] gpu: nova-core: gsp: split the reply match out of the RPC receive path John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
2026-09-18 1:07 ` [PATCH v3 33/33] gpu: nova-core: gsp: match a GMC response by flag, id and sequence John Hubbard
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
GSP-RM posts RPC and GMC messages on one queue. Every element opens with
the same transport headers, whose NVDM type selects whether an RPC
header or a GMC header follows.
Each receive path assumed the header of its own kind. An RPC wait read a
GMC element through the RPC header layout, where the GMC sequence number
occupies the function code's field, so a GMC response looked like the
RPC no-op and was consumed silently, and a GMC event looked like an
unknown function. The GMC wait logged an RPC element as a dropped
element, so an OS error record posted during the GSP_INIT wait was never
logged as an error.
Decode every element once, at the transport level, and dispatch on its
NVDM type, so that each receive path decodes its own kind and logs the
other kind as what it is. An element whose NVDM type the queue does not
use poisons the queue, as bad framing does.
The r000 GSP-RM posts no element for the path that the driver is not
waiting in, but the protocol allows it, so both receive paths must
accept both kinds.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
Documentation/gpu/nova/core/interrupts.rst | 31 +-
drivers/gpu/nova-core/gsp/cmdq.rs | 334 +++++++++++----------
drivers/gpu/nova-core/gsp/fw.rs | 30 +-
3 files changed, 201 insertions(+), 194 deletions(-)
diff --git a/Documentation/gpu/nova/core/interrupts.rst b/Documentation/gpu/nova/core/interrupts.rst
index 8519c73e9989..fa124bc8ca05 100644
--- a/Documentation/gpu/nova/core/interrupts.rst
+++ b/Documentation/gpu/nova/core/interrupts.rst
@@ -562,17 +562,23 @@ GSP.
Draining the GSP-to-CPU queue
-----------------------------
-The queue carries command replies and unsolicited events, and a message's
-function code says which it is.
+The queue carries RPC messages and GMC messages. GMC is the GPU Management
+Controller, and its API is ABI-stable. Every element opens with the same queue
+element header, which holds the MCTP and NVDM headers, and the NVDM type in
+that header selects which kind of message header follows. An RPC message is a
+command reply or an unsolicited event, and the two differ in the function code.
* A function code that matches the awaited reply: the message is decoded and
returned to the caller that sent the command.
-* Anything else is an event. An OS error record and a robust-channel record
- are logged at error level, and an unrecognized function code at warning
- level. The other known events (GSP logs, libos prints, assertion records,
- lifecycle notices) need no action and get no line of their own, because the
- receive trace at debug level already records every message's arrival with
- its sequence number, function code, and length.
+* Any other RPC message is an event. An OS error record and a robust-channel
+ record are logged at error level, and an unrecognized function code at
+ warning level. The other known events (GSP logs, LIBOS prints, assertion
+ records, lifecycle notices) need no action and get no line of their own,
+ because the receive trace at debug level already records every message's
+ arrival with its sequence number, function code, and length.
+* A GMC message carries a command id in place of a function code. Only the
+ ``GSP_INIT`` wait during boot claims GMC messages, so one that arrives
+ anywhere else is logged at warning level and dropped.
A command's reply must carry the RPC sequence number that nova-core wrote into
the command, as well as its function code. A message with the awaited function
@@ -585,10 +591,11 @@ The read pointer advances past every message, whether it matched, was an event,
or matched but failed to decode, so a message is never left at the queue head
for the next receive to parse again.
-Corrupt framing is the exception. An element that fails framing validation has
-no trustworthy length, so the read pointer cannot advance past it. Such a
-failure poisons the queue: nova-core logs it once, and every later receive
-fails with ``EIO`` until the device is reset.
+Corrupt framing is the exception. An element that fails framing validation, or
+that carries an NVDM type that the queue does not use, has no trustworthy
+length, so the read pointer cannot advance past it. Such a failure poisons the
+queue: nova-core logs it once, and every later receive fails with ``EIO`` until
+the device is reset.
The polling path and the IRQ thread both read the queue under the command-queue
mutex. Replies and events share one queue and one read pointer, so one lock is
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index 2258c4f2cfd1..2664ace70ef8 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -317,6 +317,11 @@ fn driver_write_area_size(&self) -> usize {
num::u32_as_usize(self.free_slots()) * GSP_PAGE_SIZE
}
+ /// Returns `true` if the GSP has posted a message that the driver has not consumed.
+ fn has_unread_message(&self) -> bool {
+ self.gsp_write_ptr() != self.cpu_read_ptr
+ }
+
/// Returns the region of the GSP message queue that the driver may read, as two slices
/// because the ring wraps.
fn driver_read_area(&self) -> (&[[u8; GSP_PAGE_SIZE]], &[[u8; GSP_PAGE_SIZE]]) {
@@ -462,9 +467,9 @@ struct GspCommand<'a, H> {
contents: (&'a mut [u8], &'a mut [u8]),
}
-/// A message ready to be processed from the message queue.
+/// An RPC message ready to be processed from the message queue.
///
-/// This is the type returned by [`CmdqInner::wait_for_msg`].
+/// This is the message that [`QueueElement::Rpc`] carries.
struct GspMessage<'a> {
// Reference to the header of the message.
header: &'a GspMsgElement,
@@ -492,23 +497,50 @@ struct GmcMessage<'a> {
///
/// This is the type returned by [`CmdqInner::wait_for_element`].
enum QueueElement<'a> {
+ /// An RM RPC message.
+ Rpc(GspMessage<'a>),
/// A GMC API message.
Gmc(GmcMessage<'a>),
- /// An element whose NVDM type names another kind of message, such as an RM RPC. Only its
- /// queue element header is decoded.
- Other(&'a QueueElementHeader),
}
impl QueueElement<'_> {
/// Returns the number of queue slots that the element occupies.
fn element_count(&self) -> u32 {
match self {
+ Self::Rpc(message) => message.header.element_count(),
Self::Gmc(message) => message.header.element_count(),
- Self::Other(element_header) => element_header.element_count(),
}
}
}
+/// The headers that open a queue element of one kind of message: the queue element header, then
+/// the RPC header or the GMC API header.
+trait MessageHeaders: FromBytes {
+ /// Name of the kind of message, for the log line written when an element of this kind poisons
+ /// the queue.
+ const KIND: &'static str;
+
+ /// Returns the length of the payload that follows the message header, or `None` if the queue
+ /// element header declares a message shorter than the message header.
+ fn payload_length(&self) -> Option<usize>;
+}
+
+impl MessageHeaders for GspMsgElement {
+ const KIND: &'static str = "RPC";
+
+ fn payload_length(&self) -> Option<usize> {
+ GspMsgElement::payload_length(self)
+ }
+}
+
+impl MessageHeaders for GspGmcMsgElement {
+ const KIND: &'static str = "GMC";
+
+ fn payload_length(&self) -> Option<usize> {
+ GspGmcMsgElement::payload_length(self)
+ }
+}
+
/// GSP command queue.
///
/// Provides the ability to send commands and receive messages from the GSP using a shared memory
@@ -668,16 +700,16 @@ pub(crate) fn await_msg<M: MessageFromGsp>(&self) -> Result<M>
self.inner.lock().await_msg(None)
}
- /// Logs and consumes every message the GSP has already posted, and returns without waiting for
- /// more.
+ /// Logs and consumes every element that the GSP has already posted, and returns without waiting
+ /// for more.
///
- /// No caller is waiting for a reply while this holds the queue mutex, so every message is
+ /// No caller is waiting for a reply while this holds the queue mutex, so every element is
/// logged as an event. See "Draining the GSP-to-CPU queue" in
/// `Documentation/gpu/nova/core/interrupts.rst`.
///
/// # Errors
///
- /// `EIO` if the queue is poisoned, or if a message fails framing validation.
+ /// `EIO` if the queue is poisoned, or if an element fails framing validation.
pub(crate) fn drain(&self) -> Result {
self.inner.lock().drain()
}
@@ -690,11 +722,11 @@ struct CmdqInner<'a> {
/// Next RPC sequence number, advanced once per command, however many messages the command is
/// split into.
rpc_seq: u32,
- /// Set once a message fails framing validation. Every later receive fails, since
- /// the bad message cannot be skipped. See "Draining the GSP-to-CPU queue" in
+ /// Set once an element fails framing validation. Every later receive fails, because the bad
+ /// element cannot be skipped. See "Draining the GSP-to-CPU queue" in
/// `Documentation/gpu/nova/core/interrupts.rst`.
///
- /// A [`Cell`] because [`Self::wait_for_msg`] sets it through `&self`.
+ /// A [`Cell`], because the receive path sets it through `&self`.
poisoned: Cell<bool>,
/// Memory area shared with the GSP for communicating commands and messages.
gsp_mem: DmaGspMem<'a>,
@@ -800,7 +832,7 @@ fn send_command<M>(&mut self, command: M) -> Result<u32>
/// Logs `reason`, poisons the queue, and returns `EIO` for the caller to propagate.
fn poison(&self, reason: fmt::Arguments<'_>) -> Error {
- dev_err!(&self.dev, "GSP RPC: receive: queue poisoned: {}\n", reason);
+ dev_err!(&self.dev, "GSP receive: queue poisoned: {}\n", reason);
self.poisoned.set(true);
EIO
@@ -851,81 +883,17 @@ fn send_gmc(&mut self, command_id: u32, payload: &[u8], max_response_size: u32)
Ok(())
}
- /// Waits for a message to become available on the message queue.
- ///
- /// This validates the queue element header and the lengths that it declares, and does not
- /// interpret the RPC header that follows it.
- ///
- /// Returns the message's [`GspMsgElement`] and its contents as two byte slices, the second of
- /// which is empty unless the message wraps around the end of the message queue.
- ///
- /// # Errors
- ///
- /// - `ETIMEDOUT` if `timeout` has elapsed before any message becomes available.
- /// - `EIO` if the queue is already poisoned, or if the framing is invalid, which poisons it
- /// (see [`Self::poisoned`]).
- fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {
- if self.poisoned.get() {
- return Err(EIO);
- }
-
- // Wait for a message to arrive from the GSP.
- let (slice_1, slice_2) = read_poll_timeout(
- || Ok(self.gsp_mem.driver_read_area()),
- |driver_area| !driver_area.0.is_empty(),
- Delta::from_millis(1),
- timeout,
- )
- .map(|(slice_1, slice_2)| (slice_1.as_flattened(), slice_2.as_flattened()))?;
-
- // Extract the `GspMsgElement`.
- let Some((header, slice_1)) = GspMsgElement::from_bytes_prefix(slice_1) else {
- return Err(self.poison(fmt!(
- "read area of {} bytes is shorter than a message header",
- slice_1.len()
- )));
- };
-
- if header.validate_framing().is_err() {
- return Err(self.poison(fmt!(
- "RPC element has a bad queue element header, declared length {}",
- header.length()
- )));
- }
-
- dev_dbg!(
- &self.dev,
- "GSP RPC: receive: seq# {}, function={:?}, length=0x{:x}\n",
- header.sequence(),
- header.function(),
- header.length(),
- );
-
- let Some(payload_length) = header.payload_length() else {
- return Err(self.poison(fmt!(
- "RPC message seq# {} declares a message shorter than the RPC header",
- header.sequence()
- )));
- };
-
- let contents = self.payload_slices(slice_1, slice_2, payload_length)?;
-
- Ok(GspMessage { header, contents })
- }
-
- /// Receives a message from the GSP.
- ///
- /// [`Self::match_rpc_reply`] decodes the message as the awaited reply of type `M`, or logs it.
- /// `expected_seq` narrows the match.
+ /// Receives an element from the GSP.
///
- /// The read pointer advances past the message in every case, including a decode failure.
+ /// [`Self::match_rpc_reply`] decodes an RPC message as the awaited reply of type `M`, or logs
+ /// it. `expected_seq` narrows the match. A GMC message is logged as unclaimed.
///
/// # Errors
///
- /// - `ETIMEDOUT` if `timeout` has elapsed before any message becomes available.
- /// - `EIO` if the queue is poisoned or the message fails framing validation (see
- /// [`Self::wait_for_msg`]), or if the matched message is too short for `M::Message`.
- /// - `ENOMSG` if the message was not the awaited reply.
+ /// - `ETIMEDOUT` if `timeout` has elapsed before any element becomes available.
+ /// - `EIO` if the queue is poisoned or the element fails framing validation (see
+ /// [`Self::wait_for_element`]), or if the matched message is too short for `M::Message`.
+ /// - `ENOMSG` if the element is not the awaited reply.
///
/// Error codes returned by [`MessageFromGsp::read`] are propagated as-is.
fn receive_msg<M: MessageFromGsp>(
@@ -937,17 +905,14 @@ fn receive_msg<M: MessageFromGsp>(
// This allows all error types, including `Infallible`, to be used for `M::InitError`.
Error: From<M::InitError>,
{
- let message = self.wait_for_msg(timeout)?;
-
- // An early return here would leave the read pointer on this message.
- let result = self.match_rpc_reply::<M>(&message, expected_seq);
-
- // Advance the read pointer past this message.
- self.gsp_mem.advance_cpu_read_ptr(u32::try_from(
- message.header.length().div_ceil(GSP_PAGE_SIZE),
- )?);
+ self.consume_element(timeout, |this, element| match element {
+ QueueElement::Gmc(message) => {
+ this.log_gmc_event(message.header);
- result
+ Err(ENOMSG)
+ }
+ QueueElement::Rpc(message) => this.match_rpc_reply::<M>(&message, expected_seq),
+ })
}
/// Decodes `message` as the awaited reply of type `M`, or logs it.
@@ -1020,15 +985,15 @@ fn match_rpc_reply<M: MessageFromGsp>(
/// Receives a message of type `M`, waiting up to [`Cmdq::RECEIVE_TIMEOUT`] from the call.
///
- /// Any other message that arrives first is logged as an event and does not extend the
- /// deadline. `expected_seq` narrows the match as [`Self::match_rpc_reply`] describes.
+ /// Any other element that arrives first is logged and does not extend the deadline.
+ /// `expected_seq` narrows the match as [`Self::match_rpc_reply`] describes.
///
/// # Errors
///
/// - `ETIMEDOUT` if no message of type `M` arrives before the deadline, however many other
- /// messages arrive while waiting.
- /// - `EIO` if the queue is poisoned or a message fails framing validation (see
- /// [`Self::wait_for_msg`]).
+ /// elements arrive while waiting.
+ /// - `EIO` if the queue is poisoned or an element fails framing validation (see
+ /// [`Self::wait_for_element`]).
///
/// Error codes returned by [`MessageFromGsp::read`] are propagated as-is.
fn await_msg<M: MessageFromGsp>(&mut self, expected_seq: Option<u32>) -> Result<M>
@@ -1050,11 +1015,11 @@ fn await_msg<M: MessageFromGsp>(&mut self, expected_seq: Option<u32>) -> Result<
}
}
- /// Logs an event, meaning a message that no caller was waiting for.
+ /// Logs an event, meaning an RPC message that no caller is waiting for.
///
/// An OS error or robust-channel record is logged at error level and an unknown function code
/// at warning level. Every other event is recorded only by the receive trace in
- /// [`Self::wait_for_msg`].
+ /// [`Self::consume_element`].
fn log_event(&self, function: Result<MsgFunction, u32>, seq: u32) {
match function {
Ok(MsgFunction::OsErrorLog) => {
@@ -1080,27 +1045,35 @@ fn log_event(&self, function: Result<MsgFunction, u32>, seq: u32) {
}
}
- /// Logs and consumes every message the queue holds.
+ /// Logs a GMC message that no caller is waiting for: a response to a request that has already
+ /// timed out, or an event that arrives outside the boot sequence.
+ fn log_gmc_event(&self, header: &GspGmcMsgElement) {
+ dev_warn!(
+ &self.dev,
+ "GSP GMC: dropping unclaimed message (seq {}, command_id=0x{:x})\n",
+ header.gmc.sequence,
+ header.gmc.command_id(),
+ );
+ }
+
+ /// Logs and consumes every element that the queue holds.
///
/// # Errors
///
- /// `EIO` if the queue is poisoned, a message fails framing validation, or a
- /// message's page count overflows a `u32`.
+ /// `EIO` if the queue is poisoned or an element fails framing validation.
fn drain(&mut self) -> Result {
- while !self.gsp_mem.driver_read_area().0.is_empty() {
- // A message is available, so this returns without waiting.
- let msg = self.wait_for_msg(Delta::ZERO)?;
-
- let pages =
- u32::try_from(msg.header.length().div_ceil(GSP_PAGE_SIZE)).map_err(|_| {
- dev_err!(&self.dev, "GSP drain: message length overflow\n");
- EIO
- })?;
- let function = msg.header.function();
- let seq = msg.header.sequence();
+ while self.gsp_mem.has_unread_message() {
+ // An element is available, so this returns without waiting.
+ self.consume_element(Delta::ZERO, |this, element| {
+ match element {
+ QueueElement::Rpc(message) => {
+ this.log_event(message.header.function(), message.header.sequence());
+ }
+ QueueElement::Gmc(message) => this.log_gmc_event(message.header),
+ }
- self.gsp_mem.advance_cpu_read_ptr(pages);
- self.log_event(function, seq);
+ Ok(())
+ })?;
}
Ok(())
@@ -1146,18 +1119,21 @@ fn payload_slices<'a>(
/// | queue element header: magic, MCTP | validated
/// | header, NVDM header, lengths |
/// +------------------------------------+
- /// | message header | decoded as a GMC API header when the NVDM type
- /// +------------------------------------+ is GmcApi, and left undecoded otherwise
+ /// | message header | decoded as the RPC header or the GMC API
+ /// +------------------------------------+ header. The NVDM type selects between the two.
/// | payload | truncated to the length that the queue
/// +------------------------------------+ element header declares
/// ```
///
+ /// The element stays at the queue head, and the payload slices point into it, so the read
+ /// pointer must not advance until they are dropped.
+ ///
/// # Errors
///
- /// - `ETIMEDOUT` if no element arrives within `timeout`.
- /// - `EIO` if the queue is already poisoned, or if the framing is invalid, or if the GMC API
- /// header and the queue element header declare different payload sizes. Each of these
- /// poisons the queue (see [`Self::poisoned`]).
+ /// - `ETIMEDOUT` if `timeout` has elapsed before any element becomes available.
+ /// - `EIO` if the queue is already poisoned, or if the framing, the NVDM type or a declared
+ /// length is invalid, or if the GMC API header and the queue element header declare
+ /// different payload sizes. Each of these poisons the queue (see [`Self::poisoned`]).
fn wait_for_element(&self, timeout: Delta) -> Result<QueueElement<'_>> {
if self.poisoned.get() {
return Err(EIO);
@@ -1186,38 +1162,68 @@ fn wait_for_element(&self, timeout: Delta) -> Result<QueueElement<'_>> {
)));
}
- if !element_header.is_nvdm_type(NvdmType::GmcApi) {
- return Ok(QueueElement::Other(element_header));
+ match element_header.nvdm_type() {
+ Ok(NvdmType::RmRpc) => {
+ let (header, contents) = self.split_element::<GspMsgElement>(slice_1, slice_2)?;
+
+ Ok(QueueElement::Rpc(GspMessage { header, contents }))
+ }
+ Ok(NvdmType::GmcApi) => {
+ let (header, contents) =
+ self.split_element::<GspGmcMsgElement>(slice_1, slice_2)?;
+
+ // GSP-RM writes both sizes from the same payload, so a difference means that one
+ // of the two headers is corrupt, and the driver cannot know which.
+ let payload_length = contents.0.len() + contents.1.len();
+ if payload_length != num::u32_as_usize(header.gmc.size) {
+ return Err(self.poison(fmt!(
+ "GMC seq# {}: GMC API header declares {} payload bytes, element header {}",
+ header.gmc.sequence,
+ header.gmc.size,
+ payload_length
+ )));
+ }
+
+ Ok(QueueElement::Gmc(GmcMessage { header, contents }))
+ }
+ Ok(nvdm_type) => Err(self.poison(fmt!(
+ "element carries NVDM type {:?}, which the GSP queues do not use",
+ nvdm_type
+ ))),
+ Err(_) => Err(self.poison(fmt!("element carries an unknown NVDM type"))),
}
+ }
- let Some((header, slice_1)) = GspGmcMsgElement::from_bytes_prefix(slice_1) else {
+ /// Splits the read area into the headers of type `H` that open the element and the payload
+ /// slices that follow them.
+ ///
+ /// # Errors
+ ///
+ /// - `EIO` if the read area is shorter than the headers, if the queue element header declares
+ /// a message shorter than the message header, or if fewer payload bytes are readable than
+ /// declared. Each of these poisons the queue.
+ fn split_element<'a, H: MessageHeaders>(
+ &self,
+ slice_1: &'a [u8],
+ slice_2: &'a [u8],
+ ) -> Result<(&'a H, (&'a [u8], &'a [u8]))> {
+ let Some((header, slice_1)) = H::from_bytes_prefix(slice_1) else {
return Err(self.poison(fmt!(
- "read area of {} bytes is shorter than a GMC element header",
- slice_1.len()
+ "{} element: read area is shorter than the message header",
+ H::KIND
)));
};
let Some(payload_length) = header.payload_length() else {
return Err(self.poison(fmt!(
- "GMC message seq# {} declares a message shorter than the GMC API header",
- header.gmc.sequence
+ "{} element declares a message shorter than the message header",
+ H::KIND
)));
};
- // GSP-RM writes both sizes from the same payload, so a difference means that one of the
- // two headers is corrupt, and the driver cannot know which.
- if payload_length != num::u32_as_usize(header.gmc.size) {
- return Err(self.poison(fmt!(
- "GMC seq# {}: GMC API header declares {} payload bytes, element header {}",
- header.gmc.sequence,
- header.gmc.size,
- payload_length
- )));
- }
-
let contents = self.payload_slices(slice_1, slice_2, payload_length)?;
- Ok(QueueElement::Gmc(GmcMessage { header, contents }))
+ Ok((header, contents))
}
/// Waits for the next queue element, passes it to `f`, and advances the read pointer past it.
@@ -1240,6 +1246,23 @@ fn consume_element<R>(
let element = self.wait_for_element(timeout)?;
let element_count = element.element_count();
+ match &element {
+ QueueElement::Rpc(message) => dev_dbg!(
+ &self.dev,
+ "GSP RPC: receive: seq# {}, function={:?}, length=0x{:x}\n",
+ message.header.sequence(),
+ message.header.function(),
+ message.header.length(),
+ ),
+ QueueElement::Gmc(message) => dev_dbg!(
+ &self.dev,
+ "GSP GMC: receive: seq# {}, command_id=0x{:x}, length=0x{:x}\n",
+ message.header.gmc.sequence,
+ message.header.gmc.command_id(),
+ message.header.length(),
+ ),
+ }
+
let result = f(self, element);
self.gsp_mem.advance_cpu_read_ptr(element_count);
@@ -1250,8 +1273,7 @@ fn consume_element<R>(
/// Receives the next queue element and, if it is a GMC element, passes it to `handler`.
///
/// `handler` receives the headers that open the element and the payload that follows the GMC
- /// API header, as two slices because the ring may wrap, and returns `None` for an element that
- /// it declines.
+ /// API header, as two slices because the ring may wrap. An RPC element is logged as an event.
///
/// Returns `Ok(None)` when `handler` declines the element or when the element is not a GMC
/// element.
@@ -1259,7 +1281,7 @@ fn consume_element<R>(
/// # Errors
///
/// - `ETIMEDOUT` if no element arrives within `timeout`.
- /// - `EIO` if the queue is poisoned or the queue element header is invalid, as
+ /// - `EIO` if the queue is poisoned or the element is invalid, as
/// [`Self::wait_for_element`] describes.
///
/// Errors from `handler` are propagated as-is.
@@ -1269,23 +1291,13 @@ fn receive_gmc_and_dispatch<R>(
handler: impl FnOnce(&GspGmcMsgElement, &[u8], &[u8]) -> Result<Option<R>>,
) -> Result<Option<R>> {
self.consume_element(timeout, |this, element| match element {
- QueueElement::Other(_) => {
- dev_warn!(&this.dev, "GSP GMC: dropping non-GMC queue element\n");
+ QueueElement::Rpc(message) => {
+ this.log_event(message.header.function(), message.header.sequence());
Ok(None)
}
QueueElement::Gmc(message) => {
- let header = message.header;
-
- dev_dbg!(
- &this.dev,
- "GSP GMC: event: seq# {}, command_id=0x{:x}, length=0x{:x}\n",
- header.gmc.sequence,
- header.gmc.command_id(),
- header.length(),
- );
-
- handler(header, message.contents.0, message.contents.1)
+ handler(message.header, message.contents.0, message.contents.1)
}
})
}
@@ -1295,8 +1307,8 @@ fn receive_gmc_and_dispatch<R>(
///
/// The response's payload is passed to `decode`, as two slices because the ring may wrap.
/// Every other GMC element that arrives first is passed to `on_other` with the headers that
- /// open it and its payload slices, and any other element is logged. Neither kind of element
- /// extends the deadline.
+ /// open it and its payload slices, and an RPC element is logged as an event. Neither kind of
+ /// element extends the deadline.
///
/// # Errors
///
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index ced7da14c0b2..f3dff49af2bf 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -535,23 +535,6 @@ pub(crate) fn length(&self) -> usize {
self.element_header.element_len()
}
- /// Validates the queue element header and that the element is long enough to hold the RPC
- /// header after it.
- ///
- /// # Errors
- ///
- /// - `EIO` if [`QueueElementHeader::validate`] fails, or if the declared element length is
- /// shorter than the two headers together.
- pub(crate) fn validate_framing(&self) -> Result {
- self.element_header.validate().map_err(|_| EIO)?;
-
- if self.length() < size_of::<Self>() {
- return Err(EIO);
- }
-
- Ok(())
- }
-
// Returns the sequence number of the message.
pub(crate) fn sequence(&self) -> u32 {
self.rpc.sequence
@@ -665,6 +648,15 @@ pub(crate) fn element_count(&self) -> u32 {
.div_ceil(num::usize_into_u32::<GSP_PAGE_SIZE>())
}
+ /// Returns the NVDM type.
+ ///
+ /// # Errors
+ ///
+ /// - `EINVAL` if the field holds no known NVDM type.
+ pub(crate) fn nvdm_type(&self) -> Result<NvdmType> {
+ self.nvdm.nvdm_type()
+ }
+
/// Validates the queue element header.
///
/// Returns the first check that fails as a [`QueueElementHeaderError`].
@@ -692,10 +684,6 @@ pub(crate) fn validate(&self) -> Result<(), QueueElementHeaderError> {
Ok(())
}
-
- pub(crate) fn is_nvdm_type(&self, nvdm_type: NvdmType) -> bool {
- self.nvdm.validate(nvdm_type)
- }
}
/// The check of [`QueueElementHeader::validate`] that a queue element header fails.
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread* [PATCH v3 33/33] gpu: nova-core: gsp: match a GMC response by flag, id and sequence
2026-09-18 1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
` (31 preceding siblings ...)
2026-09-18 1:07 ` [PATCH v3 32/33] gpu: nova-core: gsp: decode queue elements by their NVDM type John Hubbard
@ 2026-09-18 1:07 ` John Hubbard
32 siblings, 0 replies; 35+ messages in thread
From: John Hubbard @ 2026-09-18 1:07 UTC (permalink / raw)
To: Danilo Krummrich, Alexandre Courbot
Cc: Timur Tabi, Alistair Popple, Eliot Courtney, Zhi Wang,
David Airlie, Simona Vetter, Bjorn Helgaas, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
nova-gpu, LKML, John Hubbard
From: Zhi Wang <zhiw@nvidia.com>
GSP-RM marks a response with a flag bit in the GMC command word, and it
echoes back the RPC sequence number that the request carried. An event
carries neither, and an event can name any command id. The r000 GSP-RM
raises no event that names the GSP_INIT id.
Nova-core matched the GSP_INIT reply on the command id alone. The
receive path would have decoded an event that named the same id as the
reply, and dropped the real reply, so only the absence of such an event
kept the boot working.
Return the sequence number that a GMC send used, and require the
response flag, the command id and that sequence to match before
decoding a reply. A command that GSP-RM does not answer goes through
its own send path, which has no sequence to return.
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Assisted-by: LLM
Co-developed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
drivers/gpu/nova-core/gsp/cmdq.rs | 44 ++++++++++++++++++---------
drivers/gpu/nova-core/gsp/commands.rs | 8 +++--
drivers/gpu/nova-core/gsp/fw.rs | 26 +++++++++++++---
3 files changed, 55 insertions(+), 23 deletions(-)
diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index 2664ace70ef8..dd0b3dfc6f37 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -642,8 +642,8 @@ pub(crate) fn send_command_no_wait<M>(&self, command: M) -> Result
self.inner.lock().send_command(command).map(|_| ())
}
- /// Waits for the response to the GMC request with command id `command_id`, and passes every
- /// other GMC element that arrives first to `on_other`.
+ /// Waits for the response to the GMC request with command id `command_id` and RPC sequence
+ /// number `sequence`, and passes every other GMC element that arrives first to `on_other`.
///
/// This method may sleep while waiting. The queue mutex stays locked across the whole wait and
/// across the `on_other` and `decode` calls, so neither may call back into this [`Cmdq`].
@@ -652,16 +652,19 @@ pub(crate) fn send_command_no_wait<M>(&self, command: M) -> Result
pub(crate) fn await_gmc_response<R>(
&self,
command_id: u32,
+ sequence: u32,
on_other: impl FnMut(&GspGmcMsgElement, &[u8], &[u8]) -> Result,
decode: impl FnMut(&[u8], &[u8]) -> Result<R>,
) -> Result<R> {
self.inner
.lock()
- .await_gmc_response(command_id, on_other, decode)
+ .await_gmc_response(command_id, sequence, on_other, decode)
}
/// Sends a GMC API request to the GSP without waiting for the response.
///
+ /// Returns the RPC sequence number that the request carries.
+ ///
/// # Errors
///
/// Errors from [`DmaGspMem::allocate_command`] are propagated as-is.
@@ -670,12 +673,24 @@ pub(crate) fn send_gmc_no_wait(
command_id: u32,
payload: &[u8],
max_response_size: u32,
- ) -> Result {
+ ) -> Result<u32> {
self.inner
.lock()
.send_gmc(command_id, payload, max_response_size)
}
+ /// Sends a GMC API request that GSP-RM does not answer.
+ ///
+ /// # Errors
+ ///
+ /// Errors from [`DmaGspMem::allocate_command`] are propagated as-is.
+ pub(crate) fn send_gmc_no_reply(&self, command_id: u32, payload: &[u8]) -> Result {
+ self.inner
+ .lock()
+ .send_gmc(command_id, payload, 0)
+ .map(|_| ())
+ }
+
/// Waits for an unsolicited GSP event of type `M`. Events that arrive before it are logged and
/// consumed.
///
@@ -844,10 +859,12 @@ fn poison(&self, reason: fmt::Arguments<'_>) -> Error {
/// response that the caller accepts. The request carries the next RPC sequence number, which
/// GSP-RM copies into its response. The number is consumed even if the send fails.
///
+ /// Returns the RPC sequence number that the request carries.
+ ///
/// # Errors
///
/// Errors from [`DmaGspMem::allocate_command`] are propagated as-is.
- fn send_gmc(&mut self, command_id: u32, payload: &[u8], max_response_size: u32) -> Result {
+ fn send_gmc(&mut self, command_id: u32, payload: &[u8], max_response_size: u32) -> Result<u32> {
let rpc_seq = self.rpc_seq;
self.rpc_seq = self.rpc_seq.wrapping_add(1);
@@ -855,12 +872,8 @@ fn send_gmc(&mut self, command_id: u32, payload: &[u8], max_response_size: u32)
.gsp_mem
.allocate_command::<GspGmcMsgElement>(payload.len(), Self::ALLOCATE_TIMEOUT)?;
- let msg_element = GspGmcMsgElement::init(
- command_id,
- u64::from(rpc_seq),
- payload.len(),
- max_response_size,
- );
+ let msg_element =
+ GspGmcMsgElement::init(command_id, rpc_seq, payload.len(), max_response_size);
// SAFETY: `dst.header` is a valid reference, and not written if the initializer fails.
unsafe {
pin_init::raw_try_init(core::ptr::from_mut(dst.header), msg_element)?;
@@ -880,7 +893,7 @@ fn send_gmc(&mut self, command_id: u32, payload: &[u8], max_response_size: u32)
let elem_count = dst.header.element_count();
self.gsp_mem.advance_cpu_write_ptr(elem_count);
- Ok(())
+ Ok(rpc_seq)
}
/// Receives an element from the GSP.
@@ -1302,8 +1315,8 @@ fn receive_gmc_and_dispatch<R>(
})
}
- /// Waits for the response to the GMC request with command id `command_id`, up to
- /// [`Cmdq::RECEIVE_TIMEOUT`] from the call.
+ /// Waits for the response to the GMC request with command id `command_id` and RPC sequence
+ /// number `sequence`, up to [`Cmdq::RECEIVE_TIMEOUT`] from the call.
///
/// The response's payload is passed to `decode`, as two slices because the ring may wrap.
/// Every other GMC element that arrives first is passed to `on_other` with the headers that
@@ -1321,6 +1334,7 @@ fn receive_gmc_and_dispatch<R>(
fn await_gmc_response<R>(
&mut self,
command_id: u32,
+ sequence: u32,
mut on_other: impl FnMut(&GspGmcMsgElement, &[u8], &[u8]) -> Result,
mut decode: impl FnMut(&[u8], &[u8]) -> Result<R>,
) -> Result<R> {
@@ -1334,7 +1348,7 @@ fn await_gmc_response<R>(
let response =
self.receive_gmc_and_dispatch(remaining, |header, payload_0, payload_1| {
- if header.gmc.command_id() != command_id {
+ if !header.gmc.is_response_to(command_id, sequence) {
return on_other(header, payload_0, payload_1).map(|()| None);
}
diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs
index 128d6f8dcb43..f1a3c0613e70 100644
--- a/drivers/gpu/nova-core/gsp/commands.rs
+++ b/drivers/gpu/nova-core/gsp/commands.rs
@@ -79,10 +79,12 @@ pub(crate) fn gsp_init(
// Qualified because `zerocopy::IntoBytes` also gives `[T]` an `as_bytes`.
let payload = AsBytes::as_bytes(payload);
- cmdq.send_gmc_no_wait(GMCAPI_CMD_GSP_INIT, payload, GSP_INIT_MAX_RESPONSE_SIZE)?;
+ let sequence =
+ cmdq.send_gmc_no_wait(GMCAPI_CMD_GSP_INIT, payload, GSP_INIT_MAX_RESPONSE_SIZE)?;
cmdq.await_gmc_response(
GMCAPI_CMD_GSP_INIT,
+ sequence,
on_unsolicited_element,
decode_gsp_init_reply,
)
@@ -124,9 +126,9 @@ fn decode_gsp_init_reply(payload_0: &[u8], payload_1: &[u8]) -> Result<GspStatic
///
/// # Errors
///
-/// Errors from [`Cmdq::send_gmc_no_wait`] are propagated as-is.
+/// Errors from [`Cmdq::send_gmc_no_reply`] are propagated as-is.
pub(crate) fn gsp_suspend(cmdq: &Cmdq<'_>, level: PowerStateLevel) -> Result {
let params = fw::commands::GspSuspend::new(level);
- cmdq.send_gmc_no_wait(GMCAPI_CMD_GSP_SUSPEND, AsBytes::as_bytes(¶ms), 0)
+ cmdq.send_gmc_no_reply(GMCAPI_CMD_GSP_SUSPEND, AsBytes::as_bytes(¶ms))
}
diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs
index f3dff49af2bf..f7e83e75e53e 100644
--- a/drivers/gpu/nova-core/gsp/fw.rs
+++ b/drivers/gpu/nova-core/gsp/fw.rs
@@ -725,6 +725,9 @@ pub(crate) struct GmcApiHeader {
/// Bits of [`GmcApiHeader::command`] that hold the command id. The high byte holds flags.
const GMCAPI_COMMAND_ID_MASK: u32 = 0x00ff_ffff;
+/// Flag bit of [`GmcApiHeader::command`] that GSP-RM sets on a response.
+const GMCAPI_COMMAND_FLAGS_RESPONSE: u32 = 0x0100_0000;
+
/// GMC request that carries the system information and registry keys to GSP-RM. GSP-RM answers
/// it with the static GPU configuration once it has finished starting.
pub(crate) const GMCAPI_CMD_GSP_INIT: u32 = bindings::GMCAPI_COMMANDS_GMCAPI_CMD_GSP_INIT;
@@ -771,13 +774,26 @@ pub(crate) fn command_id(&self) -> u32 {
self.command & GMCAPI_COMMAND_ID_MASK
}
+ /// Returns `true` if GSP-RM sent this header as a response rather than an event.
+ fn is_response(&self) -> bool {
+ self.command & GMCAPI_COMMAND_FLAGS_RESPONSE != 0
+ }
+
/// Returns the `NV_STATUS` that a response carries.
///
- /// The value is meaningful only on a response, which GSP-RM marks with a flag in the command
- /// word. In a request, the same word holds the largest response that the sender accepts.
+ /// The value is meaningful only when [`Self::is_response`] is `true`. In a request, the same
+ /// word holds the largest response that the sender accepts.
pub(crate) fn status(&self) -> u32 {
self.max_resp_or_status
}
+
+ /// Returns `true` if this header answers the request with command id `command_id` and RPC
+ /// sequence number `sequence`.
+ pub(crate) fn is_response_to(&self, command_id: u32, sequence: u32) -> bool {
+ self.is_response()
+ && self.command_id() == command_id
+ && self.sequence == u64::from(sequence)
+ }
}
// SAFETY: All fields are integer types with no uninitialized padding bytes.
@@ -801,7 +817,7 @@ pub(crate) struct GspGmcMsgElement {
impl GspGmcMsgElement {
/// Creates the queue element header and the GMC API header of a request that carries
- /// `payload_size` bytes of payload.
+ /// `payload_size` bytes of payload under the RPC sequence number `sequence`.
///
/// `max_response_size` is the largest response that the sender accepts, and zero for a request
/// that GSP-RM does not answer.
@@ -811,7 +827,7 @@ impl GspGmcMsgElement {
/// - `EOVERFLOW` if a length does not fit its 32-bit field.
pub(crate) fn init(
command_id: u32,
- sequence: u64,
+ sequence: u32,
payload_size: usize,
max_response_size: u32,
) -> impl Init<Self, Error> {
@@ -825,7 +841,7 @@ pub(crate) fn init(
gmc: GmcApiHeader {
command: command_id,
size: payload_size.try_into().map_err(|_| EOVERFLOW)?,
- sequence,
+ sequence: u64::from(sequence),
max_resp_or_status: max_response_size,
reserved: [0; 5],
},
--
2.55.0
^ permalink raw reply [flat|nested] 35+ messages in thread