* [PATCH 0/1] gpu: nova-core: populate GSP_VF_INFO for vGPU @ 2026-07-30 18:14 Zhi Wang 2026-07-30 18:14 ` [PATCH 1/1] gpu: nova-core: populate GSP_VF_INFO when vGPU is enabled Zhi Wang 0 siblings, 1 reply; 4+ messages in thread From: Zhi Wang @ 2026-07-30 18:14 UTC (permalink / raw) To: dakr, acourbot Cc: airlied, simona, ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin, a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, joelagnelf, apopple, cjia, smitra, kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu, linux-kernel, zhiwang, Zhi Wang This is a follow-up to the "Boot GSP with vGPU enabled" series [1]. It provides GSP-RM with the VF topology required for per-VF event routing. This series depends on the separately posted "[PATCH v5 0/1] Rust PCI capability infrastructure and SR-IOV support" series [2], which provides the VF BAR decoding API used by this patch. [1] https://lore.kernel.org/nova-gpu/20260709150206.1046839-1-zhiw@nvidia.com/ [2] https://lore.kernel.org/rust-for-linux/20260730180349.771719-1-zhiw@nvidia.com/T/#t Zhi Wang (1): gpu: nova-core: populate GSP_VF_INFO when vGPU is enabled drivers/gpu/nova-core/gsp/boot.rs | 6 ++-- drivers/gpu/nova-core/gsp/commands.rs | 15 ++++++++-- drivers/gpu/nova-core/gsp/fw.rs | 4 +++ drivers/gpu/nova-core/gsp/fw/commands.rs | 35 +++++++++++++++++++++++- 4 files changed, 54 insertions(+), 6 deletions(-) base-commit: 2735e4c4adbae7e1eade2230d43c7f500915c46c -- 2.53.0 ^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH 1/1] gpu: nova-core: populate GSP_VF_INFO when vGPU is enabled 2026-07-30 18:14 [PATCH 0/1] gpu: nova-core: populate GSP_VF_INFO for vGPU Zhi Wang @ 2026-07-30 18:14 ` Zhi Wang 2026-07-30 21:28 ` Danilo Krummrich 2026-08-10 2:57 ` Alexandre Courbot 0 siblings, 2 replies; 4+ messages in thread From: Zhi Wang @ 2026-07-30 18:14 UTC (permalink / raw) To: dakr, acourbot Cc: airlied, simona, ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin, a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, joelagnelf, apopple, cjia, smitra, kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu, linux-kernel, zhiwang, Zhi Wang GSP firmware requires VF BAR layout information to correctly route interrupts and doorbell events to individual virtual functions. Without this data the firmware cannot set up per-VF event routing. When vGPU mode is enabled, read the VF topology from the typed PCI SR-IOV extended capability and populate the GSP_VF_INFO embedded in the GSP_SYSTEM_INFO command. Register GpuacctPerfmonUtilSamples in MsgFunction so the receive path recognizes vGPU perfmon sampling events from GSP-RM. Signed-off-by: Zhi Wang <zhiw@nvidia.com> --- drivers/gpu/nova-core/gsp/boot.rs | 6 ++-- drivers/gpu/nova-core/gsp/commands.rs | 15 ++++++++-- drivers/gpu/nova-core/gsp/fw.rs | 4 +++ drivers/gpu/nova-core/gsp/fw/commands.rs | 35 +++++++++++++++++++++++- 4 files changed, 54 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs index 727b8ae4bcb7..c086026dc2d8 100644 --- a/drivers/gpu/nova-core/gsp/boot.rs +++ b/drivers/gpu/nova-core/gsp/boot.rs @@ -86,8 +86,10 @@ pub(crate) fn boot( dev_dbg!(pdev, "RISC-V active? {}\n", gsp_falcon.is_riscv_active(),); - self.cmdq - .send_command_no_wait(bar, commands::SetSystemInfo::new(pdev, chipset))?; + self.cmdq.send_command_no_wait( + bar, + commands::SetSystemInfo::new(pdev, chipset, ctx.vgpu.state()), + )?; self.cmdq .send_command_no_wait(bar, commands::SetRegistry::new(ctx.vgpu.state())?)?; diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs index ffc25fd8c47b..11d0c7ec371e 100644 --- a/drivers/gpu/nova-core/gsp/commands.rs +++ b/drivers/gpu/nova-core/gsp/commands.rs @@ -41,12 +41,21 @@ pub(crate) struct SetSystemInfo<'a> { pdev: &'a pci::Device<device::Bound>, chipset: Chipset, + vgpu_state: VgpuState, } impl<'a> SetSystemInfo<'a> { /// Creates a new `GspSetSystemInfo` command using the parameters of `pdev`. - pub(crate) fn new(pdev: &'a pci::Device<device::Bound>, chipset: Chipset) -> Self { - Self { pdev, chipset } + pub(crate) fn new( + pdev: &'a pci::Device<device::Bound>, + chipset: Chipset, + vgpu_state: VgpuState, + ) -> Self { + Self { + pdev, + chipset, + vgpu_state, + } } } @@ -57,7 +66,7 @@ impl<'a> CommandToGsp for SetSystemInfo<'a> { type InitError = Error; fn init(&self) -> impl Init<Self::Command, Self::InitError> { - Self::Command::init(self.pdev, self.chipset) + Self::Command::init(self.pdev, self.chipset, self.vgpu_state) } } diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs index 6e8e7d822ef1..6afe08abd868 100644 --- a/drivers/gpu/nova-core/gsp/fw.rs +++ b/drivers/gpu/nova-core/gsp/fw.rs @@ -254,6 +254,7 @@ pub(crate) enum MsgFunction { GspLockdownNotice = bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE, GspPostNoCat = bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD, GspRunCpuSequencer = bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER, + GpuacctPerfmonUtilSamples = bindings::NV_VGPU_MSG_EVENT_GPUACCT_PERFMON_UTIL_SAMPLES, MmuFaultQueued = bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED, OsErrorLog = bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG, PostEvent = bindings::NV_VGPU_MSG_EVENT_POST_EVENT, @@ -303,6 +304,9 @@ fn try_from(value: u32) -> Result<MsgFunction> { bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER => { Ok(MsgFunction::GspRunCpuSequencer) } + bindings::NV_VGPU_MSG_EVENT_GPUACCT_PERFMON_UTIL_SAMPLES => { + Ok(MsgFunction::GpuacctPerfmonUtilSamples) + } bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED => Ok(MsgFunction::MmuFaultQueued), bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG => Ok(MsgFunction::OsErrorLog), bindings::NV_VGPU_MSG_EVENT_POST_EVENT => Ok(MsgFunction::PostEvent), diff --git a/drivers/gpu/nova-core/gsp/fw/commands.rs b/drivers/gpu/nova-core/gsp/fw/commands.rs index 6dc31d1bf5ae..edf21aa8c61b 100644 --- a/drivers/gpu/nova-core/gsp/fw/commands.rs +++ b/drivers/gpu/nova-core/gsp/fw/commands.rs @@ -16,11 +16,42 @@ use crate::{ gpu::Chipset, gsp::GSP_PAGE_SIZE, - num::IntoSafeCast, // + num::IntoSafeCast, + vgpu::VgpuState, // }; use super::bindings; +/// Builds the VF topology passed to GSP-RM as part of the system information. +fn build_vf_info( + dev: &pci::Device<device::Bound>, + vgpu_state: VgpuState, +) -> Result<bindings::GSP_VF_INFO> { + let VgpuState::Enabled { total_vfs } = vgpu_state else { + return Ok(Default::default()); + }; + + let sriov = dev + .config_space_extended()? + .find_ext_capability::<pci::ExtSriovRegs>()?; + + let bar0 = sriov.read_vf_bar(0)?; + let bar1 = sriov.read_vf_bar(bar0.next_index())?; + let bar2 = sriov.read_vf_bar(bar1.next_index())?; + + Ok(bindings::GSP_VF_INFO { + totalVFs: u32::from(total_vfs.get()), + firstVFOffset: u32::from(kernel::io_read!(sriov, .vf_offset)), + FirstVFBar0Address: bar0.address(), + FirstVFBar1Address: bar1.address(), + FirstVFBar2Address: bar2.address(), + b64bitBar0: u8::from(bar0.is_64bit()), + b64bitBar1: u8::from(bar1.is_64bit()), + b64bitBar2: u8::from(bar2.is_64bit()), + ..Zeroable::zeroed() + }) +} + /// Payload of the `GspSetSystemInfo` command. #[repr(transparent)] pub(crate) struct GspSetSystemInfo { @@ -33,6 +64,7 @@ impl GspSetSystemInfo { pub(crate) fn init<'a>( dev: &'a pci::Device<device::Bound>, chipset: Chipset, + vgpu_state: VgpuState, ) -> impl Init<Self, Error> + 'a { type InnerGspSystemInfo = bindings::GspSystemInfo; let pci_config_mirror_range = chipset.pci_config_mirror_range(); @@ -54,6 +86,7 @@ pub(crate) fn init<'a>( PCIRevisionID: u32::from(dev.revision_id()), bIsPrimary: 0, bPreserveVideoMemoryAllocations: 0, + gspVFInfo: build_vf_info(dev, vgpu_state)?, ..Zeroable::init_zeroed() }); -- 2.53.0 ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] gpu: nova-core: populate GSP_VF_INFO when vGPU is enabled 2026-07-30 18:14 ` [PATCH 1/1] gpu: nova-core: populate GSP_VF_INFO when vGPU is enabled Zhi Wang @ 2026-07-30 21:28 ` Danilo Krummrich 2026-08-10 2:57 ` Alexandre Courbot 1 sibling, 0 replies; 4+ messages in thread From: Danilo Krummrich @ 2026-07-30 21:28 UTC (permalink / raw) To: Zhi Wang Cc: acourbot, airlied, simona, ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin, a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, joelagnelf, apopple, cjia, smitra, kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu, linux-kernel, zhiwang On Thu Jul 30, 2026 at 8:14 PM CEST, Zhi Wang wrote: > Register GpuacctPerfmonUtilSamples in MsgFunction so the receive path > recognizes vGPU perfmon sampling events from GSP-RM. Listing something additional in the commit message usually indicates a separate patch, but I think in this case it is fine as is. > +/// Builds the VF topology passed to GSP-RM as part of the system information. > +fn build_vf_info( > + dev: &pci::Device<device::Bound>, > + vgpu_state: VgpuState, > +) -> Result<bindings::GSP_VF_INFO> { I'd make this an associated function of GspSetSystemInfo; it solely serves that struct's construction, as your own comment already states. :) ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] gpu: nova-core: populate GSP_VF_INFO when vGPU is enabled 2026-07-30 18:14 ` [PATCH 1/1] gpu: nova-core: populate GSP_VF_INFO when vGPU is enabled Zhi Wang 2026-07-30 21:28 ` Danilo Krummrich @ 2026-08-10 2:57 ` Alexandre Courbot 1 sibling, 0 replies; 4+ messages in thread From: Alexandre Courbot @ 2026-08-10 2:57 UTC (permalink / raw) To: Zhi Wang Cc: dakr, airlied, simona, ojeda, alex.gaynor, boqun.feng, gary, bjorn3_gh, lossin, a.hindborg, aliceryhl, tmgross, jhubbard, ecourtney, joelagnelf, apopple, cjia, smitra, kjaju, alkumar, ankita, aniketa, kwankhede, targupta, nova-gpu, linux-kernel, zhiwang On Fri Jul 31, 2026 at 3:14 AM JST, Zhi Wang wrote: > GSP firmware requires VF BAR layout information to correctly route > interrupts and doorbell events to individual virtual functions. > Without this data the firmware cannot set up per-VF event routing. > > When vGPU mode is enabled, read the VF topology from the typed PCI > SR-IOV extended capability and populate the GSP_VF_INFO embedded in > the GSP_SYSTEM_INFO command. > > Register GpuacctPerfmonUtilSamples in MsgFunction so the receive path > recognizes vGPU perfmon sampling events from GSP-RM. > > Signed-off-by: Zhi Wang <zhiw@nvidia.com> > --- > drivers/gpu/nova-core/gsp/boot.rs | 6 ++-- > drivers/gpu/nova-core/gsp/commands.rs | 15 ++++++++-- > drivers/gpu/nova-core/gsp/fw.rs | 4 +++ > drivers/gpu/nova-core/gsp/fw/commands.rs | 35 +++++++++++++++++++++++- > 4 files changed, 54 insertions(+), 6 deletions(-) > > diff --git a/drivers/gpu/nova-core/gsp/boot.rs b/drivers/gpu/nova-core/gsp/boot.rs > index 727b8ae4bcb7..c086026dc2d8 100644 > --- a/drivers/gpu/nova-core/gsp/boot.rs > +++ b/drivers/gpu/nova-core/gsp/boot.rs > @@ -86,8 +86,10 @@ pub(crate) fn boot( > > dev_dbg!(pdev, "RISC-V active? {}\n", gsp_falcon.is_riscv_active(),); > > - self.cmdq > - .send_command_no_wait(bar, commands::SetSystemInfo::new(pdev, chipset))?; > + self.cmdq.send_command_no_wait( > + bar, > + commands::SetSystemInfo::new(pdev, chipset, ctx.vgpu.state()), > + )?; > self.cmdq > .send_command_no_wait(bar, commands::SetRegistry::new(ctx.vgpu.state())?)?; > > diff --git a/drivers/gpu/nova-core/gsp/commands.rs b/drivers/gpu/nova-core/gsp/commands.rs > index ffc25fd8c47b..11d0c7ec371e 100644 > --- a/drivers/gpu/nova-core/gsp/commands.rs > +++ b/drivers/gpu/nova-core/gsp/commands.rs > @@ -41,12 +41,21 @@ > pub(crate) struct SetSystemInfo<'a> { > pdev: &'a pci::Device<device::Bound>, > chipset: Chipset, > + vgpu_state: VgpuState, > } > > impl<'a> SetSystemInfo<'a> { > /// Creates a new `GspSetSystemInfo` command using the parameters of `pdev`. > - pub(crate) fn new(pdev: &'a pci::Device<device::Bound>, chipset: Chipset) -> Self { > - Self { pdev, chipset } > + pub(crate) fn new( > + pdev: &'a pci::Device<device::Bound>, > + chipset: Chipset, > + vgpu_state: VgpuState, > + ) -> Self { > + Self { > + pdev, > + chipset, > + vgpu_state, > + } > } > } > > @@ -57,7 +66,7 @@ impl<'a> CommandToGsp for SetSystemInfo<'a> { > type InitError = Error; > > fn init(&self) -> impl Init<Self::Command, Self::InitError> { > - Self::Command::init(self.pdev, self.chipset) > + Self::Command::init(self.pdev, self.chipset, self.vgpu_state) > } > } > > diff --git a/drivers/gpu/nova-core/gsp/fw.rs b/drivers/gpu/nova-core/gsp/fw.rs > index 6e8e7d822ef1..6afe08abd868 100644 > --- a/drivers/gpu/nova-core/gsp/fw.rs > +++ b/drivers/gpu/nova-core/gsp/fw.rs > @@ -254,6 +254,7 @@ pub(crate) enum MsgFunction { > GspLockdownNotice = bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE, > GspPostNoCat = bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD, > GspRunCpuSequencer = bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER, > + GpuacctPerfmonUtilSamples = bindings::NV_VGPU_MSG_EVENT_GPUACCT_PERFMON_UTIL_SAMPLES, > MmuFaultQueued = bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED, > OsErrorLog = bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG, > PostEvent = bindings::NV_VGPU_MSG_EVENT_POST_EVENT, > @@ -303,6 +304,9 @@ fn try_from(value: u32) -> Result<MsgFunction> { > bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER => { > Ok(MsgFunction::GspRunCpuSequencer) > } > + bindings::NV_VGPU_MSG_EVENT_GPUACCT_PERFMON_UTIL_SAMPLES => { > + Ok(MsgFunction::GpuacctPerfmonUtilSamples) > + } > bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED => Ok(MsgFunction::MmuFaultQueued), > bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG => Ok(MsgFunction::OsErrorLog), > bindings::NV_VGPU_MSG_EVENT_POST_EVENT => Ok(MsgFunction::PostEvent), > diff --git a/drivers/gpu/nova-core/gsp/fw/commands.rs b/drivers/gpu/nova-core/gsp/fw/commands.rs > index 6dc31d1bf5ae..edf21aa8c61b 100644 > --- a/drivers/gpu/nova-core/gsp/fw/commands.rs > +++ b/drivers/gpu/nova-core/gsp/fw/commands.rs > @@ -16,11 +16,42 @@ > use crate::{ > gpu::Chipset, > gsp::GSP_PAGE_SIZE, > - num::IntoSafeCast, // > + num::IntoSafeCast, > + vgpu::VgpuState, // > }; > > use super::bindings; > > +/// Builds the VF topology passed to GSP-RM as part of the system information. > +fn build_vf_info( > + dev: &pci::Device<device::Bound>, > + vgpu_state: VgpuState, > +) -> Result<bindings::GSP_VF_INFO> { > + let VgpuState::Enabled { total_vfs } = vgpu_state else { > + return Ok(Default::default()); > + }; > + > + let sriov = dev > + .config_space_extended()? > + .find_ext_capability::<pci::ExtSriovRegs>()?; > + > + let bar0 = sriov.read_vf_bar(0)?; > + let bar1 = sriov.read_vf_bar(bar0.next_index())?; > + let bar2 = sriov.read_vf_bar(bar1.next_index())?; This basically confirms the discussion we had on the SR-IOV thread: the iterator pattern is the correct one here. The patch itself looks good, so I think we can merge it as soon as the dependency is in and Danilo's feedback is addressed. ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-08-10 2:58 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2026-07-30 18:14 [PATCH 0/1] gpu: nova-core: populate GSP_VF_INFO for vGPU Zhi Wang 2026-07-30 18:14 ` [PATCH 1/1] gpu: nova-core: populate GSP_VF_INFO when vGPU is enabled Zhi Wang 2026-07-30 21:28 ` Danilo Krummrich 2026-08-10 2:57 ` Alexandre Courbot
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®