mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Zhi Wang <zhiw@nvidia.com>
To: <dakr@kernel.org>, <acourbot@nvidia.com>
Cc: <alex@shazbot.org>, <jgg@nvidia.com>, <yishaih@nvidia.com>,
	<skolothumtho@nvidia.com>, <kevin.tian@intel.com>,
	<airlied@gmail.com>, <simona@ffwll.ch>, <ojeda@kernel.org>,
	<alex.gaynor@gmail.com>, <boqun.feng@gmail.com>,
	<gary@garyguo.net>, <bjorn3_gh@protonmail.com>,
	<lossin@kernel.org>, <a.hindborg@kernel.org>,
	<aliceryhl@google.com>, <tmgross@umich.edu>,
	<jhubbard@nvidia.com>, <ecourtney@nvidia.com>, <cjia@nvidia.com>,
	<smitra@nvidia.com>, <kjaju@nvidia.com>, <alkumar@nvidia.com>,
	<ankita@nvidia.com>, <aniketa@nvidia.com>, <kwankhede@nvidia.com>,
	<targupta@nvidia.com>, <nova-gpu@lists.linux.dev>,
	<linux-kernel@vger.kernel.org>, <zhiwang@kernel.org>,
	Zhi Wang <zhiw@nvidia.com>
Subject: [PATCH v2 23/32] gpu: nova-core: vgpu: add GSP plugin communication buffers
Date: Mon, 14 Sep 2026 10:48:58 +0300	[thread overview]
Message-ID: <20260914074907.1399629-24-zhiw@nvidia.com> (raw)
In-Reply-To: <20260914074907.1399629-1-zhiw@nvidia.com>

The GSP plugin uses a communication region in its management heap for
control data and logs. The vGPU manager reads its boot-ready marker through
BAR1.

Map this region with BarMapping and expose the firmware-defined control
and log regions. Keep the mapping's BarUser borrow until explicit teardown.

Signed-off-by: Zhi Wang <zhiw@nvidia.com>
---
 drivers/gpu/nova-core/mm.rs                   |   1 -
 drivers/gpu/nova-core/vgpu.rs                 |   1 +
 drivers/gpu/nova-core/vgpu/fw.rs              |  15 ++
 drivers/gpu/nova-core/vgpu/gsp_plugin_comm.rs | 204 ++++++++++++++++++
 4 files changed, 220 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/nova-core/vgpu/gsp_plugin_comm.rs

diff --git a/drivers/gpu/nova-core/mm.rs b/drivers/gpu/nova-core/mm.rs
index 016d99608aeb..d1cad5ad52fc 100644
--- a/drivers/gpu/nova-core/mm.rs
+++ b/drivers/gpu/nova-core/mm.rs
@@ -67,7 +67,6 @@ macro_rules! impl_pfn_bounded {
 mod regs;
 pub(super) mod tlb;
 pub(super) mod vmm;
-#[expect(dead_code)]
 pub(crate) mod vram;
 
 /// GPU Memory Manager - owns all core MM components.
diff --git a/drivers/gpu/nova-core/vgpu.rs b/drivers/gpu/nova-core/vgpu.rs
index f3e40e6de6f0..8919322dc057 100644
--- a/drivers/gpu/nova-core/vgpu.rs
+++ b/drivers/gpu/nova-core/vgpu.rs
@@ -24,6 +24,7 @@
 
 mod commands;
 mod fw;
+mod gsp_plugin_comm;
 mod hal;
 mod instance;
 mod vram;
diff --git a/drivers/gpu/nova-core/vgpu/fw.rs b/drivers/gpu/nova-core/vgpu/fw.rs
index 23097cb13121..aabe114b55c2 100644
--- a/drivers/gpu/nova-core/vgpu/fw.rs
+++ b/drivers/gpu/nova-core/vgpu/fw.rs
@@ -10,6 +10,21 @@
 
 use crate::gsp::bindings;
 
+pub(super) use bindings::{
+    GSP_PLUGIN_BOOTLOADED,
+    VGPU_CPU_GSP_COMMUNICATION_BUFF_TOTAL_SIZE,
+    VGPU_CPU_GSP_CTRL_BUFF_REGION as RawControlRegion,
+    VGPU_CPU_GSP_CTRL_BUFF_REGION_SIZE,
+    VGPU_CPU_GSP_ERROR_BUFF_REGION_SIZE,
+    VGPU_CPU_GSP_GUEST_RPC_TRACE_BUFF_REGION_SIZE,
+    VGPU_CPU_GSP_INIT_TASK_LOG_BUFF_REGION_SIZE,
+    VGPU_CPU_GSP_KERNEL_TASK_LOG_BUFF_REGION_SIZE,
+    VGPU_CPU_GSP_MESSAGE_BUFF_REGION_SIZE,
+    VGPU_CPU_GSP_MIGRATION_BUFF_REGION_SIZE,
+    VGPU_CPU_GSP_RESPONSE_BUFF_REGION_SIZE,
+    VGPU_CPU_GSP_VGPU_TASK_LOG_BUFF_REGION_SIZE, //
+};
+
 pub(super) const GMCAPI_CMD_QUERY_ASSIGNED_VF_VGPU_TYPE: u32 =
     bindings::GMCAPI_COMMANDS_GMCAPI_CMD_QUERY_ASSIGNED_VF_VGPU_TYPE;
 
diff --git a/drivers/gpu/nova-core/vgpu/gsp_plugin_comm.rs b/drivers/gpu/nova-core/vgpu/gsp_plugin_comm.rs
new file mode 100644
index 000000000000..d2139309b6be
--- /dev/null
+++ b/drivers/gpu/nova-core/vgpu/gsp_plugin_comm.rs
@@ -0,0 +1,204 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+//! GSP plugin communication buffer mappings and access.
+
+use kernel::prelude::*;
+
+use crate::mm::{
+    bar_user::{
+        BarMapping,
+        BarUser, //
+    },
+    vram::VramRegion,
+    GpuMm, //
+};
+
+use super::fw::{
+    self,
+    RawControlRegion, //
+};
+
+/// Physical VRAM regions containing the vGPU plugin logs.
+pub(super) struct PluginLogRegions {
+    init: VramRegion,
+    vgpu: VramRegion,
+    kernel: VramRegion,
+}
+
+#[expect(dead_code)]
+impl PluginLogRegions {
+    pub(super) const fn init(&self) -> &VramRegion {
+        &self.init
+    }
+
+    pub(super) const fn vgpu(&self) -> &VramRegion {
+        &self.vgpu
+    }
+
+    pub(super) const fn kernel(&self) -> &VramRegion {
+        &self.kernel
+    }
+}
+
+fn take_region(region: &VramRegion, cursor: &mut u64, size: u32) -> Result<VramRegion> {
+    let end = cursor.checked_add(u64::from(size)).ok_or(EOVERFLOW)?;
+    let subregion = region.subregion(*cursor..end)?;
+    *cursor = end;
+    Ok(subregion)
+}
+
+/// BAR1 mapping of the plugin communication region in its management heap.
+///
+/// r000 layout, with byte offsets from the management heap (not to scale):
+///
+/// ```text
+/// 0x000000 +----------------------------------------+
+///          | Control (boot-ready marker)      4 KiB |
+/// 0x001000 +----------------------------------------+
+///          | Response                         4 KiB |
+/// 0x002000 +----------------------------------------+
+///          | Message                          4 KiB |
+/// 0x003000 +----------------------------------------+
+///          | Migration                        2 MiB |
+/// 0x203000 +----------------------------------------+
+///          | Error                            4 KiB |
+/// 0x204000 +----------------------------------------+
+///          | Init task log                  128 KiB |
+/// 0x224000 +----------------------------------------+
+///          | vGPU task log                  256 KiB |
+/// 0x264000 +----------------------------------------+
+///          | Kernel task log                 64 KiB |
+/// 0x274000 +----------------------------------------+
+///          | Guest RPC trace                 64 KiB |
+/// 0x284000 +----------------------------------------+
+/// ```
+pub(super) struct CommBufferRegion<'map, 'gpu> {
+    map: BarMapping<'map, 'gpu>,
+    control: VramRegion,
+    init_log: VramRegion,
+    vgpu_log: VramRegion,
+    kernel_log: VramRegion,
+}
+
+#[expect(dead_code)]
+impl<'map, 'gpu> CommBufferRegion<'map, 'gpu> {
+    /// Map the communication portion of a plugin management heap.
+    pub(super) fn new(
+        bar_user: &'map BarUser<'gpu>,
+        mm: &mut GpuMm<'_>,
+        management_heap: &VramRegion,
+    ) -> Result<Self> {
+        let total_size = u64::from(fw::VGPU_CPU_GSP_COMMUNICATION_BUFF_TOTAL_SIZE);
+        let region = management_heap.subregion(0..total_size)?;
+        let mut cursor = 0;
+
+        let control = take_region(&region, &mut cursor, fw::VGPU_CPU_GSP_CTRL_BUFF_REGION_SIZE)?;
+        take_region(
+            &region,
+            &mut cursor,
+            fw::VGPU_CPU_GSP_RESPONSE_BUFF_REGION_SIZE,
+        )?;
+        take_region(
+            &region,
+            &mut cursor,
+            fw::VGPU_CPU_GSP_MESSAGE_BUFF_REGION_SIZE,
+        )?;
+        take_region(
+            &region,
+            &mut cursor,
+            fw::VGPU_CPU_GSP_MIGRATION_BUFF_REGION_SIZE,
+        )?;
+        take_region(
+            &region,
+            &mut cursor,
+            fw::VGPU_CPU_GSP_ERROR_BUFF_REGION_SIZE,
+        )?;
+        let init_log = take_region(
+            &region,
+            &mut cursor,
+            fw::VGPU_CPU_GSP_INIT_TASK_LOG_BUFF_REGION_SIZE,
+        )?;
+        let vgpu_log = take_region(
+            &region,
+            &mut cursor,
+            fw::VGPU_CPU_GSP_VGPU_TASK_LOG_BUFF_REGION_SIZE,
+        )?;
+        let kernel_log = take_region(
+            &region,
+            &mut cursor,
+            fw::VGPU_CPU_GSP_KERNEL_TASK_LOG_BUFF_REGION_SIZE,
+        )?;
+        take_region(
+            &region,
+            &mut cursor,
+            fw::VGPU_CPU_GSP_GUEST_RPC_TRACE_BUFF_REGION_SIZE,
+        )?;
+
+        if cursor != total_size || control.size() != u64::try_from(size_of::<RawControlRegion>())? {
+            return Err(EINVAL);
+        }
+
+        let map = BarMapping::new(bar_user, mm, region, true)?;
+
+        Ok(Self {
+            map,
+            control,
+            init_log,
+            vgpu_log,
+            kernel_log,
+        })
+    }
+
+    fn region_offset(&self, region: &VramRegion) -> Result<usize> {
+        let offset = region
+            .address()
+            .checked_sub(self.map.region().address())
+            .ok_or(EINVAL)?;
+        if offset.checked_add(region.size()).ok_or(EOVERFLOW)? > self.map.region().size() {
+            return Err(EINVAL);
+        }
+
+        usize::try_from(offset).map_err(|_| EOVERFLOW)
+    }
+
+    fn io_offset(&self, region: &VramRegion, field: usize, width: usize) -> Result<usize> {
+        let field_end = field.checked_add(width).ok_or(EOVERFLOW)?;
+        if u64::try_from(field_end).map_err(|_| EOVERFLOW)? > region.size() {
+            return Err(EINVAL);
+        }
+
+        self.region_offset(region)?
+            .checked_add(field)
+            .ok_or(EOVERFLOW)
+    }
+
+    fn read_u32(&self, region: &VramRegion, field: usize) -> Result<u32> {
+        self.map
+            .try_read32(self.io_offset(region, field, size_of::<u32>())?)
+    }
+
+    /// Return the physical regions occupied by the three plugin logs.
+    pub(super) fn plugin_logs(&self) -> PluginLogRegions {
+        PluginLogRegions {
+            init: self.init_log.clone(),
+            vgpu: self.vgpu_log.clone(),
+            kernel: self.kernel_log.clone(),
+        }
+    }
+
+    /// Return whether firmware has published the plugin boot marker.
+    pub(super) fn is_plugin_ready(&self) -> Result<bool> {
+        let value = self.read_u32(
+            &self.control,
+            core::mem::offset_of!(RawControlRegion, __bindgen_anon_1.message_seq_num),
+        )?;
+
+        Ok(value == fw::GSP_PLUGIN_BOOTLOADED)
+    }
+
+    /// Invalidate the PTEs and release the communication mapping.
+    pub(super) fn destroy(self, mm: &mut GpuMm<'_>) -> Result {
+        self.map.destroy(mm)
+    }
+}

  parent reply	other threads:[~2026-09-14  7:53 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-14  7:48 [PATCH v2 00/32] Introduce NVIDIA vGPU manager Zhi Wang
2026-09-14  7:48 ` [PATCH v2 01/32] gpu: nova-core: gsp: pass boot context through setup helpers Zhi Wang
2026-09-14  7:48 ` [PATCH v2 02/32] gpu: nova-core: gsp: decouple boot context from VgpuManager Zhi Wang
2026-09-14  7:48 ` [PATCH v2 03/32] gpu: nova-core: vgpu: detect boot state independently Zhi Wang
2026-09-14  7:48 ` [PATCH v2 04/32] gpu: nova-core: gpu: add a channel ID pool for vGPU Zhi Wang
2026-09-14  7:48 ` [PATCH v2 05/32] gpu: nova-core: gsp: expose the VMMU segment size Zhi Wang
2026-09-14  7:48 ` [PATCH v2 06/32] gpu: nova-core: gsp: decode the FIFO engine table Zhi Wang
2026-09-14  7:48 ` [PATCH v2 07/32] gpu: nova-core: vgpu: initialize runtime parameters after GSP boot Zhi Wang
2026-09-14  7:48 ` [PATCH v2 08/32] gpu: nova-core: vgpu: reserve the 48-VM WPR2 heap Zhi Wang
2026-09-14  7:48 ` [PATCH v2 09/32] gpu: nova-core: mm: borrow BarUser for temporary BAR1 access Zhi Wang
2026-09-14  7:48 ` [PATCH v2 10/32] gpu: nova-core: mm: add VramBlock Zhi Wang
2026-09-14  7:48 ` [PATCH v2 11/32] gpu: nova-core: mm: add VramRegion Zhi Wang
2026-09-14  7:48 ` [PATCH v2 12/32] gpu: nova-core: mm: add BarMapping Zhi Wang
2026-09-14  7:48 ` [PATCH v2 13/32] gpu: nova-core: gsp: dispatch RM RPC messages during GMC receive Zhi Wang
2026-09-14  7:48 ` [PATCH v2 14/32] gpu: nova-core: gsp: add synchronous GMC transactions Zhi Wang
2026-09-14  7:48 ` [PATCH v2 15/32] gpu: nova-core: gsp: add status-only " Zhi Wang
2026-09-14  7:48 ` [PATCH v2 16/32] gpu: nova-core: gsp: wait for GMC completion events Zhi Wang
2026-09-14  7:48 ` [PATCH v2 17/32] gpu: nova-core: vgpu: add r000 plugin bindings Zhi Wang
2026-09-14  7:48 ` [PATCH v2 18/32] gpu: nova-core: vgpu: add VRAM slot allocator Zhi Wang
2026-09-14  7:48 ` [PATCH v2 19/32] gpu: nova-core: gsp: factor out NVKV payload conversion Zhi Wang
2026-09-14  7:48 ` [PATCH v2 20/32] gpu: nova-core: vgpu: query VF assignments and properties Zhi Wang
2026-09-14  7:48 ` [PATCH v2 21/32] gpu: nova-core: vgpu: add instance create/destroy Zhi Wang
2026-09-14  7:48 ` [PATCH v2 22/32] gpu: nova-core: vgpu: encode vGPU boot requests Zhi Wang
2026-09-14  7:48 ` Zhi Wang [this message]
2026-09-14  7:48 ` [PATCH v2 24/32] gpu: nova-core: vgpu: add instance boot and shutdown Zhi Wang
2026-09-14  7:49 ` [PATCH v2 25/32] gpu: nova-core: vgpu: initialize GSP plugin RPC buffers Zhi Wang
2026-09-14  7:49 ` [PATCH v2 26/32] gpu: nova-core: vgpu: add GSP plugin RPC transactions Zhi Wang
2026-09-14  7:49 ` [PATCH v2 27/32] gpu: nova-core: vgpu: negotiate the GSP plugin RPC version Zhi Wang
2026-09-14  7:49 ` [PATCH v2 28/32] gpu: nova-core: vgpu: send GSP plugin configuration parameters Zhi Wang
2026-09-14  7:49 ` [PATCH v2 29/32] gpu: nova-core: vgpu: update the GSP plugin BME state Zhi Wang
2026-09-14  7:49 ` [PATCH v2 30/32] gpu: nova-core: vgpu: scrub guest VRAM with CeUtils Zhi Wang
2026-09-14  7:49 ` [PATCH v2 31/32] gpu: nova-core: add build ID headers to debugfs log buffer dumps Zhi Wang
2026-09-14  7:49 ` [PATCH v2 32/32] gpu: nova-core: vgpu: export plugin log buffers via debugfs Zhi Wang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260914074907.1399629-24-zhiw@nvidia.com \
    --to=zhiw@nvidia.com \
    --cc=a.hindborg@kernel.org \
    --cc=acourbot@nvidia.com \
    --cc=airlied@gmail.com \
    --cc=alex.gaynor@gmail.com \
    --cc=alex@shazbot.org \
    --cc=aliceryhl@google.com \
    --cc=alkumar@nvidia.com \
    --cc=aniketa@nvidia.com \
    --cc=ankita@nvidia.com \
    --cc=bjorn3_gh@protonmail.com \
    --cc=boqun.feng@gmail.com \
    --cc=cjia@nvidia.com \
    --cc=dakr@kernel.org \
    --cc=ecourtney@nvidia.com \
    --cc=gary@garyguo.net \
    --cc=jgg@nvidia.com \
    --cc=jhubbard@nvidia.com \
    --cc=kevin.tian@intel.com \
    --cc=kjaju@nvidia.com \
    --cc=kwankhede@nvidia.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lossin@kernel.org \
    --cc=nova-gpu@lists.linux.dev \
    --cc=ojeda@kernel.org \
    --cc=simona@ffwll.ch \
    --cc=skolothumtho@nvidia.com \
    --cc=smitra@nvidia.com \
    --cc=targupta@nvidia.com \
    --cc=tmgross@umich.edu \
    --cc=yishaih@nvidia.com \
    --cc=zhiwang@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®