mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Joel Fernandes <joelagnelf@nvidia.com>
To: linux-kernel@vger.kernel.org, Danilo Krummrich <dakr@kernel.org>,
	Maarten Lankhorst <maarten.lankhorst@linux.intel.com>,
	Maxime Ripard <mripard@kernel.org>,
	Thomas Zimmermann <tzimmermann@suse.de>,
	David Airlie <airlied@gmail.com>, Simona Vetter <simona@ffwll.ch>,
	Jonathan Corbet <corbet@lwn.net>
Cc: nouveau@lists.freedesktop.org, dri-devel@lists.freedesktop.org,
	Alexandre Courbot <acourbot@nvidia.com>,
	John Hubbard <jhubbard@nvidia.com>,
	Shirish Baskaran <sbaskaran@nvidia.com>,
	Alistair Popple <apopple@nvidia.com>,
	Timur Tabi <ttabi@nvidia.com>, Ben Skeggs <bskeggs@nvidia.com>,
	rust-for-linux@vger.kernel.org,
	Joel Fernandes <joelagnelf@nvidia.com>,
	linux-doc@vger.kernel.org
Subject: [PATCH v2 5/7] docs: nova-core: Document devinit process
Date: Sat,  3 May 2025 00:07:57 -0400	[thread overview]
Message-ID: <20250503040802.1411285-6-joelagnelf@nvidia.com> (raw)
In-Reply-To: <20250503040802.1411285-1-joelagnelf@nvidia.com>

devinit is mentioned in the code. This patch explains it so it is clear
what it does. devinit is not only essential at boot-time, but also at
runtime due to suspend-resume and things like re-clocking.

Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
---
 Documentation/gpu/nova/core/devinit.rst | 59 +++++++++++++++++++++++++
 Documentation/gpu/nova/index.rst        |  1 +
 drivers/gpu/nova-core/devinit.rs        |  6 +--
 3 files changed, 63 insertions(+), 3 deletions(-)
 create mode 100644 Documentation/gpu/nova/core/devinit.rst

diff --git a/Documentation/gpu/nova/core/devinit.rst b/Documentation/gpu/nova/core/devinit.rst
new file mode 100644
index 000000000000..85e7033dd3a7
--- /dev/null
+++ b/Documentation/gpu/nova/core/devinit.rst
@@ -0,0 +1,59 @@
+.. SPDX-License-Identifier: GPL-2.0
+==================================
+Device Initialization (devinit)
+==================================
+Note that devinit itself is a complicated process, subject to change, and this document
+only roughly provides an overview of the process using the Ampere GPU family as an example.
+The idea is to provide a conceptual overview of the process to help understand the kernel
+code that deals with it.
+
+Device initialization (devinit) is a crucial sequence of register read/write operations 
+that occur after a GPU reset. The devinit sequence is essential for properly configuring 
+the GPU hardware before it can be used.
+
+The devinit engine is an interpreter program that typically runs on the PMU (Power Management
+Unit) microcontroller of the GPU. This interpreter executes a "script" of initialization
+commands. The devinit engine itself is part of the VBIOS ROM in the same ROM image as the
+FWSEC (Firmware Security) image (see fwsec.rst and vbios.rst) and it runs before the
+nova-core driver is even loaded. On an Ampere GPU, the devinit ucode (which is separate
+from the FWSEC ucode), is launched by the FWSEC (FWSEC runs on GSP in heavy-secure mode
+and devinit runs on PMU in light-secure mode).
+
+Key Functions of devinit
+------------------------
+devinit performs several critical tasks:
+
+1. Programming VRAM memory controller timings
+2. Power sequencing
+3. Clock and PLL (Phase-Locked Loop) configuration
+4. Thermal management
+
+Low-level Firmware Initialization Flow
+--------------------------------------
+Upon reset, several microcontrollers on the GPU (such as PMU, SEC2, GSP, etc.) run GPU 
+firmware (gfw) code to set up the GPU and its core parameters. Most of the GPU is
+considered unusable until this initialization process completes.
+
+These low-level GPU firmware components are typically:
+1. Located in the VBIOS ROM in the same ROM partition (see vbios.rst and fwsec.rst).
+2. Executed in sequence on different microcontrollers:
+   - The devinit engine typically but not necessarily runs on the PMU.
+   - On an Ampere GPU, the FWSEC typically runs on the GSP (GPU System Processor) in
+     heavy-secure mode.
+
+Before the driver can proceed with further initialization, it must wait for a signal 
+indicating that core initialization is complete (known as GFW_BOOT). This signal is
+asserted by the FWSEC running on the GSP in heavy-secure mode.
+
+Runtime Considerations
+---------------------
+It's important to note that the devinit sequence also needs to run during suspend/resume 
+operations at runtime, not just during initial boot, as it is critical to power management.
+
+Security and Access Control
+--------------------------
+The initialization process involves careful privilege management. For example, before 
+accessing certain completion status registers, the driver must check privilege level 
+masks. Some registers are only accessible after secure firmware (FWSEC) lowers the 
+privilege level to allow CPU (LS/low-secure) access. For example, to receive the
+GFW_BOOT signal.
\ No newline at end of file
diff --git a/Documentation/gpu/nova/index.rst b/Documentation/gpu/nova/index.rst
index 22e5712ac6b0..301435c5cf67 100644
--- a/Documentation/gpu/nova/index.rst
+++ b/Documentation/gpu/nova/index.rst
@@ -29,4 +29,5 @@ vGPU manager VFIO driver and the nova-drm driver.
    core/guidelines
    core/vbios
    core/fwsec
+   core/devinit
    core/todo
diff --git a/drivers/gpu/nova-core/devinit.rs b/drivers/gpu/nova-core/devinit.rs
index 21fc9b96d498..bd6a5c57c444 100644
--- a/drivers/gpu/nova-core/devinit.rs
+++ b/drivers/gpu/nova-core/devinit.rs
@@ -10,8 +10,8 @@
 //! 3. Clock and PLL configuration.
 //! 4. Thermal management.
 //!
-//! devinit itself is a 'script' which is interpreted by the PMU microcontroller of
-//! the GPU by an interpreter program.
+//! devinit itself is a 'script' which is interpreted by a microcontroller of
+//! the GPU by an interpreter program (typically running on the PMU).
 //!
 //! Note that the devinit sequence also needs to run during suspend/resume at runtime.
 
@@ -30,7 +30,7 @@
 ///
 /// The GPU firmware (gfw) code includes several components that execute before the driver loads.
 /// These components are located in the VBIOS ROM and are executed in a sequence on these different
-/// microcontrollers. The devinit sequence itself runs on the PMU, and the FWSEC runs on the GSP.
+/// microcontrollers. The devinit sequence typically runs on the PMU, and the FWSEC runs on the GSP.
 ///
 /// This function specifically waits for a signal indicating core initialization is complete before
 /// which not much can be done. This signal is setup by the FWSEC running on the GSP in Heavy-secured
-- 
2.43.0


  parent reply	other threads:[~2025-05-03  4:08 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-05-03  4:07 [PATCH v2 0/7] Documentation for nova-core Joel Fernandes
2025-05-03  4:07 ` [PATCH v2 1/7] nova-core: doc: Add code comments related to devinit Joel Fernandes
2025-05-03  4:07 ` [PATCH v2 2/7] nova-core: doc: Clarify sysmembar operations Joel Fernandes
2025-05-03  4:07 ` [PATCH v2 3/7] nova-core: docs: Document vbios layout Joel Fernandes
2025-05-05  3:00   ` Bagas Sanjaya
2025-05-05  3:12     ` Bagas Sanjaya
2025-05-03  4:07 ` [PATCH v2 4/7] nova-core: docs: Document fwsec operation and layout Joel Fernandes
2025-05-05  3:52   ` Bagas Sanjaya
2025-05-06 16:26   ` Zhi Wang
2025-05-09 20:56     ` Joel Fernandes
2025-05-03  4:07 ` Joel Fernandes [this message]
2025-05-05  4:04   ` [PATCH v2 5/7] docs: nova-core: Document devinit process Bagas Sanjaya
2025-05-05 22:15     ` Joel Fernandes
2025-05-03  4:07 ` [PATCH v2 6/7] docs: nova-core: Document basics of the Falcon Joel Fernandes
2025-05-05  4:14   ` Bagas Sanjaya
2025-05-05 21:37     ` Joel Fernandes
2025-05-03  4:07 ` [PATCH v2 7/7] gpu: nova-core: Clarify falcon code Joel Fernandes
2025-05-06 16:21   ` Zhi Wang
2025-05-09 20:59     ` Joel Fernandes
2025-06-30 10:37 ` [PATCH v2 0/7] Documentation for nova-core Danilo Krummrich
2025-06-30 11:33   ` Alexandre Courbot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250503040802.1411285-6-joelagnelf@nvidia.com \
    --to=joelagnelf@nvidia.com \
    --cc=acourbot@nvidia.com \
    --cc=airlied@gmail.com \
    --cc=apopple@nvidia.com \
    --cc=bskeggs@nvidia.com \
    --cc=corbet@lwn.net \
    --cc=dakr@kernel.org \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=jhubbard@nvidia.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maarten.lankhorst@linux.intel.com \
    --cc=mripard@kernel.org \
    --cc=nouveau@lists.freedesktop.org \
    --cc=rust-for-linux@vger.kernel.org \
    --cc=sbaskaran@nvidia.com \
    --cc=simona@ffwll.ch \
    --cc=ttabi@nvidia.com \
    --cc=tzimmermann@suse.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®