[PATCH v2 00/12] gpu: nova-core: generic bootloader and falcon DMA changes for r000

[PATCH v2 00/12] gpu: nova-core: generic bootloader and falcon DMA changes for r000

[Alexandre Courbot]

This series contains patches 13, 14, 16 and 17 from John's r000 series
v3 [1], that prepare the generic falcon bootloader and the falcon DMA
code for the r000 boot protocol. They are submitted separately as they
do not depend on the command queue work, and are valuable on their own.
A few preparatory patches are also introduced to smoothen the
transition.

Patches 1 to 7 make the generic bootloader its own type, usable with any
DMEM descriptor and not just FWSEC's, and place it at the top of IMEM as
reported by the HWCFG register instead of assuming a 64 KiB IMEM. Patch
6 is a bit of an outlier as it allows register array locations to be
copied, in order to simplify patch 7 which makes sure the generic
bootloader can be used with the r000 boot protocol.

Patches 9 to 11 let falcon DMA transfers read from a framebuffer address
and tag DMEM blocks, and check the per-block offsets for overflow. These
three are extracted from a single commit into their own patches.

Apart from the bootloader placement, no functional change is introduced
by this series. The new DMA capabilities remain unused until the r000
load-and-execute handlers land.

Based on `drm-rust-next`, tested with TU106 and GA107 using both the
570.144 and 615.71.09 firmware images.

[1] https://lore.kernel.org/20260918010719.1176945-1-jhubbard@nvidia.com

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
Changes in v2:
- Remove `Assisted-by: LLM` tags.
- Widen the `IMEMC::offs` field to 24 bits to be able to load the
  bootloader even if IMEM is larger than 64KB. (Eliot)
- Accept unbound `Device` to create the generic bootloader. (Eliot)
- Fix documentation for GenericBootloader::new and FwsecFirmwareWithBl::new. (Eliot)
- Program `FBIF_TRANSCFG` from `GenericBootloader::run`. (Eliot)
- Bring `with_fbif_transcfg()` from the later r000 patches.
- Use the source offset as tag for tagged DMEM transfers. (Eliot,
  Sashiko)
- Set the `DMTAG` attribute from `with_falcon_mem`. (Eliot)
- Program the FBIF_TRANSCFG register in dma_wr. (Eliot)
- Link to v1: https://patch.msgid.link/20261103-r000-fw-v1-0-e3d38fc130e3@nvidia.com

Changes since John's v3:
- Split the IMEM placement change out of the bootloader extraction.
- Keep the DMEM descriptor fields private behind a constructor.
- Replace the exported bootloader pairing type with a `run()` method.
- Make `dma_wr()` able to work with different sources instead of adding
  a second DMA method.
- Split the DMEM tagging and overflow checks into their own patches.

---
Alexandre Courbot (6):
      gpu: nova-core: falcon: widen the IMEM PIO offset to 24 bits
      rust: io: register: implement Copy and Clone for RegisterArrayLoc
      gpu: nova-core: falcon: rename the local variables of dma_wr()
      gpu: nova-core: falcon: allow DMEM DMA transfers to request tagging
      gpu: nova-core: falcon: allow DMA transfers to read from a framebuffer address
      gpu: nova-core: falcon: program the FBIF_TRANSCFG register in dma_wr

John Hubbard (6):
      gpu: nova-core: separate the generic falcon bootloader from FWSEC
      gpu: nova-core: place the generic bootloader at the top of IMEM
      gpu: nova-core: move the bootloader DMEM descriptor out of FWSEC
      gpu: nova-core: run the generic bootloader from its own type
      gpu: nova-core: falcon: introduce and use with_fbif_transcfg
      gpu: nova-core: falcon: check the per-block DMA offsets for overflow

 drivers/gpu/nova-core/falcon.rs                    | 148 ++++++++---
 drivers/gpu/nova-core/firmware.rs                  |   1 +
 drivers/gpu/nova-core/firmware/fwsec/bootloader.rs | 220 +++-------------
 drivers/gpu/nova-core/firmware/gen_bootloader.rs   | 286 +++++++++++++++++++++
 drivers/gpu/nova-core/gsp/hal/tu102.rs             |   6 +-
 drivers/gpu/nova-core/regs.rs                      |  13 +-
 rust/kernel/io/register.rs                         |   8 +
 7 files changed, 449 insertions(+), 233 deletions(-)
---
base-commit: 7a6bc132f1b51437063c9203e7221067b40a791e
change-id: 20261103-r000-fw-088d39a643b7

Best regards,
--  
Alexandre Courbot <acourbot@nvidia.com>

[PATCH v2 01/12] gpu: nova-core: separate the generic falcon bootloader from FWSEC

[Alexandre Courbot]

From: John Hubbard <jhubbard@nvidia.com>

The generic falcon bootloader is a small program that loads a larger
image into a falcon. The r000 boot protocol needs it to run an image
that GSP-RM names in a load-and-execute event during boot.

Nova-core used the bootloader only to load FWSEC, so the bootloader code
was part of the FWSEC code, and there was no way to run it on another
image.

Move the bootloader into its own type, which FWSEC then uses.

No functional changes.

Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Zhi Wang <zhiw@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
[acourbot: document boot_addr(); split the change of IMEM placement into
 its own patch]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
 drivers/gpu/nova-core/firmware.rs                  |  1 +
 drivers/gpu/nova-core/firmware/fwsec/bootloader.rs | 60 ++-----------
 drivers/gpu/nova-core/firmware/gen_bootloader.rs   | 99 ++++++++++++++++++++++
 3 files changed, 107 insertions(+), 53 deletions(-)

diff --git a/drivers/gpu/nova-core/firmware.rs b/drivers/gpu/nova-core/firmware.rs
index d8f6509a35d1..358c9b8db0b8 100644
--- a/drivers/gpu/nova-core/firmware.rs
+++ b/drivers/gpu/nova-core/firmware.rs
@@ -24,6 +24,7 @@
 
 pub(crate) mod booter;
 pub(crate) mod fwsec;
+pub(crate) mod gen_bootloader;
 pub(crate) mod gsp;
 pub(crate) mod gsp_fmc;
 pub(crate) mod radix3;
diff --git a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
index 68858b48390e..80239301bbfa 100644
--- a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
@@ -17,14 +17,11 @@
         Io, //
     },
     prelude::*,
-    ptr::Alignable,
-    sizes,
     transmute::AsBytes,
 };
 
 use crate::{
     falcon::{
-        self,
         gsp::Gsp,
         Falcon,
         FalconBromParams,
@@ -38,10 +35,7 @@
     },
     firmware::{
         fwsec::FwsecFirmware,
-        tlv::{
-            request_tlv, //
-            Tlv,
-        },
+        gen_bootloader::GenericBootloader, //
     },
     gpu::Chipset,
     num::FromSafeCast, //
@@ -100,16 +94,12 @@ unsafe impl AsBytes for BootloaderDmemDescV2 {}
 pub(crate) struct FwsecFirmwareWithBl<'a> {
     /// DMA object the bootloader will copy the firmware from.
     _firmware_dma: Coherent<'a, [u8]>,
-    /// Code of the bootloader to be loaded into non-secure IMEM.
-    ucode: KVec<u8>,
+    /// The bootloader that copies the firmware.
+    bootloader: GenericBootloader,
     /// Descriptor to be loaded into DMEM for the bootloader to read.
     dmem_desc: BootloaderDmemDescV2,
-    /// Range-validated start offset of the firmware code in IMEM.
-    imem_dst_start: u16,
     /// BROM parameters of the loaded firmware.
     brom_params: FalconBromParams,
-    /// Range-validated `desc.start_tag`.
-    start_tag: u16,
 }
 
 impl<'a> FwsecFirmwareWithBl<'a> {
@@ -120,28 +110,7 @@ pub(crate) fn new(
         dev: &'a Device<device::Bound>,
         chipset: Chipset,
     ) -> Result<Self> {
-        let fw = request_tlv(dev, chipset, "gen_bootloader")?;
-        let tlv = Tlv::new(fw.data())?;
-        dev_dbg!(
-            dev,
-            "loaded generic bootloader firmware v{}\n",
-            tlv.get_string(b"VERS")?
-        );
-
-        let ucode = {
-            let blob = tlv.get_bytes(b"BLOB")?;
-            let code_size = usize::from_safe_cast(tlv.get_u32(b"CDSZ")?);
-            let code = blob.get(..code_size).ok_or(EINVAL)?;
-            let aligned_code_size = code_size
-                .align_up(cv!(falcon::MEM_BLOCK_ALIGNMENT))
-                .ok_or(EINVAL)?;
-
-            let mut ucode = KVec::with_capacity(aligned_code_size, GFP_KERNEL)?;
-            ucode.extend_from_slice(code, GFP_KERNEL)?;
-            ucode.resize(aligned_code_size, 0, GFP_KERNEL)?;
-
-            ucode
-        };
+        let bootloader = GenericBootloader::new(dev, chipset)?;
 
         // `BootloaderDmemDescV2` expects the source to be a mirror image of the destination and
         // uses the same offset parameter for both.
@@ -211,21 +180,11 @@ pub(crate) fn new(
             }
         };
 
-        // The bootloader's code must be loaded in the area right below the first 64K of IMEM.
-        const BOOTLOADER_LOAD_CEILING: usize = sizes::SZ_64K;
-        let imem_dst_start = BOOTLOADER_LOAD_CEILING
-            .checked_sub(ucode.len())
-            .ok_or(EOVERFLOW)?;
-
-        let start_tag = u16::try_from(tlv.get_u32(b"STRT")?)?;
-
         Ok(Self {
             _firmware_dma: firmware_dma,
-            ucode,
+            bootloader,
             dmem_desc,
             brom_params: firmware.brom_params(),
-            imem_dst_start: u16::try_from(imem_dst_start)?,
-            start_tag,
         })
     }
 
@@ -274,7 +233,7 @@ fn brom_params(&self) -> FalconBromParams {
     fn boot_addr(&self) -> u32 {
         // On V2 platforms, the boot address is extracted from the generic bootloader, because the
         // gbl is what actually copies FWSEC into memory, so that is what needs to be booted.
-        u32::from(self.start_tag) << 8
+        self.bootloader.boot_addr()
     }
 }
 
@@ -284,12 +243,7 @@ fn imem_sec_load_params(&self) -> Option<FalconPioImemLoadTarget<'_>> {
     }
 
     fn imem_ns_load_params(&self) -> Option<FalconPioImemLoadTarget<'_>> {
-        Some(FalconPioImemLoadTarget {
-            data: self.ucode.as_ref(),
-            dst_start: self.imem_dst_start,
-            secure: false,
-            start_tag: self.start_tag,
-        })
+        Some(self.bootloader.imem_load_params())
     }
 
     fn dmem_load_params(&self) -> FalconPioDmemLoadTarget<'_> {
diff --git a/drivers/gpu/nova-core/firmware/gen_bootloader.rs b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
new file mode 100644
index 000000000000..1e7ff9699174
--- /dev/null
+++ b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
@@ -0,0 +1,99 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+//! The generic falcon bootloader: a small program that the driver loads into a falcon's IMEM by
+//! PIO. It reads a descriptor that the driver writes at DMEM offset 0, and loads the image that
+//! the descriptor names into IMEM and DMEM by DMA.
+
+use kernel::{
+    device,
+    prelude::*,
+    ptr::Alignable,
+    sizes, //
+};
+
+use crate::{
+    falcon::{
+        self,
+        FalconPioImemLoadTarget, //
+    },
+    firmware::tlv::{
+        request_tlv, //
+        Tlv,
+    },
+    gpu::Chipset,
+    num::FromSafeCast, //
+};
+
+/// The generic falcon bootloader image and its IMEM load parameters.
+pub(crate) struct GenericBootloader {
+    /// Bootloader code, zero-padded to a whole number of falcon memory blocks.
+    ucode: KVec<u8>,
+    /// Byte offset in IMEM at which the code is loaded.
+    imem_dst_start: u16,
+    /// Tag under which the first code block is loaded.
+    start_tag: u16,
+}
+
+impl GenericBootloader {
+    /// Loads the generic bootloader image for `chipset`, placed right below the first 64 KiB of
+    /// IMEM.
+    ///
+    /// # Errors
+    ///
+    /// - `EINVAL` if a required TLV field is absent.
+    /// - `EOVERFLOW` if the image does not fit below 64 KiB.
+    /// - `ENOMEM` if the padded copy of the code cannot be allocated.
+    pub(crate) fn new(dev: &device::Device, chipset: Chipset) -> Result<Self> {
+        let fw = request_tlv(dev, chipset, "gen_bootloader")?;
+        let tlv = Tlv::new(fw.data())?;
+        dev_dbg!(
+            dev,
+            "loaded generic bootloader firmware v{}\n",
+            tlv.get_string(b"VERS")?
+        );
+
+        let ucode = {
+            let blob = tlv.get_bytes(b"BLOB")?;
+            let code_size = usize::from_safe_cast(tlv.get_u32(b"CDSZ")?);
+            let code = blob.get(..code_size).ok_or(EINVAL)?;
+            let aligned_code_size = code_size
+                .align_up(cv!(falcon::MEM_BLOCK_ALIGNMENT))
+                .ok_or(EINVAL)?;
+
+            let mut ucode = KVec::with_capacity(aligned_code_size, GFP_KERNEL)?;
+            ucode.extend_from_slice(code, GFP_KERNEL)?;
+            ucode.resize(aligned_code_size, 0, GFP_KERNEL)?;
+
+            ucode
+        };
+
+        // The bootloader's code must be loaded in the area right below the first 64K of IMEM.
+        const BOOTLOADER_LOAD_CEILING: usize = sizes::SZ_64K;
+        let imem_dst_start = BOOTLOADER_LOAD_CEILING
+            .checked_sub(ucode.len())
+            .ok_or(EOVERFLOW)?;
+
+        Ok(Self {
+            ucode,
+            imem_dst_start: u16::try_from(imem_dst_start)?,
+            start_tag: u16::try_from(tlv.get_u32(b"STRT")?)?,
+        })
+    }
+
+    /// Returns the address at which the falcon starts executing the bootloader: the IMEM virtual
+    /// address of its first code block.
+    pub(crate) fn boot_addr(&self) -> u32 {
+        u32::from(self.start_tag) << 8
+    }
+
+    /// Returns the PIO parameters that place this bootloader in non-secure IMEM.
+    pub(crate) fn imem_load_params(&self) -> FalconPioImemLoadTarget<'_> {
+        FalconPioImemLoadTarget {
+            data: self.ucode.as_ref(),
+            dst_start: self.imem_dst_start,
+            secure: false,
+            start_tag: self.start_tag,
+        }
+    }
+}

-- 
2.56.0

[PATCH v2 02/12] gpu: nova-core: falcon: widen the IMEM PIO offset to 24 bits

[Alexandre Courbot]

`NV_PFALCON_FALCON_IMEMC` addresses IMEM with a 24-bit offset since GA100
(16 bits on Turing, with the following 8 bits unused).

Nova declared it as 16 bits and carried IMEM PIO destinations as `u16`,
which prevents loading anything above 64 KiB of IMEM, a scenario that
becomes possible once the generic bootloader is placed at the top of
IMEM.

Widen the register field and carry the destination as a 24-bit `Bounded`.

Suggested-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://lore.kernel.org/all/DMNAN7KX22SC.QQ5HURCU332A@nvidia.com/
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
 drivers/gpu/nova-core/falcon.rs                  | 10 +++++++---
 drivers/gpu/nova-core/firmware/gen_bootloader.rs |  8 ++++++--
 drivers/gpu/nova-core/regs.rs                    |  2 +-
 3 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/nova-core/falcon.rs b/drivers/gpu/nova-core/falcon.rs
index 0b377bd6524a..a34748abc74a 100644
--- a/drivers/gpu/nova-core/falcon.rs
+++ b/drivers/gpu/nova-core/falcon.rs
@@ -18,6 +18,10 @@
         Io,
         Mmio, //
     },
+    num::{
+        Bounded,
+        TryIntoBounded, //
+    },
     prelude::*,
     sizes::SZ_4K,
     time::Delta,
@@ -230,13 +234,13 @@ fn try_as_pio_loadable(&self) -> Result<FalconDmaFirmwarePioAdapter<'_, Self>> {
             let end = start + usize::from_safe_cast(params.len);
             let data = self.as_slice().get(start..end).ok_or(EINVAL)?;
 
-            let dst_start = u16::try_from(params.dst_start).map_err(|_| EINVAL)?;
+            let dst_start = params.dst_start.try_into_bounded().ok_or(EINVAL)?;
 
             Ok::<_, Error>(FalconPioImemLoadTarget {
                 data,
                 dst_start,
                 secure,
-                start_tag: dst_start >> 8,
+                start_tag: dst_start.shr::<8, 16>().into(),
             })
         };
 
@@ -272,7 +276,7 @@ fn try_as_pio_loadable(&self) -> Result<FalconDmaFirmwarePioAdapter<'_, Self>> {
 #[derive(Clone)]
 pub(crate) struct FalconPioImemLoadTarget<'a> {
     pub(crate) data: &'a [u8],
-    pub(crate) dst_start: u16,
+    pub(crate) dst_start: Bounded<u32, 24>,
     pub(crate) secure: bool,
     pub(crate) start_tag: u16,
 }
diff --git a/drivers/gpu/nova-core/firmware/gen_bootloader.rs b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
index 1e7ff9699174..16ef2c79a6b1 100644
--- a/drivers/gpu/nova-core/firmware/gen_bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
@@ -7,6 +7,10 @@
 
 use kernel::{
     device,
+    num::{
+        Bounded,
+        TryIntoBounded, //
+    },
     prelude::*,
     ptr::Alignable,
     sizes, //
@@ -30,7 +34,7 @@ pub(crate) struct GenericBootloader {
     /// Bootloader code, zero-padded to a whole number of falcon memory blocks.
     ucode: KVec<u8>,
     /// Byte offset in IMEM at which the code is loaded.
-    imem_dst_start: u16,
+    imem_dst_start: Bounded<u32, 24>,
     /// Tag under which the first code block is loaded.
     start_tag: u16,
 }
@@ -76,7 +80,7 @@ pub(crate) fn new(dev: &device::Device, chipset: Chipset) -> Result<Self> {
 
         Ok(Self {
             ucode,
-            imem_dst_start: u16::try_from(imem_dst_start)?,
+            imem_dst_start: imem_dst_start.try_into_bounded().ok_or(EINVAL)?,
             start_tag: u16::try_from(tlv.get_u32(b"STRT")?)?,
         })
     }
diff --git a/drivers/gpu/nova-core/regs.rs b/drivers/gpu/nova-core/regs.rs
index df2eddc58efb..bf3a43f1c09e 100644
--- a/drivers/gpu/nova-core/regs.rs
+++ b/drivers/gpu/nova-core/regs.rs
@@ -236,7 +236,7 @@ pub(crate) fn usable_fb_size(self) -> u64 {
         /// Auto-increment on write.
         24:24     aincw => bool;
         /// IMEM block and word offset.
-        15:0      offs;
+        23:0      offs;
     }
 
     /// IMEM data register. Reading/writing this register accesses IMEM at the address

-- 
2.56.0

[PATCH v2 03/12] gpu: nova-core: place the generic bootloader at the top of IMEM

[Alexandre Courbot]

From: John Hubbard <jhubbard@nvidia.com>

The generic bootloader was loaded right below the first 64 KiB of IMEM,
on the assumption that every falcon it runs on has a 64 KiB IMEM. The
r000 load-and-execute bootloader event can name any image for it to
load, and the bootloader must then sit above the blocks that the image
occupies, that is, in the last blocks of IMEM, regardless of its size.

Place the bootloader at the top of the falcon's IMEM, using the IMEM
size that the HWCFG register reports, instead of assuming a 64 KiB IMEM.

Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Zhi Wang <zhiw@nvidia.com>
[acourbot: split out of the patch separating the bootloader from FWSEC;
 convert the IMEM size with usize::from()]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
 drivers/gpu/nova-core/falcon.rs                    |  7 +++++++
 drivers/gpu/nova-core/firmware/fwsec/bootloader.rs |  7 ++++---
 drivers/gpu/nova-core/firmware/gen_bootloader.rs   | 21 +++++++++++++--------
 drivers/gpu/nova-core/gsp/hal/tu102.rs             |  6 ++++--
 drivers/gpu/nova-core/regs.rs                      |  5 +++++
 5 files changed, 33 insertions(+), 13 deletions(-)

diff --git a/drivers/gpu/nova-core/falcon.rs b/drivers/gpu/nova-core/falcon.rs
index a34748abc74a..b17a997860ef 100644
--- a/drivers/gpu/nova-core/falcon.rs
+++ b/drivers/gpu/nova-core/falcon.rs
@@ -386,6 +386,13 @@ pub(crate) fn new(
         })
     }
 
+    /// Returns the size of this falcon's IMEM in bytes.
+    pub(crate) fn imem_size(&self) -> usize {
+        let blocks = usize::from(self.pfalcon.read(regs::NV_PFALCON_FALCON_HWCFG).imem_size());
+
+        blocks * MEM_BLOCK_ALIGNMENT
+    }
+
     /// Resets DMA-related registers.
     pub(crate) fn dma_reset(&self) {
         self.pfalcon.update(regs::NV_PFALCON_FBIF_CTL, |v| {
diff --git a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
index 80239301bbfa..00f0d9704883 100644
--- a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
@@ -103,14 +103,15 @@ pub(crate) struct FwsecFirmwareWithBl<'a> {
 }
 
 impl<'a> FwsecFirmwareWithBl<'a> {
-    /// Loads the bootloader firmware for `dev` and `chipset`, and wrap `firmware` so it can be
-    /// loaded using it.
+    /// Loads the bootloader firmware for `dev` and `chipset`, placed at the top of `falcon`'s IMEM,
+    /// and wraps `firmware` so it can be loaded using it.
     pub(crate) fn new(
         firmware: FwsecFirmware,
         dev: &'a Device<device::Bound>,
         chipset: Chipset,
+        falcon: &Falcon<'_, Gsp>,
     ) -> Result<Self> {
-        let bootloader = GenericBootloader::new(dev, chipset)?;
+        let bootloader = GenericBootloader::new(dev, chipset, falcon)?;
 
         // `BootloaderDmemDescV2` expects the source to be a mirror image of the destination and
         // uses the same offset parameter for both.
diff --git a/drivers/gpu/nova-core/firmware/gen_bootloader.rs b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
index 16ef2c79a6b1..c4f436104326 100644
--- a/drivers/gpu/nova-core/firmware/gen_bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
@@ -12,13 +12,14 @@
         TryIntoBounded, //
     },
     prelude::*,
-    ptr::Alignable,
-    sizes, //
+    ptr::Alignable, //
 };
 
 use crate::{
     falcon::{
         self,
+        gsp::Gsp,
+        Falcon,
         FalconPioImemLoadTarget, //
     },
     firmware::tlv::{
@@ -40,15 +41,19 @@ pub(crate) struct GenericBootloader {
 }
 
 impl GenericBootloader {
-    /// Loads the generic bootloader image for `chipset`, placed right below the first 64 KiB of
+    /// Loads the generic bootloader image for `chipset`, placed in the last blocks of `falcon`'s
     /// IMEM.
     ///
     /// # Errors
     ///
     /// - `EINVAL` if a required TLV field is absent.
-    /// - `EOVERFLOW` if the image does not fit below 64 KiB.
+    /// - `EOVERFLOW` if the image does not fit in IMEM.
     /// - `ENOMEM` if the padded copy of the code cannot be allocated.
-    pub(crate) fn new(dev: &device::Device, chipset: Chipset) -> Result<Self> {
+    pub(crate) fn new(
+        dev: &device::Device,
+        chipset: Chipset,
+        falcon: &Falcon<'_, Gsp>,
+    ) -> Result<Self> {
         let fw = request_tlv(dev, chipset, "gen_bootloader")?;
         let tlv = Tlv::new(fw.data())?;
         dev_dbg!(
@@ -72,9 +77,9 @@ pub(crate) fn new(dev: &device::Device, chipset: Chipset) -> Result<Self> {
             ucode
         };
 
-        // The bootloader's code must be loaded in the area right below the first 64K of IMEM.
-        const BOOTLOADER_LOAD_CEILING: usize = sizes::SZ_64K;
-        let imem_dst_start = BOOTLOADER_LOAD_CEILING
+        // The top of IMEM, above the blocks that the bootloader loads the image into.
+        let imem_dst_start = falcon
+            .imem_size()
             .checked_sub(ucode.len())
             .ok_or(EOVERFLOW)?;
 
diff --git a/drivers/gpu/nova-core/gsp/hal/tu102.rs b/drivers/gpu/nova-core/gsp/hal/tu102.rs
index e90db1a23032..1b17ece53bd6 100644
--- a/drivers/gpu/nova-core/gsp/hal/tu102.rs
+++ b/drivers/gpu/nova-core/gsp/hal/tu102.rs
@@ -165,7 +165,7 @@ fn run_fwsec_frts(
         )?;
 
         if self.needs_fwsec_bootloader {
-            let fwsec_frts_bl = FwsecFirmwareWithBl::new(fwsec_frts, dev, chipset)?;
+            let fwsec_frts_bl = FwsecFirmwareWithBl::new(fwsec_frts, dev, chipset, falcon)?;
             // Load and run the bootloader, which will load FWSEC-FRTS and run it.
             fwsec_frts_bl.run(dev, falcon)?;
         } else {
@@ -223,7 +223,9 @@ fn build_unload_bundle<'gpu>(
         // Load the FWSEC SB firmware, as well as its bootloader if required.
         let fwsec_sb = FwsecFirmware::new(dev, gsp_falcon, bios, FwsecCommand::Sb)?;
         let fwsec_sb = if self.needs_fwsec_bootloader {
-            FwsecUnloadFirmware::WithBl(FwsecFirmwareWithBl::new(fwsec_sb, dev, chipset)?)
+            FwsecUnloadFirmware::WithBl(FwsecFirmwareWithBl::new(
+                fwsec_sb, dev, chipset, gsp_falcon,
+            )?)
         } else {
             FwsecUnloadFirmware::WithoutBl(fwsec_sb)
         };
diff --git a/drivers/gpu/nova-core/regs.rs b/drivers/gpu/nova-core/regs.rs
index bf3a43f1c09e..af0bbb92e992 100644
--- a/drivers/gpu/nova-core/regs.rs
+++ b/drivers/gpu/nova-core/regs.rs
@@ -181,6 +181,11 @@ pub(crate) fn usable_fb_size(self) -> u64 {
         31:0    value => u32;
     }
 
+    pub(crate) NV_PFALCON_FALCON_HWCFG(u32) @ 0x00000108 {
+        /// Number of 256-byte blocks in this falcon's IMEM.
+        8:0     imem_size;
+    }
+
     pub(crate) NV_PFALCON_FALCON_DMACTL(u32) @ 0x0000010c {
         7:7     secure_stat => bool;
         6:3     dmaq_num;

-- 
2.56.0

[PATCH v2 04/12] gpu: nova-core: move the bootloader DMEM descriptor out of FWSEC

[Alexandre Courbot]

From: John Hubbard <jhubbard@nvidia.com>

The generic falcon bootloader reads its load parameters from a
descriptor that the driver places in DMEM. The r000 load-and-execute
bootloader event carries such a descriptor, so its handler will need the
descriptor type.

The descriptor was defined in the FWSEC code, which was its only user.
Move the definition, unchanged, into the generic bootloader module.

No functional changes.

Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
[acourbot: keep the fields private, add a constructor and the ctx_dma()
 accessor instead of making every field pub(crate)]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
 drivers/gpu/nova-core/firmware/fwsec/bootloader.rs | 95 +++++-----------------
 drivers/gpu/nova-core/firmware/gen_bootloader.rs   | 94 ++++++++++++++++++++-
 2 files changed, 112 insertions(+), 77 deletions(-)

diff --git a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
index 00f0d9704883..19e047f1c1ed 100644
--- a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
@@ -35,60 +35,16 @@
     },
     firmware::{
         fwsec::FwsecFirmware,
-        gen_bootloader::GenericBootloader, //
+        gen_bootloader::{
+            BootloaderDmemDescV2,
+            GenericBootloader, //
+        },
     },
     gpu::Chipset,
     num::FromSafeCast, //
     regs,
 };
 
-/// Structure used by the boot-loader to load the rest of the code.
-///
-/// This has to be filled by the GPU driver and copied into DMEM at offset
-/// [`BootloaderDesc.dmem_load_off`].
-#[repr(C, packed)]
-#[derive(Debug, Clone)]
-struct BootloaderDmemDescV2 {
-    /// Reserved, should always be first element.
-    reserved: [u32; 4],
-    /// 16B signature for secure code, 0s if no secure code.
-    signature: [u32; 4],
-    /// DMA context used by the bootloader while loading code/data.
-    ctx_dma: u32,
-    /// 256B-aligned physical FB address where code is located.
-    code_dma_base: u64,
-    /// Offset from `code_dma_base` where the non-secure code is located.
-    ///
-    /// Also used as destination IMEM offset of non-secure code as the DMA firmware object is
-    /// expected to be a mirror image of its loaded state.
-    ///
-    /// Must be multiple of 256.
-    non_sec_code_off: u32,
-    /// Size of the non-secure code part.
-    non_sec_code_size: u32,
-    /// Offset from `code_dma_base` where the secure code is located (must be multiple of 256).
-    ///
-    /// Also used as destination IMEM offset of secure code as the DMA firmware object is expected
-    /// to be a mirror image of its loaded state.
-    ///
-    /// Must be multiple of 256.
-    sec_code_off: u32,
-    /// Size of the secure code part.
-    sec_code_size: u32,
-    /// Code entry point invoked by the bootloader after code is loaded.
-    code_entry_point: u32,
-    /// 256B-aligned physical FB address where data is located.
-    data_dma_base: u64,
-    /// Size of data block (should be multiple of 256B).
-    data_size: u32,
-    /// Number of arguments to be passed to the target firmware being loaded.
-    argc: u32,
-    /// Arguments to be passed to the target firmware being loaded.
-    argv: u32,
-}
-// SAFETY: This struct doesn't contain uninitialized bytes and doesn't have interior mutability.
-unsafe impl AsBytes for BootloaderDmemDescV2 {}
-
 /// Wrapper for [`FwsecFirmware`] that includes the bootloader performing the actual load
 /// operation.
 pub(crate) struct FwsecFirmwareWithBl<'a> {
@@ -141,9 +97,6 @@ pub(crate) fn new(
         };
 
         let dmem_desc = {
-            // Bootloader payload is in non-coherent system memory.
-            const FALCON_DMAIDX_PHYS_SYS_NCOH: u32 = 4;
-
             let imem_sec = firmware.imem_sec_load_params();
             let imem_ns = firmware.imem_ns_load_params().ok_or(EINVAL)?;
             let dmem = firmware.dmem_load_params();
@@ -155,30 +108,20 @@ pub(crate) fn new(
                 return Err(EINVAL);
             }
 
-            BootloaderDmemDescV2 {
-                reserved: [0; 4],
-                signature: [0; 4],
-                ctx_dma: FALCON_DMAIDX_PHYS_SYS_NCOH,
-                code_dma_base: firmware_dma.dma_address(),
-                // `dst_start` is also valid as the source offset since the firmware DMA object is
-                // a mirror image of the target IMEM layout.
-                non_sec_code_off: imem_ns.dst_start,
-                non_sec_code_size: imem_ns.len,
-                // `dst_start` is also valid as the source offset since the firmware DMA object is
-                // a mirror image of the target IMEM layout.
-                sec_code_off: imem_sec.dst_start,
-                sec_code_size: imem_sec.len,
-                code_entry_point: 0,
-                // Start of data section is the added padding + the DMEM `src_start` field.
-                data_dma_base: firmware_dma
-                    .dma_address()
-                    .checked_add(u64::from_safe_cast(align_padding))
-                    .and_then(|offset| offset.checked_add(dmem.src_start.into()))
-                    .ok_or(EOVERFLOW)?,
-                data_size: dmem.len,
-                argc: 0,
-                argv: 0,
-            }
+            // Start of data section is the added padding + the DMEM `src_start` field.
+            let data_dma_base = firmware_dma
+                .dma_address()
+                .checked_add(u64::from_safe_cast(align_padding))
+                .and_then(|offset| offset.checked_add(dmem.src_start.into()))
+                .ok_or(EOVERFLOW)?;
+
+            BootloaderDmemDescV2::new(
+                &imem_ns,
+                &imem_sec,
+                &dmem,
+                firmware_dma.dma_address(),
+                data_dma_base,
+            )
         };
 
         Ok(Self {
@@ -204,7 +147,7 @@ pub(crate) fn run(&self, dev: &Device<device::Bound>, falcon: &Falcon<'_, Gsp>)
 
         // Configure DMA index for the bootloader to fetch the FWSEC firmware from system memory.
         falcon.pfalcon.update(
-            regs::NV_PFALCON_FBIF_TRANSCFG::try_at(usize::from_safe_cast(self.dmem_desc.ctx_dma))
+            regs::NV_PFALCON_FBIF_TRANSCFG::try_at(usize::from_safe_cast(self.dmem_desc.ctx_dma()))
                 .ok_or(EINVAL)?,
             |v| {
                 v.with_target(FalconFbifTarget::CoherentSysmem)
diff --git a/drivers/gpu/nova-core/firmware/gen_bootloader.rs b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
index c4f436104326..f769b2b277c0 100644
--- a/drivers/gpu/nova-core/firmware/gen_bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
@@ -12,7 +12,8 @@
         TryIntoBounded, //
     },
     prelude::*,
-    ptr::Alignable, //
+    ptr::Alignable,
+    transmute::AsBytes, //
 };
 
 use crate::{
@@ -20,6 +21,7 @@
         self,
         gsp::Gsp,
         Falcon,
+        FalconDmaLoadTarget,
         FalconPioImemLoadTarget, //
     },
     firmware::tlv::{
@@ -30,6 +32,96 @@
     num::FromSafeCast, //
 };
 
+/// Structure used by the boot-loader to load the rest of the code.
+///
+/// This has to be filled by the GPU driver and copied into DMEM at offset
+/// [`BootloaderDesc.dmem_load_off`].
+#[repr(C, packed)]
+#[derive(Debug, Clone)]
+pub(crate) struct BootloaderDmemDescV2 {
+    /// Reserved, should always be first element.
+    reserved: [u32; 4],
+    /// 16B signature for secure code, 0s if no secure code.
+    signature: [u32; 4],
+    /// DMA context used by the bootloader while loading code/data.
+    ctx_dma: u32,
+    /// 256B-aligned physical FB address where code is located.
+    code_dma_base: u64,
+    /// Offset from `code_dma_base` where the non-secure code is located.
+    ///
+    /// Also used as destination IMEM offset of non-secure code as the DMA firmware object is
+    /// expected to be a mirror image of its loaded state.
+    ///
+    /// Must be multiple of 256.
+    non_sec_code_off: u32,
+    /// Size of the non-secure code part.
+    non_sec_code_size: u32,
+    /// Offset from `code_dma_base` where the secure code is located (must be multiple of 256).
+    ///
+    /// Also used as destination IMEM offset of secure code as the DMA firmware object is expected
+    /// to be a mirror image of its loaded state.
+    ///
+    /// Must be multiple of 256.
+    sec_code_off: u32,
+    /// Size of the secure code part.
+    sec_code_size: u32,
+    /// Code entry point invoked by the bootloader after code is loaded.
+    code_entry_point: u32,
+    /// 256B-aligned physical FB address where data is located.
+    data_dma_base: u64,
+    /// Size of data block (should be multiple of 256B).
+    data_size: u32,
+    /// Number of arguments to be passed to the target firmware being loaded.
+    argc: u32,
+    /// Arguments to be passed to the target firmware being loaded.
+    argv: u32,
+}
+
+impl BootloaderDmemDescV2 {
+    /// Context DMA slot through which the bootloader reads the image: physical, non-coherent
+    /// system memory.
+    const FALCON_DMAIDX_PHYS_SYS_NCOH: u32 = 4;
+
+    /// Creates the descriptor of an image whose code is at `code_dma_base` and whose data is at
+    /// `data_dma_base`.
+    ///
+    /// `imem_ns` and `imem_sec` locate the non-secure and secure code in the image, and `dmem`
+    /// gives the size of its data. The code DMA object must be a mirror image of the target IMEM
+    /// layout, because the bootloader uses the destination offset of each code segment as its
+    /// source offset too.
+    pub(crate) fn new(
+        imem_ns: &FalconDmaLoadTarget,
+        imem_sec: &FalconDmaLoadTarget,
+        dmem: &FalconDmaLoadTarget,
+        code_dma_base: u64,
+        data_dma_base: u64,
+    ) -> Self {
+        Self {
+            reserved: [0; 4],
+            signature: [0; 4],
+            ctx_dma: Self::FALCON_DMAIDX_PHYS_SYS_NCOH,
+            code_dma_base,
+            non_sec_code_off: imem_ns.dst_start,
+            non_sec_code_size: imem_ns.len,
+            sec_code_off: imem_sec.dst_start,
+            sec_code_size: imem_sec.len,
+            code_entry_point: 0,
+            data_dma_base,
+            data_size: dmem.len,
+            argc: 0,
+            argv: 0,
+        }
+    }
+
+    /// Returns the context DMA slot through which the bootloader reads the image.
+    pub(crate) fn ctx_dma(&self) -> u32 {
+        self.ctx_dma
+    }
+}
+
+// SAFETY: This struct doesn't contain uninitialized bytes and doesn't have interior mutability.
+unsafe impl AsBytes for BootloaderDmemDescV2 {}
+
 /// The generic falcon bootloader image and its IMEM load parameters.
 pub(crate) struct GenericBootloader {
     /// Bootloader code, zero-padded to a whole number of falcon memory blocks.

-- 
2.56.0

[PATCH v2 05/12] gpu: nova-core: run the generic bootloader from its own type

[Alexandre Courbot]

From: John Hubbard <jhubbard@nvidia.com>

The generic falcon bootloader needs a descriptor in DMEM to know what to
load, so loading it into a falcon means loading the two together by PIO
and starting the falcon. The FWSEC wrapper did that through its own
implementation of the PIO loading traits. The r000 load-and-execute
bootloader event will need to do the same thing.

Give the generic bootloader a run method, which pairs it with the
descriptor that it receives, loads the pair by PIO, starts the falcon
with the given MAILBOX0 value and returns the mailboxes once the falcon
has halted. This will allow the FWSEC and r000 load-and-execute
bootloader event to share the same code.

No functional changes.

Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Co-developed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
 drivers/gpu/nova-core/firmware/fwsec/bootloader.rs |  70 ++-------------
 drivers/gpu/nova-core/firmware/gen_bootloader.rs   | 100 ++++++++++++++++++++-
 2 files changed, 102 insertions(+), 68 deletions(-)

diff --git a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
index 19e047f1c1ed..90b875945766 100644
--- a/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/fwsec/bootloader.rs
@@ -12,26 +12,15 @@
         Device, //
     },
     dma::Coherent,
-    io::{
-        register::Array,
-        Io, //
-    },
-    prelude::*,
-    transmute::AsBytes,
+    prelude::*, //
 };
 
 use crate::{
     falcon::{
         gsp::Gsp,
         Falcon,
-        FalconBromParams,
         FalconDmaLoadable,
-        FalconFbifMemType,
-        FalconFbifTarget,
-        FalconFirmware,
-        FalconPioDmemLoadTarget,
-        FalconPioImemLoadTarget,
-        FalconPioLoadable, //
+        FalconFbifTarget, //
     },
     firmware::{
         fwsec::FwsecFirmware,
@@ -42,7 +31,6 @@
     },
     gpu::Chipset,
     num::FromSafeCast, //
-    regs,
 };
 
 /// Wrapper for [`FwsecFirmware`] that includes the bootloader performing the actual load
@@ -54,8 +42,6 @@ pub(crate) struct FwsecFirmwareWithBl<'a> {
     bootloader: GenericBootloader,
     /// Descriptor to be loaded into DMEM for the bootloader to read.
     dmem_desc: BootloaderDmemDescV2,
-    /// BROM parameters of the loaded firmware.
-    brom_params: FalconBromParams,
 }
 
 impl<'a> FwsecFirmwareWithBl<'a> {
@@ -128,7 +114,6 @@ pub(crate) fn new(
             _firmware_dma: firmware_dma,
             bootloader,
             dmem_desc,
-            brom_params: firmware.brom_params(),
         })
     }
 
@@ -137,26 +122,14 @@ pub(crate) fn new(
     /// The bootloader will load the FWSEC firmware and then execute it. This function returns
     /// after FWSEC has reached completion.
     pub(crate) fn run(&self, dev: &Device<device::Bound>, falcon: &Falcon<'_, Gsp>) -> Result<()> {
-        // Reset falcon, load the firmware, and run it.
         falcon
             .reset()
             .inspect_err(|e| dev_err!(dev, "Failed to reset GSP falcon: {:?}\n", e))?;
-        falcon
-            .pio_load(self)
-            .inspect_err(|e| dev_err!(dev, "Failed to load FWSEC firmware: {:?}\n", e))?;
 
-        // Configure DMA index for the bootloader to fetch the FWSEC firmware from system memory.
-        falcon.pfalcon.update(
-            regs::NV_PFALCON_FBIF_TRANSCFG::try_at(usize::from_safe_cast(self.dmem_desc.ctx_dma()))
-                .ok_or(EINVAL)?,
-            |v| {
-                v.with_target(FalconFbifTarget::CoherentSysmem)
-                    .with_mem_type(FalconFbifMemType::Physical)
-            },
-        );
-
-        let (mbox0, _) = falcon
-            .boot(Some(0), None)
+        // Run the bootloader, which loads the FWSEC firmware and runs it.
+        let (mbox0, _) = self
+            .bootloader
+            .run(falcon, &self.dmem_desc, FalconFbifTarget::CoherentSysmem, 0)
             .inspect_err(|e| dev_err!(dev, "Failed to boot FWSEC firmware: {:?}\n", e))?;
         if mbox0 != 0 {
             dev_err!(dev, "FWSEC firmware returned error {}\n", mbox0);
@@ -166,34 +139,3 @@ pub(crate) fn run(&self, dev: &Device<device::Bound>, falcon: &Falcon<'_, Gsp>)
         }
     }
 }
-
-impl FalconFirmware for FwsecFirmwareWithBl<'_> {
-    type Target = Gsp;
-
-    fn brom_params(&self) -> FalconBromParams {
-        self.brom_params.clone()
-    }
-
-    fn boot_addr(&self) -> u32 {
-        // On V2 platforms, the boot address is extracted from the generic bootloader, because the
-        // gbl is what actually copies FWSEC into memory, so that is what needs to be booted.
-        self.bootloader.boot_addr()
-    }
-}
-
-impl FalconPioLoadable for FwsecFirmwareWithBl<'_> {
-    fn imem_sec_load_params(&self) -> Option<FalconPioImemLoadTarget<'_>> {
-        None
-    }
-
-    fn imem_ns_load_params(&self) -> Option<FalconPioImemLoadTarget<'_>> {
-        Some(self.bootloader.imem_load_params())
-    }
-
-    fn dmem_load_params(&self) -> FalconPioDmemLoadTarget<'_> {
-        FalconPioDmemLoadTarget {
-            data: self.dmem_desc.as_bytes(),
-            dst_start: 0,
-        }
-    }
-}
diff --git a/drivers/gpu/nova-core/firmware/gen_bootloader.rs b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
index f769b2b277c0..1fecb6a5f80e 100644
--- a/drivers/gpu/nova-core/firmware/gen_bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
@@ -7,6 +7,10 @@
 
 use kernel::{
     device,
+    io::{
+        register::Array,
+        Io, //
+    },
     num::{
         Bounded,
         TryIntoBounded, //
@@ -21,15 +25,22 @@
         self,
         gsp::Gsp,
         Falcon,
+        FalconBromParams,
         FalconDmaLoadTarget,
-        FalconPioImemLoadTarget, //
+        FalconFbifMemType,
+        FalconFbifTarget,
+        FalconFirmware,
+        FalconPioDmemLoadTarget,
+        FalconPioImemLoadTarget,
+        FalconPioLoadable, //
     },
     firmware::tlv::{
         request_tlv, //
         Tlv,
     },
     gpu::Chipset,
-    num::FromSafeCast, //
+    num::FromSafeCast,
+    regs, //
 };
 
 /// Structure used by the boot-loader to load the rest of the code.
@@ -184,12 +195,12 @@ pub(crate) fn new(
 
     /// Returns the address at which the falcon starts executing the bootloader: the IMEM virtual
     /// address of its first code block.
-    pub(crate) fn boot_addr(&self) -> u32 {
+    fn boot_addr(&self) -> u32 {
         u32::from(self.start_tag) << 8
     }
 
     /// Returns the PIO parameters that place this bootloader in non-secure IMEM.
-    pub(crate) fn imem_load_params(&self) -> FalconPioImemLoadTarget<'_> {
+    fn imem_load_params(&self) -> FalconPioImemLoadTarget<'_> {
         FalconPioImemLoadTarget {
             data: self.ucode.as_ref(),
             dst_start: self.imem_dst_start,
@@ -197,4 +208,85 @@ pub(crate) fn imem_load_params(&self) -> FalconPioImemLoadTarget<'_> {
             start_tag: self.start_tag,
         }
     }
+
+    /// Runs the bootloader on `falcon` to load and start the image that `dmem_desc` names.
+    ///
+    /// The bootloader and the descriptor are loaded by PIO, the bootloader is started with `mbox0`
+    /// in `MAILBOX0`, and this returns the mailbox values once the falcon has halted, that is,
+    /// once the loaded image has run to completion.
+    ///
+    /// The caller is expected to reset `falcon` beforehand.
+    ///
+    /// # Errors
+    ///
+    /// - `ETIMEDOUT` if the falcon has not halted within two seconds of starting.
+    /// - `EINVAL` if `dmem_desc`'s DMA context isn't valid.
+    ///
+    /// Errors from the PIO load are propagated as-is.
+    pub(crate) fn run(
+        &self,
+        falcon: &Falcon<'_, Gsp>,
+        dmem_desc: &BootloaderDmemDescV2,
+        target: FalconFbifTarget,
+        mbox0: u32,
+    ) -> Result<(u32, u32)> {
+        // Configure DMA index for the bootloader to fetch the FWSEC firmware from system memory.
+        falcon.pfalcon.update(
+            regs::NV_PFALCON_FBIF_TRANSCFG::try_at(usize::from_safe_cast(dmem_desc.ctx_dma()))
+                .ok_or(EINVAL)?,
+            |v| {
+                v.with_target(target)
+                    .with_mem_type(FalconFbifMemType::Physical)
+            },
+        );
+
+        falcon.pio_load(&GenericBootloaderLoad {
+            bootloader: self,
+            dmem_desc,
+        })?;
+
+        falcon.boot(Some(mbox0), None)
+    }
+}
+
+/// The generic bootloader together with the descriptor that it reads from DMEM offset 0, as one
+/// firmware that [`Falcon::pio_load`] accepts.
+struct GenericBootloaderLoad<'a> {
+    bootloader: &'a GenericBootloader,
+    dmem_desc: &'a BootloaderDmemDescV2,
+}
+
+impl FalconFirmware for GenericBootloaderLoad<'_> {
+    type Target = Gsp;
+
+    fn brom_params(&self) -> FalconBromParams {
+        // The bootloader is unsigned, and the falcon HALs of the chipsets that load it this way
+        // write no BROM registers, so these values never reach hardware.
+        FalconBromParams {
+            pkc_data_offset: 0,
+            engine_id_mask: 0,
+            ucode_id: 0,
+        }
+    }
+
+    fn boot_addr(&self) -> u32 {
+        self.bootloader.boot_addr()
+    }
+}
+
+impl FalconPioLoadable for GenericBootloaderLoad<'_> {
+    fn imem_sec_load_params(&self) -> Option<FalconPioImemLoadTarget<'_>> {
+        None
+    }
+
+    fn imem_ns_load_params(&self) -> Option<FalconPioImemLoadTarget<'_>> {
+        Some(self.bootloader.imem_load_params())
+    }
+
+    fn dmem_load_params(&self) -> FalconPioDmemLoadTarget<'_> {
+        FalconPioDmemLoadTarget {
+            data: self.dmem_desc.as_bytes(),
+            dst_start: 0,
+        }
+    }
 }

-- 
2.56.0

[PATCH v2 06/12] rust: io: register: implement Copy and Clone for RegisterArrayLoc

[Alexandre Courbot]

Register array locations are small values that are cheap to copy, but
obtaining one can fail. Making them implement `Copy` lets the same
location be checked once and used for several accesses.

Cc: Danilo Krummrich <dakr@kernel.org>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: Daniel Almeida <daniel.almeida@collabora.com>
Cc: Gary Guo <gary@garyguo.net>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
 rust/kernel/io/register.rs | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/rust/kernel/io/register.rs b/rust/kernel/io/register.rs
index b6513fa0f412..9e31e68d02cd 100644
--- a/rust/kernel/io/register.rs
+++ b/rust/kernel/io/register.rs
@@ -194,6 +194,14 @@ pub fn try_new(idx: usize) -> Option<Self> {
     }
 }
 
+impl<T: RegisterArray> Clone for RegisterArrayLoc<T> {
+    fn clone(&self) -> Self {
+        *self
+    }
+}
+
+impl<T: RegisterArray> Copy for RegisterArrayLoc<T> {}
+
 impl<Base: ?Sized, T> IoLoc<Base, T> for RegisterArrayLoc<T>
 where
     T: RegisterArray<Base = Base>,

-- 
2.56.0

[PATCH v2 07/12] gpu: nova-core: falcon: introduce and use with_fbif_transcfg

[Alexandre Courbot]

From: John Hubbard <jhubbard@nvidia.com>

The generic bootloader programs the FBIF aperture of its context DMA
slot and leaves it set after the bootloader has run. The r000
load-and-execute events will need to do the same with apertures that
GSP-RM chooses, and should not leave them behind.

Add a falcon method that programs an FBIF context DMA slot, runs a
closure, and restores the slot's previous value once the closure has
succeeded, and use it to run the generic bootloader. The slot used by
FWSEC is now restored after FWSEC has halted.

Signed-off-by: John Hubbard <jhubbard@nvidia.com>
[acourbot: extract from later patch of the r000 series, convert unneeded
 call to `update` into a `write`, only check the validity of the DMA
 context once.]
Co-developed-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
 drivers/gpu/nova-core/falcon.rs                  | 30 ++++++++++++++++++++++++
 drivers/gpu/nova-core/firmware/gen_bootloader.rs | 28 +++++++++-------------
 2 files changed, 41 insertions(+), 17 deletions(-)

diff --git a/drivers/gpu/nova-core/falcon.rs b/drivers/gpu/nova-core/falcon.rs
index b17a997860ef..ad497a5704ec 100644
--- a/drivers/gpu/nova-core/falcon.rs
+++ b/drivers/gpu/nova-core/falcon.rs
@@ -513,6 +513,36 @@ pub(crate) fn pio_load<F: FalconFirmware<Target = E> + FalconPioLoadable>(
         Ok(())
     }
 
+    /// Programs FBIF context DMA slot `ctx_dma` with the value that `configure` returns, runs `f`,
+    /// and restores the slot once `f` has returned `Ok`.
+    ///
+    /// The slot keeps the programmed value if `f` fails, since a falcon that `f` started and that
+    /// has not halted may still be reading through the aperture.
+    ///
+    /// # Errors
+    ///
+    /// - `EINVAL` if `ctx_dma` is not a context DMA slot that the falcon has.
+    ///
+    /// Errors from `f` are propagated as-is.
+    pub(crate) fn with_fbif_transcfg<R>(
+        &self,
+        ctx_dma: u32,
+        configure: impl FnOnce(regs::NV_PFALCON_FBIF_TRANSCFG) -> regs::NV_PFALCON_FBIF_TRANSCFG,
+        f: impl FnOnce() -> Result<R>,
+    ) -> Result<R> {
+        let transcfg =
+            regs::NV_PFALCON_FBIF_TRANSCFG::try_at(usize::from_safe_cast(ctx_dma)).ok_or(EINVAL)?;
+
+        let saved = self.pfalcon.read(transcfg);
+        self.pfalcon.update(transcfg, configure);
+
+        let result = f()?;
+
+        self.pfalcon.write(transcfg, saved);
+
+        Ok(result)
+    }
+
     /// Perform a DMA write according to `load_offsets` from `dma_obj` into the falcon's
     /// `target_mem`.
     ///
diff --git a/drivers/gpu/nova-core/firmware/gen_bootloader.rs b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
index 1fecb6a5f80e..252f6ac2d14b 100644
--- a/drivers/gpu/nova-core/firmware/gen_bootloader.rs
+++ b/drivers/gpu/nova-core/firmware/gen_bootloader.rs
@@ -7,10 +7,6 @@
 
 use kernel::{
     device,
-    io::{
-        register::Array,
-        Io, //
-    },
     num::{
         Bounded,
         TryIntoBounded, //
@@ -39,8 +35,7 @@
         Tlv,
     },
     gpu::Chipset,
-    num::FromSafeCast,
-    regs, //
+    num::FromSafeCast, //
 };
 
 /// Structure used by the boot-loader to load the rest of the code.
@@ -230,22 +225,21 @@ pub(crate) fn run(
         target: FalconFbifTarget,
         mbox0: u32,
     ) -> Result<(u32, u32)> {
-        // Configure DMA index for the bootloader to fetch the FWSEC firmware from system memory.
-        falcon.pfalcon.update(
-            regs::NV_PFALCON_FBIF_TRANSCFG::try_at(usize::from_safe_cast(dmem_desc.ctx_dma()))
-                .ok_or(EINVAL)?,
+        falcon.with_fbif_transcfg(
+            dmem_desc.ctx_dma(),
             |v| {
                 v.with_target(target)
                     .with_mem_type(FalconFbifMemType::Physical)
             },
-        );
+            || {
+                falcon.pio_load(&GenericBootloaderLoad {
+                    bootloader: self,
+                    dmem_desc,
+                })?;
 
-        falcon.pio_load(&GenericBootloaderLoad {
-            bootloader: self,
-            dmem_desc,
-        })?;
-
-        falcon.boot(Some(mbox0), None)
+                falcon.boot(Some(mbox0), None)
+            },
+        )
     }
 }
 

-- 
2.56.0

[PATCH v2 08/12] gpu: nova-core: falcon: rename the local variables of dma_wr()

[Alexandre Courbot]

The local variables of dma_wr() are named after a generic DMA address,
while what they hold are the values programmed into specific registers.
Worse, the `src_start` local has the same name as the `src_start` field
of `FalconDmaLoadTarget`, but a different value for DMEM transfers.

Rename them after the registers they end up in:

- `dma_address` -> `dma_base`, programmed into DMATRFBASE,
- `src_start` -> `fb_offset`, programmed into DMATRFFBOFFS,
- `dma_start` -> `base_offset`, the part of the source offset that is
  folded into the DMA base.

No functional change.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
 drivers/gpu/nova-core/falcon.rs | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/gpu/nova-core/falcon.rs b/drivers/gpu/nova-core/falcon.rs
index ad497a5704ec..3b3e8e0f821a 100644
--- a/drivers/gpu/nova-core/falcon.rs
+++ b/drivers/gpu/nova-core/falcon.rs
@@ -562,13 +562,13 @@ fn dma_wr(
         // For IMEM, we want to use the start offset as a virtual address tag for each page, since
         // code addresses in the firmware (and the boot vector) are virtual.
         //
-        // For DMEM, the start offset is folded into the DMA address.
-        let (src_start, dma_start) = match target_mem {
+        // For DMEM, the start offset is folded into the DMA base.
+        let (fb_offset, base_offset) = match target_mem {
             FalconMem::ImemSecure | FalconMem::ImemNonSecure => (load_offsets.src_start, 0),
             FalconMem::Dmem => (0, usize::from_safe_cast(load_offsets.src_start)),
         };
 
-        let dma_address = {
+        let dma_base = {
             // Upper limit of transfer is `(num_transfers * DMA_LEN) + load_offsets.src_start`.
             let dma_end = num_transfers
                 .checked_mul(DMA_LEN)
@@ -576,10 +576,10 @@ fn dma_wr(
                 .map(usize::from_safe_cast)
                 .ok_or(EOVERFLOW)?;
 
-            io_project!(dma_obj, [try: dma_start..dma_end]).dma_address()
+            io_project!(dma_obj, [try: base_offset..dma_end]).dma_address()
         };
 
-        if dma_address % DmaAddress::from(DMA_LEN) > 0 {
+        if dma_base % DmaAddress::from(DMA_LEN) > 0 {
             dev_err!(
                 self.dev,
                 "DMA transfer start addresses must be a multiple of {}\n",
@@ -594,10 +594,10 @@ fn dma_wr(
             .write_reg(regs::NV_PFALCON_FALCON_DMATRFBASE::zeroed().with_base(
                 // CAST: `as u32` is used on purpose since we do want to strip the upper bits,
                 // which will be written to `NV_PFALCON_FALCON_DMATRFBASE1`.
-                (dma_address >> 8) as u32,
+                (dma_base >> 8) as u32,
             ));
         self.pfalcon.write_reg(
-            regs::NV_PFALCON_FALCON_DMATRFBASE1::zeroed().try_with_base(dma_address >> 40)?,
+            regs::NV_PFALCON_FALCON_DMATRFBASE1::zeroed().try_with_base(dma_base >> 40)?,
         );
 
         let cmd = regs::NV_PFALCON_FALCON_DMATRFCMD::zeroed()
@@ -611,7 +611,7 @@ fn dma_wr(
                     .try_with_offs(load_offsets.dst_start + pos)?,
             );
             self.pfalcon.write_reg(
-                regs::NV_PFALCON_FALCON_DMATRFFBOFFS::zeroed().with_offs(src_start + pos),
+                regs::NV_PFALCON_FALCON_DMATRFFBOFFS::zeroed().with_offs(fb_offset + pos),
             );
 
             self.pfalcon.write_reg(cmd);

-- 
2.56.0

[PATCH v2 09/12] gpu: nova-core: falcon: allow DMEM DMA transfers to request tagging

[Alexandre Courbot]

Add a tag flag to `FalconMem::Dmem` and set DMATRFCMD's DMTAG bit from
it.

Similarly to IMEM, when tagging is requested the DMA engine tags each
DMEM block it writes with the block's source offset, which the loaded
image then uses as its DMEM virtual address. The r000 load-and-execute
HS binary event will provide a DMEM virtual address for images that
need one.

No functional change intended.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
 drivers/gpu/nova-core/falcon.rs | 23 +++++++++++++++--------
 drivers/gpu/nova-core/regs.rs   |  6 ++++--
 2 files changed, 19 insertions(+), 10 deletions(-)

diff --git a/drivers/gpu/nova-core/falcon.rs b/drivers/gpu/nova-core/falcon.rs
index 3b3e8e0f821a..00da664887fc 100644
--- a/drivers/gpu/nova-core/falcon.rs
+++ b/drivers/gpu/nova-core/falcon.rs
@@ -135,8 +135,9 @@ pub(crate) enum FalconMem {
     /// Non-Secure Instruction Memory.
     #[expect(unused)]
     ImemNonSecure,
-    /// Data Memory.
-    Dmem,
+    /// Data Memory. If `tag` is set, a transfer into DMEM also tags each block with its source
+    /// offset, which is then the DMEM virtual address of the image.
+    Dmem { tag: bool },
 }
 
 bounded_enum! {
@@ -559,13 +560,15 @@ fn dma_wr(
         // need to perform.
         let num_transfers = load_offsets.len.div_ceil(DMA_LEN);
 
-        // For IMEM, we want to use the start offset as a virtual address tag for each page, since
-        // code addresses in the firmware (and the boot vector) are virtual.
+        // For IMEM and tagged DMEM, we want to use the start offset as a virtual address tag for
+        // each page, since addresses in the firmware (and the boot vector) are virtual.
         //
-        // For DMEM, the start offset is folded into the DMA base.
+        // For untagged DMEM, the start offset is folded into the DMA base.
         let (fb_offset, base_offset) = match target_mem {
-            FalconMem::ImemSecure | FalconMem::ImemNonSecure => (load_offsets.src_start, 0),
-            FalconMem::Dmem => (0, usize::from_safe_cast(load_offsets.src_start)),
+            FalconMem::ImemSecure | FalconMem::ImemNonSecure | FalconMem::Dmem { tag: true } => {
+                (load_offsets.src_start, 0)
+            }
+            FalconMem::Dmem { tag: false } => (0, usize::from_safe_cast(load_offsets.src_start)),
         };
 
         let dma_base = {
@@ -659,7 +662,11 @@ fn dma_load<F: FalconFirmware<Target = E> + FalconDmaLoadable>(&self, fw: &F) ->
             });
 
         self.dma_wr(&dma_obj, FalconMem::ImemSecure, fw.imem_sec_load_params())?;
-        self.dma_wr(&dma_obj, FalconMem::Dmem, fw.dmem_load_params())?;
+        self.dma_wr(
+            &dma_obj,
+            FalconMem::Dmem { tag: false },
+            fw.dmem_load_params(),
+        )?;
 
         self.hal.program_brom(self, &fw.brom_params());
 
diff --git a/drivers/gpu/nova-core/regs.rs b/drivers/gpu/nova-core/regs.rs
index af0bbb92e992..7e997478ab42 100644
--- a/drivers/gpu/nova-core/regs.rs
+++ b/drivers/gpu/nova-core/regs.rs
@@ -321,9 +321,11 @@ pub(crate) fn mem_scrubbing_done(self) -> bool {
 }
 
 impl NV_PFALCON_FALCON_DMATRFCMD {
-    /// Programs the `imem` and `sec` fields for the given FalconMem
+    /// Programs the `imem`, `sec` and `set_dmtag` fields for the given FalconMem
     pub(crate) fn with_falcon_mem(self, mem: FalconMem) -> Self {
-        let this = self.with_imem(mem != FalconMem::Dmem);
+        let this = self
+            .with_imem(!matches!(mem, FalconMem::Dmem { .. }))
+            .with_set_dmtag(matches!(mem, FalconMem::Dmem { tag: true }));
 
         match mem {
             FalconMem::ImemSecure => this.with_const_sec::<1>(),

-- 
2.56.0

[PATCH v2 10/12] gpu: nova-core: falcon: allow DMA transfers to read from a framebuffer address

[Alexandre Courbot]

The r000 boot protocol has a load-and-execute event in which GSP-RM asks
the driver to load a Heavy-Secured binary from framebuffer memory into
the GSP falcon's memory by DMA in order to run it.

Nova-core's falcon DMA loading method currently only supports loading
from a DMA object that the driver has allocated, so there is no way to
copy an image from framebuffer memory that an event supplied.

Address this by making the source of a falcon DMA transfer an enum,
which can be either a DMA object, or a GPU framebuffer address.

The framebuffer address source has no user yet, but will be used by the
r000 boot protocol.

Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
 drivers/gpu/nova-core/falcon.rs | 65 ++++++++++++++++++++++++++---------------
 1 file changed, 42 insertions(+), 23 deletions(-)

diff --git a/drivers/gpu/nova-core/falcon.rs b/drivers/gpu/nova-core/falcon.rs
index 00da664887fc..b63265abd263 100644
--- a/drivers/gpu/nova-core/falcon.rs
+++ b/drivers/gpu/nova-core/falcon.rs
@@ -140,6 +140,15 @@ pub(crate) enum FalconMem {
     Dmem { tag: bool },
 }
 
+/// Where a falcon DMA transfer reads from.
+pub(crate) enum FalconDmaSource<'a> {
+    /// A DMA object in system memory.
+    Sysmem(&'a Coherent<'a, [u8]>),
+    /// GPU framebuffer memory at physical address `base`.
+    #[expect(dead_code)]
+    Framebuffer { base: u64 },
+}
+
 bounded_enum! {
     /// Defines the Framebuffer Interface (FBIF) aperture type.
     /// This determines the memory type for external memory access during a DMA transfer, which is
@@ -544,13 +553,10 @@ pub(crate) fn with_fbif_transcfg<R>(
         Ok(result)
     }
 
-    /// Perform a DMA write according to `load_offsets` from `dma_obj` into the falcon's
-    /// `target_mem`.
-    ///
-    /// `sec` is set if the loaded firmware is expected to run in secure mode.
+    /// Perform a DMA write according to `load_offsets` from `src` into the falcon's `target_mem`.
     fn dma_wr(
         &self,
-        dma_obj: &Coherent<'_, [u8]>,
+        src: FalconDmaSource<'_>,
         target_mem: FalconMem,
         load_offsets: FalconDmaLoadTarget,
     ) -> Result {
@@ -563,23 +569,32 @@ fn dma_wr(
         // For IMEM and tagged DMEM, we want to use the start offset as a virtual address tag for
         // each page, since addresses in the firmware (and the boot vector) are virtual.
         //
-        // For untagged DMEM, the start offset is folded into the DMA base.
-        let (fb_offset, base_offset) = match target_mem {
-            FalconMem::ImemSecure | FalconMem::ImemNonSecure | FalconMem::Dmem { tag: true } => {
-                (load_offsets.src_start, 0)
+        // For untagged DMEM in a DMA object, the start offset is folded into the DMA base. For a
+        // framebuffer address, the DMA engine adds the start offset to the address.
+        let (fb_offset, dma_base) = match src {
+            FalconDmaSource::Sysmem(dma_obj) => {
+                let (fb_offset, base_offset) = match target_mem {
+                    FalconMem::ImemSecure
+                    | FalconMem::ImemNonSecure
+                    | FalconMem::Dmem { tag: true } => (load_offsets.src_start, 0),
+                    FalconMem::Dmem { tag: false } => {
+                        (0, usize::from_safe_cast(load_offsets.src_start))
+                    }
+                };
+
+                // Upper limit of transfer is `(num_transfers * DMA_LEN) + load_offsets.src_start`.
+                let dma_end = num_transfers
+                    .checked_mul(DMA_LEN)
+                    .and_then(|size| size.checked_add(load_offsets.src_start))
+                    .map(usize::from_safe_cast)
+                    .ok_or(EOVERFLOW)?;
+
+                (
+                    fb_offset,
+                    io_project!(dma_obj, [try: base_offset..dma_end]).dma_address(),
+                )
             }
-            FalconMem::Dmem { tag: false } => (0, usize::from_safe_cast(load_offsets.src_start)),
-        };
-
-        let dma_base = {
-            // Upper limit of transfer is `(num_transfers * DMA_LEN) + load_offsets.src_start`.
-            let dma_end = num_transfers
-                .checked_mul(DMA_LEN)
-                .and_then(|size| size.checked_add(load_offsets.src_start))
-                .map(usize::from_safe_cast)
-                .ok_or(EOVERFLOW)?;
-
-            io_project!(dma_obj, [try: base_offset..dma_end]).dma_address()
+            FalconDmaSource::Framebuffer { base } => (load_offsets.src_start, base),
         };
 
         if dma_base % DmaAddress::from(DMA_LEN) > 0 {
@@ -661,9 +676,13 @@ fn dma_load<F: FalconFirmware<Target = E> + FalconDmaLoadable>(&self, fw: &F) ->
                     .with_mem_type(FalconFbifMemType::Physical)
             });
 
-        self.dma_wr(&dma_obj, FalconMem::ImemSecure, fw.imem_sec_load_params())?;
         self.dma_wr(
-            &dma_obj,
+            FalconDmaSource::Sysmem(&dma_obj),
+            FalconMem::ImemSecure,
+            fw.imem_sec_load_params(),
+        )?;
+        self.dma_wr(
+            FalconDmaSource::Sysmem(&dma_obj),
             FalconMem::Dmem { tag: false },
             fw.dmem_load_params(),
         )?;

-- 
2.56.0

[PATCH v2 11/12] gpu: nova-core: falcon: check the per-block DMA offsets for overflow

[Alexandre Courbot]

From: John Hubbard <jhubbard@nvidia.com>

Each block of a falcon DMA transfer is addressed by the start offset of
the transfer plus the block's position. The two additions were plain, so
a source or destination offset near the top of the 32-bit range would
have wrapped or panicked rather than failing the transfer.

Add the two with checked arithmetic and fail the transfer with EOVERFLOW
when a sum does not fit.

Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
[acourbot: split out of the DMA helper patch of v3.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
 drivers/gpu/nova-core/falcon.rs | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/nova-core/falcon.rs b/drivers/gpu/nova-core/falcon.rs
index b63265abd263..da860caac20b 100644
--- a/drivers/gpu/nova-core/falcon.rs
+++ b/drivers/gpu/nova-core/falcon.rs
@@ -626,10 +626,11 @@ fn dma_wr(
             // Perform a transfer of size `DMA_LEN`.
             self.pfalcon.write_reg(
                 regs::NV_PFALCON_FALCON_DMATRFMOFFS::zeroed()
-                    .try_with_offs(load_offsets.dst_start + pos)?,
+                    .try_with_offs(load_offsets.dst_start.checked_add(pos).ok_or(EOVERFLOW)?)?,
             );
             self.pfalcon.write_reg(
-                regs::NV_PFALCON_FALCON_DMATRFFBOFFS::zeroed().with_offs(fb_offset + pos),
+                regs::NV_PFALCON_FALCON_DMATRFFBOFFS::zeroed()
+                    .with_offs(fb_offset.checked_add(pos).ok_or(EOVERFLOW)?),
             );
 
             self.pfalcon.write_reg(cmd);

-- 
2.56.0

[PATCH v2 12/12] gpu: nova-core: falcon: program the FBIF_TRANSCFG register in dma_wr

[Alexandre Courbot]

`dma_wr` takes the source of the transfer as a parameter and can derive
the value of the `FBIF_TRANSCFG` register from it. Do it so the caller
does not need to program this register beforehand, which could leave the
door open to bogus setups.

Suggested-by: Eliot Courtney <ecourtney@nvidia.com>
Link: https://lore.kernel.org/all/DMNWO08EFJD8.D2G9X082FUKN@nvidia.com/
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
---
 drivers/gpu/nova-core/falcon.rs | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/drivers/gpu/nova-core/falcon.rs b/drivers/gpu/nova-core/falcon.rs
index da860caac20b..76f49970cade 100644
--- a/drivers/gpu/nova-core/falcon.rs
+++ b/drivers/gpu/nova-core/falcon.rs
@@ -571,7 +571,7 @@ fn dma_wr(
         //
         // For untagged DMEM in a DMA object, the start offset is folded into the DMA base. For a
         // framebuffer address, the DMA engine adds the start offset to the address.
-        let (fb_offset, dma_base) = match src {
+        let (fb_offset, dma_base, fbif_target) = match src {
             FalconDmaSource::Sysmem(dma_obj) => {
                 let (fb_offset, base_offset) = match target_mem {
                     FalconMem::ImemSecure
@@ -592,9 +592,12 @@ fn dma_wr(
                 (
                     fb_offset,
                     io_project!(dma_obj, [try: base_offset..dma_end]).dma_address(),
+                    FalconFbifTarget::CoherentSysmem,
                 )
             }
-            FalconDmaSource::Framebuffer { base } => (load_offsets.src_start, base),
+            FalconDmaSource::Framebuffer { base } => {
+                (load_offsets.src_start, base, FalconFbifTarget::LocalFb)
+            }
         };
 
         if dma_base % DmaAddress::from(DMA_LEN) > 0 {
@@ -622,6 +625,12 @@ fn dma_wr(
             .with_size(DmaTrfCmdSize::Size256B)
             .with_falcon_mem(target_mem);
 
+        self.pfalcon
+            .update(regs::NV_PFALCON_FBIF_TRANSCFG::at(0), |v| {
+                v.with_target(fbif_target)
+                    .with_mem_type(FalconFbifMemType::Physical)
+            });
+
         for pos in (0..num_transfers).map(|i| i * DMA_LEN) {
             // Perform a transfer of size `DMA_LEN`.
             self.pfalcon.write_reg(
@@ -671,11 +680,6 @@ fn dma_load<F: FalconFirmware<Target = E> + FalconDmaLoadable>(&self, fw: &F) ->
         };
 
         self.dma_reset();
-        self.pfalcon
-            .update(regs::NV_PFALCON_FBIF_TRANSCFG::at(0), |v| {
-                v.with_target(FalconFbifTarget::CoherentSysmem)
-                    .with_mem_type(FalconFbifMemType::Physical)
-            });
 
         self.dma_wr(
             FalconDmaSource::Sysmem(&dma_obj),

-- 
2.56.0

Re: [PATCH v2 06/12] rust: io: register: implement Copy and Clone for RegisterArrayLoc

[Danilo Krummrich]

On Sat Nov 7, 2026 at 10:48 AM CET, Alexandre Courbot wrote:
> Register array locations are small values that are cheap to copy, but
> obtaining one can fail. Making them implement `Copy` lets the same
> location be checked once and used for several accesses.
>
> Cc: Danilo Krummrich <dakr@kernel.org>
> Cc: Alice Ryhl <aliceryhl@google.com>
> Cc: Daniel Almeida <daniel.almeida@collabora.com>
> Cc: Gary Guo <gary@garyguo.net>
> Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>

Acked-by: Danilo Krummrich <dakr@kernel.org>