[PATCH] gpu: nova-core: gsp: zero-pad command queue elements

[PATCH] gpu: nova-core: gsp: zero-pad command queue elements

[haider]

GSP-RM checksums a command queue element by XORing it in 64-bit words,
so for an element whose size is not a multiple of 8 it also reads up to
7 bytes past its end, and expects them to be zero: its _checkSum32()
"assumes that the data is padded out with zeros to the next 8-byte
alignment". NVIDIA's kernel driver zeroes these bytes before computing
the checksum, and nouveau builds each element in a zeroed buffer.

allocate_command() hands out the queue slots as they are, so these bytes
still hold whatever an earlier, longer element left there. Once the
queue has wrapped, a command of such a size can then fail GSP-RM's
checksum: GSP-RM never answers it, and soon stops answering at all.

Nova does not send enough commands for the queue to wrap yet, as it is
zeroed at probe, but this will break as soon as it does.

Clear each element and its padding before writing it.

Fixes: 75f6b1de8133 ("gpu: nova-core: gsp: Add GSP command queue bindings and handling")
Assisted-by: LLM
Signed-off-by: haider <soka11223344k@gmail.com>
---

Notes:
    Found and tested on an out-of-tree nova branch that sends RM object
    allocations and controls through the command queue, so the queue wraps
    there. Once it had wrapped, the allocation of a subdevice failed the
    checksum, and GSP-RM stopped answering, logging Display NOCAT 0x28,
    assertions and Xid 45, within the first 5-10 cycles of creating and
    freeing a client, device and subdevice. With this patch, 288 such cycles
    ran clean (RTX 3070 / GA104, GSP firmware 570.144).
    
    On drm-rust-next as is the queue never wraps, so there it is only
    build-tested (W=1, CLIPPY=1), along with checkpatch and rustfmt.
    
    An LLM found the cause by comparing the command queue code with NVIDIA's
    open kernel modules and nouveau, and wrote the fix. I reviewed and
    tested it.

 drivers/gpu/nova-core/gsp/cmdq.rs | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/gpu/nova-core/gsp/cmdq.rs b/drivers/gpu/nova-core/gsp/cmdq.rs
index 1b2f346a23..ff323bb37c 100644
--- a/drivers/gpu/nova-core/gsp/cmdq.rs
+++ b/drivers/gpu/nova-core/gsp/cmdq.rs
@@ -384,6 +384,15 @@ fn allocate_command(&mut self, size: usize, timeout: Delta) -> Result<GspCommand
             (slice_1.as_flattened_mut(), slice_2.as_flattened_mut())
         };
 
+        // GSP-RM checksums an element in 64-bit words, reading up to 7 bytes past its end, and
+        // expects them to be zero (`_checkSum32` in NVIDIA's `message_queue_priv.h`). The ring
+        // still holds what earlier elements left there, so clear the element and its padding.
+        // Elements take whole pages, so the padding fits in the space that was waited for.
+        let padded = (size_of::<GspMsgElement>() + size).next_multiple_of(size_of::<u64>());
+        let first = padded.min(slice_1.len());
+        slice_1[..first].fill(0);
+        slice_2.get_mut(..padded - first).ok_or(EIO)?.fill(0);
+
         // Extract area for the `GspMsgElement`.
         let (header, slice_1) = GspMsgElement::from_bytes_mut_prefix(slice_1).ok_or(EIO)?;
 

base-commit: 5a1b7121a6c4db6c0239d3ac3fd1c2bb33a40025
-- 
2.55.0

Re: [PATCH] gpu: nova-core: gsp: zero-pad command queue elements

[Alexandre Courbot]

On Sun Oct 11, 2026 at 11:51 PM JST, haider wrote:
> GSP-RM checksums a command queue element by XORing it in 64-bit words,
> so for an element whose size is not a multiple of 8 it also reads up to
> 7 bytes past its end, and expects them to be zero: its _checkSum32()
> "assumes that the data is padded out with zeros to the next 8-byte
> alignment". NVIDIA's kernel driver zeroes these bytes before computing
> the checksum, and nouveau builds each element in a zeroed buffer.
>
> allocate_command() hands out the queue slots as they are, so these bytes
> still hold whatever an earlier, longer element left there. Once the
> queue has wrapped, a command of such a size can then fail GSP-RM's
> checksum: GSP-RM never answers it, and soon stops answering at all.
>
> Nova does not send enough commands for the queue to wrap yet, as it is
> zeroed at probe, but this will break as soon as it does.
>
> Clear each element and its padding before writing it.

The checksum code is imminently going away with the switch to the r000
firmware version, so I don't think there is much value in fixing it now.

>
> Fixes: 75f6b1de8133 ("gpu: nova-core: gsp: Add GSP command queue bindings and handling")
> Assisted-by: LLM
> Signed-off-by: haider <soka11223344k@gmail.com>

For future contributions, please use as real name as per the
contribution guidelines [1].

Also as a general comment, both the commit log and code comment are
unsufferably long and tedious. When working with a LLM, please
edit/rewrite its output to something concise and to the point.

[1] https://www.kernel.org/doc/html/v4.17/process/submitting-patches.html#developer-s-certificate-of-origin-1-1