mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Gary Guo" <gary@garyguo.net>
To: "Alexandre Courbot" <acourbot@nvidia.com>, "Gary Guo" <gary@garyguo.net>
Cc: "John Hubbard" <jhubbard@nvidia.com>,
	"Danilo Krummrich" <dakr@kernel.org>,
	"Timur Tabi" <ttabi@nvidia.com>,
	"Alistair Popple" <apopple@nvidia.com>,
	"Eliot Courtney" <ecourtney@nvidia.com>,
	"Zhi Wang" <zhiw@nvidia.com>, "David Airlie" <airlied@gmail.com>,
	"Simona Vetter" <simona@ffwll.ch>,
	"Bjorn Helgaas" <bhelgaas@google.com>,
	"Miguel Ojeda" <ojeda@kernel.org>,
	"Alex Gaynor" <alex.gaynor@gmail.com>,
	"Boqun Feng" <boqun.feng@gmail.com>,
	"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
	"Benno Lossin" <lossin@kernel.org>,
	"Andreas Hindborg" <a.hindborg@kernel.org>,
	"Alice Ryhl" <aliceryhl@google.com>,
	"Trevor Gross" <tmgross@umich.edu>,
	nova-gpu@lists.linux.dev, LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v3 08/33] gpu: nova-core: gsp: compute the queue regions from a count and a slot
Date: Wed, 23 Sep 2026 17:47:10 +0100	[thread overview]
Message-ID: <DLMUKFL5CAZ5.2ICMKG94JXP52@garyguo.net> (raw)
In-Reply-To: <DLMQ5TVNFSC1.2AGU37ACR0Y9Q@nvidia.com>

On Wed Sep 23, 2026 at 2:20 PM BST, Alexandre Courbot wrote:
> On Wed Sep 23, 2026 at 8:30 PM JST, Gary Guo wrote:
>> On Wed Sep 23, 2026 at 5:55 AM BST, Alexandre Courbot wrote:
>>> On Fri Sep 18, 2026 at 10:06 AM JST, John Hubbard wrote:
>>>>      fn driver_write_area(&mut self) -> (&mut [[u8; GSP_PAGE_SIZE]], &mut [[u8; GSP_PAGE_SIZE]]) {
>>>> -        let tx = self.cpu_write_ptr();
>>>> -        let rx = self.gsp_read_ptr();
>>>> +        let avail = num::u32_as_usize(self.free_slots());
>>>> +        let w_slot = num::u32_as_usize(self.cpu_write_ptr());
>>>>  
>>>>          // Pointer to the first entry of the CPU message queue.
>>>>          let data = ptr::project!(mut self.mem.as_mut_ptr(), .cpuq.msgq.data[build: 0]);
>>>>  
>>>> -        let (tail_end, wrap_end) = if rx == 0 {
>>>> -            // The write area is non-wrapping, and stops at the second-to-last entry of the command
>>>> -            // queue (to leave the last one empty).
>>>> -            (MSGQ_NUM_PAGES - 1, 0)
>>>> -        } else if rx <= tx {
>>>> -            // The write area wraps and continues until `rx - 1`.
>>>> -            (MSGQ_NUM_PAGES, rx - 1)
>>>> -        } else {
>>>> -            // The write area doesn't wrap and stops at `rx - 1`.
>>>> -            (rx - 1, 0)
>>>> -        };
>>>> -
>>>>          // SAFETY:
>>>> -        // - `data` was created from a valid pointer, and `rx` and `tx` are in the
>>>> -        //   `0..MSGQ_NUM_PAGES` range per the invariants of `cpu_write_ptr` and `gsp_read_ptr`,
>>>> -        //   thus the created slices are valid.
>>>> -        // - The area starting at `tx` and ending at `rx - 2` modulo `MSGQ_NUM_PAGES`,
>>>> -        //   inclusive, belongs to the driver for writing and is not accessed concurrently by
>>>> -        //   the GSP.
>>>> -        // - The caller holds a reference to `self` for as long as the returned slices are live,
>>>> -        //   meaning the CPU write pointer cannot be advanced and thus that the returned area
>>>> -        //   remains exclusive to the CPU for the duration of the slices.
>>>> -        // - The created slices point to non-overlapping sub-ranges of `data` in all
>>>> -        //   branches (in the `rx <= tx` case, the second slice ends at `rx - 1` which is strictly
>>>> -        //   less than `tx` where the first slice starts; in the other cases the second slice is
>>>> -        //   empty), so creating two `&mut` references from them does not violate aliasing rules.
>>>> -        unsafe {
>>>> -            (
>>>> -                core::slice::from_raw_parts_mut(
>>>> -                    data.add(num::u32_as_usize(tx)),
>>>> -                    num::u32_as_usize(tail_end - tx),
>>>> -                ),
>>>> -                core::slice::from_raw_parts_mut(data, num::u32_as_usize(wrap_end)),
>>>> -            )
>>>> -        }
>>>> +        // - `data` points to the `MSGQ_NUM_PAGES` initialized entries of the CPU message queue.
>>>> +        // - The returned slices cover the `avail` free slots from the write pointer on, which the
>>>> +        //   GSP does not read until `advance_cpu_write_ptr` publishes them.
>>>> +        // - `split_at_mut` gives two non-overlapping halves, and the `&mut self` borrow lasts as
>>>> +        //   long as the returned slices, so that no other call hands out the same region while
>>>> +        //   they live.
>>>> +        let data =
>>>> +            unsafe { core::slice::from_raw_parts_mut(data, num::u32_as_usize(MSGQ_NUM_PAGES)) };
>>>> +        let (before_w, after_w) = data.split_at_mut(w_slot);
>>>
>>> This creates a reference over the whole ring, including the parts owned
>>> by the GSP, which breaks the `Coherent` safety contract that the device
>>> must not be able to read or write to a live slice. So we'll need to call
>>> `from_raw_parts_mut` twice, with the correct sizes, instead of
>>> splitting.
>>>
>>> (also `split_at_mut` is panicking and should have a `PANIC:` comment
>>> justifying why it cannot, but once the point above is addressed that
>>> call will go away).
>>>
>>> I wanted to try it locally and ended up with something that seems to
>>> work, so let me share it to save some time:
>>>
>>>     fn driver_write_area(&mut self) -> (&mut [[u8; GSP_PAGE_SIZE]], &mut [[u8; GSP_PAGE_SIZE]]) {
>>>         let avail = self.free_slots();
>>>         let w_slot = self.cpu_write_ptr();
>>>
>>>         // Pointer to the first entry of the CPU message queue.
>>>         let data = ptr::project!(mut self.mem.as_mut_ptr(), .cpuq.msgq.data[build: 0]);
>>>
>>>         let in_after = avail.min(MSGQ_NUM_PAGES - w_slot);
>>>         let in_before = avail - in_after;
>>>
>>>         // SAFETY:
>>>         // - `data` was created from a valid pointer of `MSGQ_NUM_PAGES` entries.
>>>         // - The `in_after` entries after `w_slot` belong to the `avail` entries that the driver is
>>>         //   currently allowed to write.
>>>         // - The `in_before` first entries belong to the `avail` entries that the driver is
>>>         //   currently allowed to write.
>>>         // - The slices do not overlap.
>>>         unsafe {
>>>             (
>>>                 core::slice::from_raw_parts_mut(
>>>                     data.add(num::u32_as_usize(w_slot)),
>>>                     num::u32_as_usize(in_after),
>>>                 ),
>>>                 core::slice::from_raw_parts_mut(data, num::u32_as_usize(in_before)),
>>>             )
>>>         }
>>>     }
>>>
>>> It has turned out quite short, which I like! I also opted to work with
>>> the original `u32` until the very end, as it results in less conversions
>>> overall.
>>
>> Possibly take some thing from the old projection syntax rework series?
>>
>> https://lore.kernel.org/rust-for-linux/20260415-projection-syntax-rework-v1-4-450723cb3727@garyguo.net/
>
> Oh yes, I forgot about this patch. Do you mean using `ptr::project` to
> create the final sub-slices, or am I missing something else?

I think it's possible to use I/O projection on `CoherentView` and then you
assert that it's not concurrently accessed by turning them from `CoherentView`
to `&mut []` using `CoherentView::as_mut`.

Best,
Gary


  reply	other threads:[~2026-09-23 16:47 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-18  1:06 [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware John Hubbard
2026-09-18  1:06 ` [PATCH v3 01/33] rust: pci: add domain_nr() accessor John Hubbard
2026-09-18  1:06 ` [PATCH v3 02/33] gpu: nova-core: set MCTP transport header version to 1 John Hubbard
2026-09-22 13:23   ` Alexandre Courbot
2026-09-18  1:06 ` [PATCH v3 03/33] gpu: nova-core: gsp: give the command queue its own BAR0 mapping John Hubbard
2026-09-22 13:24   ` Alexandre Courbot
2026-09-18  1:06 ` [PATCH v3 04/33] gpu: nova-core: firmware: add r000 bindings John Hubbard
2026-09-18 17:42   ` Timur Tabi
2026-09-22  1:51     ` John Hubbard
2026-09-23  1:34       ` John Hubbard
2026-09-18  1:06 ` [PATCH v3 05/33] gpu: nova-core: regs: add msgq v2 BAR0 register declarations John Hubbard
2026-09-23 12:20   ` Alexandre Courbot
2026-09-18  1:06 ` [PATCH v3 06/33] gpu: nova-core: gsp: ring the GSP doorbell from the queue memory John Hubbard
2026-09-18  1:06 ` [PATCH v3 07/33] gpu: nova-core: gsp: make command allocation generic over the header John Hubbard
2026-09-18  1:06 ` [PATCH v3 08/33] gpu: nova-core: gsp: compute the queue regions from a count and a slot John Hubbard
2026-09-23  4:55   ` Alexandre Courbot
2026-09-23 11:30     ` Gary Guo
2026-09-23 13:20       ` Alexandre Courbot
2026-09-23 16:47         ` Gary Guo [this message]
2026-09-23 20:36         ` John Hubbard
2026-09-18  1:06 ` [PATCH v3 09/33] gpu: nova-core: add GMC API message types John Hubbard
2026-09-18 21:45   ` Timur Tabi
2026-09-22  2:18     ` John Hubbard
2026-09-23 11:18   ` Alexandre Courbot
2026-09-18  1:06 ` [PATCH v3 10/33] gpu: nova-core: add GMC send path John Hubbard
2026-09-23 14:12   ` Alexandre Courbot
2026-09-18  1:06 ` [PATCH v3 11/33] gpu: nova-core: add GMC transport receive path John Hubbard
2026-09-18 21:57   ` Timur Tabi
2026-09-22  2:28     ` John Hubbard
2026-09-18  1:06 ` [PATCH v3 12/33] gpu: nova-core: gsp: add GMC dispatch on receive John Hubbard
2026-09-18 22:01   ` Timur Tabi
2026-09-22  2:50     ` John Hubbard
2026-09-22 20:11       ` Timur Tabi
2026-09-18  1:06 ` [PATCH v3 13/33] gpu: nova-core: separate the generic falcon bootloader from FWSEC John Hubbard
2026-09-18 22:04   ` Timur Tabi
2026-09-22  2:39     ` John Hubbard
2026-09-18  1:07 ` [PATCH v3 14/33] gpu: nova-core: add the falcon DMA and suspend helpers for r000 boot John Hubbard
2026-09-18 22:07   ` Timur Tabi
2026-09-22  2:39     ` John Hubbard
2026-09-18  1:07 ` [PATCH v3 15/33] gpu: nova-core: add the r000 load-and-execute HS binary handler John Hubbard
2026-09-18  1:07 ` [PATCH v3 16/33] gpu: nova-core: move the bootloader DMEM descriptor out of FWSEC John Hubbard
2026-09-18  1:07 ` [PATCH v3 17/33] gpu: nova-core: add the r000 load-and-execute bootloader handler John Hubbard
2026-09-18  3:32   ` Timur Tabi
2026-09-22  2:05     ` John Hubbard
2026-09-18  1:07 ` [PATCH v3 18/33] gpu: nova-core: gsp: add the GMC boot event dispatcher John Hubbard
2026-09-18  1:07 ` [PATCH v3 19/33] gpu: nova-core: gsp: rename the static configuration type John Hubbard
2026-09-18  1:07 ` [PATCH v3 20/33] gpu: nova-core: gsp: return the static GPU configuration from boot John Hubbard
2026-09-18  1:07 ` [PATCH v3 21/33] gpu: nova-core: gsp: add the GSP_INIT request builder John Hubbard
2026-09-18 22:26   ` Timur Tabi
2026-09-22  2:42     ` John Hubbard
2026-09-22 20:05       ` Timur Tabi
2026-09-23  1:51   ` Alexandre Courbot
2026-09-18  1:07 ` [PATCH v3 22/33] gpu: nova-core: gsp: send GSP_INIT and decode its reply John Hubbard
2026-09-18  1:07 ` [PATCH v3 23/33] gpu: nova-core: add LIBOS3 log buffers and state monitor buffer John Hubbard
2026-09-18 22:34   ` Timur Tabi
2026-09-22  2:34     ` John Hubbard
2026-09-18  1:07 ` [PATCH v3 24/33] gpu: nova-core: add the ucodes firmware loader John Hubbard
2026-09-18  1:07 ` [PATCH v3 25/33] gpu: nova-core: gsp: let the GSP HAL load the generic bootloader John Hubbard
2026-09-18  1:07 ` [PATCH v3 26/33] gpu: nova-core: gsp: add the GSP_SUSPEND request John Hubbard
2026-09-18  1:07 ` [PATCH v3 27/33] gpu: nova-core: switch to the r000 GSP firmware John Hubbard
2026-09-18 22:48   ` Timur Tabi
2026-09-22  2:44     ` John Hubbard
2026-09-18  1:07 ` [PATCH v3 28/33] gpu: nova-core: gsp: make the GSP_INIT reply the static configuration John Hubbard
2026-09-18  1:07 ` [PATCH v3 29/33] gpu: nova-core: firmware: delete the r570 bindings John Hubbard
2026-09-18  1:07 ` [PATCH v3 30/33] gpu: nova-core: match GSP RPC replies by sequence, not just function John Hubbard
2026-09-23 14:20   ` Alexandre Courbot
2026-09-18  1:07 ` [PATCH v3 31/33] gpu: nova-core: gsp: split the reply match out of the RPC receive path John Hubbard
2026-09-18  1:07 ` [PATCH v3 32/33] gpu: nova-core: gsp: decode queue elements by their NVDM type John Hubbard
2026-09-18  1:07 ` [PATCH v3 33/33] gpu: nova-core: gsp: match a GMC response by flag, id and sequence John Hubbard
2026-09-23 14:10   ` Alexandre Courbot
2026-09-18 23:08 ` [PATCH v3 00/33] gpu: nova-core: boot on the r000 GSP firmware Timur Tabi
2026-09-22  2:45   ` John Hubbard

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DLMUKFL5CAZ5.2ICMKG94JXP52@garyguo.net \
    --to=gary@garyguo.net \
    --cc=a.hindborg@kernel.org \
    --cc=acourbot@nvidia.com \
    --cc=airlied@gmail.com \
    --cc=alex.gaynor@gmail.com \
    --cc=aliceryhl@google.com \
    --cc=apopple@nvidia.com \
    --cc=bhelgaas@google.com \
    --cc=bjorn3_gh@protonmail.com \
    --cc=boqun.feng@gmail.com \
    --cc=dakr@kernel.org \
    --cc=ecourtney@nvidia.com \
    --cc=jhubbard@nvidia.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lossin@kernel.org \
    --cc=nova-gpu@lists.linux.dev \
    --cc=ojeda@kernel.org \
    --cc=simona@ffwll.ch \
    --cc=tmgross@umich.edu \
    --cc=ttabi@nvidia.com \
    --cc=zhiw@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®