Re: [PATCH 7/7] nova-core: mm: Add data structures for page table management

[Joel Fernandes <joelagnelf@nvidia.com>]

Hi John,

On 10/20/2025 4:59 PM, John Hubbard wrote:
> On 10/20/25 11:55 AM, Joel Fernandes wrote:
>> Add data structures and helpers for page table management. Uses
>> bitfield for cleanly representing and accessing the bitfields in the
>> structures.
>>
> ...
>> +bitfield! {
>> +    pub(crate) struct Pte(u64), "Page Table Entry (PTE) to map virtual pages to physical frames." {
>> +        0:0     valid           as bool;    // (1 = valid for PTEs)
>> +        1:1     privilege       as bool;    // P - Privileged/kernel-only access
>> +        2:2     read_only       as bool;    // RO - Write protection
>> +        3:3     atomic_disable  as bool;    // AD - Disable atomic ops
>> +        4:4     encrypted       as bool;    // E - Encryption enabled
>> +        39:8    frame_number    as u64;     // PA[39:8] - Physical frame number (32 bits)
>> +        41:40   aperture        as u8 => AperturePte;   // Memory aperture type.
>> +        42:42   volatile        as bool;    // VOL - Volatile flag
>> +        50:43   kind            as u8;      // K[7:0] - Compression/tiling kind
>> +        63:51   comptag_line    as u16;     // CTL[12:0] - Compression tag line
>> +    }
>> +}
> 
> Hi Joel,
> 
> What GPUs is this good for? I ask because I seem to recall that
> the format has changed over the years, on a per-GPU-architecture
> basis...

Yes, there's different format versions.

This patch supports all Turing and later GPUs because all GPUs including Hopper+
are backward compatible with version 1. However they wont be able to use the
version 2 and 3 features with only this patch.

I kind of intentionally did this for a first cut. But yes, I could split it into
versions. The 3 MMU structures (PTE, PDE and Dual PDE) are versioned. Version 2
is Turing and later. Hopper+ is when Version 3 got introduced and it is also
backward compatible with Version 2.

We could eventually support versions 2 and 3 (instead of just version 1 as I am
doing), however my working MMU translation prototype is based on version 1 (I
did not have to configure anything in the MMU to switch versions, this was default).

There are a couple of options:

1. For starters, support only version 1. Drawback is, when/if we want to use
version 2 and 3 features, it may require some rework.

2. Have the following hierarchy:
mm/types.rs - all common structures (stuff that is generic like Pfn).
mm/types/ver1.rs - Version 1 specific types.
mm/types/ver2.rs - Version 2 specific types.
mm/types/ver3.rs - Version 3 specific types.
The advantage of this is it keeps the structs namespaced. So it'd be
nova_core::mm:types::ver2::Pte or nova_core::mm:types::ver3::Pte. And the
nova-core MMU code can pick the correct version.

3. Keep the single file types.rs and just suffix the structs with version
numbers. This is attractive because there are only 3 types that have version
flavors (pte, pde and dual pde). So instead of Pte, we would have PteVersion1,
PteVersion2 etc, and a helper abstraction can pick the correct struct.

4. Any of the options 1-3, but dropping version 1 since Turing+ supports version
2 and later. I do have to figure out how to configure the MMU to use a specific
version (which is reasonable).

5. Your options here.

Btw, I used Nouveau as a reference as well, so likely Nouveau doesn't support
version 2 and 3 features. Not that that matters (we should support newer
features in nova-core), but just thought I'd mention it.

Other thoughts?

thanks,

 - Joel