Re: [PATCH v7 00/10] x86,fs/resctrl: Support L3 Smart Data Cache Injection Allocation Enforcement (SDCIAE)

["Moger, Babu" <babu.moger@amd.com>]

Hi Reinette,


On 7/21/25 18:28, Reinette Chatre wrote:
> Hi Babu,
> 
> On 7/10/25 10:16 AM, Babu Moger wrote:
>>
>> This series adds the support for L3 Smart Data Cache Injection Allocation
>> Enforcement (SDCIAE) to resctrl infrastructure. It is referred to as
>> "io_alloc" in resctrl subsystem.
>>
>> Upcoming AMD hardware implements Smart Data Cache Injection (SDCI).
>> Smart Data Cache Injection (SDCI) is a mechanism that enables direct
>> insertion of data from I/O devices into the L3 cache. By directly caching
>> data from I/O devices rather than first storing the I/O data in DRAM, SDCI
>> reduces demands on DRAM bandwidth and reduces latency to the processor
>> consuming the I/O data.
>>
>> The SDCIAE (SDCI Allocation Enforcement) PQE feature allows system software
>> to control the portion of the L3 cache used for SDCI devices.
>>
>> When enabled, SDCIAE forces all SDCI lines to be placed into the L3 cache
>> partitions identified by the highest-supported L3_MASK_n register, where n
>> is the maximum supported CLOSID.
> 
> Even though this CLOSID use is unique to AMD this implementation makes it part
> of resctrl fs's support of io_alloc. It is confusing to have architectural
> specific features be handled by resctrl fs so I think it will be useful to
> add a snippet here to help folks trying to decipher this work. Consider
> for example a snippet like:
> 
>   Since CLOSIDs are managed by resctrl fs it is least invasive to make
>   the "io_alloc is supported by maximum supported CLOSID" part of the
>   initial resctrl fs support for io_alloc. Take care not to expose this
>   use of CLOSID for io_alloc to user space so that this is not required from
>   other architectures that may support io_alloc differently in the future.
> 

Sure.

>>
>> The feature details are documented in the APM listed below [1].
>> [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming
>> Publication # 24593 Revision 3.41 section 19.4.7 L3 Smart Data Cache
>> Injection Allocation Enforcement (SDCIAE)
>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537
>>
>> The feature requires linux support of TPH (TLP Processing Hints).
>> The support is available in linux kernel after the commit
>> 48d0fd2b903e3 ("PCI/TPH: Add TPH documentation")
>>
>> The patches are based on top of commit (6.16.0-rc5)
>> commit b4ec95e3bc3f ("Merge x86/microcode into tip/master")
>>
>> # Linux Implementation
>>
>> Feature adds following interface files when the resctrl "io_alloc" feature
>> is supported on the resource:
>>
>> /sys/fs/resctrl/info/L3/io_alloc: Report the feature status. Enable/disable the
>> 				  feature by writing to the interface.
>>
>> /sys/fs/resctrl/info/L3/io_alloc_cbm:  List the Capacity Bit Masks (CBMs) available
>> 				       for I/O devices when io_alloc feature is enabled.
>> 				       Configure the CBM by writing to the interface.
>>
>> When CDP is enabled, these files will be created both in L3CODE and L3DATA.
> 
> "will be" -> "are"
> 

Sure.

>>
>> # Examples:
>>
>> a. Check if io_alloc feature is available
>> 	# mount -t resctrl resctrl /sys/fs/resctrl/
>>
>> 	# cat /sys/fs/resctrl/info/L3/io_alloc
>> 	disabled
>>
>> b. Enable the io_alloc feature. 
>>
>> 	# echo 1 > /sys/fs/resctrl/info/L3/io_alloc 
>> 	# cat /sys/fs/resctrl/info/L3/io_alloc
>> 	enabled
>>
>> c. Check the CBM values for the io_alloc feature.
>>
>> 	# cat /sys/fs/resctrl/info/L3/io_alloc_cbm 
>> 	0=ffff;1=ffff
>>
>> d. Change the CBM value for the domain 1:
> 
> "for the domain 1" -> "of domain 1"?

Sure.

> 
> (nit: inconsistent use of "." vs. ":" in items)

Fixed it.

> 
>> 	# echo 1=FF > /sys/fs/resctrl/info/L3/io_alloc_cbm
>>
>> 	# cat /sys/fs/resctrl/info/L3/io_alloc_cbm 
>> 	0=ffff;1=00ff
>>
>> d. Disable io_alloc feature and exit.
>>
>> 	# echo 0 > /sys/fs/resctrl/info/L3/io_alloc 
>> 	# cat /sys/fs/resctrl/info/L3/io_alloc
>> 	disabled
>>
>> 	# umount /sys/fs/resctrl/
>>
>> ---
> 
> Reinette
> 
> 

-- 
Thanks
Babu Moger