Re: [PATCH] x86/resctrl: Require force for L3 CAT on early Skylake-X

["Graf (AWS), Alexander" <graf@amazon.de>]

Hi Reinette and Tony,

On 14.10.26 22:44, Reinette Chatre wrote:
> Hi Alexander,
>
> On 10/14/26 1:36 AM, Alexander Graf wrote:
>> When L3 CAT gets enabled with rdt=l3cat on Skylake-X stepping 3,
>> consecutive kexec boots hang for me during secondary socket bringup.
>> Cold boots on the same machine work.
> Any insight into where the hang occurs?
>
> What is state of resctrl fs when kexec runs? Is it mounted (with which options)?
>
> Does the boot hang whether resctrl fs is mounted or not?
>
> If resctrl is mounted at time of kexec, has it been set up with different groups?
> Does kexec task run from a particular group?
>   
> Does it make a difference if resctrl fs is unmounted before kexec? As I understand
> kexec takes system from resctrl without giving opportunity to reset. There should
> be no accumulation of state but system may boot up with settings left over from
> previous run. Unmounting resctrl would give opportunity to reset state so that
> new kernel can start with clean slate.
>
> What does "during secondary socket bringup" mean exactly? For example, is
> the hang when first CPU of second socket is onlined and found to be hanging
> in the resctrl online handler?


I ran a few more experiments. This seems un- or if anything only mildly 
related to kexec. The real culprit is the microcode version. In my 
environment, we boot with older microcode (0x1000159), then hot load the 
latest microcode (0x1000191) and then kexec. In that flow, any write to 
IA32_L3_QOS_MASK_n stalls the socket that write happens on. I'm 
collecting a bit more data - including microcode version bisects, early 
load, etc - and give you a full rundown tomorrow or on Monday, depending 
on how quickly it finishes.


Alex