Re: [PATCH v1 1/1] x86/fred: Clear the WFE bit in missing-ENDBRANCH #CP

[Xin Li <xin@zytor.com>]

On 9/11/2024 5:22 PM, Dave Hansen wrote:
> On 9/11/24 16:19, Xin Li (Intel) wrote:
>> +/*
>> + * The WFE (WAIT_FOR_ENDBRANCH) bit in the augmented CS of FRED stack frame is
>> + * set to 1 in missing-ENDBRANCH #CP exceptions.
> 
> I think there's a bit of relatively irrelevant info in there.  For
> instance, I don't think it's super important to mention that FRED is
> involved and where the WFE bit is in memory.
> 
> FRED's involvement is kinda a no-brainer from the whole X86_FEATURE_FRED
> thing, and if you're reading exception handler code and don't know that
> 'regs' is on the stack, this probably isn't the place to explain that.

I often find myself in a dilemma, should I mention some technical
background which sometimes could also be distracting :(

Based on your feedback, maybe the following is better?

static void ibt_clear_fred_wfe(struct pt_regs *regs)
{
	if (regs->fred_cs.wfe)
		regs->fred_cs.wfe = 0;
}

And we know only FRED hardware will set the WFE bit.

> 
>> + * If the WFE bit is left as 1, the CPU will generate another missing-ENDBRANCH
>> + * #CP because the indirect branch tracker will be set in the WAIT_FOR_ENDBRANCH
>> + * state upon completion of the following ERETS instruction and the CPU will
>> + * restart from the IP that just caused a previous missing-ENDBRANCH #CP.
>> + *
>> + * Clear the WFE bit to avoid dead looping due to the above reason.
>> + */
>> +static void ibt_clear_fred_wfe(struct pt_regs *regs)
>> +{
>> +	if (cpu_feature_enabled(X86_FEATURE_FRED))
>> +		regs->fred_cs.wfe = 0;
>> +}
> 
> Can I suggest a slightly different comment?
> 
> /*
>   * WFE==1 (WAIT_FOR_ENDBRANCH) means that the CPU expects the next ERETS
>   * to jump to an ENDBR instruction. If the ENDBR is missing, the CPU
>   * raises a #CP.
>   *
>   * Clear WFE to avoid that #CP.
>   *
>   * Use this function in a #CP handler to effectively give the next
>   * ERETS a free pass to ignore IBT for a single instruction.
>   */
> 
> I think original comment really needs a "How do I use this?" sentence or
> two.
> 
> A comment at the call site also wouldn't hurt:
> 
>   	if (unlikely(regs->ip == (unsigned long)&ibt_selftest_noendbr)){
>   		regs->ax = 0;
> +		/* Disable IBT enforcement for one exception return: */
> +		ibt_clear_fred_wfe(regs);
>   		return;
>   	}
> 
> I'm finding it kinda hard to concisely differentiate between the
> "disable IBT at one ERETS" and "disable IBT forever", but I hope this
> sounds good to folks.
> 

My understanding is that a missing-ENDBRANCH #CP is triggered in two
steps:

     1) Upon completion of an indirect call/jmp, or an event return
        instruction, the CPU indirect branch tracker is put in the
        WAIT_FOR_ENDBRANCH state.

     2) As the CPU is in WAIT_FOR_ENDBRANCH state, if the instruction to
        be executed is ENDBR, the CPU indirect branch tracker exits
        WAIT_FOR_ENDBRANCH state, otherwise a #CP is generated.

So this is more of "preserve WAIT_FOR_ENDBRANCH state" or not.

IDT is unable to preserve WAIT_FOR_ENDBRANCH state when returning from
event handling, which as Andrew mentioned is a security hole.