[PATCH 3/3] x86/retpoline: Ensure default return thunk isn't used at runtime

[David Kaplan <david.kaplan@amd.com>]

All CPU bugs that require a return thunk define a special return thunk
to use (e.g., srso_return_thunk).  The default thunk,
__x86_return_thunk, should never be used after apply_returns() completes.
Otherwise this could lead to potential speculation holes.

Enforce this by replacing this thunk with a ud2 when alternatives are
applied.  Alternative instructions are applied after apply_returns().

The default thunk is only used during kernel boot, it is not used during
module init since that occurs after apply_returns().

Signed-off-by: David Kaplan <david.kaplan@amd.com>
---
 arch/x86/lib/retpoline.S | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/arch/x86/lib/retpoline.S b/arch/x86/lib/retpoline.S
index 3da768a71cf9..10212cf4a9af 100644
--- a/arch/x86/lib/retpoline.S
+++ b/arch/x86/lib/retpoline.S
@@ -358,15 +358,17 @@ SYM_FUNC_END(call_depth_return_thunk)
  * This function name is magical and is used by -mfunction-return=thunk-extern
  * for the compiler to generate JMPs to it.
  *
- * This code is only used during kernel boot or module init.  All
+ * This code is only used during kernel boot.  All
  * 'JMP __x86_return_thunk' sites are changed to something else by
  * apply_returns().
+ *
+ * This thunk is turned into a ud2 to ensure it is never used at runtime.
+ * Alternative instructions are applied after apply_returns().
  */
 SYM_CODE_START(__x86_return_thunk)
 	UNWIND_HINT_FUNC
 	ANNOTATE_NOENDBR
-	ANNOTATE_UNRET_SAFE
-	ret
+	ALTERNATIVE __stringify(ANNOTATE_UNRET_SAFE;ret),"ud2", X86_FEATURE_RETHUNK
 	int3
 SYM_CODE_END(__x86_return_thunk)
 EXPORT_SYMBOL(__x86_return_thunk)
-- 
2.25.1