From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1FC33CF211; Mon, 24 Aug 2026 08:34:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.11 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787560488; cv=none; b=kiRtfytAuVWO3ZjcXUN1rWxMeUKpC35CxBCIuVofE51LBXE9XdtPquAHMlI/tvpYeown829yJipJcI3rDqc1YA9/QK7xy3YfQ0tejz1YjuZurm6wkexZA88OK/XIVQsHdZM8Qu7/qQAtpw+7oBVEtCMmLWwgYTZfjSXZ5+siJBE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787560488; c=relaxed/simple; bh=nC74cNfjP2+EsKrsD4m63iGzVE8gu+HzDH11FE4a1d8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=b1jY8a6GE82xO4B1FBKGWjbn4ik+WBBXyeDwlJRqLEdO068LqTaybjk55023Ar8s8E/HGlnTlLRnEnC4F+V2t21e3MrEFr7LmpkgATlEVARMDlfrN8VZBhquYW4itA9ZIH5SaN+xfoI3+zd3viQ502xrYGn4ZpTh1LX6Aw5o/Iw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=X3AmFWd0; arc=none smtp.client-ip=192.198.163.11 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="X3AmFWd0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1787560486; x=1819096486; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nC74cNfjP2+EsKrsD4m63iGzVE8gu+HzDH11FE4a1d8=; b=X3AmFWd05SUOJd1ZCTRBcfIguBho8X8zJXAYiRwQCNqUQHeRctzin/UJ TX0HnTHBdPAKzJRqS21HGirDAaTRCfZr9z/Y3qmNkeRqBDBwR6M56WxOQ PX3KJ93fsGiv9kez88kCRFMZuxw7MxQtuIJJJR2lkPuMjaWzNGI5xlefI mYQXEi48H5vR+4rJb4gOzYbD26SCtQNBSTcOshzlDOOLbWTyjfoxowEnq bL7fF/2fX8agBlib/0GxpufJaoqW8HPSoxN0nLcY54+ryDx++g0VPRR4j NzRJhXjaw69wAhZVFGhRTobGIlITjf9zrrsJXeihJEsfNIaj4FD5m8n2A Q==; X-CSE-ConnectionGUID: fmr4nBjTRIaRGV0cyVoSbw== X-CSE-MsgGUID: XfMOXgt4RjCKixG6v/lWPg== X-IronPort-AV: E=McAfee;i="6800,10657,11884"; a="98598758" X-IronPort-AV: E=Sophos;i="6.25,240,1779174000"; d="scan'208";a="98598758" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Aug 2026 01:34:46 -0700 X-CSE-ConnectionGUID: 0VMcyBvVQNmbAEh7jWIG/Q== X-CSE-MsgGUID: FxxcdYJlQAKAi1clHpCrRw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.25,240,1779174000"; d="scan'208";a="264326180" Received: from spr.sh.intel.com ([10.112.229.196]) by fmviesa008.fm.intel.com with ESMTP; 24 Aug 2026 01:34:41 -0700 From: Dapeng Mi To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Thomas Gleixner , Dave Hansen , Ian Rogers , Adrian Hunter , Jiri Olsa , Alexander Shishkin , Andi Kleen , Eranian Stephane Cc: Mark Rutland , broonie@kernel.org, Ravi Bangoria , linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Zide Chen , Falcon Thomas , Dapeng Mi , Xudong Hao , Dapeng Mi Subject: [RESEND Patch v10 07/23] x86/fpu: Add update_fpu_state_and_flag() helper Date: Mon, 24 Aug 2026 16:27:15 +0800 Message-Id: <20260824082731.1013973-8-dapeng1.mi@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260824082731.1013973-1-dapeng1.mi@linux.intel.com> References: <20260824082731.1013973-1-dapeng1.mi@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Add update_fpu_state_and_flag() as suggested by Peter and Dave. The helper saves user FPU state and then sets TIF_NEED_FPU_LOAD, ensuring the task FPU state is saved whenever the flag is set. Subsequent patches will use this guarantee in NMI context by checking TIF_NEED_FPU_LOAD before retrieving user FPU state from the saved task FPU state. Also add barrier() in the host/guest FPU state switch path and move clearing of fpu->__task_fpstate after switching back to host state. So fpu->__task_fpstate is always observed as host FPU state when non-NULL. Link: https://lore.kernel.org/all/20251204154721.GB2619703@noisy.programming.kicks-ass.net/ Signed-off-by: Dapeng Mi --- arch/x86/include/asm/fpu/sched.h | 5 +++-- arch/x86/kernel/fpu/core.c | 33 ++++++++++++++++++++++++++------ 2 files changed, 30 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/fpu/sched.h b/arch/x86/include/asm/fpu/sched.h index 89004f4ca208..dcb2fa5f06d6 100644 --- a/arch/x86/include/asm/fpu/sched.h +++ b/arch/x86/include/asm/fpu/sched.h @@ -10,6 +10,8 @@ #include extern void save_fpregs_to_fpstate(struct fpu *fpu); +extern void update_fpu_state_and_flag(struct fpu *fpu, + struct task_struct *task); extern void fpu__drop(struct task_struct *tsk); extern int fpu_clone(struct task_struct *dst, u64 clone_flags, bool minimal, unsigned long shstk_addr); @@ -36,8 +38,7 @@ static inline void switch_fpu(struct task_struct *old, int cpu) !(old->flags & (PF_KTHREAD | PF_USER_WORKER))) { struct fpu *old_fpu = x86_task_fpu(old); - set_tsk_thread_flag(old, TIF_NEED_FPU_LOAD); - save_fpregs_to_fpstate(old_fpu); + update_fpu_state_and_flag(old_fpu, old); /* * The save operation preserved register state, so the * fpu_fpregs_owner_ctx is still @old_fpu. Store the diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 584fb9913be4..9e029b2f8937 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -213,6 +213,19 @@ void restore_fpregs_from_fpstate(struct fpstate *fpstate, u64 mask) } } +/* + * Save the FPU register state in fpu->fpstate->regs and set + * TIF_NEED_FPU_LOAD subsequently. + * + * Must be called with fpregs_lock() held, ensuring flag + * TIF_NEED_FPU_LOAD is set last. + */ +void update_fpu_state_and_flag(struct fpu *fpu, struct task_struct *task) +{ + save_fpregs_to_fpstate(fpu); + set_tsk_thread_flag(task, TIF_NEED_FPU_LOAD); +} + void fpu_reset_from_exception_fixup(void) { restore_fpregs_from_fpstate(&init_fpstate, XFEATURE_MASK_FPSTATE); @@ -383,13 +396,13 @@ int fpu_swap_kvm_fpstate(struct fpu_guest *guest_fpu, bool enter_guest) /* Swap fpstate */ if (enter_guest) { - fpu->__task_fpstate = cur_fps; + WRITE_ONCE(fpu->__task_fpstate, cur_fps); + barrier(); fpu->fpstate = guest_fps; guest_fps->in_use = true; } else { guest_fps->in_use = false; fpu->fpstate = fpu->__task_fpstate; - fpu->__task_fpstate = NULL; } cur_fps = fpu->fpstate; @@ -406,6 +419,16 @@ int fpu_swap_kvm_fpstate(struct fpu_guest *guest_fpu, bool enter_guest) xfd_update_state(cur_fps); } + /* + * Clear fpu->__task_fpstate after switching back to host state. + * A non-NULL __task_fpstate means guest state is still resident in + * hardware; reset it only once host state has been restored. + */ + if (!enter_guest) { + barrier(); + WRITE_ONCE(fpu->__task_fpstate, NULL); + } + fpregs_mark_activate(); fpregs_unlock(); return 0; @@ -481,10 +504,8 @@ void kernel_fpu_begin_mask(unsigned int kfpu_mask) this_cpu_write(kernel_fpu_allowed, false); if (!(current->flags & (PF_KTHREAD | PF_USER_WORKER)) && - !test_thread_flag(TIF_NEED_FPU_LOAD)) { - set_thread_flag(TIF_NEED_FPU_LOAD); - save_fpregs_to_fpstate(x86_task_fpu(current)); - } + !test_thread_flag(TIF_NEED_FPU_LOAD)) + update_fpu_state_and_flag(x86_task_fpu(current), current); __cpu_invalidate_fpregs_state(); /* Put sane initial values into the control registers. */ -- 2.34.1