From: "H. Peter Anvin" <hpa@zytor.com>
To: "Rafael J. Wysocki" <rafael@kernel.org>,
"Xin Li (Intel)" <xin@zytor.com>
Cc: linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org,
stable@vger.kernel.org, rafael@kernel.org, pavel@kernel.org,
tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
dave.hansen@linux.intel.com, x86@kernel.org, xi.pardee@intel.com,
todd.e.brandt@intel.com
Subject: Re: [PATCH v1 1/1] x86/fred: Fix system hang during S4 resume with FRED enabled
Date: Mon, 31 Mar 2025 13:03:51 -0700 [thread overview]
Message-ID: <148C8753-8972-4970-8951-E2D1CB26D8B0@zytor.com> (raw)
In-Reply-To: <CAJZ5v0jfak9K_7b=adf5ew-xDiGHUEPSp5ZpAGt66Okj-ovsGQ@mail.gmail.com>
On March 31, 2025 8:30:49 AM PDT, "Rafael J. Wysocki" <rafael@kernel.org> wrote:
>On Wed, Mar 26, 2025 at 7:26 AM Xin Li (Intel) <xin@zytor.com> wrote:
>>
>> During an S4 resume, the system first performs a cold power-on. The
>> kernel image is initially loaded to a random linear address, and the
>> FRED MSRs are initialized. Subsequently, the S4 image is loaded,
>> and the kernel image is relocated to its original address from before
>> the S4 suspend. Due to changes in the kernel text and data mappings,
>> the FRED MSRs must be reinitialized.
>
>To be precise, the above description of the hibernation control flow
>doesn't exactly match the code.
>
>Yes, a new kernel is booted upon a wakeup from S4, but this is not "a
>cold power-on", strictly speaking. This kernel is often referred to
>as the restore kernel and yes, it initializes the FRED MSRs as
>appropriate from its perspective.
>
>Yes, it loads a hibernation image, including the kernel that was
>running before hibernation, often referred to as the image kernel, but
>it does its best to load image pages directly into the page frames
>occupied by them before hibernation unless those page frames are
>currently in use. In that case, the given image pages are loaded into
>currently free page frames, but they may or may not be part of the
>image kernel (they may as well belong to user space processes that
>were running before hibernation). Yes, all of these pages need to be
>moved to their original locations before the last step of restore,
>which is a jump into a "trampoline" page in the image kernel, but this
>is sort of irrelevant to the issue at hand.
>
>At this point, the image kernel has control, but the FRED MSRs still
>contain values written to them by the restore kernel and there is no
>guarantee that those values are the same as the ones written into them
>by the image kernel before hibernation. Thus the image kernel must
>ensure that the values of the FRED MSRs will be the same as they were
>before hibernation, and because they only depend on the location of
>the kernel text and data, they may as well be recomputed from scratch.
>
>> Reported-by: Xi Pardee <xi.pardee@intel.com>
>> Reported-and-Tested-by: Todd Brandt <todd.e.brandt@intel.com>
>> Suggested-by: H. Peter Anvin (Intel) <hpa@zytor.com>
>> Signed-off-by: Xin Li (Intel) <xin@zytor.com>
>> Cc: stable@kernel.org # 6.9+
>> ---
>> arch/x86/power/cpu.c | 16 ++++++++++++++++
>> 1 file changed, 16 insertions(+)
>>
>> diff --git a/arch/x86/power/cpu.c b/arch/x86/power/cpu.c
>> index 63230ff8cf4f..ef3c152c319c 100644
>> --- a/arch/x86/power/cpu.c
>> +++ b/arch/x86/power/cpu.c
>> @@ -27,6 +27,7 @@
>> #include <asm/mmu_context.h>
>> #include <asm/cpu_device_id.h>
>> #include <asm/microcode.h>
>> +#include <asm/fred.h>
>>
>> #ifdef CONFIG_X86_32
>> __visible unsigned long saved_context_ebx;
>> @@ -231,6 +232,21 @@ static void notrace __restore_processor_state(struct saved_context *ctxt)
>> */
>> #ifdef CONFIG_X86_64
>> wrmsrl(MSR_GS_BASE, ctxt->kernelmode_gs_base);
>> +
>> + /*
>> + * Restore FRED configs.
>> + *
>> + * FRED configs are completely derived from current kernel text and
>> + * data mappings, thus nothing needs to be saved and restored.
>> + *
>> + * As such, simply re-initialize FRED to restore FRED configs.
>
>Instead of the above, I would just say "Reinitialize FRED to ensure
>that the FRED registers contain the same values as before
>hibernation."
>
>> + *
>> + * Note, FRED RSPs setup needs to access percpu data structures.
>
>And I'm not sure what you wanted to say here? Does this refer to the
>ordering of the code below or to something else?
>
>> + */
>> + if (ctxt->cr4 & X86_CR4_FRED) {
>> + cpu_init_fred_exceptions();
>> + cpu_init_fred_rsps();
>> + }
>> #else
>> loadsegment(fs, __KERNEL_PERCPU);
>> #endif
>> --
>
Just to make it clear: the patch is correct, the shortcoming is in the description.
I would say that Xin's description, although perhaps excessively brief, is correct from the *hardware* point of view, whereas Rafael adds the much needed *software* perspective.
As far as hardware is concerned, Linux S4 is just a power on (we don't use any BIOS support for S4 even if it exists, which it rarely does anymore, and for very good reasons.) From a software point of view, it is more like a kexec into the frozen kernel image, which then has to re-establish its runtime execution environment – (including the FRED state, which is what this patch does.)
For the APs this is done through the normal AP bringup mechanism, it is only the BSP that needs special treatment.
next prev parent reply other threads:[~2025-03-31 20:04 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-03-26 6:25 Xin Li (Intel)
2025-03-31 12:36 ` [tip: x86/urgent] " tip-bot2 for Xin Li (Intel)
2025-03-31 15:30 ` [PATCH v1 1/1] " Rafael J. Wysocki
2025-03-31 16:19 ` Ingo Molnar
2025-03-31 20:03 ` H. Peter Anvin [this message]
2025-03-31 20:25 ` Rafael J. Wysocki
2025-04-01 8:22 ` Ingo Molnar
2025-04-01 6:34 ` Xin Li
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=148C8753-8972-4970-8951-E2D1CB26D8B0@zytor.com \
--to=hpa@zytor.com \
--cc=bp@alien8.de \
--cc=dave.hansen@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=pavel@kernel.org \
--cc=rafael@kernel.org \
--cc=stable@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=todd.e.brandt@intel.com \
--cc=x86@kernel.org \
--cc=xi.pardee@intel.com \
--cc=xin@zytor.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®