From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754041AbeATG50 (ORCPT ); Sat, 20 Jan 2018 01:57:26 -0500 Received: from mail-pf0-f169.google.com ([209.85.192.169]:44950 "EHLO mail-pf0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750922AbeATG5U (ORCPT ); Sat, 20 Jan 2018 01:57:20 -0500 X-Google-Smtp-Source: AH8x227AKqN19g0vjBctBlvN9U5WNoRSVWkUjn7E+bopUI2Y0pfeDOo/F0BXPvSWngPspX6SK9yG3w== Subject: Re: Boot regression with bacf6b499e11 ("x86/mm: Use a struct to reduce parameters for SME PGD mapping") on top of -rc8 To: Tom Lendacky , Gabriel C Cc: Borislav Petkov , Linus Torvalds , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Brijesh Singh , X86 ML , Linux Kernel Mailing List References: <9fdddcb1-d122-7d52-9204-7066ada5ccba@redhat.com> <44505ab1-237b-88ea-1fb1-f80de9b3025a@redhat.com> <6277b77a-d4a0-5669-e5aa-7a850436227c@amd.com> From: Laura Abbott Message-ID: <6b616ebc-1a8f-0bd7-68be-3674ff99500b@redhat.com> Date: Fri, 19 Jan 2018 22:57:17 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <6277b77a-d4a0-5669-e5aa-7a850436227c@amd.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 01/19/2018 10:15 PM, Tom Lendacky wrote: > On 1/19/2018 11:25 PM, Gabriel C wrote: >> 2018-01-20 5:02 GMT+01:00 Laura Abbott : >>> On 01/19/2018 06:23 PM, Gabriel C wrote: >>>> >>>> 2018-01-20 2:23 GMT+01:00 Laura Abbott : >>>>> >>>>> Hi, >>>> >>>> >>>> Hi , >>>> >>>>> >>>>> Fedora got multiple reports of an early bootup crash post -rc8. >>>>> Bisection showed bacf6b499e11 ("x86/mm: Use a struct to reduce >>>>> parameters for SME PGD mapping") . It doesn't revert cleanly >>>>> but if I revert the few other changes in arch/x86/mm/mem_encrypt.c >>>>> as well it boots up fine. >>>>> >>>>> Annoyingly, I can't seem to get any actual kernel logs even with >>>>> earlyprintk. It just reboots immediately (triple fault?). This >>>>> happens on both of my Lenovo machines and I can ask other reporters >>>>> for details as well. >>>>> >>>> >>>> I tested these patches on 2 Lenovo Ideapad both with Skylake CPUs >>>> on a older dual Xeon box , on 2 Toshibas with AMD APUs , on a RYZEN box , >>>> on dual EPYC box .. ofc on EPYC with mem_encrypt=on on the Intel CPUs >>>> disabled. >>>> >>>> Also tested on top 4.14.13 , 4.14.14 as well on top 4.15.0-rc7 and on >>>> current master/rc8++ without to see something like this. >>>> >>>> Also we pushed these patches on 4.14.13/14 and didn't got any reports >>>> about >>>> something like this. >>>> >>>> What Lenovo boxes are these ? maybe I find one to reproduce. >>>> >>>> >>>>> $ git bisect log >>>>> # bad: [ec835f8104a21f4d4eeb9d316ee71d2b4a7f00de] Merge tag >>>>> 'trace-v4.15-rc4-3' of >>>>> git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace >>>>> # good: [a8750ddca918032d6349adbf9a4b6555e7db20da] Linux 4.15-rc8 >>>>> git bisect start 'origin/master' 'v4.15-rc8' >>>>> # bad: [79683f80e4f07dba13cc08d0ebcf5c7b0aa1bf68] Merge tag >>>>> 'mmc-v4.15-rc2-3' of >>>>> git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc >>>>> git bisect bad 79683f80e4f07dba13cc08d0ebcf5c7b0aa1bf68 >>>>> # good: [161f72ed6dbe7fb176585091d3b797125d310399] Merge tag >>>>> 'mac80211-for-davem-2018-01-15' of >>>>> git://git.kernel.org/pub/scm/linux/kernel/git/jberg/mac80211 >>>>> git bisect good 161f72ed6dbe7fb176585091d3b797125d310399 >>>>> # good: [88dc7fca18001fd883e5ace775afa316b68c8f2c] Merge branch >>>>> 'x86-pti-for-linus' of >>>>> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip >>>>> git bisect good 88dc7fca18001fd883e5ace775afa316b68c8f2c >>>>> # bad: [d47924417319e3b6a728c0b690f183e75bc2a702] x86/intel_rdt/cqm: >>>>> Prevent >>>>> use after free >>>>> git bisect bad d47924417319e3b6a728c0b690f183e75bc2a702 >>>>> # good: [fc90ccfd286eabb05ec54521367df8663cf0bbbf] Revert "x86/apic: >>>>> Remove >>>>> init_bsp_APIC()" >>>>> git bisect good fc90ccfd286eabb05ec54521367df8663cf0bbbf >>>>> # bad: [bacf6b499e11760aef73a3bb5ce4e5eea74a3fd4] x86/mm: Use a struct to >>>>> reduce parameters for SME PGD mapping >>>>> git bisect bad bacf6b499e11760aef73a3bb5ce4e5eea74a3fd4 >>>>> # good: [1303880179e67c59e801429b7e5d0f6b21137d99] x86/mm: Clean up >>>>> register >>>>> saving in the __enc_copy() assembly code >>>>> git bisect good 1303880179e67c59e801429b7e5d0f6b21137d99 >>>>> # first bad commit: [bacf6b499e11760aef73a3bb5ce4e5eea74a3fd4] x86/mm: >>>>> Use a >>>>> struct to reduce parameters for SME PGD mapping >>>>> >>>>> >>>>> Configuration is at >>>>> >>>>> https://git.kernel.org/pub/scm/linux/kernel/git/jwboyer/fedora.git/commit/?h=rawhide >>>>> Note that I do think this is something in the Fedora configuration >>>>> because a generic "make defconfig" booted just fine. >>>> >>>> >>>> But maybe some of the Fedora patches ? >>>> >>>> Can you try an kernel with the config but without any patches ? >>>> Or a defconfig and just enable CONFIG_AMD_MEM_ENCRYPT ? >>>> >>> >>> The bisect was a vanilla kernel without Fedora patches. >> >> Ok . I did an build ( v4.15-rc8-225-g8dd903d2cf7b ) on my AMD >> Workstation ( EPYC CPU ) > > I've tried multiple config combinations on my EPYC system and have not > been able to reproduce this issue and have not had any boot issues with > mem_encrypt=on or mem_encrypt=off. I don't have access to a non-AMD box > at the moment, but I'm really scratching my head on this one. > >> with your 64bit config and one on the Ideapad ( Intel i7-6498DU ) .. >> I disabled Selinux since I don't use it here and module signing. >> >> Also with your config my serial setup won't work and the kernel hangs >> but mem_encrypt=on/off works just fine. > > If your using the EPYC serial device and you haven't enabled legacy > serial device support (if your BIOS supports that), then you're using > it as a platform device. The fedora config has not set the > CONFIG_X86_AMD_PLATFORM_DEVICE setting so you won't get the module > to load and give you serial output. > > I'm confused when you say the kernel hangs but mem_encrypt=on/off works > just fine, can you explain that a bit more? > >> >> Also I notice on the Workstation it takes forever to boot untill >> 'DMA-API' reports out-of-memory >> ( dunno how much memory it need but the box has 128GB of RAM ).. > The machines I have are a Lenovo X1 Carbon and a Lenovo T470s. A Lenovo X220 ThinkPad also reported the problem. If I comment out sme_encrypt_kernel it boots: diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 7ba5d819ebe3..443ef5d3f1fa 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -158,7 +158,7 @@ unsigned long __head __startup_64(unsigned long physaddr, *p += load_delta - sme_get_me_mask(); /* Encrypt the kernel and related (if SME is active) */ - sme_encrypt_kernel(bp); + //sme_encrypt_kernel(bp); /* * Return the SME encryption mask (if SME is active) to be used as a Interestingly, I tried to print the values in sme_active (sme_me_mask , sev_enabled) followed by a return at the very start of sme_encrypt_kernel and that rebooted as well, vs booting if I just kept the return. sme_me_mask and sev_enabled are explicitly marked as being in .data, is it possible they are ending up in a section that isn't yet mapped or did I hit print too early? Thanks, Laura