From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753228AbcDZWrJ (ORCPT ); Tue, 26 Apr 2016 18:47:09 -0400 Received: from mail-bl2on0089.outbound.protection.outlook.com ([65.55.169.89]:19717 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752330AbcDZWrE (ORCPT ); Tue, 26 Apr 2016 18:47:04 -0400 Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=amd.com; From: Tom Lendacky Subject: [RFC PATCH v1 09/18] x86: Insure that memory areas are encrypted when possible To: , , , , , , , , CC: Radim =?utf-8?b?S3LEjW3DocWZ?= , Arnd Bergmann , Jonathan Corbet , Matt Fleming , Joerg Roedel , "Konrad Rzeszutek Wilk" , Paolo Bonzini , "Ingo Molnar" , Borislav Petkov , "H. Peter Anvin" , Andrey Ryabinin , "Alexander Potapenko" , Thomas Gleixner , "Dmitry Vyukov" Date: Tue, 26 Apr 2016 17:46:50 -0500 Message-ID: <20160426224650.13079.35775.stgit@tlendack-t1.amdoffice.net> In-Reply-To: <20160426224508.13079.90373.stgit@tlendack-t1.amdoffice.net> References: <20160426224508.13079.90373.stgit@tlendack-t1.amdoffice.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: BY1PR15CA0032.namprd15.prod.outlook.com (10.162.17.170) To BY2PR1201MB1112.namprd12.prod.outlook.com (10.164.168.20) X-MS-Office365-Filtering-Correlation-Id: 5422496f-fc46-425b-af3d-08d36e24ab1e X-Microsoft-Exchange-Diagnostics: 1;BY2PR1201MB1112;2:yDHDlKPcCGyqSe3BmDNc7mEEIgHrJBhAuDJQ5Xyn6Je8oHn9CZQvfn4W0sdswrY8DuLmapCChqM5HnxN2Gqw0IoiUNXNfbWQUzDLfHMLv7PCctV5qGs2/t0pUT0ijn42hBWNxonPmGa9iQoT9ieltBBbROWkhrGbkFXks85QkU4CrthKJ4aYRF29cBX3Yvkr;3:I4o5kMsNEVzb1B4PGsYnKGmZxVnIsz/hzMSEtnB1pQ228wJ8X8mCIg6ykuRA1SXfaZ4f5L3qDwgU4gZ5k7JmjIxkc1d/MDg8yCshRMo8YFXI6w8IlzW7X1AwI7LLgaH2 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY2PR1201MB1112; X-Microsoft-Exchange-Diagnostics: 1;BY2PR1201MB1112;25:MdNjipknyg4T6KAREoteupZi0+gNgpAvPK+x2Ar7kdRUdqL+5lDMTJwvjFCk0hKF2lc+9B9++jD3xAuw4S/RC5KvOhT2HeFmlJtHS1AW11FwDB0Lwh+EmlN4rlZzrszLUMKOdz/oqcoPHdlVlnR/eVt3+rWtUobi1waS/C3/XiqISLJXrGyG2v4hQNNqLZy0cYocxc95kM3H4ephVHaWmv+eAPpSe6hPaQmNakpknb2y62/79s9GnH+lUEkOACj1fBhKlBSZYRJMbZwx/4SYM685/MNXHjBtcgAWuKdfYtlkFDS92dj7gneiL9vWRc5Og7aQhMBLoSsdAp2B6z1PPcdq8MUtZPEMfP4Z+RZT6hzmzxjtvDZbYsi04SndLimWie3qruozsrWilA+kC5pE3kM0YZyXZwHa79aZseFZvoVctZsmTCbK9L65mOPIdZgxIMq09nUICYoLBOUiNNOsZfzQJ5brNysF9bSXbE9jQRzpJQZN2Clhwb1fuKdjvnKD9doefMMiLMp0nmDNEdN1NF/ossTrEfD+BoNWvQIagn/zXmNdPRcutq0nVqFq4OK464vSDEK87yNGhJ8W5KcsHaOBvy99Ac8UcoMNLYJXKqGbm+CkqiSDCkUtIQ+zsAlhgn94SJQcOv34SeZEbt8vKA== X-Microsoft-Exchange-Diagnostics: 1;BY2PR1201MB1112;20:hAn1l9FPob+GHohDrlkJjzgM+j4bA6OYI0nvBPAF0QnCGKb6htpyUbU4SwUA4Y3dxF37CfQs1LKOVP5VSRjwRKWHieH+CjMxttQk2id32zX30Vx+LTQGTiJ+4KL4kteX7DChqigcJjrnXUZzHzzDoJ71AznE2AmWoN4oCEUnVI5MoFHu3EhYbws5/D2hFjsZtkfdM3hjZq9QQ9xBYp4a54s5Nj3kAMaaOYoyHSaDUSjR0RsRsKnzQDW6+96pY46MrMg1h94JHOkDTB/cECFIKGZyIqmGtiZygK5gXa6fv1PcGd23apnTPFoose84A9cZOvPGoECqlHSP8hFHst38jxizEEWBpbcRZ7O4Hs1WycZr4e+zQQGAvVMp5PSzQOoLVAHfAjl3pf3kB/TKl6Pn5vB4fWom+ygSHsZFOyRsx2Zrj/tfM0zVr7rmKrfs59AnbvaUN476BG7H0Y/TXMYUYMhVEI6WYYytZFFF5A/2oXQhZlLXZmiGGK3pktu4IyEq X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(187975280028177); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(9101521072)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6055026);SRVR:BY2PR1201MB1112;BCL:0;PCL:0;RULEID:;SRVR:BY2PR1201MB1112; X-Microsoft-Exchange-Diagnostics: 1;BY2PR1201MB1112;4:VllTM/A43dg5FR39HtJHPUHzycgiy4oc4pC3znuEKOkpNSiGfgV+VHJyhGzabF1pjg0ztYDgetI2qVCyQ0zBUgJZKDBh0vkqDpQPLfvedUWSynWoA7cKzBGXqtj+Ejc9U4wqUjgThGlFmecvS7U0vuk1SJJnev/AE/b+rb9slRwvoeXhhASsOP5wUbq0UonqeOr/nlXszMv3r0Xb3+Xs0PkKvTMyf+Ow3pPF/tV2oEO1iFewv6fsBC+Zmli3PpRAD//jxhnx2cgDe+ry/aJRQ1dTnPEsEmfFHdIK4/AsCMXFxo79KMLVcsC6DXDE5WClcSfsMge9U0omOQgJJ+Du6ps5+EpX/AGgfyva8w1rJZ9BI5H/z8bM5qu2C1VfTl0XOPeyxtMI55XTE7QCwlSGDlfJSBarLarCfbKsx1Ci915gN77wISSH4PQ0KytW5gONB4p7pnrCLtO/fgew3JD1Yw== X-Forefront-PRVS: 0924C6A0D5 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(50466002)(5001770100001)(189998001)(9686002)(229853001)(42186005)(4001350100001)(2201001)(81166005)(230700001)(103116003)(53416004)(86362001)(1076002)(2950100001)(23676002)(6116002)(2906002)(4326007)(92566002)(50986999)(586003)(77096005)(76176999)(19580395003)(97746001)(19580405001)(33646002)(1096002)(66066001)(5004730100002)(54356999)(5008740100001)(47776003)(71626007)(217873001);DIR:OUT;SFP:1101;SCL:1;SRVR:BY2PR1201MB1112;H:tlendack-t1.amdoffice.net;FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCWTJQUjEyMDFNQjExMTI7MjM6a1E1Sm1ybjJxYWNkaVV0d2liK1FzaGJN?= =?utf-8?B?OW9JMHNiWUl3Sks2YTJMeFJpQytRQTcxeVRyQVhUNXNIU1dTcE9mcGloSmtB?= =?utf-8?B?WDJiVnQvZzRlL2Rhb0NNZjI0ZGNDczMxMzdabmdDSkt4Z1d1Nlk4VFhMUS9I?= =?utf-8?B?Q2RjYnlIb0hMTW5oVnVFeHRHcWc4MTdxQVZwUGNWUXUxbk93RkJGOXFHNytw?= =?utf-8?B?dDEvemtwWE5SVDltZ3JxSmUvRGJmK2RNRU9hem5PMVVDSFJGK2YrdzR2QVU3?= =?utf-8?B?RkxVNzdvYnk2RmFTcXRVN1ZYM2cvM3RLZmptL2F5MkVrV1dlTCtzRE11ZVBL?= =?utf-8?B?aFFmSGxxZ1ZVUnE5TC9yZ3B6U2c3UjJMTGhsQmdNWW84Z0M2dG5rTzgyR2dJ?= =?utf-8?B?blVIZWRsdDJCdS9vTi9XdEJQK1ZyUUtReDFycy9XeEkzNWk2cFZ4ZG9UUFBK?= =?utf-8?B?YWVvcERiUmNFUmRpNEtzWkRFb0xQcEJZYW9Pa28xcTFVQzQra2RYbVM3Rmx2?= =?utf-8?B?SlF6NllwYlJ0VGIyU2J1UGxvOXFEQ2YzVEhXaUhZNjhSS2Z5endvcEoxNWJC?= =?utf-8?B?Wm8vU3BSa3dqUFdPS01jR1pjdFdveGx1dkRZM09sTzlLSnVoc3ZsODV3TTNO?= =?utf-8?B?OUlJSVJvY3ViMndJVFdXU2dHWkJDMXE5d0FUK1QreW1ybzZ4bTBpUm8vWGFP?= =?utf-8?B?SDZ3WE1WWU1pbzdaZ2pqZTZhUUZXblA2R2Zyd0dGaTdwdlZjVVVmWHRUOS93?= =?utf-8?B?TjVrZnRNZjhBbWJJdXEyR29sVzF1ZXg1cllWdElFMlJMb3NabzRMM3dsM1pz?= =?utf-8?B?dWNTNTM3OE1mczl6akc2L01YYTFXTDUxcVZ6eTQwRDUwWm13d1c4bzVHcGdr?= =?utf-8?B?UkIrVVNaMHh4WTlTUDJSZGRraFVOZUpGVmVwU1d6Smt0YThlaXBoOCs4N2c3?= =?utf-8?B?c1JEamZZVWFSdHRzMS9BT21uRlVjUXFIVFAzdCt3d2d4b0JramJseXBpd2t3?= =?utf-8?B?bmlWOVVJYVBvUTBNSHYzeDFKVFQrL0RvcFNNTkdYT0dUSGNnTVBsUVVUUG5W?= =?utf-8?B?ejVJcGZkUFNyWUxiMlVreTV5OWNJbjFXTjJHWnhOZ1JJeE5pSFQvblAyeFFH?= =?utf-8?B?U1pnckhmVEFZY1hueG9uK2ZxU0ZGWjg2NkVoSzc5eFhaOWtraVA0aUorNFFE?= =?utf-8?B?SWRicElJVDNRSDl0MU9wNjNreWl3dFZ0Um1Yc0llV1lTTmNHSmVLTTBoRWxj?= =?utf-8?B?WjZqcWtrOUlIbW1LS3ozQjhQS25MS01Jc0s1YkFrSlBaU3F2VTdDRCs5bCs3?= =?utf-8?B?aG03MjhITGYzWlowdVgrZUdlT2NaWmdOcFNmR3BkbkRCbmQ5UTlCbjhZcVpB?= =?utf-8?B?c3R3ei8yZlJKOUdFY2JlZTVjcUlJZW1zelcxeUwrbEE9PQ==?= X-Microsoft-Exchange-Diagnostics: 1;BY2PR1201MB1112;5:U5GsAjeBWZzd3tWsl6c1Qjsncsdi2z0q8EHsM1npWjzzicsXlbwlrsFLmiydO/kDkRHElf058QTXtzAIALm0a1t+X+g7jaQLLffSWxy8ae0iDNMm/xvWOhG5EdJ2jPa5JOzWx6JoSr03FmpULCnaRA==;24:aODTrcvldGt+GyDaI00qsB01pR8uajjk897l58eOeFBUe3b6UuTTaGfyB+euaDrNQd6ilr+h/yYfnpDVHhnXtJxx+kNFS07WbjQ4w2Q+gbw=;7:C3WVKOQWaYu0qdYHyPJnZ/k3Xw3yeItv4Omg8BkobfE5IweYPZd5SSvIJjKZYujhul7FesyUkySDJoHpPLLZ6Rz8KpYAqPf6mnNx96n79hUWGjzMjj3OfRLZU+IvsfQN/gXylSPyly6YfKPVFHTQyvkk0FZ8k4M2byfgatniXk0=;20:pys6e99DI7qkI/C55SmJbZ6Cwi9PgVhEc7nVrUo7bBTPF0bB9qO82plqQpR+A+EBQVBGZvOE1B3B/JTX0Rb5hSWAsEE6/E9jXVDswkjB5MvhBz/nyaqmAgLbef+5XlL8cKEQWfcZyBux4j7jyq99jxZiguv80xZ1YiFyvCQJG4XYdxUruWoqUQaC4TJdNkn2sGI/z/vbuPP83SR/2MuE701lcDlHDXD0m+uCRu/xu0yHjo1QupMT/BOW8MIzb76g X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Apr 2016 22:46:54.4726 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR1201MB1112 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Encrypt memory areas in place when possible (e.g. zero page, etc.) so that special handling isn't needed afterwards. Signed-off-by: Tom Lendacky --- arch/x86/kernel/head64.c | 90 +++++++++++++++++++++++++++++++++++++++++++--- arch/x86/kernel/setup.c | 8 ++++ 2 files changed, 93 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 3516f9f..ac3a2bf 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -47,12 +47,12 @@ static void __init reset_early_page_tables(void) } /* Create a new PMD entry */ -int __init early_make_pgtable(unsigned long address) +static int __init __early_make_pgtable(unsigned long address, pmdval_t pmd) { unsigned long physaddr = address - __PAGE_OFFSET; pgdval_t pgd, *pgd_p; pudval_t pud, *pud_p; - pmdval_t pmd, *pmd_p; + pmdval_t *pmd_p; /* Invalid address or early pgt is done ? */ if (physaddr >= MAXMEM || read_cr3() != __sme_pa_nodebug(early_level4_pgt)) @@ -94,12 +94,92 @@ again: memset(pmd_p, 0, sizeof(*pmd_p) * PTRS_PER_PMD); *pud_p = (pudval_t)pmd_p - __START_KERNEL_map + phys_base + _KERNPG_TABLE; } - pmd = (physaddr & PMD_MASK) + early_pmd_flags; pmd_p[pmd_index(address)] = pmd; return 0; } +int __init early_make_pgtable(unsigned long address) +{ + unsigned long physaddr = address - __PAGE_OFFSET; + pmdval_t pmd; + + pmd = (physaddr & PMD_MASK) + early_pmd_flags; + + return __early_make_pgtable(address, pmd); +} + +static void __init create_unencrypted_mapping(void *address, unsigned long size) +{ + unsigned long physaddr = (unsigned long)address - __PAGE_OFFSET; + pmdval_t pmd_flags, pmd; + + if (!sme_me_mask) + return; + + /* Clear the encryption mask from the early_pmd_flags */ + pmd_flags = early_pmd_flags & ~sme_me_mask; + + do { + pmd = (physaddr & PMD_MASK) + pmd_flags; + __early_make_pgtable((unsigned long)address, pmd); + + address += PMD_SIZE; + physaddr += PMD_SIZE; + size = (size < PMD_SIZE) ? 0 : size - PMD_SIZE; + } while (size); +} + +static void __init __clear_mapping(unsigned long address) +{ + unsigned long physaddr = address - __PAGE_OFFSET; + pgdval_t pgd, *pgd_p; + pudval_t pud, *pud_p; + pmdval_t *pmd_p; + + /* Invalid address or early pgt is done ? */ + if (physaddr >= MAXMEM || + read_cr3() != __sme_pa_nodebug(early_level4_pgt)) + return; + + pgd_p = &early_level4_pgt[pgd_index(address)].pgd; + pgd = *pgd_p; + + if (!pgd) + return; + + /* + * The use of __START_KERNEL_map rather than __PAGE_OFFSET here matches + * __early_make_pgtable where the entry was created. + */ + pud_p = (pudval_t *)((pgd & PTE_PFN_MASK) + __START_KERNEL_map - phys_base); + pud_p += pud_index(address); + pud = *pud_p; + + if (!pud) + return; + + pmd_p = (pmdval_t *)((pud & PTE_PFN_MASK) + __START_KERNEL_map - phys_base); + pmd_p[pmd_index(address)] = 0; +} + +static void __init clear_mapping(void *address, unsigned long size) +{ + do { + __clear_mapping((unsigned long)address); + + address += PMD_SIZE; + size = (size < PMD_SIZE) ? 0 : size - PMD_SIZE; + } while (size); +} + +static void __init sme_memcpy(void *dst, void *src, unsigned long size) +{ + create_unencrypted_mapping(src, size); + memcpy(dst, src, size); + clear_mapping(src, size); +} + /* Don't add a printk in there. printk relies on the PDA which is not initialized yet. */ static void __init clear_bss(void) @@ -122,12 +202,12 @@ static void __init copy_bootdata(char *real_mode_data) char * command_line; unsigned long cmd_line_ptr; - memcpy(&boot_params, real_mode_data, sizeof boot_params); + sme_memcpy(&boot_params, real_mode_data, sizeof boot_params); sanitize_boot_params(&boot_params); cmd_line_ptr = get_cmd_line_ptr(); if (cmd_line_ptr) { command_line = __va(cmd_line_ptr); - memcpy(boot_command_line, command_line, COMMAND_LINE_SIZE); + sme_memcpy(boot_command_line, command_line, COMMAND_LINE_SIZE); } } diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 2367ae0..1d29cf9 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -113,6 +113,7 @@ #include #include #include +#include /* * max_low_pfn_mapped: highest direct mapped pfn under 4GB @@ -375,6 +376,13 @@ static void __init reserve_initrd(void) !ramdisk_image || !ramdisk_size) return; /* No initrd provided by bootloader */ + /* + * This memory is marked encrypted by the kernel but the ramdisk + * was loaded in the clear by the bootloader, so make sure that + * the ramdisk image is encrypted. + */ + sme_early_mem_enc(ramdisk_image, ramdisk_end - ramdisk_image); + initrd_start = 0; mapped_size = memblock_mem_size(max_pfn_mapped);