From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 030EEC4167B for ; Wed, 29 Nov 2023 20:02:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233626AbjK2UCI (ORCPT ); Wed, 29 Nov 2023 15:02:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229741AbjK2UCG (ORCPT ); Wed, 29 Nov 2023 15:02:06 -0500 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14B77122 for ; Wed, 29 Nov 2023 12:02:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1701288132; x=1732824132; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=HJouOlC70iGhWVB4TgDhN5rkdx7hDjM9My1HatSaiqE=; b=hqVK6QZ/xXVKSfkK+Nh5KSwlU/4vr7W7AMCYrrKTua9F6h4z83I47vcl 2dyFuSfLZsI4nnegUzaHU/YPYg/1sqn6DTq7lXu2yqbvRKPmibSweQVHp CwITY6KDVp1TkkRow9zbvUw0KirnRZzXyu3N8JNyS19KeB1Kq9KxaT3Fh 1z10dJNOOqiY+wcQy7jRjyVrcBS7DsRMRY86gSUKkbMLq1tV0FZ+glKtJ 8Vb0QzNlRW3H1UrGDtVR9CSi9JVsdQJ6TxY5YqWDnUWTg4bP1uSAaEcCz gMn6lbrHy1qPrMNXwWev4QTYJp/k8rIj2buRdiz53j7lyRCcL1iP7dpMW Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10909"; a="14776280" X-IronPort-AV: E=Sophos;i="6.04,237,1695711600"; d="scan'208";a="14776280" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2023 12:01:56 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10909"; a="745374069" X-IronPort-AV: E=Sophos;i="6.04,237,1695711600"; d="scan'208";a="745374069" Received: from npandey-mobl.amr.corp.intel.com (HELO [10.212.143.94]) ([10.212.143.94]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2023 12:01:42 -0800 Message-ID: <26d91827-8308-42e2-969e-993c82623bb0@intel.com> Date: Wed, 29 Nov 2023 12:01:42 -0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: AMD Memory encryption vs. kexec Content-Language: en-US To: Tom Lendacky , Borislav Petkov , "Shutemov, Kirill" , Ashish Kalra , Kai Huang Cc: LKML , the arch/x86 maintainers References: <82b2e0d5-edcc-44f3-b759-d6b8c8788ea6@intel.com> <29e235f7-add2-47ae-b06e-a717202c4faf@amd.com> From: Dave Hansen Autocrypt: addr=dave.hansen@intel.com; keydata= xsFNBE6HMP0BEADIMA3XYkQfF3dwHlj58Yjsc4E5y5G67cfbt8dvaUq2fx1lR0K9h1bOI6fC oAiUXvGAOxPDsB/P6UEOISPpLl5IuYsSwAeZGkdQ5g6m1xq7AlDJQZddhr/1DC/nMVa/2BoY 2UnKuZuSBu7lgOE193+7Uks3416N2hTkyKUSNkduyoZ9F5twiBhxPJwPtn/wnch6n5RsoXsb ygOEDxLEsSk/7eyFycjE+btUtAWZtx+HseyaGfqkZK0Z9bT1lsaHecmB203xShwCPT49Blxz VOab8668QpaEOdLGhtvrVYVK7x4skyT3nGWcgDCl5/Vp3TWA4K+IofwvXzX2ON/Mj7aQwf5W iC+3nWC7q0uxKwwsddJ0Nu+dpA/UORQWa1NiAftEoSpk5+nUUi0WE+5DRm0H+TXKBWMGNCFn c6+EKg5zQaa8KqymHcOrSXNPmzJuXvDQ8uj2J8XuzCZfK4uy1+YdIr0yyEMI7mdh4KX50LO1 pmowEqDh7dLShTOif/7UtQYrzYq9cPnjU2ZW4qd5Qz2joSGTG9eCXLz5PRe5SqHxv6ljk8mb ApNuY7bOXO/A7T2j5RwXIlcmssqIjBcxsRRoIbpCwWWGjkYjzYCjgsNFL6rt4OL11OUF37wL QcTl7fbCGv53KfKPdYD5hcbguLKi/aCccJK18ZwNjFhqr4MliQARAQABzUVEYXZpZCBDaHJp c3RvcGhlciBIYW5zZW4gKEludGVsIFdvcmsgQWRkcmVzcykgPGRhdmUuaGFuc2VuQGludGVs LmNvbT7CwXgEEwECACIFAlQ+9J0CGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEGg1 lTBwyZKwLZUP/0dnbhDc229u2u6WtK1s1cSd9WsflGXGagkR6liJ4um3XCfYWDHvIdkHYC1t MNcVHFBwmQkawxsYvgO8kXT3SaFZe4ISfB4K4CL2qp4JO+nJdlFUbZI7cz/Td9z8nHjMcWYF IQuTsWOLs/LBMTs+ANumibtw6UkiGVD3dfHJAOPNApjVr+M0P/lVmTeP8w0uVcd2syiaU5jB aht9CYATn+ytFGWZnBEEQFnqcibIaOrmoBLu2b3fKJEd8Jp7NHDSIdrvrMjYynmc6sZKUqH2 I1qOevaa8jUg7wlLJAWGfIqnu85kkqrVOkbNbk4TPub7VOqA6qG5GCNEIv6ZY7HLYd/vAkVY E8Plzq/NwLAuOWxvGrOl7OPuwVeR4hBDfcrNb990MFPpjGgACzAZyjdmYoMu8j3/MAEW4P0z F5+EYJAOZ+z212y1pchNNauehORXgjrNKsZwxwKpPY9qb84E3O9KYpwfATsqOoQ6tTgr+1BR CCwP712H+E9U5HJ0iibN/CDZFVPL1bRerHziuwuQuvE0qWg0+0SChFe9oq0KAwEkVs6ZDMB2 P16MieEEQ6StQRlvy2YBv80L1TMl3T90Bo1UUn6ARXEpcbFE0/aORH/jEXcRteb+vuik5UGY 5TsyLYdPur3TXm7XDBdmmyQVJjnJKYK9AQxj95KlXLVO38lczsFNBFRjzmoBEACyAxbvUEhd GDGNg0JhDdezyTdN8C9BFsdxyTLnSH31NRiyp1QtuxvcqGZjb2trDVuCbIzRrgMZLVgo3upr MIOx1CXEgmn23Zhh0EpdVHM8IKx9Z7V0r+rrpRWFE8/wQZngKYVi49PGoZj50ZEifEJ5qn/H Nsp2+Y+bTUjDdgWMATg9DiFMyv8fvoqgNsNyrrZTnSgoLzdxr89FGHZCoSoAK8gfgFHuO54B lI8QOfPDG9WDPJ66HCodjTlBEr/Cwq6GruxS5i2Y33YVqxvFvDa1tUtl+iJ2SWKS9kCai2DR 3BwVONJEYSDQaven/EHMlY1q8Vln3lGPsS11vSUK3QcNJjmrgYxH5KsVsf6PNRj9mp8Z1kIG qjRx08+nnyStWC0gZH6NrYyS9rpqH3j+hA2WcI7De51L4Rv9pFwzp161mvtc6eC/GxaiUGuH BNAVP0PY0fqvIC68p3rLIAW3f97uv4ce2RSQ7LbsPsimOeCo/5vgS6YQsj83E+AipPr09Caj 0hloj+hFoqiticNpmsxdWKoOsV0PftcQvBCCYuhKbZV9s5hjt9qn8CE86A5g5KqDf83Fxqm/ vXKgHNFHE5zgXGZnrmaf6resQzbvJHO0Fb0CcIohzrpPaL3YepcLDoCCgElGMGQjdCcSQ+Ci FCRl0Bvyj1YZUql+ZkptgGjikQARAQABwsFfBBgBAgAJBQJUY85qAhsMAAoJEGg1lTBwyZKw l4IQAIKHs/9po4spZDFyfDjunimEhVHqlUt7ggR1Hsl/tkvTSze8pI1P6dGp2XW6AnH1iayn yRcoyT0ZJ+Zmm4xAH1zqKjWplzqdb/dO28qk0bPso8+1oPO8oDhLm1+tY+cOvufXkBTm+whm +AyNTjaCRt6aSMnA/QHVGSJ8grrTJCoACVNhnXg/R0g90g8iV8Q+IBZyDkG0tBThaDdw1B2l asInUTeb9EiVfL/Zjdg5VWiF9LL7iS+9hTeVdR09vThQ/DhVbCNxVk+DtyBHsjOKifrVsYep WpRGBIAu3bK8eXtyvrw1igWTNs2wazJ71+0z2jMzbclKAyRHKU9JdN6Hkkgr2nPb561yjcB8 sIq1pFXKyO+nKy6SZYxOvHxCcjk2fkw6UmPU6/j/nQlj2lfOAgNVKuDLothIxzi8pndB8Jju KktE5HJqUUMXePkAYIxEQ0mMc8Po7tuXdejgPMwgP7x65xtfEqI0RuzbUioFltsp1jUaRwQZ MTsCeQDdjpgHsj+P2ZDeEKCbma4m6Ez/YWs4+zDm1X8uZDkZcfQlD9NldbKDJEXLIjYWo1PH hYepSffIWPyvBMBTW2W5FRjJ4vLRrJSUoEfJuPQ3vW9Y73foyo/qFoURHO48AinGPZ7PC7TF vUaNOTjKedrqHkaOcqB185ahG2had0xnFsDPlx5y In-Reply-To: <29e235f7-add2-47ae-b06e-a717202c4faf@amd.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 11/28/23 06:03, Tom Lendacky wrote: ... >> By my reading, the CC_ATTR_HOST_MEM_ENCRYPT is basically a check for >> whether the current kernel has enabled SME but not SEV while the >> stop_this_cpu() site is driven purely by whether the hardware *supports* >> SME. >> >> The whole supposed reason stop_this_cpu() checks CPUID directly is that >> the current kernel SME/SEV enabling might not match the _next_ kernel's >> enabling choices. > > Correct. > >> So, why is a _current_ kernel check OK for relocate_kernel(), but not OK >> for stop_this_cpu()? > > The relocate_kernel() check provides an indication of whether SME is > actually active. The kexec kernel is placed in unencrypted memory to > match how the system was booted - where the kernel is loaded into > unencrypted memory and then encrypted in-place if SME is desired > (mem_encrypt=on). Since the kexec kernel will be unencrypted, the > cc_platform_has() call is used to indicate whether to perform a wbinvd > to remove encrypted cache line entries. If SME is not active, then there > is no need to flush caches prior to booting the kexec kernel. Ahh, so that wbinvd is truly specific to kexec. It protects the always-unencrypted kexec area from being zapped by encrypted lines. It isn't necessary when the old kexec kernel is mem_encrypt=off because the unencrypted old kernel matches the always unencrypted kexec area. What I was worried about was the _larger_ case. Not the kexec area, the *rest* of memory. But I think that's irrelevant because there's yet *another* wbinvd in __enc_copy() that is will flush the rest of memory when going from mem_encrypt=off=>on. I'd like to propose a simplification. Let's add a CC_ATTR_HOST_MEM_INCOHERENT. That bit gets set on all hardware that needs WBVINDs at kexec. On AMD, it can use the stop_this_cpu() logic. This will cause an additional wbinvd in case where a mem_encrypt=off kernel is kexec'ing. We can also set it on any TDX-enabled Intel hardware. That leads to very simple logic at kexec: Could the old kernel leave incoherent caches around? If so, do WBINVD. That logic gets applied to all CPUs, both boot and secondary. It applies to all the SME-only systems (currently CC_ATTR_HOST_MEM_ENCRYPT) and also all TDX systems. It would not depend on the current kernel's SME enabling and it would allow both kexec-related sites to share the same logic. I don't really like the idea of yet another CC_ATTR_HOST_MEM_INCOHERENT bit, but I do think it's better than adding some TDX-specific paths.