From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753207AbdJTOdZ (ORCPT ); Fri, 20 Oct 2017 10:33:25 -0400 Received: from mail-dm3nam03on0088.outbound.protection.outlook.com ([104.47.41.88]:47104 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752465AbdJTObo (ORCPT ); Fri, 20 Oct 2017 10:31:44 -0400 From: Brijesh Singh To: x86@kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Borislav Petkov , Tom Lendacky , Brijesh Singh , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Borislav Petkov , Andy Shevchenko , David Laight , Arnd Bergmann Subject: [Part1 PATCH v7 13/17] x86/io: Unroll string I/O when SEV is active Date: Fri, 20 Oct 2017 09:30:55 -0500 Message-Id: <20171020143059.3291-14-brijesh.singh@amd.com> X-Mailer: git-send-email 2.9.5 In-Reply-To: <20171020143059.3291-1-brijesh.singh@amd.com> References: <20171020143059.3291-1-brijesh.singh@amd.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [165.204.78.1] X-ClientProxiedBy: BN6PR14CA0029.namprd14.prod.outlook.com (10.171.172.143) To SN1PR12MB0160.namprd12.prod.outlook.com (10.162.3.147) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 10ed9531-8880-44b7-7deb-08d517c741dd X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(48565401081)(4534020)(4602075)(4627075)(201703031133081)(201702281549075)(2017052603229);SRVR:SN1PR12MB0160; X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0160;3:Ux+BZvwPOv/1PNdPeGDwRC8fECZAMc3qDkHdGchivnj8WEWCRsHugigq84Jxlsd8y3k3/Uwg4mum7pUfCFCVZhpcWjXKLpDI+QuhKNPWfMAWRQsZIV/RSNbwNsDy9yFEOmT9M49/E9xbTH/DvwK+SbnfCv4HutExUXqpJEk4zW/P0yr+yUXG/ixi80t9Q2AE0y/Tv1I9WgKq3jz6tfQgK8Q0nao80zJGf8oQ7WC8D2zKc5X7R0lpdNj3+jQu0YHF;25:DEwgPRq+1rj4H+ZTIpElTvaOb8SSIg162reWhiiodjCR5RCNxIKQ++pRFWsEX+O79hdoXETn14Bo6XXrnLfzoQF6Ow3sYGrh+TBnEwlEa4A4OyKLDjSmm9sAOgJ5cHSoIRPEYIdMFyUV5xI4OLTPNTcnASNCSRwOU41kGPktroarjtgdpP8dI171eTktI6GwhHUrPKbl0F4PWnxODFgOZhiTBweD3yQDM2glrJD99H0LE2tQcCimLaJVLesMmXUdpXwp0+1B/YRvVVqJtvITGaWrgx7pQgr2Cw1hge8poWqSCVvC41zLFDtbtonkISzP4UYVl+KIXBePux9KtVxagg==;31:y5mc854wSZp8Av2fSDey0vG12GQYXhrAUlSvatdS5LHC2N3g/Pq0sYmDsEtkedPMjeyQ/p/edDWBfxtP40QUQr8vls/WYd4N5gRkFcLbgG6pz+jttDOhM2fHNCjo9SPklC2Vuzei8JLXVm1ZFhmF11BhpRwJkkCnWVJCOXQiVLEpvQJiTw7xfJ4s9e4nDzki3pEF3CoUHfLVYLNvs4wgMudyTRo7rn+zNVMS5/Y2lqI= X-MS-TrafficTypeDiagnostic: SN1PR12MB0160: X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0160;20:X19/9UyrcQpKlW0wd1HR01BcRWrvUCQhtEFlAD9bb2ALv/RT8OT44AQoTLNZB4ysoVxr2NoXxTQnohIjU9E7DHg9gEr+KX+nhOPiV+GTH8XyyrzYCFFLQZFryh6l6IGtmQgCUd2Msdv3eNIysr7pkfMZHpOZ3jV5ETcGIOh4bI2iCtU+Gw96ZSXto2H7H/MNQN6UYkgw7dGS5SRP0zXaUv+n5iWwvr99X7epDrxuBHqkTjh/R1PBzPtWw9h2tykeBfVzHiAAzP/12jtlbPMUgsFek8i4LTt5hxkjT69WoSUSpX7YU0tKYlv3Qbdv1pVI7CrTtyb0BnwqSvrWMzhuaNzU1oacC/v+kcgOOEZMMjTtiZHKPyNwf3mdo1TdxNFRkNGoyqycE54nPYeZR0F4LhXom3MvDKumoVu/iuBCS2cPczoMfiTjBw+KqDmK/W4ccZ/M0OvMDULFH2EwyEMhOwplixOPLKfR8VW2LSc1VdJsNCWhrgllWaYU/AP39w6n;4:Yi/4zF/WPZujalBi/NhCGnXIvvfYtiDAl9SWuDAY1IK+o5Wgn4zXh/yruskBSEJMVwh8FjlIHQ2xeLbajSbSJTwX+blSX52U3GXKJrkHy/MvCM77Dsb1hknuJ2BtVYNg71GQgGhDK2+T1qmONE3ka2Q2fG8ur33m8YmCzVJ1PkbjNOqq/oh25X+TM9BwkoUigY2AtP628k8DRb8eT/Mb8rKQdZrw/eXh8CsdorVy6FFNDYwHYT4BFNKdee4qovC6raDRU85Gx6CmxYFUB5JdXjjVtoUViKqYxQvOTpUnQqaHXl6RvroCUedk1c5zpEK6tgwSaLEiS/QtRMu4kD56eUNCriAN4mHK/8nu11Dx8kI= X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110)(228905959029699); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(3002001)(3231020)(93006095)(93001095)(10201501046)(100000703101)(100105400095)(6055026)(6041248)(20161123562025)(20161123564025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:SN1PR12MB0160;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:SN1PR12MB0160; X-Forefront-PRVS: 0466CA5A45 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6009001)(39860400002)(376002)(346002)(199003)(189002)(54906003)(1076002)(6666003)(189998001)(68736007)(8676002)(305945005)(50226002)(7736002)(101416001)(76176999)(8936002)(97736004)(25786009)(81166006)(81156014)(16526018)(6486002)(478600001)(48376002)(2906002)(50466002)(4326008)(7416002)(47776003)(66066001)(5660300001)(2950100002)(53936002)(106356001)(3846002)(50986999)(16586007)(53416004)(6116002)(5003940100001)(36756003)(33646002)(86362001)(105586002)(316002);DIR:OUT;SFP:1101;SCL:1;SRVR:SN1PR12MB0160;H:ubuntu-010236106000.amd.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;SN1PR12MB0160;23:ZVYmaz8Zw1TvXgUHAdkK5e6sy2EfCr3BgC4RmCFWp?= =?us-ascii?Q?LxBsGDHUSTHAml8/6KOmwDh70yrQpyBhQYN1mkJm2NMke/utbnD7xCOnMJ+b?= =?us-ascii?Q?wDe0VMMM2G/GQpS2Gv+mqvoAcQSiARH6JogSwTMkbZjmrdoZb51nbCyyOgAg?= =?us-ascii?Q?rMwovHzhv7G49X3mYG5yCPrhWQWfGQ2I6tvkj85r8AIydcSDa3SlGj8D6+ZF?= =?us-ascii?Q?KtNPHzKxlRa9NSJ16PYh8p8w1kjdM5QUYv4Ek1PWVJa01MYm7Itv51gw0ny5?= =?us-ascii?Q?dhJWhRxhylb3cuLe9aqnJxNvgKMo1qDsYtcF+qkyR+MjBgf8v6uC7BYmrWV3?= =?us-ascii?Q?Z5vfypre0vudKf+kRD6QUFaHahbhriR2pgFZM7fJ/++PrmOktyKVz3ySc58P?= =?us-ascii?Q?lhkTcNcVoTp+yHPJlRz1WdUofjFvM/jN+lwNmzyImk8xF6FpIwTCgjR0cAAn?= =?us-ascii?Q?ls+bOgDngQaP321uvOqRrlE6XWCSSmYLEbOCgSfhqLF3YEPYP84pw60ZoE6m?= =?us-ascii?Q?H+dNPeY7Ab1nOWsJ3z9QkMOMjb0q1lpLiUBBVCa77vX9/bpkZB+t0yC6cT59?= =?us-ascii?Q?KMSE+Lol5LsJS4NuWHuN1bUY1TbqO3MaIMeTPM9xmQ+JfL6YQ+xoJqzh/NBp?= =?us-ascii?Q?9RoJmWh0++kmJVJSJ58PfSXIpLHOwlwKDWFUV20lwYYw6TqYZLyLMxIhsu1y?= =?us-ascii?Q?Qryf0JWz5yzf1TykFIPjTmFjvPjhqZWetJcqzKo6fDtO2rPiygInPaPgOeM8?= =?us-ascii?Q?Ol7nWQzY5wN7zx31q1a11fCWfMEGOS8mzOaIh1/rPe5p7fUAOaXSZbbGFgC4?= =?us-ascii?Q?pXgVXDWgjs/M7Yd+RbQwTB6FoFkizT1V9pSm6Bx3EgBo9pUngC+4z9do/R21?= =?us-ascii?Q?dmOtnLbZd2huyKiozeMHM6FGPvAg4tl9RyEjE8DCE7Aw5SDq08pDbnPqqpWG?= =?us-ascii?Q?zBzILE3d1AoW55GW7RWbwJ5uq57K6LAosQj8hkT61/Yc4gsov2JUOIWFbqSo?= =?us-ascii?Q?oN/J1aePnG5lKeFZx5ZxuDbqdWLb2pZ/LCrTpVX9+Jn0KnSsTETTcpMr7mYO?= =?us-ascii?Q?QntrOILOEW8943xy/C0LsRmBelaYwPhZpbxl264pWbbos1nTi8mO5VMfa0qH?= =?us-ascii?Q?yzVKrfRH44=3D?= X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0160;6:8eEvd4dpAPdi+TkLurOz8nNzCL2Uot3cBkJgeaXufBDFXP8o5tnebbXlFhwR+ENcnQToK/AtucKqftJhm5p2Jgn7KUGyRx2+o55vZ7NJ3ahDR22wJPlnt2Q6Zav9CGzKB1Ip/Oq9gKmBBeHHSaGMv2qyq54MsOHrL/Z02zJu8wP3//dWNgjCyO2PAWbfomN+qyO15RnYFS8pFyS33ohacvlz+1wWCfpv7FMAxRDIm2ZSnGTY8tMYEfViEDaj5hBybAdL6vbeAOBY41lfQQFIFnidacHy8RQJ/slipxXVFvvGYTBfx7HUd1u3qYlAotsoDmX0Nqp3VnRpQgHFAc+nGQ==;5:hy6bplGuKfRJd+DkSFwdKY8QIYMi5hw5W9d96tz+nJ2Yf4RJkdwe42mPK4nnxDqVg8zkGYbg6X9O3s53IYQWooH40ypvBTuBCQvY6tR4UNqJwadyB298AaBiyuhKKIDU7WOiw0vlrjf6SAZd97YmQg==;24:MxVSGYAFir4zqOmLgO/lGQb7elvx1qb6pYvUQm8Fxrn+6DGbiiKTiJHo/v7FgxgGOmZCUzNFBFoW6qHaDgZzAA5gOdr/R9FasS1Gj50aHd4=;7:bbPlvEfWQdOWPoFXvqvTZ/tseuYgOHlOL3on/Z2CGUxBi0Oke+MmPzT5AlDL7J3h34LxGzJ4x+9f2Y74ZmusoHJuhUgrID1MMBgTRU96Sb4FENC3+ejSuqKwmegHVBasQXp4huzNmBzt4Yh8h04AHKCGv60jg5ytjgoodFEUPi3xKrMUy+szo5yIi5XYE6MzCf3keJwGuhO6VhpRhm3x2oXyxQ/bICKCFJNDYwrgXTw= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;SN1PR12MB0160;20:n439NAv+PSjtyztcEznUxnNc/4D4vktO/LX1RGL4bVZxsH2n5tsj8Kof3r1mljbpMPpR75qvjO2YddXPRwkPRKz+5jFkhm8sV03jUyB347AbCo+NizbQC/2FMJm8+V2OPK0Se6PmfQ02TmmjoEfkRrdIYe9exrn+d/kcMoDLU4jPEem9zVt2hM+I7DILpwg5onjZyf3Enj9D42eg+HSvtUKYBxxzV4QcGKFByii04QP01rYGiuGkq8Ji5ZwRfRhi X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Oct 2017 14:31:30.2665 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 10ed9531-8880-44b7-7deb-08d517c741dd X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB0160 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Tom Lendacky Secure Encrypted Virtualization (SEV) does not support string I/O, so unroll the string I/O operation into a loop operating on one element at a time. Signed-off-by: Tom Lendacky Signed-off-by: Brijesh Singh Reviewed-by: Borislav Petkov Tested-by: Borislav Petkov Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Borislav Petkov Cc: Andy Shevchenko Cc: David Laight Cc: Arnd Bergmann Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org --- arch/x86/include/asm/io.h | 43 +++++++++++++++++++++++++++++++++++++++---- arch/x86/mm/mem_encrypt.c | 8 ++++++++ 2 files changed, 47 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h index c40a95c33bb8..b8b8c2e3fb36 100644 --- a/arch/x86/include/asm/io.h +++ b/arch/x86/include/asm/io.h @@ -265,6 +265,21 @@ static inline void slow_down_io(void) #endif +#ifdef CONFIG_AMD_MEM_ENCRYPT +#include + +extern struct static_key_false __sev; +static inline bool sev_key_active(void) +{ + return static_branch_unlikely(&__sev); +} + +#else /* !CONFIG_AMD_MEM_ENCRYPT */ + +static inline bool sev_key_active(void) { return false; } + +#endif /* CONFIG_AMD_MEM_ENCRYPT */ + #define BUILDIO(bwl, bw, type) \ static inline void out##bwl(unsigned type value, int port) \ { \ @@ -295,14 +310,34 @@ static inline unsigned type in##bwl##_p(int port) \ \ static inline void outs##bwl(int port, const void *addr, unsigned long count) \ { \ - asm volatile("rep; outs" #bwl \ - : "+S"(addr), "+c"(count) : "d"(port) : "memory"); \ + if (sev_key_active()) { \ + unsigned type *value = (unsigned type *)addr; \ + while (count) { \ + out##bwl(*value, port); \ + value++; \ + count--; \ + } \ + } else { \ + asm volatile("rep; outs" #bwl \ + : "+S"(addr), "+c"(count) \ + : "d"(port) : "memory"); \ + } \ } \ \ static inline void ins##bwl(int port, void *addr, unsigned long count) \ { \ - asm volatile("rep; ins" #bwl \ - : "+D"(addr), "+c"(count) : "d"(port) : "memory"); \ + if (sev_key_active()) { \ + unsigned type *value = (unsigned type *)addr; \ + while (count) { \ + *value = in##bwl(port); \ + value++; \ + count--; \ + } \ + } else { \ + asm volatile("rep; ins" #bwl \ + : "+D"(addr), "+c"(count) \ + : "d"(port) : "memory"); \ + } \ } BUILDIO(b, b, char) diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c index fd9c7bb51664..45110b60be21 100644 --- a/arch/x86/mm/mem_encrypt.c +++ b/arch/x86/mm/mem_encrypt.c @@ -41,6 +41,8 @@ static char sme_cmdline_off[] __initdata = "off"; */ u64 sme_me_mask __section(.data) = 0; EXPORT_SYMBOL_GPL(sme_me_mask); +DEFINE_STATIC_KEY_FALSE(__sev); +EXPORT_SYMBOL_GPL(__sev); static bool sev_enabled __section(.data); @@ -313,6 +315,12 @@ void __init mem_encrypt_init(void) if (sev_active()) dma_ops = &sev_dma_ops; + /* + * With SEV, we need to unroll the rep string I/O instructions. + */ + if (sev_active()) + static_branch_enable(&__sev); + pr_info("AMD %s active\n", sev_active() ? "Secure Encrypted Virtualization (SEV)" : "Secure Memory Encryption (SME)"); -- 2.9.5