From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758676AbdJQBn3 (ORCPT ); Mon, 16 Oct 2017 21:43:29 -0400 Received: from mail-co1nam03on0064.outbound.protection.outlook.com ([104.47.40.64]:35469 "EHLO NAM03-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1756661AbdJQBnZ (ORCPT ); Mon, 16 Oct 2017 21:43:25 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; Cc: brijesh.singh@amd.com, x86@kernel.org, Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Paolo Bonzini , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Tom Lendacky , linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: Re: [Part1 PATCH v6 16/17] X86/KVM: Decrypt shared per-cpu variables when SEV is active To: Borislav Petkov References: <20171016153423.2980-1-brijesh.singh@amd.com> <20171016153423.2980-17-brijesh.singh@amd.com> <20171016222416.x26tp222x2ox3tkz@pd.tnic> From: Brijesh Singh Message-ID: Date: Mon, 16 Oct 2017 20:43:15 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20171016222416.x26tp222x2ox3tkz@pd.tnic> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: BN6PR17CA0002.namprd17.prod.outlook.com (2603:10b6:404:65::12) To DM2PR12MB0153.namprd12.prod.outlook.com (2a01:111:e400:50ce::16) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 41297cd4-5f71-4293-a222-08d515007323 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075);SRVR:DM2PR12MB0153; X-Microsoft-Exchange-Diagnostics: 1;DM2PR12MB0153;3:KKJAEljb2IMT4K6EVidFBBttZ2K2R4tt4uwQ9vBkz1+QDVRfCS05tXJ8/cFMxoC1l0BVwg89qLAzkYK1pact2wngo8YhFJJLz+tjNcZQHQPsOi3shLVZtOtbP1jLA+F3QQKJ92NOABJm3NvobOGEK3La297HaGY4bs/7LA6hVJ3YUB8JM74XHHcAZ5rjGj/EQsiy5WvJhTqELUpKjPlJfUX1RGBCSFk4Z6ySlUwPHtM+6JN4FAI7D6Ylxp7t3p7Y;25:Bgs9RjSa09pCId1cQN5NHHibSO0vlm9veGRjYy6J8PzogQ2ThHy4Aj8sCuQ+XjZBUypg5FFs+p7GiFCQyq3KG2uYkdLahjlrbqlzquSaTiJg56hNc3+k/LQzbcqVfStgG5WCJddBZ/S2lPR7yx3x3OUg979TOwB2dZIlTGVI2oN4sCMgEPjqHoQeXxy4p48rZo04dU34PgvW7UJSc6+q2JvIfVT2TVL75GjDvYgIcZ7+TwySMR6OrtFRuhaticpWOD+mlT2OsRrqjMauSVhXmvVTb2FwFKHDyf1B6R2/PqKH7v1mAYGplBY8lTcbBWar0+TE75icXk/EXSQao/Ci4Q==;31:yp7wBMXoI+bqfdVY9vBW13RotOxnJ3MUCVl6AFQKIAFLb4RMumcm3HtLD8r1DA/vDJZpgf76UWTUtS4C+wFz4X14QjQQSW/LN0h88gjljx1B3btgzrFpH3GXqM1bvIZ+6HxI4VLS1LM4LpFce3ef2tDCPxVsutmh9EyK8a9okXqZBcx1Et9BUPBzM6AT+dood9AQ75oN5e4H8DHJAWWFG/3o9FTydsoqI6YnvwsRAYw= X-MS-TrafficTypeDiagnostic: DM2PR12MB0153: X-Microsoft-Exchange-Diagnostics: 1;DM2PR12MB0153;20:CRuuB0p0Y9hc0l4ln7ujgdw7vBlUzMFhtboJSOShd8yb0yHPo8366KX6e8jf04siUDL+5uq6iPlilerMZdW+YYqAyET+y3c8uPofrdQL/WJubF9ra/Z+oQJWEubUyeZ/fLeH+Y4tearQYXepFhdl9wOaX4SWW7Rzhbab7SB4OmGaGPLdLGJU80QNCF9fZuCOXsMi8C9XgZctZCO44JeYrbYXvXcQ0QROwa1AEai7COSZ4iVWm+6wyiyOnVjtpoYcmKA04+RkzNgVqftLM6hL2MjqiGlhi89nlPLKEMFItSfuYGV9FAqIHH5kfiBluDoajBmqRMdDTLWuJtuKupF8plAOwvmB8rxVrgZg1IQ5GSqDa5r7BHOKNBVCK6siUpy/RVzApLgANcHiq/D22ylCsvQYbisa3Dg6r/6USeIOTEi3WSth/7nRO1IgVaqaVYhrWWQBEWqOUV0u6tLFnnDkpx5XlyGdZE7sa8dmwU/BKw7khSMGqBC1Nww6zAsYOwgA;4:662tICtr13xbkeLzIpQb+L7Pc/6XmFygmw2sliiUV/BiVDI50UeaEgaVA1MSFvqhq/894xPlnNSE9Bxcb4G7vpa7Xto5ZQ6fVy2gpF81YFeLcRHPaV1IDzgQZeKtk3D1GTyRud5SOz0N8XnUZn3cmeJ/wA6lRuJ4rdPwptwlPw+RDEBfdVVPXVg6eASz7OvnO5P1EBiB+QFvZH/udfp6xSqklFYQ9D1UwfQLJz3M4kr1RtH3FjEnwFNuQ2gLqQC+ X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(93006095)(93001095)(10201501046)(3002001)(6055026)(6041248)(20161123564025)(20161123558100)(20161123560025)(20161123555025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:DM2PR12MB0153;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:DM2PR12MB0153; X-Forefront-PRVS: 04631F8F77 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6009001)(39860400002)(346002)(376002)(24454002)(189002)(199003)(377454003)(64126003)(6486002)(31696002)(86362001)(23676002)(50986999)(76176999)(54356999)(53546010)(305945005)(101416001)(7736002)(50466002)(31686004)(33646002)(81156014)(8936002)(81166006)(8676002)(53416004)(106356001)(105586002)(65806001)(6666003)(65956001)(66066001)(47776003)(230700001)(478600001)(3846002)(36756003)(83506001)(6916009)(97736004)(53936002)(6116002)(25786009)(2950100002)(229853002)(65826007)(189998001)(54906003)(316002)(16526018)(68736007)(58126008)(4326008)(5660300001)(2906002)(6246003);DIR:OUT;SFP:1101;SCL:1;SRVR:DM2PR12MB0153;H:wsp094296wss.amd.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtETTJQUjEyTUIwMTUzOzIzOnFNaElWQW1zSHFDbG5HdXhqb0FobDhleWdm?= =?utf-8?B?R0JTTFRoSlh0YW1vOVJOV2kralJvVXBneHQ0cFZVWXI3cjhwQXBEN2pIbGdi?= =?utf-8?B?QlhNWC9tRnlIQm13YXpBMWIzMG02Z1lTdXBUQzJDYzRKUFM4dHU1ZUdmVWdr?= =?utf-8?B?bVI2ZFBzaG02TzhjRjNSVUFpUUJNVytnZDhBeGk0dmlYRlhlV0tZYXk4bWZh?= =?utf-8?B?cVFPUjhYdlJ2Z3UrcEZkbDcrVlVOZ0x4SlAwY0U0VXNiVU9GdG93cndLODJU?= =?utf-8?B?azJlMkgvaTl1MHJ4R3lIdHVCUm56NklHN1BsUW10RGhOd290dlNXRlEydFNX?= =?utf-8?B?RWNYSkF4N0R4R0NRTmpNeHNoN2xoUXZRNUhZQXk2NjRBcnB4VjVZZzNuNnF2?= =?utf-8?B?ay9uQlE1S1J1RGtoM2d1T1ZRbXJtVThtM29vSlNSZHo0ZnJXSE9TWldXYit0?= =?utf-8?B?WlVXeEpUWER2UFBsNStzbVFOUVpyb3BMVys5bGRJL3JKVldLUUR5N1FZalVv?= =?utf-8?B?ZDN1R05pNDhFZTlRZDNHSXQzK1NpTUhXUVNrYW1QZGJIbVZQTXoxd2JKa3Rn?= =?utf-8?B?K3RmV3U2Ti93MkttZDVwei9HWS9kd2o5Tnl5aFdobnB6Q2pxQXNZN1FPWUtF?= =?utf-8?B?anAzRHh1aVBQWkhndEdsUXdVZkhET1VQUmRQRXF1SkVoWFd6blJyVmlEbmdh?= =?utf-8?B?eFhCbjAzWHFrYXBUellRcDNYKzU2RUV5KzdXc0Q2bWNrWUhxRGk5M1RzZHo0?= =?utf-8?B?NXlZRC94ZFU0TUR5Y0VTdnhQU0lnYkl5TlBKM3NFOER6Sk95NlhYUFhPby9x?= =?utf-8?B?NWRta3Q3ZXYwcXdPVm5VNmVwWW1BbHRLWWNXclR0K0lydXVZc2ovdmgzV1lU?= =?utf-8?B?TzNBOTFhQTBrakFqOGYya1F4SVdNSUQwc1c0SUNFSXlUNHI5UDRjK3prT0Vq?= =?utf-8?B?UGR1L1FzSlBDYUdOZnQyWDJ4TzQ4WC9HTGhwNUQ4dmNscVJlSGROL010TkZP?= =?utf-8?B?UzI4c2VCZkZPaGYrTnFsL1JiWklIOXVhMlRhem5VODZNNDhMSlZMdUxaRTNT?= =?utf-8?B?TXU0NHdJWk9NL0lDWE4zM3NuenRPRmp5WXNOeEpOOWZVZjJEbGcyQTdQK1Mz?= =?utf-8?B?ZnpDTkpVSUYxNmYzZ3RDQXhhZEU3bHNSRFdqaXhvdENxalBicTU3cDV5dXpL?= =?utf-8?B?M1lZRGdzYTFDY1JDNUM4c2ZYeTlDVlVXTTlpNURZc2hvUm1qMGMrcTMzbEpm?= =?utf-8?B?enNhZzNSM1BTNjA0OWNnT0wrc0p6bFYvdUkzQU92NndJZzl5ckNZa2xlMko1?= =?utf-8?B?ZG96d2lSZk9HQzl4ZVZDZllINkUwRThIK1MwMHJYMEg4NlNlU1JJUElGamtR?= =?utf-8?B?NVBORE9SY2Frd1JjOEcxUzR2eTVhbVJJTysrWmwzckRvYmpoWXR2dW1UNlZx?= =?utf-8?B?U1J1RkQ1OEJCcVhxVGVYemllRExKeEdWTXQrV1JGbCs3OVdkWmxzY0VoeldZ?= =?utf-8?B?SWhwZmg3SXdJeW1Od1dlVjFvNnZkY3AwbVZPWlJvV21UT3BVUlJMZUp2QThU?= =?utf-8?B?ZERFMk11a2ljbDRvQ0gvOXgrUG0zZEZqSFpqdUZBMnNHYVpaVHVBQlpmS0RV?= =?utf-8?B?OVJkOVRieVl6SkNvQjllM2g0Z0E4RXY1QmJ3dy9WSWJaNjJvK1NXb0VUUnlM?= =?utf-8?B?a2RRWGcrTzJ5MjF3L2NlVVZoS3BXdndlbE9GSW1BUFFwQUNFOWVQeTBmK20x?= =?utf-8?B?anQwUkhSUzhldmxjV3FETHhiS0o5V0hKWTZ3cVNDWEM5SWltazdSOENiNk82?= =?utf-8?B?aWsxdE11ZXN3Wm1lMWRRMW9xSDcwa2RUaHg1SzM0UFBaK1E9PQ==?= X-Microsoft-Exchange-Diagnostics: 1;DM2PR12MB0153;6:jLwZNknbVNkmJcXzUl7D/+tciY3FjEoIKU9OcZb9CqWuBYlQwoGmxG9wC7SDq8QjdeMK4gKvGwmop06bgkVr2epB8FIjo6FHBrMZmSd8PcEsd6AFOgHHYifAx9/kT18B/0hXVXhKR6/23VqHZqkYRhmLBJjHgB0kOUkwq2DIoj99RnWCJ1yd0Ac+bZBFjzaIhbODBE3GggbCfBrcG+/4RrpCU1Yii0b3gdLVmpzrSUo10/tMYf8gjuJqGFc6guDGCSXOe+lMOykC3MOIkYIpdgDdqZU5ZWfAFxf1zRl8kTTcUziAf8PByuzT7HkNnzX+2aY5+MtGNCcwXhyIZr8fMg==;5:zHIJYKCHe5YmnYEohEuKbkIx2O1PpHQh+8UYTs7gsSPINVmg6y+8ynFa7DEr8p7mpY7vx4p6TqJsamab942MAaf0O2vqiPq5mA5uLmF4tY4vZDS3P6NAZtTH5S/RGt9YjUKXOtHBANyU4f2S0GcX6A==;24:CGyTTRLkmQJz7tNvlceKRD7vpMcTs7X2Tv1LpSxoEoyq+QYixTUaM8MXlozX9vmuNwES5Ht1SCPFcdVaEObl4M3Hrx3ZJ4EeNOlP+Aa/8vY=;7:PhQtZGTBzXkx7Weg9ClsoWbIPdd3tVJN1bHWflcBTI3o821sk1a2QFkhFaFPJzIpuwyXxERrYoSogbLUgea1WD0D7hXLF48yzYgUddb9tI5v/vAuvZG5BoIc3h+JidTpcslZlQRca57hRyL7nAFmaUL186/tGDgb/S38Hv2WKXH//EHC9Vr+p/7kCtRSw4EHu5gulPFH68TEmCRaNLOWSKDf4xgm+yRzj0k4FtJy8o4= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;DM2PR12MB0153;20:4cgKtbiOo4PjIFW40joD+rQx9ockxvtZcK/yh/ufEptpA+yuVzMbPvW3rHUtH96vbxR8EwSQIVBfOY+F4RYZgVvGC4QiTd0p7dMcFtZuVnqpSiK4AeRR32OicTZp/TqjlWVgEIBIcX/5YH/SnpfaCR6VUPPtSbdxKb90o564PkvxnT09791ZhlUFHRVOZH7ZjYKikKz9UzsCATrpvbQ2fs/7jxLd2H1kLB9jx1LnGKOxSyCGf2705O+lMPi5xdZZ X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2017 01:43:20.0566 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR12MB0153 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/16/17 5:24 PM, Borislav Petkov wrote: ... >> >> +static inline void __set_percpu_decrypted(void *ptr, unsigned long size) >> +{ >> + early_set_memory_decrypted(slow_virt_to_phys(ptr), size); >> +} > Ok, so this looks like useless conversion: > > you pass in a virtual address, it gets converted to a physical address > with slow_virt_to_phys() and then in early_set_memory_enc_dec() gets > converted to a virtual address again. > > Why? Why not pass the virtual address directly? Actually, I worked to enable the kvmclock support before the kvm-stealtime, eoi and apf_reason. The kvmclock uses memblock_alloc() to allocate the shared memory and since the memblock_alloc() returns the physical address hence I used the same input type as a argument to the early_set_memory_decrypted(). If you want me to change the input to accept the virtual address then I have no issue doing so. But the changes need to propagated to kvmclock (i.e PATCH 17/17) to use __va(). Please let me know if you want me to pass the virtual address. >> +/* >> + * Iterate through all possible CPUs and map the memory region pointed >> + * by apf_reason, steal_time and kvm_apic_eoi as decrypted at once. >> + * >> + * Note: we iterate through all possible CPUs to ensure that CPUs >> + * hotplugged will have their per-cpu variable already mapped as >> + * decrypted. >> + */ >> +static void __init sev_map_percpu_data(void) >> +{ >> + int cpu; >> + >> + if (!sev_active()) >> + return; >> + >> + for_each_possible_cpu(cpu) { >> + __set_percpu_decrypted(&per_cpu(apf_reason, cpu), sizeof(apf_reason)); >> + __set_percpu_decrypted(&per_cpu(steal_time, cpu), sizeof(steal_time)); >> + __set_percpu_decrypted(&per_cpu(kvm_apic_eoi, cpu), sizeof(kvm_apic_eoi)); >> + } >> +} >> + >> #ifdef CONFIG_SMP >> static void __init kvm_smp_prepare_boot_cpu(void) >> { >> + sev_map_percpu_data(); >> kvm_guest_cpu_init(); >> native_smp_prepare_boot_cpu(); >> kvm_spinlock_init(); >> @@ -496,6 +524,7 @@ void __init kvm_guest_init(void) >> kvm_cpu_online, kvm_cpu_down_prepare) < 0) >> pr_err("kvm_guest: Failed to install cpu hotplug callbacks\n"); >> #else >> + sev_map_percpu_data(); >> kvm_guest_cpu_init(); >> #endif > Why isn't it enough to call > > sev_map_percpu_data() > > at the end of kvm_guest_init() only but you have to call it in > kvm_smp_prepare_boot_cpu() too? I mean, once you map those things > decrypted, there's no need to do them again... > IIRC, we tried clearing C bit in kvm_guest_init() but since the kvm_guest_init() is called before setup_per_cpu_areas() hence per_cpu_ptr(var, cpu_id) was not able to get another processors copy of the variable.