From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D752C433DB for ; Thu, 4 Feb 2021 00:44:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 10AF464DF0 for ; Thu, 4 Feb 2021 00:44:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234531AbhBDAo3 (ORCPT ); Wed, 3 Feb 2021 19:44:29 -0500 Received: from mail-bn7nam10on2057.outbound.protection.outlook.com ([40.107.92.57]:26369 "EHLO NAM10-BN7-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S234397AbhBDAnn (ORCPT ); Wed, 3 Feb 2021 19:43:43 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=j+Yze3vkaza5NHiZAZ0SZh6f+5Fq2vfcVsCXEjQwO7/qcaIjKfeXORXNY612IrliLB93HQShJkOYEZ54GBmx2SB7Q2j1jiimYJ+JCqFt4TUD1T1LFTTnwZmU6oXH5h1Pz+PtkaT0m44Zze5jpLTz9Sfy4rcdUOweOA7BAuNCi86oq/Fk22fc9OBuhKXYiQIxi8kmjSVqVSpHW+dOQOhaTESP4Mop9lsLKXeZ8I2TyszY6YvVcjzBnOi/YBrhZuebwE2xHJtt55Q1JCvqB3R9LHFZWhgmJ0be3d0oNJzH9nokMmm5MPPyyrDVGQgzwCnoOhYVvMii2fcQhz9dESu/eA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VYTRVC+NdwDsMrIAb8kXUlNawYxLjr7S0rWuSm3vygY=; b=KRaVQJxhJI/DCzWjLneIZlHZEJo1lHjH0oqzZXAoja9yEAmfYHoDdNMgcjsCkbwcgKBF/FYusJOplaYiR1cyrztjxeWx67tnqaaad7OgoV6+MLASA4qMFPJLDM43JHjBgX/sEXQkcnPCIZaX+30ABk7UlHAsJ6m9zAsT9aQU1J0rzdGzRCbcPcRTbQqD4+BZNEUbycfn80MzFfpw0MiMOANa8e32SykmVsQFTeldE6ziu2ZT6ZcrCmdSM2MI1iNgepiui6vBy1DoVVSGKZ1yoPi1OxmVux+WBGQLXAv5bBKbCbkOdOLqFArpH8K48k0suXxeEHcgY0QYDnSK77bR4Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VYTRVC+NdwDsMrIAb8kXUlNawYxLjr7S0rWuSm3vygY=; b=CgU2HnesCJa2vMCzuFhdcnhEdTsZ9Pu1euXxEG2V1GAxWdmcp28UyXuvNxJL65pLLYj9E4n0Tae5FtxXencFFfGNepEklOcVjiLKNfX4UNbFM+l+SRm467iLZWk8HwaIH4+X2jYUwWHHRcx5qKxE8exFOC2LD69AvDgyVW+kxaw= Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB2767.namprd12.prod.outlook.com (2603:10b6:805:75::23) by SA0PR12MB4384.namprd12.prod.outlook.com (2603:10b6:806:9f::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3805.16; Thu, 4 Feb 2021 00:39:09 +0000 Received: from SN6PR12MB2767.namprd12.prod.outlook.com ([fe80::24bb:3e53:c95e:cb8e]) by SN6PR12MB2767.namprd12.prod.outlook.com ([fe80::24bb:3e53:c95e:cb8e%7]) with mapi id 15.20.3805.028; Thu, 4 Feb 2021 00:39:09 +0000 From: Ashish Kalra To: pbonzini@redhat.com Cc: tglx@linutronix.de, mingo@redhat.com, hpa@zytor.com, rkrcmar@redhat.com, joro@8bytes.org, bp@suse.de, thomas.lendacky@amd.com, x86@kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, srutherford@google.com, seanjc@google.com, venu.busireddy@oracle.com, brijesh.singh@amd.com Subject: [PATCH v10 09/16] mm: x86: Invoke hypercall when page encryption status is changed Date: Thu, 4 Feb 2021 00:39:00 +0000 Message-Id: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: SA0PR11CA0145.namprd11.prod.outlook.com (2603:10b6:806:131::30) To SN6PR12MB2767.namprd12.prod.outlook.com (2603:10b6:805:75::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ashkalra_ubuntu_server.amd.com (165.204.77.1) by SA0PR11CA0145.namprd11.prod.outlook.com (2603:10b6:806:131::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.20 via Frontend Transport; Thu, 4 Feb 2021 00:39:08 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 394b8b67-f6e0-4e32-95fd-08d8c8a548e8 X-MS-TrafficTypeDiagnostic: SA0PR12MB4384: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zwtIG6ASS+v0yGGfr5Oh7SiZDGJCoyWrMsK1is4qhnVS5GtQpoBopXcOxAggv+cwhPHaWFbbz1q1r2sY5ysIB3413frydXQKccO3RvX7nmrQfRp/TSxzAxRE8JGlfZsyDutWaZmsvKnVy9jjG5vAo3lmNvMhoD8MR4JV8l0+XDeXiCijUkWlUiX6YYICdh1At2SkPqMuONCuyMzWeBf0Kv3D5T1gTw7xfm+58VWJcdv4eZ01VELslg5Ck/RA5hCKs8x+deNcBn78IYIk3qGJY9psvb1AvaNWrxMOs0SfFW4LkzdnKf7hjknPwPAK62OymVUHAEA7fHMYfkDQKpR1vdEyn0Nnd0WgB/JAacKssneReKTE523JD8jELeWzuzKHBAUEK7DLYFPmcgPcSlHS93g6kbFCNti0r+NdapDOxtVmMSGmyiKTypqO8xj6UWxmRBru/fXCXsYY5gHoyzNBAybVNeVmI62D7t/o21mzvm5E5JroGhUL72aU58+BimNlBSjOgERGzhNTi0Ktgodwew== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR12MB2767.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(39850400004)(366004)(346002)(376002)(396003)(8936002)(4326008)(66556008)(86362001)(6666004)(66476007)(2906002)(186003)(478600001)(26005)(6916009)(6486002)(5660300002)(52116002)(16526019)(36756003)(7696005)(7416002)(66946007)(83380400001)(8676002)(956004)(2616005)(66574015)(316002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?ZFk4eDhUY2dvL3BMVE1KdVNpeEVsSHFnQjBEY3MyaEZPY0p2WUk5ZXNIV1V4?= =?utf-8?B?ZGlHL0pxRHJMd1oxR3BtL3IwSlk4RFVOT0dRKzVsdVZOTmVXS0tNdVhlVzcy?= =?utf-8?B?alowblpJS290VlF5RUpkNGZMM3FBMVVaVjJIeFYrekxZa21VYXkyeldOSUI4?= =?utf-8?B?UUo2bEk1VnIrOC83STJzK2MybnppQS9XY2tEUFZSS09KYXpZT3MyZ2xMcndt?= =?utf-8?B?blVBNWhSRTU5aXBsUmRvN0E4ekd6OUlNZ2ZGcmZnVTJ6bnVHN09KY3p3aU1O?= =?utf-8?B?aVZkd1g4dVN4UjcrT0tEUHJoZkJkaFpwTURwVUN6czhCTzdRSmxOcStWeko4?= =?utf-8?B?UVZFdTRQQzdXYVVjSjhEZU9ZMDVtejZBc05pa1F6OTR6K1JTMC9mN2xTM0Ix?= =?utf-8?B?M2dYUkp3SGxvWVFxTnhPUnlSRjJnODlIaXlFdVQxOEF3RklMcHBNQisyM2hO?= =?utf-8?B?SEdqaiszVlYxaVZlVVYxRlpBQ1pPS2JtNVMvSVhtVWMxeVNVd1IwbCthNzlO?= =?utf-8?B?N2I2aGRZcmppUDhTTmZsWjQ3eDBhbnUxRU5URHBSanhvMlpIcG4vWTF1Y1d0?= =?utf-8?B?VEdHMlg2LzlNdlRsQWVtaERKalRydFZPUTZhVGNZaEpJZnNnYS80a1B5aWc0?= =?utf-8?B?KzdCbjIrOFdmOTFDNFpteTQ4bWd4N04rM0QzN1hUTTMwcGgwYWZHT3FYejJy?= =?utf-8?B?a0lNb0NRMXJXTDhndk4rY1c1elgwMDQ3U2tsdUNDVmZSTFc3dXFlelFsV0Zi?= =?utf-8?B?enFiQjl1eUJoWjltTnZlMDdhQ0Y1Vy9qQmJWQVFEUXVwelppMUZ5OW1Sb1Fi?= =?utf-8?B?MW9qbGZ0S1RPYmNpY3c0bnhjZjFpbzFQYTRuZlk0eGVKbVJOdmFBelZjYm9T?= =?utf-8?B?QlFKblZtak1JMktBRVREUlphVVVBUFBsN2hsbDhickEyUm1BYW1DdC84SzdP?= =?utf-8?B?TTBpR2JPdEVmMTNlaldFL0JtbUFFdDdCTzFucllkSHdoTVNWRjBJZTAzc3Yv?= =?utf-8?B?UWRhRTI5UzVyRmZsTWVVeDVNWEcvTTZXVVUzTEZJSzJINTFKdWxWQTNJaUYv?= =?utf-8?B?NTdncUt6MlhMeGR3ZUtzbkx1b3NxOHdrK1Y5N0JZYStVdWp3K2VOTkw4Zkp5?= =?utf-8?B?R29MWmlubVg5Qlp4LzRVYzFveDlvNEgzSjVPUjFVZG52aDhEdmVCOC9OQ2du?= =?utf-8?B?YUIzU0Y3NE16ek5POTlVWEZ5bVhnRlZiU1R2RFVvVGVhQ1h2Q0dnSnBHZTFM?= =?utf-8?B?M1BPWTR5MFZ4aWY2cEJaSC9Hb2EveWFNZS85cVVvTlUxaFpJOEQ2eGppTVZQ?= =?utf-8?B?Z0c2bXBFbk9rZmc4Qy80bWlUckVORTJWekFCbU4ycDJIbmlpWkJBVFdRaVFx?= =?utf-8?B?UUwvcFhITHFPQnIvM2YzT3kvTkZNRjYrOTUxeHlRd05NeE9IKzFaWG4rSXd3?= =?utf-8?B?MXByQ1p4Tm8rd0RGa0hxSjBYTGVzY29VNG5FbEl4V2g5QXlUYmQ5K0JUNTlS?= =?utf-8?B?TW9KSkZQZ3lkaDA4OVFjWi9SK0JodmR1cTNqckpBZEkrbzY4VldIZE1HTG5B?= =?utf-8?B?WkNydDk0NDA5MmVmSXNheU10VUlrYVJLdVRKeXF2Qll1VWtIMlBDMkpUS0RN?= =?utf-8?B?WlVybzdrUkhQL1YzRGNiS2JscXd3andxK01hZzF0OGoxSC9ya2tMaXR0VDIw?= =?utf-8?B?UmN0cEE4ODlQUFRHMjNNUjhaQjkvREZENVdyWHFEOWRxdXRvZFEzWW1NTTVY?= =?utf-8?Q?OrSwJrvs9rL6kQKvyzmvWROHU4kTYTIZpYVAmeG?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 394b8b67-f6e0-4e32-95fd-08d8c8a548e8 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2767.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Feb 2021 00:39:09.2835 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jMZcDv3rSlP29H3EiBeDHznRLJc1yyxRWQ0Dwm9IYk4L1Cvzixe2BS0kclnebHjU0AJ3P+Q6iemmGyUAExtdBA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4384 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Brijesh Singh Invoke a hypercall when a memory region is changed from encrypted -> decrypted and vice versa. Hypervisor needs to know the page encryption status during the guest migration. Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Paolo Bonzini Cc: "Radim Krčmář" Cc: Joerg Roedel Cc: Borislav Petkov Cc: Tom Lendacky Cc: x86@kernel.org Cc: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Reviewed-by: Venu Busireddy Signed-off-by: Brijesh Singh Signed-off-by: Ashish Kalra --- arch/x86/include/asm/paravirt.h | 10 +++++ arch/x86/include/asm/paravirt_types.h | 2 + arch/x86/kernel/paravirt.c | 1 + arch/x86/mm/mem_encrypt.c | 57 ++++++++++++++++++++++++++- arch/x86/mm/pat/set_memory.c | 7 ++++ 5 files changed, 76 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index f8dce11d2bc1..1265e1f5db5f 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -84,6 +84,12 @@ static inline void paravirt_arch_exit_mmap(struct mm_struct *mm) PVOP_VCALL1(mmu.exit_mmap, mm); } +static inline void page_encryption_changed(unsigned long vaddr, int npages, + bool enc) +{ + PVOP_VCALL3(mmu.page_encryption_changed, vaddr, npages, enc); +} + #ifdef CONFIG_PARAVIRT_XXL static inline void load_sp0(unsigned long sp0) { @@ -829,6 +835,10 @@ static inline void paravirt_arch_dup_mmap(struct mm_struct *oldmm, static inline void paravirt_arch_exit_mmap(struct mm_struct *mm) { } + +static inline void page_encryption_changed(unsigned long vaddr, int npages, bool enc) +{ +} #endif #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_PARAVIRT_H */ diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h index b6b02b7c19cc..6a83821cf758 100644 --- a/arch/x86/include/asm/paravirt_types.h +++ b/arch/x86/include/asm/paravirt_types.h @@ -208,6 +208,8 @@ struct pv_mmu_ops { /* Hook for intercepting the destruction of an mm_struct. */ void (*exit_mmap)(struct mm_struct *mm); + void (*page_encryption_changed)(unsigned long vaddr, int npages, + bool enc); #ifdef CONFIG_PARAVIRT_XXL struct paravirt_callee_save read_cr2; diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 6c3407ba6ee9..52913356b6fa 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -340,6 +340,7 @@ struct paravirt_patch_template pv_ops = { (void (*)(struct mmu_gather *, void *))tlb_remove_page, .mmu.exit_mmap = paravirt_nop, + .mmu.page_encryption_changed = paravirt_nop, #ifdef CONFIG_PARAVIRT_XXL .mmu.read_cr2 = __PV_IS_CALLEE_SAVE(native_read_cr2), diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c index c79e5736ab2b..dc17d14f9bcd 100644 --- a/arch/x86/mm/mem_encrypt.c +++ b/arch/x86/mm/mem_encrypt.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include @@ -29,6 +30,7 @@ #include #include #include +#include #include "mm_internal.h" @@ -229,6 +231,47 @@ void __init sev_setup_arch(void) swiotlb_adjust_size(size); } +static void set_memory_enc_dec_hypercall(unsigned long vaddr, int npages, + bool enc) +{ + unsigned long sz = npages << PAGE_SHIFT; + unsigned long vaddr_end, vaddr_next; + + vaddr_end = vaddr + sz; + + for (; vaddr < vaddr_end; vaddr = vaddr_next) { + int psize, pmask, level; + unsigned long pfn; + pte_t *kpte; + + kpte = lookup_address(vaddr, &level); + if (!kpte || pte_none(*kpte)) + return; + + switch (level) { + case PG_LEVEL_4K: + pfn = pte_pfn(*kpte); + break; + case PG_LEVEL_2M: + pfn = pmd_pfn(*(pmd_t *)kpte); + break; + case PG_LEVEL_1G: + pfn = pud_pfn(*(pud_t *)kpte); + break; + default: + return; + } + + psize = page_level_size(level); + pmask = page_level_mask(level); + + kvm_sev_hypercall3(KVM_HC_PAGE_ENC_STATUS, + pfn << PAGE_SHIFT, psize >> PAGE_SHIFT, enc); + + vaddr_next = (vaddr & pmask) + psize; + } +} + static void __init __set_clr_pte_enc(pte_t *kpte, int level, bool enc) { pgprot_t old_prot, new_prot; @@ -286,12 +329,13 @@ static void __init __set_clr_pte_enc(pte_t *kpte, int level, bool enc) static int __init early_set_memory_enc_dec(unsigned long vaddr, unsigned long size, bool enc) { - unsigned long vaddr_end, vaddr_next; + unsigned long vaddr_end, vaddr_next, start; unsigned long psize, pmask; int split_page_size_mask; int level, ret; pte_t *kpte; + start = vaddr; vaddr_next = vaddr; vaddr_end = vaddr + size; @@ -346,6 +390,8 @@ static int __init early_set_memory_enc_dec(unsigned long vaddr, ret = 0; + set_memory_enc_dec_hypercall(start, PAGE_ALIGN(size) >> PAGE_SHIFT, + enc); out: __flush_tlb_all(); return ret; @@ -479,6 +525,15 @@ void __init mem_encrypt_init(void) if (sev_active()) static_branch_enable(&sev_enable_key); +#ifdef CONFIG_PARAVIRT + /* + * With SEV, we need to make a hypercall when page encryption state is + * changed. + */ + if (sev_active()) + pv_ops.mmu.page_encryption_changed = set_memory_enc_dec_hypercall; +#endif + print_mem_encrypt_feature_info(); } diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index 16f878c26667..3576b583ac65 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -27,6 +27,7 @@ #include #include #include +#include #include "../mm_internal.h" @@ -2012,6 +2013,12 @@ static int __set_memory_enc_dec(unsigned long addr, int numpages, bool enc) */ cpa_flush(&cpa, 0); + /* Notify hypervisor that a given memory range is mapped encrypted + * or decrypted. The hypervisor will use this information during the + * VM migration. + */ + page_encryption_changed(addr, numpages, enc); + return ret; } -- 2.17.1