From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06ECEE82CA6 for ; Wed, 27 Sep 2023 15:35:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232487AbjI0Pfc (ORCPT ); Wed, 27 Sep 2023 11:35:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232363AbjI0Pf3 (ORCPT ); Wed, 27 Sep 2023 11:35:29 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AFFB3192; Wed, 27 Sep 2023 08:35:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1695828928; x=1727364928; h=to:cc:subject:references:date:mime-version: content-transfer-encoding:from:message-id:in-reply-to; bh=8QSD0wD8cLSsvCvzs8DbCnWU+wjcfau6s4KUAC2usv4=; b=QYcSNotmDcpFnJxJ1ruEK3h78UVdOJJ7JmfyGkWDMROlB+hzP5+yNtsb RDWnC8iqA8OPN33tn6AeDpZ/gNWpDuOr22GF7Ec6dbnRluqxYOMIAUCb2 ulnFqoVlzH3/4QiTvcQytXCf3PNAK9RxyFY5g66SmmmeHCDpT7zPY1aHk raw51FQHuKnpo4gOV7HY53ME6Kda+KqTYuSt0AkHLV6TL9T3eifmtDJRC nRJX6r/8pz/w1oH6naiC1CyRUleAqsTNXqX0Xtik2FlO7JIUQ2+2w/SGT WEddOdsHXtXnx0kqU3hMpc6TB0fJV91qndDY3rkmRYeMRUjODmZktCXgU w==; X-IronPort-AV: E=McAfee;i="6600,9927,10846"; a="445989627" X-IronPort-AV: E=Sophos;i="6.03,181,1694761200"; d="scan'208";a="445989627" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Sep 2023 08:35:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10846"; a="742752632" X-IronPort-AV: E=Sophos;i="6.03,181,1694761200"; d="scan'208";a="742752632" Received: from hhuan26-mobl.amr.corp.intel.com ([10.92.96.100]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 27 Sep 2023 08:35:25 -0700 Content-Type: text/plain; charset=iso-8859-15; format=flowed; delsp=yes To: "hpa@zytor.com" , "linux-sgx@vger.kernel.org" , "x86@kernel.org" , "dave.hansen@linux.intel.com" , "cgroups@vger.kernel.org" , "bp@alien8.de" , "linux-kernel@vger.kernel.org" , "jarkko@kernel.org" , "tglx@linutronix.de" , "Mehta, Sohil" , "tj@kernel.org" , "mingo@redhat.com" , "Huang, Kai" Cc: "kristen@linux.intel.com" , "yangjie@microsoft.com" , "Li, Zhiquan1" , "Christopherson,, Sean" , "mikko.ylinen@linux.intel.com" , "Zhang, Bo" , "anakrish@microsoft.com" Subject: Re: [PATCH v5 09/18] x86/sgx: Store struct sgx_encl when allocating new VA pages References: <20230923030657.16148-1-haitao.huang@linux.intel.com> <20230923030657.16148-10-haitao.huang@linux.intel.com> <46fecb95ae2c4cd156ad7bda99522214fcfe5774.camel@intel.com> Date: Wed, 27 Sep 2023 10:35:24 -0500 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Haitao Huang" Organization: Intel Message-ID: In-Reply-To: <46fecb95ae2c4cd156ad7bda99522214fcfe5774.camel@intel.com> User-Agent: Opera Mail/1.0 (Win32) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Kai, On Wed, 27 Sep 2023 06:14:20 -0500, Huang, Kai wrote: > On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: >> From: Sean Christopherson >> >> In a later patch, when a cgroup has exceeded the max capacity for EPC >> pages, it may need to identify and OOM kill a less active enclave to >> make room for other enclaves within the same group. Such a victim >> enclave would have no active pages other than the unreclaimable Version >> Array (VA) and SECS pages. Therefore, the cgroup needs examine its >> unreclaimable page list, and finding an enclave given a SECS page or a >> VA page. This will require a backpointer from a page to an enclave, >> which is not available for VA pages. >> >> Because struct sgx_epc_page instances of VA pages are not owned by an >> sgx_encl_page instance, mark their owner as sgx_encl: pass the struct >> sgx_encl of the enclave allocating the VA page to sgx_alloc_epc_page(), >> which will store this value in the owner field of the struct >> sgx_epc_page. In a later patch, VA pages will be placed in an >> unreclaimable queue that can be examined by the cgroup to select the OOM >> killed enclave. >> >> Signed-off-by: Sean Christopherson >> Co-developed-by: Kristen Carlson Accardi >> Signed-off-by: Kristen Carlson Accardi >> Co-developed-by: Haitao Huang >> Signed-off-by: Haitao Huang >> Cc: Sean Christopherson >> > > [...] > >> @@ -562,7 +562,7 @@ struct sgx_epc_page *sgx_alloc_epc_page(void >> *owner, bool reclaim) >> for ( ; ; ) { >> page = __sgx_alloc_epc_page(); >> if (!IS_ERR(page)) { >> - page->owner = owner; >> + page->encl_page = owner; > > Looks using 'encl_page' is arbitrary. > > Also actually for virtual EPC page the owner is set to the 'sgx_vepc' > instance. > >> break; >> } >> >> @@ -607,7 +607,7 @@ void sgx_free_epc_page(struct sgx_epc_page *page) >> >> spin_lock(&node->lock); >> >> - page->owner = NULL; >> + page->encl_page = NULL; > > Ditto. > >> if (page->poison) >> list_add(&page->list, &node->sgx_poison_page_list); >> else >> @@ -642,7 +642,7 @@ static bool __init sgx_setup_epc_section(u64 >> phys_addr, u64 size, >> for (i = 0; i < nr_pages; i++) { >> section->pages[i].section = index; >> section->pages[i].flags = 0; >> - section->pages[i].owner = NULL; >> + section->pages[i].encl_page = NULL; >> section->pages[i].poison = 0; >> list_add_tail(§ion->pages[i].list, &sgx_dirty_page_list); >> } >> diff --git a/arch/x86/kernel/cpu/sgx/sgx.h >> b/arch/x86/kernel/cpu/sgx/sgx.h >> index 764cec23f4e5..5110dd433b80 100644 >> --- a/arch/x86/kernel/cpu/sgx/sgx.h >> +++ b/arch/x86/kernel/cpu/sgx/sgx.h >> @@ -68,7 +68,12 @@ struct sgx_epc_page { >> unsigned int section; >> u16 flags; >> u16 poison; >> - struct sgx_encl_page *owner; >> + >> + /* Possible owner types */ >> + union { >> + struct sgx_encl_page *encl_page; >> + struct sgx_encl *encl; >> + }; > > Sadly for virtual EPC page the owner is set to the 'sgx_vepc' instance it > belongs to. > > Given how sgx_{alloc|free}_epc_page() arbitrarily uses encl_page, > perhaps we > should do below? > > union { > struct sgx_encl_page *encl_page; > struct sgx_encl *encl; > struct sgx_vepc *vepc; > void *owner; > }; > > And in sgx_{alloc|free}_epc_page() we can use 'owner' instead. > As I mentioned in cover letter and change log in 11/18, this series does not track virtual EPC. We can add vepc field into the union in future if such tracking is needed. Don't think "void *owner" is needed though. Thanks Haitao