From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from mail.kernel.org (mail.kernel.org [198.145.29.99])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 7225DC433EF
	for <linux-kernel@archiver.kernel.org>; Tue, 16 Nov 2021 16:21:25 +0000 (UTC)
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by mail.kernel.org (Postfix) with ESMTP id 5787261BD3
	for <linux-kernel@archiver.kernel.org>; Tue, 16 Nov 2021 16:21:25 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S231314AbhKPQYV (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Tue, 16 Nov 2021 11:24:21 -0500
Received: from mail.kernel.org ([198.145.29.99]:41998 "EHLO mail.kernel.org"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S230232AbhKPQYT (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Tue, 16 Nov 2021 11:24:19 -0500
Received: by mail.kernel.org (Postfix) with ESMTPSA id 3E3AC61452;
        Tue, 16 Nov 2021 16:21:22 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
        s=k20201202; t=1637079682;
        bh=ywIASA+kCwdP8ALqLLEnfdf/kPnlO3UamUlO4O1gU+A=;
        h=From:To:Cc:Subject:Date:From;
        b=qfD8iQrLERqwBCh/MzbhGIKM3xc9ARwi2e2hxBHwZd8rdfPJLsfh0RxUD5o3kdkdQ
         v9rNA9dOAX4H2u4uA7qpPK+8VHgCPE4gxJTLZWG8MpJ2Nf+/sTg7rdLfFjDCfHpSqP
         /d52kCtLDxf6MosXKoyHTB1iOMkoNfSoDHBssfQkTS7QaR/HJHUNjRFbsmaBpK1GWN
         uEIeJfQoxvfYW3sG1ESwT8FJ+CbJ90RSdNRrUAAHXKgKHZ5eTZJsGtKlp3g82nxuoQ
         7ueqdg2Yx5WYKPtU1V1ekbfwTmX53o3o0nYX2gD+d1HE8TGbWYGX0FE61t48QgKQ7p
         CHp+IhqZWOPRA==
From:   Jarkko Sakkinen <jarkko@kernel.org>
To:     Jarkko Sakkinen <jarkko@kernel.org>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>
Cc:     reinette.chatre@intel.com, linux-sgx@vger.kernel.org,
        linux-kernel@vger.kernel.org
Subject: [PATCH v13 1/2] x86/sgx: Rename fallback labels in sgx_init()
Date:   Tue, 16 Nov 2021 18:21:15 +0200
Message-Id: <20211116162116.93081-1-jarkko@kernel.org>
X-Mailer: git-send-email 2.32.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

It's hard to add new content to this function because it is time
consuming to match fallback and its cause. Rename labels in a way
that the name of error label refers to the site where failure
happened. This way it is easier to keep on track what is going
on.

Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
---
v5:
* A new patch.
---
 arch/x86/kernel/cpu/sgx/main.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 231c494dfd40..b8e1703eecae 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -914,12 +914,12 @@ static int __init sgx_init(void)
 
 	if (!sgx_page_reclaimer_init()) {
 		ret = -ENOMEM;
-		goto err_page_cache;
+		goto err_reclaimer;
 	}
 
 	ret = misc_register(&sgx_dev_provision);
 	if (ret)
-		goto err_kthread;
+		goto err_provision;
 
 	/*
 	 * Always try to initialize the native *and* KVM drivers.
@@ -932,17 +932,17 @@ static int __init sgx_init(void)
 	ret = sgx_drv_init();
 
 	if (sgx_vepc_init() && ret)
-		goto err_provision;
+		goto err_driver;
 
 	return 0;
 
-err_provision:
+err_driver:
 	misc_deregister(&sgx_dev_provision);
 
-err_kthread:
+err_provision:
 	kthread_stop(ksgxd_tsk);
 
-err_page_cache:
+err_reclaimer:
 	for (i = 0; i < sgx_nr_epc_sections; i++) {
 		vfree(sgx_epc_sections[i].pages);
 		memunmap(sgx_epc_sections[i].virt_addr);
-- 
2.32.0


From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from mail.kernel.org (mail.kernel.org [198.145.29.99])
	by smtp.lore.kernel.org (Postfix) with ESMTP id D6ED4C433EF
	for <linux-kernel@archiver.kernel.org>; Tue, 16 Nov 2021 16:21:33 +0000 (UTC)
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by mail.kernel.org (Postfix) with ESMTP id C0FEF61BD3
	for <linux-kernel@archiver.kernel.org>; Tue, 16 Nov 2021 16:21:33 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S230232AbhKPQY3 (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Tue, 16 Nov 2021 11:24:29 -0500
Received: from mail.kernel.org ([198.145.29.99]:42104 "EHLO mail.kernel.org"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S231324AbhKPQYX (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Tue, 16 Nov 2021 11:24:23 -0500
Received: by mail.kernel.org (Postfix) with ESMTPSA id 9A70F61BE2;
        Tue, 16 Nov 2021 16:21:25 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
        s=k20201202; t=1637079686;
        bh=dVeRd+rpu6BuxDclanJO71ukMtRRehqTwia8OT676z4=;
        h=From:To:Cc:Subject:Date:In-Reply-To:References:From;
        b=SEJoEoMIrJ5qjx2IfB565iw4NrOdsIOlqrqqX/iqz0yROQLRyYI/E1cluCmFKpS7M
         ilA0hjfhbLdF9VRuug/QyS0CARA6fzKAuHlqeghQPQuVdaL0vtbdolliB7lopkGehQ
         hMGSjjwsEcPp5SxYj8CjXK/i7HRZlHW3rSd1pC1e8zkCbjWcmzml07+tSq3XrRMAK1
         e8R4syORC5D7hLHC58OyUva3jOOg1B1FmGEp+TJgKwrAWZW6/BHc7PylSoIbhDQqJ5
         kooGSzBE0l915viMBoWj+ig7TuOMfJ+KdIQTBndqFFio6udleeMMOA8MixqWUWRV/f
         kT3nDyDolDz/A==
From:   Jarkko Sakkinen <jarkko@kernel.org>
To:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Jarkko Sakkinen <jarkko@kernel.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>
Cc:     reinette.chatre@intel.com, linux-kernel@vger.kernel.org,
        linux-sgx@vger.kernel.org
Subject: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX memory in a NUMA node
Date:   Tue, 16 Nov 2021 18:21:16 +0200
Message-Id: <20211116162116.93081-2-jarkko@kernel.org>
X-Mailer: git-send-email 2.32.0
In-Reply-To: <20211116162116.93081-1-jarkko@kernel.org>
References: <20211116162116.93081-1-jarkko@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

The amount of SGX memory on the system is determined by the BIOS and it
varies wildly between systems.  It can be from dozens of MB's on desktops
or VM's, up to many GB's on servers.  Just like for regular memory, it is
sometimes useful to know the amount of usable SGX memory in the system.

Introduce CONFIG_HAVE_ARCH_NODE_DEV_GROUP opt-in flag to expose an arch
specific attribute group, and add an attribute for the amount of SGX
memory in bytes to each NUMA node:

/sys/devices/system/node/nodeX/x86/sgx_total_bytes

Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
---
v13:
* Rebased on top of tip/x86/sgx.

v12:
* Device ID is set to same as NUMA node ID. Use that to get the correct
  NUMA node ID in sgx_total_bytes_show().

v11:
* Fix documentation and the commit message.

v10:
* Change DEVICE_ATTR_RO() to static (Greg K-H)
* Change the attribute name as sgx_total_bytes, and attribute group
  name as "x86" (Dave).
* Add a new config flag HAVE_ARCH_NODE_DEV_GROUP to identify, whether
  an arch exports arch specific attribute group for NUMA nodes.

v9:
* Fix racy initialization of sysfs attributes:
  https://lore.kernel.org/linux-sgx/YXOsx8SvFJV5R7lU@kroah.com/

v8:
* Fix a bug in sgx_numa_init(): node->dev should be only set after
  sysfe_create_group().  Otherwise, sysfs_remove_group() will issue a
  warning in sgx_numa_exit(), when sgx_create_group() is unsuccessful,
  because the group does not exist.

v7:
* Shorten memory_size to size. The prefix makes the name only longer
  but does not clarify things more than "size" would.
* Use device_attribute instead of kobj_attribute.
* Use named attribute group instead of creating raw kobject just for
  the "sgx" subdirectory.

v6:
* Initialize node->size to zero in sgx_setup_epc_section(), when the
  node is first accessed.

v5
* A new patch based on the discussion on
  https://lore.kernel.org/linux-sgx/3a7cab4115b4f902f3509ad8652e616b91703e1d.camel@kernel.org/T/#t
---
 Documentation/ABI/stable/sysfs-devices-node |  6 ++++++
 arch/Kconfig                                |  4 ++++
 arch/x86/Kconfig                            |  1 +
 arch/x86/kernel/cpu/sgx/main.c              | 20 ++++++++++++++++++++
 arch/x86/kernel/cpu/sgx/sgx.h               |  1 +
 drivers/base/node.c                         |  3 +++
 include/linux/numa.h                        |  4 ++++
 7 files changed, 39 insertions(+)

diff --git a/Documentation/ABI/stable/sysfs-devices-node b/Documentation/ABI/stable/sysfs-devices-node
index 484fc04bcc25..8db67aa472f1 100644
--- a/Documentation/ABI/stable/sysfs-devices-node
+++ b/Documentation/ABI/stable/sysfs-devices-node
@@ -176,3 +176,9 @@ Contact:	Keith Busch <keith.busch@intel.com>
 Description:
 		The cache write policy: 0 for write-back, 1 for write-through,
 		other or unknown.
+
+What:		/sys/devices/system/node/nodeX/x86/sgx_total_bytes
+Date:		November 2021
+Contact:	Jarkko Sakkinen <jarkko@kernel.org>
+Description:
+		The total amount of SGX physical memory in bytes.
diff --git a/arch/Kconfig b/arch/Kconfig
index 26b8ed11639d..0a9dadb00b61 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1302,6 +1302,10 @@ config ARCH_HAS_PARANOID_L1D_FLUSH
 config DYNAMIC_SIGFRAME
 	bool
 
+# Select, if arch has a named attribute group bound to NUMA device nodes.
+config HAVE_ARCH_NODE_DEV_GROUP
+	bool
+
 source "kernel/gcov/Kconfig"
 
 source "scripts/gcc-plugins/Kconfig"
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index b9281fab4e3e..f2b699d12eb8 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -269,6 +269,7 @@ config X86
 	select HAVE_ARCH_KCSAN			if X86_64
 	select X86_FEATURE_NAMES		if PROC_FS
 	select PROC_PID_ARCH_STATUS		if PROC_FS
+	select HAVE_ARCH_NODE_DEV_GROUP		if X86_SGX
 	imply IMA_SECURE_AND_OR_TRUSTED_BOOT    if EFI
 
 config INSTRUCTION_DECODER
diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index b8e1703eecae..00af6f90d31a 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -825,9 +825,11 @@ static bool __init sgx_page_cache_init(void)
 			INIT_LIST_HEAD(&sgx_numa_nodes[nid].free_page_list);
 			INIT_LIST_HEAD(&sgx_numa_nodes[nid].sgx_poison_page_list);
 			node_set(nid, sgx_numa_mask);
+			sgx_numa_nodes[nid].size = 0;
 		}
 
 		sgx_epc_sections[i].node =  &sgx_numa_nodes[nid];
+		sgx_numa_nodes[nid].size += size;
 
 		sgx_nr_epc_sections++;
 	}
@@ -901,6 +903,24 @@ int sgx_set_attribute(unsigned long *allowed_attributes,
 }
 EXPORT_SYMBOL_GPL(sgx_set_attribute);
 
+#ifdef CONFIG_NUMA
+static ssize_t sgx_total_bytes_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+	return sysfs_emit(buf, "%lu\n", sgx_numa_nodes[dev->id].size);
+}
+static DEVICE_ATTR_RO(sgx_total_bytes);
+
+static struct attribute *arch_node_dev_attrs[] = {
+	&dev_attr_sgx_total_bytes.attr,
+	NULL,
+};
+
+const struct attribute_group arch_node_dev_group = {
+	.name = "x86",
+	.attrs = arch_node_dev_attrs,
+};
+#endif /* CONFIG_NUMA */
+
 static int __init sgx_init(void)
 {
 	int ret;
diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
index 9ec3136c7800..0f17def9fe6f 100644
--- a/arch/x86/kernel/cpu/sgx/sgx.h
+++ b/arch/x86/kernel/cpu/sgx/sgx.h
@@ -44,6 +44,7 @@ struct sgx_epc_page {
 struct sgx_numa_node {
 	struct list_head free_page_list;
 	struct list_head sgx_poison_page_list;
+	unsigned long size;
 	spinlock_t lock;
 };
 
diff --git a/drivers/base/node.c b/drivers/base/node.c
index b5a4ba18f9f9..87acc47e8951 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -581,6 +581,9 @@ static const struct attribute_group node_dev_group = {
 
 static const struct attribute_group *node_dev_groups[] = {
 	&node_dev_group,
+#ifdef CONFIG_HAVE_ARCH_NODE_DEV_GROUP
+	&arch_node_dev_group,
+#endif
 	NULL
 };
 
diff --git a/include/linux/numa.h b/include/linux/numa.h
index cb44cfe2b725..59df211d051f 100644
--- a/include/linux/numa.h
+++ b/include/linux/numa.h
@@ -58,4 +58,8 @@ static inline int phys_to_target_node(u64 start)
 }
 #endif
 
+#ifdef CONFIG_HAVE_ARCH_NODE_DEV_GROUP
+extern const struct attribute_group arch_node_dev_group;
+#endif
+
 #endif /* _LINUX_NUMA_H */
-- 
2.32.0


From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id CDB61C433F5
	for <linux-kernel@archiver.kernel.org>; Sat,  4 Dec 2021 23:51:57 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1345441AbhLDXzW (ORCPT
        <rfc822;linux-kernel@archiver.kernel.org>);
        Sat, 4 Dec 2021 18:55:22 -0500
Received: from ams.source.kernel.org ([145.40.68.75]:56388 "EHLO
        ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1345387AbhLDXzW (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Sat, 4 Dec 2021 18:55:22 -0500
Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140])
        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
        (No client certificate requested)
        by ams.source.kernel.org (Postfix) with ESMTPS id C6A38B80CF7;
        Sat,  4 Dec 2021 23:51:54 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9CE52C341C2;
        Sat,  4 Dec 2021 23:51:52 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
        s=k20201202; t=1638661913;
        bh=gIwRRmmFn/lz89is2lQVmQzHi97B5tvfl0lAabRnaqo=;
        h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
        b=lUu4R+9wdXcGbgk4IYXK94StXVkRz2nrWqOGhv1+etoIsDspobZ4Qpn7ixeiR65TV
         Ssm/c5wGquPRzma3/gNFwFh9Ff0GirXhUgln2g8TFWRWgojItqD17lnKHWjRFdvB+C
         fk2PtksFk7F6xiySvW1MaDWoLGc+fQ537UgquSz4xMB2nCOI5hAWJ+V4KnmgKQI+2T
         Lc4VDjXsBG6QpHwFs7im+A8lzRzRETT+gvBvXnY6dsMKW8LZlljZk73GSgTrmxVbfe
         zxFrXGYal7MeUKh18ZgIT/iLQAsnXjgXEpEys6TEmfYskLQQ7ujrDV42xTluW5br8r
         ZqCtFAhXnUEcQ==
Date:   Sun, 5 Dec 2021 01:51:49 +0200
From:   Jarkko Sakkinen <jarkko@kernel.org>
To:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>
Cc:     reinette.chatre@intel.com, linux-kernel@vger.kernel.org,
        linux-sgx@vger.kernel.org
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
Message-ID: <Yav/FXZhUL3od1qx@iki.fi>
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20211116162116.93081-2-jarkko@kernel.org>
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Nov 16, 2021 at 06:21:16PM +0200, Jarkko Sakkinen wrote:
> The amount of SGX memory on the system is determined by the BIOS and it
> varies wildly between systems.  It can be from dozens of MB's on desktops
> or VM's, up to many GB's on servers.  Just like for regular memory, it is
> sometimes useful to know the amount of usable SGX memory in the system.
> 
> Introduce CONFIG_HAVE_ARCH_NODE_DEV_GROUP opt-in flag to expose an arch
> specific attribute group, and add an attribute for the amount of SGX
> memory in bytes to each NUMA node:
> 
> /sys/devices/system/node/nodeX/x86/sgx_total_bytes
> 
> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
> ---
> v13:
> * Rebased on top of tip/x86/sgx.
> 
> v12:
> * Device ID is set to same as NUMA node ID. Use that to get the correct
>   NUMA node ID in sgx_total_bytes_show().
> 
> v11:
> * Fix documentation and the commit message.
> 
> v10:
> * Change DEVICE_ATTR_RO() to static (Greg K-H)
> * Change the attribute name as sgx_total_bytes, and attribute group
>   name as "x86" (Dave).
> * Add a new config flag HAVE_ARCH_NODE_DEV_GROUP to identify, whether
>   an arch exports arch specific attribute group for NUMA nodes.
> 
> v9:
> * Fix racy initialization of sysfs attributes:
>   https://lore.kernel.org/linux-sgx/YXOsx8SvFJV5R7lU@kroah.com/
> 
> v8:
> * Fix a bug in sgx_numa_init(): node->dev should be only set after
>   sysfe_create_group().  Otherwise, sysfs_remove_group() will issue a
>   warning in sgx_numa_exit(), when sgx_create_group() is unsuccessful,
>   because the group does not exist.
> 
> v7:
> * Shorten memory_size to size. The prefix makes the name only longer
>   but does not clarify things more than "size" would.
> * Use device_attribute instead of kobj_attribute.
> * Use named attribute group instead of creating raw kobject just for
>   the "sgx" subdirectory.
> 
> v6:
> * Initialize node->size to zero in sgx_setup_epc_section(), when the
>   node is first accessed.
> 
> v5
> * A new patch based on the discussion on
>   https://lore.kernel.org/linux-sgx/3a7cab4115b4f902f3509ad8652e616b91703e1d.camel@kernel.org/T/#t
> ---
>  Documentation/ABI/stable/sysfs-devices-node |  6 ++++++
>  arch/Kconfig                                |  4 ++++
>  arch/x86/Kconfig                            |  1 +
>  arch/x86/kernel/cpu/sgx/main.c              | 20 ++++++++++++++++++++
>  arch/x86/kernel/cpu/sgx/sgx.h               |  1 +
>  drivers/base/node.c                         |  3 +++
>  include/linux/numa.h                        |  4 ++++
>  7 files changed, 39 insertions(+)
> 
> diff --git a/Documentation/ABI/stable/sysfs-devices-node b/Documentation/ABI/stable/sysfs-devices-node
> index 484fc04bcc25..8db67aa472f1 100644
> --- a/Documentation/ABI/stable/sysfs-devices-node
> +++ b/Documentation/ABI/stable/sysfs-devices-node
> @@ -176,3 +176,9 @@ Contact:	Keith Busch <keith.busch@intel.com>
>  Description:
>  		The cache write policy: 0 for write-back, 1 for write-through,
>  		other or unknown.
> +
> +What:		/sys/devices/system/node/nodeX/x86/sgx_total_bytes
> +Date:		November 2021
> +Contact:	Jarkko Sakkinen <jarkko@kernel.org>
> +Description:
> +		The total amount of SGX physical memory in bytes.
> diff --git a/arch/Kconfig b/arch/Kconfig
> index 26b8ed11639d..0a9dadb00b61 100644
> --- a/arch/Kconfig
> +++ b/arch/Kconfig
> @@ -1302,6 +1302,10 @@ config ARCH_HAS_PARANOID_L1D_FLUSH
>  config DYNAMIC_SIGFRAME
>  	bool
>  
> +# Select, if arch has a named attribute group bound to NUMA device nodes.
> +config HAVE_ARCH_NODE_DEV_GROUP
> +	bool
> +
>  source "kernel/gcov/Kconfig"
>  
>  source "scripts/gcc-plugins/Kconfig"
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index b9281fab4e3e..f2b699d12eb8 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -269,6 +269,7 @@ config X86
>  	select HAVE_ARCH_KCSAN			if X86_64
>  	select X86_FEATURE_NAMES		if PROC_FS
>  	select PROC_PID_ARCH_STATUS		if PROC_FS
> +	select HAVE_ARCH_NODE_DEV_GROUP		if X86_SGX
>  	imply IMA_SECURE_AND_OR_TRUSTED_BOOT    if EFI
>  
>  config INSTRUCTION_DECODER
> diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> index b8e1703eecae..00af6f90d31a 100644
> --- a/arch/x86/kernel/cpu/sgx/main.c
> +++ b/arch/x86/kernel/cpu/sgx/main.c
> @@ -825,9 +825,11 @@ static bool __init sgx_page_cache_init(void)
>  			INIT_LIST_HEAD(&sgx_numa_nodes[nid].free_page_list);
>  			INIT_LIST_HEAD(&sgx_numa_nodes[nid].sgx_poison_page_list);
>  			node_set(nid, sgx_numa_mask);
> +			sgx_numa_nodes[nid].size = 0;
>  		}
>  
>  		sgx_epc_sections[i].node =  &sgx_numa_nodes[nid];
> +		sgx_numa_nodes[nid].size += size;
>  
>  		sgx_nr_epc_sections++;
>  	}
> @@ -901,6 +903,24 @@ int sgx_set_attribute(unsigned long *allowed_attributes,
>  }
>  EXPORT_SYMBOL_GPL(sgx_set_attribute);
>  
> +#ifdef CONFIG_NUMA
> +static ssize_t sgx_total_bytes_show(struct device *dev, struct device_attribute *attr, char *buf)
> +{
> +	return sysfs_emit(buf, "%lu\n", sgx_numa_nodes[dev->id].size);
> +}
> +static DEVICE_ATTR_RO(sgx_total_bytes);
> +
> +static struct attribute *arch_node_dev_attrs[] = {
> +	&dev_attr_sgx_total_bytes.attr,
> +	NULL,
> +};
> +
> +const struct attribute_group arch_node_dev_group = {
> +	.name = "x86",
> +	.attrs = arch_node_dev_attrs,
> +};
> +#endif /* CONFIG_NUMA */
> +
>  static int __init sgx_init(void)
>  {
>  	int ret;
> diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
> index 9ec3136c7800..0f17def9fe6f 100644
> --- a/arch/x86/kernel/cpu/sgx/sgx.h
> +++ b/arch/x86/kernel/cpu/sgx/sgx.h
> @@ -44,6 +44,7 @@ struct sgx_epc_page {
>  struct sgx_numa_node {
>  	struct list_head free_page_list;
>  	struct list_head sgx_poison_page_list;
> +	unsigned long size;
>  	spinlock_t lock;
>  };
>  
> diff --git a/drivers/base/node.c b/drivers/base/node.c
> index b5a4ba18f9f9..87acc47e8951 100644
> --- a/drivers/base/node.c
> +++ b/drivers/base/node.c
> @@ -581,6 +581,9 @@ static const struct attribute_group node_dev_group = {
>  
>  static const struct attribute_group *node_dev_groups[] = {
>  	&node_dev_group,
> +#ifdef CONFIG_HAVE_ARCH_NODE_DEV_GROUP
> +	&arch_node_dev_group,
> +#endif
>  	NULL
>  };
>  
> diff --git a/include/linux/numa.h b/include/linux/numa.h
> index cb44cfe2b725..59df211d051f 100644
> --- a/include/linux/numa.h
> +++ b/include/linux/numa.h
> @@ -58,4 +58,8 @@ static inline int phys_to_target_node(u64 start)
>  }
>  #endif
>  
> +#ifdef CONFIG_HAVE_ARCH_NODE_DEV_GROUP
> +extern const struct attribute_group arch_node_dev_group;
> +#endif
> +
>  #endif /* _LINUX_NUMA_H */
> -- 
> 2.32.0
> 

So what still needs to be done to this?

/Jarkko

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 26BCAC433EF
	for <linux-kernel@archiver.kernel.org>; Tue,  7 Dec 2021 19:37:00 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S235812AbhLGTk3 (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Tue, 7 Dec 2021 14:40:29 -0500
Received: from mga04.intel.com ([192.55.52.120]:24373 "EHLO mga04.intel.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S231629AbhLGTkZ (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Tue, 7 Dec 2021 14:40:25 -0500
X-IronPort-AV: E=McAfee;i="6200,9189,10191"; a="236407307"
X-IronPort-AV: E=Sophos;i="5.87,295,1631602800"; 
   d="scan'208";a="236407307"
Received: from orsmga008.jf.intel.com ([10.7.209.65])
  by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Dec 2021 11:36:53 -0800
X-IronPort-AV: E=Sophos;i="5.87,295,1631602800"; 
   d="scan'208";a="515431114"
Received: from yperng-mobl1.amr.corp.intel.com (HELO [10.209.19.84]) ([10.209.19.84])
  by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Dec 2021 11:36:52 -0800
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
To:     Jarkko Sakkinen <jarkko@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>
Cc:     reinette.chatre@intel.com, linux-kernel@vger.kernel.org,
        linux-sgx@vger.kernel.org
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org>
From:   Dave Hansen <dave.hansen@intel.com>
Autocrypt: addr=dave.hansen@intel.com; keydata=
 xsFNBE6HMP0BEADIMA3XYkQfF3dwHlj58Yjsc4E5y5G67cfbt8dvaUq2fx1lR0K9h1bOI6fC
 oAiUXvGAOxPDsB/P6UEOISPpLl5IuYsSwAeZGkdQ5g6m1xq7AlDJQZddhr/1DC/nMVa/2BoY
 2UnKuZuSBu7lgOE193+7Uks3416N2hTkyKUSNkduyoZ9F5twiBhxPJwPtn/wnch6n5RsoXsb
 ygOEDxLEsSk/7eyFycjE+btUtAWZtx+HseyaGfqkZK0Z9bT1lsaHecmB203xShwCPT49Blxz
 VOab8668QpaEOdLGhtvrVYVK7x4skyT3nGWcgDCl5/Vp3TWA4K+IofwvXzX2ON/Mj7aQwf5W
 iC+3nWC7q0uxKwwsddJ0Nu+dpA/UORQWa1NiAftEoSpk5+nUUi0WE+5DRm0H+TXKBWMGNCFn
 c6+EKg5zQaa8KqymHcOrSXNPmzJuXvDQ8uj2J8XuzCZfK4uy1+YdIr0yyEMI7mdh4KX50LO1
 pmowEqDh7dLShTOif/7UtQYrzYq9cPnjU2ZW4qd5Qz2joSGTG9eCXLz5PRe5SqHxv6ljk8mb
 ApNuY7bOXO/A7T2j5RwXIlcmssqIjBcxsRRoIbpCwWWGjkYjzYCjgsNFL6rt4OL11OUF37wL
 QcTl7fbCGv53KfKPdYD5hcbguLKi/aCccJK18ZwNjFhqr4MliQARAQABzShEYXZpZCBDaHJp
 c3RvcGhlciBIYW5zZW4gPGRhdmVAc3I3MS5uZXQ+wsF7BBMBAgAlAhsDBgsJCAcDAgYVCAIJ
 CgsEFgIDAQIeAQIXgAUCTo3k0QIZAQAKCRBoNZUwcMmSsMO2D/421Xg8pimb9mPzM5N7khT0
 2MCnaGssU1T59YPE25kYdx2HntwdO0JA27Wn9xx5zYijOe6B21ufrvsyv42auCO85+oFJWfE
 K2R/IpLle09GDx5tcEmMAHX6KSxpHmGuJmUPibHVbfep2aCh9lKaDqQR07gXXWK5/yU1Dx0r
 VVFRaHTasp9fZ9AmY4K9/BSA3VkQ8v3OrxNty3OdsrmTTzO91YszpdbjjEFZK53zXy6tUD2d
 e1i0kBBS6NLAAsqEtneplz88T/v7MpLmpY30N9gQU3QyRC50jJ7LU9RazMjUQY1WohVsR56d
 ORqFxS8ChhyJs7BI34vQusYHDTp6PnZHUppb9WIzjeWlC7Jc8lSBDlEWodmqQQgp5+6AfhTD
 kDv1a+W5+ncq+Uo63WHRiCPuyt4di4/0zo28RVcjtzlGBZtmz2EIC3vUfmoZbO/Gn6EKbYAn
 rzz3iU/JWV8DwQ+sZSGu0HmvYMt6t5SmqWQo/hyHtA7uF5Wxtu1lCgolSQw4t49ZuOyOnQi5
 f8R3nE7lpVCSF1TT+h8kMvFPv3VG7KunyjHr3sEptYxQs4VRxqeirSuyBv1TyxT+LdTm6j4a
 mulOWf+YtFRAgIYyyN5YOepDEBv4LUM8Tz98lZiNMlFyRMNrsLV6Pv6SxhrMxbT6TNVS5D+6
 UorTLotDZKp5+M7BTQRUY85qARAAsgMW71BIXRgxjYNCYQ3Xs8k3TfAvQRbHccky50h99TUY
 sqdULbsb3KhmY29raw1bgmyM0a4DGS1YKN7qazCDsdQlxIJp9t2YYdBKXVRzPCCsfWe1dK/q
 66UVhRPP8EGZ4CmFYuPTxqGY+dGRInxCeap/xzbKdvmPm01Iw3YFjAE4PQ4hTMr/H76KoDbD
 cq62U50oKC83ca/PRRh2QqEqACvIH4BR7jueAZSPEDnzwxvVgzyeuhwqHY05QRK/wsKuhq7s
 UuYtmN92Fasbxbw2tbVLZfoidklikvZAmotg0dwcFTjSRGEg0Gr3p/xBzJWNavFZZ95Rj7Et
 db0lCt0HDSY5q4GMR+SrFbH+jzUY/ZqfGdZCBqo0cdPPp58krVgtIGR+ja2Mkva6ah94/oQN
 lnCOw3udS+Eb/aRcM6detZr7XOngvxsWolBrhwTQFT9D2NH6ryAuvKd6yyAFt3/e7r+HHtkU
 kOy27D7IpjngqP+b4EumELI/NxPgIqT69PQmo9IZaI/oRaKorYnDaZrMXViqDrFdD37XELwQ
 gmLoSm2VfbOYY7fap/AhPOgOYOSqg3/Nxcapv71yoBzRRxOc4FxmZ65mn+q3rEM27yRztBW9
 AnCKIc66T2i92HqXCw6AgoBJRjBkI3QnEkPgohQkZdAb8o9WGVKpfmZKbYBo4pEAEQEAAcLB
 XwQYAQIACQUCVGPOagIbDAAKCRBoNZUwcMmSsJeCEACCh7P/aaOLKWQxcnw47p4phIVR6pVL
 e4IEdR7Jf7ZL00s3vKSNT+nRqdl1ugJx9Ymsp8kXKMk9GSfmZpuMQB9c6io1qZc6nW/3TtvK
 pNGz7KPPtaDzvKA4S5tfrWPnDr7n15AU5vsIZvgMjU42gkbemkjJwP0B1RkifIK60yQqAAlT
 YZ14P0dIPdIPIlfEPiAWcg5BtLQU4Wg3cNQdpWrCJ1E3m/RIlXy/2Y3YOVVohfSy+4kvvYU3
 lXUdPb04UPw4VWwjcVZPg7cgR7Izion61bGHqVqURgSALt2yvHl7cr68NYoFkzbNsGsye9ft
 M9ozM23JSgMkRylPSXTeh5JIK9pz2+etco3AfLCKtaRVysjvpysukmWMTrx8QnI5Nn5MOlJj
 1Ov4/50JY9pXzgIDVSrgy6LYSMc4vKZ3QfCY7ipLRORyalFDF3j5AGCMRENJjHPD6O7bl3Xo
 4DzMID+8eucbXxKiNEbs21IqBZbbKdY1GkcEGTE7AnkA3Y6YB7I/j9mQ3hCgm5muJuhM/2Fr
 OPsw5tV/LmQ5GXH0JQ/TZXWygyRFyyI2FqNTx4WHqUn3yFj8rwTAU1tluRUYyeLy0ayUlKBH
 ybj0N71vWO936MqP6haFERzuPAIpxj2ezwu0xb1GjTk4ynna6h5GjnKgdfOWoRtoWndMZxbA
 z5cecg==
Message-ID: <f25d95e6-e129-597b-5d93-d7264feae8b8@intel.com>
Date:   Tue, 7 Dec 2021 11:36:50 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20211116162116.93081-2-jarkko@kernel.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 11/16/21 8:21 AM, Jarkko Sakkinen wrote:
> The amount of SGX memory on the system is determined by the BIOS and it
> varies wildly between systems.  It can be from dozens of MB's on desktops
> or VM's, up to many GB's on servers.  Just like for regular memory, it is
> sometimes useful to know the amount of usable SGX memory in the system.
> 
> Introduce CONFIG_HAVE_ARCH_NODE_DEV_GROUP opt-in flag to expose an arch
> specific attribute group, and add an attribute for the amount of SGX
> memory in bytes to each NUMA node:
> 
> /sys/devices/system/node/nodeX/x86/sgx_total_bytes

There's some context missing here:

This serves the same function for SGX memory as /proc/meminfo or
/sys/devices/system/node/nodeX/meminfo does for normal RAM.  It
enumerates how much physical SGX memory is present so that you can size
enclaves on different systems.

This specific file (sgx_total_bytes) is needed today to help drive the
SGX selftests.  The SGX selftests need to create overcommitted enclaves
which are larger than the physical SGX memory on the system.  They
currently use a CPUID-based approach which can diverge from the actual
amount of SGX memory available.  This file ensures that the selftests
can work efficiently and do not attempt stupid things like creating a
100,000 MB enclave on a system with 128 MB  of SGX memory.

The nodeX/x86 directory is used because SGX is highly x86-specific.
It's very unlikely that any other architecture (or even non-Intel x86
vendor) will ever implement SGX.  It needs its own directory (as opposed
to being in the nodeX/ "root") because this is expected to be the first
of a few different things that need to get exported.  This avoids
cluttering the root with several "sgx_*" files.

How many of these files will there be?  Just scanning /proc/meminfo,
these are the no-brainers that we have for RAM, but we need for SGX:

MemTotal:       xxxx kB // sgx_total_bytes (this patch)
MemFree:        yyyy kB // sgx_free_bytes
SwapTotal:      zzzz kB // sgx_swapped_bytes

So, at *least* three.  I think we will eventually end up needing
something more along the lines of a dozen.

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 6A6CCC433F5
	for <linux-kernel@archiver.kernel.org>; Wed,  8 Dec 2021 10:10:37 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S230341AbhLHKOH (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Wed, 8 Dec 2021 05:14:07 -0500
Received: from mail.skyhub.de ([5.9.137.197]:49818 "EHLO mail.skyhub.de"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S229453AbhLHKOG (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Wed, 8 Dec 2021 05:14:06 -0500
Received: from zn.tnic (dslb-088-067-202-008.088.067.pools.vodafone-ip.de [88.67.202.8])
        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
        (No client certificate requested)
        by mail.skyhub.de (SuperMail on ZX Spectrum 128k) with ESMTPSA id B64E91EC04EC;
        Wed,  8 Dec 2021 11:10:29 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alien8.de; s=dkim;
        t=1638958229;
        h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
         to:to:cc:cc:mime-version:mime-version:content-type:content-type:
         content-transfer-encoding:in-reply-to:in-reply-to:  references:references;
        bh=DpqXWMf3G5gZNv0d220PhTyu5Y62F7o91lIeySiWr5s=;
        b=nN0vx6QK48dPO4Egzyeff6bSZCydmc5t9l/MCVGNJ5N+8xVRvsBYVldAxIkB0eD7srP+oz
        RFruwsxuI94VGdN1hllSGkZulEzcbVB1l3cmgFVGzRGu7glNh814PPYy2qOq5Y0UG122j/
        OXedxb7Ky1ZIGwReu29lOjzxEMzT96w=
Date:   Wed, 8 Dec 2021 11:10:31 +0100
From:   Borislav Petkov <bp@alien8.de>
To:     Dave Hansen <dave.hansen@intel.com>
Cc:     Jarkko Sakkinen <jarkko@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
Message-ID: <YbCEl9kqhTz2iOY2@zn.tnic>
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org>
 <f25d95e6-e129-597b-5d93-d7264feae8b8@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <f25d95e6-e129-597b-5d93-d7264feae8b8@intel.com>
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Dec 07, 2021 at 11:36:50AM -0800, Dave Hansen wrote:
> On 11/16/21 8:21 AM, Jarkko Sakkinen wrote:
> > The amount of SGX memory on the system is determined by the BIOS and it
> > varies wildly between systems.  It can be from dozens of MB's on desktops
> > or VM's, up to many GB's on servers.  Just like for regular memory, it is
> > sometimes useful to know the amount of usable SGX memory in the system.
> > 
> > Introduce CONFIG_HAVE_ARCH_NODE_DEV_GROUP opt-in flag to expose an arch
> > specific attribute group, and add an attribute for the amount of SGX
> > memory in bytes to each NUMA node:
> > 
> > /sys/devices/system/node/nodeX/x86/sgx_total_bytes
> 
> There's some context missing here:

I'll say. "it is sometimes useful" is not the proper argumentation for
adding new stuff.

> This serves the same function for SGX memory as /proc/meminfo or
> /sys/devices/system/node/nodeX/meminfo does for normal RAM.  It
> enumerates how much physical SGX memory is present so that you can size
> enclaves on different systems.
> 
> This specific file (sgx_total_bytes) is needed today to help drive the
> SGX selftests.  The SGX selftests need to create overcommitted enclaves
> which are larger than the physical SGX memory on the system.  They
> currently use a CPUID-based approach which can diverge from the actual
> amount of SGX memory available.  This file ensures that the selftests
> can work efficiently and do not attempt stupid things like creating a
> 100,000 MB enclave on a system with 128 MB  of SGX memory.

Thanks, that's exactly what I was missing. Please stick that explanation
somewhere prominent.

> The nodeX/x86 directory is used because SGX is highly x86-specific.
> It's very unlikely that any other architecture (or even non-Intel x86
> vendor) will ever implement SGX.  It needs its own directory (as opposed
> to being in the nodeX/ "root") because this is expected to be the first
> of a few different things that need to get exported.  This avoids
> cluttering the root with several "sgx_*" files.
> 
> How many of these files will there be?  Just scanning /proc/meminfo,
> these are the no-brainers that we have for RAM, but we need for SGX:
> 
> MemTotal:       xxxx kB // sgx_total_bytes (this patch)
> MemFree:        yyyy kB // sgx_free_bytes
> SwapTotal:      zzzz kB // sgx_swapped_bytes
> 
> So, at *least* three.  I think we will eventually end up needing
> something more along the lines of a dozen.

Oh well, I guess.

I was going to propose to make that

	...nodeX/sgx/

but having it be arch-specific would mean that theoretically we could
add some non-sgx files there too, in the future, and if/when needed.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 5EB56C433F5
	for <linux-kernel@archiver.kernel.org>; Wed,  8 Dec 2021 19:38:38 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S239654AbhLHTmJ (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Wed, 8 Dec 2021 14:42:09 -0500
Received: from mga07.intel.com ([134.134.136.100]:44815 "EHLO mga07.intel.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S229759AbhLHTmI (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Wed, 8 Dec 2021 14:42:08 -0500
X-IronPort-AV: E=McAfee;i="6200,9189,10192"; a="301304548"
X-IronPort-AV: E=Sophos;i="5.88,190,1635231600"; 
   d="scan'208";a="301304548"
Received: from fmsmga005.fm.intel.com ([10.253.24.32])
  by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Dec 2021 11:38:20 -0800
X-IronPort-AV: E=Sophos;i="5.88,190,1635231600"; 
   d="scan'208";a="752007966"
Received: from eatci-mobl.amr.corp.intel.com (HELO [10.212.194.205]) ([10.212.194.205])
  by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Dec 2021 11:38:19 -0800
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
To:     Borislav Petkov <bp@alien8.de>
Cc:     Jarkko Sakkinen <jarkko@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org>
 <f25d95e6-e129-597b-5d93-d7264feae8b8@intel.com> <YbCEl9kqhTz2iOY2@zn.tnic>
From:   Dave Hansen <dave.hansen@intel.com>
Autocrypt: addr=dave.hansen@intel.com; keydata=
 xsFNBE6HMP0BEADIMA3XYkQfF3dwHlj58Yjsc4E5y5G67cfbt8dvaUq2fx1lR0K9h1bOI6fC
 oAiUXvGAOxPDsB/P6UEOISPpLl5IuYsSwAeZGkdQ5g6m1xq7AlDJQZddhr/1DC/nMVa/2BoY
 2UnKuZuSBu7lgOE193+7Uks3416N2hTkyKUSNkduyoZ9F5twiBhxPJwPtn/wnch6n5RsoXsb
 ygOEDxLEsSk/7eyFycjE+btUtAWZtx+HseyaGfqkZK0Z9bT1lsaHecmB203xShwCPT49Blxz
 VOab8668QpaEOdLGhtvrVYVK7x4skyT3nGWcgDCl5/Vp3TWA4K+IofwvXzX2ON/Mj7aQwf5W
 iC+3nWC7q0uxKwwsddJ0Nu+dpA/UORQWa1NiAftEoSpk5+nUUi0WE+5DRm0H+TXKBWMGNCFn
 c6+EKg5zQaa8KqymHcOrSXNPmzJuXvDQ8uj2J8XuzCZfK4uy1+YdIr0yyEMI7mdh4KX50LO1
 pmowEqDh7dLShTOif/7UtQYrzYq9cPnjU2ZW4qd5Qz2joSGTG9eCXLz5PRe5SqHxv6ljk8mb
 ApNuY7bOXO/A7T2j5RwXIlcmssqIjBcxsRRoIbpCwWWGjkYjzYCjgsNFL6rt4OL11OUF37wL
 QcTl7fbCGv53KfKPdYD5hcbguLKi/aCccJK18ZwNjFhqr4MliQARAQABzShEYXZpZCBDaHJp
 c3RvcGhlciBIYW5zZW4gPGRhdmVAc3I3MS5uZXQ+wsF7BBMBAgAlAhsDBgsJCAcDAgYVCAIJ
 CgsEFgIDAQIeAQIXgAUCTo3k0QIZAQAKCRBoNZUwcMmSsMO2D/421Xg8pimb9mPzM5N7khT0
 2MCnaGssU1T59YPE25kYdx2HntwdO0JA27Wn9xx5zYijOe6B21ufrvsyv42auCO85+oFJWfE
 K2R/IpLle09GDx5tcEmMAHX6KSxpHmGuJmUPibHVbfep2aCh9lKaDqQR07gXXWK5/yU1Dx0r
 VVFRaHTasp9fZ9AmY4K9/BSA3VkQ8v3OrxNty3OdsrmTTzO91YszpdbjjEFZK53zXy6tUD2d
 e1i0kBBS6NLAAsqEtneplz88T/v7MpLmpY30N9gQU3QyRC50jJ7LU9RazMjUQY1WohVsR56d
 ORqFxS8ChhyJs7BI34vQusYHDTp6PnZHUppb9WIzjeWlC7Jc8lSBDlEWodmqQQgp5+6AfhTD
 kDv1a+W5+ncq+Uo63WHRiCPuyt4di4/0zo28RVcjtzlGBZtmz2EIC3vUfmoZbO/Gn6EKbYAn
 rzz3iU/JWV8DwQ+sZSGu0HmvYMt6t5SmqWQo/hyHtA7uF5Wxtu1lCgolSQw4t49ZuOyOnQi5
 f8R3nE7lpVCSF1TT+h8kMvFPv3VG7KunyjHr3sEptYxQs4VRxqeirSuyBv1TyxT+LdTm6j4a
 mulOWf+YtFRAgIYyyN5YOepDEBv4LUM8Tz98lZiNMlFyRMNrsLV6Pv6SxhrMxbT6TNVS5D+6
 UorTLotDZKp5+M7BTQRUY85qARAAsgMW71BIXRgxjYNCYQ3Xs8k3TfAvQRbHccky50h99TUY
 sqdULbsb3KhmY29raw1bgmyM0a4DGS1YKN7qazCDsdQlxIJp9t2YYdBKXVRzPCCsfWe1dK/q
 66UVhRPP8EGZ4CmFYuPTxqGY+dGRInxCeap/xzbKdvmPm01Iw3YFjAE4PQ4hTMr/H76KoDbD
 cq62U50oKC83ca/PRRh2QqEqACvIH4BR7jueAZSPEDnzwxvVgzyeuhwqHY05QRK/wsKuhq7s
 UuYtmN92Fasbxbw2tbVLZfoidklikvZAmotg0dwcFTjSRGEg0Gr3p/xBzJWNavFZZ95Rj7Et
 db0lCt0HDSY5q4GMR+SrFbH+jzUY/ZqfGdZCBqo0cdPPp58krVgtIGR+ja2Mkva6ah94/oQN
 lnCOw3udS+Eb/aRcM6detZr7XOngvxsWolBrhwTQFT9D2NH6ryAuvKd6yyAFt3/e7r+HHtkU
 kOy27D7IpjngqP+b4EumELI/NxPgIqT69PQmo9IZaI/oRaKorYnDaZrMXViqDrFdD37XELwQ
 gmLoSm2VfbOYY7fap/AhPOgOYOSqg3/Nxcapv71yoBzRRxOc4FxmZ65mn+q3rEM27yRztBW9
 AnCKIc66T2i92HqXCw6AgoBJRjBkI3QnEkPgohQkZdAb8o9WGVKpfmZKbYBo4pEAEQEAAcLB
 XwQYAQIACQUCVGPOagIbDAAKCRBoNZUwcMmSsJeCEACCh7P/aaOLKWQxcnw47p4phIVR6pVL
 e4IEdR7Jf7ZL00s3vKSNT+nRqdl1ugJx9Ymsp8kXKMk9GSfmZpuMQB9c6io1qZc6nW/3TtvK
 pNGz7KPPtaDzvKA4S5tfrWPnDr7n15AU5vsIZvgMjU42gkbemkjJwP0B1RkifIK60yQqAAlT
 YZ14P0dIPdIPIlfEPiAWcg5BtLQU4Wg3cNQdpWrCJ1E3m/RIlXy/2Y3YOVVohfSy+4kvvYU3
 lXUdPb04UPw4VWwjcVZPg7cgR7Izion61bGHqVqURgSALt2yvHl7cr68NYoFkzbNsGsye9ft
 M9ozM23JSgMkRylPSXTeh5JIK9pz2+etco3AfLCKtaRVysjvpysukmWMTrx8QnI5Nn5MOlJj
 1Ov4/50JY9pXzgIDVSrgy6LYSMc4vKZ3QfCY7ipLRORyalFDF3j5AGCMRENJjHPD6O7bl3Xo
 4DzMID+8eucbXxKiNEbs21IqBZbbKdY1GkcEGTE7AnkA3Y6YB7I/j9mQ3hCgm5muJuhM/2Fr
 OPsw5tV/LmQ5GXH0JQ/TZXWygyRFyyI2FqNTx4WHqUn3yFj8rwTAU1tluRUYyeLy0ayUlKBH
 ybj0N71vWO936MqP6haFERzuPAIpxj2ezwu0xb1GjTk4ynna6h5GjnKgdfOWoRtoWndMZxbA
 z5cecg==
Message-ID: <32abd0a7-bc3c-f7a7-3330-8287ef05131c@intel.com>
Date:   Wed, 8 Dec 2021 11:38:11 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <YbCEl9kqhTz2iOY2@zn.tnic>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

I reworked the changelog quite a bit, addressing some of Borislav's
questions.  No code changes, though.

The result is below.  I've retained Greg's ack.  I'll stick this in
tip/x86/sgx if this looks OK to everyone.

---

From: Jarkko Sakkinen <jarkko@kernel.org>

== Problem ==

The amount of SGX memory on a system is determined by the BIOS and it
varies wildly between systems.  It can be as small as dozens of MB's
and as large as many GB's on servers.  Just like how applications need
to know how much regular RAM is available, enclave builders need to
know how much SGX memory an enclave can consume.

== Solution ==

Introduce a new sysfs file:

	/sys/devices/system/node/nodeX/x86/sgx_total_bytes

to enumerate the amount of SGX memory available in each NUMA node.
This serves the same function for SGX as /proc/meminfo or
/sys/devices/system/node/nodeX/meminfo does for normal RAM.

'sgx_total_bytes' is needed today to help drive the SGX selftests.
SGX-specific swap code is exercised by creating overcommitted enclaves
which are larger than the physical SGX memory on the system.  They
currently use a CPUID-based approach which can diverge from the actual
amount of SGX memory available.  'sgx_total_bytes' ensures that the
selftests can work efficiently and do not attempt stupid things like
creating a 100,000 MB enclave on a system with 128 MB of SGX memory.

== Implementation Details ==

Introduce CONFIG_HAVE_ARCH_NODE_DEV_GROUP opt-in flag to expose an
arch specific attribute group, and add an attribute for the amount of
SGX memory in bytes to each NUMA node:

== ABI Design Discussion ==

As opposed to the per-node ABI, a single, global ABI was considered.
However, this would prevent enclaves from being able to size
themselves so that they fit on a single NUMA node.  Essentially, a
single value would rule out NUMA optimizations for enclaves.

Create a new "x86/" directory inside each "nodeX/" sysfs directory.
'sgx_total_bytes' is expected to be the first of at least a few
sgx-specific files to be placed in the new directory.  Just scanning
/proc/meminfo, these are the no-brainers that we have for RAM, but we
need for SGX:

	MemTotal:       xxxx kB // sgx_total_bytes (implemented here)
	MemFree:        yyyy kB // sgx_free_bytes
	SwapTotal:      zzzz kB // sgx_swapped_bytes

So, at *least* three.  I think we will eventually end up needing
something more along the lines of a dozen.  A new directory (as
opposed to being in the nodeX/ "root") directory avoids cluttering the
root with several "sgx_*" files.

Place the new file in a new "nodeX/x86/" directory because SGX is
highly x86-specific.  It is very unlikely that any other architecture
(or even non-Intel x86 vendor) will ever implement SGX.  Using "sgx/"
as opposed to "x86/" was also considered.  But, there is a real chance
this can get used for other arch-specific purposes.

[ dhansen: rewrite changelog ]

Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Link: https://lkml.kernel.org/r/20211116162116.93081-2-jarkko@kernel.org
---
 Documentation/ABI/stable/sysfs-devices-node |  6 ++++++
 arch/Kconfig                                |  4 ++++
 arch/x86/Kconfig                            |  1 +
 arch/x86/kernel/cpu/sgx/main.c              | 20 ++++++++++++++++++++
 arch/x86/kernel/cpu/sgx/sgx.h               |  1 +
 drivers/base/node.c                         |  3 +++
 include/linux/numa.h                        |  4 ++++
 7 files changed, 39 insertions(+)

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 88F25C433EF
	for <linux-kernel@archiver.kernel.org>; Thu,  9 Dec 2021 12:08:23 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S236291AbhLIMLz (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Thu, 9 Dec 2021 07:11:55 -0500
Received: from mail.skyhub.de ([5.9.137.197]:48908 "EHLO mail.skyhub.de"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S233676AbhLIMLs (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 9 Dec 2021 07:11:48 -0500
Received: from zn.tnic (dslb-088-067-202-008.088.067.pools.vodafone-ip.de [88.67.202.8])
        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
        (No client certificate requested)
        by mail.skyhub.de (SuperMail on ZX Spectrum 128k) with ESMTPSA id 6EFB41EC04D3;
        Thu,  9 Dec 2021 13:08:09 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alien8.de; s=dkim;
        t=1639051689;
        h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
         to:to:cc:cc:mime-version:mime-version:content-type:content-type:
         content-transfer-encoding:in-reply-to:in-reply-to:  references:references;
        bh=dMxp/gGeAoEUpCT2uqegcuVBMsanPN/e8GbMQ6Bh64s=;
        b=VL92LyaeWAhSUNBriozC7zdOYcTYSsP4lck2rgTJwnI6oRHAWxayYBJnajv3sWKP6qST88
        lvDkL5wT+G7MYKy5IHzG1qIxEeN8Z2aMe3B2rq84GeZBBodrvPEyNaJJ80s7joTIXhPaGf
        dx8koIQ+ULNOC5oA1qfVbZ5uiUwpcUo=
Date:   Thu, 9 Dec 2021 13:08:11 +0100
From:   Borislav Petkov <bp@alien8.de>
To:     Dave Hansen <dave.hansen@intel.com>
Cc:     Jarkko Sakkinen <jarkko@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
Message-ID: <YbHxq9oGAdY09gVb@zn.tnic>
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org>
 <f25d95e6-e129-597b-5d93-d7264feae8b8@intel.com>
 <YbCEl9kqhTz2iOY2@zn.tnic>
 <32abd0a7-bc3c-f7a7-3330-8287ef05131c@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <32abd0a7-bc3c-f7a7-3330-8287ef05131c@intel.com>
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Dec 08, 2021 at 11:38:11AM -0800, Dave Hansen wrote:
> I reworked the changelog quite a bit, addressing some of Borislav's
> questions.  No code changes, though.
> 
> The result is below.  I've retained Greg's ack.  I'll stick this in
> tip/x86/sgx if this looks OK to everyone.

It does, thanks for taking the time!

Acked-by: Borislav Petkov <bp@suse.de>

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 59AD0C433EF
	for <linux-kernel@archiver.kernel.org>; Thu,  9 Dec 2021 15:35:40 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S239829AbhLIPjM (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Thu, 9 Dec 2021 10:39:12 -0500
Received: from Galois.linutronix.de ([193.142.43.55]:39850 "EHLO
        galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S232844AbhLIPjL (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 9 Dec 2021 10:39:11 -0500
Date:   Thu, 09 Dec 2021 15:35:35 -0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de;
        s=2020; t=1639064137;
        h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date:
         message-id:message-id:to:to:cc:cc:mime-version:mime-version:
         content-type:content-type:
         content-transfer-encoding:content-transfer-encoding:
         in-reply-to:in-reply-to:references:references;
        bh=Zu/1bGzgFrU72uxhtpp9m5ujWsKUY0BkquhPhHCCzQY=;
        b=02dJsQyl/Oy23ftn25TIiQZOoTLkZTwE9WRRL/P1TrUxns1d++fNHCKmaBSEIc1ptYeKLn
        DY4XhL4OHC2ptEsujMicWPk/6ew0YphnW7ydpgPfWQDOXAF9Y4jOzQxoWrMkL+qlggSlCR
        mU/Ckdj8lUAX623KMOqwOXQHuuzUTN/aoWtbplBXUb3AHonXXWnbyTZHQm2zAVYhQpalbm
        CXDs8ixN0PmLPwL3VzBtajyEug07Rnxj6cTNH0RLiQFbt4tXXAfZE3JxKn+4x/3fFFNgoX
        10u9DhVjw/7elccZ2wYovjxD8jTFJRGgdGDkxyhWdTPO8GJ1r4sXo5rjcwxm5g==
DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de;
        s=2020e; t=1639064137;
        h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date:
         message-id:message-id:to:to:cc:cc:mime-version:mime-version:
         content-type:content-type:
         content-transfer-encoding:content-transfer-encoding:
         in-reply-to:in-reply-to:references:references;
        bh=Zu/1bGzgFrU72uxhtpp9m5ujWsKUY0BkquhPhHCCzQY=;
        b=6u06CVyX3Q0SMIhmbfMsgReOhTXnE3dzKINk4puJCDmXEsFtSIPpWb8ah3UN3R4Rc1vUDo
        hvuOgNDyflxvTYBQ==
From:   "tip-bot2 for Jarkko Sakkinen" <tip-bot2@linutronix.de>
Sender: tip-bot2@linutronix.de
Reply-to: linux-kernel@vger.kernel.org
To:     linux-tip-commits@vger.kernel.org
Subject: [tip: x86/sgx] x86/sgx: Add an attribute for the amount of SGX memory
 in a NUMA node
Cc:     Jarkko Sakkinen <jarkko@kernel.org>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        "Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
        Borislav Petkov <bp@suse.de>, x86@kernel.org,
        linux-kernel@vger.kernel.org
In-Reply-To: <20211116162116.93081-2-jarkko@kernel.org>
References: <20211116162116.93081-2-jarkko@kernel.org>
MIME-Version: 1.0
Message-ID: <163906413588.11128.16644856184890792513.tip-bot2@tip-bot2>
Robot-ID: <tip-bot2@linutronix.de>
Robot-Unsubscribe: Contact <mailto:tglx@linutronix.de> to get blacklisted from these emails
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

The following commit has been merged into the x86/sgx branch of tip:

Commit-ID:     50468e4313355b161cac8a5155a45832995b7f25
Gitweb:        https://git.kernel.org/tip/50468e4313355b161cac8a5155a45832995b7f25
Author:        Jarkko Sakkinen <jarkko@kernel.org>
AuthorDate:    Tue, 16 Nov 2021 18:21:16 +02:00
Committer:     Dave Hansen <dave.hansen@linux.intel.com>
CommitterDate: Thu, 09 Dec 2021 07:02:22 -08:00

x86/sgx: Add an attribute for the amount of SGX memory in a NUMA node

== Problem ==

The amount of SGX memory on a system is determined by the BIOS and it
varies wildly between systems.  It can be as small as dozens of MB's
and as large as many GB's on servers.  Just like how applications need
to know how much regular RAM is available, enclave builders need to
know how much SGX memory an enclave can consume.

== Solution ==

Introduce a new sysfs file:

	/sys/devices/system/node/nodeX/x86/sgx_total_bytes

to enumerate the amount of SGX memory available in each NUMA node.
This serves the same function for SGX as /proc/meminfo or
/sys/devices/system/node/nodeX/meminfo does for normal RAM.

'sgx_total_bytes' is needed today to help drive the SGX selftests.
SGX-specific swap code is exercised by creating overcommitted enclaves
which are larger than the physical SGX memory on the system.  They
currently use a CPUID-based approach which can diverge from the actual
amount of SGX memory available.  'sgx_total_bytes' ensures that the
selftests can work efficiently and do not attempt stupid things like
creating a 100,000 MB enclave on a system with 128 MB of SGX memory.

== Implementation Details ==

Introduce CONFIG_HAVE_ARCH_NODE_DEV_GROUP opt-in flag to expose an
arch specific attribute group, and add an attribute for the amount of
SGX memory in bytes to each NUMA node:

== ABI Design Discussion ==

As opposed to the per-node ABI, a single, global ABI was considered.
However, this would prevent enclaves from being able to size
themselves so that they fit on a single NUMA node.  Essentially, a
single value would rule out NUMA optimizations for enclaves.

Create a new "x86/" directory inside each "nodeX/" sysfs directory.
'sgx_total_bytes' is expected to be the first of at least a few
sgx-specific files to be placed in the new directory.  Just scanning
/proc/meminfo, these are the no-brainers that we have for RAM, but we
need for SGX:

	MemTotal:       xxxx kB // sgx_total_bytes (implemented here)
	MemFree:        yyyy kB // sgx_free_bytes
	SwapTotal:      zzzz kB // sgx_swapped_bytes

So, at *least* three.  I think we will eventually end up needing
something more along the lines of a dozen.  A new directory (as
opposed to being in the nodeX/ "root") directory avoids cluttering the
root with several "sgx_*" files.

Place the new file in a new "nodeX/x86/" directory because SGX is
highly x86-specific.  It is very unlikely that any other architecture
(or even non-Intel x86 vendor) will ever implement SGX.  Using "sgx/"
as opposed to "x86/" was also considered.  But, there is a real chance
this can get used for other arch-specific purposes.

[ dhansen: rewrite changelog ]

Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20211116162116.93081-2-jarkko@kernel.org
---
 Documentation/ABI/stable/sysfs-devices-node |  6 ++++++-
 arch/Kconfig                                |  4 ++++-
 arch/x86/Kconfig                            |  1 +-
 arch/x86/kernel/cpu/sgx/main.c              | 20 ++++++++++++++++++++-
 arch/x86/kernel/cpu/sgx/sgx.h               |  1 +-
 drivers/base/node.c                         |  3 +++-
 include/linux/numa.h                        |  4 ++++-
 7 files changed, 39 insertions(+)

diff --git a/Documentation/ABI/stable/sysfs-devices-node b/Documentation/ABI/stable/sysfs-devices-node
index 484fc04..8db67aa 100644
--- a/Documentation/ABI/stable/sysfs-devices-node
+++ b/Documentation/ABI/stable/sysfs-devices-node
@@ -176,3 +176,9 @@ Contact:	Keith Busch <keith.busch@intel.com>
 Description:
 		The cache write policy: 0 for write-back, 1 for write-through,
 		other or unknown.
+
+What:		/sys/devices/system/node/nodeX/x86/sgx_total_bytes
+Date:		November 2021
+Contact:	Jarkko Sakkinen <jarkko@kernel.org>
+Description:
+		The total amount of SGX physical memory in bytes.
diff --git a/arch/Kconfig b/arch/Kconfig
index 26b8ed1..0a9dadb 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1302,6 +1302,10 @@ config ARCH_HAS_PARANOID_L1D_FLUSH
 config DYNAMIC_SIGFRAME
 	bool
 
+# Select, if arch has a named attribute group bound to NUMA device nodes.
+config HAVE_ARCH_NODE_DEV_GROUP
+	bool
+
 source "kernel/gcov/Kconfig"
 
 source "scripts/gcc-plugins/Kconfig"
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index b9281fa..f2b699d 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -269,6 +269,7 @@ config X86
 	select HAVE_ARCH_KCSAN			if X86_64
 	select X86_FEATURE_NAMES		if PROC_FS
 	select PROC_PID_ARCH_STATUS		if PROC_FS
+	select HAVE_ARCH_NODE_DEV_GROUP		if X86_SGX
 	imply IMA_SECURE_AND_OR_TRUSTED_BOOT    if EFI
 
 config INSTRUCTION_DECODER
diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 6036328..2857a49 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -825,9 +825,11 @@ static bool __init sgx_page_cache_init(void)
 			INIT_LIST_HEAD(&sgx_numa_nodes[nid].free_page_list);
 			INIT_LIST_HEAD(&sgx_numa_nodes[nid].sgx_poison_page_list);
 			node_set(nid, sgx_numa_mask);
+			sgx_numa_nodes[nid].size = 0;
 		}
 
 		sgx_epc_sections[i].node =  &sgx_numa_nodes[nid];
+		sgx_numa_nodes[nid].size += size;
 
 		sgx_nr_epc_sections++;
 	}
@@ -901,6 +903,24 @@ int sgx_set_attribute(unsigned long *allowed_attributes,
 }
 EXPORT_SYMBOL_GPL(sgx_set_attribute);
 
+#ifdef CONFIG_NUMA
+static ssize_t sgx_total_bytes_show(struct device *dev, struct device_attribute *attr, char *buf)
+{
+	return sysfs_emit(buf, "%lu\n", sgx_numa_nodes[dev->id].size);
+}
+static DEVICE_ATTR_RO(sgx_total_bytes);
+
+static struct attribute *arch_node_dev_attrs[] = {
+	&dev_attr_sgx_total_bytes.attr,
+	NULL,
+};
+
+const struct attribute_group arch_node_dev_group = {
+	.name = "x86",
+	.attrs = arch_node_dev_attrs,
+};
+#endif /* CONFIG_NUMA */
+
 static int __init sgx_init(void)
 {
 	int ret;
diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
index 9ec3136..0f17def 100644
--- a/arch/x86/kernel/cpu/sgx/sgx.h
+++ b/arch/x86/kernel/cpu/sgx/sgx.h
@@ -44,6 +44,7 @@ struct sgx_epc_page {
 struct sgx_numa_node {
 	struct list_head free_page_list;
 	struct list_head sgx_poison_page_list;
+	unsigned long size;
 	spinlock_t lock;
 };
 
diff --git a/drivers/base/node.c b/drivers/base/node.c
index b5a4ba1..87acc47 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -581,6 +581,9 @@ static const struct attribute_group node_dev_group = {
 
 static const struct attribute_group *node_dev_groups[] = {
 	&node_dev_group,
+#ifdef CONFIG_HAVE_ARCH_NODE_DEV_GROUP
+	&arch_node_dev_group,
+#endif
 	NULL
 };
 
diff --git a/include/linux/numa.h b/include/linux/numa.h
index cb44cfe..59df211 100644
--- a/include/linux/numa.h
+++ b/include/linux/numa.h
@@ -58,4 +58,8 @@ static inline int phys_to_target_node(u64 start)
 }
 #endif
 
+#ifdef CONFIG_HAVE_ARCH_NODE_DEV_GROUP
+extern const struct attribute_group arch_node_dev_group;
+#endif
+
 #endif /* _LINUX_NUMA_H */

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 1CA11C433EF
	for <linux-kernel@archiver.kernel.org>; Sat, 11 Dec 2021 15:36:44 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S231347AbhLKPgl (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Sat, 11 Dec 2021 10:36:41 -0500
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55634 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S229836AbhLKPgk (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Sat, 11 Dec 2021 10:36:40 -0500
Received: from sin.source.kernel.org (sin.source.kernel.org [IPv6:2604:1380:40e1:4800::1])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7622C061714;
        Sat, 11 Dec 2021 07:36:39 -0800 (PST)
Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140])
        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
        (No client certificate requested)
        by sin.source.kernel.org (Postfix) with ESMTPS id 63688CE094C;
        Sat, 11 Dec 2021 15:36:37 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id F3763C004DD;
        Sat, 11 Dec 2021 15:36:34 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
        s=k20201202; t=1639236995;
        bh=cF1A5jbPqge4rANTbCPUUfKutHCzr7QR7uCHRv4qXi0=;
        h=Subject:From:To:Cc:Date:In-Reply-To:References:From;
        b=j6Qk1xF7jgVkoGnjV3SqPCZ5dkkRFjKppH4Xd4AlQ13H7eaZOIt+UV/oS6a5fKGED
         OFqSAuEXDMjRTzTEV2x3u0tO7nF1saG4+Jx6Hv28l8hGImmW3W0jyuCCAw1FFzR53n
         yMPPm6f29MK9zT+NN6Ma7WxrJogXAKOQvpNJqpETjSAuMSLQME6/T8kqYAzVkZdCxq
         yuu/XjMdExdC7ABaEWKd2LZypFCLW9B/PsjSs1caCO67hvYV1WCMYvufcCOOJpHnVw
         ZZbMQXMde5/4l0d15pvXyhaEX/tGRB8ySSOD0fvEMGucOoiuwRje3elVaOJe+d7cU5
         sw+ydLt/q0dIA==
Message-ID: <d6ec8429763db7ddba42d9ff658a929b274f2711.camel@kernel.org>
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
From:   Jarkko Sakkinen <jarkko@kernel.org>
To:     Dave Hansen <dave.hansen@intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>
Cc:     reinette.chatre@intel.com, linux-kernel@vger.kernel.org,
        linux-sgx@vger.kernel.org
Date:   Sat, 11 Dec 2021 17:36:31 +0200
In-Reply-To: <f25d95e6-e129-597b-5d93-d7264feae8b8@intel.com>
References: <20211116162116.93081-1-jarkko@kernel.org>
         <20211116162116.93081-2-jarkko@kernel.org>
         <f25d95e6-e129-597b-5d93-d7264feae8b8@intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.42.2 
MIME-Version: 1.0
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, 2021-12-07 at 11:36 -0800, Dave Hansen wrote:
> On 11/16/21 8:21 AM, Jarkko Sakkinen wrote:
> > The amount of SGX memory on the system is determined by the BIOS and it
> > varies wildly between systems.  It can be from dozens of MB's on deskto=
ps
> > or VM's, up to many GB's on servers.  Just like for regular memory, it =
is
> > sometimes useful to know the amount of usable SGX memory in the system.
> >=20
> > Introduce CONFIG_HAVE_ARCH_NODE_DEV_GROUP opt-in flag to expose an arch
> > specific attribute group, and add an attribute for the amount of SGX
> > memory in bytes to each NUMA node:
> >=20
> > /sys/devices/system/node/nodeX/x86/sgx_total_bytes
>=20
> There's some context missing here:
>=20
> This serves the same function for SGX memory as /proc/meminfo or
> /sys/devices/system/node/nodeX/meminfo does for normal RAM.  It
> enumerates how much physical SGX memory is present so that you can size
> enclaves on different systems.
>=20
> This specific file (sgx_total_bytes) is needed today to help drive the
> SGX selftests.  The SGX selftests need to create overcommitted enclaves
> which are larger than the physical SGX memory on the system.  They
> currently use a CPUID-based approach which can diverge from the actual
> amount of SGX memory available.  This file ensures that the selftests
> can work efficiently and do not attempt stupid things like creating a
> 100,000 MB enclave on a system with 128 MB  of SGX memory.
>=20
> The nodeX/x86 directory is used because SGX is highly x86-specific.
> It's very unlikely that any other architecture (or even non-Intel x86
> vendor) will ever implement SGX.  It needs its own directory (as opposed
> to being in the nodeX/ "root") because this is expected to be the first
> of a few different things that need to get exported.  This avoids
> cluttering the root with several "sgx_*" files.
>=20
> How many of these files will there be?  Just scanning /proc/meminfo,
> these are the no-brainers that we have for RAM, but we need for SGX:
>=20
> MemTotal:       xxxx kB // sgx_total_bytes (this patch)
> MemFree:        yyyy kB // sgx_free_bytes
> SwapTotal:      zzzz kB // sgx_swapped_bytes
>=20
> So, at *least* three.  I think we will eventually end up needing
> something more along the lines of a dozen.

These three I've had in mind for the moment. The latter two will be
trivial to add now that we have pattern how to add the sysfs attribute
in the correct way.

/Jarkko

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id D4041C433F5
	for <linux-kernel@archiver.kernel.org>; Sat, 11 Dec 2021 15:37:57 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S231360AbhLKPh4 (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Sat, 11 Dec 2021 10:37:56 -0500
Received: from ams.source.kernel.org ([145.40.68.75]:54920 "EHLO
        ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S230201AbhLKPhw (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Sat, 11 Dec 2021 10:37:52 -0500
Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140])
        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
        (No client certificate requested)
        by ams.source.kernel.org (Postfix) with ESMTPS id 8A495B80AD2;
        Sat, 11 Dec 2021 15:37:51 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id BB725C004DD;
        Sat, 11 Dec 2021 15:37:49 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
        s=k20201202; t=1639237070;
        bh=ezOXs6CcJHDlycNj8lmQU7rq1j6Bz02x9rKOsPw8OgE=;
        h=Subject:From:To:Cc:Date:In-Reply-To:References:From;
        b=A7oGnlOMw//1MxT92AF6CRcLDc+qYZ13y96+6IjTymZPMxDZ0jMx6GOn6AYu+gb+y
         oG0Lxs5rXlOCCJDACsCPts5OTscst3mt8f5Eismwsji2y0F2kh/557Yi5AwVYIaIXO
         uPzqJCZ8mWlIAqvzlW/0HvwcltgjPA5Nl23gN8TioSLo7RLewimtpup9wJqaSR6N73
         2upqBdhuMfnK8Eqo10fD4k3Zu4HJp5wOaMyvI8mLI9Z68IWgjdzSuaz7NFwsrRB0OB
         k/+TSiH9cXpUcFRiMSnG9lGbYhRLV2lmo7DG3yAvQ1lPPQO/PSyMBPivTxDPzNVhim
         oqOI9qzBm0jYg==
Message-ID: <233a2fa4355b060361b9e9a6dc98b3175d1143fb.camel@kernel.org>
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
From:   Jarkko Sakkinen <jarkko@kernel.org>
To:     Dave Hansen <dave.hansen@intel.com>, Borislav Petkov <bp@alien8.de>
Cc:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
Date:   Sat, 11 Dec 2021 17:37:46 +0200
In-Reply-To: <32abd0a7-bc3c-f7a7-3330-8287ef05131c@intel.com>
References: <20211116162116.93081-1-jarkko@kernel.org>
         <20211116162116.93081-2-jarkko@kernel.org>
         <f25d95e6-e129-597b-5d93-d7264feae8b8@intel.com> <YbCEl9kqhTz2iOY2@zn.tnic>
         <32abd0a7-bc3c-f7a7-3330-8287ef05131c@intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.42.2 
MIME-Version: 1.0
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, 2021-12-08 at 11:38 -0800, Dave Hansen wrote:
> I reworked the changelog quite a bit, addressing some of Borislav's
> questions.  No code changes, though.
>=20
> The result is below.  I've retained Greg's ack.  I'll stick this in
> tip/x86/sgx if this looks OK to everyone.
>=20
> ---
>=20
> From: Jarkko Sakkinen <jarkko@kernel.org>
>=20
> =3D=3D Problem =3D=3D
>=20
> The amount of SGX memory on a system is determined by the BIOS and it
> varies wildly between systems.  It can be as small as dozens of MB's
> and as large as many GB's on servers.  Just like how applications need
> to know how much regular RAM is available, enclave builders need to
> know how much SGX memory an enclave can consume.
>=20
> =3D=3D Solution =3D=3D
>=20
> Introduce a new sysfs file:
>=20
> 	/sys/devices/system/node/nodeX/x86/sgx_total_bytes
>=20
> to enumerate the amount of SGX memory available in each NUMA node.
> This serves the same function for SGX as /proc/meminfo or
> /sys/devices/system/node/nodeX/meminfo does for normal RAM.
>=20
> 'sgx_total_bytes' is needed today to help drive the SGX selftests.
> SGX-specific swap code is exercised by creating overcommitted enclaves
> which are larger than the physical SGX memory on the system.  They
> currently use a CPUID-based approach which can diverge from the actual
> amount of SGX memory available.  'sgx_total_bytes' ensures that the
> selftests can work efficiently and do not attempt stupid things like
> creating a 100,000 MB enclave on a system with 128 MB of SGX memory.
>=20
> =3D=3D Implementation Details =3D=3D
>=20
> Introduce CONFIG_HAVE_ARCH_NODE_DEV_GROUP opt-in flag to expose an
> arch specific attribute group, and add an attribute for the amount of
> SGX memory in bytes to each NUMA node:
>=20
> =3D=3D ABI Design Discussion =3D=3D
>=20
> As opposed to the per-node ABI, a single, global ABI was considered.
> However, this would prevent enclaves from being able to size
> themselves so that they fit on a single NUMA node.  Essentially, a
> single value would rule out NUMA optimizations for enclaves.
>=20
> Create a new "x86/" directory inside each "nodeX/" sysfs directory.
> 'sgx_total_bytes' is expected to be the first of at least a few
> sgx-specific files to be placed in the new directory.  Just scanning
> /proc/meminfo, these are the no-brainers that we have for RAM, but we
> need for SGX:
>=20
> 	MemTotal:       xxxx kB // sgx_total_bytes (implemented here)
> 	MemFree:        yyyy kB // sgx_free_bytes
> 	SwapTotal:      zzzz kB // sgx_swapped_bytes
>=20
> So, at *least* three.  I think we will eventually end up needing
> something more along the lines of a dozen.  A new directory (as
> opposed to being in the nodeX/ "root") directory avoids cluttering the
> root with several "sgx_*" files.
>=20
> Place the new file in a new "nodeX/x86/" directory because SGX is
> highly x86-specific.  It is very unlikely that any other architecture
> (or even non-Intel x86 vendor) will ever implement SGX.  Using "sgx/"
> as opposed to "x86/" was also considered.  But, there is a real chance
> this can get used for other arch-specific purposes.
>=20
> [ dhansen: rewrite changelog ]
>=20
> Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Link: https://lkml.kernel.org/r/20211116162116.93081-2-jarkko@kernel.org
> ---
>  Documentation/ABI/stable/sysfs-devices-node |  6 ++++++
>  arch/Kconfig                                |  4 ++++
>  arch/x86/Kconfig                            |  1 +
>  arch/x86/kernel/cpu/sgx/main.c              | 20 ++++++++++++++++++++
>  arch/x86/kernel/cpu/sgx/sgx.h               |  1 +
>  drivers/base/node.c                         |  3 +++
>  include/linux/numa.h                        |  4 ++++
>  7 files changed, 39 insertions(+)

Thank you! Looks good to me.

/Jarkko


From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 67565C433F5
	for <linux-kernel@archiver.kernel.org>; Fri, 17 Dec 2021 19:12:14 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S240581AbhLQTMN (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Fri, 17 Dec 2021 14:12:13 -0500
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53760 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S230326AbhLQTMM (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Fri, 17 Dec 2021 14:12:12 -0500
Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17AF5C061574;
        Fri, 17 Dec 2021 11:12:12 -0800 (PST)
Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140])
        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
        (No client certificate requested)
        by dfw.source.kernel.org (Postfix) with ESMTPS id B574B62384;
        Fri, 17 Dec 2021 19:12:11 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id CE00FC36AE2;
        Fri, 17 Dec 2021 19:12:08 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
        s=k20201202; t=1639768331;
        bh=abrj6yZx1CS+4y6rWVn3ZtFqCwKgtGVKaalJpu0TSPw=;
        h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
        b=FFvjRiN6uSj8POfLPxfk9KGf6ZdjpD+TnGDpp9MGbIAz1LAjqAdYKmLTfC6BpGnj1
         oTYHkkU5U2duNyVqLi8UivDcqe/kPxza/hz0G+mUYZhXwZsPug04xNRWFzTeR4twbS
         7fb/f+UOonucgP8HSJhyuHK014XzSmjZ8FEcP2lSr/7Rl/TKmIwyYXcGd/ev2nCiv3
         u/bVR8z2NDMcyAqUFfQnm525Ov6KYtqfacvhvi0rzWBy7y5jMxlcOHVIz5V5/aoxyd
         LSHihi8FSO5U3l0UGv7gyxXbej+kSvpiQcE5ltAZRIn9tUWNmJ1SyOxj8qy+nxDh0t
         0h3gDLGhjYoug==
Date:   Fri, 17 Dec 2021 12:12:06 -0700
From:   Nathan Chancellor <nathan@kernel.org>
To:     Jarkko Sakkinen <jarkko@kernel.org>
Cc:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
Message-ID: <YbzhBrimHGGpddDM@archlinux-ax161>
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20211116162116.93081-2-jarkko@kernel.org>
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Jarkko,

On Tue, Nov 16, 2021 at 06:21:16PM +0200, Jarkko Sakkinen wrote:
> The amount of SGX memory on the system is determined by the BIOS and it
> varies wildly between systems.  It can be from dozens of MB's on desktops
> or VM's, up to many GB's on servers.  Just like for regular memory, it is
> sometimes useful to know the amount of usable SGX memory in the system.
> 
> Introduce CONFIG_HAVE_ARCH_NODE_DEV_GROUP opt-in flag to expose an arch
> specific attribute group, and add an attribute for the amount of SGX
> memory in bytes to each NUMA node:
> 
> /sys/devices/system/node/nodeX/x86/sgx_total_bytes
> 
> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>

Apologies if this has already been reported or fixed, I haven't seen
anything from a full search of lore.kernel.org.

With this patch in -next as commit 50468e431335 ("x86/sgx: Add an
attribute for the amount of SGX memory in a NUMA node"), this sysfs node
causes an OOPS for me on at least one of my test systems (Intel based):

# cat /sys/devices/system/node/node0/x86/sgx_total_bytes
fish: Job 1, 'cat /sys/devices/system/node/no…' terminated by signal SIGKILL (Forced quit)

# dmesg
[   56.956995] BUG: kernel NULL pointer dereference, address: 0000000000000020
[   56.957003] #PF: supervisor read access in kernel mode
[   56.957006] #PF: error_code(0x0000) - not-present page
[   56.957009] PGD 0 P4D 0
[   56.957013] Oops: 0000 [#2] PREEMPT SMP PTI
[   56.957017] CPU: 1 PID: 866 Comm: cat Tainted: G      D           5.16.0-rc5-next-20211217-debug #1 15ae5b0f28a4b9b6343440ee595affa8e1b5cf57
[   56.957022] Hardware name: ASUSTeK COMPUTER INC. Q302LA/Q302LA, BIOS Q302LA.203 05/15/2014
[   56.957024] RIP: 0010:sgx_total_bytes_show+0x28/0x40
[   56.957032] Code: 66 90 0f 1f 44 00 00 49 89 f8 48 89 d7 48 c7 c6 5f b1 52 a4 41 8b 80 98 02 00 00 48 8d 04 40 48 c1 e0 04 48 03 05 38 b1 3c 02 <48> 8b 50 20 e8 ff ee 3b 00 48 98 31 d2 89 d6 89 d7 41 89 d0 c3 0f
[   56.957035] RSP: 0018:ffffb79c81447d60 EFLAGS: 00010246
[   56.957039] RAX: 0000000000000000 RBX: ffffffffa4c30ec0 RCX: 0000000000000000
[   56.957041] RDX: ffff95d502b95000 RSI: ffffffffa452b15f RDI: ffff95d502b95000
[   56.957044] RBP: ffffffffa41567c0 R08: ffff95d5002a0400 R09: 0000000000000000
[   56.957046] R10: 0000000000000000 R11: 0000000000000000 R12: ffffb79c81447e28
[   56.957048] R13: ffffb79c81447e00 R14: ffff95d5041247a8 R15: 0000000000000001
[   56.957051] FS:  00007f4ab25af600(0000) GS:ffff95d616e80000(0000) knlGS:0000000000000000
[   56.957054] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   56.957056] CR2: 0000000000000020 CR3: 00000001029e8005 CR4: 00000000001706e0
[   56.957060] Call Trace:
[   56.957062]  <TASK>
[   56.957064]  dev_attr_show+0x19/0x40
[   56.957072]  sysfs_kf_seq_show+0xa1/0x120
[   56.957080]  seq_read_iter+0x12e/0x4c0
[   56.957085]  new_sync_read+0x159/0x1f0
[   56.957093]  vfs_read+0xff/0x1a0
[   56.957099]  ksys_read+0x67/0xf0
[   56.957105]  do_syscall_64+0x5c/0x90
[   56.957110]  ? syscall_exit_to_user_mode+0x23/0x50
[   56.957116]  ? exc_page_fault+0x72/0x180
[   56.957121]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[   56.957128] RIP: 0033:0x7f4ab24d1862
[   56.957131] Code: c0 e9 b2 fe ff ff 50 48 8d 3d 5a 29 0a 00 e8 55 e4 01 00 0f 1f 44 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 0f 05 <48> 3d 00 f0 ff ff 77 56 c3 0f 1f 44 00 00 48 83 ec 28 48 89 54 24
[   56.957134] RSP: 002b:00007ffff14dac78 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[   56.957137] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007f4ab24d1862
[   56.957140] RDX: 0000000000020000 RSI: 00007f4ab20d8000 RDI: 0000000000000003
[   56.957142] RBP: 00007f4ab20d8000 R08: 00007f4ab20d7010 R09: 0000000000000000
[   56.957144] R10: 0000000000000022 R11: 0000000000000246 R12: 0000000000020000
[   56.957146] R13: 0000000000000003 R14: 0000000000000000 R15: 00007ffff14daf18
[   56.957150]  </TASK>
[   56.957152] Modules linked in: 8021q garp mrp stp llc snd_hda_codec_hdmi ccm
algif_aead cbc x86_pkg_temp_thermal des_generic intel_powerclamp libdes
coretemp ecb kvm_intel algif_skcipher kvm ax88796b irqbypass cmac
crct10dif_pclmul i915 crc32_pclmul md4 ghash_clmulni_intel algif_hash
snd_hda_codec_realtek af_alg snd_hda_codec_generic ledtrig_audio uvcvideo
asus_nb_wmi aesni_intel snd_hda_intel intel_spi_platform i2c_algo_bit iwlmvm
videobuf2_vmalloc asus_wmi snd_intel_dspcfg ttm snd_intel_sdw_acpi crypto_simd
joydev ak8975 intel_spi iTCO_wdt btusb mousedev spi_nor sparse_keymap
intel_pmc_bxt at24 cryptd videobuf2_memops iTCO_vendor_support mac80211
hid_multitouch mtd platform_profile btrtl intel_rapl_msr mei_hdcp snd_hda_codec
btbcm rapl libarc4 videobuf2_v4l2 drm_kms_helper intel_cstate
processor_thermal_device_pci_legacy asix intel_uncore syscopyarea selftests
btintel snd_hda_core videobuf2_common pcspkr iwlwifi snd_hwdep
processor_thermal_device vfat btmtk i2c_i801 sysfillrect usbnet
[   56.957229]  snd_pcm processor_thermal_rfim psmouse fat sysimgblt i2c_smbus
bluetooth videodev processor_thermal_mbox snd_timer processor_thermal_rapl
fb_sys_fops mii cfg80211 mei_me usbhid snd ecdh_generic mdio_devres mc
inv_mpu6050_i2c intel_rapl_common cec crc16 int340x_thermal_zone libphy lpc_ich
rfkill intel_gtt mei soundcore intel_soc_dts_iosf inv_mpu6050 acpi_als wmi
dell_smo8800 industrialio_triggered_buffer int3400_thermal video i2c_mux
kfifo_buf soc_button_array industrialio acpi_thermal_rel asus_wireless mac_hid
drm pkcs8_key_parser fuse bpf_preload ip_tables x_tables xfs libcrc32c
crc32c_generic serio_raw atkbd libps2 i8042 crc32c_intel xhci_pci
xhci_pci_renesas serio
[   56.957286] CR2: 0000000000000020
[   56.957289] ---[ end trace 0000000000000000 ]---
[   56.957291] RIP: 0010:sgx_total_bytes_show+0x28/0x40
[   56.957296] Code: 66 90 0f 1f 44 00 00 49 89 f8 48 89 d7 48 c7 c6 5f b1 52 a4 41 8b 80 98 02 00 00 48 8d 04 40 48 c1 e0 04 48 03 05 38 b1 3c 02 <48> 8b 50 20 e8 ff ee 3b 00 48 98 31 d2 89 d6 89 d7 41 89 d0 c3 0f
[   56.957299] RSP: 0018:ffffb79c8118fd58 EFLAGS: 00010246
[   56.957301] RAX: 0000000000000000 RBX: ffffffffa4c30ec0 RCX: 0000000000000000
[   56.957304] RDX: ffff95d50403d000 RSI: ffffffffa452b15f RDI: ffff95d50403d000
[   56.957306] RBP: ffffffffa41567c0 R08: ffff95d5002a0400 R09: 0000000000000000
[   56.957308] R10: 0000000000000000 R11: 0000000000000000 R12: ffffb79c8118fe20
[   56.957310] R13: ffffb79c8118fdf8 R14: ffff95d503fd0550 R15: 0000000000000001
[   56.957312] FS:  00007f4ab25af600(0000) GS:ffff95d616e80000(0000) knlGS:0000000000000000
[   56.957315] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   56.957317] CR2: 0000000000000020 CR3: 00000001029e8005 CR4: 00000000001706e0

If I can provide any further information or testing, let me know!

Cheers,
Nathan

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id B7D6CC433F5
	for <linux-kernel@archiver.kernel.org>; Fri, 17 Dec 2021 21:17:47 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S241024AbhLQVRr (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Fri, 17 Dec 2021 16:17:47 -0500
Received: from mga11.intel.com ([192.55.52.93]:6486 "EHLO mga11.intel.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S237156AbhLQVRp (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Fri, 17 Dec 2021 16:17:45 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
  d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
  t=1639775865; x=1671311865;
  h=subject:to:cc:references:from:message-id:date:
   mime-version:in-reply-to:content-transfer-encoding;
  bh=0MGr81aqkNa4jKgg4hdu/6la1f3mjEWrRC+B7fBMBug=;
  b=fcdskudsWqc1h0PTqvNUHMpLoUmtmmD5GmeIsIO2bqcI0rQrHlYOcii6
   rBlOVQFuLYVHbSXxB8VczIfA0533ElSEhure4pNCY7Ft8zzw+tDArPcO4
   7+lZtWrjeZ1XtjeRZoN/YK4Oqz+WT4uJPLf1FcDfzgjA0qtqYtD6d63LT
   F5lJeva2kmqT0qJ83R4KYr0Rmt30VmUv3vSpvCSdRRZLyShK1RwomsLY7
   zMLbZ2i6dMPrro85NxSZPyQfOWmaO0TeY37hUskWl+RTMmL0h+BZRQfoY
   djnKuoVUP9QFEnhsIuu3rF4SCk/mikmF68dymhUjNYilUkoshksd0JwJH
   A==;
X-IronPort-AV: E=McAfee;i="6200,9189,10201"; a="237376125"
X-IronPort-AV: E=Sophos;i="5.88,214,1635231600"; 
   d="scan'208";a="237376125"
Received: from orsmga005.jf.intel.com ([10.7.209.41])
  by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Dec 2021 13:17:45 -0800
X-IronPort-AV: E=Sophos;i="5.88,214,1635231600"; 
   d="scan'208";a="683522144"
Received: from mkundu-mobl1.amr.corp.intel.com (HELO [10.212.216.75]) ([10.212.216.75])
  by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Dec 2021 13:17:45 -0800
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
To:     Nathan Chancellor <nathan@kernel.org>,
        Jarkko Sakkinen <jarkko@kernel.org>
Cc:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org> <YbzhBrimHGGpddDM@archlinux-ax161>
From:   Dave Hansen <dave.hansen@intel.com>
Autocrypt: addr=dave.hansen@intel.com; keydata=
 xsFNBE6HMP0BEADIMA3XYkQfF3dwHlj58Yjsc4E5y5G67cfbt8dvaUq2fx1lR0K9h1bOI6fC
 oAiUXvGAOxPDsB/P6UEOISPpLl5IuYsSwAeZGkdQ5g6m1xq7AlDJQZddhr/1DC/nMVa/2BoY
 2UnKuZuSBu7lgOE193+7Uks3416N2hTkyKUSNkduyoZ9F5twiBhxPJwPtn/wnch6n5RsoXsb
 ygOEDxLEsSk/7eyFycjE+btUtAWZtx+HseyaGfqkZK0Z9bT1lsaHecmB203xShwCPT49Blxz
 VOab8668QpaEOdLGhtvrVYVK7x4skyT3nGWcgDCl5/Vp3TWA4K+IofwvXzX2ON/Mj7aQwf5W
 iC+3nWC7q0uxKwwsddJ0Nu+dpA/UORQWa1NiAftEoSpk5+nUUi0WE+5DRm0H+TXKBWMGNCFn
 c6+EKg5zQaa8KqymHcOrSXNPmzJuXvDQ8uj2J8XuzCZfK4uy1+YdIr0yyEMI7mdh4KX50LO1
 pmowEqDh7dLShTOif/7UtQYrzYq9cPnjU2ZW4qd5Qz2joSGTG9eCXLz5PRe5SqHxv6ljk8mb
 ApNuY7bOXO/A7T2j5RwXIlcmssqIjBcxsRRoIbpCwWWGjkYjzYCjgsNFL6rt4OL11OUF37wL
 QcTl7fbCGv53KfKPdYD5hcbguLKi/aCccJK18ZwNjFhqr4MliQARAQABzShEYXZpZCBDaHJp
 c3RvcGhlciBIYW5zZW4gPGRhdmVAc3I3MS5uZXQ+wsF7BBMBAgAlAhsDBgsJCAcDAgYVCAIJ
 CgsEFgIDAQIeAQIXgAUCTo3k0QIZAQAKCRBoNZUwcMmSsMO2D/421Xg8pimb9mPzM5N7khT0
 2MCnaGssU1T59YPE25kYdx2HntwdO0JA27Wn9xx5zYijOe6B21ufrvsyv42auCO85+oFJWfE
 K2R/IpLle09GDx5tcEmMAHX6KSxpHmGuJmUPibHVbfep2aCh9lKaDqQR07gXXWK5/yU1Dx0r
 VVFRaHTasp9fZ9AmY4K9/BSA3VkQ8v3OrxNty3OdsrmTTzO91YszpdbjjEFZK53zXy6tUD2d
 e1i0kBBS6NLAAsqEtneplz88T/v7MpLmpY30N9gQU3QyRC50jJ7LU9RazMjUQY1WohVsR56d
 ORqFxS8ChhyJs7BI34vQusYHDTp6PnZHUppb9WIzjeWlC7Jc8lSBDlEWodmqQQgp5+6AfhTD
 kDv1a+W5+ncq+Uo63WHRiCPuyt4di4/0zo28RVcjtzlGBZtmz2EIC3vUfmoZbO/Gn6EKbYAn
 rzz3iU/JWV8DwQ+sZSGu0HmvYMt6t5SmqWQo/hyHtA7uF5Wxtu1lCgolSQw4t49ZuOyOnQi5
 f8R3nE7lpVCSF1TT+h8kMvFPv3VG7KunyjHr3sEptYxQs4VRxqeirSuyBv1TyxT+LdTm6j4a
 mulOWf+YtFRAgIYyyN5YOepDEBv4LUM8Tz98lZiNMlFyRMNrsLV6Pv6SxhrMxbT6TNVS5D+6
 UorTLotDZKp5+M7BTQRUY85qARAAsgMW71BIXRgxjYNCYQ3Xs8k3TfAvQRbHccky50h99TUY
 sqdULbsb3KhmY29raw1bgmyM0a4DGS1YKN7qazCDsdQlxIJp9t2YYdBKXVRzPCCsfWe1dK/q
 66UVhRPP8EGZ4CmFYuPTxqGY+dGRInxCeap/xzbKdvmPm01Iw3YFjAE4PQ4hTMr/H76KoDbD
 cq62U50oKC83ca/PRRh2QqEqACvIH4BR7jueAZSPEDnzwxvVgzyeuhwqHY05QRK/wsKuhq7s
 UuYtmN92Fasbxbw2tbVLZfoidklikvZAmotg0dwcFTjSRGEg0Gr3p/xBzJWNavFZZ95Rj7Et
 db0lCt0HDSY5q4GMR+SrFbH+jzUY/ZqfGdZCBqo0cdPPp58krVgtIGR+ja2Mkva6ah94/oQN
 lnCOw3udS+Eb/aRcM6detZr7XOngvxsWolBrhwTQFT9D2NH6ryAuvKd6yyAFt3/e7r+HHtkU
 kOy27D7IpjngqP+b4EumELI/NxPgIqT69PQmo9IZaI/oRaKorYnDaZrMXViqDrFdD37XELwQ
 gmLoSm2VfbOYY7fap/AhPOgOYOSqg3/Nxcapv71yoBzRRxOc4FxmZ65mn+q3rEM27yRztBW9
 AnCKIc66T2i92HqXCw6AgoBJRjBkI3QnEkPgohQkZdAb8o9WGVKpfmZKbYBo4pEAEQEAAcLB
 XwQYAQIACQUCVGPOagIbDAAKCRBoNZUwcMmSsJeCEACCh7P/aaOLKWQxcnw47p4phIVR6pVL
 e4IEdR7Jf7ZL00s3vKSNT+nRqdl1ugJx9Ymsp8kXKMk9GSfmZpuMQB9c6io1qZc6nW/3TtvK
 pNGz7KPPtaDzvKA4S5tfrWPnDr7n15AU5vsIZvgMjU42gkbemkjJwP0B1RkifIK60yQqAAlT
 YZ14P0dIPdIPIlfEPiAWcg5BtLQU4Wg3cNQdpWrCJ1E3m/RIlXy/2Y3YOVVohfSy+4kvvYU3
 lXUdPb04UPw4VWwjcVZPg7cgR7Izion61bGHqVqURgSALt2yvHl7cr68NYoFkzbNsGsye9ft
 M9ozM23JSgMkRylPSXTeh5JIK9pz2+etco3AfLCKtaRVysjvpysukmWMTrx8QnI5Nn5MOlJj
 1Ov4/50JY9pXzgIDVSrgy6LYSMc4vKZ3QfCY7ipLRORyalFDF3j5AGCMRENJjHPD6O7bl3Xo
 4DzMID+8eucbXxKiNEbs21IqBZbbKdY1GkcEGTE7AnkA3Y6YB7I/j9mQ3hCgm5muJuhM/2Fr
 OPsw5tV/LmQ5GXH0JQ/TZXWygyRFyyI2FqNTx4WHqUn3yFj8rwTAU1tluRUYyeLy0ayUlKBH
 ybj0N71vWO936MqP6haFERzuPAIpxj2ezwu0xb1GjTk4ynna6h5GjnKgdfOWoRtoWndMZxbA
 z5cecg==
Message-ID: <b744d47e-4b7f-d372-f8ba-758555c08993@intel.com>
Date:   Fri, 17 Dec 2021 13:17:42 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <YbzhBrimHGGpddDM@archlinux-ax161>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 12/17/21 11:12 AM, Nathan Chancellor wrote:
> With this patch in -next as commit 50468e431335 ("x86/sgx: Add an
> attribute for the amount of SGX memory in a NUMA node"), this sysfs node
> causes an OOPS for me on at least one of my test systems (Intel based):

Thanks for the report!

Could you share /proc/cpuinfo and a full dmesg from when this happens?
I'm curious if your system has SGX at all and if it had any issues
during initialization.

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id E2470C433EF
	for <linux-kernel@archiver.kernel.org>; Fri, 17 Dec 2021 22:04:48 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S229569AbhLQWEs (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Fri, 17 Dec 2021 17:04:48 -0500
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36628 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S229451AbhLQWEq (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Fri, 17 Dec 2021 17:04:46 -0500
Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 961CCC061574;
        Fri, 17 Dec 2021 14:04:46 -0800 (PST)
Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140])
        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
        (No client certificate requested)
        by dfw.source.kernel.org (Postfix) with ESMTPS id F191B623FC;
        Fri, 17 Dec 2021 22:04:45 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 81F84C36AE2;
        Fri, 17 Dec 2021 22:04:42 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
        s=k20201202; t=1639778685;
        bh=wreOyrCZydsXTxmLWbMLdJQSRKhIhiIFjgkG/aj9NCI=;
        h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
        b=tWRORviYOsq688Z99Nga1Wov/Xjd2I9gWW7fy0Xwb2Tlrh0hI0d76Zo8oo2S5bmE7
         dADXa3tV4GBHWV0oKQvEPz6gd+WM90xlAPH6IQjbQw2qqJiDmgqbsv8jUNBUdzjCpm
         vdbGFS8yniXYzbTVIUx9GfOR1jKFY1l+mcML6JRH0APDHQXNX610mKkqOqJew04Zju
         GY2hkZlquNwgq6UMeowXqEa4isEIx0wYqVH8V45P8vPaGFYuHVthVA2YpF/fBJS8+K
         rA5EzNtepq1MEobImcrDxU7sr0CyYAS2wxsJnXNV4R9wtDFyjEVTrOM1Z1GuDIucX3
         CN4jJ3JigGs4w==
Date:   Fri, 17 Dec 2021 15:04:39 -0700
From:   Nathan Chancellor <nathan@kernel.org>
To:     Dave Hansen <dave.hansen@intel.com>
Cc:     Jarkko Sakkinen <jarkko@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
Message-ID: <Yb0Jd6Ad6TfTgAEC@archlinux-ax161>
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org>
 <YbzhBrimHGGpddDM@archlinux-ax161>
 <b744d47e-4b7f-d372-f8ba-758555c08993@intel.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="kO/GpfGXlbU26p1X"
Content-Disposition: inline
In-Reply-To: <b744d47e-4b7f-d372-f8ba-758555c08993@intel.com>
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


--kO/GpfGXlbU26p1X
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Fri, Dec 17, 2021 at 01:17:42PM -0800, Dave Hansen wrote:
> On 12/17/21 11:12 AM, Nathan Chancellor wrote:
> > With this patch in -next as commit 50468e431335 ("x86/sgx: Add an
> > attribute for the amount of SGX memory in a NUMA node"), this sysfs node
> > causes an OOPS for me on at least one of my test systems (Intel based):
> 
> Thanks for the report!
> 
> Could you share /proc/cpuinfo and a full dmesg from when this happens?
> I'm curious if your system has SGX at all and if it had any issues
> during initialization.

Sure thing, find them attached (does not look like it does). Let me know
if there is any more information I can provide or test fixes.

Cheers,
Nathan

--kO/GpfGXlbU26p1X
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="cpuinfo_plus_dmesg.txt"
Content-Transfer-Encoding: quoted-printable

# cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 69
model name	: Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz
stepping	: 1
microcode	: 0x26
cpu MHz		: 800.000
cache size	: 3072 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 2
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat =
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rd=
tscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_=
tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 =
sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer =
aes xsave avx f16c rdrand lahf_lm abm cpuid_fault epb invpcid_single pti ss=
bd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase ts=
c_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt dtherm ida arat pln pts =
md_clear flush_l1d
vmx flags	: vnmi preemption_timer invvpid ept_x_only ept_ad ept_1gb flexpri=
ority tsc_offset vtpr mtf vapic ept vpid unrestricted_guest ple
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapg=
s itlb_multihit srbds
bogomips	: 4790.43
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 69
model name	: Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz
stepping	: 1
microcode	: 0x26
cpu MHz		: 1700.000
cache size	: 3072 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 2
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat =
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rd=
tscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_=
tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 =
sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer =
aes xsave avx f16c rdrand lahf_lm abm cpuid_fault epb invpcid_single pti ss=
bd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase ts=
c_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt dtherm ida arat pln pts =
md_clear flush_l1d
vmx flags	: vnmi preemption_timer invvpid ept_x_only ept_ad ept_1gb flexpri=
ority tsc_offset vtpr mtf vapic ept vpid unrestricted_guest ple
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapg=
s itlb_multihit srbds
bogomips	: 4790.43
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 69
model name	: Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz
stepping	: 1
microcode	: 0x26
cpu MHz		: 2171.251
cache size	: 3072 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 2
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat =
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rd=
tscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_=
tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 =
sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer =
aes xsave avx f16c rdrand lahf_lm abm cpuid_fault epb invpcid_single pti ss=
bd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase ts=
c_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt dtherm ida arat pln pts =
md_clear flush_l1d
vmx flags	: vnmi preemption_timer invvpid ept_x_only ept_ad ept_1gb flexpri=
ority tsc_offset vtpr mtf vapic ept vpid unrestricted_guest ple
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapg=
s itlb_multihit srbds
bogomips	: 4790.43
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 69
model name	: Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz
stepping	: 1
microcode	: 0x26
cpu MHz		: 800.000
cache size	: 3072 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 2
apicid		: 3
initial apicid	: 3
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat =
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rd=
tscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_=
tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 =
sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer =
aes xsave avx f16c rdrand lahf_lm abm cpuid_fault epb invpcid_single pti ss=
bd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase ts=
c_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt dtherm ida arat pln pts =
md_clear flush_l1d
vmx flags	: vnmi preemption_timer invvpid ept_x_only ept_ad ept_1gb flexpri=
ority tsc_offset vtpr mtf vapic ept vpid unrestricted_guest ple
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapg=
s itlb_multihit srbds
bogomips	: 4790.43
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:

# dmesg
[    0.000000] microcode: microcode updated early to revision 0x26, date =
=3D 2019-11-12
[    0.000000] Linux version 5.16.0-rc5-next-20211217-debug (nathan@archlin=
ux-ax161) (gcc (GCC) 11.1.0, GNU ld (GNU Binutils) 2.37.50.20211212) #1 SMP=
 PREEMPT Fri Dec 17 18:40:53 UTC 2021
[    0.000000] Command line: initrd=3D\intel-ucode.img initrd=3D\initramfs-=
linux-debug.img root=3DPARTUUID=3D8dd2ef78-91f0-40ff-96ad-6273e7a97147 rw i=
ntel_pstate=3Dno_hwp
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point=
 registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 by=
tes, using 'standard' format.
[    0.000000] signal: max sigframe size: 1776
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x0000000000057fff] usable
[    0.000000] BIOS-e820: [mem 0x0000000000058000-0x0000000000058fff] reser=
ved
[    0.000000] BIOS-e820: [mem 0x0000000000059000-0x000000000009dfff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009e000-0x000000000009ffff] reser=
ved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000c9abafff] usable
[    0.000000] BIOS-e820: [mem 0x00000000c9abb000-0x00000000c9ac1fff] ACPI =
NVS
[    0.000000] BIOS-e820: [mem 0x00000000c9ac2000-0x00000000ca318fff] usable
[    0.000000] BIOS-e820: [mem 0x00000000ca319000-0x00000000ca597fff] reser=
ved
[    0.000000] BIOS-e820: [mem 0x00000000ca598000-0x00000000d9917fff] usable
[    0.000000] BIOS-e820: [mem 0x00000000d9918000-0x00000000d9b1ffff] reser=
ved
[    0.000000] BIOS-e820: [mem 0x00000000d9b20000-0x00000000d9e5dfff] usable
[    0.000000] BIOS-e820: [mem 0x00000000d9e5e000-0x00000000dab29fff] ACPI =
NVS
[    0.000000] BIOS-e820: [mem 0x00000000dab2a000-0x00000000daffefff] reser=
ved
[    0.000000] BIOS-e820: [mem 0x00000000dafff000-0x00000000daffffff] usable
[    0.000000] BIOS-e820: [mem 0x00000000dbc00000-0x00000000dfdfffff] reser=
ved
[    0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reser=
ved
[    0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reser=
ved
[    0.000000] BIOS-e820: [mem 0x00000000fed00000-0x00000000fed03fff] reser=
ved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reser=
ved
[    0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reser=
ved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reser=
ved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000021f1fffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] efi: EFI v2.31 by American Megatrends
[    0.000000] efi: ACPI=3D0xdaae5000 ACPI 2.0=3D0xdaae5000 SMBIOS=3D0xdaf5=
2318=20
[    0.000000] SMBIOS 2.7 present.
[    0.000000] DMI: ASUSTeK COMPUTER INC. Q302LA/Q302LA, BIOS Q302LA.203 05=
/15/2014
[    0.000000] tsc: Fast TSC calibration using PIT
[    0.000000] tsc: Detected 2394.447 MHz processor
[    0.000078] e820: update [mem 0x00000000-0x00000fff] usable =3D=3D> rese=
rved
[    0.000082] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.000090] last_pfn =3D 0x21f200 max_arch_pfn =3D 0x400000000
[    0.000204] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT=
 =20
[    0.000876] total RAM covered: 8110M
[    0.001045] Found optimal setting for mtrr clean up
[    0.001046]  gran_size: 64K 	chunk_size: 128M 	num_reg: 10  	lose cover =
RAM: 0G
[    0.001394] e820: update [mem 0xdbc00000-0xffffffff] usable =3D=3D> rese=
rved
[    0.001399] last_pfn =3D 0xdb000 max_arch_pfn =3D 0x400000000
[    0.010564] Using GB pages for direct mapping
[    0.011324] Secure boot disabled
[    0.011325] RAMDISK: [mem 0x7ef14000-0x7fb72fff]
[    0.011330] ACPI: Early table checksum verification disabled
[    0.011334] ACPI: RSDP 0x00000000DAAE5000 000024 (v02 _ASUS_)
[    0.011338] ACPI: XSDT 0x00000000DAAE5090 0000A4 (v01 _ASUS_ Notebook 01=
072009 AMI  00010013)
[    0.011345] ACPI: FACP 0x00000000DAAFB750 00010C (v05 _ASUS_ Notebook 01=
072009 AMI  00010013)
[    0.011352] ACPI: DSDT 0x00000000DAAE5248 016508 (v02 _ASUS_ Notebook 00=
000012 INTL 20120711)
[    0.011356] ACPI: FACS 0x00000000DAB27F80 000040
[    0.011359] ACPI: APIC 0x00000000DAAFB860 000072 (v03 _ASUS_ Notebook 01=
072009 AMI  00010013)
[    0.011362] ACPI: FPDT 0x00000000DAAFB8D8 000044 (v01 _ASUS_ Notebook 01=
072009 AMI  00010013)
[    0.011366] ACPI: ECDT 0x00000000DAAFB920 0000C1 (v01 _ASUS_ Notebook 01=
072009 AMI. 00000005)
[    0.011369] ACPI: LPIT 0x00000000DAAFB9E8 000094 (v01 ALASKA          00=
000000 AMI. 00000005)
[    0.011373] ACPI: SSDT 0x00000000DAAFBA80 000539 (v01 PmRef  Cpu0Ist  00=
003000 INTL 20120711)
[    0.011377] ACPI: SSDT 0x00000000DAAFBFC0 000AD8 (v01 PmRef  CpuPm    00=
003000 INTL 20120711)
[    0.011380] ACPI: SSDT 0x00000000DAAFCA98 0002DE (v01 PmRef  Cpu0Tst  00=
003000 INTL 20120711)
[    0.011384] ACPI: SSDT 0x00000000DAAFCD78 000348 (v01 PmRef  ApTst    00=
003000 INTL 20120711)
[    0.011388] ACPI: MCFG 0x00000000DAAFD0C0 00003C (v01 _ASUS_ Notebook 01=
072009 MSFT 00000097)
[    0.011391] ACPI: SSDT 0x00000000DAAFD100 00306D (v01 DptfTa DptfTabl 00=
001000 INTL 20120711)
[    0.011395] ACPI: HPET 0x00000000DAB00170 000038 (v01 _ASUS_ Notebook 01=
072009 AMI. 00000005)
[    0.011398] ACPI: SSDT 0x00000000DAB001A8 0002D7 (v01 SataRe SataTabl 00=
001000 INTL 20120711)
[    0.011402] ACPI: SSDT 0x00000000DAB00480 003F21 (v01 SaSsdt SaSsdt   00=
003000 INTL 20091112)
[    0.011405] ACPI: BGRT 0x00000000DAB043A8 000038 (v00 _ASUS_ Notebook 01=
072009 ASUS 00010013)
[    0.011409] ACPI: MSDM 0x00000000D9B1EE18 000055 (v03 _ASUS_ Notebook 00=
000000 ASUS 00000001)
[    0.011412] ACPI: Reserving FACP table memory at [mem 0xdaafb750-0xdaafb=
85b]
[    0.011414] ACPI: Reserving DSDT table memory at [mem 0xdaae5248-0xdaafb=
74f]
[    0.011415] ACPI: Reserving FACS table memory at [mem 0xdab27f80-0xdab27=
fbf]
[    0.011416] ACPI: Reserving APIC table memory at [mem 0xdaafb860-0xdaafb=
8d1]
[    0.011417] ACPI: Reserving FPDT table memory at [mem 0xdaafb8d8-0xdaafb=
91b]
[    0.011418] ACPI: Reserving ECDT table memory at [mem 0xdaafb920-0xdaafb=
9e0]
[    0.011420] ACPI: Reserving LPIT table memory at [mem 0xdaafb9e8-0xdaafb=
a7b]
[    0.011421] ACPI: Reserving SSDT table memory at [mem 0xdaafba80-0xdaafb=
fb8]
[    0.011422] ACPI: Reserving SSDT table memory at [mem 0xdaafbfc0-0xdaafc=
a97]
[    0.011423] ACPI: Reserving SSDT table memory at [mem 0xdaafca98-0xdaafc=
d75]
[    0.011424] ACPI: Reserving SSDT table memory at [mem 0xdaafcd78-0xdaafd=
0bf]
[    0.011426] ACPI: Reserving MCFG table memory at [mem 0xdaafd0c0-0xdaafd=
0fb]
[    0.011427] ACPI: Reserving SSDT table memory at [mem 0xdaafd100-0xdab00=
16c]
[    0.011428] ACPI: Reserving HPET table memory at [mem 0xdab00170-0xdab00=
1a7]
[    0.011429] ACPI: Reserving SSDT table memory at [mem 0xdab001a8-0xdab00=
47e]
[    0.011430] ACPI: Reserving SSDT table memory at [mem 0xdab00480-0xdab04=
3a0]
[    0.011432] ACPI: Reserving BGRT table memory at [mem 0xdab043a8-0xdab04=
3df]
[    0.011433] ACPI: Reserving MSDM table memory at [mem 0xd9b1ee18-0xd9b1e=
e6c]
[    0.011453] ACPI: CEDT not present
[    0.011489] No NUMA configuration found
[    0.011490] Faking a node at [mem 0x0000000000000000-0x000000021f1fffff]
[    0.011494] NODE_DATA(0) allocated [mem 0x21f1fc000-0x21f1fffff]
[    0.011521] Zone ranges:
[    0.011522]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.011524]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.011526]   Normal   [mem 0x0000000100000000-0x000000021f1fffff]
[    0.011528]   Device   empty
[    0.011529] Movable zone start for each node
[    0.011530] Early memory node ranges
[    0.011531]   node   0: [mem 0x0000000000001000-0x0000000000057fff]
[    0.011533]   node   0: [mem 0x0000000000059000-0x000000000009dfff]
[    0.011534]   node   0: [mem 0x0000000000100000-0x00000000c9abafff]
[    0.011535]   node   0: [mem 0x00000000c9ac2000-0x00000000ca318fff]
[    0.011536]   node   0: [mem 0x00000000ca598000-0x00000000d9917fff]
[    0.011538]   node   0: [mem 0x00000000d9b20000-0x00000000d9e5dfff]
[    0.011539]   node   0: [mem 0x00000000dafff000-0x00000000daffffff]
[    0.011540]   node   0: [mem 0x0000000100000000-0x000000021f1fffff]
[    0.011542] Initmem setup node 0 [mem 0x0000000000001000-0x000000021f1ff=
fff]
[    0.011546] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.011548] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.011571] On node 0, zone DMA: 98 pages in unavailable ranges
[    0.017483] On node 0, zone DMA32: 7 pages in unavailable ranges
[    0.017944] On node 0, zone DMA32: 639 pages in unavailable ranges
[    0.017959] On node 0, zone DMA32: 520 pages in unavailable ranges
[    0.018005] On node 0, zone DMA32: 4513 pages in unavailable ranges
[    0.026672] On node 0, zone Normal: 20480 pages in unavailable ranges
[    0.026711] On node 0, zone Normal: 3584 pages in unavailable ranges
[    0.026720] Reserving Intel graphics memory at [mem 0xdbe00000-0xdfdffff=
f]
[    0.026798] ACPI: PM-Timer IO Port: 0x1808
[    0.026806] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[    0.026819] IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-=
39
[    0.026822] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.026824] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.026828] ACPI: Using ACPI (MADT) for SMP configuration information
[    0.026830] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.026838] e820: update [mem 0xd758c000-0xd75a4fff] usable =3D=3D> rese=
rved
[    0.026850] TSC deadline timer available
[    0.026851] smpboot: Allowing 4 CPUs, 0 hotplug CPUs
[    0.026878] PM: hibernation: Registered nosave memory: [mem 0x00000000-0=
x00000fff]
[    0.026880] PM: hibernation: Registered nosave memory: [mem 0x00058000-0=
x00058fff]
[    0.026883] PM: hibernation: Registered nosave memory: [mem 0x0009e000-0=
x0009ffff]
[    0.026884] PM: hibernation: Registered nosave memory: [mem 0x000a0000-0=
x000fffff]
[    0.026886] PM: hibernation: Registered nosave memory: [mem 0xc9abb000-0=
xc9ac1fff]
[    0.026888] PM: hibernation: Registered nosave memory: [mem 0xca319000-0=
xca597fff]
[    0.026891] PM: hibernation: Registered nosave memory: [mem 0xd758c000-0=
xd75a4fff]
[    0.026893] PM: hibernation: Registered nosave memory: [mem 0xd9918000-0=
xd9b1ffff]
[    0.026895] PM: hibernation: Registered nosave memory: [mem 0xd9e5e000-0=
xdab29fff]
[    0.026896] PM: hibernation: Registered nosave memory: [mem 0xdab2a000-0=
xdaffefff]
[    0.026898] PM: hibernation: Registered nosave memory: [mem 0xdb000000-0=
xdbbfffff]
[    0.026899] PM: hibernation: Registered nosave memory: [mem 0xdbc00000-0=
xdfdfffff]
[    0.026900] PM: hibernation: Registered nosave memory: [mem 0xdfe00000-0=
xf7ffffff]
[    0.026901] PM: hibernation: Registered nosave memory: [mem 0xf8000000-0=
xfbffffff]
[    0.026902] PM: hibernation: Registered nosave memory: [mem 0xfc000000-0=
xfebfffff]
[    0.026903] PM: hibernation: Registered nosave memory: [mem 0xfec00000-0=
xfec00fff]
[    0.026904] PM: hibernation: Registered nosave memory: [mem 0xfec01000-0=
xfecfffff]
[    0.026905] PM: hibernation: Registered nosave memory: [mem 0xfed00000-0=
xfed03fff]
[    0.026906] PM: hibernation: Registered nosave memory: [mem 0xfed04000-0=
xfed1bfff]
[    0.026907] PM: hibernation: Registered nosave memory: [mem 0xfed1c000-0=
xfed1ffff]
[    0.026908] PM: hibernation: Registered nosave memory: [mem 0xfed20000-0=
xfedfffff]
[    0.026909] PM: hibernation: Registered nosave memory: [mem 0xfee00000-0=
xfee00fff]
[    0.026910] PM: hibernation: Registered nosave memory: [mem 0xfee01000-0=
xfeffffff]
[    0.026911] PM: hibernation: Registered nosave memory: [mem 0xff000000-0=
xffffffff]
[    0.026913] [mem 0xdfe00000-0xf7ffffff] available for PCI devices
[    0.026915] Booting paravirtualized kernel on bare hardware
[    0.026917] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0=
xffffffff, max_idle_ns: 6370452778343963 ns
[    0.032536] setup_percpu: NR_CPUS:320 nr_cpumask_bits:320 nr_cpu_ids:4 n=
r_node_ids:1
[    0.032748] percpu: Embedded 61 pages/cpu s212992 r8192 d28672 u524288
[    0.032757] pcpu-alloc: s212992 r8192 d28672 u524288 alloc=3D1*2097152
[    0.032760] pcpu-alloc: [0] 0 1 2 3=20
[    0.032787] Fallback order for Node 0: 0=20
[    0.032791] Built 1 zonelists, mobility grouping on.  Total pages: 20348=
49
[    0.032793] Policy zone: Normal
[    0.032795] Kernel command line: initrd=3D\intel-ucode.img initrd=3D\ini=
tramfs-linux-debug.img root=3DPARTUUID=3D8dd2ef78-91f0-40ff-96ad-6273e7a971=
47 rw intel_pstate=3Dno_hwp
[    0.033300] Dentry cache hash table entries: 1048576 (order: 11, 8388608=
 bytes, linear)
[    0.033527] Inode-cache hash table entries: 524288 (order: 10, 4194304 b=
ytes, linear)
[    0.033691] mem auto-init: stack:byref_all(zero), heap alloc:on, heap fr=
ee:off
[    0.066173] Memory: 7965936K/8269236K available (14343K kernel code, 207=
0K rwdata, 10544K rodata, 1784K init, 4020K bss, 303040K reserved, 0K cma-r=
eserved)
[    0.066180] random: get_random_u64 called from __kmem_cache_create+0x2a/=
0x560 with crng_init=3D0
[    0.066343] SLUB: HWalign=3D64, Order=3D0-3, MinObjects=3D0, CPUs=3D4, N=
odes=3D1
[    0.066356] Kernel/User page tables isolation: enabled
[    0.066379] ftrace: allocating 43546 entries in 171 pages
[    0.083163] ftrace: allocated 171 pages with 5 groups
[    0.083236] Dynamic Preempt: full
[    0.083269] rcu: Preemptible hierarchical RCU implementation.
[    0.083270] rcu: 	RCU restricting CPUs from NR_CPUS=3D320 to nr_cpu_ids=
=3D4.
[    0.083272] rcu: 	RCU priority boosting: priority 1 delay 500 ms.
[    0.083273] 	Trampoline variant of Tasks RCU enabled.
[    0.083274] 	Rude variant of Tasks RCU enabled.
[    0.083275] 	Tracing variant of Tasks RCU enabled.
[    0.083275] rcu: RCU calculated value of scheduler-enlistment delay is 3=
0 jiffies.
[    0.083276] rcu: Adjusting geometry for rcu_fanout_leaf=3D16, nr_cpu_ids=
=3D4
[    0.088276] NR_IRQS: 20736, nr_irqs: 728, preallocated irqs: 16
[    0.088571] kfence: initialized - using 2097152 bytes for 255 objects at=
 0x(____ptrval____)-0x(____ptrval____)
[    0.088703] Console: colour dummy device 80x25
[    0.088720] printk: console [tty0] enabled
[    0.088741] ACPI: Core revision 20210930
[    0.088893] hpet: HPET dysfunctional in PC10. Force disabled.
[    0.088894] APIC: Switch to symmetric I/O mode setup
[    0.089218] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles:=
 0x2283ba1244f, max_idle_ns: 440795326940 ns
[    0.089224] Calibrating delay loop (skipped), value calculated using tim=
er frequency.. 4790.43 BogoMIPS (lpj=3D7981490)
[    0.089227] pid_max: default: 32768 minimum: 301
[    0.091127] LSM: Security Framework initializing
[    0.091135] landlock: Up and running.
[    0.091136] Yama: becoming mindful.
[    0.091143] LSM support for eBPF active
[    0.091168] Mount-cache hash table entries: 16384 (order: 5, 131072 byte=
s, linear)
[    0.091185] Mountpoint-cache hash table entries: 16384 (order: 5, 131072=
 bytes, linear)
[    0.091430] CPU0: Thermal monitoring enabled (TM1)
[    0.091467] process: using mwait in idle threads
[    0.091469] Last level iTLB entries: 4KB 1024, 2MB 1024, 4MB 1024
[    0.091471] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 1024, 1GB 4
[    0.091475] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user=
 pointer sanitization
[    0.091477] Spectre V2 : Mitigation: Full generic retpoline
[    0.091477] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB=
 on context switch
[    0.091478] Spectre V2 : Enabling Restricted Speculation for firmware ca=
lls
[    0.091480] Spectre V2 : mitigation: Enabling conditional Indirect Branc=
h Prediction Barrier
[    0.091481] Spectre V2 : User space: Mitigation: STIBP via prctl
[    0.091482] Speculative Store Bypass: Mitigation: Speculative Store Bypa=
ss disabled via prctl
[    0.091486] SRBDS: Mitigation: Microcode
[    0.091487] MDS: Mitigation: Clear CPU buffers
[    0.092555] Freeing SMP alternatives memory: 36K
[    0.092555] smpboot: Estimated ratio of average max frequency by base fr=
equency (times 1024): 1024
[    0.092555] smpboot: CPU0: Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz (fam=
ily: 0x6, model: 0x45, stepping: 0x1)
[    0.092555] cblist_init_generic: Setting adjustable number of callback q=
ueues.
[    0.092555] cblist_init_generic: Setting shift to 2 and lim to 1.
[    0.092555] cblist_init_generic: Setting shift to 2 and lim to 1.
[    0.092555] cblist_init_generic: Setting shift to 2 and lim to 1.
[    0.092555] Performance Events: PEBS fmt2+, Haswell events, 16-deep LBR,=
 full-width counters, Intel PMU driver.
[    0.092555] ... version:                3
[    0.092555] ... bit width:              48
[    0.092555] ... generic registers:      4
[    0.092555] ... value mask:             0000ffffffffffff
[    0.092555] ... max period:             00007fffffffffff
[    0.092555] ... fixed-purpose events:   3
[    0.092555] ... event mask:             000000070000000f
[    0.092555] rcu: Hierarchical SRCU implementation.
[    0.092555] NMI watchdog: Enabled. Permanently consumes one hw-PMU count=
er.
[    0.092555] smp: Bringing up secondary CPUs ...
[    0.092555] x86: Booting SMP configuration:
[    0.092555] .... node  #0, CPUs:      #1 #2
[    0.093085] MDS CPU bug present and SMT on, data leak possible. See http=
s://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more de=
tails.
[    0.093085]  #3
[    0.093085] smp: Brought up 1 node, 4 CPUs
[    0.093085] smpboot: Max logical packages: 1
[    0.093085] smpboot: Total of 4 processors activated (19163.72 BogoMIPS)
[    0.096189] devtmpfs: initialized
[    0.096189] x86/mm: Memory block size: 128MB
[    0.096519] ACPI: PM: Registering ACPI NVS region [mem 0xc9abb000-0xc9ac=
1fff] (28672 bytes)
[    0.096519] ACPI: PM: Registering ACPI NVS region [mem 0xd9e5e000-0xdab2=
9fff] (13418496 bytes)
[    0.096519] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xfffffff=
f, max_idle_ns: 6370867519511994 ns
[    0.096519] futex hash table entries: 1024 (order: 4, 65536 bytes, linea=
r)
[    0.096519] pinctrl core: initialized pinctrl subsystem
[    0.096519] PM: RTC time: 21:56:55, date: 2021-12-17
[    0.096519] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.096601] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic alloca=
tions
[    0.096677] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomi=
c allocations
[    0.096751] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for ato=
mic allocations
[    0.096765] audit: initializing netlink subsys (disabled)
[    0.096774] audit: type=3D2000 audit(1639778215.006:1): state=3Dinitiali=
zed audit_enabled=3D0 res=3D1
[    0.096774] thermal_sys: Registered thermal governor 'fair_share'
[    0.096774] thermal_sys: Registered thermal governor 'bang_bang'
[    0.096774] thermal_sys: Registered thermal governor 'step_wise'
[    0.096774] thermal_sys: Registered thermal governor 'user_space'
[    0.096774] thermal_sys: Registered thermal governor 'power_allocator'
[    0.096774] cpuidle: using governor ladder
[    0.096774] cpuidle: using governor menu
[    0.096774] HugeTLB: can free 4095 vmemmap pages for hugepages-1048576kB
[    0.096774] ACPI FADT declares the system doesn't support PCIe ASPM, so =
disable it
[    0.096774] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.096774] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000=
-0xfbffffff] (base 0xf8000000)
[    0.096774] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved in E820
[    0.096774] PCI: Using configuration type 1 for base access
[    0.096774] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on
[    0.096774] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[    0.099657] kprobes: kprobe jump-optimization is enabled. All kprobes ar=
e optimized if possible.
[    0.099665] HugeTLB: can free 7 vmemmap pages for hugepages-2048kB
[    0.099665] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.099665] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.099665] ACPI: Added _OSI(Module Device)
[    0.099665] ACPI: Added _OSI(Processor Device)
[    0.099665] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.099665] ACPI: Added _OSI(Processor Aggregator Device)
[    0.099665] ACPI: Added _OSI(Linux-Dell-Video)
[    0.099665] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    0.099665] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[    0.127364] ACPI: 8 ACPI AML tables successfully acquired and loaded
[    0.128317] ACPI: EC: EC started
[    0.128318] ACPI: EC: interrupt blocked
[    0.129550] ACPI: EC: EC_CMD/EC_SC=3D0x66, EC_DATA=3D0x62
[    0.129552] ACPI: EC: Boot ECDT EC used to handle transactions
[    0.130631] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored
[    0.133682] ACPI: Dynamic OEM Table Load:
[    0.133690] ACPI: SSDT 0xFFFF99D5402AF400 0003D3 (v01 PmRef  Cpu0Cst  00=
003001 INTL 20120711)
[    0.135085] ACPI: Dynamic OEM Table Load:
[    0.135091] ACPI: SSDT 0xFFFF99D540BFA800 0005AA (v01 PmRef  ApIst    00=
003000 INTL 20120711)
[    0.136573] ACPI: Dynamic OEM Table Load:
[    0.136578] ACPI: SSDT 0xFFFF99D54021BA00 000119 (v01 PmRef  ApCst    00=
003000 INTL 20120711)
[    0.138810] ACPI: Interpreter enabled
[    0.138848] ACPI: PM: (supports S0 S3 S4 S5)
[    0.138850] ACPI: Using IOAPIC for interrupt routing
[    0.138883] PCI: Using host bridge windows from ACPI; if necessary, use =
"pci=3Dnocrs" and report a bug
[    0.139540] ACPI: Enabled 6 GPEs in block 00 to 7F
[    0.155371] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-3e])
[    0.155379] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM Cloc=
kPM Segments MSI EDR HPX-Type3]
[    0.155695] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplu=
g SHPCHotplug PME]
[    0.156058] acpi PNP0A08:00: _OSC: OS now controls [AER PCIeCapability L=
TR DPC]
[    0.156060] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using B=
IOS configuration
[    0.156642] PCI host bridge to bus 0000:00
[    0.156645] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.156648] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.156650] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000efff=
f window]
[    0.156651] pci_bus 0000:00: root bus resource [mem 0xdfe00000-0xfeaffff=
f window]
[    0.156653] pci_bus 0000:00: root bus resource [bus 00-3e]
[    0.156668] pci 0000:00:00.0: [8086:0a04] type 00 class 0x060000
[    0.156739] pci 0000:00:02.0: [8086:0a16] type 00 class 0x030000
[    0.156748] pci 0000:00:02.0: reg 0x10: [mem 0xf7800000-0xf7bfffff 64bit]
[    0.156755] pci 0000:00:02.0: reg 0x18: [mem 0xe0000000-0xefffffff 64bit=
 pref]
[    0.156759] pci 0000:00:02.0: reg 0x20: [io  0xf000-0xf03f]
[    0.156772] pci 0000:00:02.0: BAR 2: assigned to efifb
[    0.156852] pci 0000:00:03.0: [8086:0a0c] type 00 class 0x040300
[    0.156861] pci 0000:00:03.0: reg 0x10: [mem 0xf7d1c000-0xf7d1ffff 64bit]
[    0.156953] pci 0000:00:04.0: [8086:0a03] type 00 class 0x118000
[    0.156962] pci 0000:00:04.0: reg 0x10: [mem 0xf7d10000-0xf7d17fff 64bit]
[    0.157066] pci 0000:00:14.0: [8086:9c31] type 00 class 0x0c0330
[    0.157081] pci 0000:00:14.0: reg 0x10: [mem 0xf7d00000-0xf7d0ffff 64bit]
[    0.157136] pci 0000:00:14.0: PME# supported from D3hot D3cold
[    0.157211] pci 0000:00:16.0: [8086:9c3a] type 00 class 0x078000
[    0.157228] pci 0000:00:16.0: reg 0x10: [mem 0xf7d25000-0xf7d2501f 64bit]
[    0.157294] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold
[    0.157351] pci 0000:00:1b.0: [8086:9c20] type 00 class 0x040300
[    0.157366] pci 0000:00:1b.0: reg 0x10: [mem 0xf7d18000-0xf7d1bfff 64bit]
[    0.157430] pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
[    0.157512] pci 0000:00:1c.0: [8086:9c10] type 01 class 0x060400
[    0.157596] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
[    0.158041] pci 0000:00:1c.3: [8086:9c16] type 01 class 0x060400
[    0.158131] pci 0000:00:1c.3: PME# supported from D0 D3hot D3cold
[    0.158563] pci 0000:00:1d.0: [8086:9c26] type 00 class 0x0c0320
[    0.158579] pci 0000:00:1d.0: reg 0x10: [mem 0xf7d23000-0xf7d233ff]
[    0.158662] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold
[    0.158747] pci 0000:00:1f.0: [8086:9c43] type 00 class 0x060100
[    0.158920] pci 0000:00:1f.2: [8086:9c03] type 00 class 0x010601
[    0.158931] pci 0000:00:1f.2: reg 0x10: [io  0xf0b0-0xf0b7]
[    0.158938] pci 0000:00:1f.2: reg 0x14: [io  0xf0a0-0xf0a3]
[    0.158945] pci 0000:00:1f.2: reg 0x18: [io  0xf090-0xf097]
[    0.158951] pci 0000:00:1f.2: reg 0x1c: [io  0xf080-0xf083]
[    0.158958] pci 0000:00:1f.2: reg 0x20: [io  0xf060-0xf07f]
[    0.158964] pci 0000:00:1f.2: reg 0x24: [mem 0xf7d22000-0xf7d227ff]
[    0.158997] pci 0000:00:1f.2: PME# supported from D3hot
[    0.159060] pci 0000:00:1f.3: [8086:9c22] type 00 class 0x0c0500
[    0.159075] pci 0000:00:1f.3: reg 0x10: [mem 0xf7d21000-0xf7d210ff 64bit]
[    0.159093] pci 0000:00:1f.3: reg 0x20: [io  0xf040-0xf05f]
[    0.159279] acpiphp: Slot [1] registered
[    0.159291] pci 0000:00:1c.0: PCI bridge to [bus 01]
[    0.159589] pci 0000:02:00.0: [8086:08b1] type 00 class 0x028000
[    0.159679] pci 0000:02:00.0: reg 0x10: [mem 0xf7c00000-0xf7c01fff 64bit]
[    0.159983] pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
[    0.160418] pci 0000:00:1c.3: PCI bridge to [bus 02]
[    0.160423] pci 0000:00:1c.3:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.162656] ACPI: PCI: Interrupt link LNKA configured for IRQ 0
[    0.162658] ACPI: PCI: Interrupt link LNKA disabled
[    0.162732] ACPI: PCI: Interrupt link LNKB configured for IRQ 0
[    0.162734] ACPI: PCI: Interrupt link LNKB disabled
[    0.162806] ACPI: PCI: Interrupt link LNKC configured for IRQ 0
[    0.162807] ACPI: PCI: Interrupt link LNKC disabled
[    0.162879] ACPI: PCI: Interrupt link LNKD configured for IRQ 0
[    0.162880] ACPI: PCI: Interrupt link LNKD disabled
[    0.162952] ACPI: PCI: Interrupt link LNKE configured for IRQ 0
[    0.162953] ACPI: PCI: Interrupt link LNKE disabled
[    0.163027] ACPI: PCI: Interrupt link LNKF configured for IRQ 0
[    0.163028] ACPI: PCI: Interrupt link LNKF disabled
[    0.163099] ACPI: PCI: Interrupt link LNKG configured for IRQ 0
[    0.163101] ACPI: PCI: Interrupt link LNKG disabled
[    0.163172] ACPI: PCI: Interrupt link LNKH configured for IRQ 0
[    0.163173] ACPI: PCI: Interrupt link LNKH disabled
[    0.163835] ACPI: EC: interrupt unblocked
[    0.163837] ACPI: EC: event unblocked
[    0.163843] ACPI: EC: EC_CMD/EC_SC=3D0x66, EC_DATA=3D0x62
[    0.163844] ACPI: EC: GPE=3D0xa
[    0.163846] ACPI: \_SB_.PCI0.LPCB.EC0_: Boot ECDT EC initialization comp=
lete
[    0.163848] ACPI: \_SB_.PCI0.LPCB.EC0_: EC: Used to handle transactions =
and events
[    0.163911] iommu: Default domain type: Translated=20
[    0.163913] iommu: DMA domain TLB invalidation policy: lazy mode=20
[    0.163928] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[    0.163928] pci 0000:00:02.0: vgaarb: VGA device added: decodes=3Dio+mem=
,owns=3Dio+mem,locks=3Dnone
[    0.163928] pci 0000:00:02.0: vgaarb: bridge control possible
[    0.163928] vgaarb: loaded
[    0.163928] SCSI subsystem initialized
[    0.163928] libata version 3.00 loaded.
[    0.163928] ACPI: bus type USB registered
[    0.163928] usbcore: registered new interface driver usbfs
[    0.163928] usbcore: registered new interface driver hub
[    0.163928] usbcore: registered new device driver usb
[    0.184015] pps_core: LinuxPPS API ver. 1 registered
[    0.184017] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo =
Giometti <giometti@linux.it>
[    0.184021] PTP clock support registered
[    0.184030] EDAC MC: Ver: 3.0.0
[    0.184140] Registered efivars operations
[    0.184140] NetLabel: Initializing
[    0.184140] NetLabel:  domain hash size =3D 128
[    0.184140] NetLabel:  protocols =3D UNLABELED CIPSOv4 CALIPSO
[    0.184140] NetLabel:  unlabeled traffic allowed by default
[    0.184140] PCI: Using ACPI for IRQ routing
[    0.184140] PCI: pci_cache_line_size set to 64 bytes
[    0.184148] e820: reserve RAM buffer [mem 0x00058000-0x0005ffff]
[    0.184151] e820: reserve RAM buffer [mem 0x0009e000-0x0009ffff]
[    0.184152] e820: reserve RAM buffer [mem 0xc9abb000-0xcbffffff]
[    0.184154] e820: reserve RAM buffer [mem 0xca319000-0xcbffffff]
[    0.184155] e820: reserve RAM buffer [mem 0xd758c000-0xd7ffffff]
[    0.184156] e820: reserve RAM buffer [mem 0xd9918000-0xdbffffff]
[    0.184158] e820: reserve RAM buffer [mem 0xd9e5e000-0xdbffffff]
[    0.184160] e820: reserve RAM buffer [mem 0xdb000000-0xdbffffff]
[    0.184161] e820: reserve RAM buffer [mem 0x21f200000-0x21fffffff]
[    0.186808] clocksource: Switched to clocksource tsc-early
[    0.195630] VFS: Disk quotas dquot_6.6.0
[    0.195648] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 byte=
s)
[    0.195744] pnp: PnP ACPI init
[    0.195913] system 00:00: [mem 0xfed40000-0xfed44fff] has been reserved
[    0.196264] system 00:01: [io  0x0680-0x069f] has been reserved
[    0.196267] system 00:01: [io  0xffff] has been reserved
[    0.196269] system 00:01: [io  0xffff] has been reserved
[    0.196271] system 00:01: [io  0xffff] has been reserved
[    0.196272] system 00:01: [io  0x1c00-0x1cfe] has been reserved
[    0.196274] system 00:01: [io  0x1d00-0x1dfe] has been reserved
[    0.196276] system 00:01: [io  0x1e00-0x1efe] has been reserved
[    0.196277] system 00:01: [io  0x1f00-0x1ffe] has been reserved
[    0.196279] system 00:01: [io  0x1800-0x18fe] has been reserved
[    0.196281] system 00:01: [io  0x164e-0x164f] has been reserved
[    0.196370] system 00:03: [io  0x1854-0x1857] has been reserved
[    0.196429] system 00:04: [io  0x04d0-0x04d1] has been reserved
[    0.196464] system 00:05: [io  0x0240-0x0259] has been reserved
[    0.197041] system 00:08: [mem 0xfe104000-0xfe104fff] has been reserved
[    0.197595] system 00:09: [mem 0xfed1c000-0xfed1ffff] has been reserved
[    0.197598] system 00:09: [mem 0xfed10000-0xfed17fff] has been reserved
[    0.197600] system 00:09: [mem 0xfed18000-0xfed18fff] has been reserved
[    0.197602] system 00:09: [mem 0xfed19000-0xfed19fff] has been reserved
[    0.197605] system 00:09: [mem 0xf8000000-0xfbffffff] has been reserved
[    0.197607] system 00:09: [mem 0xfed20000-0xfed3ffff] has been reserved
[    0.197609] system 00:09: [mem 0xfed90000-0xfed93fff] has been reserved
[    0.197611] system 00:09: [mem 0xfed45000-0xfed8ffff] has been reserved
[    0.197613] system 00:09: [mem 0xff000000-0xffffffff] has been reserved
[    0.197615] system 00:09: [mem 0xfee00000-0xfeefffff] could not be reser=
ved
[    0.197617] system 00:09: [mem 0xf7fdf000-0xf7fdffff] has been reserved
[    0.197620] system 00:09: [mem 0xf7fe0000-0xf7feffff] has been reserved
[    0.198247] pnp: PnP ACPI: found 11 devices
[    0.204128] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, m=
ax_idle_ns: 2085701024 ns
[    0.204189] NET: Registered PF_INET protocol family
[    0.204286] IP idents hash table entries: 131072 (order: 8, 1048576 byte=
s, linear)
[    0.205638] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4,=
 65536 bytes, linear)
[    0.205702] TCP established hash table entries: 65536 (order: 7, 524288 =
bytes, linear)
[    0.205838] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes,=
 linear)
[    0.205962] TCP: Hash tables configured (established 65536 bind 65536)
[    0.206022] MPTCP token hash table entries: 8192 (order: 5, 196608 bytes=
, linear)
[    0.206073] UDP hash table entries: 4096 (order: 5, 131072 bytes, linear)
[    0.206099] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes, l=
inear)
[    0.206152] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.206158] NET: Registered PF_XDP protocol family
[    0.206166] pci 0000:00:1c.0: PCI bridge to [bus 01]
[    0.206186] pci 0000:00:1c.3: PCI bridge to [bus 02]
[    0.206191] pci 0000:00:1c.3:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.206199] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
[    0.206201] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
[    0.206203] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000effff windo=
w]
[    0.206205] pci_bus 0000:00: resource 7 [mem 0xdfe00000-0xfeafffff windo=
w]
[    0.206206] pci_bus 0000:02: resource 1 [mem 0xf7c00000-0xf7cfffff]
[    0.206381] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x0=
00c0000-0x000dffff]
[    0.206848] PCI: CLS 64 bytes, default 64
[    0.206857] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    0.206858] software IO TLB: mapped [mem 0x00000000d347a000-0x00000000d7=
47a000] (64MB)
[    0.206906] Trying to unpack rootfs image as initramfs...
[    0.216457] Initialise system trusted keyrings
[    0.216471] Key type blacklist registered
[    0.216524] workingset: timestamp_bits=3D41 max_order=3D21 bucket_order=
=3D0
[    0.219140] zbud: loaded
[    0.232653] Key type asymmetric registered
[    0.232658] Asymmetric key parser 'x509' registered
[    0.267448] Freeing initrd memory: 12668K
[    0.270129] alg: self-tests for CTR-KDF (hmac(sha256)) passed
[    0.270176] Block layer SCSI generic (bsg) driver version 0.4 loaded (ma=
jor 243)
[    0.270210] io scheduler mq-deadline registered
[    0.270212] io scheduler kyber registered
[    0.270244] io scheduler bfq registered
[    0.270890] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    0.271449] ACPI: AC: AC Adapter [AC0] (on-line)
[    0.271522] input: Lid Switch as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0=
D:00/input/input0
[    0.271539] ACPI: button: Lid Switch [LID]
[    0.271566] input: Sleep Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0=
C0E:00/input/input1
[    0.271600] ACPI: button: Sleep Button [SLPB]
[    0.271628] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/inpu=
t/input2
[    0.271650] ACPI: button: Power Button [PWRF]
[    0.275046] thermal LNXTHERM:00: registered as thermal_zone0
[    0.275049] ACPI: thermal: Thermal Zone [THRM] (35 C)
[    0.275284] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[    0.279386] hpet_acpi_add: no address or irqs in _CRS
[    0.279446] Non-volatile memory driver v1.3
[    0.279447] Linux agpgart interface v0.103
[    0.279449] AMD-Vi: AMD IOMMUv2 functionality not available on this syst=
em - This is not a bug.
[    0.280498] ahci 0000:00:1f.2: version 3.0
[    0.280737] ahci 0000:00:1f.2: SSS flag set, parallel bus scan disabled
[    0.290839] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 4 ports 6 Gbps 0x=
1 impl SATA mode
[    0.290844] ahci 0000:00:1f.2: flags: 64bit ncq stag pm led clo only pio=
 slum part deso sadm sds apst=20
[    0.291281] scsi host0: ahci
[    0.291546] scsi host1: ahci
[    0.291701] scsi host2: ahci
[    0.291796] scsi host3: ahci
[    0.291836] ata1: SATA max UDMA/133 abar m2048@0xf7d22000 port 0xf7d2210=
0 irq 40
[    0.291838] ata2: DUMMY
[    0.291839] ata3: DUMMY
[    0.291840] ata4: DUMMY
[    0.291884] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.291888] ehci-pci: EHCI PCI platform driver
[    0.292035] ehci-pci 0000:00:1d.0: EHCI Host Controller
[    0.292041] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus =
number 1
[    0.292053] ehci-pci 0000:00:1d.0: debug port 2
[    0.295977] ehci-pci 0000:00:1d.0: irq 23, io mem 0xf7d23000
[    0.309237] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00
[    0.309293] usb usb1: New USB device found, idVendor=3D1d6b, idProduct=
=3D0002, bcdDevice=3D 5.16
[    0.309296] usb usb1: New USB device strings: Mfr=3D3, Product=3D2, Seri=
alNumber=3D1
[    0.309298] usb usb1: Product: EHCI Host Controller
[    0.309299] usb usb1: Manufacturer: Linux 5.16.0-rc5-next-20211217-debug=
 ehci_hcd
[    0.309301] usb usb1: SerialNumber: 0000:00:1d.0
[    0.309417] hub 1-0:1.0: USB hub found
[    0.309423] hub 1-0:1.0: 2 ports detected
[    0.309592] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.309596] ohci-pci: OHCI PCI platform driver
[    0.309605] uhci_hcd: USB Universal Host Controller Interface driver
[    0.309639] usbcore: registered new interface driver usbserial_generic
[    0.309643] usbserial: USB Serial support registered for generic
[    0.309742] rtc_cmos 00:02: RTC can wake from S4
[    0.309980] rtc_cmos 00:02: registered as rtc0
[    0.310015] rtc_cmos 00:02: setting system clock to 2021-12-17T21:56:55 =
UTC (1639778215)
[    0.310030] rtc_cmos 00:02: alarms up to one month, y3k, 242 bytes nvram
[    0.310168] intel_pstate: Intel P-state driver initializing
[    0.310407] ledtrig-cpu: registered to indicate activity on CPUs
[    0.310523] efifb: probing for efifb
[    0.310531] efifb: framebuffer at 0xe0000000, using 4128k, total 4128k
[    0.310533] efifb: mode is 1366x768x32, linelength=3D5504, pages=3D1
[    0.310535] efifb: scrolling: redraw
[    0.310535] efifb: Truecolor: size=3D8:8:8:8, shift=3D24:16:8:0
[    0.310573] fbcon: Deferring console take-over
[    0.310574] fb0: EFI VGA frame buffer device
[    0.310589] hid: raw HID events driver (C) Jiri Kosina
[    0.310660] drop_monitor: Initializing network drop monitor service
[    0.310724] Initializing XFRM netlink socket
[    0.310818] NET: Registered PF_INET6 protocol family
[    0.315634] Segment Routing with IPv6
[    0.315637] RPL Segment Routing with IPv6
[    0.315646] In-situ OAM (IOAM) with IPv6
[    0.315670] NET: Registered PF_PACKET protocol family
[    0.315916] microcode: sig=3D0x40651, pf=3D0x40, revision=3D0x26
[    0.315930] microcode: Microcode Update Driver: v2.2.
[    0.315937] IPI shorthand broadcast: enabled
[    0.315952] sched_clock: Marking stable (315158161, 734865)->(323371507,=
 -7478481)
[    0.316118] registered taskstats version 1
[    0.316188] Loading compiled-in X.509 certificates
[    0.323398] ACPI: battery: Slot [BAT0] (battery present)
[    0.325529] Loaded X.509 cert 'Build time autogenerated kernel key: 894e=
a34ad0509d87200dab462aa75a4a6d8980bd'
[    0.325756] zswap: loaded using pool lz4/z3fold
[    0.325921] Key type ._fscrypt registered
[    0.325923] Key type .fscrypt registered
[    0.325924] Key type fscrypt-provisioning registered
[    0.327157] PM:   Magic number: 1:435:955
[    0.327221] memory memory8: hash matches
[    0.327324] RAS: Correctable Errors collector initialized.
[    0.559285] usb 1-1: new high-speed USB device number 2 using ehci-pci
[    0.606325] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    0.607144] ata1.00: ATA-10: WD easystore 240GB, EF430400, max UDMA/133
[    0.607150] ata1.00: 468862128 sectors, multi 1: LBA48 NCQ (depth 32)
[    0.608990] ata1.00: Features: Dev-Sleep
[    0.614493] ata1.00: configured for UDMA/133
[    0.625138] scsi 0:0:0:0: Direct-Access     ATA      WD easystore 240 04=
00 PQ: 0 ANSI: 5
[    0.625812] sd 0:0:0:0: [sda] 468862128 512-byte logical blocks: (240 GB=
/224 GiB)
[    0.625833] sd 0:0:0:0: [sda] Write Protect is off
[    0.625838] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    0.625862] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled,=
 doesn't support DPO or FUA
[    0.653109]  sda: sda1 sda2
[    0.669369] sd 0:0:0:0: [sda] Attached SCSI disk
[    0.670485] Freeing unused decrypted memory: 2036K
[    0.670773] Freeing unused kernel image (initmem) memory: 1784K
[    0.670816] Write protecting the kernel read-only data: 28672k
[    0.671304] Freeing unused kernel image (text/rodata gap) memory: 2040K
[    0.671615] Freeing unused kernel image (rodata/data gap) memory: 1744K
[    0.706302] usb 1-1: New USB device found, idVendor=3D8087, idProduct=3D=
8000, bcdDevice=3D 0.04
[    0.706307] usb 1-1: New USB device strings: Mfr=3D0, Product=3D0, Seria=
lNumber=3D0
[    0.706591] hub 1-1:1.0: USB hub found
[    0.706674] hub 1-1:1.0: 8 ports detected
[    0.729423] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[    0.729427] rodata_test: all tests were successful
[    0.729429] x86/mm: Checking user space page tables
[    0.778523] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[    0.778527] Run /init as init process
[    0.778528]   with arguments:
[    0.778529]     /init
[    0.778530]   with environment:
[    0.778530]     HOME=3D/
[    0.778531]     TERM=3Dlinux
[    0.793317] fbcon: Taking over console
[    0.793384] Console: switching to colour frame buffer device 170x48
[    0.917349] xhci_hcd 0000:00:14.0: xHCI Host Controller
[    0.917361] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus =
number 2
[    0.918419] i8042: PNP: PS/2 Controller [PNP030b:PS2K,PNP0f03:PS2M] at 0=
x60,0x64 irq 1,12
[    0.918441] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x1=
00 quirks 0x000000000004b810
[    0.918635] usb usb2: New USB device found, idVendor=3D1d6b, idProduct=
=3D0002, bcdDevice=3D 5.16
[    0.918640] usb usb2: New USB device strings: Mfr=3D3, Product=3D2, Seri=
alNumber=3D1
[    0.918642] usb usb2: Product: xHCI Host Controller
[    0.918645] usb usb2: Manufacturer: Linux 5.16.0-rc5-next-20211217-debug=
 xhci-hcd
[    0.918647] usb usb2: SerialNumber: 0000:00:14.0
[    0.918949] hub 2-0:1.0: USB hub found
[    0.918966] hub 2-0:1.0: 9 ports detected
[    0.920723] i8042: Detected active multiplexing controller, rev 1.1
[    0.922409] serio: i8042 KBD port at 0x60,0x64 irq 1
[    0.922465] serio: i8042 AUX0 port at 0x60,0x64 irq 12
[    0.922500] serio: i8042 AUX1 port at 0x60,0x64 irq 12
[    0.922536] serio: i8042 AUX2 port at 0x60,0x64 irq 12
[    0.922578] serio: i8042 AUX3 port at 0x60,0x64 irq 12
[    0.923653] xhci_hcd 0000:00:14.0: xHCI Host Controller
[    0.923660] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus =
number 3
[    0.923665] xhci_hcd 0000:00:14.0: Host supports USB 3.0 SuperSpeed
[    0.923722] usb usb3: New USB device found, idVendor=3D1d6b, idProduct=
=3D0003, bcdDevice=3D 5.16
[    0.923727] usb usb3: New USB device strings: Mfr=3D3, Product=3D2, Seri=
alNumber=3D1
[    0.923730] usb usb3: Product: xHCI Host Controller
[    0.923732] usb usb3: Manufacturer: Linux 5.16.0-rc5-next-20211217-debug=
 xhci-hcd
[    0.923734] usb usb3: SerialNumber: 0000:00:14.0
[    0.923872] hub 3-0:1.0: USB hub found
[    0.923883] hub 3-0:1.0: 4 ports detected
[    0.933001] random: fast init done
[    0.992593] usb 1-1.3: new high-speed USB device number 3 using ehci-pci
[    1.007071] input: AT Translated Set 2 keyboard as /devices/platform/i80=
42/serio0/input/input3
[    1.105426] usb 1-1.3: New USB device found, idVendor=3D0b95, idProduct=
=3D772b, bcdDevice=3D 0.01
[    1.105432] usb 1-1.3: New USB device strings: Mfr=3D1, Product=3D2, Ser=
ialNumber=3D3
[    1.105434] usb 1-1.3: Product: AX88772B
[    1.105435] usb 1-1.3: Manufacturer: ASIX Elec. Corp.
[    1.105436] usb 1-1.3: SerialNumber: 12AC31
[    1.150812] SGI XFS with ACLs, security attributes, realtime, scrub, rep=
air, quota, no debug enabled
[    1.153002] XFS (sda2): Mounting V5 Filesystem
[    1.179246] usb 1-1.4: new full-speed USB device number 4 using ehci-pci
[    1.184508] XFS (sda2): Ending clean mount
[    1.188958] xfs filesystem being mounted at /new_root supports timestamp=
s until 2038 (0x7fffffff)
[    1.232609] tsc: Refined TSC clocksource calibration: 2394.457 MHz
[    1.232616] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x228=
3c360c9e, max_idle_ns: 440795302742 ns
[    1.232633] clocksource: Switched to clocksource tsc
[    1.281433] usb 1-1.4: New USB device found, idVendor=3D8087, idProduct=
=3D07dc, bcdDevice=3D 0.01
[    1.281439] usb 1-1.4: New USB device strings: Mfr=3D0, Product=3D0, Ser=
ialNumber=3D0
[    1.352575] usb 1-1.5: new high-speed USB device number 5 using ehci-pci
[    1.438770] random: crng init done
[    1.438781] systemd[1]: Successfully credited entropy passed from boot l=
oader.
[    1.442374] systemd[1]: systemd 249.7-2-arch running in system mode (+PA=
M +AUDIT -SELINUX -APPARMOR -IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +=
ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LI=
BFDISK +PCRE2 -PWQUALITY +P11KIT -QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +XKB=
COMMON +UTMP -SYSVINIT default-hierarchy=3Dunified)
[    1.456050] systemd[1]: Detected architecture x86-64.
[    1.459568] systemd[1]: Hostname set to <asus>.
[    1.475918] usb 1-1.5: New USB device found, idVendor=3D064e, idProduct=
=3D9700, bcdDevice=3D 1.01
[    1.475924] usb 1-1.5: New USB device strings: Mfr=3D3, Product=3D1, Ser=
ialNumber=3D2
[    1.475926] usb 1-1.5: Product: USB2.0 UVC HD Webcam
[    1.475927] usb 1-1.5: Manufacturer: SuYin
[    1.475929] usb 1-1.5: SerialNumber: HF1019-T838-SN03-Rev0101
[    1.532067] systemd-fstab-generator[220]: Mount point  is not a valid pa=
th, ignoring.
[    1.532698] systemd-fstab-generator[220]: Mount point  is not a valid pa=
th, ignoring.
[    1.552588] usb 1-1.7: new full-speed USB device number 6 using ehci-pci
[    1.651670] usb 1-1.7: New USB device found, idVendor=3D03eb, idProduct=
=3D8a41, bcdDevice=3D10.32
[    1.651676] usb 1-1.7: New USB device strings: Mfr=3D1, Product=3D0, Ser=
ialNumber=3D0
[    1.651678] usb 1-1.7: Manufacturer: Atmel
[    1.685307] systemd[1]: Queued start job for default target Graphical In=
terface.
[    1.686368] systemd[1]: Created slice Slice /system/dhcpcd.
[    1.686731] systemd[1]: Created slice Slice /system/getty.
[    1.687062] systemd[1]: Created slice Slice /system/modprobe.
[    1.687312] systemd[1]: Created slice User and Session Slice.
[    1.687440] systemd[1]: Started Dispatch Password Requests to Console Di=
rectory Watch.
[    1.687561] systemd[1]: Started Forward Password Requests to Wall Direct=
ory Watch.
[    1.687765] systemd[1]: Set up automount Arbitrary Executable File Forma=
ts File System Automount Point.
[    1.687868] systemd[1]: Reached target Local Encrypted Volumes.
[    1.687931] systemd[1]: Reached target Login Prompts.
[    1.688002] systemd[1]: Reached target Path Units.
[    1.688058] systemd[1]: Reached target Remote File Systems.
[    1.688112] systemd[1]: Reached target Slice Units.
[    1.688172] systemd[1]: Reached target Swaps.
[    1.688226] systemd[1]: Reached target Local Verity Protected Volumes.
[    1.688345] systemd[1]: Listening on Device-mapper event daemon FIFOs.
[    1.689662] systemd[1]: Listening on Process Core Dump Socket.
[    1.693095] systemd[1]: Listening on Journal Audit Socket.
[    1.693272] systemd[1]: Listening on Journal Socket (/dev/log).
[    1.693443] systemd[1]: Listening on Journal Socket.
[    1.697578] systemd[1]: Listening on udev Control Socket.
[    1.697744] systemd[1]: Listening on udev Kernel Socket.
[    1.698585] systemd[1]: Mounting Huge Pages File System...
[    1.699431] systemd[1]: Mounting POSIX Message Queue File System...
[    1.700286] systemd[1]: Mounting Kernel Debug File System...
[    1.701118] systemd[1]: Mounting Kernel Trace File System...
[    1.702133] systemd[1]: Starting Create List of Static Device Nodes...
[    1.703133] systemd[1]: Starting Load Kernel Module configfs...
[    1.704493] systemd[1]: Starting Load Kernel Module drm...
[    1.705999] systemd[1]: Starting Load Kernel Module fuse...
[    1.706284] systemd[1]: Condition check resulted in Set Up Additional Bi=
nary Formats being skipped.
[    1.706359] systemd[1]: Condition check resulted in File System Check on=
 Root Device being skipped.
[    1.708502] systemd[1]: Starting Journal Service...
[    1.710122] systemd[1]: Starting Load Kernel Modules...
[    1.711349] systemd[1]: Starting Remount Root and Kernel File Systems...
[    1.711562] systemd[1]: Condition check resulted in Repartition Root Dis=
k being skipped.
[    1.712892] systemd[1]: Starting Coldplug All udev Devices...
[    1.715245] systemd[1]: Mounted Huge Pages File System.
[    1.715507] systemd[1]: Mounted POSIX Message Queue File System.
[    1.716547] systemd[1]: Mounted Kernel Debug File System.
[    1.717575] systemd[1]: Mounted Kernel Trace File System.
[    1.718896] systemd[1]: Finished Create List of Static Device Nodes.
[    1.720376] systemd[1]: modprobe@configfs.service: Deactivated successfu=
lly.
[    1.720705] systemd[1]: Finished Load Kernel Module configfs.
[    1.722434] systemd[1]: Mounting Kernel Configuration File System...
[    1.724722] systemd[1]: Mounted Kernel Configuration File System.
[    1.726705] xfs filesystem being remounted at / supports timestamps unti=
l 2038 (0x7fffffff)
[    1.727556] systemd[1]: Finished Remount Root and Kernel File Systems.
[    1.728308] systemd[1]: Condition check resulted in First Boot Wizard be=
ing skipped.
[    1.736148] Asymmetric key parser 'pkcs8' registered
[    1.737165] systemd[1]: Condition check resulted in Rebuild Hardware Dat=
abase being skipped.
[    1.738366] systemd[1]: Starting Load/Save Random Seed...
[    1.740678] systemd[1]: Starting Create System Users...
[    1.742409] systemd[1]: Finished Load Kernel Modules.
[    1.743434] audit: type=3D1130 audit(1639778216.929:2): pid=3D1 uid=3D0 =
auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dsystemd-modules-load comm=
=3D"systemd" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? termina=
l=3D? res=3Dsuccess'
[    1.744854] systemd[1]: Starting Apply Kernel Variables...
[    1.750052] fuse: init (API version 7.36)
[    1.751127] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[    1.751482] systemd[1]: Finished Load Kernel Module fuse.
[    1.752649] audit: type=3D1130 audit(1639778216.939:3): pid=3D1 uid=3D0 =
auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dmodprobe@fuse comm=3D"syst=
emd" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? terminal=3D? re=
s=3Dsuccess'
[    1.752683] audit: type=3D1131 audit(1639778216.939:4): pid=3D1 uid=3D0 =
auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dmodprobe@fuse comm=3D"syst=
emd" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? terminal=3D? re=
s=3Dsuccess'
[    1.754072] systemd[1]: Mounting FUSE Control File System...
[    1.759203] systemd[1]: Mounted FUSE Control File System.
[    1.773901] systemd[1]: Finished Apply Kernel Variables.
[    1.775861] audit: type=3D1130 audit(1639778216.959:5): pid=3D1 uid=3D0 =
auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dsystemd-sysctl comm=3D"sys=
temd" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? terminal=3D? r=
es=3Dsuccess'
[    1.776475] systemd[1]: Finished Load/Save Random Seed.
[    1.777755] audit: type=3D1130 audit(1639778216.962:6): pid=3D1 uid=3D0 =
auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dsystemd-random-seed comm=
=3D"systemd" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? termina=
l=3D? res=3Dsuccess'
[    1.777983] systemd[1]: Condition check resulted in First Boot Complete =
being skipped.
[    1.779834] ACPI: bus type drm_connector registered
[    1.781274] systemd[1]: modprobe@drm.service: Deactivated successfully.
[    1.781618] systemd[1]: Finished Load Kernel Module drm.
[    1.782775] audit: type=3D1130 audit(1639778216.969:7): pid=3D1 uid=3D0 =
auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dmodprobe@drm comm=3D"syste=
md" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? terminal=3D? res=
=3Dsuccess'
[    1.782812] audit: type=3D1131 audit(1639778216.969:8): pid=3D1 uid=3D0 =
auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dmodprobe@drm comm=3D"syste=
md" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? terminal=3D? res=
=3Dsuccess'
[    1.785702] systemd[1]: Started Journal Service.
[    1.787009] audit: type=3D1130 audit(1639778216.972:9): pid=3D1 uid=3D0 =
auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dsystemd-journald comm=3D"s=
ystemd" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? terminal=3D?=
 res=3Dsuccess'
[    1.798881] systemd-journald[235]: Received client request to flush runt=
ime journal.
[    1.805966] audit: type=3D1130 audit(1639778216.992:10): pid=3D1 uid=3D0=
 auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dsystemd-sysusers comm=3D"=
systemd" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? terminal=3D=
? res=3Dsuccess'
[    2.213927] input: Asus Wireless Radio Control as /devices/LNXSYSTM:00/L=
NXSYBUS:00/ATK4002:00/input/input8
[    2.290211] Userspace governor deprecated: use thermal netlink notificat=
ion instead
[    2.295877] i801_smbus 0000:00:1f.3: SPD Write Disable is set
[    2.295922] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt
[    2.299286] i2c i2c-1: 2/4 memory slots populated (from DMI)
[    2.300171] i2c i2c-1: Successfully instantiated SPD at 0x50
[    2.409398] inv-mpu6050-i2c i2c-INVN6500:00: mounting matrix not found: =
using identity...
[    2.409406] inv-mpu6050-i2c i2c-INVN6500:00: supply vdd not found, using=
 dummy regulator
[    2.409448] inv-mpu6050-i2c i2c-INVN6500:00: supply vddio not found, usi=
ng dummy regulator
[    2.471803] mc: Linux media interface: v0.10
[    2.514666] hid-generic 0003:03EB:8A41.0001: hiddev96,hidraw0: USB HID v=
1.11 Device [Atmel] on usb-0000:00:1d.0-1.7/input0
[    2.526213] input: Atmel as /devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-=
1.7/1-1.7:1.1/0003:03EB:8A41.0002/input/input9
[    2.527576] hid-generic 0003:03EB:8A41.0002: input,hidraw1: USB HID v1.1=
1 Device [Atmel] on usb-0000:00:1d.0-1.7/input1
[    2.527635] usbcore: registered new interface driver usbhid
[    2.527638] usbhid: USB HID core driver
[    2.680110] videodev: Linux video capture interface: v2.00
[    2.723227] input: PC Speaker as /devices/platform/pcspkr/input/input11
[    2.723407] Bluetooth: Core ver 2.22
[    2.723437] NET: Registered PF_BLUETOOTH protocol family
[    2.723438] Bluetooth: HCI device and connection manager initialized
[    2.723443] Bluetooth: HCI socket layer initialized
[    2.723446] Bluetooth: L2CAP socket layer initialized
[    2.723450] Bluetooth: SCO socket layer initialized
[    2.724114] cfg80211: Loading compiled-in X.509 certificates for regulat=
ory database
[    2.726052] FAT-fs (sda1): Volume was not properly unmounted. Some data =
may be corrupt. Please run fsck.
[    2.738112] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[    2.741685] platform regulatory.0: Direct firmware load for regulatory.d=
b failed with error -2
[    2.741693] cfg80211: failed to load regulatory.db
[    2.847868] input: FocalTechPS/2 FocalTech Touchpad as /devices/platform=
/i8042/serio4/input/input14
[    2.899183] Intel(R) Wireless WiFi driver for Linux
[    2.923164] i2c i2c-0: Added multiplexed i2c bus 2
[    2.927092] iwlwifi 0000:02:00.0: loaded firmware version 17.3216344376.=
0 7260-17.ucode op_mode iwlmvm
[    2.966887] iTCO_vendor_support: vendor-support=3D0
[    3.001080] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC3236: l=
ine_outs=3D1 (0x14/0x0/0x0/0x0/0x0) type:speaker
[    3.001088] snd_hda_codec_realtek hdaudioC1D0:    speaker_outs=3D0 (0x0/=
0x0/0x0/0x0/0x0)
[    3.001093] snd_hda_codec_realtek hdaudioC1D0:    hp_outs=3D1 (0x21/0x0/=
0x0/0x0/0x0)
[    3.001096] snd_hda_codec_realtek hdaudioC1D0:    mono: mono_out=3D0x0
[    3.001099] snd_hda_codec_realtek hdaudioC1D0:    inputs:
[    3.001101] snd_hda_codec_realtek hdaudioC1D0:      Mic=3D0x12
[    3.015964] RAPL PMU: API unit is 2^-32 Joules, 4 fixed counters, 655360=
 ms ovfl timer
[    3.015970] RAPL PMU: hw unit of domain pp0-core 2^-14 Joules
[    3.015972] RAPL PMU: hw unit of domain package 2^-14 Joules
[    3.015974] RAPL PMU: hw unit of domain dram 2^-14 Joules
[    3.015975] RAPL PMU: hw unit of domain pp1-gpu 2^-14 Joules
[    3.046856] input: HDA Intel PCH Headphone as /devices/pci0000:00/0000:0=
0:1b.0/sound/card1/input15
[    3.091418] intel_rapl_common: Found RAPL domain package
[    3.091423] intel_rapl_common: Found RAPL domain core
[    3.091425] intel_rapl_common: Found RAPL domain uncore
[    3.091427] intel_rapl_common: Found RAPL domain dram
[    3.091433] intel_rapl_common: RAPL package-0 domain package locked by B=
IOS
[    3.091442] intel_rapl_common: RAPL package-0 domain dram locked by BIOS
[    3.093293] at24 1-0050: supply vcc not found, using dummy regulator
[    3.093527] usbcore: registered new interface driver btusb
[    3.093932] at24 1-0050: 256 byte spd EEPROM, read-only
[    3.119292] Bluetooth: hci0: Legacy ROM 2.5 revision 8.0 build 1 week 45=
 2013
[    3.130852] Bluetooth: hci0: Intel Bluetooth firmware file: intel/ibt-hw=
-37.7.10-fw-1.80.1.2d.d.bseq
[    3.151217] iwlwifi 0000:02:00.0: Detected Intel(R) Dual Band Wireless A=
C 7260, REV=3D0x144
[    3.152518] checking generic (e0000000 408000) vs hw (f7800000 400000)
[    3.152523] checking generic (e0000000 408000) vs hw (e0000000 10000000)
[    3.152526] fb0: switching to i915 from EFI VGA
[    3.152703] Console: switching to colour dummy device 80x25
[    3.152757] i915 0000:00:02.0: vgaarb: deactivate vga console
[    3.153397] cryptd: max_cpu_qlen set to 1000
[    3.154338] mousedev: PS/2 mouse device common for all mice
[    3.154500] usb 1-1.5: Found UVC 1.00 device USB2.0 UVC HD Webcam (064e:=
9700)
[    3.161974] i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=
=3Dio+mem,decodes=3Dio+mem:owns=3Dio+mem
[    3.170888] iwlwifi 0000:02:00.0: base HW address: a0:a8:cd:7a:2c:81
[    3.175918] input: USB2.0 UVC HD Webcam: USB2.0 UV as /devices/pci0000:0=
0/0000:00:1d.0/usb1/1-1/1-1.5/1-1.5:1.0/input/input16
[    3.186137] asus_wmi: ASUS WMI generic driver loaded
[    3.195864] iTCO_wdt iTCO_wdt.1.auto: Found a Lynx Point_LP TCO device (=
Version=3D2, TCOBASE=3D0x1860)
[    3.196207] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=3D30 sec (n=
owayout=3D0)
[    3.202648] asus_wmi: Initialization: 0x1
[    3.202749] asus_wmi: BIOS WMI version: 7.9
[    3.202894] asus_wmi: SFUN value: 0x4a2877
[    3.202898] asus-nb-wmi asus-nb-wmi: Detected ATK, not ASUSWMI, use DSTS
[    3.202902] asus-nb-wmi asus-nb-wmi: Detected ATK, enable event queue
[    3.203398] AVX2 version of gcm_enc/dec engaged.
[    3.203441] AES CTR mode by8 optimization enabled
[    3.212764] intel-spi intel-spi: s25fl064k (8192 Kbytes)
[    3.214523] input: Asus WMI hotkeys as /devices/platform/asus-nb-wmi/inp=
ut/input17
[    3.218843] Creating 1 MTD partitions on "intel-spi":
[    3.218850] 0x000000000000-0x000000800000 : "BIOS"
[    3.234043] usbcore: registered new interface driver uvcvideo
[    3.234713] input: Atmel as /devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-=
1.7/1-1.7:1.1/0003:03EB:8A41.0002/input/input18
[    3.235192] hid-multitouch 0003:03EB:8A41.0002: input,hidraw1: USB HID v=
1.11 Device [Atmel] on usb-0000:00:1d.0-1.7/input1
[    3.245167] [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on mi=
nor 0
[    3.274091] ACPI: video: Video Device [GFX0] (multi-head: yes  rom: no  =
post: no)
[    3.280624] input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08=
:00/LNXVIDEO:00/input/input19
[    3.280775] snd_hda_intel 0000:00:03.0: bound 0000:00:02.0 (ops i915_aud=
io_component_bind_ops [i915])
[    3.284153] fbcon: i915drmfb (fb0) is primary device
[    3.294057] xfs filesystem being remounted at /run/systemd/unit-root/etc=
/iwd supports timestamps until 2038 (0x7fffffff)
[    3.294128] xfs filesystem being remounted at /run/systemd/unit-root/var=
/lib/iwd supports timestamps until 2038 (0x7fffffff)
[    3.294195] xfs filesystem being remounted at /run/systemd/unit-root/var=
/tmp supports timestamps until 2038 (0x7fffffff)
[    3.296212] xfs filesystem being remounted at /run/systemd/unit-root/etc=
/iwd supports timestamps until 2038 (0x7fffffff)
[    3.307342] xfs filesystem being remounted at /run/systemd/unit-root/var=
/lib/iwd supports timestamps until 2038 (0x7fffffff)
[    3.308143] xfs filesystem being remounted at /run/systemd/unit-root/var=
/tmp supports timestamps until 2038 (0x7fffffff)
[    3.310111] xfs filesystem being remounted at /run/systemd/unit-root/var=
/lib/systemd/linger supports timestamps until 2038 (0x7fffffff)
[    3.310255] xfs filesystem being remounted at /run/systemd/unit-root/etc=
 supports timestamps until 2038 (0x7fffffff)
[    3.310408] xfs filesystem being remounted at /run/systemd/unit-root/var=
/tmp supports timestamps until 2038 (0x7fffffff)
[    3.313285] Bluetooth: hci0: unexpected event for opcode 0xfc2f
[    3.314286] Bluetooth: hci0: unexpected event 0xff length: 2 > 0
[    3.316168] xfs filesystem being remounted at /run/systemd/unit-root/etc=
 supports timestamps until 2038 (0x7fffffff)
[    3.330298] Bluetooth: hci0: Intel BT fw patch 0x2a completed & activated
[    3.339210] xfs filesystem being remounted at /run/systemd/unit-root/var=
/lib/systemd/linger supports timestamps until 2038 (0x7fffffff)
[    3.340631] xfs filesystem being remounted at /run/systemd/unit-root/var=
/tmp supports timestamps until 2038 (0x7fffffff)
[    3.343980] NET: Registered PF_ALG protocol family
[    3.382281] ieee80211 phy0: Selected rate control algorithm 'iwl-mvm-rs'
[    3.604052] libphy: Asix MDIO Bus: probed
[    3.670778] Asix Electronics AX88772C usb-001:003:10: attached PHY drive=
r (mii_bus:phy_addr=3Dusb-001:003:10, irq=3DPOLL)
[    3.670973] asix 1-1.3:1.0 eth0: register 'asix' at usb-0000:00:1d.0-1.3=
, ASIX AX88772B USB 2.0 Ethernet, 80:6d:97:12:ac:31
[    3.671034] usbcore: registered new interface driver asix
[    3.674892] asix 1-1.3:1.0 enp0s29u1u3: renamed from eth0
[    4.435435] Console: switching to colour frame buffer device 170x48
[    4.452282] i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device
[    4.499174] input: HDA Intel HDMI HDMI/DP,pcm=3D3 as /devices/pci0000:00=
/0000:00:03.0/sound/card0/input20
[    4.499262] input: HDA Intel HDMI HDMI/DP,pcm=3D7 as /devices/pci0000:00=
/0000:00:03.0/sound/card0/input21
[    4.499343] input: HDA Intel HDMI HDMI/DP,pcm=3D8 as /devices/pci0000:00=
/0000:00:03.0/sound/card0/input22
[    4.499440] input: HDA Intel HDMI HDMI/DP,pcm=3D9 as /devices/pci0000:00=
/0000:00:03.0/sound/card0/input23
[    4.499522] input: HDA Intel HDMI HDMI/DP,pcm=3D10 as /devices/pci0000:0=
0/0000:00:03.0/sound/card0/input24
[    4.552334] 8021q: 802.1Q VLAN Support v1.8
[    4.825281] asix 1-1.3:1.0 enp0s29u1u3: Link is Down
[    6.864741] IPv6: ADDRCONF(NETDEV_CHANGE): enp0s29u1u3: link becomes rea=
dy
[    6.864821] asix 1-1.3:1.0 enp0s29u1u3: Link is Up - 100Mbps/Full - flow=
 control off
[    8.401734] kauditd_printk_skb: 23 callbacks suppressed
[    8.401743] audit: type=3D1131 audit(1639778223.585:34): pid=3D1 uid=3D0=
 auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dsystemd-rfkill comm=3D"sy=
stemd" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? terminal=3D? =
res=3Dsuccess'
[   12.457912] audit: type=3D1130 audit(1639778227.642:35): pid=3D1 uid=3D0=
 auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Ddhcpcd@enp0s29u1u3 comm=
=3D"systemd" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? termina=
l=3D? res=3Dsuccess'
[   12.462623] audit: type=3D1130 audit(1639778227.649:36): pid=3D1 uid=3D0=
 auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dsshd comm=3D"systemd" exe=
=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? terminal=3D? res=3Dsucc=
ess'
[   12.470107] audit: type=3D1130 audit(1639778227.655:37): pid=3D1 uid=3D0=
 auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dsystemd-user-sessions com=
m=3D"systemd" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? termin=
al=3D? res=3Dsuccess'
[   12.554056] audit: type=3D1130 audit(1639778227.739:38): pid=3D1 uid=3D0=
 auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dntpd comm=3D"systemd" exe=
=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? terminal=3D? res=3Dsucc=
ess'
[   12.558887] audit: type=3D1130 audit(1639778227.742:39): pid=3D1 uid=3D0=
 auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Dlightdm comm=3D"systemd" =
exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? terminal=3D? res=3Ds=
uccess'
[   13.656521] audit: type=3D1103 audit(1639778228.842:40): pid=3D524 uid=
=3D0 auid=3D4294967295 ses=3D4294967295 msg=3D'op=3DPAM:setcred grantors=3D=
pam_env,pam_permit acct=3D"lightdm" exe=3D"/usr/bin/lightdm" hostname=3D? a=
ddr=3D? terminal=3D:0 res=3Dsuccess'
[   13.676160] audit: type=3D1130 audit(1639778228.862:41): pid=3D1 uid=3D0=
 auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Duser-runtime-dir@973 comm=
=3D"systemd" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? termina=
l=3D? res=3Dsuccess'
[   13.693912] audit: type=3D1101 audit(1639778228.879:42): pid=3D528 uid=
=3D0 auid=3D4294967295 ses=3D4294967295 msg=3D'op=3DPAM:accounting grantors=
=3Dpam_access,pam_unix,pam_permit,pam_time acct=3D"lightdm" exe=3D"/usr/lib=
/systemd/systemd" hostname=3D? addr=3D? terminal=3D? res=3Dsuccess'
[   13.693960] audit: type=3D1103 audit(1639778228.879:43): pid=3D528 uid=
=3D0 auid=3D4294967295 ses=3D4294967295 msg=3D'op=3DPAM:setcred grantors=3D=
? acct=3D"lightdm" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? t=
erminal=3D? res=3Dfailed'
[   13.694046] audit: type=3D1006 audit(1639778228.879:44): pid=3D528 uid=
=3D0 old-auid=3D4294967295 auid=3D973 tty=3D(none) old-ses=3D4294967295 ses=
=3D1 res=3D1
[   13.694049] audit: type=3D1300 audit(1639778228.879:44): arch=3Dc000003e=
 syscall=3D1 success=3Dyes exit=3D3 a0=3D9 a1=3D7ffe27ac2f20 a2=3D3 a3=3D3c=
d items=3D0 ppid=3D1 pid=3D528 auid=3D973 uid=3D0 gid=3D0 euid=3D0 suid=3D0=
 fsuid=3D0 egid=3D0 sgid=3D0 fsgid=3D0 tty=3D(none) ses=3D1 comm=3D"(system=
d)" exe=3D"/usr/lib/systemd/systemd" key=3D(null)
[   13.694052] audit: type=3D1327 audit(1639778228.879:44): proctitle=3D"(s=
ystemd)"
[   13.695553] audit: type=3D1105 audit(1639778228.879:45): pid=3D528 uid=
=3D0 auid=3D973 ses=3D1 msg=3D'op=3DPAM:session_open grantors=3Dpam_loginui=
d,pam_loginuid,pam_keyinit,pam_systemd_home,pam_limits,pam_unix,pam_permit,=
pam_mail,pam_systemd,pam_env acct=3D"lightdm" exe=3D"/usr/lib/systemd/syste=
md" hostname=3D? addr=3D? terminal=3D? res=3Dsuccess'
[   13.701906] audit: type=3D1334 audit(1639778228.885:46): prog-id=3D20 op=
=3DLOAD
[   13.701912] audit: type=3D1300 audit(1639778228.885:46): arch=3Dc000003e=
 syscall=3D321 success=3Dyes exit=3D8 a0=3D5 a1=3D7ffe1c961620 a2=3D78 a3=
=3D7ffe1c961620 items=3D0 ppid=3D1 pid=3D528 auid=3D973 uid=3D973 gid=3D973=
 euid=3D973 suid=3D973 fsuid=3D973 egid=3D973 sgid=3D973 fsgid=3D973 tty=3D=
(none) ses=3D1 comm=3D"systemd" exe=3D"/usr/lib/systemd/systemd" key=3D(nul=
l)
[   45.891048] kauditd_printk_skb: 4 callbacks suppressed
[   45.891052] audit: type=3D1101 audit(1639778261.075:50): pid=3D566 uid=
=3D0 auid=3D4294967295 ses=3D4294967295 msg=3D'op=3DPAM:accounting grantors=
=3Dpam_access,pam_unix,pam_permit,pam_time acct=3D"nathan" exe=3D"/usr/bin/=
sshd" hostname=3D192.168.4.54 addr=3D192.168.4.54 terminal=3Dssh res=3Dsucc=
ess'
[   45.893077] audit: type=3D1103 audit(1639778261.079:51): pid=3D566 uid=
=3D0 auid=3D4294967295 ses=3D4294967295 msg=3D'op=3DPAM:setcred grantors=3D=
pam_shells,pam_faillock,pam_permit,pam_env,pam_faillock acct=3D"nathan" exe=
=3D"/usr/bin/sshd" hostname=3D192.168.4.54 addr=3D192.168.4.54 terminal=3Ds=
sh res=3Dsuccess'
[   45.893127] audit: type=3D1006 audit(1639778261.079:52): pid=3D566 uid=
=3D0 old-auid=3D4294967295 auid=3D1000 tty=3D(none) old-ses=3D4294967295 se=
s=3D2 res=3D1
[   45.893132] audit: type=3D1300 audit(1639778261.079:52): arch=3Dc000003e=
 syscall=3D1 success=3Dyes exit=3D4 a0=3D3 a1=3D7fff98116f50 a2=3D4 a3=3D3e=
8 items=3D0 ppid=3D505 pid=3D566 auid=3D1000 uid=3D0 gid=3D0 euid=3D0 suid=
=3D0 fsuid=3D0 egid=3D0 sgid=3D0 fsgid=3D0 tty=3D(none) ses=3D2 comm=3D"ssh=
d" exe=3D"/usr/bin/sshd" key=3D(null)
[   45.893136] audit: type=3D1327 audit(1639778261.079:52): proctitle=3D737=
368643A206E617468616E205B707269765D
[   45.906886] audit: type=3D1130 audit(1639778261.092:53): pid=3D1 uid=3D0=
 auid=3D4294967295 ses=3D4294967295 msg=3D'unit=3Duser-runtime-dir@1000 com=
m=3D"systemd" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? termin=
al=3D? res=3Dsuccess'
[   45.913224] audit: type=3D1101 audit(1639778261.099:54): pid=3D569 uid=
=3D0 auid=3D4294967295 ses=3D4294967295 msg=3D'op=3DPAM:accounting grantors=
=3Dpam_access,pam_unix,pam_permit,pam_time acct=3D"nathan" exe=3D"/usr/lib/=
systemd/systemd" hostname=3D? addr=3D? terminal=3D? res=3Dsuccess'
[   45.913302] audit: type=3D1103 audit(1639778261.099:55): pid=3D569 uid=
=3D0 auid=3D4294967295 ses=3D4294967295 msg=3D'op=3DPAM:setcred grantors=3D=
? acct=3D"nathan" exe=3D"/usr/lib/systemd/systemd" hostname=3D? addr=3D? te=
rminal=3D? res=3Dfailed'
[   45.913408] audit: type=3D1006 audit(1639778261.099:56): pid=3D569 uid=
=3D0 old-auid=3D4294967295 auid=3D1000 tty=3D(none) old-ses=3D4294967295 se=
s=3D3 res=3D1
[   45.913412] audit: type=3D1300 audit(1639778261.099:56): arch=3Dc000003e=
 syscall=3D1 success=3Dyes exit=3D4 a0=3D9 a1=3D7ffe27ac2f20 a2=3D4 a3=3D3e=
8 items=3D0 ppid=3D1 pid=3D569 auid=3D1000 uid=3D0 gid=3D0 euid=3D0 suid=3D=
0 fsuid=3D0 egid=3D0 sgid=3D0 fsgid=3D0 tty=3D(none) ses=3D3 comm=3D"(syste=
md)" exe=3D"/usr/lib/systemd/systemd" key=3D(null)
[   59.424292] kauditd_printk_skb: 11 callbacks suppressed
[   59.424298] audit: type=3D1100 audit(1639778274.609:65): pid=3D780 uid=
=3D1000 auid=3D1000 ses=3D2 msg=3D'op=3DPAM:authentication grantors=3Dpam_f=
aillock,pam_permit,pam_faillock acct=3D"nathan" exe=3D"/usr/bin/doas" hostn=
ame=3Dasus addr=3D? terminal=3Dpts/0 res=3Dsuccess'
[   59.424553] audit: type=3D1101 audit(1639778274.609:66): pid=3D780 uid=
=3D1000 auid=3D1000 ses=3D2 msg=3D'op=3DPAM:accounting grantors=3Dpam_unix,=
pam_permit,pam_time acct=3D"nathan" exe=3D"/usr/bin/doas" hostname=3Dasus a=
ddr=3D? terminal=3Dpts/0 res=3Dsuccess'
[   59.424671] audit: type=3D1110 audit(1639778274.609:67): pid=3D780 uid=
=3D1000 auid=3D1000 ses=3D2 msg=3D'op=3DPAM:setcred grantors=3Dpam_faillock=
,pam_permit,pam_faillock acct=3D"root" exe=3D"/usr/bin/doas" hostname=3Dasu=
s addr=3D? terminal=3Dpts/0 res=3Dsuccess'
[   59.424890] audit: type=3D1105 audit(1639778274.609:68): pid=3D780 uid=
=3D1000 auid=3D1000 ses=3D2 msg=3D'op=3DPAM:session_open grantors=3Dpam_sys=
temd_home,pam_limits,pam_unix,pam_permit acct=3D"root" exe=3D"/usr/bin/doas=
" hostname=3Dasus addr=3D? terminal=3Dpts/0 res=3Dsuccess'
[   91.992495] BUG: kernel NULL pointer dereference, address: 0000000000000=
020
[   91.992503] #PF: supervisor read access in kernel mode
[   91.992506] #PF: error_code(0x0000) - not-present page
[   91.992508] PGD 0 P4D 0=20
[   91.992512] Oops: 0000 [#1] PREEMPT SMP PTI
[   91.992516] CPU: 0 PID: 878 Comm: cat Not tainted 5.16.0-rc5-next-202112=
17-debug #1 15ae5b0f28a4b9b6343440ee595affa8e1b5cf57
[   91.992520] Hardware name: ASUSTeK COMPUTER INC. Q302LA/Q302LA, BIOS Q30=
2LA.203 05/15/2014
[   91.992522] RIP: 0010:sgx_total_bytes_show+0x28/0x40
[   91.992530] Code: 66 90 0f 1f 44 00 00 49 89 f8 48 89 d7 48 c7 c6 5f b1 =
b2 89 41 8b 80 98 02 00 00 48 8d 04 40 48 c1 e0 04 48 03 05 38 b1 3c 02 <48=
> 8b 50 20 e8 ff ee 3b 00 48 98 31 d2 89 d6 89 d7 41 89 d0 c3 0f
[   91.992533] RSP: 0018:ffffaf6141367d38 EFLAGS: 00010246
[   91.992536] RAX: 0000000000000000 RBX: ffffffff8a230ec0 RCX: 00000000000=
00000
[   91.992539] RDX: ffff99d540eab000 RSI: ffffffff89b2b15f RDI: ffff99d540e=
ab000
[   91.992541] RBP: ffffffff897567c0 R08: ffff99d5402a8000 R09: 00000000000=
00000
[   91.992543] R10: 0000000000000000 R11: 0000000000000000 R12: ffffaf61413=
67e00
[   91.992545] R13: ffffaf6141367dd8 R14: ffff99d5422585c8 R15: 00000000000=
00001
[   91.992547] FS:  00007f6825ba2600(0000) GS:ffff99d656e00000(0000) knlGS:=
0000000000000000
[   91.992550] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   91.992553] CR2: 0000000000000020 CR3: 000000010d190003 CR4: 00000000001=
706f0
[   91.992555] Call Trace:
[   91.992558]  <TASK>
[   91.992560]  dev_attr_show+0x19/0x40
[   91.992568]  sysfs_kf_seq_show+0xa1/0x120
[   91.992577]  seq_read_iter+0x12e/0x4c0
[   91.992583]  ? __mod_memcg_lruvec_state+0x41/0x80
[   91.992592]  ? __mod_lruvec_page_state+0xa7/0x150
[   91.992599]  new_sync_read+0x159/0x1f0
[   91.992608]  vfs_read+0xff/0x1a0
[   91.992614]  ksys_read+0x67/0xf0
[   91.992619]  do_syscall_64+0x5c/0x90
[   91.992624]  ? do_user_addr_fault+0x1d7/0x6a0
[   91.992630]  ? do_syscall_64+0x69/0x90
[   91.992633]  ? exc_page_fault+0x72/0x180
[   91.992638]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[   91.992644] RIP: 0033:0x7f6825ac4862
[   91.992648] Code: c0 e9 b2 fe ff ff 50 48 8d 3d 5a 29 0a 00 e8 55 e4 01 =
00 0f 1f 44 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 0f 05 <48=
> 3d 00 f0 ff ff 77 56 c3 0f 1f 44 00 00 48 83 ec 28 48 89 54 24
[   91.992650] RSP: 002b:00007ffe0cefe458 EFLAGS: 00000246 ORIG_RAX: 000000=
0000000000
[   91.992654] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007f6825a=
c4862
[   91.992656] RDX: 0000000000020000 RSI: 00007f68256cb000 RDI: 00000000000=
00003
[   91.992658] RBP: 00007f68256cb000 R08: 00007f68256ca010 R09: 00000000000=
00000
[   91.992660] R10: 0000000000000022 R11: 0000000000000246 R12: 00000000000=
20000
[   91.992662] R13: 0000000000000003 R14: 0000000000000000 R15: 00007ffe0ce=
fe6f8
[   91.992666]  </TASK>
[   91.992668] Modules linked in: 8021q garp mrp stp llc snd_hda_codec_hdmi=
 ccm algif_aead ax88796b cbc des_generic libdes ecb algif_skcipher x86_pkg_=
temp_thermal intel_powerclamp cmac coretemp kvm_intel md4 algif_hash kvm af=
_alg irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel intel_spi_=
platform aesni_intel asus_nb_wmi asus_wmi joydev intel_spi iTCO_wdt crypto_=
simd iwlmvm spi_nor mousedev sparse_keymap hid_multitouch cryptd intel_pmc_=
bxt i915 at24 btusb platform_profile intel_rapl_msr mac80211 mtd ak8975 btr=
tl rapl snd_hda_codec_realtek uvcvideo asix iTCO_vendor_support libarc4 snd=
_hda_codec_generic ledtrig_audio videobuf2_vmalloc mei_hdcp btbcm snd_hda_i=
ntel selftests videobuf2_memops iwlwifi intel_cstate btintel i2c_algo_bit s=
nd_intel_dspcfg videobuf2_v4l2 intel_uncore usbnet ttm snd_intel_sdw_acpi b=
tmtk videobuf2_common vfat snd_hda_codec mii psmouse pcspkr drm_kms_helper =
snd_hda_core mdio_devres bluetooth fat processor_thermal_device_pci_legacy =
cfg80211 syscopyarea videodev
[   91.992742]  processor_thermal_device snd_hwdep sysfillrect usbhid libph=
y ecdh_generic processor_thermal_rfim sysimgblt crc16 mc rfkill snd_pcm wmi=
 fb_sys_fops processor_thermal_mbox inv_mpu6050_i2c snd_timer processor_the=
rmal_rapl cec inv_mpu6050 snd intel_rapl_common acpi_als dell_smo8800 indus=
trialio_triggered_buffer mei_me int3400_thermal intel_gtt i2c_i801 int340x_=
thermal_zone i2c_mux soundcore kfifo_buf video mei intel_soc_dts_iosf lpc_i=
ch acpi_thermal_rel mac_hid i2c_smbus asus_wireless soc_button_array indust=
rialio drm fuse pkcs8_key_parser bpf_preload ip_tables x_tables xfs libcrc3=
2c crc32c_generic serio_raw atkbd libps2 crc32c_intel i8042 xhci_pci xhci_p=
ci_renesas serio
[   91.992794] CR2: 0000000000000020
[   91.992824] ---[ end trace 0000000000000000 ]---
[   91.992826] RIP: 0010:sgx_total_bytes_show+0x28/0x40
[   91.992829] Code: 66 90 0f 1f 44 00 00 49 89 f8 48 89 d7 48 c7 c6 5f b1 =
b2 89 41 8b 80 98 02 00 00 48 8d 04 40 48 c1 e0 04 48 03 05 38 b1 3c 02 <48=
> 8b 50 20 e8 ff ee 3b 00 48 98 31 d2 89 d6 89 d7 41 89 d0 c3 0f
[   91.992831] RSP: 0018:ffffaf6141367d38 EFLAGS: 00010246
[   91.992833] RAX: 0000000000000000 RBX: ffffffff8a230ec0 RCX: 00000000000=
00000
[   91.992835] RDX: ffff99d540eab000 RSI: ffffffff89b2b15f RDI: ffff99d540e=
ab000
[   91.992836] RBP: ffffffff897567c0 R08: ffff99d5402a8000 R09: 00000000000=
00000
[   91.992838] R10: 0000000000000000 R11: 0000000000000000 R12: ffffaf61413=
67e00
[   91.992839] R13: ffffaf6141367dd8 R14: ffff99d5422585c8 R15: 00000000000=
00001
[   91.992841] FS:  00007f6825ba2600(0000) GS:ffff99d656e00000(0000) knlGS:=
0000000000000000
[   91.992843] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   91.992844] CR2: 0000000000000020 CR3: 000000010d190003 CR4: 00000000001=
706f0

--kO/GpfGXlbU26p1X--

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 8EDD9C433F5
	for <linux-kernel@archiver.kernel.org>; Tue, 28 Dec 2021 23:45:16 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S237848AbhL1XpP (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Tue, 28 Dec 2021 18:45:15 -0500
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38714 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S237810AbhL1XpN (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Tue, 28 Dec 2021 18:45:13 -0500
Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68105C061574;
        Tue, 28 Dec 2021 15:45:13 -0800 (PST)
Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140])
        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
        (No client certificate requested)
        by ams.source.kernel.org (Postfix) with ESMTPS id BE3D9B808CF;
        Tue, 28 Dec 2021 23:45:11 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id ED149C36AE9;
        Tue, 28 Dec 2021 23:45:09 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
        s=k20201202; t=1640735110;
        bh=7u7GV1qH6g+5RnFKcsD4IgyME2WTGsGMyR16yxq1efw=;
        h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
        b=TtqWnAYq+n2jlsdzL8GoUzKndGkmc40PorhcmXredy6bIlhzfYkt6BO1BMtAXtVz+
         EXu6H+Wyqw2+JOesXHd4KBxWWipwT5wuDBOxJPH9um+vdWPLsA+pyp2JpnbMnOflop
         FnTamg2EtKaIhVxk33b3DkHG5wPF7jtI2HqSCU6Qvu3qgSYRsemtCwQNpriTGgTOYa
         nvIzshTEWzyI6ek2LC9gGxU2yqf2CYyFyYIm2bYcCC8e4ppiCrjh+wYhneMLWAoQXA
         gYE/DsiyU5HvLNirZpUNjkcw8LWJ6cUhLaHQqqBeuC7/kB12qDJpuQOwzqJDNrud61
         onimBdoPb316Q==
Date:   Wed, 29 Dec 2021 01:45:08 +0200
From:   Jarkko Sakkinen <jarkko@kernel.org>
To:     Nathan Chancellor <nathan@kernel.org>
Cc:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
Message-ID: <YcuhhI2+k0XVuTb1@iki.fi>
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org>
 <YbzhBrimHGGpddDM@archlinux-ax161>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <YbzhBrimHGGpddDM@archlinux-ax161>
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Dec 17, 2021 at 12:12:06PM -0700, Nathan Chancellor wrote:
> Hi Jarkko,
> 
> On Tue, Nov 16, 2021 at 06:21:16PM +0200, Jarkko Sakkinen wrote:
> > The amount of SGX memory on the system is determined by the BIOS and it
> > varies wildly between systems.  It can be from dozens of MB's on desktops
> > or VM's, up to many GB's on servers.  Just like for regular memory, it is
> > sometimes useful to know the amount of usable SGX memory in the system.
> > 
> > Introduce CONFIG_HAVE_ARCH_NODE_DEV_GROUP opt-in flag to expose an arch
> > specific attribute group, and add an attribute for the amount of SGX
> > memory in bytes to each NUMA node:
> > 
> > /sys/devices/system/node/nodeX/x86/sgx_total_bytes
> > 
> > Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
> 
> Apologies if this has already been reported or fixed, I haven't seen
> anything from a full search of lore.kernel.org.
> 
> With this patch in -next as commit 50468e431335 ("x86/sgx: Add an
> attribute for the amount of SGX memory in a NUMA node"), this sysfs node
> causes an OOPS for me on at least one of my test systems (Intel based):
> 
> # cat /sys/devices/system/node/node0/x86/sgx_total_bytes
> fish: Job 1, 'cat /sys/devices/system/node/no…' terminated by signal SIGKILL (Forced quit)
> 
> # dmesg
> [   56.956995] BUG: kernel NULL pointer dereference, address: 0000000000000020
> [   56.957003] #PF: supervisor read access in kernel mode
> [   56.957006] #PF: error_code(0x0000) - not-present page
> [   56.957009] PGD 0 P4D 0
> [   56.957013] Oops: 0000 [#2] PREEMPT SMP PTI
> [   56.957017] CPU: 1 PID: 866 Comm: cat Tainted: G      D           5.16.0-rc5-next-20211217-debug #1 15ae5b0f28a4b9b6343440ee595affa8e1b5cf57
> [   56.957022] Hardware name: ASUSTeK COMPUTER INC. Q302LA/Q302LA, BIOS Q302LA.203 05/15/2014
> [   56.957024] RIP: 0010:sgx_total_bytes_show+0x28/0x40
> [   56.957032] Code: 66 90 0f 1f 44 00 00 49 89 f8 48 89 d7 48 c7 c6 5f b1 52 a4 41 8b 80 98 02 00 00 48 8d 04 40 48 c1 e0 04 48 03 05 38 b1 3c 02 <48> 8b 50 20 e8 ff ee 3b 00 48 98 31 d2 89 d6 89 d7 41 89 d0 c3 0f
> [   56.957035] RSP: 0018:ffffb79c81447d60 EFLAGS: 00010246
> [   56.957039] RAX: 0000000000000000 RBX: ffffffffa4c30ec0 RCX: 0000000000000000
> [   56.957041] RDX: ffff95d502b95000 RSI: ffffffffa452b15f RDI: ffff95d502b95000
> [   56.957044] RBP: ffffffffa41567c0 R08: ffff95d5002a0400 R09: 0000000000000000
> [   56.957046] R10: 0000000000000000 R11: 0000000000000000 R12: ffffb79c81447e28
> [   56.957048] R13: ffffb79c81447e00 R14: ffff95d5041247a8 R15: 0000000000000001
> [   56.957051] FS:  00007f4ab25af600(0000) GS:ffff95d616e80000(0000) knlGS:0000000000000000
> [   56.957054] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   56.957056] CR2: 0000000000000020 CR3: 00000001029e8005 CR4: 00000000001706e0
> [   56.957060] Call Trace:
> [   56.957062]  <TASK>
> [   56.957064]  dev_attr_show+0x19/0x40
> [   56.957072]  sysfs_kf_seq_show+0xa1/0x120
> [   56.957080]  seq_read_iter+0x12e/0x4c0
> [   56.957085]  new_sync_read+0x159/0x1f0
> [   56.957093]  vfs_read+0xff/0x1a0
> [   56.957099]  ksys_read+0x67/0xf0
> [   56.957105]  do_syscall_64+0x5c/0x90
> [   56.957110]  ? syscall_exit_to_user_mode+0x23/0x50
> [   56.957116]  ? exc_page_fault+0x72/0x180
> [   56.957121]  entry_SYSCALL_64_after_hwframe+0x44/0xae
> [   56.957128] RIP: 0033:0x7f4ab24d1862
> [   56.957131] Code: c0 e9 b2 fe ff ff 50 48 8d 3d 5a 29 0a 00 e8 55 e4 01 00 0f 1f 44 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 0f 05 <48> 3d 00 f0 ff ff 77 56 c3 0f 1f 44 00 00 48 83 ec 28 48 89 54 24
> [   56.957134] RSP: 002b:00007ffff14dac78 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
> [   56.957137] RAX: ffffffffffffffda RBX: 0000000000020000 RCX: 00007f4ab24d1862
> [   56.957140] RDX: 0000000000020000 RSI: 00007f4ab20d8000 RDI: 0000000000000003
> [   56.957142] RBP: 00007f4ab20d8000 R08: 00007f4ab20d7010 R09: 0000000000000000
> [   56.957144] R10: 0000000000000022 R11: 0000000000000246 R12: 0000000000020000
> [   56.957146] R13: 0000000000000003 R14: 0000000000000000 R15: 00007ffff14daf18
> [   56.957150]  </TASK>
> [   56.957152] Modules linked in: 8021q garp mrp stp llc snd_hda_codec_hdmi ccm
> algif_aead cbc x86_pkg_temp_thermal des_generic intel_powerclamp libdes
> coretemp ecb kvm_intel algif_skcipher kvm ax88796b irqbypass cmac
> crct10dif_pclmul i915 crc32_pclmul md4 ghash_clmulni_intel algif_hash
> snd_hda_codec_realtek af_alg snd_hda_codec_generic ledtrig_audio uvcvideo
> asus_nb_wmi aesni_intel snd_hda_intel intel_spi_platform i2c_algo_bit iwlmvm
> videobuf2_vmalloc asus_wmi snd_intel_dspcfg ttm snd_intel_sdw_acpi crypto_simd
> joydev ak8975 intel_spi iTCO_wdt btusb mousedev spi_nor sparse_keymap
> intel_pmc_bxt at24 cryptd videobuf2_memops iTCO_vendor_support mac80211
> hid_multitouch mtd platform_profile btrtl intel_rapl_msr mei_hdcp snd_hda_codec
> btbcm rapl libarc4 videobuf2_v4l2 drm_kms_helper intel_cstate
> processor_thermal_device_pci_legacy asix intel_uncore syscopyarea selftests
> btintel snd_hda_core videobuf2_common pcspkr iwlwifi snd_hwdep
> processor_thermal_device vfat btmtk i2c_i801 sysfillrect usbnet
> [   56.957229]  snd_pcm processor_thermal_rfim psmouse fat sysimgblt i2c_smbus
> bluetooth videodev processor_thermal_mbox snd_timer processor_thermal_rapl
> fb_sys_fops mii cfg80211 mei_me usbhid snd ecdh_generic mdio_devres mc
> inv_mpu6050_i2c intel_rapl_common cec crc16 int340x_thermal_zone libphy lpc_ich
> rfkill intel_gtt mei soundcore intel_soc_dts_iosf inv_mpu6050 acpi_als wmi
> dell_smo8800 industrialio_triggered_buffer int3400_thermal video i2c_mux
> kfifo_buf soc_button_array industrialio acpi_thermal_rel asus_wireless mac_hid
> drm pkcs8_key_parser fuse bpf_preload ip_tables x_tables xfs libcrc32c
> crc32c_generic serio_raw atkbd libps2 i8042 crc32c_intel xhci_pci
> xhci_pci_renesas serio
> [   56.957286] CR2: 0000000000000020
> [   56.957289] ---[ end trace 0000000000000000 ]---
> [   56.957291] RIP: 0010:sgx_total_bytes_show+0x28/0x40
> [   56.957296] Code: 66 90 0f 1f 44 00 00 49 89 f8 48 89 d7 48 c7 c6 5f b1 52 a4 41 8b 80 98 02 00 00 48 8d 04 40 48 c1 e0 04 48 03 05 38 b1 3c 02 <48> 8b 50 20 e8 ff ee 3b 00 48 98 31 d2 89 d6 89 d7 41 89 d0 c3 0f
> [   56.957299] RSP: 0018:ffffb79c8118fd58 EFLAGS: 00010246
> [   56.957301] RAX: 0000000000000000 RBX: ffffffffa4c30ec0 RCX: 0000000000000000
> [   56.957304] RDX: ffff95d50403d000 RSI: ffffffffa452b15f RDI: ffff95d50403d000
> [   56.957306] RBP: ffffffffa41567c0 R08: ffff95d5002a0400 R09: 0000000000000000
> [   56.957308] R10: 0000000000000000 R11: 0000000000000000 R12: ffffb79c8118fe20
> [   56.957310] R13: ffffb79c8118fdf8 R14: ffff95d503fd0550 R15: 0000000000000001
> [   56.957312] FS:  00007f4ab25af600(0000) GS:ffff95d616e80000(0000) knlGS:0000000000000000
> [   56.957315] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   56.957317] CR2: 0000000000000020 CR3: 00000001029e8005 CR4: 00000000001706e0
> 
> If I can provide any further information or testing, let me know!

Dave, when is the fix going to be applied [*]?

> 
> Cheers,
> Nathan

[*] https://lore.kernel.org/linux-sgx/YcGTePmWDMOQU1pn@iki.fi/T/#m831a01bdde347f9e0af2c973986fae0499718201

BR,
Jarkko

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id D52A6C433FE
	for <linux-kernel@archiver.kernel.org>; Sun,  2 Jan 2022 04:54:58 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S229887AbiABEy5 (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Sat, 1 Jan 2022 23:54:57 -0500
Received: from mga03.intel.com ([134.134.136.65]:36204 "EHLO mga03.intel.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S229474AbiABEyz (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Sat, 1 Jan 2022 23:54:55 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
  d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
  t=1641099295; x=1672635295;
  h=subject:to:cc:references:from:message-id:date:
   mime-version:in-reply-to;
  bh=ybr9h0Tjlfh98g0+tHbnAWtp1IJZwelyZh6ylaQ0k0w=;
  b=QpqK9AxCjaqSqFJw2g6XZTRDJ1WV0rfBVyTRJc6f3/t+3dB9jnm3m2Wj
   dV+THN8Zwf6BplXCaFRNrQRsqKOmteH5KxnEheEDSpylFrGT6mzj6V5ff
   KnGmPNU+esrzrSSbwLuhUzA77g7uIxBKNox8Vvo61Qr41JslcECOH0v+S
   3t+rcQrXOAKvlZJ3PTvIlC0VsPUn0SlN6+9iADav4CioQYjcBLf8C1C0a
   S76GOZGsZokVuF4FQ+0y4VUwQL8tkUHGxoQ0zbiMUIfNLspd52oKS9MLU
   6qT6Skog8pw9ulhKo3BHDKitwmp9AVHjXFWfcsjWU1i6xloreDy6ZZ+QT
   g==;
X-IronPort-AV: E=McAfee;i="6200,9189,10214"; a="241900492"
X-IronPort-AV: E=Sophos;i="5.88,255,1635231600"; 
   d="scan'208";a="241900492"
Received: from fmsmga006.fm.intel.com ([10.253.24.20])
  by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jan 2022 20:54:55 -0800
X-IronPort-AV: E=Sophos;i="5.88,255,1635231600"; 
   d="scan'208";a="759785140"
Received: from gkogo-mobl1.amr.corp.intel.com (HELO [10.209.55.50]) ([10.209.55.50])
  by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jan 2022 20:54:54 -0800
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
To:     Jarkko Sakkinen <jarkko@kernel.org>,
        Nathan Chancellor <nathan@kernel.org>
Cc:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org> <YbzhBrimHGGpddDM@archlinux-ax161>
 <YcuhhI2+k0XVuTb1@iki.fi>
From:   Dave Hansen <dave.hansen@intel.com>
Message-ID: <c857c964-89b9-d827-74ec-32cf874e8d8b@intel.com>
Date:   Sat, 1 Jan 2022 20:54:51 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <YcuhhI2+k0XVuTb1@iki.fi>
Content-Type: multipart/mixed;
 boundary="------------A9E940C37A18674D71DEB08B"
Content-Language: en-US
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

This is a multi-part message in MIME format.
--------------A9E940C37A18674D71DEB08B
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

On 12/28/21 3:45 PM, Jarkko Sakkinen wrote:
>> If I can provide any further information or testing, let me know!
> Dave, when is the fix going to be applied [*]?
> 
>> Cheers,
>> Nathan
> [*] https://lore.kernel.org/linux-sgx/YcGTePmWDMOQU1pn@iki.fi/T/#m831a01bdde347f9e0af2c973986fae0499718201

Greg preferred hiding the file as opposed to faking a number in there.
Any testing of the attached would be appreciated.

--------------A9E940C37A18674D71DEB08B
Content-Type: text/x-patch; charset=UTF-8;
 name="sgx-null-ptr.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="sgx-null-ptr.patch"


From: Dave Hansen <dave.hansen@linux.intel.com>

Nathan Chancellor reported an oops when aceessing the
'sgx_total_bytes' sysfs file:

	https://lore.kernel.org/all/YbzhBrimHGGpddDM@archlinux-ax161/

The sysfs output code accesses the sgx_numa_nodes[] array
unconditionally.  However, this array is allocated during SGX
initialization, which only occurs on systems where SGX is
supported.

If the sysfs file is accessed on systems without SGX support,
sgx_numa_nodes[] is NULL and an oops occurs.

To fix this, hide the entire nodeX/x86/ attribute group on
systems without SGX support using the ->is_visible attribute
group callback.

Fixes: 50468e431335 ("x86/sgx: Add an attribute for the amount of SGX memory in a NUMA node")
Reported-by: Nathan Chancellor <nathan@kernel.org>
CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Jarkko Sakkinen <jarkko@kernel.org>
Cc: linux-sgx@vger.kernel.org
Cc: x86@kernel.org
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
---

 b/arch/x86/kernel/cpu/sgx/main.c |   11 +++++++++++
 1 file changed, 11 insertions(+)

diff -puN arch/x86/kernel/cpu/sgx/main.c~sgx-null-ptr arch/x86/kernel/cpu/sgx/main.c
--- a/arch/x86/kernel/cpu/sgx/main.c~sgx-null-ptr	2021-12-20 07:56:38.309584807 -0800
+++ b/arch/x86/kernel/cpu/sgx/main.c	2021-12-20 08:17:28.997705149 -0800
@@ -910,6 +910,16 @@ static ssize_t sgx_total_bytes_show(stru
 }
 static DEVICE_ATTR_RO(sgx_total_bytes);
 
+static umode_t arch_node_attr_is_visible(struct kobject * kobj,
+		struct attribute * attr, int idx)
+{
+	/* Make all x86/ attributes invisible when SGX is not initialized: */
+	if (nodes_empty(sgx_numa_mask))
+		return 0;
+
+	return attr->mode;
+}
+
 static struct attribute *arch_node_dev_attrs[] = {
 	&dev_attr_sgx_total_bytes.attr,
 	NULL,
@@ -918,6 +928,7 @@ static struct attribute *arch_node_dev_a
 const struct attribute_group arch_node_dev_group = {
 	.name = "x86",
 	.attrs = arch_node_dev_attrs,
+	.is_visible = arch_node_attr_is_visible,
 };
 #endif /* CONFIG_NUMA */
 
_

--------------A9E940C37A18674D71DEB08B--

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 8B0EBC433FE
	for <linux-kernel@archiver.kernel.org>; Sun,  2 Jan 2022 23:20:50 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S230049AbiABXUt (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Sun, 2 Jan 2022 18:20:49 -0500
Received: from ams.source.kernel.org ([145.40.68.75]:54524 "EHLO
        ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S229723AbiABXUs (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Sun, 2 Jan 2022 18:20:48 -0500
Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140])
        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
        (No client certificate requested)
        by ams.source.kernel.org (Postfix) with ESMTPS id 142E0B80DB9;
        Sun,  2 Jan 2022 23:20:47 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 39BE0C36AEE;
        Sun,  2 Jan 2022 23:20:43 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
        s=k20201202; t=1641165645;
        bh=A34D33k6RMW8JdsozuY0EuUv/rE0L1gufityMIS5DNQ=;
        h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
        b=NFHQq0le1HnsDP0BxzlgvSkG04+H7Z15u6tHu/o5OBWhjhfqb9B1PHnEmO9OVJe4C
         eqk5CR7Rdn16ouoXFclIp6MG0JmOMVtHOJahFY9/aA80xTvKJG1/zVY5f6bI+v25Gx
         NE1PiSLP/5mep8zcuuryJux0URfZ7juLuQTCdzCK0ZU5GrnRF8IRsWL6s8h8q4aTLw
         rCOF9riD/c2ftriYuxNLJNjjAG1zyzl10aJts1UYhUM+BXLlbGiVwwgs2F+mO2ZBY2
         NoBtsyIay7vD3O0bAwJ2tIgaHrY55tfQ9NWvXeCebt4yN7BJ+vj1cIZNU4PjEV3gCU
         ZTajFL5u6JQrw==
Date:   Sun, 2 Jan 2022 16:20:40 -0700
From:   Nathan Chancellor <nathan@kernel.org>
To:     Dave Hansen <dave.hansen@intel.com>
Cc:     Jarkko Sakkinen <jarkko@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
Message-ID: <YdIzSGjxvgiWwkXu@archlinux-ax161>
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org>
 <YbzhBrimHGGpddDM@archlinux-ax161>
 <YcuhhI2+k0XVuTb1@iki.fi>
 <c857c964-89b9-d827-74ec-32cf874e8d8b@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <c857c964-89b9-d827-74ec-32cf874e8d8b@intel.com>
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sat, Jan 01, 2022 at 08:54:51PM -0800, Dave Hansen wrote:
> On 12/28/21 3:45 PM, Jarkko Sakkinen wrote:
> >> If I can provide any further information or testing, let me know!
> > Dave, when is the fix going to be applied [*]?
> > 
> >> Cheers,
> >> Nathan
> > [*] https://lore.kernel.org/linux-sgx/YcGTePmWDMOQU1pn@iki.fi/T/#m831a01bdde347f9e0af2c973986fae0499718201
> 
> Greg preferred hiding the file as opposed to faking a number in there.
> Any testing of the attached would be appreciated.

> 
> From: Dave Hansen <dave.hansen@linux.intel.com>
> 
> Nathan Chancellor reported an oops when aceessing the
> 'sgx_total_bytes' sysfs file:
> 
> 	https://lore.kernel.org/all/YbzhBrimHGGpddDM@archlinux-ax161/
> 
> The sysfs output code accesses the sgx_numa_nodes[] array
> unconditionally.  However, this array is allocated during SGX
> initialization, which only occurs on systems where SGX is
> supported.
> 
> If the sysfs file is accessed on systems without SGX support,
> sgx_numa_nodes[] is NULL and an oops occurs.
> 
> To fix this, hide the entire nodeX/x86/ attribute group on
> systems without SGX support using the ->is_visible attribute
> group callback.
> 
> Fixes: 50468e431335 ("x86/sgx: Add an attribute for the amount of SGX memory in a NUMA node")
> Reported-by: Nathan Chancellor <nathan@kernel.org>
> CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Jarkko Sakkinen <jarkko@kernel.org>
> Cc: linux-sgx@vger.kernel.org
> Cc: x86@kernel.org

On both my older Intel based test system and AMD based test system, the
sgx_total_bytes node is no longer visible.

$ ls -al /sys/devices/system/node/node0/x86/

Tested-by: Nathan Chancellor <nathan@kernel.org>

> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> ---
> 
>  b/arch/x86/kernel/cpu/sgx/main.c |   11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff -puN arch/x86/kernel/cpu/sgx/main.c~sgx-null-ptr arch/x86/kernel/cpu/sgx/main.c
> --- a/arch/x86/kernel/cpu/sgx/main.c~sgx-null-ptr	2021-12-20 07:56:38.309584807 -0800
> +++ b/arch/x86/kernel/cpu/sgx/main.c	2021-12-20 08:17:28.997705149 -0800
> @@ -910,6 +910,16 @@ static ssize_t sgx_total_bytes_show(stru
>  }
>  static DEVICE_ATTR_RO(sgx_total_bytes);
>  
> +static umode_t arch_node_attr_is_visible(struct kobject * kobj,
> +		struct attribute * attr, int idx)
> +{
> +	/* Make all x86/ attributes invisible when SGX is not initialized: */
> +	if (nodes_empty(sgx_numa_mask))
> +		return 0;
> +
> +	return attr->mode;
> +}
> +
>  static struct attribute *arch_node_dev_attrs[] = {
>  	&dev_attr_sgx_total_bytes.attr,
>  	NULL,
> @@ -918,6 +928,7 @@ static struct attribute *arch_node_dev_a
>  const struct attribute_group arch_node_dev_group = {
>  	.name = "x86",
>  	.attrs = arch_node_dev_attrs,
> +	.is_visible = arch_node_attr_is_visible,
>  };
>  #endif /* CONFIG_NUMA */
>  
> _


From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id C93D2C433FE
	for <linux-kernel@archiver.kernel.org>; Tue,  4 Jan 2022 16:52:12 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S235524AbiADQwL (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Tue, 4 Jan 2022 11:52:11 -0500
Received: from mga03.intel.com ([134.134.136.65]:2738 "EHLO mga03.intel.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S234046AbiADQwK (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Tue, 4 Jan 2022 11:52:10 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
  d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
  t=1641315130; x=1672851130;
  h=subject:to:cc:references:from:message-id:date:
   mime-version:in-reply-to:content-transfer-encoding;
  bh=B4JSVksj4KqzEAOO0ubUoTxIOso8jy7LZ/EF3a/TNdA=;
  b=Qmqoa5O2LOjmiWSZ3feg/288LWpJMLnrGiBcCgGoW9gbX1PRtEkeh32J
   xIcFIMTiphMRZFZrX6/RB3WJLl5nskh3QLeipAA8R3VQT4wk2oEL+o69j
   VE21ZUi2O217hvwQAfdmCi+ODS/xV0BWzThf8nEzy3qxPtYw/TbUDshVm
   3oB1nC6a1jOojuyPhwIKD17tsrknnmjC+dIqWPGVhKGUBoZCRMnpAt5qp
   TU4atKEWKpBF5KSIbC3jUjuFYU/lC//f0yMz6bHp7/JPrEd/L9oKMF7KE
   pj5YLjKy31xziNxWEL5+yqrHH/VVcsxzJFT7xOHeEXtqHyyxjD3TPatnA
   g==;
X-IronPort-AV: E=McAfee;i="6200,9189,10216"; a="242214257"
X-IronPort-AV: E=Sophos;i="5.88,261,1635231600"; 
   d="scan'208";a="242214257"
Received: from orsmga001.jf.intel.com ([10.7.209.18])
  by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2022 08:52:10 -0800
X-IronPort-AV: E=Sophos;i="5.88,261,1635231600"; 
   d="scan'208";a="556218187"
Received: from mncallah-mobl.amr.corp.intel.com (HELO [10.209.35.108]) ([10.209.35.108])
  by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2022 08:52:10 -0800
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
To:     Jarkko Sakkinen <jarkko@kernel.org>,
        Nathan Chancellor <nathan@kernel.org>
Cc:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org> <YbzhBrimHGGpddDM@archlinux-ax161>
 <YcuhhI2+k0XVuTb1@iki.fi> <c857c964-89b9-d827-74ec-32cf874e8d8b@intel.com>
From:   Dave Hansen <dave.hansen@intel.com>
Message-ID: <2da95214-8b1c-46e7-f6d1-a35ef2697c26@intel.com>
Date:   Tue, 4 Jan 2022 08:52:07 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <c857c964-89b9-d827-74ec-32cf874e8d8b@intel.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 1/1/22 8:54 PM, Dave Hansen wrote:
> On 12/28/21 3:45 PM, Jarkko Sakkinen wrote:
>>> If I can provide any further information or testing, let me know!
>> Dave, when is the fix going to be applied [*]?
>>
>>> Cheers,
>>> Nathan
>> [*] https://lore.kernel.org/linux-sgx/YcGTePmWDMOQU1pn@iki.fi/T/#m831a01bdde347f9e0af2c973986fae0499718201
> Greg preferred hiding the file as opposed to faking a number in there.
> Any testing of the attached would be appreciated.

Well, that didn't work on real SGX hardware.  The sysfs node code calls
the ->is_visible() callback before SGX is initialized.

I'll send out and updated version shortly that uses sysfs_update_group()
instead.

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 102FDC433F5
	for <linux-kernel@archiver.kernel.org>; Thu,  6 Jan 2022 19:18:42 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S243293AbiAFTSk (ORCPT <rfc822;linux-kernel@archiver.kernel.org>);
        Thu, 6 Jan 2022 14:18:40 -0500
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38798 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S243235AbiAFTSj (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 6 Jan 2022 14:18:39 -0500
Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 948BFC061245;
        Thu,  6 Jan 2022 11:18:39 -0800 (PST)
Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140])
        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
        (No client certificate requested)
        by ams.source.kernel.org (Postfix) with ESMTPS id 6098AB82369;
        Thu,  6 Jan 2022 19:18:38 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id D6C3DC36AE3;
        Thu,  6 Jan 2022 19:18:36 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
        s=k20201202; t=1641496717;
        bh=YDuZHR0Qu3o0lNe3alckOklSPNdvNLbGwS8If0Z9lz8=;
        h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
        b=LXBsbGmQ5hH0jNtJC2pOfCC5iLP1G9BVWiLNOIaJoPkS3EMfMU823YEynflzLCjt1
         6gqzrK40nhR0azq2MXorY0iuwjNjBip2s4dIWsBfmeXSTSsOKVdnsxYvZQARSEFLPC
         SfI6kw/plfUUzQ7hf+V7RvFf7bezH0e0zPmdegEETtwCUugPGpJgwty6/h1eW7bBUy
         dVEs4VOYnAHKd4SrvXHt4PkycK5+BeGSieEnq1R1bc9PpqTh1SxpxUuqIrbgwTRsy2
         /oIzy63kSgU3ZpLi6v7zhqwnv1VAfb6J0c3qUNU/sa/afNG3XlF7FYHXZOx0xHINDE
         b2STsNLJBoGyA==
Date:   Thu, 6 Jan 2022 21:18:32 +0200
From:   Jarkko Sakkinen <jarkko@kernel.org>
To:     Dave Hansen <dave.hansen@intel.com>
Cc:     Nathan Chancellor <nathan@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
Message-ID: <YddAiHSSX0V7UUik@iki.fi>
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org>
 <YbzhBrimHGGpddDM@archlinux-ax161>
 <YcuhhI2+k0XVuTb1@iki.fi>
 <c857c964-89b9-d827-74ec-32cf874e8d8b@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <c857c964-89b9-d827-74ec-32cf874e8d8b@intel.com>
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sat, Jan 01, 2022 at 08:54:51PM -0800, Dave Hansen wrote:
> On 12/28/21 3:45 PM, Jarkko Sakkinen wrote:
> >> If I can provide any further information or testing, let me know!
> > Dave, when is the fix going to be applied [*]?
> > 
> >> Cheers,
> >> Nathan
> > [*] https://lore.kernel.org/linux-sgx/YcGTePmWDMOQU1pn@iki.fi/T/#m831a01bdde347f9e0af2c973986fae0499718201
> 
> Greg preferred hiding the file as opposed to faking a number in there.
> Any testing of the attached would be appreciated.

> 
> From: Dave Hansen <dave.hansen@linux.intel.com>
> 
> Nathan Chancellor reported an oops when aceessing the
> 'sgx_total_bytes' sysfs file:
> 
> 	https://lore.kernel.org/all/YbzhBrimHGGpddDM@archlinux-ax161/
> 
> The sysfs output code accesses the sgx_numa_nodes[] array
> unconditionally.  However, this array is allocated during SGX
> initialization, which only occurs on systems where SGX is
> supported.
> 
> If the sysfs file is accessed on systems without SGX support,
> sgx_numa_nodes[] is NULL and an oops occurs.
> 
> To fix this, hide the entire nodeX/x86/ attribute group on
> systems without SGX support using the ->is_visible attribute
> group callback.
> 
> Fixes: 50468e431335 ("x86/sgx: Add an attribute for the amount of SGX memory in a NUMA node")
> Reported-by: Nathan Chancellor <nathan@kernel.org>
> CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Jarkko Sakkinen <jarkko@kernel.org>
> Cc: linux-sgx@vger.kernel.org
> Cc: x86@kernel.org
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> ---
> 
>  b/arch/x86/kernel/cpu/sgx/main.c |   11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff -puN arch/x86/kernel/cpu/sgx/main.c~sgx-null-ptr arch/x86/kernel/cpu/sgx/main.c
> --- a/arch/x86/kernel/cpu/sgx/main.c~sgx-null-ptr	2021-12-20 07:56:38.309584807 -0800
> +++ b/arch/x86/kernel/cpu/sgx/main.c	2021-12-20 08:17:28.997705149 -0800
> @@ -910,6 +910,16 @@ static ssize_t sgx_total_bytes_show(stru
>  }
>  static DEVICE_ATTR_RO(sgx_total_bytes);
>  
> +static umode_t arch_node_attr_is_visible(struct kobject * kobj,
> +		struct attribute * attr, int idx)
> +{
> +	/* Make all x86/ attributes invisible when SGX is not initialized: */
> +	if (nodes_empty(sgx_numa_mask))
> +		return 0;
> +
> +	return attr->mode;
> +}
> +
>  static struct attribute *arch_node_dev_attrs[] = {
>  	&dev_attr_sgx_total_bytes.attr,
>  	NULL,
> @@ -918,6 +928,7 @@ static struct attribute *arch_node_dev_a
>  const struct attribute_group arch_node_dev_group = {
>  	.name = "x86",
>  	.attrs = arch_node_dev_attrs,
> +	.is_visible = arch_node_attr_is_visible,
>  };
>  #endif /* CONFIG_NUMA */
>  
> _

I'm compiling now kernel with this applied, reporting soon but the fix
looks good to me.

/Jarkko

From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 68742C433F5
	for <linux-kernel@archiver.kernel.org>; Fri,  7 Jan 2022 11:43:03 +0000 (UTC)
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1347187AbiAGLnC (ORCPT
        <rfc822;linux-kernel@archiver.kernel.org>);
        Fri, 7 Jan 2022 06:43:02 -0500
Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34830 "EHLO
        lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S237668AbiAGLnB (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Fri, 7 Jan 2022 06:43:01 -0500
Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1])
        by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 255A9C061245;
        Fri,  7 Jan 2022 03:43:01 -0800 (PST)
Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140])
        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
        (No client certificate requested)
        by ams.source.kernel.org (Postfix) with ESMTPS id E1179B825E5;
        Fri,  7 Jan 2022 11:42:59 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id 48E70C36AE0;
        Fri,  7 Jan 2022 11:42:58 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
        s=k20201202; t=1641555778;
        bh=gwX1wUVYeNEw/0pE+CCT1kIe3J4A7iSduIbxxDzZtdg=;
        h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
        b=govdLCzbMCFUyMaBAshxtMCsjQGHpUayH2Zyrdg06qgGS4ayj90tVw2WWbmfLi0GY
         E4jtmO/ExrqldNMRaqpaYB4K02IxPg5StiFXYYc/Apso6nZgc2wJgsjL459Vj0JpLA
         IhucE310IuTJOzvU8Y1rbFqKc+zzIGjba3eW2WCJ7TLUcsI13HGGWe/jgCH2kA0PB+
         i+1IiQmT4BnMqIzoSRjQDMu+7bFBmhtohpFdnlIK6EpKuubtljck4mfl0hqiBIgtvU
         4dGbXe5Bc1HsNf+QCR9b0pMi7earlt4+Q4560Zei9XX4pZ9fjqNBwN2hgKnbFsCJzE
         1vE9Vsmla9UpQ==
Date:   Fri, 7 Jan 2022 13:42:53 +0200
From:   Jarkko Sakkinen <jarkko@kernel.org>
To:     Dave Hansen <dave.hansen@intel.com>
Cc:     Nathan Chancellor <nathan@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>, reinette.chatre@intel.com,
        linux-kernel@vger.kernel.org, linux-sgx@vger.kernel.org
Subject: Re: [PATCH v13 2/2] x86/sgx: Add an attribute for the amount of SGX
 memory in a NUMA node
Message-ID: <YdgnPTE5cnrptniM@iki.fi>
References: <20211116162116.93081-1-jarkko@kernel.org>
 <20211116162116.93081-2-jarkko@kernel.org>
 <YbzhBrimHGGpddDM@archlinux-ax161>
 <YcuhhI2+k0XVuTb1@iki.fi>
 <c857c964-89b9-d827-74ec-32cf874e8d8b@intel.com>
 <YddAiHSSX0V7UUik@iki.fi>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <YddAiHSSX0V7UUik@iki.fi>
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Jan 06, 2022 at 09:18:35PM +0200, Jarkko Sakkinen wrote:
> On Sat, Jan 01, 2022 at 08:54:51PM -0800, Dave Hansen wrote:
> > On 12/28/21 3:45 PM, Jarkko Sakkinen wrote:
> > >> If I can provide any further information or testing, let me know!
> > > Dave, when is the fix going to be applied [*]?
> > > 
> > >> Cheers,
> > >> Nathan
> > > [*] https://lore.kernel.org/linux-sgx/YcGTePmWDMOQU1pn@iki.fi/T/#m831a01bdde347f9e0af2c973986fae0499718201
> > 
> > Greg preferred hiding the file as opposed to faking a number in there.
> > Any testing of the attached would be appreciated.
> 
> > 
> > From: Dave Hansen <dave.hansen@linux.intel.com>
> > 
> > Nathan Chancellor reported an oops when aceessing the
> > 'sgx_total_bytes' sysfs file:
> > 
> > 	https://lore.kernel.org/all/YbzhBrimHGGpddDM@archlinux-ax161/
> > 
> > The sysfs output code accesses the sgx_numa_nodes[] array
> > unconditionally.  However, this array is allocated during SGX
> > initialization, which only occurs on systems where SGX is
> > supported.
> > 
> > If the sysfs file is accessed on systems without SGX support,
> > sgx_numa_nodes[] is NULL and an oops occurs.
> > 
> > To fix this, hide the entire nodeX/x86/ attribute group on
> > systems without SGX support using the ->is_visible attribute
> > group callback.
> > 
> > Fixes: 50468e431335 ("x86/sgx: Add an attribute for the amount of SGX memory in a NUMA node")
> > Reported-by: Nathan Chancellor <nathan@kernel.org>
> > CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > Cc: Jarkko Sakkinen <jarkko@kernel.org>
> > Cc: linux-sgx@vger.kernel.org
> > Cc: x86@kernel.org
> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > ---
> > 
> >  b/arch/x86/kernel/cpu/sgx/main.c |   11 +++++++++++
> >  1 file changed, 11 insertions(+)
> > 
> > diff -puN arch/x86/kernel/cpu/sgx/main.c~sgx-null-ptr arch/x86/kernel/cpu/sgx/main.c
> > --- a/arch/x86/kernel/cpu/sgx/main.c~sgx-null-ptr	2021-12-20 07:56:38.309584807 -0800
> > +++ b/arch/x86/kernel/cpu/sgx/main.c	2021-12-20 08:17:28.997705149 -0800
> > @@ -910,6 +910,16 @@ static ssize_t sgx_total_bytes_show(stru
> >  }
> >  static DEVICE_ATTR_RO(sgx_total_bytes);
> >  
> > +static umode_t arch_node_attr_is_visible(struct kobject * kobj,
> > +		struct attribute * attr, int idx)
> > +{
> > +	/* Make all x86/ attributes invisible when SGX is not initialized: */
> > +	if (nodes_empty(sgx_numa_mask))
> > +		return 0;
> > +
> > +	return attr->mode;
> > +}
> > +
> >  static struct attribute *arch_node_dev_attrs[] = {
> >  	&dev_attr_sgx_total_bytes.attr,
> >  	NULL,
> > @@ -918,6 +928,7 @@ static struct attribute *arch_node_dev_a
> >  const struct attribute_group arch_node_dev_group = {
> >  	.name = "x86",
> >  	.attrs = arch_node_dev_attrs,
> > +	.is_visible = arch_node_attr_is_visible,
> >  };
> >  #endif /* CONFIG_NUMA */
> >  
> > _
> 
> I'm compiling now kernel with this applied, reporting soon but the fix
> looks good to me.

Tested-by: Jarkko Sakkinen <jarkko@kernel.org>

Did two tests in a virtual machine:

1. Without the patch reproduced the crash.
2. With the patch verified that the crash does not occur.

/Jarkko

