From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751371AbbJEEdo (ORCPT ); Mon, 5 Oct 2015 00:33:44 -0400 Received: from mail-db3on0123.outbound.protection.outlook.com ([157.55.234.123]:8835 "EHLO emea01-db3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750793AbbJEEdm (ORCPT ); Mon, 5 Oct 2015 00:33:42 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=daniel@numascale.com; From: Daniel J Blueman To: Ingo Molnar CC: Daniel J Blueman , Denys Vlasenko , Thomas Gleixner , Jiang Liu , Len Brown , Steffen Persvold , , Subject: [PATCH v2] x86/apic: Use smaller array for __apicid_to_node[] mapping Date: Mon, 5 Oct 2015 12:32:30 +0800 Message-ID: <1444019550-21045-1-git-send-email-daniel@numascale.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <20151003074428.GA25143@gmail.com> References: <20151003074428.GA25143@gmail.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [175.156.157.249] X-ClientProxiedBy: DB4PR05CA0013.eurprd05.prod.outlook.com (25.160.40.23) To HE1PR07MB1306.eurprd07.prod.outlook.com (25.164.51.156) X-Microsoft-Exchange-Diagnostics: 1;HE1PR07MB1306;2:vjKD2eR3u/fQsgEyNk5XjoRvb0/fkvqLjc6gjofF8Oxg9siYifIswCmuKFTmqNzTm5QyID/kafETq06BGgz1b9friOKtz1R/OK2bQKG6tYz3Qry0hWrsHnWRd8Mkq3AJOhCHyzi5xPQZ2fN7De72GtTxa9YlBRFT9824MHxy4+U=;3:nWUfALjfULhUblJG1RXxjfWvT+IIz/hXldnbs9QDpfDXN8u/sE3Q33EnPrWM43/FzPCoMPkHLSZZcBG4KYJ9Deq4M7XyW4n2RGKdv/2G6epQsk1NvZhw9FpW+4+RVGj0w5rEoXl2GaQAw4ku3N3tUw==;25:jWY0PPNpHt/9iNBLFShpreITK5LUX0jw8xquNtnPjcgmKUw9SOLkYFn6ghf/+XgmnP1U3SKVb/vk8/s33VGZBSC+vNpKzeE4S1I2PNbFkVORJj3q9y4e2iANGD+hx3EfjiXRRUiKcvVCsvwRLYLXFswHut3J6LuOLjv+LowrGguwOlqXoKVkcgoRFDrIKL9QFTfgI6ZiXJLKzszd+KZ98uCXM/pURXX5PCk40hNNob0mrWz3vx9/oLefWbUNA4sOdQdZdSxovDP1fYB+zbOjsw== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:HE1PR07MB1306; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(8121501046)(520078)(5005006)(3002001);SRVR:HE1PR07MB1306;BCL:0;PCL:0;RULEID:;SRVR:HE1PR07MB1306; X-Microsoft-Exchange-Diagnostics: 1;HE1PR07MB1306;4:H46l9f8g8gO4L8WvpUWngtYKZ8G2ZogMobAhVERWpMrIn0QSggzDU1r8gmyEZhhsVOtIeWmXbmPGNfRRQudRJNxTXhHKD67hcyt8YmHcWgC/zQlMSgTXcXnO1T/kfNMpnkYJ6rYMCvVC0tfGJzWtKl5ho1pFlUdKNP7xFu3ln8oyq01pZX2YXIvXftVNrfO5NvVi83U/CGdWAZe3MtK0rX1lYMbvdS9uLAeBsDxMCvMMTKO0a5pMNc7lzi+Vf1fUiizYWOBZ5VYn5I494QiAaxT3/3LGDssFHCXE6RApEhj7pjWwn2q/elg1PLnRSzYXjTuefwQD/wjamYlT+zVAqYVEWs96uRdVkz6w6J2RbHg= X-Forefront-PRVS: 07200C0526 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(6009001)(199003)(189002)(243025005)(36756003)(19580395003)(97736004)(81156007)(110136002)(189998001)(19580405001)(87976001)(229853001)(5004730100002)(15975445007)(64706001)(2950100001)(92566002)(5001830100001)(47776003)(4001540100001)(77096005)(68736005)(5008740100001)(50226001)(77156002)(50466002)(40100003)(5007970100001)(46102003)(122386002)(66066001)(33646002)(62966003)(5001960100002)(101416001)(5001860100001)(86362001)(48376002)(50986999)(5003940100001)(76176999)(106356001)(42186005)(575784001)(105586002);DIR:OUT;SFP:1102;SCL:1;SRVR:HE1PR07MB1306;H:exige.int.numascale.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;HE1PR07MB1306;23:urP2pxwBsfdjqveustoM6DxkDKVgpVBSI1fGkp1e9?= =?us-ascii?Q?vzfjU2B/8GWhrwOEg8BjJv9mkNXNE6+kN7gzSjreZnaK2PGepcdvbK1vmJSA?= =?us-ascii?Q?lZizpygOXCqjHpeETCLDZPJLrJm/hOk3ny9spj00nStUaQsMrTQzpN1pqyz9?= =?us-ascii?Q?NtPSCZy29NM8615h9Bb5UBjT0qSuaILGyzDEHPPqO0vX4mzVRVInp/+kxU25?= =?us-ascii?Q?HIcaIbrOzx1U4aK41Kr9gGKTCErx3X5VeGVOLNpoI31HoMXQpQR0GhNlgBot?= =?us-ascii?Q?jQVOx6TbPWOy91Q7KJ7a2CrQG9t16TgtYJBv0dXnhskwVeTv9YXRAT7xyNS2?= =?us-ascii?Q?OQaXPQ7HELAGfqKAGPXpFoGTLZ+mUoK5UEYrb1hQYsWpqGg9joxUU5F0jthv?= =?us-ascii?Q?3fFgZ4NKU8gIBKV44vWIAznelaFr+VMTUEnW3O2Clq+e7tBwWRWcCuPRSBAu?= =?us-ascii?Q?wD0xfqGFEiwmskUb194W4X2CN2mGfS+vYQBU+ADrHVB1dLvarrbLkNFhzYXy?= =?us-ascii?Q?vHdpy1q5SnYu0x271q67fN2c9TeZ/MHn+mRULmpVf7s35SNcMoBD4Gicp9+E?= =?us-ascii?Q?n42SpxQ/DzHWtcHLFieWYWaDDP/b7teqbFoFtGvjVP+721W7keTdvq0+1qLC?= =?us-ascii?Q?N7qVc16Cs7ZEiRdZZ9or4FP3YntYtwB+3gYv6NuxXpLNCbox6/diIULydJmI?= =?us-ascii?Q?giVW4XCS6WhbzOS/Js/LqBxmYwnj0RIp+wOKoujUf/DC7lJLbPuEJBD20Pi/?= =?us-ascii?Q?UPDLUjPyosRCLCRJQGiUHExJ4+wFRobj+3cWX5dBMumkjka9ARYCoRiQzNLL?= =?us-ascii?Q?NIVstQRcXFy07J7VTFYMVo1TT1wPYU625Ts7Uhorb26b9/3sWyHfKn9HCj8M?= =?us-ascii?Q?M4CFj5nY0pVSFecMgID9Vr65SlJZLYWHg6VXTKruBSpLPr8PhKWvlZQ8TAy9?= =?us-ascii?Q?fi0b/FWjSe7bLOCH4G1ftXkNVoo4XfynOeMXzNHynfWqMM8tZETxxVVBKNWa?= =?us-ascii?Q?l1tjgnLoZ9HX2cCU53zGICsHF4Oqfc8uOKP9nZ/2ZEsdVdE6IbHl23tlsoJ+?= =?us-ascii?Q?v31Rz965Wg5q6hVyEgou0YBdgUHicuoS7fZfCB4choEGMNw4Dh+5EjyvoKkZ?= =?us-ascii?Q?Mgz8XbWm62hDvUVUktekg8qa+KtDDWk6WDVdPxmYcumqGM3HN8+cTFywqSqY?= =?us-ascii?Q?ttsQARAN1XYTQKJMJJ381EVLRYrwwa0Zm8M+5zZ9tiCPdQ0f63vjwunnNmSq?= =?us-ascii?Q?LegZap0shAPW1cnLbKlRdo+PLpFaZ/Mv0L2Owec?= X-Microsoft-Exchange-Diagnostics: 1;HE1PR07MB1306;5:NnMfxB/Rv5LpYnlQgpm69bHqxceF8BVKhj+bE6ORBB9My0lC1P/Ugv5NsEoWxVGq9WrRXXNxTxSWnlL9daaJA8Q7/PWnj+ahwSZoPRIzjqI4JyEVHJ2AcO6h5ntLLplDWaVXzoPyko/lVnRl8cw3FA==;24:M2BB7QIwcfcOEAFJV2Rnwxy5EF+yAVO/dcpKW+F9mK7++5HNPyOFPMOrZg4Yo8c+404U/16JfsXmcM+yfTIX5F/4QO4SVoMDFe5qn4k3eOY=;20:1/QuZskTM+4Jli2N53Ef6v46l+de4GKnVa2UDNOCFZjYWDmUQETQGxZNjI+HdbdflMG1avYA39kYIpYVCfYcnA== SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: numascale.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Oct 2015 04:33:33.6049 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR07MB1306 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The Intel x2APIC spec states the upper 16-bits of APIC ID is the cluster ID [1, p2-12], intended for future distributed systems. Beyond the legacy 8-bit APIC ID, Numascale NumaConnect uses 4-bits for the position of a server on each axis of a multi-dimension torus; SGI NUMAlink also structures the APIC ID space. Instead, define an array based on NR_CPUs to achieve a 1:1 mapping and perform linear search; we see "ACPI: NR_CPUS/possible_cpus limit of X reached. Processor 8/0x16 ignored." when config-limited. This addresses the binary bloat and the present artificial APIC ID limits. With CONFIG_NR_CPUS=256, we save ~64KB of vmlinux data: $ size vmlinux vmlinux-patched text data bss dec hex filename 18232877 1849656 2281472 22364005 1553f65 vmlinux 18233034 1786168 2281472 22300674 1544802 vmlinux-patched Tested on a 256-core system with a 20-bit APIC ID space, and on a 48-core legacy 8-bit APIC ID system with and without CONFIG_NUMA, CONFIG_NUMA_EMU and CONFIG_AMD_NUMA. v2: Improved readability by moving static variable out; integrated Denys's numa emulation fix Signed-off-by: Daniel J Blueman CC: Denys Vlasenko CC: Ingo Molnar CC: Thomas Gleixner CC: Jiang Liu CC: Len Brown CC: Steffen Persvold CC: linux-kernel@vger.kernel.org CC: x86@kernel.org [1] http://www.intel.com/content/dam/doc/specification-update/64-architecture-x2apic-specification.pdf --- arch/x86/include/asm/numa.h | 13 +++++++------ arch/x86/kernel/cpu/amd.c | 11 ++++++----- arch/x86/mm/numa.c | 29 +++++++++++++++++++++-------- arch/x86/mm/numa_emulation.c | 6 +++--- 4 files changed, 37 insertions(+), 22 deletions(-) diff --git a/arch/x86/include/asm/numa.h b/arch/x86/include/asm/numa.h index 01b493e..33becb8 100644 --- a/arch/x86/include/asm/numa.h +++ b/arch/x86/include/asm/numa.h @@ -17,6 +17,11 @@ */ #define NODE_MIN_SIZE (4*1024*1024) +struct apicid_to_node { + int apicid; + s16 node; +}; + extern int numa_off; /* @@ -27,17 +32,13 @@ extern int numa_off; * should be accessed by the accessors - set_apicid_to_node() and * numa_cpu_node(). */ -extern s16 __apicid_to_node[MAX_LOCAL_APIC]; +extern struct apicid_to_node __apicid_to_node[NR_CPUS]; extern nodemask_t numa_nodes_parsed __initdata; extern int __init numa_add_memblk(int nodeid, u64 start, u64 end); extern void __init numa_set_distance(int from, int to, int distance); -static inline void set_apicid_to_node(int apicid, s16 node) -{ - __apicid_to_node[apicid] = node; -} - +extern void set_apicid_to_node(int apicid, s16 node); extern int numa_cpu_node(int cpu); #else /* CONFIG_NUMA */ diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c index 4a70fc6..9494f0e 100644 --- a/arch/x86/kernel/cpu/amd.c +++ b/arch/x86/kernel/cpu/amd.c @@ -277,12 +277,13 @@ static int nearby_node(int apicid) int i, node; for (i = apicid - 1; i >= 0; i--) { - node = __apicid_to_node[i]; + node = __apicid_to_node[i].node; if (node != NUMA_NO_NODE && node_online(node)) return node; } - for (i = apicid + 1; i < MAX_LOCAL_APIC; i++) { - node = __apicid_to_node[i]; + for (i = apicid + 1; i < NR_CPUS; i++) { + node = __apicid_to_node[i].node; + if (node != NUMA_NO_NODE && node_online(node)) return node; } @@ -422,8 +423,8 @@ static void srat_detect_node(struct cpuinfo_x86 *c) int ht_nodeid = c->initial_apicid; if (ht_nodeid >= 0 && - __apicid_to_node[ht_nodeid] != NUMA_NO_NODE) - node = __apicid_to_node[ht_nodeid]; + __apicid_to_node[ht_nodeid].node != NUMA_NO_NODE) + node = __apicid_to_node[ht_nodeid].node; /* Pick a nearby node */ if (!node_online(node)) node = nearby_node(apicid); diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c index c3b3f65..849a113 100644 --- a/arch/x86/mm/numa.c +++ b/arch/x86/mm/numa.c @@ -26,6 +26,7 @@ nodemask_t numa_nodes_parsed __initdata; struct pglist_data *node_data[MAX_NUMNODES] __read_mostly; EXPORT_SYMBOL(node_data); +static unsigned apicids; static struct numa_meminfo numa_meminfo #ifndef CONFIG_MEMORY_HOTPLUG __initdata @@ -56,16 +57,31 @@ early_param("numa", numa_setup); /* * apicid, cpu, node mappings */ -s16 __apicid_to_node[MAX_LOCAL_APIC] = { - [0 ... MAX_LOCAL_APIC-1] = NUMA_NO_NODE +struct apicid_to_node __apicid_to_node[NR_CPUS] = { + [0 ... NR_CPUS-1] = {-1, NUMA_NO_NODE} }; +void set_apicid_to_node(int apicid, s16 node) +{ + /* Protect against small kernel on large system */ + if (apicids >= NR_CPUS) + return; + + __apicid_to_node[apicids].apicid = apicid; + __apicid_to_node[apicids].node = node; + apicids++; +} + int numa_cpu_node(int cpu) { - int apicid = early_per_cpu(x86_cpu_to_apicid, cpu); + int ent, apicid = early_per_cpu(x86_cpu_to_apicid, cpu); + if (apicid == BAD_APICID) + return NUMA_NO_NODE; + + for (ent = 0; ent < NR_CPUS; ent++) + if (__apicid_to_node[ent].apicid == apicid) + return __apicid_to_node[ent].node; - if (apicid != BAD_APICID) - return __apicid_to_node[apicid]; return NUMA_NO_NODE; } @@ -607,9 +623,6 @@ static int __init numa_init(int (*init_func)(void)) int i; int ret; - for (i = 0; i < MAX_LOCAL_APIC; i++) - set_apicid_to_node(i, NUMA_NO_NODE); - nodes_clear(numa_nodes_parsed); nodes_clear(node_possible_map); nodes_clear(node_online_map); diff --git a/arch/x86/mm/numa_emulation.c b/arch/x86/mm/numa_emulation.c index a8f90ce..1a0e112 100644 --- a/arch/x86/mm/numa_emulation.c +++ b/arch/x86/mm/numa_emulation.c @@ -399,12 +399,12 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt) * back to zero just in case. */ for (i = 0; i < ARRAY_SIZE(__apicid_to_node); i++) { - if (__apicid_to_node[i] == NUMA_NO_NODE) + if (__apicid_to_node[i].node == NUMA_NO_NODE) continue; for (j = 0; j < ARRAY_SIZE(emu_nid_to_phys); j++) - if (__apicid_to_node[i] == emu_nid_to_phys[j]) + if (__apicid_to_node[i].node == emu_nid_to_phys[j]) break; - __apicid_to_node[i] = j < ARRAY_SIZE(emu_nid_to_phys) ? j : 0; + __apicid_to_node[i].node = j < ARRAY_SIZE(emu_nid_to_phys) ? j : 0; } /* make sure all emulated nodes are mapped to a physical node */ -- 2.5.0