From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1765404AbdEWWla (ORCPT ); Tue, 23 May 2017 18:41:30 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:60815 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1765083AbdEWWl2 (ORCPT ); Tue, 23 May 2017 18:41:28 -0400 Subject: Re: [Patch 2/2]: powerpc/hotplug/mm: Fix hot-add memory node assoc To: Reza Arbab References: <3bb44d92-b2ff-e197-4bdf-ec6d588d6dab@linux.vnet.ibm.com> <20170523155251.bqwc5mc4jpgzkqlm@arbab-laptop.localdomain> <1c1d70e3-4e45-b035-0e75-1b0f531c111b@linux.vnet.ibm.com> <20170523214922.bns675oqzqj4pkhc@arbab-laptop.localdomain> Cc: linux-kernel@vger.kernel.org, Paul Mackerras , "Aneesh Kumar K.V" , Bharata B Rao , Shailendra Singh , Thomas Gleixner , linuxppc-dev@lists.ozlabs.org, Sebastian Andrzej Siewior From: Michael Bringmann Organization: IBM Linux Technology Center Date: Tue, 23 May 2017 17:41:22 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20170523214922.bns675oqzqj4pkhc@arbab-laptop.localdomain> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 x-cbid: 17052322-0020-0000-0000-00000C011695 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007107; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000212; SDB=6.00864674; UDB=6.00429266; IPR=6.00644454; BA=6.00005369; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015556; XFM=3.00000015; UTC=2017-05-23 22:41:25 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17052322-0021-0000-0000-00005C7521CF Message-Id: <48d09a3b-f2cf-a98c-8a26-46877af8b43f@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-05-23_10:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705230115 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/23/2017 04:49 PM, Reza Arbab wrote: > On Tue, May 23, 2017 at 03:05:08PM -0500, Michael Bringmann wrote: >> On 05/23/2017 10:52 AM, Reza Arbab wrote: >>> On Tue, May 23, 2017 at 10:15:44AM -0500, Michael Bringmann wrote: >>>> +static void setup_nodes(void) >>>> +{ >>>> + int i, l = 32 /* MAX_NUMNODES */; >>>> + >>>> + for (i = 0; i < l; i++) { >>>> + if (!node_possible(i)) { >>>> + setup_node_data(i, 0, 0); >>>> + node_set(i, node_possible_map); >>>> + } >>>> + } >>>> +} >>> >>> This seems to be a workaround for 3af229f2071f ("powerpc/numa: Reset node_possible_map to only node_online_map"). >> >> They may be related, but that commit is not a replacement. The above patch ensures that >> there are enough of the nodes initialized at startup to allow for memory hot-add into a >> node that was not used at boot. (See 'setup_node_data' function in 'numa.c'.) That and >> recording that the node was initialized. > > Is it really necessary to preinitialize these empty nodes using setup_node_data()? When you do memory hotadd into a node that was not used at boot, the node data already gets set up by > > add_memory > add_memory_resource > hotadd_new_pgdat > arch_alloc_nodedata <-- allocs the pg_data_t > ... > free_area_init_node <-- sets NODE_DATA(nid)->node_id, etc. I see that code now, but for some reason it did not work when I hot-added memory. > > Removing setup_node_data() from that loop leaves only the call to node_set(). If 3af229f2071f (which reduces node_possible_map) was reverted, you wouldn't need to do that either. > >> I didn't see where any part of commit 3af229f2071f would touch the 'node_possible_map' >> which is needed by 'numa.c' and 'workqueue.c'. The nodemask created and updated by >> 'mem_cgroup_may_update_nodemask()' does not appear to be the same mask. > > Are you sure you're looking at 3af229f2071f? It only adds one line of code; the reduction of node_possible_map. > The 3rd file in the patch set removes, - nodes_and(node_possible_map, node_possible_map, node_online_map); I need to add bits to 'node_possible_map' -- bits which may not be used for the memory at boot, but which would be used when memory is hot-added later. I haven't found anything outside of the boot code that adds bits to the 'possible' mask. -- Michael W. Bringmann Linux Technology Center IBM Corporation Tie-Line 363-5196 External: (512) 286-5196 Cell: (512) 466-0650 mwb@linux.vnet.ibm.com