From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1946778AbdEZDX0 (ORCPT <rfc822;w@1wt.eu>);
        Thu, 25 May 2017 23:23:26 -0400
Received: from mail-pf0-f193.google.com ([209.85.192.193]:34883 "EHLO
        mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1946760AbdEZDXY (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 25 May 2017 23:23:24 -0400
Date: Fri, 26 May 2017 13:23:05 +1000
From: Balbir Singh <bsingharora@gmail.com>
To: Michael Bringmann <mwb@linux.vnet.ibm.com>
Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
        Benjamin Herrenschmidt <benh@kernel.crashing.org>,
        Paul Mackerras <paulus@samba.org>,
        Michael Ellerman <mpe@ellerman.id.au>,
        Reza Arbab <arbab@linux.vnet.ibm.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Bharata B Rao <bharata@linux.vnet.ib>,
        Shailendra Singh <shailendras@nvidia.com>,
        "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>,
        Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Subject: Re: [PATCH V2 2/2]: powerpc/hotplug/mm: Fix hot-add memory node
 assoc
Message-ID: <20170526132305.19ef3590@firefly.ozlabs.ibm.com>
In-Reply-To: <9b006866-5a8e-dc05-d603-bfe8eb02eef2@linux.vnet.ibm.com>
References: <20170525171932.29159.89311.stgit@ltcalpine2-lp20.aus.stglabs.ibm.com>
        <9b006866-5a8e-dc05-d603-bfe8eb02eef2@linux.vnet.ibm.com>
X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.31; x86_64-redhat-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, 25 May 2017 12:37:40 -0500
Michael Bringmann <mwb@linux.vnet.ibm.com> wrote:

> Removing or adding memory via the PowerPC hotplug interface shows
> anomalies in the association between memory and nodes.  The code
> was updated to ensure that all nodes found at boot are still available
> to subsequent DLPAR hotplug-memory operations, even if they are not
> needed at boot time.
> 
> Signed-off-by: Michael Bringmann <mwb@linux.vnet.ibm.com>
> ---
> Changes in V2:
>   -- Simplify patches to ensure more nodes in possible map, removing
>      code from PowerPC numa.c that constrained possible map to size
>      of online map.
> ---
>  arch/powerpc/mm/numa.c |    7 -------
>  1 file changed, 7 deletions(-)
> 
> diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c
> index 15c2dd5..18f3038 100644
> --- a/arch/powerpc/mm/numa.c
> +++ b/arch/powerpc/mm/numa.c
> @@ -907,13 +907,6 @@ void __init initmem_init(void)
>  
>  	memblock_dump_all();
>  
> -	/*
> -	 * Reduce the possible NUMA nodes to the online NUMA nodes,
> -	 * since we do not support node hotplug. This ensures that  we
> -	 * lower the maximum NUMA node ID to what is actually present.
> -	 */
> -	nodes_and(node_possible_map, node_possible_map, node_online_map);
> -

There is an overhead with turning this off if you have too many cgroups
with the memory controller. I think this fix was added for a pathological
test case. On my system I see 84 cgroups with 1 node, so the probable
overhead is 84*255*sizeof(struct mem_cgroup_tree_per_node).

I tried some patches to reduce the overhead, but those need more overhauling
and rework.

Balbir Singh.