From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S965005AbbD0UHr (ORCPT <rfc822;w@1wt.eu>);
	Mon, 27 Apr 2015 16:07:47 -0400
Received: from g1t5425.austin.hp.com ([15.216.225.55]:49529 "EHLO
	g1t5425.austin.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S932964AbbD0UHq (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Mon, 27 Apr 2015 16:07:46 -0400
Message-ID: <553E970E.2040406@hp.com>
Date: Mon, 27 Apr 2015 16:07:42 -0400
From: Waiman Long <waiman.long@hp.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12
MIME-Version: 1.0
To: Mel Gorman <mgorman@suse.de>
CC: Andrew Morton <akpm@linux-foundation.org>, Linux-MM <linux-mm@kvack.org>,
        Nathan Zimmer <nzimmer@sgi.com>, Dave Hansen <dave.hansen@intel.com>,
        Scott Norton <scott.norton@hp.com>,
        Daniel J Blueman <daniel@numascale.com>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 10/13] x86: mm: Enable deferred struct page initialisation
 on x86-64
References: <1429722473-28118-1-git-send-email-mgorman@suse.de> <1429722473-28118-11-git-send-email-mgorman@suse.de> <20150422164500.121a355e6b578243cb3650e3@linux-foundation.org> <20150423092327.GJ14842@suse.de> <553A54C5.3060106@hp.com> <20150424152007.GD2449@suse.de> <553A93BB.1010404@hp.com> <20150425172859.GE2449@suse.de>
In-Reply-To: <20150425172859.GE2449@suse.de>
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 04/25/2015 01:28 PM, Mel Gorman wrote:
> On Fri, Apr 24, 2015 at 03:04:27PM -0400, Waiman Long wrote:
>>>> Within a NUMA node, however, we can split the
>>>> memory initialization to 2 or more local CPUs if the memory size is
>>>> big enough.
>>>>
>>> I considered it but discarded the idea. It'd be more complex to setup and
>>> the two CPUs could simply end up contending on the same memory bus as
>>> well as contending on zone->lock.
>>>
>> I don't think we need that now. However, we may have to consider
>> this when one day even a single node can have TBs of memory unless
>> we move to a page size larger than 4k.
>>
> We'll cross that bridge when we come to it. I suspect there is more room
> for improvement in the initialisation that would be worth trying before
> resorting to more threads. With more threads there is a risk that we hit
> memory bus contention and a high risk that it actually is worse due to
> contending on zone->lock when freeing the pages.
>
> In the meantime, do you mind updating the before/after figures for your
> test machine with this series please?
>

I will test the latest patch once I got my hand on a 12TB machine.

Cheers,
Longman