From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756242AbdCWXsC (ORCPT ); Thu, 23 Mar 2017 19:48:02 -0400 Received: from userp1040.oracle.com ([156.151.31.81]:21178 "EHLO userp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751813AbdCWXsA (ORCPT ); Thu, 23 Mar 2017 19:48:00 -0400 Subject: Re: [v1 0/5] parallelized "struct page" zeroing To: David Miller , willy@infradead.org References: <1490310113-824438-1-git-send-email-pasha.tatashin@oracle.com> <20170323232638.GB29134@bombadil.infradead.org> <20170323.163520.123614131649571916.davem@davemloft.net> Cc: linux-kernel@vger.kernel.org, sparclinux@vger.kernel.org, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.or From: Pasha Tatashin Message-ID: Date: Thu, 23 Mar 2017 19:47:23 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20170323.163520.123614131649571916.davem@davemloft.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Source-IP: userv0021.oracle.com [156.151.31.71] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 03/23/2017 07:35 PM, David Miller wrote: > From: Matthew Wilcox > Date: Thu, 23 Mar 2017 16:26:38 -0700 > >> On Thu, Mar 23, 2017 at 07:01:48PM -0400, Pavel Tatashin wrote: >>> When deferred struct page initialization feature is enabled, we get a >>> performance gain of initializing vmemmap in parallel after other CPUs are >>> started. However, we still zero the memory for vmemmap using one boot CPU. >>> This patch-set fixes the memset-zeroing limitation by deferring it as well. >>> >>> Here is example performance gain on SPARC with 32T: >>> base >>> https://hastebin.com/ozanelatat.go >>> >>> fix >>> https://hastebin.com/utonawukof.go >>> >>> As you can see without the fix it takes: 97.89s to boot >>> With the fix it takes: 46.91 to boot. >> >> How long does it take if we just don't zero this memory at all? We seem >> to be initialising most of struct page in __init_single_page(), so it >> seems like a lot of additional complexity to conditionally zero the rest >> of struct page. > > Alternatively, just zero out the entire vmemmap area when it is setup > in the kernel page tables. Hi Dave, I can do this, either way is fine with me. It would be a little slower compared to the current approach where we benefit from having memset() to work as prefetch. But that would become negligible, once in the future we will increase the granularity of multi-threading, currently it is only one thread per-mnode to multithread vmemamp. Your call. Thank you, Pasha