From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756095AbdGXO2j (ORCPT ); Mon, 24 Jul 2017 10:28:39 -0400 Received: from mx2.suse.de ([195.135.220.15]:45625 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751827AbdGXO2a (ORCPT ); Mon, 24 Jul 2017 10:28:30 -0400 Date: Mon, 24 Jul 2017 16:28:27 +0200 From: Michal Hocko To: Tejun Heo Cc: Michael Ellerman , Jiri Kosina , linux-mm@kvack.org, LKML Subject: Re: pcpu allocator on large NUMA machines Message-ID: <20170724142826.GN25221@dhcp22.suse.cz> References: <20170724134240.GL25221@dhcp22.suse.cz> <20170724135714.GA3240919@devbig577.frc2.facebook.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170724135714.GA3240919@devbig577.frc2.facebook.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon 24-07-17 09:57:14, Tejun Heo wrote: > Hello, Hi, and thanks for ths swift answer > On Mon, Jul 24, 2017 at 03:42:40PM +0200, Michal Hocko wrote: [...] > > My understanding of the pcpu allocator is basically close to zero but it > > seems weird to me that we would need many TB of vmalloc address space > > just to allocate vmalloc areas that are in range of hundreds of MB. So I > > am wondering whether this is an expected behavior of the allocator or > > there is a problem somwehere else. > > It's not actually using the entire region but the area allocations try > to follow the same topology as kernel linear address layouts. ie. if > kernel address for different NUMA nodes are apart by certain amount, > the percpu allocator tries to replicate that for dynamic allocations > which allows leaving the static and first dynamic area in the kernel > linear address which helps reducing TLB pressure. > > This optimization can be turned off when vmalloc area isn't spacious > enough by using pcpu_page_first_chunk() instead of > pcpu_embed_first_chunk() while initializing percpu allocator. Thanks for the clarification, this is really helpful! > Can you > see whether replacing that in arch/powerpc/kernel/setup_64.c fixes the > issue? If so, all it needs to do is figuring out what conditions we > need to check to opt out of embedding the first chunk. Note that x86 > 32bit does about the same thing. Hmm, I will need some help from PPC guys here. I cannot find something ready to implement pcpup_populate_pte and I am not familiar with ppc memory model to implement one myself. -- Michal Hocko SUSE Labs