From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2122D2FF164 for ; Tue, 21 Apr 2026 09:27:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776763626; cv=none; b=Vh++7Su0KbZ7BoJUCueBK2V8qYSVQ+QIsHhjLyZn8naEOOvtFrchmgMzJ2f8IzGA/8cpR3p2IzWgX60jltlBYm1r9QcGN36dDNdgKmh808Ezr/EXtq8kORtPlZkBDw3tI+l/BDFty3IGQBPavwPDyQPkwx6LlKmUMBbj7g+zsw8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776763626; c=relaxed/simple; bh=MR/GwJdH+bPFnJA+jLQ/+UtlUiXYet/1ZH/US7v3bHE=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=V7QQKXHXDugZ6bpVdzCHI/zm6JkJC9O4CfwdNtFyOlJRumvEvXtCTl6YSMJ5ICbihMjvqUxr9o+jU6bZCaIjYN3/0IcpYshM+NUv4gMIfWYksLC2a9RCVMk/68jSSHnXOIL/zv7fqzOvIk+5Drr8qw6q8deUOXGJ88klu1Tf/SA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Lm8NxsWt; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Lm8NxsWt" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8ABB3C2BCB0; Tue, 21 Apr 2026 09:27:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1776763625; bh=MR/GwJdH+bPFnJA+jLQ/+UtlUiXYet/1ZH/US7v3bHE=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=Lm8NxsWtNqVyR+muCCnRFtHlpHTjvsA01spMNnq0NEPA9JbrqgrK7fsn7nJQctOph 4I2w+QkPdRXDO8IUCvVMv0h95A1KrZy5QWxGRg4uaJK1MNS0OEpOQbJN2N6McpSu36 FDx3JdDMTUQNzwGkKKGQNLpFGlPelpPFddNM+MuiaZy7pSJ1KLgamqL3ZmIpobmiVX Qs83No089Dt+C4YANol96fE1BFYW2cF+LYu82ot0qUr8UK6bMHWLwgWwjJMP+pBZq4 bSKOvJdyNs8332tAB2cYTby9Eh78OHkOhL0uxanWJUaHSt+1aLGOdK8ggPRpGQ6Bl3 xD52IqviaC9sQ== Message-ID: <378ae036-6559-4b81-9bac-c8f0b73e1c98@kernel.org> Date: Tue, 21 Apr 2026 11:27:00 +0200 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2] mm/page_alloc: use batch page clearing in kernel_init_pages() Content-Language: en-US To: Hrushikesh Salunke , akpm@linux-foundation.org, david@kernel.org, ljs@kernel.org, Liam.Howlett@oracle.com, rppt@kernel.org, surenb@google.com, mhocko@suse.com, jackmanb@google.com, hannes@cmpxchg.org, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, rkodsara@amd.com, bharata@amd.com, ankur.a.arora@oracle.com, shivankg@amd.com References: <20260421042451.76918-1-hsalunke@amd.com> From: "Vlastimil Babka (SUSE)" In-Reply-To: <20260421042451.76918-1-hsalunke@amd.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 4/21/26 06:24, Hrushikesh Salunke wrote: > When init_on_alloc is enabled, kernel_init_pages() clears every page > one at a time via clear_highpage_kasan_tagged(), which incurs per-page > kmap_local_page()/kunmap_local() overhead and prevents the architecture > clearing primitive from operating on contiguous ranges. > > Introduce clear_highpages_kasan_tagged() in highmem.h, a batch > clearing helper that calls clear_pages() for the full contiguous range > on !HIGHMEM systems, bypassing the per-page kmap overhead and allowing > a single invocation of the arch clearing primitive across the entire > allocation. The HIGHMEM path falls back to per-page clearing since > those pages require kmap. > > Use it in kernel_init_pages() to replace the per-page loop. > > Allocating 8192 x 2MB HugeTLB pages (16GB) with init_on_alloc=1: > > Before: 0.445s > After: 0.166s (-62.7%, 2.68x faster) > > Kernel time (sys) reduction per workload with init_on_alloc=1: > > Workload Before After Change > Graph500 64C128T 30m 41.8s 15m 14.8s -50.3% > Graph500 16C32T 15m 56.7s 9m 43.7s -39.0% > Pagerank 32T 1m 58.5s 1m 12.8s -38.5% > Pagerank 128T 2m 36.3s 1m 40.4s -35.7% > > Signed-off-by: Hrushikesh Salunke > --- > base commit: f1541b40cd422d7e22273be9b7e9edfc9ea4f0d7 > > v1: https://lore.kernel.org/all/20260408092441.435133-1-hsalunke@amd.com/ > > Changes since v1: > - Dropped cond_resched() and PROCESS_PAGES_NON_PREEMPT_BATCH as > kernel_init_pages() runs inside the page allocator and can be > called from atomic context, making cond_resched() unsafe. The > original code never had a cond_resched() here, and the > performance gain comes from batching, not rescheduling. > > - Moved the !HIGHMEM/HIGHMEM branching into a new > clear_highpages_kasan_tagged() helper in highmem.h, per David's > suggestion. > > include/linux/highmem.h | 12 ++++++++++++ > mm/page_alloc.c | 5 +---- > 2 files changed, 13 insertions(+), 4 deletions(-) Acked-by: Vlastimil Babka (SUSE) > > diff --git a/include/linux/highmem.h b/include/linux/highmem.h > index af03db851a1d..ad0f42d06ce6 100644 > --- a/include/linux/highmem.h > +++ b/include/linux/highmem.h > @@ -345,6 +345,18 @@ static inline void clear_highpage_kasan_tagged(struct page *page) > kunmap_local(kaddr); > } > > +static inline void clear_highpages_kasan_tagged(struct page *page, int numpages) > +{ > + if (!IS_ENABLED(CONFIG_HIGHMEM)) { > + clear_pages(kasan_reset_tag(page_address(page)), numpages); > + } else { > + int i; > + > + for (i = 0; i < numpages; i++) > + clear_highpage_kasan_tagged(page + i); > + } > +} > + > #ifndef __HAVE_ARCH_TAG_CLEAR_HIGHPAGES > > /* Return false to let people know we did not initialize the pages */ > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > index b1c5430cad4e..1aaf7f839ff4 100644 > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -1220,12 +1220,9 @@ static inline bool should_skip_kasan_poison(struct page *page) > > static void kernel_init_pages(struct page *page, int numpages) > { > - int i; > - > /* s390's use of memset() could override KASAN redzones. */ > kasan_disable_current(); > - for (i = 0; i < numpages; i++) > - clear_highpage_kasan_tagged(page + i); > + clear_highpages_kasan_tagged(page, numpages); > kasan_enable_current(); > } >