From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B4213D666F for ; Tue, 21 Apr 2026 13:57:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776779831; cv=none; b=VDjEsZ9/ZM914zQrSRoxALRm6q2ZH80Utv98qwLI5/s58QD2Ao8F0bk8tF3vv0wCjD5PR6vE+d2Vy5XoI+gzOM4btXBHgpcP5Fy1C7ZKHYW+sS0pMvDZlb8rXBJaLli/SxWLtAPkS7tjFe8XfM7onLU1gmCQi3OtFRHqusac3iU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776779831; c=relaxed/simple; bh=dHSSxhVkg9bb3OFn8crTULXaFwNtsY6qMfn9yU6Cs9U=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=RjVqRWX9+zEGbMZXqQHt9/BVLUQwfmGALpbGLbpGp57q2ZQoRfs2ac3pJ98KA4k5WKd8J5qjb3NIjtnmk6UdJyPzB5i8lptgt1achumKw5IbNgRiOBS++5AQlCyeO1N1vIz2IxYsbdaHDT5dqF6eorv3nw9BnMpPzAPGtLHTrJM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nCiSy1UD; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nCiSy1UD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 45468C2BCFD; Tue, 21 Apr 2026 13:57:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1776779831; bh=dHSSxhVkg9bb3OFn8crTULXaFwNtsY6qMfn9yU6Cs9U=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=nCiSy1UDJwK/BAgBOZNPukrXxRM6ZUA3u3iEBLaMLMwOUFGZnWfr+DGLlsi/9VubI tsvG9oZ6Owq5xwdgKbAFbBPO0l5ZOY6xJ4R2ZL4Cuw8JnalH+0GSQfJUTnJWUQrOpy SOAaEFGCmL6hxLA0ytJN6bb/8QFE7zPyMLq/qOyvr7Yho0Haqk5MQpT15EPdzL17Rh ZgjEufjSXeDFnqDtwtKraONH0pcwKca9UX9L44VOrP+0uAcIDnBvN0eIK5xjdwvfNt dUxF5y9iWuiayI28SvB8L1cjSjkqGnivM0aiLFfQfwgzStwQRmfQM9luuw75ghxkgp pjXh9AcQ0cBdw== Message-ID: Date: Tue, 21 Apr 2026 15:57:05 +0200 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2] mm/page_alloc: use batch page clearing in kernel_init_pages() To: Zi Yan , Hrushikesh Salunke Cc: akpm@linux-foundation.org, ljs@kernel.org, Liam.Howlett@oracle.com, vbabka@kernel.org, rppt@kernel.org, surenb@google.com, mhocko@suse.com, jackmanb@google.com, hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, rkodsara@amd.com, bharata@amd.com, ankur.a.arora@oracle.com, shivankg@amd.com References: <20260421042451.76918-1-hsalunke@amd.com> From: "David Hildenbrand (Arm)" Content-Language: en-US Autocrypt: addr=david@kernel.org; keydata= xsFNBFXLn5EBEAC+zYvAFJxCBY9Tr1xZgcESmxVNI/0ffzE/ZQOiHJl6mGkmA1R7/uUpiCjJ dBrn+lhhOYjjNefFQou6478faXE6o2AhmebqT4KiQoUQFV4R7y1KMEKoSyy8hQaK1umALTdL QZLQMzNE74ap+GDK0wnacPQFpcG1AE9RMq3aeErY5tujekBS32jfC/7AnH7I0v1v1TbbK3Gp XNeiN4QroO+5qaSr0ID2sz5jtBLRb15RMre27E1ImpaIv2Jw8NJgW0k/D1RyKCwaTsgRdwuK Kx/Y91XuSBdz0uOyU/S8kM1+ag0wvsGlpBVxRR/xw/E8M7TEwuCZQArqqTCmkG6HGcXFT0V9 PXFNNgV5jXMQRwU0O/ztJIQqsE5LsUomE//bLwzj9IVsaQpKDqW6TAPjcdBDPLHvriq7kGjt WhVhdl0qEYB8lkBEU7V2Yb+SYhmhpDrti9Fq1EsmhiHSkxJcGREoMK/63r9WLZYI3+4W2rAc UucZa4OT27U5ZISjNg3Ev0rxU5UH2/pT4wJCfxwocmqaRr6UYmrtZmND89X0KigoFD/XSeVv jwBRNjPAubK9/k5NoRrYqztM9W6sJqrH8+UWZ1Idd/DdmogJh0gNC0+N42Za9yBRURfIdKSb B3JfpUqcWwE7vUaYrHG1nw54pLUoPG6sAA7Mehl3nd4pZUALHwARAQABzS5EYXZpZCBIaWxk ZW5icmFuZCAoQ3VycmVudCkgPGRhdmlkQGtlcm5lbC5vcmc+wsGQBBMBCAA6AhsDBQkmWAik AgsJBBUKCQgCFgICHgUCF4AWIQQb2cqtc1xMOkYN/MpN3hD3AP+DWgUCaYJt/AIZAQAKCRBN 3hD3AP+DWriiD/9BLGEKG+N8L2AXhikJg6YmXom9ytRwPqDgpHpVg2xdhopoWdMRXjzOrIKD g4LSnFaKneQD0hZhoArEeamG5tyo32xoRsPwkbpIzL0OKSZ8G6mVbFGpjmyDLQCAxteXCLXz ZI0VbsuJKelYnKcXWOIndOrNRvE5eoOfTt2XfBnAapxMYY2IsV+qaUXlO63GgfIOg8RBaj7x 3NxkI3rV0SHhI4GU9K6jCvGghxeS1QX6L/XI9mfAYaIwGy5B68kF26piAVYv/QZDEVIpo3t7 /fjSpxKT8plJH6rhhR0epy8dWRHk3qT5tk2P85twasdloWtkMZ7FsCJRKWscm1BLpsDn6EQ4 jeMHECiY9kGKKi8dQpv3FRyo2QApZ49NNDbwcR0ZndK0XFo15iH708H5Qja/8TuXCwnPWAcJ DQoNIDFyaxe26Rx3ZwUkRALa3iPcVjE0//TrQ4KnFf+lMBSrS33xDDBfevW9+Dk6IISmDH1R HFq2jpkN+FX/PE8eVhV68B2DsAPZ5rUwyCKUXPTJ/irrCCmAAb5Jpv11S7hUSpqtM/6oVESC 3z/7CzrVtRODzLtNgV4r5EI+wAv/3PgJLlMwgJM90Fb3CB2IgbxhjvmB1WNdvXACVydx55V7 LPPKodSTF29rlnQAf9HLgCphuuSrrPn5VQDaYZl4N/7zc2wcWM7BTQRVy5+RARAA59fefSDR 9nMGCb9LbMX+TFAoIQo/wgP5XPyzLYakO+94GrgfZjfhdaxPXMsl2+o8jhp/hlIzG56taNdt VZtPp3ih1AgbR8rHgXw1xwOpuAd5lE1qNd54ndHuADO9a9A0vPimIes78Hi1/yy+ZEEvRkHk /kDa6F3AtTc1m4rbbOk2fiKzzsE9YXweFjQvl9p+AMw6qd/iC4lUk9g0+FQXNdRs+o4o6Qvy iOQJfGQ4UcBuOy1IrkJrd8qq5jet1fcM2j4QvsW8CLDWZS1L7kZ5gT5EycMKxUWb8LuRjxzZ 3QY1aQH2kkzn6acigU3HLtgFyV1gBNV44ehjgvJpRY2cC8VhanTx0dZ9mj1YKIky5N+C0f21 zvntBqcxV0+3p8MrxRRcgEtDZNav+xAoT3G0W4SahAaUTWXpsZoOecwtxi74CyneQNPTDjNg azHmvpdBVEfj7k3p4dmJp5i0U66Onmf6mMFpArvBRSMOKU9DlAzMi4IvhiNWjKVaIE2Se9BY FdKVAJaZq85P2y20ZBd08ILnKcj7XKZkLU5FkoA0udEBvQ0f9QLNyyy3DZMCQWcwRuj1m73D sq8DEFBdZ5eEkj1dCyx+t/ga6x2rHyc8Sl86oK1tvAkwBNsfKou3v+jP/l14a7DGBvrmlYjO 59o3t6inu6H7pt7OL6u6BQj7DoMAEQEAAcLBfAQYAQgAJgIbDBYhBBvZyq1zXEw6Rg38yk3e EPcA/4NaBQJonNqrBQkmWAihAAoJEE3eEPcA/4NaKtMQALAJ8PzprBEXbXcEXwDKQu+P/vts IfUb1UNMfMV76BicGa5NCZnJNQASDP/+bFg6O3gx5NbhHHPeaWz/VxlOmYHokHodOvtL0WCC 8A5PEP8tOk6029Z+J+xUcMrJClNVFpzVvOpb1lCbhjwAV465Hy+NUSbbUiRxdzNQtLtgZzOV Zw7jxUCs4UUZLQTCuBpFgb15bBxYZ/BL9MbzxPxvfUQIPbnzQMcqtpUs21CMK2PdfCh5c4gS sDci6D5/ZIBw94UQWmGpM/O1ilGXde2ZzzGYl64glmccD8e87OnEgKnH3FbnJnT4iJchtSvx yJNi1+t0+qDti4m88+/9IuPqCKb6Stl+s2dnLtJNrjXBGJtsQG/sRpqsJz5x1/2nPJSRMsx9 5YfqbdrJSOFXDzZ8/r82HgQEtUvlSXNaXCa95ez0UkOG7+bDm2b3s0XahBQeLVCH0mw3RAQg r7xDAYKIrAwfHHmMTnBQDPJwVqxJjVNr7yBic4yfzVWGCGNE4DnOW0vcIeoyhy9vnIa3w1uZ 3iyY2Nsd7JxfKu1PRhCGwXzRw5TlfEsoRI7V9A8isUCoqE2Dzh3FvYHVeX4Us+bRL/oqareJ CIFqgYMyvHj7Q06kTKmauOe4Nf0l0qEkIuIzfoLJ3qr5UyXc2hLtWyT9Ir+lYlX9efqh7mOY qIws/H2t In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 4/21/26 15:44, Zi Yan wrote: > On 21 Apr 2026, at 0:24, Hrushikesh Salunke wrote: > >> When init_on_alloc is enabled, kernel_init_pages() clears every page >> one at a time via clear_highpage_kasan_tagged(), which incurs per-page >> kmap_local_page()/kunmap_local() overhead and prevents the architecture >> clearing primitive from operating on contiguous ranges. >> >> Introduce clear_highpages_kasan_tagged() in highmem.h, a batch >> clearing helper that calls clear_pages() for the full contiguous range >> on !HIGHMEM systems, bypassing the per-page kmap overhead and allowing >> a single invocation of the arch clearing primitive across the entire >> allocation. The HIGHMEM path falls back to per-page clearing since >> those pages require kmap. >> >> Use it in kernel_init_pages() to replace the per-page loop. >> >> Allocating 8192 x 2MB HugeTLB pages (16GB) with init_on_alloc=1: >> >> Before: 0.445s >> After: 0.166s (-62.7%, 2.68x faster) >> >> Kernel time (sys) reduction per workload with init_on_alloc=1: >> >> Workload Before After Change >> Graph500 64C128T 30m 41.8s 15m 14.8s -50.3% >> Graph500 16C32T 15m 56.7s 9m 43.7s -39.0% >> Pagerank 32T 1m 58.5s 1m 12.8s -38.5% >> Pagerank 128T 2m 36.3s 1m 40.4s -35.7% >> >> Signed-off-by: Hrushikesh Salunke >> --- >> base commit: f1541b40cd422d7e22273be9b7e9edfc9ea4f0d7 >> >> v1: https://lore.kernel.org/all/20260408092441.435133-1-hsalunke@amd.com/ >> >> Changes since v1: >> - Dropped cond_resched() and PROCESS_PAGES_NON_PREEMPT_BATCH as >> kernel_init_pages() runs inside the page allocator and can be >> called from atomic context, making cond_resched() unsafe. The >> original code never had a cond_resched() here, and the >> performance gain comes from batching, not rescheduling. >> >> - Moved the !HIGHMEM/HIGHMEM branching into a new >> clear_highpages_kasan_tagged() helper in highmem.h, per David's >> suggestion. >> >> include/linux/highmem.h | 12 ++++++++++++ >> mm/page_alloc.c | 5 +---- >> 2 files changed, 13 insertions(+), 4 deletions(-) >> >> diff --git a/include/linux/highmem.h b/include/linux/highmem.h >> index af03db851a1d..ad0f42d06ce6 100644 >> --- a/include/linux/highmem.h >> +++ b/include/linux/highmem.h >> @@ -345,6 +345,18 @@ static inline void clear_highpage_kasan_tagged(struct page *page) >> kunmap_local(kaddr); >> } >> >> +static inline void clear_highpages_kasan_tagged(struct page *page, int numpages) >> +{ >> + if (!IS_ENABLED(CONFIG_HIGHMEM)) { >> + clear_pages(kasan_reset_tag(page_address(page)), numpages); > > kasan_reset_tag() here removes the tag from page address, so that > clear_pages() can use the right kaddr. I thought each page needs > a kasan_reset_tag(). No need to respond here, as I am reading > the code and trying to understand how it works. It's all confusing. But we really just turn the pointer into an untagged pointer here, once. So I think this is ok. I do wonder, though, whether we want to move the kasan_disable_current/kasan_enable_current into the clear_highpages_kasan_tagged(). -- Cheers, David