From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A02DF7081F for ; Sat, 7 Mar 2026 01:28:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.97 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772846918; cv=none; b=W+VYNEo8lClmeQE95bbR+JwDhkxIzv1RvreDrCXGb24w1ZxWe/FRGxQ5s46YJKEdIhEkRi9BuwSsgYkuC4L8P+RS2D0Rr14c3H/V51ZiVw3XNhbrlL362N5LQUsCdFG7jFnaZUv9Q6z4irk7NXKIo8P+e29PLJ7QlZ+OgbrvUhE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772846918; c=relaxed/simple; bh=uSG9eIJhOXu3uQPwzO+MDPCA0bQwq9oozlaMtmHUCh0=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=OfgoqsGLIN+2h7hJwQoafVDRpuCOF3XukcpR1OhjPaTS6DJyRna5jGZP4rALQmA7efmr15P0hRzq1lMj8KA5+k04NFLkcN+HWLIvZBUGSmwufo+F751uGJy643LP5euK7HADAzVY6rBQ/lEjo0AKDz9NaPBoIH1JhO7hlasREDE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=aAuJR7XS; arc=none smtp.client-ip=115.124.30.97 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="aAuJR7XS" DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1772846912; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type; bh=kzar5nNFlAv1Huoqo+I+k0T1XEOUIvMAuNiWgwlVg7Y=; b=aAuJR7XSCbY/gsKxSmIVhl/WGbt2EQhHKbCZXReNx+tSKAF03aIAZeuIqkZA3rwAYBZfCuqVIZETCL63xzNRaenWRmeBdePMQodsg4pbUZM37uiIaLNk3pmG0MEov3Jr6/MOjNnQfdHk6dNQVzG/Z8tpi7e6YRIUvuzWxM1c7oA= Received: from 30.42.98.36(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0X-OMjm5_1772846909 cluster:ay36) by smtp.aliyun-inc.com; Sat, 07 Mar 2026 09:28:30 +0800 Message-ID: Date: Sat, 7 Mar 2026 09:28:29 +0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 6/6] arm64: mm: implement the architecture-specific test_and_clear_young_ptes() To: "David Hildenbrand (Arm)" , akpm@linux-foundation.org Cc: catalin.marinas@arm.com, will@kernel.org, lorenzo.stoakes@oracle.com, ryan.roberts@arm.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, riel@surriel.com, harry.yoo@oracle.com, jannh@google.com, willy@infradead.org, baohua@kernel.org, dev.jain@arm.com, axelrasmussen@google.com, yuanchu@google.com, weixugc@google.com, hannes@cmpxchg.org, zhengqi.arch@bytedance.com, shakeel.butt@linux.dev, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org References: <7f891d42a720cc2e57862f3b79e4f774404f313c.1772778858.git.baolin.wang@linux.alibaba.com> <6305e05e-2911-42b0-b6f5-7fdde787b778@kernel.org> From: Baolin Wang In-Reply-To: <6305e05e-2911-42b0-b6f5-7fdde787b778@kernel.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 3/6/26 10:47 PM, David Hildenbrand (Arm) wrote: > On 3/6/26 07:43, Baolin Wang wrote: >> Implement the Arm64 architecture-specific test_and_clear_young_ptes() to enable >> batched checking of young flags, improving performance during large folio >> reclamation when MGLRU is enabled. >> >> While we're at it, simplify ptep_test_and_clear_young() by calling >> test_and_clear_young_ptes(). Since callers guarantee that PTEs are present >> before calling these functions, we can use pte_cont() to check the CONT_PTE >> flag instead of pte_valid_cont(). >> >> Performance testing: >> Enable MGLRU, then allocate 10G clean file-backed folios by mmap() in a memory >> cgroup, and try to reclaim 8G file-backed folios via the memory.reclaim interface. >> I can observe 60%+ performance improvement on my Arm64 32-core server (and about >> 15% improvement on my X86 machine). >> >> W/o patchset: >> real 0m0.470s >> user 0m0.000s >> sys 0m0.470s >> >> W/ patchset: >> real 0m0.180s >> user 0m0.001s >> sys 0m0.179s >> >> Reviewed-by: Rik van Riel >> Signed-off-by: Baolin Wang >> --- >> arch/arm64/include/asm/pgtable.h | 18 ++++++++++++------ >> 1 file changed, 12 insertions(+), 6 deletions(-) >> >> diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h >> index aa4b13da6371..ab451d20e4c5 100644 >> --- a/arch/arm64/include/asm/pgtable.h >> +++ b/arch/arm64/include/asm/pgtable.h >> @@ -1812,16 +1812,22 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm, >> return __ptep_get_and_clear(mm, addr, ptep); >> } >> >> +#define test_and_clear_young_ptes test_and_clear_young_ptes >> +static inline int test_and_clear_young_ptes(struct vm_area_struct *vma, >> + unsigned long addr, pte_t *ptep, >> + unsigned int nr) >> +{ >> + if (likely(nr == 1 && !pte_cont(__ptep_get(ptep)))) >> + return __ptep_test_and_clear_young(vma, addr, ptep); >> + >> + return contpte_test_and_clear_young_ptes(vma, addr, ptep, nr); >> +} > > Thinking out loud, what would happen if Good questions, I think the contpte_test_and_clear_young_ptes() takes that into account. > (a) The range spans multiple possible cont ranges (like, 64 ptes). The contpte_test_and_clear_young_ptes() will call contpte_align_addr_ptep() to align the range to cont‑block boundary, that means the range can span multiple cont blocks. int contpte_test_and_clear_young_ptes(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, unsigned int nr) { unsigned long end = addr + nr * PAGE_SIZE; int young = 0; ptep = contpte_align_addr_ptep(&addr, &end, ptep, nr); for (; addr != end; ptep++, addr += PAGE_SIZE) young |= __ptep_test_and_clear_young(vma, addr, ptep); return young; } > > (b) The first pte is !pte_cont(), but some others in there are? IMO they can’t be handled in a single batch. Since the folio_pte_batch() will group consecutive !cont PTEs into one batch and consecutive cont PTEs into another (assume all PTEs belong to a single large folio), because their PTE entries have different CONT bits. Even if the callers do so, contpte_align_addr_ptep() will check the pte_cont() of the start and end address to align the range appropriately.