From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 53EAD43440F; Mon, 13 Jul 2026 14:03:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783951417; cv=none; b=kmkhzIVSIil+0BMGtd/pP9s08xfX5u6ta0GvznKpDpz1h061aWuZE3oG6NBdENzsJfzsDOkeOXl3wyFg48cJafL5Is68T5Q9ryetKKqGiZbYgAfdKBlfpcgQQ+d4HgRU07oEWflOLAV7zmQdGDqeSnt2txkcHxryzDIowp0F4U4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783951417; c=relaxed/simple; bh=QAL4GYmN8cF0pmxGGh9bWrEOsYJOq6TzQh5SyXgApoA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XamWiigeduuKWi4fawwqqiYUYTUwCUaANlQ6oCJT1hs377vM4LjXirjR3Uua4zB46Dg1eHr4G/fTqXOKitlztkQS+8SSgWlPOSMhjhFyTgMZ+nml0WX0Rs1/OGYCGB1UMATvFrWOJ1Rz2bt2M6x9EAQIZn6vuoJ5UjV0NRE6KYY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=nFe7xAlP; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="nFe7xAlP" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9D6A62309; Mon, 13 Jul 2026 07:03:31 -0700 (PDT) Received: from e129823.cambridge.arm.com (e129823.arm.com [10.2.213.3]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id AF31A3F7B4; Mon, 13 Jul 2026 07:03:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1783951415; bh=QAL4GYmN8cF0pmxGGh9bWrEOsYJOq6TzQh5SyXgApoA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nFe7xAlP2LDvWkRo0KA0ExYj7kOvwZ3/PeBr/ZxemEskfg5Bkh306EL0w/W9x9JIN 6Rx+rn4+oXKAD3tpAI7lV/BuzB2WGkXK7sBpbgChBd8nXbW9Icfb0KCMOKBYCtCIox XMYIO7XpJyV2WpqTO0aUkz0XGFIpt8zutTPK0j+A= From: Yeoreum Yun To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-arch@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, x86@kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org Cc: david@kernel.org, linux@armlinux.org.uk, akpm@linux-foundation.org, ankur.a.arora@oracle.com, rppt@kernel.org, linmag7@gmail.com, chleroy@kernel.org, klarasmodin@gmail.com, chenhuacai@kernel.org, kernel@xen0n.name, kas@kernel.org, zhangtianyang@loongson.cn, wangyuli@aosc.io, tsbogend@alpha.franken.de, ljs@kernel.org, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, ryan.roberts@arm.com, pasha.tatashin@soleen.com, rmclure@linux.ibm.com, baolin.wang@linux.alibaba.com, tj@kernel.org, kevin.brodsky@arm.com, anup@brainfault.org, atish.patra@linux.dev, pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, hannes@cmpxchg.org, mhocko@kernel.org, qi.zheng@linux.dev, shakeel.butt@linux.dev, kasong@tencent.com, baohua@kernel.org, axelrasmussen@google.com, yuanchu@google.com, weixugc@google.com, ryabinin.a.a@gmail.com, glider@google.com, andreyknvl@gmail.com, dvyukov@google.com, vincenzo.frascino@arm.com, anshuman.khandual@arm.com, yang@os.amperecomputing.com, chaitanyas.prakash@arm.com, ardb@kernel.org, guoren@kernel.org, yang.li85200@gmail.com, viro@zeniv.linux.org.uk, dinguyen@kernel.org, schuster.simon@siemens-energy.com, wangruikang@iscas.ac.cn, junhui.liu@pigmoral.tech, muchun.song@linux.dev, vishal.moola@gmail.com, namcao@linutronix.de, pavel@kernel.org, djbw@kernel.org, yu-cheng.yu@intel.com, baolu.lu@linux.intel.com, Jonathan.Cameron@huawei.com, coxu@redhat.com, andreas@gaisler.com, liam@infradead.org, vbabka@kernel.org, surenb@google.com, mhocko@suse.com, geert@linux-m68k.org, shorne@gmail.com, jonas@southpole.se, stefan.kristiansson@saunalahti.fi, Yeoreum Yun Subject: [RFC PATCH 34/34] arm64: pgtable: convert pte_present() from macro to static inline Date: Mon, 13 Jul 2026 14:56:13 +0100 Message-ID: <20260713135614.1618183-35-yeoreum.yun@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260713135614.1618183-1-yeoreum.yun@arm.com> References: <20260713135614.1618183-1-yeoreum.yun@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit pte_present() is used as the basis for both pmd_present() and pud_present(). It is currently implemented as a macro composed of pte_val() and pte_present_invalid(). When pte_present() or its higher-level variants are used directly with ptep_get() or pXdp_get(), for example: pte_present(ptep_get(pte)); pmd_present(pmdp_get(pmd)); pud_present(pudp_get(pud)); the macro expansion causes the compiler to evaluate the argument twice, resulting in redundant loads. For example, pte_present() expands to: !pte_val(READ_ONCE(*pte) || pte_present_invalid(READ_ONCE(*pte)) A typical example is pud_free_pmd_page(), where the expansion of pmd_present() generates: ... /* pmd_present() (x20 = pmdp) */ 1b88: f9400288 ldr x8, [x20] // read pmdp. 1b8c: f9000fa8 str x8, [x29, #0x18] 1b90: 3707fec8 tbnz w8, #0x0, 0x1b68 1b94: f9400288 ldr x8, [x20] // redundant read of pmdp. 1b98: 8a170109 and x9, x8, x23 1b9c: f9000fa8 str x8, [x29, #0x18] 1ba0: f120013f cmp x9, #0x800 1ba4: 54fffe20 b.eq 0x1b68 1ba8: 17fffff4 b 0x1b78 ... Convert pte_present() to static inline function so that prevent the generation of redundant code and move pte_valid() and pte_present_invalid() further up so the inline function can use them. After this change, the generated code becomes: ... /* pmd_present() (x20 = pmdp) */ 1a30: f9400288 ldr x8, [x20] 1a34: 8a170109 and x9, x8, x23 1a38: f9000fa8 str x8, [x29, #0x18] 1a3c: f120013f cmp x9, #0x800 1a40: 54fffe80 b.eq 0x1a10 1a44: 3607fee8 tbz w8, #0x0, 0x1a20 1a48: 17fffff2 b 0x1a10 ... This eliminates the redundant load and also reduces code size at call sites using this pattern. For example, pud_free_pmd_page() shrinks from 7,500 bytes to 7,148 bytes, a reduction of approximately 4.7%. Signed-off-by: Yeoreum Yun --- arch/arm64/include/asm/pgtable.h | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 6185fc291fd7d..aaae3d14895f6 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -140,10 +140,17 @@ static inline pteval_t __phys_to_pte_val(phys_addr_t phys) #define pte_none(pte) (!pte_val(pte)) #define pte_page(pte) (pfn_to_page(pte_pfn(pte))) +#define pte_valid(pte) (!!(pte_val(pte) & PTE_VALID)) +#define pte_present_invalid(pte) \ + ((pte_val(pte) & (PTE_VALID | PTE_PRESENT_INVALID)) == PTE_PRESENT_INVALID) + /* * The following only work if pte_present(). Undefined behaviour otherwise. */ -#define pte_present(pte) (pte_valid(pte) || pte_present_invalid(pte)) +static __always_inline bool pte_present(pte_t pte) +{ + return pte_valid(pte) || pte_present_invalid(pte); +} #define pte_young(pte) (!!(pte_val(pte) & PTE_AF)) #define pte_special(pte) (!!(pte_val(pte) & PTE_SPECIAL)) #define pte_write(pte) (!!(pte_val(pte) & PTE_WRITE)) @@ -168,9 +175,6 @@ static inline pteval_t __phys_to_pte_val(phys_addr_t phys) #define pte_sw_dirty(pte) (!!(pte_val(pte) & PTE_DIRTY)) #define pte_dirty(pte) (pte_sw_dirty(pte) || pte_hw_dirty(pte)) -#define pte_valid(pte) (!!(pte_val(pte) & PTE_VALID)) -#define pte_present_invalid(pte) \ - ((pte_val(pte) & (PTE_VALID | PTE_PRESENT_INVALID)) == PTE_PRESENT_INVALID) /* * Execute-only user mappings do not have the PTE_USER bit set. All valid * kernel mappings have the PTE_UXN bit set. -- LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7}