From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1793618541; b=1u98+ZKHd8zdNMrenY+UZYcbMqsF2L4b1neuc0gQisktuu+yhwmEkGkMWPVHFlDTAo6F F7YcaG7co5tyc+ckz5cT1JvQek5ykeTiN3rlqQmFKEaWI9B2gFBSqURnckjwxRREv04Lp XBx7ohxZZVTiwO/c3+dvqo8/8Q5tOBmO2+aC7yxIjO7MNChlml5cxjxs698N63e8cGbar jUMhLHS0kgYYcfThNB7PE3lzu3Eb6zvHt8SKR3/qpiAVCe182o+TJsjf58xRac/oV0xNG Zt5VJYcfY7Rj/kjP8iJlxh2vqQ5a88F2eNAX4Zr7a0uA9d9N8mwOODOSPGbYlzXt2zQ== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1793618541; h=DMARC-Filter:Received:Received:DKIM-Signature:From:Date:Subject: MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id: References:In-Reply-To:To:Cc:X-Mailer:X-Developer-Signature: X-Developer-Key; bh=AhZPH2Id1V3UJZ0PUDk/Q6jSjnsEPYgfRRz51ht6Rp4=; b=lo4sMriqfdxb1BwpURpg0crY39/agBY1L0vs5k+IJeJA27EMH9h6/lMInDLDDou7MtfT pqgo/EtHM92cXG7FzWTbqa2t1mnyEyJT3w8/h1+gI5bdCl46T8oFBIyy4D9UsykVZXgi1 tCZwCtYInRN21dVTVmhgqhfRgkkWeuDLg01bX77rNMMQ3dTF3fnCChQHf6l4jOLT5N6LQ x4mYHwG7EH48AVFa5YLzxJ/40nvxc0XeOK4Krtp89PJJkAgjIsDgvT51wxFGPsLOUC2Ec DYBLJNB/r853Fh0fXlpwBHEvVzHoX9eQnaMYQA6Misv8m2FW4PX7U/9EAp7alEEJidA== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=foss header.b="PsZ/xV55"; arc=none smtp.remote-ip=217.140.110.172 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C5AAB60A2633; Mon, 2 Nov 2026 11:22:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org C5AAB60A2633 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key, unprotected) header.d=arm.com header.i=@arm.com header.a=rsa-sha256 header.s=foss header.b=PsZ/xV55 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1E6232972; Mon, 2 Nov 2026 03:22:17 -0800 (PST) Received: from e129823.arm.com (e129823.arm.com [10.2.213.3]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2DD373F632; Mon, 2 Nov 2026 03:22:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1793618540; bh=iAKWhkLBUL8Dbje3w+lN/zrtNjFP/+2E4XMbJAzkSwE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=PsZ/xV55JWI15/OuGGIzIxarKfdNw1USnk6yd5/qGxAT2iQNKUyjmZiknVsIG9EAM FHVIK/r0dSwZlieqJ6i4Ub47uuLajcoviLMk9DLJUT8YpfdyuFurfnAbmwFqi15Qes azpHEA/ThKpZh5kfXCYp/IH1C3ruHlJgAZacMHks= From: Yeoreum Yun Date: Mon, 02 Nov 2026 11:21:20 +0000 Subject: [PATCH RESEND v2 07/21] mm/pgtable: define (pgd|p4d|pud)_offset_lockless() for folded page tables X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20261102-dummy_ptxp3-v2-7-78840b2ec73c@arm.com> References: <20261102-dummy_ptxp3-v2-0-78840b2ec73c@arm.com> In-Reply-To: <20261102-dummy_ptxp3-v2-0-78840b2ec73c@arm.com> To: Russell King , Huacai Chen , WANG Xuerui , Thomas Bogendoerfer , Catalin Marinas , Will Deacon , Arnd Bergmann , Andrew Morton , Kairui Song , Qi Zheng , Shakeel Butt , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Johannes Weiner , David Hildenbrand , Michal Hocko , Lorenzo Stoakes , Tianrui Zhao , Bibo Mao , Anup Patel , Atish Patra , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H. Peter Anvin" , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Jonas Bonn , Stefan Kristiansson , Stafford Horne Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-openrisc@vger.kernel.org X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4366; i=yeoreum.yun@arm.com; h=from:subject:message-id; bh=u76iVQJbWCdkEbiJ5jZQxWZGCtfm9Pp2msKkXET/qwE=; b=owEB7QES/pANAwAKAW3Vw9FaxTEzAcsmYgBq6HIypEb9aXVoCMG7pxsrAGeq/LTHoW1uWXEBM pIwPB40LByJAbMEAAEKAB0WIQQtg+CS3QUzuFh1pJ1t1cPRWsUxMwUCauhyMgAKCRBt1cPRWsUx MzzkDADQPtRN+DQ0QdZ/E5ulISLF3JkAQe33ZVd/+S87WO+Sy3zOmf/s77mek/A86gwCB52rGz6 NsFtjGs+ALYco+k3aCZaOJ+4OI+GsRLUJBA8kK1l/X00CILC9t0qmJGYamB8dsPxPMO+M21DwVI /rk19ar+YAisNFKK7f0r5JRGn1r6rtA/zTelmesjg8qXXW2d3R/orYzgSRJNRVGfHplzMvW5WF7 2SV4kMDEQq/4kLJUcVebxdFAS9HJCf3XrJTRtiDgy0aEVXoT+q2DWd2LJKPSA+QL6zMvxFgIt8D t6P+wsuF+ihqujWnCZw/I+PE5C+Jd11Rk9vGE3JKnMrNmMnR1Sp2jL30ZQcG/HwQ7coqXiYVVSk vhNQdA4aQ9WI4ceztWVvVzhXt1frhLl1HR8P0NGGqLVFaAJL3L+M+HjZBrU9dMhkbtwMTx/1+S9 SzYhL7QXCQWSjEax1AfhVDQ8sPZIqGPWAtZQ/BR8IPZRp8AoNMk/iHqhsX16+teiHOk3Y= X-Developer-Key: i=yeoreum.yun@arm.com; a=openpgp; fpr=2D83E092DD0533B85875A49D6DD5C3D15AC53133 From: "David Hildenbrand (Arm)" The default helpers fallback to pud_offset(), passing a pointer to the stack value. Let's provide variants that do exactly what (pgd|p4d|pud)_offset() do, but ignore the passed pgd_t/p4d_t/pud_t, consequently not working on a pointer to the stack value like the default handlers would. With this patch, Drop the arm64 custom variant that did that. As it documents: Note: reusing the original pointer means that it may be dereferenced the same (live) page-table entry multiple times. This is safe because it is still only loaded once in the context of each level and the CPU guarantees same-address read-after-read ordering. Do not document this, as the double read will be removed in the next patch. Signed-off-by: David Hildenbrand (Arm) --- arch/arm64/include/asm/pgtable.h | 23 ----------------------- include/asm-generic/pgtable-nop4d.h | 7 +++++++ include/asm-generic/pgtable-nopmd.h | 7 +++++++ include/asm-generic/pgtable-nopud.h | 7 +++++++ 4 files changed, 21 insertions(+), 23 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 763c5a411d64e..b391f325670f8 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1141,29 +1141,6 @@ static inline bool pgtable_l5_enabled(void) { return false; } #define p4d_clear_fixmap() #define p4d_offset_kimg(dir,addr) ((p4d_t *)dir) - -static inline -p4d_t *p4d_offset_lockless_folded(pgd_t *pgdp, pgd_t pgd, unsigned long addr) -{ - /* - * With runtime folding of the pud, pud_offset_lockless() passes - * the 'pgd_t *' we return here to p4d_to_folded_pud(), which - * will offset the pointer assuming that it points into - * a page-table page. However, the fast GUP path passes us a - * pgd_t allocated on the stack and so we must use the original - * pointer in 'pgdp' to construct the p4d pointer instead of - * using the generic p4d_offset_lockless() implementation. - * - * Note: reusing the original pointer means that we may - * dereference the same (live) page-table entry multiple times. - * This is safe because it is still only loaded once in the - * context of each level and the CPU guarantees same-address - * read-after-read ordering. - */ - return p4d_offset(pgdp, addr); -} -#define p4d_offset_lockless p4d_offset_lockless_folded - #endif /* CONFIG_PGTABLE_LEVELS > 4 */ #define pgd_set_fixmap(addr) ((pgd_t *)set_fixmap_offset(FIX_PGD, addr)) diff --git a/include/asm-generic/pgtable-nop4d.h b/include/asm-generic/pgtable-nop4d.h index c6a5a43899b50..019c3f074b771 100644 --- a/include/asm-generic/pgtable-nop4d.h +++ b/include/asm-generic/pgtable-nop4d.h @@ -39,6 +39,13 @@ static inline p4d_t *p4d_offset(pgd_t *pgd, unsigned long address) return (p4d_t *)pgd; } +static inline p4d_t *p4d_offset_lockless(pgd_t *pgdp, pgd_t pgd, + unsigned long address) +{ + return (p4d_t *)pgdp; +} +#define p4d_offset_lockless p4d_offset_lockless + #define p4d_val(x) (pgd_val((x).pgd)) #define __p4d(x) ((p4d_t) { __pgd(x) }) diff --git a/include/asm-generic/pgtable-nopmd.h b/include/asm-generic/pgtable-nopmd.h index 4e5aaf253b6a5..1dd5b165234be 100644 --- a/include/asm-generic/pgtable-nopmd.h +++ b/include/asm-generic/pgtable-nopmd.h @@ -49,6 +49,13 @@ static inline pmd_t * pmd_offset(pud_t * pud, unsigned long address) } #define pmd_offset pmd_offset +static inline pmd_t *pmd_offset_lockless(pud_t *pudp, pud_t pud, + unsigned long address) +{ + return (pmd_t *)pudp; +} +#define pmd_offset_lockless pmd_offset_lockless + #define pmd_val(x) (pud_val((x).pud)) #define __pmd(x) ((pmd_t) { __pud(x) } ) diff --git a/include/asm-generic/pgtable-nopud.h b/include/asm-generic/pgtable-nopud.h index 6c9bca78047c4..5a2b0a81ae197 100644 --- a/include/asm-generic/pgtable-nopud.h +++ b/include/asm-generic/pgtable-nopud.h @@ -47,6 +47,13 @@ static inline pud_t *pud_offset(p4d_t *p4d, unsigned long address) } #define pud_offset pud_offset +static inline pud_t *pud_offset_lockless(p4d_t *p4dp, p4d_t p4d, + unsigned long address) +{ + return (pud_t *)p4dp; +} +#define pud_offset_lockless pud_offset_lockless + #define pud_val(x) (p4d_val((x).p4d)) #define __pud(x) ((pud_t) { __p4d(x) }) -- 2.43.0