From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 09D9481AA8 for ; Fri, 17 Jul 2026 17:00:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=96.67.55.147 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784307644; cv=none; b=VF3cEcKnWfZma14fb8R9kzm163NLT12Ywfitidx5sVlu5i27cqOmuU1kSgQBbZFKVAHcpsx6ApjOiyJvzkyes+dO41MEye/3T0NVHFKL8XC3xdDL3VKWNqZZVKJtyxpyfLHomexOPoj0blQtHD4DxYBGuPqlxzN/etYhH4hZXUE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784307644; c=relaxed/simple; bh=PgtKYMJnpgoVxoWhxkkLNSTm2PQ3iOoRAiNeH/3ro+o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=piNsYzBlKoH9Jpi7ttg9p7pt6m/S/xle90bsG1HWK2/CHbTpH1mgLt9oJCBKVgZGNbL6WQF62m/bFG22JKou+G2ztggLRnL6zXR75V30DVmyqfp7yqOp6ByZksftG9g2mXOOu2Z0dXVeE9HsKSaYmaT8DXC/WnB0COeCNxSBROU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=surriel.com; spf=pass smtp.mailfrom=surriel.com; dkim=pass (2048-bit key) header.d=surriel.com header.i=@surriel.com header.b=keJ/nUbp; arc=none smtp.client-ip=96.67.55.147 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=surriel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=surriel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=surriel.com header.i=@surriel.com header.b="keJ/nUbp" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=surriel.com ; s=mail; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=iUviapJKXkmCmGZC2J3yV4WlPPv4IevEHJi7vfOXIEY=; b=keJ/nUbptoUuFh/dToE97NtnUe VF5w74JwYDghMknIO1vCHJRLKILijbOsApAT2KOhOMApTwn3oSPvSO13CSd9sdT4/uIBn/JSu2n1W KmwmQLCY5Fc1HbS1rG6cq7Ff2QCxlzAeJnGmGYHr+9TXVsCX7of/X4tLnbiK7BHXHX2pwNyxw0yB3 ZPU9ADhqr5QkLi74kuE9XKZSM8lUNJUK+4Xi6xVvaZzaICVclMULUqYXAqvVP/9OVNO+2sEcc2Jha kn2HLdLkIL3x/SCM1fP0tXsnO7AeL6olcpxFOY5X7yJpijb9R9LZV3kgyAcp/fOGTSV52EktqYOn0 UxSzeejg==; Received: from fangorn.home.surriel.com ([10.0.13.7]) by shelob.surriel.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.97.1) (envelope-from ) id 1wklvS-000000001w4-0DlG; Fri, 17 Jul 2026 13:00:38 -0400 From: Rik van Riel To: linux-kernel@vger.kernel.org, Andrew Morton Cc: kernel-team@meta.com, Rik van Riel , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , linux-mm@kvack.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Usama Arif Subject: [PATCH RFC v3 1/6] x86/mm: add untagged_addr_remote_unlocked() Date: Fri, 17 Jul 2026 13:00:31 -0400 Message-ID: <20260717170036.743149-2-riel@surriel.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260717170036.743149-1-riel@surriel.com> References: <20260717170036.743149-1-riel@surriel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit __access_remote_vm() reads another process's memory under the mmap lock. On large machines that lock is contended by tasks polling /proc/PID/cmdline, /proc/PID/environ, or calling process_vm_readv(), even though the memory they read is almost always resident and could be reached under the per-VMA lock instead. Looking up the VMA first requires untagging the address. untagged_addr_remote() asserts the mmap lock only because it reads mm->context.untag_mask, which can race with the write in mm_enable_lam(). That mask is set once, when LAM is enabled, and never changes afterwards, so the read itself does not need the lock. It is already read without it, from the context switch path and /proc/PID/status. Add untagged_addr_remote_unlocked() for callers that have not taken the mmap lock, and annotate access to mm->context.untag_mask with READ_ONCE() and WRITE_ONCE() so the existing lockless reads are explicit and KCSAN-clean. untagged_addr_remote() keeps its assertion and shares the same code. Assisted-by: Claude:claude-opus-4.8 Acked-by: Usama Arif Signed-off-by: Rik van Riel --- arch/x86/include/asm/mmu_context.h | 6 +++--- arch/x86/include/asm/uaccess_64.h | 14 +++++++++++--- arch/x86/kernel/process_64.c | 4 ++-- include/linux/uaccess.h | 7 +++++++ 4 files changed, 23 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h index ef5b507de34e..cee710f64658 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -100,18 +100,18 @@ static inline unsigned long mm_lam_cr3_mask(struct mm_struct *mm) static inline void dup_lam(struct mm_struct *oldmm, struct mm_struct *mm) { mm->context.lam_cr3_mask = oldmm->context.lam_cr3_mask; - mm->context.untag_mask = oldmm->context.untag_mask; + WRITE_ONCE(mm->context.untag_mask, READ_ONCE(oldmm->context.untag_mask)); } #define mm_untag_mask mm_untag_mask static inline unsigned long mm_untag_mask(struct mm_struct *mm) { - return mm->context.untag_mask; + return READ_ONCE(mm->context.untag_mask); } static inline void mm_reset_untag_mask(struct mm_struct *mm) { - mm->context.untag_mask = -1UL; + WRITE_ONCE(mm->context.untag_mask, -1UL); } #define arch_pgtable_dma_compat arch_pgtable_dma_compat diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uaccess_64.h index 20de34cc9aa6..05377ad804d1 100644 --- a/arch/x86/include/asm/uaccess_64.h +++ b/arch/x86/include/asm/uaccess_64.h @@ -39,18 +39,26 @@ static inline unsigned long __untagged_addr(unsigned long addr) (__force __typeof__(addr))__untagged_addr(__addr); \ }) +/* + * mm->context.untag_mask is set once, when LAM is enabled, and never + * changes afterwards, so it can be read without holding the mmap lock. + */ static inline unsigned long __untagged_addr_remote(struct mm_struct *mm, unsigned long addr) { - mmap_assert_locked(mm); - return addr & (mm)->context.untag_mask; + return addr & READ_ONCE(mm->context.untag_mask); } -#define untagged_addr_remote(mm, addr) ({ \ +#define untagged_addr_remote_unlocked(mm, addr) ({ \ unsigned long __addr = (__force unsigned long)(addr); \ (__force __typeof__(addr))__untagged_addr_remote(mm, __addr); \ }) +#define untagged_addr_remote(mm, addr) ({ \ + mmap_assert_locked(mm); \ + untagged_addr_remote_unlocked(mm, addr); \ +}) + #endif #define valid_user_address(x) \ diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index d44afbe005bb..9fa659117f38 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -814,7 +814,7 @@ static void enable_lam_func(void *__mm) static void mm_enable_lam(struct mm_struct *mm) { mm->context.lam_cr3_mask = X86_CR3_LAM_U57; - mm->context.untag_mask = ~GENMASK(62, 57); + WRITE_ONCE(mm->context.untag_mask, ~GENMASK(62, 57)); /* * Even though the process must still be single-threaded at this @@ -952,7 +952,7 @@ long do_arch_prctl_64(struct task_struct *task, int option, unsigned long arg2) #endif #ifdef CONFIG_ADDRESS_MASKING case ARCH_GET_UNTAG_MASK: - return put_user(task->mm->context.untag_mask, + return put_user(mm_untag_mask(task->mm), (unsigned long __user *)arg2); case ARCH_ENABLE_TAGGED_ADDR: return prctl_enable_tagged_addr(task->mm, arg2); diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h index eddbbb65ccc4..7e6e4c89184c 100644 --- a/include/linux/uaccess.h +++ b/include/linux/uaccess.h @@ -34,6 +34,13 @@ }) #endif +#ifndef untagged_addr_remote_unlocked +#define untagged_addr_remote_unlocked(mm, addr) ({ \ + (void)(mm); \ + untagged_addr(addr); \ +}) +#endif + #ifdef masked_user_access_begin #define can_do_masked_user_access() 1 # ifndef masked_user_write_access_begin -- 2.53.0-Meta