From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DEE2C470120 for ; Fri, 24 Jul 2026 22:30:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=96.67.55.147 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784932218; cv=none; b=U+sEWVhCKwwgxH1oWlpVz05QrRThVjOb8Q99UEfepL2PNiNOi/YYFV7sQ5ggXXx4VAbR9KCMgciHEdBuCsC1CRsic1zRopT7wWj/kWl510k/FmxkN0/CPN0Gr1VO0KmtN7zFntEaG1KYKu0erTSMzc5P1BoryrBR13KOd03joXQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784932218; c=relaxed/simple; bh=VEwMMIEFAluWDJ0OS1P3q5nheOQxX3odzvtWygaRusE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DFQcz8unAji00e/rEnOF1j5WQ+5N6L4js++OCvD4oY9MUn7liPLyLYjia7NyMy/+6p2ZJSd17VlLdD7lousND5bgZKDxO8gB36POSuw2VqfL+AE4gNAq4A6pQYfoDje5T4SSbuKwZOihXTn/eRlzGv+/Q8cg2mdABOxeZNa3cD0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=surriel.com; spf=pass smtp.mailfrom=surriel.com; dkim=pass (2048-bit key) header.d=surriel.com header.i=@surriel.com header.b=DwRMlYPQ; arc=none smtp.client-ip=96.67.55.147 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=surriel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=surriel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=surriel.com header.i=@surriel.com header.b="DwRMlYPQ" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=surriel.com ; s=mail; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=IF4h0IOGesCPD6JnizKVFYQDqwHWpzKQAHqpitemykk=; b=DwRMlYPQpIsFsEsXZzPPL3sxm/ XwEli1sW2qLlJIwTseXX2o/q1glQCXOW/6qb3eWO+YB9NsZJ21DXr8JDOmcD4G8gCuqrbySUNPK0X AeN9ZFPsQawYFLRUwxaTF0qcs/+YwkcQm3g17KMh1EuuHv6EHqkdEfHW5IDK3KUSQ5zpTkQ75MAe7 xMin7ON9QwNgwRZWPvemLN7ReOBz/2+blnX+hqbzOcZ+CVBqoJItKziZOvlfBfywfefvI80kHSH7u 48Irw4hOCzBUYEdb5XsYKbjnzXSQK4vHCnAHc0nabbE43UPB8NzmeRB55+/i970oo+hEwo0d6bduS oRzqlBOA==; Received: from fangorn.home.surriel.com ([10.0.13.7]) by shelob.surriel.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.97.1) (envelope-from ) id 1wnOOs-0000000027h-14Iy; Fri, 24 Jul 2026 18:29:50 -0400 From: Rik van Riel To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel-team@meta.com, Dave Hansen , Peter Zijlstra , Suren Baghdasaryan , Lorenzo Stoakes , Vlastimil Babka , David Hildenbrand , "Liam R. Howlett" , Mike Rapoport , Michal Hocko , Jason Gunthorpe , John Hubbard , Peter Xu , Matthew Wilcox , Usama Arif , Rik van Riel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , linux-riscv@lists.infradead.org Subject: [PATCH RFC v4 02/12] riscv/mm: add untagged_addr_remote_unlocked() Date: Fri, 24 Jul 2026 18:29:24 -0400 Message-ID: <20260724222934.1463812-3-riel@surriel.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260724222934.1463812-1-riel@surriel.com> References: <20260724222934.1463812-1-riel@surriel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit __access_remote_vm() reads another task's memory under the mmap lock. That lock is contended on large multi-threaded workloads, where an mmap() or munmap() stalls readers like /proc/PID/cmdline or process_vm_readv() even though the target memory is resident and reachable under the per-VMA lock. Reaching it under the per-VMA lock means looking up the VMA first, which requires untagging the remote address without the mmap lock. riscv's untagged_addr_remote() asserts that lock and has no unlocked variant; the generic untagged_addr_remote_unlocked() falls back to untagged_addr(), which masks against current->mm, not the target mm. Add untagged_addr_remote_unlocked(), which masks @addr against the target mm without asserting the lock, and redefine untagged_addr_remote() on top of it so it keeps the assertion. The mask width is mm->context.pmlen, written while a program is single threaded, and frozen with MM_CONTEXT_LOCK_PMLEN once it spawns a second thread. That read need not be stable: pmlen is already read locklessly by untagged_addr() and mm_untag_mask(), and a remote untag is best-effort. Annotate the lockless reads with READ_ONCE() and the writes with WRITE_ONCE(), so a reader sees one whole value and the accesses are KCSAN-clean. Assisted-by: Claude:claude-opus-4.8 Signed-off-by: Rik van Riel --- arch/riscv/include/asm/mmu_context.h | 4 ++-- arch/riscv/include/asm/uaccess.h | 10 +++++++--- arch/riscv/kernel/process.c | 12 +++++++----- 3 files changed, 16 insertions(+), 10 deletions(-) diff --git a/arch/riscv/include/asm/mmu_context.h b/arch/riscv/include/asm/mmu_context.h index dbf27a78df6c..3ce16796e5a2 100644 --- a/arch/riscv/include/asm/mmu_context.h +++ b/arch/riscv/include/asm/mmu_context.h @@ -21,7 +21,7 @@ static inline void activate_mm(struct mm_struct *prev, struct mm_struct *next) { #ifdef CONFIG_RISCV_ISA_SUPM - next->context.pmlen = 0; + WRITE_ONCE(next->context.pmlen, 0); #endif switch_mm(prev, next, NULL); } @@ -44,7 +44,7 @@ DECLARE_STATIC_KEY_FALSE(use_asid_allocator); #define mm_untag_mask mm_untag_mask static inline unsigned long mm_untag_mask(struct mm_struct *mm) { - return -1UL >> mm->context.pmlen; + return -1UL >> READ_ONCE(mm->context.pmlen); } #endif diff --git a/arch/riscv/include/asm/uaccess.h b/arch/riscv/include/asm/uaccess.h index 5d4ec15584cf..53806e0f7dcf 100644 --- a/arch/riscv/include/asm/uaccess.h +++ b/arch/riscv/include/asm/uaccess.h @@ -16,7 +16,7 @@ static inline unsigned long __untagged_addr_remote(struct mm_struct *mm, unsigned long addr) { if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SUPM)) { - u8 pmlen = mm->context.pmlen; + u8 pmlen = READ_ONCE(mm->context.pmlen); /* Virtual addresses are sign-extended; physical addresses are zero-extended. */ if (IS_ENABLED(CONFIG_MMU)) @@ -33,12 +33,16 @@ static inline unsigned long __untagged_addr_remote(struct mm_struct *mm, unsigne (__force __typeof__(addr))__untagged_addr_remote(current->mm, __addr); \ }) -#define untagged_addr_remote(mm, addr) ({ \ +#define untagged_addr_remote_unlocked(mm, addr) ({ \ unsigned long __addr = (__force unsigned long)(addr); \ - mmap_assert_locked(mm); \ (__force __typeof__(addr))__untagged_addr_remote(mm, __addr); \ }) +#define untagged_addr_remote(mm, addr) ({ \ + mmap_assert_locked(mm); \ + untagged_addr_remote_unlocked(mm, addr); \ +}) + #define access_ok(addr, size) likely(__access_ok(untagged_addr(addr), size)) #else #define untagged_addr(addr) (addr) diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c index b2df7f72241a..6ae7552fed09 100644 --- a/arch/riscv/kernel/process.c +++ b/arch/riscv/kernel/process.c @@ -357,13 +357,15 @@ long set_tagged_addr_ctrl(struct task_struct *task, unsigned long arg) if (mmap_write_lock_killable(mm)) return -EINTR; - if (test_bit(MM_CONTEXT_LOCK_PMLEN, &mm->context.flags) && mm->context.pmlen != pmlen) { - mmap_write_unlock(mm); - return -EBUSY; + if (test_bit(MM_CONTEXT_LOCK_PMLEN, &mm->context.flags)) { + if (READ_ONCE(mm->context.pmlen) != pmlen) { + mmap_write_unlock(mm); + return -EBUSY; + } } envcfg_update_bits(task, ENVCFG_PMM, pmm); - mm->context.pmlen = pmlen; + WRITE_ONCE(mm->context.pmlen, pmlen); mmap_write_unlock(mm); @@ -394,7 +396,7 @@ long get_tagged_addr_ctrl(struct task_struct *task) break; } - if (task->mm->context.pmlen) + if (READ_ONCE(task->mm->context.pmlen)) ret |= PR_TAGGED_ADDR_ENABLE; return ret; -- 2.53.0-Meta