From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCAE3451068 for ; Fri, 22 May 2026 14:25:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459911; cv=none; b=jeP4A9+2tNSuIVbgIFc2ep6/aFVAWQbeMcWAoDwXr8y2li2jNUeao880LAUdshf88dKWP6vpkdY7NKT+xPaFKGtnsMt/tc6LRn7T+ccZS7W5iCZCawC/pZPNBT5UaZ8ryoJzwacFfIny3SMEvdFXxGzc6uOsMZy0kX3IHh/IpA0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459911; c=relaxed/simple; bh=qcH8PMPAyJiTkepaghd+65iBhrd37pavq9xijlY+swk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=i7cMrGFTGX5r8xm4GxzGmGzxdLTwBHwnKbgk/bMrYuamHnvb+fzZkPss+FzXLhClA3xOCyeIx7YgNiGqJ+t4j6gPwawYE7f94+Ts6RBkswUG600sN55ILGb8lbNtnZSZ5jLbR3f1CkzmdfzpRo1Ni8CNEN9z35sN9ybGsp1aU/0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=ehPhuicX; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="ehPhuicX" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-82f9fdfc965so3461363b3a.1 for ; Fri, 22 May 2026 07:25:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1779459909; x=1780064709; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=h4fHtLWbjgxAw/RuryI1Zn991wS9DJVk1T6sfJuVej8=; b=ehPhuicXKAIR9B7gwXfffscORkPFNTsNMUCooxYJwqYWVwoJfo0wqo2BDRlmJkq48/ n3zs0564k65PnlB7+Zauptzaz4clljngdY+fX1ook/uM6OFKwt1McaZJQmP7JPapdtNS nT367SAr+n6gT0XpM73iv170XKtrZUCd5M5rs/f1wpuIA90CCGNqEj+r9RHk0Pvw0w2r Oiu4ZucMxSJo8Wl4bZ61WoGC8k9SeCX+P6hHgcgL1fIYD++S2aBwvSkYgPuHN7jzWLoR jMOOP3UQSYmqAeD/qkhrvoCRlbnhYIrQ7ithpX8wxWgfW8UOXXEZWJr0x3e7i+kKb7SH LOrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779459909; x=1780064709; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=h4fHtLWbjgxAw/RuryI1Zn991wS9DJVk1T6sfJuVej8=; b=rE2n2i9C9IuZgRgibzqwU4r75DOeUYCluIrBFETjffqVPI9VlbhO7CoaNiP8Ro+7Xe wHLW61EMUbHyH8AYQfOujob992RzzlKW2RRVP36sXaMiwb4TR19yYQBqPy4YjKilwBPe Y0Tx9MPtJ9NtpxEMLhikJkWQoBmjzfG65rLuvTrKaWqLSBB/P3Re3/kYRRFlnS4tfHQM 3XslbZhE7kiqlO2RwnrzeC0Ci2DI3mYucV6TPiQACtq3wE1UN7eaLSQ/9x9a0tmXidsC C3kBXOusU0EHjfdjEIDzl8s1Pybh9HMEBFA7Q2Hw/vdwl0FCq7UCcGm7rXAaimhAPLeV cDWQ== X-Forwarded-Encrypted: i=1; AFNElJ9FvMuKn+I/qQrprzW1k6D5UCLGG9W8IKViF76haPyeC/gClS5RflLyea1rii1DmA3Q1b6MoZPbuFsiep8=@vger.kernel.org X-Gm-Message-State: AOJu0YwUSdP0BsldWja9eHrVGSZnoefQqI6VbKSdZOnBpcJpAa2NjPTK XjaeAs0XexLndcwivzVeJVwg078NmaMBsBm9V6ZQQ4GrsTWlNds+31YV2hmDLZpkU04= X-Gm-Gg: Acq92OEe7j4JRKW6/VaGTs26+40rXngqjECDzbqKkMWAuS0OoRahPCDboBXtfp4ufJX rlqVFm7jgrWpwNQThq/hbxNVueVCZTLtEsB1AdD7HuVk8Kla9S90Lh2Up2LL3j71fNL6BJxo8J8 Xl5XiTvzYeoG9v013rBU4UL0dUwegCMa7NpTjiG8nShfEbMnLAFdmJx5CKxRSW68/z3JttXjORr kraQ3i8WZwwqltimM9Vuh1+Nqtrabhy7/hy8+8BfPna1anhiNIv+iN0Y51G5GMB/f9+2+7uPAFw MnO/uXU66xUryIVSv5FfOQMWtXjhhJXiQXpw81Lre9eJKI2z6Rq/7ROUYx6u07aPGa3Uoe5j7ME 8IbilSWCwuaDjluN3U85r1kM6UrUpCJYVdabp1SuIp8c6G0RFRhEsKJoqby8uGZ61d8ceocLbFZ 7A8NqjCd6zPX1Ns4Nv0HxsOvi0GUKN5yi95LBdfSHpGyN7eEaLqUWkfLAur1YEuu5lClvcMS8R X-Received: by 2002:a05:6a00:3498:b0:837:fe75:104e with SMTP id d2e1a72fcca58-8415f0f0279mr4202939b3a.5.1779459908747; Fri, 22 May 2026 07:25:08 -0700 (PDT) Received: from L6YN4KR4K9.bytedance.net ([139.177.225.253]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84164ea14d0sm1963997b3a.37.2026.05.22.07.25.00 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 22 May 2026 07:25:08 -0700 (PDT) From: Yunhui Cui To: pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, alex@ghiti.fr, akpm@linux-foundation.org, pasha.tatashin@soleen.com, andrew+kernel@donnellan.id.au, rmclure@linux.ibm.com, debug@rivosinc.com, baolin.wang@linux.alibaba.com, zhangchunyan@iscas.ac.cn, cuiyunhui@bytedance.com, apopple@nvidia.com, namcao@linutronix.de, wangruikang@iscas.ac.cn, apatel@ventanamicro.com, liu.xuemei1@zte.com.cn, ajones@ventanamicro.com, cleger@rivosinc.com, charlie@rivosinc.com, hui.wang@canonical.com, guodong@riscstar.com, pincheng.plct@isrc.iscas.ac.cn, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Qingwei Hu Subject: [PATCH v2 2/2] riscv: preserve hardware-updated A/D bits in PTE accessors Date: Fri, 22 May 2026 22:23:58 +0800 Message-Id: <20260522142358.87589-2-cuiyunhui@bytedance.com> X-Mailer: git-send-email 2.39.2 (Apple Git-143) In-Reply-To: <20260522142358.87589-1-cuiyunhui@bytedance.com> References: <20260522142358.87589-1-cuiyunhui@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Use cmpxchg-based merges for live RISC-V PTE permission updates so software changes do not lose concurrently hardware-updated accessed and dirty state. Cover ptep_set_access_flags(), ptep_test_and_clear_young(), and ptep_set_wrprotect(), and extend the same wrprotect handling to the PUD leaf helper used by huge mappings. Keep the existing Svvptc flush behaviour, but only flush when the merged PTE value actually changed. Signed-off-by: Yunhui Cui Reviewed-by: Qingwei Hu --- arch/riscv/include/asm/pgtable.h | 19 +++++++-- arch/riscv/mm/pgtable.c | 68 ++++++++++++++++++++++++++------ 2 files changed, 73 insertions(+), 14 deletions(-) diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 20663a466cf6c..984c37ca8aef7 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -668,15 +668,21 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm, static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long address, pte_t *ptep) { - pte_t read_pte = READ_ONCE(*ptep); + pte_t old_pte; + pte_t pte; /* * ptep_set_wrprotect can be called for shadow stack ranges too. * shadow stack memory is XWR = 010 and thus clearing _PAGE_WRITE will lead to * encoding 000b which is wrong encoding with V = 1. This should lead to page fault * but we dont want this wrong configuration to be set in page tables. */ - atomic_long_set((atomic_long_t *)ptep, - ((pte_val(read_pte) & ~(unsigned long)_PAGE_WRITE) | _PAGE_READ)); + pte = READ_ONCE(*ptep); + do { + old_pte = pte; + pte = pte_wrprotect(pte); + pte_val(pte) = cmpxchg_relaxed(&pte_val(*ptep), pte_val(old_pte), + pte_val(pte)); + } while (pte_val(pte) != pte_val(old_pte)); } #define __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH @@ -1030,6 +1036,13 @@ static inline void pmdp_set_wrprotect(struct mm_struct *mm, ptep_set_wrprotect(mm, address, (pte_t *)pmdp); } +#define __HAVE_ARCH_PUDP_SET_WRPROTECT +static inline void pudp_set_wrprotect(struct mm_struct *mm, + unsigned long address, pud_t *pudp) +{ + ptep_set_wrprotect(mm, address, (pte_t *)pudp); +} + #define pmdp_establish pmdp_establish static inline pmd_t pmdp_establish(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp, pmd_t pmd) diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c index 9c4427d0b1874..b77e82362442e 100644 --- a/arch/riscv/mm/pgtable.c +++ b/arch/riscv/mm/pgtable.c @@ -5,23 +5,55 @@ #include #include +#define RISCV_PTE_ACCESS_FLAG_MASK (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC | \ + _PAGE_ACCESSED | _PAGE_DIRTY | \ + _PAGE_SOFT_DIRTY) + +static inline unsigned long riscv_pte_access_flags(unsigned long cur, + unsigned long entry) +{ + unsigned long pteval; + unsigned long hw_flags; + + hw_flags = _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_SOFT_DIRTY; + pteval = cur & ~RISCV_PTE_ACCESS_FLAG_MASK; + pteval |= entry & (RISCV_PTE_ACCESS_FLAG_MASK & ~hw_flags); + pteval |= (cur | entry) & hw_flags; + + return pteval; +} + int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address, pte_t *ptep, pte_t entry, int dirty) { + unsigned long old_pteval; + unsigned long new_pteval; + unsigned long prev_pteval; + bool changed; + + old_pteval = pte_val(ptep_get(ptep)); + do { + new_pteval = riscv_pte_access_flags(old_pteval, pte_val(entry)); + if (new_pteval == old_pteval) + break; + + prev_pteval = cmpxchg_relaxed(&pte_val(*ptep), old_pteval, + new_pteval); + if (prev_pteval == old_pteval) + break; + + old_pteval = prev_pteval; + } while (1); + + changed = old_pteval != new_pteval; if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SVVPTC)) { - if (!pte_same(ptep_get(ptep), entry)) { - __set_pte_at(vma->vm_mm, ptep, entry); - /* Here only not svadu is impacted */ + if (changed) flush_tlb_page(vma, address); - return true; - } - return false; + return changed; } - if (!pte_same(ptep_get(ptep), entry)) - __set_pte_at(vma->vm_mm, ptep, entry); /* * update_mmu_cache will unconditionally execute, handling both * the case that the PTE changed and the spurious fault case. @@ -32,9 +64,23 @@ int ptep_set_access_flags(struct vm_area_struct *vma, bool ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep) { - if (!pte_young(ptep_get(ptep))) - return false; - return test_and_clear_bit(_PAGE_ACCESSED_OFFSET, &pte_val(*ptep)); + unsigned long old_pteval; + unsigned long new_pteval; + unsigned long prev_pteval; + + old_pteval = pte_val(ptep_get(ptep)); + do { + if (!(old_pteval & _PAGE_ACCESSED)) + return false; + + new_pteval = pte_val(pte_mkold(__pte(old_pteval))); + prev_pteval = cmpxchg_relaxed(&pte_val(*ptep), old_pteval, + new_pteval); + if (prev_pteval == old_pteval) + return true; + + old_pteval = prev_pteval; + } while (1); } EXPORT_SYMBOL_GPL(ptep_test_and_clear_young); -- 2.39.5