From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj2-f7.google.com (mail-pj2-f7.google.com [74.125.227.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A9B337DAA2 for ; Fri, 7 Aug 2026 07:50:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.135 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786089012; cv=none; b=DRxVf9KRJlecpyk0NML0vzDYqIo13fAoAPXl9rLzFB5f60CrjnLGVr/GfXN1FRXo4Lu5WOLyWuB5GuckC/R/5VAQMM8NcrBS4m8Tv5N9oyXuEYmfZnWI8/KpRILjZhgOr33aAH9Ce3/z/e7ESqdynoiBc7C7/mBsUjuse1kBhWE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786089012; c=relaxed/simple; bh=a9doi47rGq5GGgQE1nU+FIuUB9ds8tSrmQqJ45euQhU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=t0t4GWi1a3zsDXdtwYxyo2bBKsC3Bg7CPAKkiB9sOl7q5zMnyyG16sal7rx4LWLxqHgQs1EuFUIUMeCdMj5MkcV5U+IQpIxSAXjElhE1lxfepjX/UPhzgRJl0nf4v2RWAKyRc11XU4/ZawNe44ZYfp8by1XHEhKZ4SA7TmvjPoA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ldA2rUUZ; arc=none smtp.client-ip=74.125.227.135 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ldA2rUUZ" Received: by mail-pj2-f7.google.com with SMTP id 98e67ed59e1d1-38dcd98e5easo1823223a91.0 for ; Fri, 07 Aug 2026 00:50:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1786089008; x=1786693808; darn=vger.kernel.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=gy9L9/LoV9ZFrTZKgyHXZWp3XQGkD3Y92WNpBSmA0ec=; b=ldA2rUUZdnZ9VY9HTE5Bym+MnNpdBtgwjK5yGxp/8L3RivqbOW3uqULOSCbaLJCaqW BMZGaGFeQz419AG+e52DYimqINq6PQGH8imUOZC5j0bvcJgSUO/hwY4UsNsiuD4sJzOQ X0zxm1ZaHSlM0djKxi+v3wQY74TrAAOiDsceRWWkWeE644MXJuSIEhHJZZQaBpKEq1QL VC/xgHnDxmqDEUjyzJLq3IgIjgq8EyScs0ckvYfW/0TNCnSdXL5ndPmQVFNfE3IKO/7q R69z2ZWMYfkcV1EEY12xpQ1M3/wl5IMJl69FdMyDFjrw0ePkKT8/rTM8fyaWbGZ2k3Mw pspQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786089008; x=1786693808; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=gy9L9/LoV9ZFrTZKgyHXZWp3XQGkD3Y92WNpBSmA0ec=; b=krQTKcLh7weEmlSnoDsE2eH36I+SUWxt25zLHfmmAbTwBEssOuRotusGTaRlKq0kQ5 DvHwKk06RGERG2cSGxe9HqEje++yaNNzKEqhYP52OOoYvtvDxcvdFqEFIlLw+aQDurvK vuVpZj5m0zS8n2JfnMIGr4AglweLhyikx7MmrNj39JtKKT2gWSN0kSL4KLbSjQJzDzjK tG3fZ5dDJ6JLGeAjF3ZZFa/zn9WBS0LElDYUJIez0U5EQoWFa2rWI0KujBbQ+8YyuCdB PR3HbdExjXSss4PiBsva3Wxj5bMJRnd37EMZF659rfybmuZvgr9SydfQX45HkvWJuCCC iBjw== X-Forwarded-Encrypted: i=1; AHgh+Rq0RkK2d6h16LlDpks2aaPFH6DpFvf6cvQJQ9W8me+LDYuZ5r7oaJtjCcc1mHAo8ElSxNy29OKFmwLy0fM=@vger.kernel.org X-Gm-Message-State: AOJu0YyEFuWl6PjCVqq966Z9/UN7420so5c8YCBvkzyS6aANXIzpdDM8 VrMq0/M+4jd2kKaxhJvRzjNlNi0OtXKBkZqwEQQQOFtTvPNCHZi5172x X-Gm-Gg: AR+sD13GUAgK95LXJ7IUCCpSIPiifbbYqOn2dpuLGyChjsS+1cwmcl2FPss69/ESGx/ 4vR8W5Ei5n4dod4Fo0sHUxT6FtGVTYES8UdD+OtWtFr48Cnn5AAJyhqW1VRbmKdDKjgLCVGKBJe dN4VLV9a+8evPpfpp+NLPj14ZQmNxoDz7VWYDrSxIkcWDsT9VdjWBgSWc6pihGUn3Ygr5hoCVga XwteI5nt5up0w6mRnkW8Mts+X/ol0UN6RmsQCt4eO5Ayhbhokpd7bxG6itTIncDuk5H+mFIKt3L QP39BcOR4CLYjr/AxEWq0AKe1+z/FEB4esA2lglV9CYDGGe+t8PA/ltFLjcU3Vg+aKHG3xRI+5X Bp6Yk1fH6RObq6z+pWR4jPlHS7+yeyrffFRJasxuL34moo7KhrT28PlpRljmYjFeKZZeRIErok6 bYtPh+Gdgfj5yNQ/rLocu9njztSoKBWm6cKH3SLEw+d4KGiRHpyL3cfP3ikaodJhYUkbYOYHQ/M xR6SCD98fkiuZmgOyUG2XN2Ng== X-Received: by 2002:a17:90b:1843:b0:37f:bfa2:1887 with SMTP id 98e67ed59e1d1-3903c53639amr22774995a91.8.1786089008044; Fri, 07 Aug 2026 00:50:08 -0700 (PDT) Received: from zhr-ThinkStation-K.mioffice.cn ([2408:8607:1b00:8:3629:bdc:8549:6d81]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3925fa4fc7fsm1453369a91.0.2026.08.07.00.49.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Aug 2026 00:50:07 -0700 (PDT) From: Hongru Zhang X-Google-Original-From: Hongru Zhang To: baohua@kernel.org Cc: akpm@linux-foundation.org, david@kernel.org, liam@infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, ljs@kernel.org, mhocko@suse.com, rppt@kernel.org, shakeel.butt@linux.dev, surenb@google.com, vbabka@kernel.org, willy@infradead.org, zhanghongru06@gmail.com, zhanghongru@xiaomi.com Subject: Re: [RFC PATCH v4 1/3] mm: allow page faults to request VMA-lock retry Date: Fri, 7 Aug 2026 15:49:47 +0800 Message-ID: <20260807074948.28953-1-zhanghongru@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit > On Thu, Aug 6, 2026 at 3:30 PM Hongru Zhang wrote: > > > > > On Tue, Aug 4, 2026 at 8:32 PM Lorenzo Stoakes (ARM) wrote: > > > > > > > > On Tue, Aug 04, 2026 at 05:52:19PM +0800, Hongru Zhang wrote: > > > > > From: Hongru Zhang > > > > > > > > > > Page faults handled under the per-VMA lock currently fall back to the > > > > > mmap_lock path whenever handle_mm_fault() returns VM_FAULT_RETRY. This > > > > > means that lower-level fault handlers have no way to tell the > > > > > architecture fault handler that the retry can safely continue under the > > > > > per-VMA lock. > > > > > > > > > > Add VM_FAULT_MAY_USE_VMA_LOCK as an advisory bit that can be returned > > > > > > > > I don't love that name or that faulting retry behaviour is _modified_ by a > > > > value that indicates fault resolution state... ugh. > > > > > > > > It's kinda confusing things 'VM_FAULT_RETRY' is 'you have to retry this > > > > fault'. > > > > > > > > 'VM_FAULT_MAY_...' is starting to bring in effectively configuration > > > > options into it and that's kinda horrible. > > > > > > > > I mean is there any reason we shouldn't ALWAYS do this if a VMA lock was > > > > used? > > > > > > > > It's not too expensive to do a single retry with the VMA lock before > > > > falling back to the mmap lock. > > > > > > > > So maybe simplify like that? > > > > > > > > And like that this series becomes a single patch right? > > > > > > This is a brilliant idea. That's a genius insight, Lorenzo. > > > > > > I guess the conceptual model could simply be: > > > > > > diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c > > > index 45b99c3b1442..3592bcc9bbd7 100644 > > > --- a/arch/x86/mm/fault.c > > > +++ b/arch/x86/mm/fault.c > > > @@ -1222,6 +1222,7 @@ void do_user_addr_fault(struct pt_regs *regs, > > > struct mm_struct *mm; > > > vm_fault_t fault; > > > unsigned int flags = FAULT_FLAG_DEFAULT; > > > + bool vma_lock_retried = false; > > > > > > tsk = current; > > > mm = tsk->mm; > > > @@ -1331,6 +1332,7 @@ void do_user_addr_fault(struct pt_regs *regs, > > > if (!(flags & FAULT_FLAG_USER)) > > > goto lock_mmap; > > > > > > +vma_lock: > > > vma = lock_vma_under_rcu(mm, address); > > > if (!vma) > > > goto lock_mmap; > > > @@ -1352,6 +1354,11 @@ void do_user_addr_fault(struct pt_regs *regs, > > > if (fault & VM_FAULT_MAJOR) > > > flags |= FAULT_FLAG_TRIED; > > > > > > + if (!vma_lock_retried) { > > > + vma_lock_retried = true; > > > + goto vma_lock; > > > + } > > > + > > > /* Quick path to respond to signals */ > > > if (fault_signal_pending(fault, regs)) { > > > if (!user_mode(regs)) > > > > > > Nothing else needs to change then. I wonder if there is a cleaner > > > way to implement the idea, but it is really stunning. > > > > > > Best Regards > > > Barry > > > > Filemap Throughput (higher is better): > > +---------+------------+---------------------+---------------------+---------------------+ > > | Threads | Vanilla | RFC v4 | P1 | P2 | > > +---------+------------+---------------------+---------------------+---------------------+ > > | 40 | 1069.34 /s | 1404.47 /s (+31.3%) | 1400.13 /s (+30.9%) | 1412.38 /s (+32.1%) | > > +---------+------------+---------------------+---------------------+---------------------+ > > | 60 | 1038.12 /s | 1682.88 /s (+62.1%) | 1683.37 /s (+62.2%) | 1685.23 /s (+62.3%) | > > +---------+------------+---------------------+---------------------+---------------------+ > > | 80 | 1042.62 /s | 1766.72 /s (+69.5%) | 1767.83 /s (+69.6%) | 1771.73 /s (+69.9%) | > > +---------+------------+---------------------+---------------------+---------------------+ > > > > Swap Throughput (higher is better): > > +--------------+-------------+----------------------+----------------------+----------------------+ > > | mmap writers | Vanilla | RFC v4 | P1 | P2 | > > +--------------+-------------+----------------------+----------------------+----------------------+ > > | 0 | 17303.09 /s | 18394.51 /s (+6.3%) | 17899.48 /s (+3.4%) | 18337.30 /s (+6.0%) | > > +--------------+-------------+----------------------+----------------------+----------------------+ > > | 2 | 16728.04 /s | 18591.68 /s (+11.1%) | 18346.72 /s (+9.7%) | 18848.17 /s (+12.7%) | > > +--------------+-------------+----------------------+----------------------+----------------------+ > > | 4 | 12596.23 /s | 18534.00 /s (+47.1%) | 16095.20 /s (+27.8%) | 18507.62 /s (+46.9%) | > > +--------------+-------------+----------------------+----------------------+----------------------+ > > > > The key difference between P1 and P2 is how FAULT_FLAG_TRIED is handled: P1 > > keeps the existing major-fault-only setting before the retry, while P2 sets it > > before the VMA-lock retry for all retrying faults. > > > > In filemap throughput test, each reader thread operates on its own file. > > In swap throughput test, all reader threads fault the same memory area. > > > > > > P1: > > > > diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c > > index 45b99c3b1442..c3ab30d32a15 100644 > > --- a/arch/x86/mm/fault.c > > +++ b/arch/x86/mm/fault.c > > @@ -1222,6 +1222,7 @@ void do_user_addr_fault(struct pt_regs *regs, > > struct mm_struct *mm; > > vm_fault_t fault; > > unsigned int flags = FAULT_FLAG_DEFAULT; > > + bool vma_lock_retried = false; > > > > tsk = current; > > mm = tsk->mm; > > @@ -1331,6 +1332,7 @@ void do_user_addr_fault(struct pt_regs *regs, > > if (!(flags & FAULT_FLAG_USER)) > > goto lock_mmap; > > > > +lock_vma: > > vma = lock_vma_under_rcu(mm, address); > > if (!vma) > > goto lock_mmap; > > @@ -1360,6 +1362,12 @@ void do_user_addr_fault(struct pt_regs *regs, > > ARCH_DEFAULT_PKEY); > > return; > > } > > + > > + if (!vma_lock_retried) { > > + vma_lock_retried = true; > > + goto lock_vma; > > + } > > + > > lock_mmap: > > > > > > P2: > > > > diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c > > index 45b99c3b1442..9507b8a0fe18 100644 > > --- a/arch/x86/mm/fault.c > > +++ b/arch/x86/mm/fault.c > > @@ -1222,6 +1222,7 @@ void do_user_addr_fault(struct pt_regs *regs, > > struct mm_struct *mm; > > vm_fault_t fault; > > unsigned int flags = FAULT_FLAG_DEFAULT; > > + bool vma_lock_retried = false; > > > > tsk = current; > > mm = tsk->mm; > > @@ -1331,6 +1332,7 @@ void do_user_addr_fault(struct pt_regs *regs, > > if (!(flags & FAULT_FLAG_USER)) > > goto lock_mmap; > > > > +lock_vma: > > vma = lock_vma_under_rcu(mm, address); > > if (!vma) > > goto lock_mmap; > > @@ -1349,8 +1351,6 @@ void do_user_addr_fault(struct pt_regs *regs, > > goto done; > > } > > count_vm_vma_lock_event(VMA_LOCK_RETRY); > > - if (fault & VM_FAULT_MAJOR) > > - flags |= FAULT_FLAG_TRIED; > > > > /* Quick path to respond to signals */ > > if (fault_signal_pending(fault, regs)) { > > @@ -1360,6 +1360,13 @@ void do_user_addr_fault(struct pt_regs *regs, > > ARCH_DEFAULT_PKEY); > > return; > > } > > + > > + if (!vma_lock_retried) { > > + flags |= FAULT_FLAG_TRIED; > > + vma_lock_retried = true; > > + goto lock_vma; > > + } > > + > > lock_mmap: > > Thanks! > > As Lorenzo pointed out, this would break major fault accounting, so I > think it is better suited as P1. > > The performance difference you are seeing is probably because another > thread is concurrently swapping in the same address, taking the > folio_lock and installing the PTE. That is a separate issue and could > be addressed by a separate patch, likely an updated version of this: > > mm: Don't retry page fault if folio is uptodate during swap-in > > https://lore.kernel.org/all/20260430040427.4672-5-baohua@kernel.org/ Swap Throughput (higher is better): +--------------+-------------+----------------------+----------------------+ | mmap writers | Vanilla | P1 | P3 | +--------------+-------------+----------------------+----------------------+ | 0 | 17303.09 /s | 17899.48 /s (+3.4%) | 18190.62 /s (+5.1%) | +--------------+-------------+----------------------+----------------------+ | 2 | 16728.04 /s | 18346.72 /s (+9.7%) | 18162.03 /s (+8.6%) | +--------------+-------------+----------------------+----------------------+ | 4 | 12596.23 /s | 16095.20 /s (+27.8%) | 17991.45 /s (+42.8%) | +--------------+-------------+----------------------+----------------------+ P3: diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 45b99c3b1442..c3ab30d32a15 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -1222,6 +1222,7 @@ void do_user_addr_fault(struct pt_regs *regs, struct mm_struct *mm; vm_fault_t fault; unsigned int flags = FAULT_FLAG_DEFAULT; + bool vma_lock_retried = false; tsk = current; mm = tsk->mm; @@ -1331,6 +1332,7 @@ void do_user_addr_fault(struct pt_regs *regs, if (!(flags & FAULT_FLAG_USER)) goto lock_mmap; +lock_vma: vma = lock_vma_under_rcu(mm, address); if (!vma) goto lock_mmap; @@ -1360,6 +1362,12 @@ void do_user_addr_fault(struct pt_regs *regs, ARCH_DEFAULT_PKEY); return; } + + if (!vma_lock_retried) { + vma_lock_retried = true; + goto lock_vma; + } + lock_mmap: retry: diff --git a/mm/memory.c b/mm/memory.c index 428eb555ecb7..8c34a857548b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4987,6 +4987,13 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) } swapcache = folio; + /* + * If the folio is uptodate, we are likely only waiting for + * another concurrent PTE mapping to complete, which should + * be brief. No need to drop the lock and retry the fault. + */ + if (folio_test_uptodate(folio)) + vmf->flags &= ~FAULT_FLAG_ALLOW_RETRY; ret |= folio_lock_or_retry(folio, vmf); if (ret & VM_FAULT_RETRY) goto out_release; Thanks, Hongru