From: "Barry Song (Xiaomi)" <baohua@kernel.org>
To: zippermonkey@icloud.com
Cc: akpm@linux-foundation.org, axelrasmussen@google.com,
baohua@kernel.org, baolin.wang@linux.alibaba.com,
baoquan.he@linux.dev, bruzzhang@tencent.com, david@kernel.org,
hannes@cmpxchg.org, kasong@tencent.com,
linux-kernel@vger.kernel.org, linux-mm@kvack.org, ljs@kernel.org,
mhocko@kernel.org, qi.zheng@linux.dev, shakeel.butt@linux.dev,
weixugc@google.com, yuanchu@google.com
Subject: [PATCH v6 0/5] mm: batch TLB flushing for dirty folios in vmscan
Date: Mon, 16 Nov 2026 07:02:31 +0800 [thread overview]
Message-ID: <20261115230231.21194-1-baohua@kernel.org> (raw)
In-Reply-To: <20261031-batch-tlb-flush-v6-0-190539839eec@tencent.com>
On Sat, Oct 31, 2026 at 6:35 PM Zhang Peng <zippermonkey@icloud.com> wrote:
>
> This series introduces batch TLB flushing optimization for dirty folios
> during memory reclaim, aiming to reduce IPI overhead on multi-core systems.
>
> Background
> ----------
> Reclaim calls try_to_unmap_flush_dirty() before writing out each dirty
> folio. When writable TLB entries are pending, this triggers a flush.
> On systems that use IPIs for TLB shootdowns, frequent flushes can add
> significant overhead.
>
> Approach
> --------
> Collect dirty folios into a folio_batch, releasing their locks while
> collecting the batch so swap faults can progress. Before writeout, try
> locking each folio again and recheck writeback, mappings and DMA pins.
> Keep eligible folios locked, flush pending writable TLB entries once,
> then page them out. Process a final partial batch at the end of the scan.
>
> Changes
> -------
> Patch 1: Extract the folio activation block at activate_locked into
> folio_activate_locked().
> Patch 2: Extract the folio-freeing path (buffer release, lazyfree,
> __remove_mapping, folio_batch drain) into folio_try_reclaim_free().
> Group retained folios, deferred frees and the reclaimed count
> in reclaim_folio_state, local to each shrink_folio_list() call.
> Patch 3: Extract the pageout() dispatch state machine into folio_try_pageout().
> Patch 4: Extract the TTU setup and try_to_unmap() block into folio_try_unmap().
> Patch 5: Implement batch TLB flushing logic. Dirty folios are accumulated in
> batches and a single TLB flush is performed for each batch before
> calling pageout.
>
> Testing
> -------
> The benchmark script uses stress-ng to compare TLB shootdown behavior before and
> after this patch. It constrains a stress-ng workload via memcg to force reclaim
> through shrink_folio_list(), reporting TLB shootdowns and IPIs.
>
> Core benchmark command: stress-ng --vm 16 --vm-bytes 2G --vm-keep --timeout 60
>
> ==========================================================================
> batch_dirty_tlb_flush Benchmark Results
> ==========================================================================
> Kernel: 7.0.0-rc1+ CPUs: 16
> MemTotal: 31834M SwapTotal: 8191M
> memcg limit: 512M alloc: 2G workers: 16 duration: 60s
> --------------------------------------------------------------------------
> Metric Before After Delta (abs / %)
> --------------------------------------------------------------------------
> bogo ops/s 28238.63 35833.97 +7595.34 (+26.9%)
> TLB shootdowns 55428953 17621697 -37807256 (-68.2%)
> Function call IPIs 34073695 14498768 -19574927 (-57.4%)
> pgscan_anon (pages) 52856224 60252894 7396670 (+14.0%)
> pgsteal_anon (pages) 29004962 34054753 5049791 (+17.4%)
> --------------------------------------------------------------------------
>
> Suggested-by: Kairui Song <kasong@tencent.com>
> Signed-off-by: Zhang Peng <bruzzhang@tencent.com>
> ---
Hi Peng,
Just a quick update: this patch is still on my radar. I ran a
quick perf test while building the kernel. Without your patch,
try_to_unmap_flush_dirty() accounted for 1.75% of CPU cycles.
I guess there could be workloads where try_to_unmap_flush_dirty()
accounts for an even larger share of CPU cycles.
# Children Self Command Shared Object Symbol
# ........ ........ ....... ................. ............................
#
1.75% 0.01% cc1 [kernel.kallsyms] [.] try_to_unmap_flush_dirty
|
--1.74%--try_to_unmap_flush_dirty
try_to_unmap_flush (inlined)
|
--1.73%--arch_tlbbatch_flush
|
--1.70%--flush_tlb_multi (inlined)
__flush_tlb_multi (inlined)
|
--1.69%--native_flush_tlb_multi
|
--1.68%--on_each_cpu_cond_mask
|
--1.62%--smp_call_function_many_cond
|
--1.23%--csd_lock_wait (inlined)
With your patch, the CPU overhead drops by roughly 7x:
# Children Self Command Shared Object Symbol
# ........ ........ ....... ................. ............................
#
0.25% 0.00% cc1 [kernel.kallsyms] [.] try_to_unmap_flush_dirty
|
--0.24%--try_to_unmap_flush_dirty
try_to_unmap_flush (inlined)
|
--0.24%--arch_tlbbatch_flush
|
--0.24%--flush_tlb_multi (inlined)
__flush_tlb_multi (inlined)
|
--0.24%--native_flush_tlb_multi
|
--0.23%--on_each_cpu_cond_mask
|
--0.21%--smp_call_function_many_cond
|
--0.14%--csd_lock_wait (inlined)
0.00% 0.00% fixdep [kernel.kallsyms] [.] try_to_unmap_flush_dirty
Overall, I think this is a good approach. I'll be traveling soon
and will return to this patchset for more testing and a thorough
review in about two and a half weeks.
Thanks
Barry
next prev parent reply other threads:[~2026-11-15 23:02 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-31 10:35 Zhang Peng
2026-10-31 10:35 ` [PATCH v6 1/5] mm/vmscan: introduce folio_activate_locked() helper Zhang Peng
2026-10-31 10:35 ` [PATCH v6 2/5] mm/vmscan: extract folio reclaim freeing from shrink_folio_list() Zhang Peng
2026-10-31 10:35 ` [PATCH v6 3/5] mm/vmscan: extract folio pageout " Zhang Peng
2026-10-31 10:35 ` [PATCH v6 4/5] mm/vmscan: extract folio unmap logic into folio_try_unmap() Zhang Peng
2026-10-31 10:35 ` [PATCH v6 5/5] mm/vmscan: batch dirty TLB flushes before pageout Zhang Peng
2026-11-15 23:02 ` Barry Song (Xiaomi) [this message]
2026-11-15 23:27 ` [PATCH v6 0/5] mm: batch TLB flushing for dirty folios in vmscan Barry Song
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261115230231.21194-1-baohua@kernel.org \
--to=baohua@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=axelrasmussen@google.com \
--cc=baolin.wang@linux.alibaba.com \
--cc=baoquan.he@linux.dev \
--cc=bruzzhang@tencent.com \
--cc=david@kernel.org \
--cc=hannes@cmpxchg.org \
--cc=kasong@tencent.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=ljs@kernel.org \
--cc=mhocko@kernel.org \
--cc=qi.zheng@linux.dev \
--cc=shakeel.butt@linux.dev \
--cc=weixugc@google.com \
--cc=yuanchu@google.com \
--cc=zippermonkey@icloud.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®