From: Andrew Morton <akpm@linux-foundation.org>
To: Jiebin Sun <jiebin.sun@intel.com>
Cc: dennis@kernel.org, tj@kernel.org, cl@gentwo.org,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
bigeasy@linutronix.de, clrkwllms@kernel.org, rostedt@goodmis.org,
linux-rt-devel@lists.linux.dev, tim.c.chen@linux.intel.com,
tianyou.li@intel.com, wangyang.guo@intel.com
Subject: Re: [PATCH] lib/percpu_counter: allow concurrent sums with an rwlock
Date: Sun, 15 Nov 2026 19:16:36 -0800 [thread overview]
Message-ID: <20261115191636.f18859e4dd5f58b9a33e7814@linux-foundation.org> (raw)
In-Reply-To: <20261116015655.2744577-1-jiebin.sun@intel.com>
On Mon, 16 Nov 2026 09:56:55 +0800 Jiebin Sun <jiebin.sun@intel.com> wrote:
> __percpu_counter_sum() holds a raw spinlock while reading the global
> count and per-CPU deltas, serializing concurrent sums of the same
> counter. Since a sum changes neither the global count nor the per-CPU
> deltas, use rwlock_t on !PREEMPT_RT to allow concurrent sums with a
> read lock. Global updates take the write lock. The per-CPU fast path
> remains lockless, so sums keep their existing snapshot semantics.
>
> stress-ng dentry reaches this path through ext4_statfs():
>
> statfs -> ext4_statfs -> __percpu_counter_sum
> -> native_queued_spin_lock_slowpath
>
> Keep the raw spinlock on PREEMPT_RT, where rwlock_t can sleep.
> Global counter updates can run in atomic context and must not sleep.
>
> stress-ng dentry test average throughput change:
>
> +------------------------+------------+--------------------+
> | Case | Unit | Average change (%) |
> +------------------------+------------+--------------------+
> | Dentry, 64 workers | bogo ops/s | +47.34 |
> +------------------------+------------+--------------------+
>
> Dentry ran on ext4 with 64 workers pinned to physical CPUs:
>
> stress-ng --dentry 64 --dentries 4096 --timeout 20s
lol, this is a fun patch - it reminds me of the old days. All those
mission-critical workloads which sit in a loop running statfs will love
it!
I had $sidekick do a scan and it told me "the only plausible routine
win is dirty throttling: with low dirty limits, concurrent writers sum
the same counters. ext4 and XFS sum near ENOSPC, where updates also
contend. BFQ's sums are debug-only and serialized. Most other callers
report statistics". You might want to add something like this to
changelog.
Also mention that percpu_counter_sum_positive callers will benefit.
> --- a/include/linux/percpu_counter.h
> +++ b/include/linux/percpu_counter.h
> @@ -20,7 +20,11 @@
> #ifdef CONFIG_SMP
>
> struct percpu_counter {
> +#ifdef CONFIG_PREEMPT_RT
Might be worth a wee comment telling readers why RT is special?
> raw_spinlock_t lock;
> +#else
> + rwlock_t lock; /* shared for sums, exclusive for updates */
> +#endif
> s64 count;
> #ifdef CONFIG_HOTPLUG_CPU
> struct list_head list; /* All percpu_counters are on a list */
>
> --- a/lib/percpu_counter.c
> +++ b/lib/percpu_counter.c
> @@ -15,6 +15,46 @@ static LIST_HEAD(percpu_counters);
> static DEFINE_SPINLOCK(percpu_counters_lock);
> #endif
>
> +/*
> + * Non-RT sums share the lock; global updates take it exclusively.
> + * Keep raw spinlocks on RT: rwlocks may sleep and their irqsave variants
> + * do not disable interrupts. Updates run in atomic contexts and need
> + * IRQ exclusion for CPU-local read-modify-write sequences.
> + */
> +#ifdef CONFIG_PREEMPT_RT
> +#define fbc_lock(fbc) raw_spin_lock(&(fbc)->lock)
> +#define fbc_unlock(fbc) raw_spin_unlock(&(fbc)->lock)
> +#define fbc_lock_irqsave(fbc, f) raw_spin_lock_irqsave(&(fbc)->lock, f)
> +#define fbc_unlock_irqrestore(fbc, f) raw_spin_unlock_irqrestore(&(fbc)->lock, f)
> +#define fbc_read_lock_irqsave fbc_lock_irqsave
> +#define fbc_read_unlock_irqrestore fbc_unlock_irqrestore
> +#else
> +#define fbc_lock(fbc) write_lock(&(fbc)->lock)
> +#define fbc_unlock(fbc) write_unlock(&(fbc)->lock)
> +#define fbc_lock_irqsave(fbc, f) write_lock_irqsave(&(fbc)->lock, f)
> +#define fbc_unlock_irqrestore(fbc, f) write_unlock_irqrestore(&(fbc)->lock, f)
> +#define fbc_read_lock_irqsave(fbc, f) read_lock_irqsave(&(fbc)->lock, f)
> +#define fbc_read_unlock_irqrestore(fbc, f) read_unlock_irqrestore(&(fbc)->lock, f)
> +#endif
Please, no!
https://www.google.com/search?q=why+does+Andrew+Morton+prefer+static+inlines+over+macros
lots of good stuff there ;)
next prev parent reply other threads:[~2026-11-16 3:16 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 1:56 Jiebin Sun
2026-11-16 3:16 ` Andrew Morton [this message]
2026-11-16 3:44 ` Andrew Morton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261115191636.f18859e4dd5f58b9a33e7814@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=bigeasy@linutronix.de \
--cc=cl@gentwo.org \
--cc=clrkwllms@kernel.org \
--cc=dennis@kernel.org \
--cc=jiebin.sun@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-rt-devel@lists.linux.dev \
--cc=rostedt@goodmis.org \
--cc=tianyou.li@intel.com \
--cc=tim.c.chen@linux.intel.com \
--cc=tj@kernel.org \
--cc=wangyang.guo@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®