From: Namhyung Kim <namhyung@kernel.org>
To: Harry Yoo <harry@kernel.org>
Cc: Tim Menninger <tmenninger@everpuredata.com>,
linux-mm@kvack.org, Chuck Lever <cel@kernel.org>,
linux-nfs@vger.kernel.org, Jon Curley <jcurley@everpuredata.com>,
Eric Badger <ebadger@everpuredata.com>,
Vlastimil Babka <vbabka@kernel.org>,
Andrew Morton <akpm@linux-foundation.org>,
Hao Li <hao.li@linux.dev>, Christoph Lameter <cl@gentwo.org>,
David Rientjes <rientjes@google.com>,
Roman Gushchin <roman.gushchin@linux.dev>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
James Clark <james.clark@linaro.org>,
linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [SLUB] nfs_page cmpxchg_double_fail and perf lock perturbation on dual-socket NFS/RDMA
Date: Fri, 18 Sep 2026 00:14:02 -0700 [thread overview]
Message-ID: <aqzkugwB_I5X2JYx@z2> (raw)
In-Reply-To: <aqvy8NqRRRmUlxjR@thinkstation>
On Thu, Sep 17, 2026 at 03:10:47PM +0100, Harry Yoo wrote:
> ... now I realize we need to add PERFORMANCE EVENTS SUBSYSTEM folks
> as well ;-)
>
> Hmm, sounds like perf lock is somehow triggering slab allocations
> and interfering the workload.
>
> It could be because SLUB is merging nfs_page cache with some other
> cache that perf uses.
>
> Could you please check if it reproduces with slab_nomerge kernel
> parameter?
>
> On Thu, Sep 17, 2026 at 02:46:24PM +0100, Harry Yoo wrote:
> > Hi Tim and Chuck, thanks for reporting this to linux-mm.
> > Will take a look at this but let me Cc SLAB ALLOCATOR folks here first.
> >
> > --
> > Cheers,
> > Harry / Hyeonggon
> >
[SNIP]
> > On Wed, Sep 16, 2026 at 11:22:27PM +0000, Tim Menninger wrote:
> > > My questions are:
> > >
> > > 1. Do the uninstrumented cmpxchg_double_fail rates above look abnormal
> > > for this workload/topology, or are they within the range one would
> > > expect from this degree of concurrency and NUMA placement?
> > >
> > > 2. Is there a less invasive way you would recommend measuring the
> > > nfs_page list_lock/freelist contention? I would like to distinguish
> > > the steady-state behavior from what is observed during the perf-lock
> > > capture.
You can use the BPF version of perf lock to check lock contention like
below. (it only work with 'contention' subcommand.) It won't create
a data file and just shows the result.
$ sudo perf lock con -ab sleep 10
or
$ sudo perf lock con -ab -E 5 sleep 10
to see top-5 locks only.
Thanks,
Namhyung
prev parent reply other threads:[~2026-09-18 7:14 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20260916232227.4098143-1-tmenninger@everpuredata.com>
[not found] ` <aqvq8-6IJBDer90O@thinkstation>
2026-09-17 14:10 ` Harry Yoo
2026-09-17 15:29 ` Peter Zijlstra
2026-09-18 7:04 ` Namhyung Kim
2026-09-17 16:25 ` Tim Menninger
2026-09-18 7:08 ` Vlastimil Babka (SUSE)
2026-09-18 7:14 ` Namhyung Kim [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aqzkugwB_I5X2JYx@z2 \
--to=namhyung@kernel.org \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=akpm@linux-foundation.org \
--cc=alexander.shishkin@linux.intel.com \
--cc=cel@kernel.org \
--cc=cl@gentwo.org \
--cc=ebadger@everpuredata.com \
--cc=hao.li@linux.dev \
--cc=harry@kernel.org \
--cc=irogers@google.com \
--cc=james.clark@linaro.org \
--cc=jcurley@everpuredata.com \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nfs@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rientjes@google.com \
--cc=roman.gushchin@linux.dev \
--cc=tmenninger@everpuredata.com \
--cc=vbabka@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®