mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Namhyung Kim <namhyung@kernel.org>
To: Harry Yoo <harry@kernel.org>
Cc: Tim Menninger <tmenninger@everpuredata.com>,
	linux-mm@kvack.org, Chuck Lever <cel@kernel.org>,
	linux-nfs@vger.kernel.org, Jon Curley <jcurley@everpuredata.com>,
	Eric Badger <ebadger@everpuredata.com>,
	Vlastimil Babka <vbabka@kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Hao Li <hao.li@linux.dev>, Christoph Lameter <cl@gentwo.org>,
	David Rientjes <rientjes@google.com>,
	Roman Gushchin <roman.gushchin@linux.dev>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
	Adrian Hunter <adrian.hunter@intel.com>,
	James Clark <james.clark@linaro.org>,
	linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [SLUB] nfs_page cmpxchg_double_fail and perf lock perturbation on dual-socket NFS/RDMA
Date: Fri, 18 Sep 2026 00:14:02 -0700	[thread overview]
Message-ID: <aqzkugwB_I5X2JYx@z2> (raw)
In-Reply-To: <aqvy8NqRRRmUlxjR@thinkstation>

On Thu, Sep 17, 2026 at 03:10:47PM +0100, Harry Yoo wrote:
> ... now I realize we need to add PERFORMANCE EVENTS SUBSYSTEM folks 
> as well ;-)
> 
> Hmm, sounds like perf lock is somehow triggering slab allocations
> and interfering the workload.
> 
> It could be because SLUB is merging nfs_page cache with some other
> cache that perf uses.
> 
> Could you please check if it reproduces with slab_nomerge kernel
> parameter?
> 
> On Thu, Sep 17, 2026 at 02:46:24PM +0100, Harry Yoo wrote:
> > Hi Tim and Chuck, thanks for reporting this to linux-mm.
> > Will take a look at this but let me Cc SLAB ALLOCATOR folks here first.
> > 
> > -- 
> > Cheers,
> > Harry / Hyeonggon
> > 
[SNIP]
> > On Wed, Sep 16, 2026 at 11:22:27PM +0000, Tim Menninger wrote:
> > > My questions are:
> > > 
> > > 1. Do the uninstrumented cmpxchg_double_fail rates above look abnormal
> > >    for this workload/topology, or are they within the range one would
> > >    expect from this degree of concurrency and NUMA placement?
> > > 
> > > 2. Is there a less invasive way you would recommend measuring the
> > >    nfs_page list_lock/freelist contention? I would like to distinguish
> > >    the steady-state behavior from what is observed during the perf-lock
> > >    capture.

You can use the BPF version of perf lock to check lock contention like
below.  (it only work with 'contention' subcommand.)  It won't create
a data file and just shows the result.

  $ sudo perf lock con -ab sleep 10

or

  $ sudo perf lock con -ab -E 5 sleep 10

to see top-5 locks only.

Thanks,
Namhyung


      parent reply	other threads:[~2026-09-18  7:14 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20260916232227.4098143-1-tmenninger@everpuredata.com>
     [not found] ` <aqvq8-6IJBDer90O@thinkstation>
2026-09-17 14:10   ` Harry Yoo
2026-09-17 15:29     ` Peter Zijlstra
2026-09-18  7:04       ` Namhyung Kim
2026-09-17 16:25     ` Tim Menninger
2026-09-18  7:08       ` Vlastimil Babka (SUSE)
2026-09-18  7:14     ` Namhyung Kim [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aqzkugwB_I5X2JYx@z2 \
    --to=namhyung@kernel.org \
    --cc=acme@kernel.org \
    --cc=adrian.hunter@intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=cel@kernel.org \
    --cc=cl@gentwo.org \
    --cc=ebadger@everpuredata.com \
    --cc=hao.li@linux.dev \
    --cc=harry@kernel.org \
    --cc=irogers@google.com \
    --cc=james.clark@linaro.org \
    --cc=jcurley@everpuredata.com \
    --cc=jolsa@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rientjes@google.com \
    --cc=roman.gushchin@linux.dev \
    --cc=tmenninger@everpuredata.com \
    --cc=vbabka@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®