[linus:master] [fs/pipe] f7f4665dc5: lmbench3.PIPE.bandwidth.MB/sec 18.5% regression

[linus:master] [fs/pipe] f7f4665dc5: lmbench3.PIPE.bandwidth.MB/sec 18.5% regression

[kernel test robot]

Hello,

kernel test robot noticed a 18.5% regression of lmbench3.PIPE.bandwidth.MB/sec on:


commit: f7f4665dc520fd8bbc1db20e59945773e03744a0 ("fs/pipe: unify the page pools into a single per-pipe pool") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

testcase: lmbench3
config: x86_64-rhel-9.4
compiler: gcc-14
test machine: 224 threads 2 sockets Intel(R) Xeon(R) (Sapphire Rapids) with 128G memory
parameters:

	test_memory_size: 50%
	nr_threads: 20%
	mode: development
	test: PIPE
	cpufreq_governor: performance

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <yi1.lai@intel.com>
| Closes: https://lore.kernel.org/oe-lkp/202611041404.befe64cd-lkp@intel.com

Details are as below:
-------------------------------------------------------------------------------------------------->

The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20261104/202611041404.befe64cd-lkp@intel.com

=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_threads/rootfs/tbox_group/test/test_memory_size/testcase:
  gcc-14/performance/x86_64-rhel-9.4/development/20%/debian-13-x86_64-20250902.cgz/lkp-spr-2sp2/PIPE/50%/lmbench3

commit: 
  c610d2d078 ("fs: annotate inode timestamp accessors")
  f7f4665dc5 ("fs/pipe: unify the page pools into a single per-pipe pool")

c610d2d0787961cd f7f4665dc520fd8bbc1db20e599
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
     29142 ±  2%     -18.5%      23737        lmbench3.PIPE.bandwidth.MB/sec
    191.80 ±  8%     +54.1%     295.65 ±  5%  lmbench3.time.elapsed_time
    191.80 ±  8%     +54.1%     295.65 ±  5%  lmbench3.time.elapsed_time.max
     23261 ± 16%     +19.3%      27743 ±  5%  lmbench3.time.involuntary_context_switches
      9853 ± 16%     +27.9%      12605 ±  5%  lmbench3.time.system_time
     51.80 ±  2%     +11.6%      57.80 ±  3%  lmbench3.time.user_time
 2.928e+08 ±  3%     +26.6%  3.706e+08 ±  4%  lmbench3.time.voluntary_context_switches


Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.


-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [linus:master] [fs/pipe] f7f4665dc5: lmbench3.PIPE.bandwidth.MB/sec 18.5% regression

[Christian Brauner]

On Thu, Nov 05, 2026 at 08:38:45AM +0800, kernel test robot wrote:
> 
> 
> Hello,
> 
> kernel test robot noticed a 18.5% regression of lmbench3.PIPE.bandwidth.MB/sec on:
> 
> 
> commit: f7f4665dc520fd8bbc1db20e59945773e03744a0 ("fs/pipe: unify the page pools into a single per-pipe pool") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

@Breno can you reproduce this?
It's something we should at least investigate.

> 
> testcase: lmbench3
> config: x86_64-rhel-9.4
> compiler: gcc-14
> test machine: 224 threads 2 sockets Intel(R) Xeon(R) (Sapphire Rapids) with 128G memory
> parameters:
> 
> 	test_memory_size: 50%
> 	nr_threads: 20%
> 	mode: development
> 	test: PIPE
> 	cpufreq_governor: performance
> 
> If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
> | Reported-by: kernel test robot <yi1.lai@intel.com>
> | Closes: https://lore.kernel.org/oe-lkp/202611041404.befe64cd-lkp@intel.com
> 
> Details are as below:
> -------------------------------------------------------------------------------------------------->
> 
> The kernel config and materials to reproduce are available at:
> https://download.01.org/0day-ci/archive/20261104/202611041404.befe64cd-lkp@intel.com
> 
> =========================================================================================
> compiler/cpufreq_governor/kconfig/mode/nr_threads/rootfs/tbox_group/test/test_memory_size/testcase:
>   gcc-14/performance/x86_64-rhel-9.4/development/20%/debian-13-x86_64-20250902.cgz/lkp-spr-2sp2/PIPE/50%/lmbench3
> 
> commit: 
>   c610d2d078 ("fs: annotate inode timestamp accessors")
>   f7f4665dc5 ("fs/pipe: unify the page pools into a single per-pipe pool")
> 
> c610d2d0787961cd f7f4665dc520fd8bbc1db20e599
> ---------------- --------------------------- 
>          %stddev     %change         %stddev
>              \          |                \  
>      29142 ±  2%     -18.5%      23737        lmbench3.PIPE.bandwidth.MB/sec
>     191.80 ±  8%     +54.1%     295.65 ±  5%  lmbench3.time.elapsed_time
>     191.80 ±  8%     +54.1%     295.65 ±  5%  lmbench3.time.elapsed_time.max
>      23261 ± 16%     +19.3%      27743 ±  5%  lmbench3.time.involuntary_context_switches
>       9853 ± 16%     +27.9%      12605 ±  5%  lmbench3.time.system_time
>      51.80 ±  2%     +11.6%      57.80 ±  3%  lmbench3.time.user_time
>  2.928e+08 ±  3%     +26.6%  3.706e+08 ±  4%  lmbench3.time.voluntary_context_switches
> 
> 
> Disclaimer:
> Results have been estimated based on internal Intel analysis and are provided
> for informational purposes only. Any difference in system hardware or software
> design or configuration may affect actual performance.
> 
> 
> -- 
> 0-DAY CI Kernel Test Service
> https://github.com/intel/lkp-tests/wiki
> 
> 
> 

-- 

Re: [linus:master] [fs/pipe] f7f4665dc5: lmbench3.PIPE.bandwidth.MB/sec 18.5% regression

[Breno Leitao]

On Thu, Nov 05, 2026 at 04:04:58PM +0100, Christian Brauner wrote:
> On Thu, Nov 05, 2026 at 08:38:45AM +0800, kernel test robot wrote:
> > kernel test robot noticed a 18.5% regression of lmbench3.PIPE.bandwidth.MB/sec on:
> > 
> > commit: f7f4665dc520fd8bbc1db20e59945773e03744a0 ("fs/pipe: unify the page pools into a single per-pipe pool") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master


> @Breno can you reproduce this?
> It's something we should at least investigate.

Ack, I will check it , but not today unfortunately.

I will come back to it tomorrow, and until end of week I will report my
findings.

Re: [linus:master] [fs/pipe] f7f4665dc5: lmbench3.PIPE.bandwidth.MB/sec 18.5% regression

[Mateusz Guzik]

On Thu, Nov 5, 2026 at 4:28 PM Breno Leitao <leitao@debian.org> wrote:
>
> On Thu, Nov 05, 2026 at 04:04:58PM +0100, Christian Brauner wrote:
> > On Thu, Nov 05, 2026 at 08:38:45AM +0800, kernel test robot wrote:
> > > kernel test robot noticed a 18.5% regression of lmbench3.PIPE.bandwidth.MB/sec on:
> > >
> > > commit: f7f4665dc520fd8bbc1db20e59945773e03744a0 ("fs/pipe: unify the page pools into a single per-pipe pool") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>
>
> > @Breno can you reproduce this?
> > It's something we should at least investigate.
>
> Ack, I will check it , but not today unfortunately.
>
> I will come back to it tomorrow, and until end of week I will report my
> findings.

I figured it out.

The test at hand has multiple readers and writers using the same pipe,
with 64KB i/os.

Prior to the patch each writer had its own private pool to draw from,
guaranteeing no allocation will happen under the mutex.

With the patch the reader exit also drains the pool.

As a result tons of allocation ends up happening under the lock again:

-   94.88%    94.88%  bw_pipe          [kernel.kallsyms]     [k]
__alloc_frozen_pages_noprof
     94.88% __internal_syscall_cancel
        entry_SYSCALL_64_after_hwframe
        do_syscall_64
        ksys_write
        vfs_write
      - anon_pipe_write
         - 47.46% alloc_pages_noprof
              alloc_pages_mpol
              __alloc_frozen_pages_noprof
         - 47.42% anon_pipe_prefill_and_lock
              alloc_pages_noprof
              alloc_pages_mpol
              __alloc_frozen_pages_noprof

I don't think multiple high-volume writers vs multiple high
volume-readers is a realistic scenario and I would not bother fixing
it.

However, should someone want to address it, I think the patch should
be reverted and the resulting state tweaked. The patch at hand came
with some win for throughput in one reader vs one writer scenario,
which I'm pretty sure stems from postponing page deallocation until
after the mutex is released -- the kernel prior to any of this would
still put_page on the reader side under the lock. This bit would have
to get tweaked to recover the loss in that scenario (not to be
confused with the loss reported here).

tl;dr i think this is a EIRRELEVANT kind of a report

Re: [linus:master] [fs/pipe] f7f4665dc5: lmbench3.PIPE.bandwidth.MB/sec 18.5% regression

[Breno Leitao]

On Thu, Nov 05, 2026 at 04:57:33PM +0100, Mateusz Guzik wrote:
> On Thu, Nov 5, 2026 at 4:28 PM Breno Leitao <leitao@debian.org> wrote:
>
> The test at hand has multiple readers and writers using the same pipe,
> with 64KB i/os.
> 
> Prior to the patch each writer had its own private pool to draw from,
> guaranteeing no allocation will happen under the mutex.
> 
> With the patch the reader exit also drains the pool.
> 
> As a result tons of allocation ends up happening under the lock again:
> 
> -   94.88%    94.88%  bw_pipe          [kernel.kallsyms]     [k]
> __alloc_frozen_pages_noprof
>      94.88% __internal_syscall_cancel
>         entry_SYSCALL_64_after_hwframe
>         do_syscall_64
>         ksys_write
>         vfs_write
>       - anon_pipe_write
>          - 47.46% alloc_pages_noprof
>               alloc_pages_mpol
>               __alloc_frozen_pages_noprof
>          - 47.42% anon_pipe_prefill_and_lock
>               alloc_pages_noprof
>               alloc_pages_mpol
>               __alloc_frozen_pages_noprof

Ack, this matches what I found as well. Thanks for investigating it.

Because the preallocated pages are not private to the writer anymore,
the reader flushes them while the writer waits for room in the pipe,
and the writer then has to allocate them inside the lock.

This basically loses the wins[0] from commit 212ed884a1aeef ("fs/pipe:
pre-allocate pages outside pipe->mutex in anon_pipe_write") for this
specific scenario.

> I don't think multiple high-volume writers vs multiple high
> volume-readers is a realistic scenario and I would not bother fixing
> it.

Agree, but I would like to spend some cycles on it.

> However, should someone want to address it, I think the patch should
> be reverted and the resulting state tweaked. The patch at hand came
> after the mutex is released -- the kernel prior to any of this would
> still put_page on the reader side under the lock. This bit would have
> to get tweaked to recover the loss in that scenario (not to be
> confused with the loss reported here).

We can revert the patch, but that would bring back the two page pools
(the on-stack prealloc and tmp_page[]), which is probably worse than
this regression itself (IIRC it would also lose the gains for smaller
writes (about 20% for 8-16 KiB writes IIRC ))

That said, these are the items I am planning to spend some time one:

1) Reclaiming the pooled pages from a shrinker, as suggested in [1]

2) Trimming the pool gradually on the reader side, instead of cutting it
   down to two pages in one go. Or not trimming it while a writer is
   in the waiting room (!?)

3) Keeping the writer's preallocated pages private again, while keeping
   the unified pool (i.e. without bringing tmp_page[] back).

4) Freeing the pages the reader drops after releasing pipe->mutex,
   instead of under it, as you suggested. This is independent of the
   options above.

5) Longer term: copying the data into the preallocated pages before
   taking pipe->mutex, so the writer only links them into the pipe
   under the lock. This doesn't seem trivial, but it would make this
   whole pre-allocated buffers go away

6) Any other idea?

[0] https://lore.kernel.org/all/202607091015.c3936260-lkp@intel.com/
[1] https://lore.kernel.org/all/CAGudoHGKyN6k+eSAUm35_WDSZcxxtm4QL6fO+4rYJTvtObMqzA@mail.gmail.com/

Thanks for looking at it,
--breno

Re: [linus:master] [fs/pipe] f7f4665dc5: lmbench3.PIPE.bandwidth.MB/sec 18.5% regression

[Mateusz Guzik]

On Fri, Nov 6, 2026 at 12:11 PM Breno Leitao <leitao@debian.org> wrote:
>
> On Thu, Nov 05, 2026 at 04:57:33PM +0100, Mateusz Guzik wrote:
> > On Thu, Nov 5, 2026 at 4:28 PM Breno Leitao <leitao@debian.org> wrote:
> >
> > The test at hand has multiple readers and writers using the same pipe,
> > with 64KB i/os.
> >
> > Prior to the patch each writer had its own private pool to draw from,
> > guaranteeing no allocation will happen under the mutex.
> >
> > With the patch the reader exit also drains the pool.
> >
> > As a result tons of allocation ends up happening under the lock again:
> >
> > -   94.88%    94.88%  bw_pipe          [kernel.kallsyms]     [k]
> > __alloc_frozen_pages_noprof
> >      94.88% __internal_syscall_cancel
> >         entry_SYSCALL_64_after_hwframe
> >         do_syscall_64
> >         ksys_write
> >         vfs_write
> >       - anon_pipe_write
> >          - 47.46% alloc_pages_noprof
> >               alloc_pages_mpol
> >               __alloc_frozen_pages_noprof
> >          - 47.42% anon_pipe_prefill_and_lock
> >               alloc_pages_noprof
> >               alloc_pages_mpol
> >               __alloc_frozen_pages_noprof
>
> Ack, this matches what I found as well. Thanks for investigating it.
>
> Because the preallocated pages are not private to the writer anymore,
> the reader flushes them while the writer waits for room in the pipe,
> and the writer then has to allocate them inside the lock.
>
> This basically loses the wins[0] from commit 212ed884a1aeef ("fs/pipe:
> pre-allocate pages outside pipe->mutex in anon_pipe_write") for this
> specific scenario.
>
> > I don't think multiple high-volume writers vs multiple high
> > volume-readers is a realistic scenario and I would not bother fixing
> > it.
>
> Agree, but I would like to spend some cycles on it.
>
> > However, should someone want to address it, I think the patch should
> > be reverted and the resulting state tweaked. The patch at hand came
> > after the mutex is released -- the kernel prior to any of this would
> > still put_page on the reader side under the lock. This bit would have
> > to get tweaked to recover the loss in that scenario (not to be
> > confused with the loss reported here).
>
> We can revert the patch, but that would bring back the two page pools
> (the on-stack prealloc and tmp_page[]), which is probably worse than
> this regression itself (IIRC it would also lose the gains for smaller
> writes (about 20% for 8-16 KiB writes IIRC ))
>

Like I said above, per my understanding the win is not specific to the
unified pool thing, but merely to the fact that the reader would put
pages *after* releasing the lock. this can be done in either approach.

> That said, these are the items I am planning to spend some time one:
>
> 1) Reclaiming the pooled pages from a shrinker, as suggested in [1]
>

The more I look at this the more the extra memory usage bothers me --
even with shrinker support there is just plain waste until it runs and
it is going to cost elsewhere. If this was "free" performance I would
have no comments.

Makes me wonder if there is any practical win to pursuing it in the first place.

But here is an idea just to ponder: what if the threads using the pipe
were guaranteed to trim it at some point, even if it remains open and
unused?

Currently there is the notion of task_work, which allows for arbitrary
callbacks to execute on the user<->kernel boundary. As implemented it
is rather expensive (atomics) to use and always flushes the entire
work queue.

But imagine the mechanism got extended such that there is a separate
queue where only current can add stuff (so it is cheap). Getting to
the code could be gated with the same branch which tells the kernel
about task_work, so no extra overhead in syscall handling as long as
you don't have anything queued up.

With this in place pipe code seeing a big buffer could add a callback,
which would do some magic to decide whether to trim the pool and by
what amount. Just for the sake of argument, here is an example: 1/16
chance it will do anything to begin with. if it chooses to do
something, it checks if there is a "trimmer was here" bit. If no, sets
the bit and leaves. If yes, shrinks the pool in half. The pipe write
code would clear the bit, so a pipe with continuous usage would mostly
remain intact and an unused pipe would eventually drain to nothing
(including the currently held 2 pages).

> 2) Trimming the pool gradually on the reader side, instead of cutting it
>    down to two pages in one go. Or not trimming it while a writer is
>    in the waiting room (!?)
>

There is no indicator there is a writer pending, at best you can check
if the lock is contested. This seems like gaming the benchmark to me
though, as a real program who happened to generate data a little later
would suddenly find itself with the pool trimmed.

Granular trimming definitely makes sense, provided shrinker support is
implemented.

> 3) Keeping the writer's preallocated pages private again, while keeping
>    the unified pool (i.e. without bringing tmp_page[] back).
>
> 4) Freeing the pages the reader drops after releasing pipe->mutex,
>    instead of under it, as you suggested. This is independent of the
>    options above.
>
> 5) Longer term: copying the data into the preallocated pages before
>    taking pipe->mutex, so the writer only links them into the pipe
>    under the lock. This doesn't seem trivial, but it would make this
>    whole pre-allocated buffers go away
>

I think this is sounds awfully complicated in face of multiple writers.

> 6) Any other idea?

The SMAP stuff is the primary problem right now. To reiterate: every
single buffer read/write takes a SMAP trip, which is very expensive
even on modern uarchs. Refactoring this into i/o under one SMAP round
would provide a visible speed up. It also happens to not pose a
tradeoff in terms of memory usage and would be my first avenue of
attack if I was working on this.

All that said, one reader vs one writer scenario definitely makes
sense to optimize.

Apart from that, highly parallel i/o on one pipe is imo the wrong
thing to look at in the first place. What is more likely to occur in
the real world is the user having separate processes operating on
distinct pipes, but this was not benchmarked at all. Another case to
look at is one reader vs one writer, but all the tuples have unique
uids.