mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers
@ 2026-11-11 21:02 Mani Tofigh via B4 Relay
  2026-11-11 23:34 ` Oleg Nesterov
                   ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: Mani Tofigh via B4 Relay @ 2026-11-11 21:02 UTC (permalink / raw)
  To: Christian Brauner, Alexander Viro
  Cc: Jan Kara, Oleg Nesterov, linux-fsdevel, linux-kernel, tz2294,
	stable, Mani Tofigh

From: Mani Tofigh <mt4022@columbia.edu>

A task blocked in read() on a pipe can sleep while the pipe holds data,
when an epoll(EPOLLEXCLUSIVE) watcher of the read end was added before
it.  The watcher's entry on ->rd_wait is exclusive and sits ahead of the
reader, so a write into an empty pipe wakes only the watcher.  The
watcher reads without sleeping in anon_pipe_read() and doesn't pass the
wakeup on.  Since commit f1cebdec9739 ("pipe: only enable the extra
wake_up(rd_wait) for EPOLLET consumers") the next writes into the
non-empty pipe don't wake anyone anymore:

  Task E                   Task A                 Task C
  epoll_wait(ep, -1)
    /* EPOLLEXCLUSIVE */
                           read(p[0])
                                                  write(p[1], "1")
                                                    /* wakes E only */
                                                  write(p[1], "2")
                                                    /* no wakeup */
  read(p[0], 1 byte)
  exit()
                           /* still asleep, "2" is in the pipe */

A then sleeps until the next write into an empty pipe, if ever.  Before
that commit any pipe_poll() enabled the per-write wakeups.  Those
reached A because an EPOLLEXCLUSIVE entry only takes a wakeup while a
task sleeps in epoll_wait().

So EPOLLEXCLUSIVE consumers depend on the per-write wakeups too, for a
different reason than EPOLLET.  Enable them for EPOLLEXCLUSIVE as well
and say why in the comment.  With the fix, A reads the second byte.

This bug was identified and confirmed with the help of a deterministic
hypervisor currently under development.  The C reproducer below however
hits it within about 2s.

Fixes: f1cebdec9739 ("pipe: only enable the extra wake_up(rd_wait) for EPOLLET consumers")
Cc: stable@vger.kernel.org
Signed-off-by: Mani Tofigh <mt4022@columbia.edu>
---
Reproducer: https://github.com/manitofigh/files/blob/main/linux-repros/pipeexcl.c

There are a couple of older gaps like this (a single write, and the
write side with EPOLLOUT|EPOLLEXCLUSIVE).  I left both alone here though.
---
 fs/pipe.c                 | 6 +++++-
 include/linux/pipe_fs_i.h | 2 +-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/fs/pipe.c b/fs/pipe.c
index 4c1122597a3c..8ad792d300c9 100644
--- a/fs/pipe.c
+++ b/fs/pipe.c
@@ -700,9 +700,13 @@ pipe_poll(struct file *filp, poll_table *wait)
 	 * and the ->pseudo_edgetrigger check in anon_pipe_write().
 	 * Currently io_uring sets EPOLLET for multishot polls, so it gets the
 	 * same behaviour.
+	 *
+	 * EPOLLEXCLUSIVE needs them too.  Its entry is an exclusive waiter on
+	 * ->rd_wait that can take the wakeup meant for a blocked reader, and
+	 * unlike a woken reader in anon_pipe_read() it doesn't pass it on.
 	 */
 	if ((filp->f_mode & FMODE_READ) &&
-	    wait && (wait->_key & EPOLLET) &&
+	    wait && (wait->_key & (EPOLLET | EPOLLEXCLUSIVE)) &&
 	    unlikely(!READ_ONCE(pipe->pseudo_edgetrigger)))
 		WRITE_ONCE(pipe->pseudo_edgetrigger, true);
 
diff --git a/include/linux/pipe_fs_i.h b/include/linux/pipe_fs_i.h
index 95f6236c5c5b..6b2e14ec6ef8 100644
--- a/include/linux/pipe_fs_i.h
+++ b/include/linux/pipe_fs_i.h
@@ -90,7 +90,7 @@ struct anon_pipe_prealloc {
  *	@files: number of struct file referring this pipe (protected by ->i_lock)
  *	@r_counter: reader counter
  *	@w_counter: writer counter
- *	@pseudo_edgetrigger: has an EPOLLET consumer, enable per-write wakeups
+ *	@pseudo_edgetrigger: enable per-write wakeups for EPOLLET/EPOLLEXCLUSIVE
  *	@fasync_readers: reader side fasync
  *	@fasync_writers: writer side fasync
  *	@bufs: the circular array of pipe buffers

---
base-commit: b0993210e420f2f1d61e6719fbcd3966b96ff9ed
change-id: 20261111-pipe-epollexclusive-wakeup-981528041b4d

Best regards,
--  
Mani Tofigh <mt4022@columbia.edu>



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers
  2026-11-11 21:02 [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers Mani Tofigh via B4 Relay
@ 2026-11-11 23:34 ` Oleg Nesterov
  2026-11-12 10:26   ` Oleg Nesterov
  2026-11-12 12:56 ` Oleg Nesterov
  2026-11-17 13:27 ` Oleg Nesterov
  2 siblings, 1 reply; 9+ messages in thread
From: Oleg Nesterov @ 2026-11-11 23:34 UTC (permalink / raw)
  To: Mani Tofigh via B4 Relay
  Cc: Christian Brauner, Alexander Viro, Jan Kara, linux-fsdevel,
	linux-kernel, tz2294, stable, Mani Tofigh

On 11/11, Mani Tofigh via B4 Relay wrote:
>
> From: Mani Tofigh <mt4022@columbia.edu>
>
> A task blocked in read() on a pipe can sleep while the pipe holds data,
> when an epoll(EPOLLEXCLUSIVE) watcher of the read end was added before
> it.  The watcher's entry on ->rd_wait is exclusive and sits ahead of the
> reader, so a write into an empty pipe wakes only the watcher.

I am already sleeping, but at first glance this is already wrong?

I mean, a WQ_FLAG_EXCLUSIVE waiter should not sit ahead of non-exclusive
waiter, no?

Oleg.


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers
  2026-11-11 23:34 ` Oleg Nesterov
@ 2026-11-12 10:26   ` Oleg Nesterov
  0 siblings, 0 replies; 9+ messages in thread
From: Oleg Nesterov @ 2026-11-12 10:26 UTC (permalink / raw)
  To: Mani Tofigh via B4 Relay
  Cc: Christian Brauner, Alexander Viro, Jan Kara, linux-fsdevel,
	linux-kernel, tz2294, stable, Mani Tofigh

On 11/12, Oleg Nesterov wrote:
>
> On 11/11, Mani Tofigh via B4 Relay wrote:
> >
> > From: Mani Tofigh <mt4022@columbia.edu>
> >
> > A task blocked in read() on a pipe can sleep while the pipe holds data,
> > when an epoll(EPOLLEXCLUSIVE) watcher of the read end was added before
> > it.  The watcher's entry on ->rd_wait is exclusive and sits ahead of the
> > reader, so a write into an empty pipe wakes only the watcher.
>
> I am already sleeping, but at first glance this is already wrong?
>
> I mean, a WQ_FLAG_EXCLUSIVE waiter should not sit ahead of non-exclusive
> waiter, no?

Ah, I forgot that anon_pipe_read() uses wait_event_exclusive(), and it
assumes that this wait-event is the only reason to set wake_next_reader...

I seem to understand, let me think about it.

Oleg.


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers
  2026-11-11 21:02 [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers Mani Tofigh via B4 Relay
  2026-11-11 23:34 ` Oleg Nesterov
@ 2026-11-12 12:56 ` Oleg Nesterov
  2026-11-17 13:27 ` Oleg Nesterov
  2 siblings, 0 replies; 9+ messages in thread
From: Oleg Nesterov @ 2026-11-12 12:56 UTC (permalink / raw)
  To: Mani Tofigh via B4 Relay
  Cc: Christian Brauner, Alexander Viro, Jan Kara, linux-fsdevel,
	linux-kernel, tz2294, stable, Mani Tofigh

This probably needs more discussion... I will write another email.
But so far I think we don't really care.

On 11/11, Mani Tofigh via B4 Relay wrote:
>
> A task blocked in read() on a pipe can sleep while the pipe holds data,
> when an epoll(EPOLLEXCLUSIVE) watcher of the read end was added before
> it.  The watcher's entry on ->rd_wait is exclusive and sits ahead of the
> reader, so a write into an empty pipe wakes only the watcher.  The
> watcher reads without sleeping in anon_pipe_read() and doesn't pass the
> wakeup on.  Since commit f1cebdec9739 ("pipe: only enable the extra
> wake_up(rd_wait) for EPOLLET consumers") the next writes into the
> non-empty pipe don't wake anyone anymore:
>
>   Task E                   Task A                 Task C
>   epoll_wait(ep, -1)
>     /* EPOLLEXCLUSIVE */
>                            read(p[0])
>                                                   write(p[1], "1")
>                                                     /* wakes E only */
>                                                   write(p[1], "2")
>                                                     /* no wakeup */
>   read(p[0], 1 byte)
>   exit()
>                            /* still asleep, "2" is in the pipe */

Yes, in this case f1cebdec9739 ("pipe: only enable the extra wake_up(rd_wait)
for EPOLLET consumers") makes a difference.

However, this trivial test-case

	#include <unistd.h>
	#include <sys/epoll.h>
	#include <sys/wait.h>
	#include <assert.h>
	#include <stdio.h>

	int main(void)
	{
		int pfd[2], efd;
		struct epoll_event evt = { .events = EPOLLIN | EPOLLEXCLUSIVE };
		char c;

		pipe(pfd);
		efd = epoll_create1(0);
		epoll_ctl(efd, EPOLL_CTL_ADD, pfd[0], &evt);

		if (!fork()) {
			epoll_wait(efd, &evt, 1, -1);
			read(pfd[0], &c, 1);
			printf("1st: %c\n", c);
			return 0;
		}

		sleep(1);

		if (!fork()) {
			read(pfd[0], &c, 1);
			printf("2nd: %c\n", c);
			return 0;
		}

		sleep(1);

		write(pfd[1], "12", 2);
		assert(wait(NULL) > 0);
		assert(wait(NULL) > 0);

		return 0;
	}

hangs even with this patch

	+++ b/fs/pipe.c
	@@ -749,6 +749,7 @@ pipe_poll(struct file *filp, poll_table *wait)
		    unlikely(!READ_ONCE(pipe->pseudo_edgetrigger)))
			WRITE_ONCE(pipe->pseudo_edgetrigger, true);
	 
	+WRITE_ONCE(pipe->pseudo_edgetrigger, true);
		/*
		 * Reading pipe state only -- no need for acquiring the semaphore.
		 *

which reverts f1cebdec9739.

The 2nd child sleeps in read(pfd[0]) forever even if the pipe is not empty.
So I am not sure we want to enforce another oddity in pipe_poll/read ...

Oleg.


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers
  2026-11-11 21:02 [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers Mani Tofigh via B4 Relay
  2026-11-11 23:34 ` Oleg Nesterov
  2026-11-12 12:56 ` Oleg Nesterov
@ 2026-11-17 13:27 ` Oleg Nesterov
  2026-11-17 16:22   ` Linus Torvalds
  2 siblings, 1 reply; 9+ messages in thread
From: Oleg Nesterov @ 2026-11-17 13:27 UTC (permalink / raw)
  To: Mani Tofigh via B4 Relay, Breno Leitao, Christian Brauner,
	Linus Torvalds, Mateusz Guzik
  Cc: Alexander Viro, Jan Kara, linux-fsdevel, linux-kernel, tz2294,
	stable, Mani Tofigh

Mani reports the case when f1cebdec9739 ("pipe: only enable the extra
wake_up(rd_wait) for EPOLLET consumers") makes a difference, see his
email below.

But lets forget about that commit for the moment. With or without that
commit this trivial test-case hangs,

	#include <unistd.h>
	#include <sys/epoll.h>
	#include <sys/wait.h>
	#include <assert.h>
	#include <stdio.h>

	int main(void)
	{
		int pfd[2], efd;
		struct epoll_event evt = { .events = EPOLLIN | EPOLLEXCLUSIVE };
		char c;

		pipe(pfd);
		efd = epoll_create1(0);
		epoll_ctl(efd, EPOLL_CTL_ADD, pfd[0], &evt);

		if (!fork()) {
			epoll_wait(efd, &evt, 1, -1);
			read(pfd[0], &c, 1);
			printf("1st: %c\n", c);
			return 0;
		}

		sleep(1);

		if (!fork()) {
			read(pfd[0], &c, 1);
			printf("2nd: %c\n", c);
			return 0;
		}

		sleep(1);

		write(pfd[1], "12", 2);
		assert(wait(NULL) > 0);
		assert(wait(NULL) > 0);

		return 0;
	}

the 2nd non-epoll child sleeps in read(pfd[0]) even if pipe is not empty.

anon_pipe_read() assumes that all exclusive waiters on pipe->rd_wait come
from anon_pipe_read()->wait_event_interruptible_exclusive(), and the woken
reader will set wake_next_reader = true.

However, EPOLLEXCLUSIVE breaks this logic. ep_ptable_queue_proc() does
add_wait_queue_exclusive() if EPOLLEXCLUSIVE is set. So write(pfd[1]) above
wakes up the 1st "epoll" child, which then reads one byte from the pipe without
sleeping, so it doesn't set wake_next_reader and thus it doesn't wakeup the
2nd waiter.

Again, this happens with or without f1cebdec9739. But that commit makes a
difference: the 2nd child will continue to sleep even after another write
to this pipe. So Mani suggests the trivial change which should restore the
previous behaviour

	--- a/fs/pipe.c
	+++ b/fs/pipe.c
	@@ -700,9 +700,13 @@ pipe_poll(struct file *filp, poll_table *wait)
		 * and the ->pseudo_edgetrigger check in anon_pipe_write().
		 * Currently io_uring sets EPOLLET for multishot polls, so it gets the
		 * same behaviour.
	+	 *
	+	 * EPOLLEXCLUSIVE needs them too.  Its entry is an exclusive waiter on
	+	 * ->rd_wait that can take the wakeup meant for a blocked reader, and
	+	 * unlike a woken reader in anon_pipe_read() it doesn't pass it on.
		 */
		if ((filp->f_mode & FMODE_READ) &&
	-	    wait && (wait->_key & EPOLLET) &&
	+	    wait && (wait->_key & (EPOLLET | EPOLLEXCLUSIVE)) &&
		    unlikely(!READ_ONCE(pipe->pseudo_edgetrigger)))
			WRITE_ONCE(pipe->pseudo_edgetrigger, true);

But I can't decide if we actually want to bless another historical
"nonsensical" wakeup until have a user application that actually cares?

Again, the test-case above will hang with or without the patch above, with
or without f1cebdec9739. So do we really care? If yes, then perhaps we should
also change anon_pipe_read(), something like

	--- a/fs/pipe.c
	+++ b/fs/pipe.c
	@@ -468,6 +468,8 @@ anon_pipe_read(struct kiocb *iocb, struct iov_iter *to)
		}
		if (pipe_is_empty(pipe))
			wake_next_reader = false;
	+	else if (pipe->pseudo_edgetrigger) // for EPOLLEXCLUSIVE
	+		wake_next_reader = true;
		/* Consumed buffers may have refilled the pool; trim it and unlock. */
		anon_pipe_trim_and_unlock(pipe);

?

(of course, this won't help if the 1st child doesn't do read() after epoll_wait()).

What do you think?

Oleg.

On 11/11, Mani Tofigh via B4 Relay wrote:
>
> From: Mani Tofigh <mt4022@columbia.edu>
>
> A task blocked in read() on a pipe can sleep while the pipe holds data,
> when an epoll(EPOLLEXCLUSIVE) watcher of the read end was added before
> it.  The watcher's entry on ->rd_wait is exclusive and sits ahead of the
> reader, so a write into an empty pipe wakes only the watcher.  The
> watcher reads without sleeping in anon_pipe_read() and doesn't pass the
> wakeup on.  Since commit f1cebdec9739 ("pipe: only enable the extra
> wake_up(rd_wait) for EPOLLET consumers") the next writes into the
> non-empty pipe don't wake anyone anymore:
>
>   Task E                   Task A                 Task C
>   epoll_wait(ep, -1)
>     /* EPOLLEXCLUSIVE */
>                            read(p[0])
>                                                   write(p[1], "1")
>                                                     /* wakes E only */
>                                                   write(p[1], "2")
>                                                     /* no wakeup */
>   read(p[0], 1 byte)
>   exit()
>                            /* still asleep, "2" is in the pipe */
>
> A then sleeps until the next write into an empty pipe, if ever.  Before
> that commit any pipe_poll() enabled the per-write wakeups.  Those
> reached A because an EPOLLEXCLUSIVE entry only takes a wakeup while a
> task sleeps in epoll_wait().
>
> So EPOLLEXCLUSIVE consumers depend on the per-write wakeups too, for a
> different reason than EPOLLET.  Enable them for EPOLLEXCLUSIVE as well
> and say why in the comment.  With the fix, A reads the second byte.
>
> This bug was identified and confirmed with the help of a deterministic
> hypervisor currently under development.  The C reproducer below however
> hits it within about 2s.
>
> Fixes: f1cebdec9739 ("pipe: only enable the extra wake_up(rd_wait) for EPOLLET consumers")
> Cc: stable@vger.kernel.org
> Signed-off-by: Mani Tofigh <mt4022@columbia.edu>
> ---
> Reproducer: https://github.com/manitofigh/files/blob/main/linux-repros/pipeexcl.c
>
> There are a couple of older gaps like this (a single write, and the
> write side with EPOLLOUT|EPOLLEXCLUSIVE).  I left both alone here though.
> ---
>  fs/pipe.c                 | 6 +++++-
>  include/linux/pipe_fs_i.h | 2 +-
>  2 files changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/fs/pipe.c b/fs/pipe.c
> index 4c1122597a3c..8ad792d300c9 100644
> --- a/fs/pipe.c
> +++ b/fs/pipe.c
> @@ -700,9 +700,13 @@ pipe_poll(struct file *filp, poll_table *wait)
>  	 * and the ->pseudo_edgetrigger check in anon_pipe_write().
>  	 * Currently io_uring sets EPOLLET for multishot polls, so it gets the
>  	 * same behaviour.
> +	 *
> +	 * EPOLLEXCLUSIVE needs them too.  Its entry is an exclusive waiter on
> +	 * ->rd_wait that can take the wakeup meant for a blocked reader, and
> +	 * unlike a woken reader in anon_pipe_read() it doesn't pass it on.
>  	 */
>  	if ((filp->f_mode & FMODE_READ) &&
> -	    wait && (wait->_key & EPOLLET) &&
> +	    wait && (wait->_key & (EPOLLET | EPOLLEXCLUSIVE)) &&
>  	    unlikely(!READ_ONCE(pipe->pseudo_edgetrigger)))
>  		WRITE_ONCE(pipe->pseudo_edgetrigger, true);
>
> diff --git a/include/linux/pipe_fs_i.h b/include/linux/pipe_fs_i.h
> index 95f6236c5c5b..6b2e14ec6ef8 100644
> --- a/include/linux/pipe_fs_i.h
> +++ b/include/linux/pipe_fs_i.h
> @@ -90,7 +90,7 @@ struct anon_pipe_prealloc {
>   *	@files: number of struct file referring this pipe (protected by ->i_lock)
>   *	@r_counter: reader counter
>   *	@w_counter: writer counter
> - *	@pseudo_edgetrigger: has an EPOLLET consumer, enable per-write wakeups
> + *	@pseudo_edgetrigger: enable per-write wakeups for EPOLLET/EPOLLEXCLUSIVE
>   *	@fasync_readers: reader side fasync
>   *	@fasync_writers: writer side fasync
>   *	@bufs: the circular array of pipe buffers
>
> ---
> base-commit: b0993210e420f2f1d61e6719fbcd3966b96ff9ed
> change-id: 20261111-pipe-epollexclusive-wakeup-981528041b4d
>
> Best regards,
> --
> Mani Tofigh <mt4022@columbia.edu>
>
>


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers
  2026-11-17 13:27 ` Oleg Nesterov
@ 2026-11-17 16:22   ` Linus Torvalds
  2026-11-17 17:25     ` Oleg Nesterov
  0 siblings, 1 reply; 9+ messages in thread
From: Linus Torvalds @ 2026-11-17 16:22 UTC (permalink / raw)
  To: Oleg Nesterov
  Cc: Mani Tofigh via B4 Relay, Breno Leitao, Christian Brauner,
	Mateusz Guzik, Alexander Viro, Jan Kara, linux-fsdevel,
	linux-kernel, tz2294, stable, Mani Tofigh

On Tue, 17 Nov 2026 at 05:27, Oleg Nesterov <oleg@redhat.com> wrote:
>
> However, EPOLLEXCLUSIVE breaks this logic. ep_ptable_queue_proc() does
> add_wait_queue_exclusive() if EPOLLEXCLUSIVE is set.

Christ, how I hate epoll. It's a pile of stinking shit.

This literally violates THE WHOLE AND ONLY POINT of exclusive waits.

The rule for an exclusive wait is that it *will* act on the wakeup -
either by consuming the data, or - if it can't consume it all - by
waking up the next waiter itself.

So it's not that "EPOLLEXCLUSIVE breaks this logic".

The logic is fine. EPOLLEXCLUSIVE is broken beyond words.

I really think we need to just actively de-emphasize epoll. Starting
with saying "you can't add EPOLLEXCLUSIVE, because you don't honor
it".

               Linus

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers
  2026-11-17 16:22   ` Linus Torvalds
@ 2026-11-17 17:25     ` Oleg Nesterov
  2026-11-17 17:35       ` Linus Torvalds
  0 siblings, 1 reply; 9+ messages in thread
From: Oleg Nesterov @ 2026-11-17 17:25 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Mani Tofigh via B4 Relay, Breno Leitao, Christian Brauner,
	Mateusz Guzik, Alexander Viro, Jan Kara, linux-fsdevel,
	linux-kernel, tz2294, stable, Mani Tofigh

On 11/17, Linus Torvalds wrote:
>
> On Tue, 17 Nov 2026 at 05:27, Oleg Nesterov <oleg@redhat.com> wrote:
> >
> > However, EPOLLEXCLUSIVE breaks this logic. ep_ptable_queue_proc() does
> > add_wait_queue_exclusive() if EPOLLEXCLUSIVE is set.
>
> Christ, how I hate epoll. It's a pile of stinking shit.
>
> This literally violates THE WHOLE AND ONLY POINT of exclusive waits.
>
> The rule for an exclusive wait is that it *will* act on the wakeup -
> either by consuming the data, or - if it can't consume it all - by
> waking up the next waiter itself.
>
> So it's not that "EPOLLEXCLUSIVE breaks this logic".
>
> The logic is fine. EPOLLEXCLUSIVE is broken beyond words.

Well, I thought so too. That is why initially I replied "we don't care"
to Mani. But then I decided to summarize and add CC's.

So. Do you agree that we can "ignore" this report from Mani for now?
Until we find a user application that already depends on the behaviour
before f1cebdec9739 (the next write() wakes up a reader which sleeps on
->rd_wait for no reason).

Just in case... io_uring/poll.c:__io_queue_proc does the same:

	if (poll->events & EPOLLEXCLUSIVE) {
		add_wait_queue_exclusive(head, &poll->wait);
	} else {
		add_wait_queue(head, &poll->wait);
	}

Oleg.


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers
  2026-11-17 17:25     ` Oleg Nesterov
@ 2026-11-17 17:35       ` Linus Torvalds
  2026-11-17 17:58         ` Oleg Nesterov
  0 siblings, 1 reply; 9+ messages in thread
From: Linus Torvalds @ 2026-11-17 17:35 UTC (permalink / raw)
  To: Oleg Nesterov
  Cc: Mani Tofigh via B4 Relay, Breno Leitao, Christian Brauner,
	Mateusz Guzik, Alexander Viro, Jan Kara, linux-fsdevel,
	linux-kernel, tz2294, stable, Mani Tofigh

On Tue, 17 Nov 2026 at 09:25, Oleg Nesterov <oleg@redhat.com> wrote:
>
> So. Do you agree that we can "ignore" this report from Mani for now?

I think we can ignore it, but I also have a cunning plan.

Normally, exclusive wait events always go at the tail of the wait
queue. We do that mainly because we want to wake all non-exclusive
ones first, and because it's fair.

But pipes don't *have* non-exclusive waiters (ignoring the splice
case, which waits for space, and if there are readers or writes that
would take the space we don't care).

So what we could do is to just make the pipe code open-code its own
wait event logic, and add its waiters at the head.

Voilà - we never even care about epoll braindamage.

Yes, that makes pipes unfair in that new readers get preference over
old readers, and maybe that's not great. It might affect things like
the make jobserver. I'd expect it would probably improve throughput -
new waiters are hot waiters - but it's most definitely worth
mentioning.

It won't fix the "EPOLL edge triggered braindamage", where old
applications asked for edge triggered things and actually wanted level
triggers. But it should solve *this* issue.

Hmm?

Let me stew on this a bit.

           Linus

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers
  2026-11-17 17:35       ` Linus Torvalds
@ 2026-11-17 17:58         ` Oleg Nesterov
  0 siblings, 0 replies; 9+ messages in thread
From: Oleg Nesterov @ 2026-11-17 17:58 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Mani Tofigh via B4 Relay, Breno Leitao, Christian Brauner,
	Mateusz Guzik, Alexander Viro, Jan Kara, linux-fsdevel,
	linux-kernel, tz2294, stable, Mani Tofigh

On 11/17, Linus Torvalds wrote:
>
> On Tue, 17 Nov 2026 at 09:25, Oleg Nesterov <oleg@redhat.com> wrote:
> >
> > So. Do you agree that we can "ignore" this report from Mani for now?
>
> I think we can ignore it, but I also have a cunning plan.

Good. this was my main (the only) question.

> So what we could do is to just make the pipe code open-code its own
> wait event logic, and add its waiters at the head.

perhaps... and we already have WQ_FLAG_PRIORITY, but we can't re-use it
without more changes...

Oleg.


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2026-11-17 17:58 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-11 21:02 [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers Mani Tofigh via B4 Relay
2026-11-11 23:34 ` Oleg Nesterov
2026-11-12 10:26   ` Oleg Nesterov
2026-11-12 12:56 ` Oleg Nesterov
2026-11-17 13:27 ` Oleg Nesterov
2026-11-17 16:22   ` Linus Torvalds
2026-11-17 17:25     ` Oleg Nesterov
2026-11-17 17:35       ` Linus Torvalds
2026-11-17 17:58         ` Oleg Nesterov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®