From: Tejun Heo <tj@kernel.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>, linux-kernel@vger.kernel.org
Subject: Re: [GIT PULL] workqueue: Fixes for v7.4-rc2
Date: Mon, 16 Nov 2026 07:09:13 -1000 [thread overview]
Message-ID: <avs4uWg9ch-Bqobj@slm.duckdns.org> (raw)
In-Reply-To: <CAHk-=wg4jAWZvxWo0OZ+fFTU=SBtQ7dnsUCV7hx+UAifkwPUwQ@mail.gmail.com>
Hello, Linus.
On Sun, Nov 15, 2026 at 02:14:41PM -0800, Linus Torvalds wrote:
> On Sun, 15 Nov 2026 at 12:45, Tejun Heo <tj@kernel.org> wrote:
> >
> > - Draining a workqueue could return while a work item queued by another work
> > item was still pending, making workqueue destruction warn and leak the
> > workqueue.
>
> Why is this re-creating - badly - the existing sequence counter code?
I was trying to plug the hole minimally and it just didn't trigger "oh this
is seqcount".
> Not only does it do the memory barriers by hand - possibly getting
> them wrong, but it seems actively broken.
>
> There's a reason proper sequence counters have that "odd/even" rule,
> and a writer increments the sequence counter *twice* - once when it
> starts the modification and once when it is done.
>
> As far as I can tell, your "fix" is still broken:
>
> drain_workqueue:
>
> mutex_lock(&wq->mutex);
> if (!wq->nr_drainers++)
> wq->flags |= __WQ_DRAINING;
> mutex_unlock(&wq->mutex);
>
> seq = atomic_read(&wq->drain_seq);
>
>
> __queue_work:
>
> if (unlikely(wq->flags & (__WQ_DESTROYING | __WQ_DRAINING))) {
> ...
> smp_mb__before_atomic();
> atomic_inc(&wq->drain_seq);
> smp_mb__after_atomic();
>
> but what if __queue_work() was called before __WQ_DRAINING was set and
> is running concurrently and never incremented the sequence number?
>
> Of even if it *did* increment the sequence number, what keeps it from
> doing that *before* the atomic_read() in drain_workqueue() and then
>
> Either way drain_workqueue() never sees that drain_seq change, and
> thinks everythings is fine, even though __queue_work added an entry?
>
> Is there some other serialization that I'm missing?
Hmm... so, draining looks like this:
set DRAINING;
read seq;
flush wq:
for each pwq:
flush them;
for each pwq:
lock and check whether empty;
if not drained or seq bumped, try again;
and the queueing side tests DRAINING and incs the seq while the queueing
work item is still in flight on its pwq. The chained insertion is under the
pool lock.
Any work item that hasn't seen DRAINING or inc'd the seq before draining
reads it should be visible to the flush wq step, and by the time flush
finishes, all the immediate chained work items it queued should be visible
to the following pwq empty test step.
A chained work item queued after the flush started saw DRAINING and inc'd
the seq after the read, so if it chains further and finishes before the
empty test, the seq check catches it.
> Anyway, sequence numbers almost never work with a single write,
> exactly beause the single write is itself then not ordered with the
> changes that the writer does (ie the sequence point update happens
> before them or after them, but either way the reader might miss the
> sequence number update, becuase it's not atomic wrt the actual change
> it is supposed to be about.
>
> The actual 8real* sequence numbers we have do that two-step change
> exactly because of these issues. The writer increments once before it
> starts writing, and increments it again after it is done. That way the
> reader actually can check whether something changed (an odd value
> means that change is happening and needs retrying regardless - which
> is why read_seqcount_begin(() will loop on odd numbers)
>
> It is possible that I'm missing something that makes your ad-hoc
> version ok, but this really does look just plain broken.
>
> I pulled it and then unpulled it, waiting for explanations.
I don't think it's broken but it is adhoc and difficult to get one's head
around including my own. It'd be a lot simpler w/ seqcount. I'm traveling
this week, so might be slower but will redo it.
Thanks.
--
tejun
next prev parent reply other threads:[~2026-11-16 17:09 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-15 20:45 Tejun Heo
2026-11-15 22:14 ` Linus Torvalds
2026-11-16 17:09 ` Tejun Heo [this message]
2026-11-16 17:29 ` Linus Torvalds
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=avs4uWg9ch-Bqobj@slm.duckdns.org \
--to=tj@kernel.org \
--cc=jiangshanlai@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®