From: Seiji Nishikawa <snishika@redhat.com>
To: Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@kernel.dk>,
Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>
Cc: Hannes Reinecke <hare@suse.de>, Chris Leech <cleech@redhat.com>,
Seiji Nishikawa <snishika@redhat.com>,
linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org
Subject: [RFC PATCH 1/2] nvme-tcp: remove the direct send path
Date: Sun, 22 Nov 2026 10:18:25 +0900 [thread overview]
Message-ID: <20261122011826.1959697-2-snishika@redhat.com> (raw)
In-Reply-To: <20261122011826.1959697-1-snishika@redhat.com>
nvme_tcp_queue_request() sends the request directly from the submitting
context when it runs on the io_cpu of the queue, the queue is empty and
it can take send_mutex. Commit db5ad6b7f8cd ("nvme-tcp: try to send
request in queue_rq context") added this to skip the workqueue hop when
the queue is idle.
Since then this path has needed a series of fixes for races and
priority inversions between it and io_work. These are
commit 5c11f7d9f843 ("nvme-tcp: Fix possible race of io_work and direct
send"), commit a0fdd1418007 ("nvme-tcp: rerun io_work if req_list is
not empty"), commit 70f437fb4395 ("nvme-tcp: fix io_work priority
inversion"), commit 08acb54b063a ("nvme-tcp: defer TLS inline send to
io_work") and commit 5db358f3a7f9 ("nvme-tcp: fix io_work priority
inversion when send_mutex is held").
All of them come from having 2 contexts that send on the same queue and
coordinate through send_mutex, while io_work runs on a high priority
workqueue and the submitting task does not.
Remove the direct send path and always send from io_work. TLS queues
already work this way since 08acb54b063a. With a single sender, io_work
has to be queued whenever the last request of a batch is added, so the
has_pending check in nvme_tcp_queue_request() goes away too, and
nvme_tcp_send_all() has no caller left.
Suggested-by: Hannes Reinecke <hare@suse.de>
Assisted-by: LLM
Signed-off-by: Seiji Nishikawa <snishika@redhat.com>
---
drivers/nvme/host/tcp.c | 33 ++-------------------------------
1 file changed, 2 insertions(+), 31 deletions(-)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 62a594b8ab9b..2b832ac3068a 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -213,7 +213,6 @@ struct nvme_tcp_ctrl {
static struct workqueue_struct *nvme_tcp_wq;
static const struct blk_mq_ops nvme_tcp_mq_ops;
static const struct blk_mq_ops nvme_tcp_admin_mq_ops;
-static int nvme_tcp_try_send(struct nvme_tcp_queue *queue);
#ifdef CONFIG_DEBUG_LOCK_ALLOC
/* lockdep can detect a circular dependency of the form
@@ -420,16 +419,6 @@ static inline void nvme_tcp_advance_req(struct nvme_tcp_request *req,
}
}
-static inline void nvme_tcp_send_all(struct nvme_tcp_queue *queue)
-{
- int ret;
-
- /* drain the send queue as much as we can... */
- do {
- ret = nvme_tcp_try_send(queue);
- } while (ret > 0);
-}
-
static inline bool nvme_tcp_queue_has_pending(struct nvme_tcp_queue *queue)
{
return !list_empty(&queue->send_list) ||
@@ -446,28 +435,10 @@ static inline void nvme_tcp_queue_request(struct nvme_tcp_request *req,
bool last)
{
struct nvme_tcp_queue *queue = req->queue;
- bool empty;
- empty = llist_add(&req->lentry, &queue->req_list) &&
- list_empty(&queue->send_list) && !queue->request;
-
- /*
- * if we're the first on the send_list and we can try to send
- * directly, otherwise queue io_work. Also, only do that if we
- * are on the same cpu, so we don't introduce contention.
- *
- * TLS kTLS send takes ctx->tx_lock while blk_mq holds set->srcu.
- * lockdep reports circular locking via elevator_lock. Defer TLS
- * sends to the io workqueue instead of inline from this path.
- */
- if (queue->io_cpu == raw_smp_processor_id() &&
- !nvme_tcp_queue_tls(queue) &&
- empty && mutex_trylock(&queue->send_mutex)) {
- nvme_tcp_send_all(queue);
- mutex_unlock(&queue->send_mutex);
- }
+ llist_add(&req->lentry, &queue->req_list);
- if (last && nvme_tcp_queue_has_pending(queue))
+ if (last)
queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue->io_work);
}
--
2.56.1
next prev parent reply other threads:[~2026-11-22 1:19 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-22 1:18 [RFC PATCH 0/2] " Seiji Nishikawa
2026-11-22 1:18 ` Seiji Nishikawa [this message]
2026-11-22 1:18 ` [RFC PATCH 2/2] nvme-tcp: remove send_mutex Seiji Nishikawa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261122011826.1959697-2-snishika@redhat.com \
--to=snishika@redhat.com \
--cc=axboe@kernel.dk \
--cc=cleech@redhat.com \
--cc=hare@suse.de \
--cc=hch@lst.de \
--cc=kbusch@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®