* [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect
@ 2026-09-22 20:07 Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 1/3] nvme-tcp: store and use the caller's network namespace Surabhi Gogte
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Surabhi Gogte @ 2026-09-22 20:07 UTC (permalink / raw)
To: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, linux-kernel, mkhalfella, randyj, adailey, Surabhi Gogte
Similar to commit 2a8513091d2f ("nvme-rdma: parallelize I/O queue
allocation and startup"), I/O queue allocation and start steps
(including authentication) can be parallelized for nvme-tcp as well.
Patch 1 fixes a pre-existing bug: nvme_tcp_alloc_queue() used
current->nsproxy->net_ns to create sockets during initial connect
while it used kernel's init_net for reconnect and error recovery
paths. The fix captures the caller's network namespace at controller
allocation time.
Patch 2 is a preparatory refactor: it splits the queue-count
negotiation and TLS PSK checks into helpers so that the individual
steps can be called directly from nvme_tcp_configure_io_queues(). No
functional change.
Patch 3 has the async implementation: it fans the per-queue alloc and
start out over an async domain, serializes the per-cpu queue-count in
nvme_tcp_set_queue_io_cpu() that can now run concurrently, and uses
ctrl->net for all socket creation.
Testing on a 64-core host with 64 IO-queues shows nvme-tcp connection
time reduced from 61ms to 11ms.
blktests - nvme suite, TCP transport (nvme_trtype=tcp)
Tested with:
- Ubuntu 22.04 LTS (Jammy Jellyfish), x86_64, KubeVirt VM
- nvme-cli v2.13 / libnvme v1.14 (built with libkeyutils)
- ktls-utils tag ktls-utils-1.4.0 (c63c04b) (tlshd) linked
against gnutls 3.8.5
---
v4:
- Moved conditional from nvme_tcp_configure_io_queues() to
nvme_tcp_setup_io_queues().
v3:
- Added patch 1 fixing a network namespace bug: reconnect and error
recovery paths were creating sockets in init_net instead of the
controller's namespace. Patch 3 now uses ctrl->net (from patch 1)
instead of hardcoding init_net.
- Patches renumbered accordingly (v2 patch 1->2, patch 2->3).
v2:
- Resending the entire series to fix missing numbering in the
original patch 2 subject line. No functional code changes.
v1: https://lore.kernel.org/20260824222939.301887-1-sgogte@purestorage.com/
v2: https://lore.kernel.org/20260824225648.316962-1-sgogte@purestorage.com/
v3: https://lore.kernel.org/20260910202812.1642832-1-sgogte@purestorage.com/
---
Surabhi Gogte (3):
nvme-tcp: store and use the caller's network namespace
nvme-tcp: refactor I/O queue setup path
nvme-tcp: parallelize I/O queue allocation and startup
drivers/nvme/host/tcp.c | 139 +++++++++++++++++++++++++++-------------
1 file changed, 96 insertions(+), 43 deletions(-)
--
2.55.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v4 1/3] nvme-tcp: store and use the caller's network namespace
2026-09-22 20:07 [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Surabhi Gogte
@ 2026-09-22 20:07 ` Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 2/3] nvme-tcp: refactor I/O queue setup path Surabhi Gogte
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Surabhi Gogte @ 2026-09-22 20:07 UTC (permalink / raw)
To: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, linux-kernel, mkhalfella, randyj, adailey,
Surabhi Gogte, Hannes Reinecke
nvme_tcp_alloc_queue() creates each socket using
current->nsproxy->net_ns, which is correct for the initial connect that
runs in userspace context. However, the reconnect and error recovery
paths run from a workqueue where current netns is a kernel thread in
init_net, so sockets created during reconnect end up in a different
namespace than the one the controller was originally connected from.
Fix this by capturing the caller's network namespace at controller
allocation time. Use ctrl->net for all socket creation so that every
queue, on initial connect, reset, or reconnect, is created in the same
namespace.
Fixes: 3f2304f8c6d6 ("nvme-tcp: add NVMe over TCP host driver")
Signed-off-by: Surabhi Gogte <sgogte@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/tcp.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 921934028e0b..f916bcca304f 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -171,6 +171,7 @@ struct nvme_tcp_ctrl {
struct delayed_work connect_work;
struct nvme_tcp_request async_req;
u32 io_queues[HCTX_MAX_TYPES];
+ struct net *net;
};
static struct workqueue_struct *nvme_tcp_wq;
@@ -1858,7 +1859,7 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
queue->cmnd_capsule_len = sizeof(struct nvme_command) +
NVME_TCP_ADMIN_CCSZ;
- ret = sock_create_kern(current->nsproxy->net_ns,
+ ret = sock_create_kern(ctrl->net,
ctrl->addr.ss_family, SOCK_STREAM,
IPPROTO_TCP, &queue->sock);
if (ret) {
@@ -2650,6 +2651,7 @@ static void nvme_tcp_free_ctrl(struct nvme_ctrl *nctrl)
nvmf_free_options(nctrl->opts);
free_ctrl:
+ put_net(ctrl->net);
kfree(ctrl->queues);
kfree(ctrl);
}
@@ -3048,12 +3050,15 @@ static struct nvme_tcp_ctrl *nvme_tcp_alloc_ctrl(struct device *dev,
goto out_free_ctrl;
}
+ ctrl->net = get_net(current->nsproxy->net_ns);
+
ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_tcp_ctrl_ops, 0);
if (ret)
goto out_kfree_queues;
return ctrl;
out_kfree_queues:
+ put_net(ctrl->net);
kfree(ctrl->queues);
out_free_ctrl:
kfree(ctrl);
--
2.55.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v4 2/3] nvme-tcp: refactor I/O queue setup path
2026-09-22 20:07 [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 1/3] nvme-tcp: store and use the caller's network namespace Surabhi Gogte
@ 2026-09-22 20:07 ` Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 3/3] nvme-tcp: parallelize I/O queue allocation and startup Surabhi Gogte
2026-10-13 15:33 ` [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Keith Busch
3 siblings, 0 replies; 6+ messages in thread
From: Surabhi Gogte @ 2026-09-22 20:07 UTC (permalink / raw)
To: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, linux-kernel, mkhalfella, randyj, adailey,
Surabhi Gogte, Hannes Reinecke
Split the I/O queue setup helpers apart so that the individual steps can
be called directly from nvme_tcp_configure_io_queues():
- Queue count negotiation moves out of nvme_tcp_alloc_io_queues() into a
new nvme_tcp_set_io_queue_count(), leaving the allocator with just the
per-queue allocation loop.
- TLS PSK validation moves out of __nvme_tcp_alloc_io_queues() into a
new nvme_tcp_tls_check_psk().
- nvme_tcp_configure_io_queues() now calls the three steps directly in
the same order as before.
Signed-off-by: Surabhi Gogte <sgogte@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/tcp.c | 25 +++++++++++++++++++------
1 file changed, 19 insertions(+), 6 deletions(-)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index f916bcca304f..9308feba8a74 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2184,10 +2184,8 @@ static int nvme_tcp_alloc_admin_queue(struct nvme_ctrl *ctrl)
return ret;
}
-static int __nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
+static int nvme_tcp_tls_check_psk(struct nvme_ctrl *ctrl)
{
- int i, ret;
-
if (nvme_tcp_tls_configured(ctrl)) {
if (ctrl->opts->concat) {
/*
@@ -2209,6 +2207,13 @@ static int __nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
}
}
+ return 0;
+}
+
+static int __nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
+{
+ int i, ret;
+
for (i = 1; i < ctrl->queue_count; i++) {
ret = nvme_tcp_alloc_queue(ctrl, i,
ctrl->tls_pskid);
@@ -2225,7 +2230,7 @@ static int __nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
return ret;
}
-static int nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
+static int nvme_tcp_set_io_queue_count(struct nvme_ctrl *ctrl)
{
unsigned int nr_io_queues;
int ret;
@@ -2247,14 +2252,22 @@ static int nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
nvmf_set_io_queues(ctrl->opts, nr_io_queues,
to_tcp_ctrl(ctrl)->io_queues);
- return __nvme_tcp_alloc_io_queues(ctrl);
+ return 0;
}
static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
{
int ret, nr_queues;
- ret = nvme_tcp_alloc_io_queues(ctrl);
+ ret = nvme_tcp_set_io_queue_count(ctrl);
+ if (ret)
+ return ret;
+
+ ret = nvme_tcp_tls_check_psk(ctrl);
+ if (ret)
+ return ret;
+
+ ret = __nvme_tcp_alloc_io_queues(ctrl);
if (ret)
return ret;
--
2.55.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v4 3/3] nvme-tcp: parallelize I/O queue allocation and startup
2026-09-22 20:07 [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 1/3] nvme-tcp: store and use the caller's network namespace Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 2/3] nvme-tcp: refactor I/O queue setup path Surabhi Gogte
@ 2026-09-22 20:07 ` Surabhi Gogte
2026-10-18 12:25 ` Shin'ichiro Kawasaki
2026-10-13 15:33 ` [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Keith Busch
3 siblings, 1 reply; 6+ messages in thread
From: Surabhi Gogte @ 2026-09-22 20:07 UTC (permalink / raw)
To: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, linux-kernel, mkhalfella, randyj, adailey,
Surabhi Gogte, Hannes Reinecke
Similar to commit 2a8513091d2f ("nvme-rdma: parallelize I/O queue
allocation and startup"), refactor nvme tcp I/O queue setup to use async
API, combining allocation and startup into a single parallel operation
per queue. This reduces connection and reconnection setup time when
there are delays in establishing connections, which is especially
important for high-core-count hosts.
Key changes:
- Use async API to facilitate parallel calls for io queue setup.
- Add nvme_tcp_queue_setup_ctx for propagating errors from async
workers.
- Remove nvme_tcp_start_io_queues() and __nvme_tcp_alloc_io_queues();
their logic is folded into nvme_tcp_setup_io_queues() and
nvme_tcp_configure_io_queues().
- Allocate the io tag set before the queues so that the queue range is
known, and only set up the reconnect grow case if the queue count
actually increased.
- Serialize the cpu scan and claim in nvme_tcp_set_queue_io_cpu() with a
spinlock, as concurrent callers would otherwise select the same cpu.
The per-cpu counters no longer need to be atomics.
Testing on a 64-core host with 64 IO-queues shows nvme-tcp connection
time reduced from 61ms to 11ms.
Signed-off-by: Surabhi Gogte <sgogte@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
---
drivers/nvme/host/tcp.c | 119 ++++++++++++++++++++++++++--------------
1 file changed, 77 insertions(+), 42 deletions(-)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 9308feba8a74..ec678ed07af8 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -7,6 +7,7 @@
#include <linux/module.h>
#include <linux/init.h>
#include <linux/slab.h>
+#include <linux/async.h>
#include <linux/err.h>
#include <linux/crc32.h>
#include <linux/nvme-tcp.h>
@@ -54,7 +55,8 @@ MODULE_PARM_DESC(tls_handshake_timeout,
"nvme TLS handshake timeout in seconds (default 10)");
#endif
-static atomic_t nvme_tcp_cpu_queues[NR_CPUS];
+static int nvme_tcp_cpu_queues[NR_CPUS];
+static DEFINE_SPINLOCK(nvme_tcp_cpu_queues_lock);
enum nvme_tcp_send_state {
NVME_TCP_SEND_CMD_PDU = 0,
@@ -154,6 +156,12 @@ struct nvme_tcp_queue {
static DEFINE_MUTEX(nvme_tcp_ctrl_mutex);
static LIST_HEAD_GUARDED(nvme_tcp_ctrl_list, nvme_tcp_ctrl_mutex);
+struct nvme_tcp_queue_setup_ctx {
+ struct nvme_ctrl *ctrl;
+ int qid;
+ int *err;
+};
+
struct nvme_tcp_ctrl {
/* read only in the hot path */
struct nvme_tcp_queue *queues;
@@ -1731,9 +1739,10 @@ static void nvme_tcp_set_queue_io_cpu(struct nvme_tcp_queue *queue)
goto out;
/* Search for the least used cpu from the mq_map */
+ spin_lock(&nvme_tcp_cpu_queues_lock);
io_cpu = WORK_CPU_UNBOUND;
for_each_online_cpu(cpu) {
- int num_queues = atomic_read(&nvme_tcp_cpu_queues[cpu]);
+ int num_queues = nvme_tcp_cpu_queues[cpu];
if (mq_map[cpu] != qid)
continue;
@@ -1744,9 +1753,10 @@ static void nvme_tcp_set_queue_io_cpu(struct nvme_tcp_queue *queue)
}
if (io_cpu != WORK_CPU_UNBOUND) {
queue->io_cpu = io_cpu;
- atomic_inc(&nvme_tcp_cpu_queues[io_cpu]);
+ nvme_tcp_cpu_queues[io_cpu]++;
set_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags);
}
+ spin_unlock(&nvme_tcp_cpu_queues_lock);
out:
dev_dbg(ctrl->ctrl.device, "queue %d: using cpu %d\n",
qid, queue->io_cpu);
@@ -2023,8 +2033,11 @@ static void nvme_tcp_stop_queue_nowait(struct nvme_ctrl *nctrl, int qid)
if (!test_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
return;
- if (test_and_clear_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags))
- atomic_dec(&nvme_tcp_cpu_queues[queue->io_cpu]);
+ if (test_and_clear_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags)) {
+ spin_lock(&nvme_tcp_cpu_queues_lock);
+ nvme_tcp_cpu_queues[queue->io_cpu]--;
+ spin_unlock(&nvme_tcp_cpu_queues_lock);
+ }
mutex_lock(&queue->queue_lock);
if (test_and_clear_bit(NVME_TCP_Q_LIVE, &queue->flags))
@@ -2131,25 +2144,6 @@ static void nvme_tcp_stop_io_queues(struct nvme_ctrl *ctrl)
nvme_tcp_wait_queue(ctrl, i);
}
-static int nvme_tcp_start_io_queues(struct nvme_ctrl *ctrl,
- int first, int last)
-{
- int i, ret;
-
- for (i = first; i < last; i++) {
- ret = nvme_tcp_start_queue(ctrl, i);
- if (ret)
- goto out_stop_queues;
- }
-
- return 0;
-
-out_stop_queues:
- for (i--; i >= first; i--)
- nvme_tcp_stop_queue(ctrl, i);
- return ret;
-}
-
static int nvme_tcp_alloc_admin_queue(struct nvme_ctrl *ctrl)
{
int ret;
@@ -2210,22 +2204,67 @@ static int nvme_tcp_tls_check_psk(struct nvme_ctrl *ctrl)
return 0;
}
-static int __nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
+static void nvme_tcp_setup_queue_async(void *data, async_cookie_t cookie)
{
- int i, ret;
+ struct nvme_tcp_queue_setup_ctx *ctx = data;
+ struct nvme_ctrl *ctrl = ctx->ctrl;
+ int ret;
- for (i = 1; i < ctrl->queue_count; i++) {
- ret = nvme_tcp_alloc_queue(ctrl, i,
- ctrl->tls_pskid);
- if (ret)
- goto out_free_queues;
+ ret = nvme_tcp_alloc_queue(ctrl, ctx->qid, ctrl->tls_pskid);
+ if (ret)
+ goto out_err;
+
+ ret = nvme_tcp_start_queue(ctrl, ctx->qid);
+ if (ret)
+ goto out_err;
+
+ return;
+
+out_err:
+ WRITE_ONCE(*ctx->err, ret);
+}
+
+static int nvme_tcp_setup_io_queues(struct nvme_ctrl *ctrl, unsigned int first,
+ unsigned int last)
+{
+ ASYNC_DOMAIN_EXCLUSIVE(queue_domain);
+ struct nvme_tcp_queue_setup_ctx *ctxs;
+ int nr_queues = last - first;
+ int err = 0, i, ret;
+
+ if (first >= last)
+ return 0;
+
+ ctxs = kmalloc_objs(*ctxs, nr_queues);
+ if (!ctxs)
+ return -ENOMEM;
+
+ for (i = 0; i < nr_queues; i++) {
+ ctxs[i].ctrl = ctrl;
+ ctxs[i].qid = first + i;
+ ctxs[i].err = &err;
+ async_schedule_domain(nvme_tcp_setup_queue_async, &ctxs[i],
+ &queue_domain);
}
+ async_synchronize_full_domain(&queue_domain);
+ kfree(ctxs);
+
+ ret = READ_ONCE(err);
+ if (ret)
+ goto out_free_queues;
+
return 0;
out_free_queues:
- for (i--; i >= 1; i--)
- nvme_tcp_free_queue(ctrl, i);
+ for (i = first; i < last; i++) {
+ struct nvme_tcp_queue *queue = &to_tcp_ctrl(ctrl)->queues[i];
+
+ if (test_bit(NVME_TCP_Q_LIVE, &queue->flags))
+ nvme_tcp_stop_queue(ctrl, i);
+ if (test_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
+ nvme_tcp_free_queue(ctrl, i);
+ }
return ret;
}
@@ -2267,10 +2306,6 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
if (ret)
return ret;
- ret = __nvme_tcp_alloc_io_queues(ctrl);
- if (ret)
- return ret;
-
if (new) {
ret = nvme_alloc_io_tag_set(ctrl, &to_tcp_ctrl(ctrl)->tag_set,
&nvme_tcp_mq_ops,
@@ -2281,12 +2316,12 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
}
/*
- * Only start IO queues for which we have allocated the tagset
+ * Only setup IO queues for which we have allocated the tagset
* and limited it to the available queues. On reconnects, the
* queue number might have changed.
*/
nr_queues = min(ctrl->tagset->nr_hw_queues + 1, ctrl->queue_count);
- ret = nvme_tcp_start_io_queues(ctrl, 1, nr_queues);
+ ret = nvme_tcp_setup_io_queues(ctrl, 1, nr_queues);
if (ret)
goto out_cleanup_connect_q;
@@ -2310,9 +2345,9 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
/*
* If the number of queues has increased (reconnect case)
- * start all new queues now.
+ * setup all new queues now.
*/
- ret = nvme_tcp_start_io_queues(ctrl, nr_queues,
+ ret = nvme_tcp_setup_io_queues(ctrl, nr_queues,
ctrl->tagset->nr_hw_queues + 1);
if (ret)
goto out_wait_freeze_timed_out;
@@ -3157,7 +3192,7 @@ static int __init nvme_tcp_init_module(void)
return -ENOMEM;
for_each_possible_cpu(cpu)
- atomic_set(&nvme_tcp_cpu_queues[cpu], 0);
+ nvme_tcp_cpu_queues[cpu] = 0;
nvmf_register_transport(&nvme_tcp_transport);
return 0;
--
2.55.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect
2026-09-22 20:07 [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Surabhi Gogte
` (2 preceding siblings ...)
2026-09-22 20:07 ` [PATCH v4 3/3] nvme-tcp: parallelize I/O queue allocation and startup Surabhi Gogte
@ 2026-10-13 15:33 ` Keith Busch
3 siblings, 0 replies; 6+ messages in thread
From: Keith Busch @ 2026-10-13 15:33 UTC (permalink / raw)
To: Surabhi Gogte
Cc: Jens Axboe, Christoph Hellwig, Sagi Grimberg, linux-nvme,
linux-kernel, mkhalfella, randyj, adailey
On Tue, Sep 22, 2026 at 02:07:15PM -0600, Surabhi Gogte wrote:
> Similar to commit 2a8513091d2f ("nvme-rdma: parallelize I/O queue
> allocation and startup"), I/O queue allocation and start steps
> (including authentication) can be parallelized for nvme-tcp as well.
>
> Patch 1 fixes a pre-existing bug: nvme_tcp_alloc_queue() used
> current->nsproxy->net_ns to create sockets during initial connect
> while it used kernel's init_net for reconnect and error recovery
> paths. The fix captures the caller's network namespace at controller
> allocation time.
>
> Patch 2 is a preparatory refactor: it splits the queue-count
> negotiation and TLS PSK checks into helpers so that the individual
> steps can be called directly from nvme_tcp_configure_io_queues(). No
> functional change.
>
> Patch 3 has the async implementation: it fans the per-queue alloc and
> start out over an async domain, serializes the per-cpu queue-count in
> nvme_tcp_set_queue_io_cpu() that can now run concurrently, and uses
> ctrl->net for all socket creation.
>
> Testing on a 64-core host with 64 IO-queues shows nvme-tcp connection
> time reduced from 61ms to 11ms.
Thanks, applied to nvme-7.4.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v4 3/3] nvme-tcp: parallelize I/O queue allocation and startup
2026-09-22 20:07 ` [PATCH v4 3/3] nvme-tcp: parallelize I/O queue allocation and startup Surabhi Gogte
@ 2026-10-18 12:25 ` Shin'ichiro Kawasaki
0 siblings, 0 replies; 6+ messages in thread
From: Shin'ichiro Kawasaki @ 2026-10-18 12:25 UTC (permalink / raw)
To: Surabhi Gogte
Cc: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg,
linux-nvme, linux-kernel, mkhalfella, randyj, adailey,
Hannes Reinecke
On Sep 22, 2026 / 14:07, Surabhi Gogte wrote:
> Similar to commit 2a8513091d2f ("nvme-rdma: parallelize I/O queue
> allocation and startup"), refactor nvme tcp I/O queue setup to use async
> API, combining allocation and startup into a single parallel operation
> per queue. This reduces connection and reconnection setup time when
> there are delays in establishing connections, which is especially
> important for high-core-count hosts.
>
> Key changes:
> - Use async API to facilitate parallel calls for io queue setup.
> - Add nvme_tcp_queue_setup_ctx for propagating errors from async
> workers.
> - Remove nvme_tcp_start_io_queues() and __nvme_tcp_alloc_io_queues();
> their logic is folded into nvme_tcp_setup_io_queues() and
> nvme_tcp_configure_io_queues().
> - Allocate the io tag set before the queues so that the queue range is
> known, and only set up the reconnect grow case if the queue count
> actually increased.
> - Serialize the cpu scan and claim in nvme_tcp_set_queue_io_cpu() with a
> spinlock, as concurrent callers would otherwise select the same cpu.
> The per-cpu counters no longer need to be atomics.
>
> Testing on a 64-core host with 64 IO-queues shows nvme-tcp connection
> time reduced from 61ms to 11ms.
Recently, blktests-CI runs for the kernel at block/for-next branch started
reporting failures of the test case nvme/004 (with tcp transport) and md/001.
The kernel reported a WARN related to lock->magic, and it caused the failures.
FYI, here I share the failure run log at md/001 [1] and dmesg at the failure.
The md/001 confirms a bug fix in md driver. The bug was triggered when a
nvme device with tcp transport is used as its backend. Then the test case
exercises the nvme-tcp driver, which this patch touches.
I bisected and found that this commit is the trigger of the failures. When I
revert this commit from the block/for-next branch tip with git hash
34fbfb9d8d1f, the failure goes away.
Surabhi, could you take a look in the failure?
[1] failure of md/001
md/001 (Raid with bitmap on tcp nvmet with opt-io-size over bitmap size) [failed]
runtime 3.021s ... 2.970s
something found in dmesg:
[ 79.786635] [ T966] run blktests md/001 at 2026-10-18 20:35:29.594227680
[ 80.157789] [ T984] brd: module loaded
[ 80.566933] [ T1008] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[ 80.637888] [ T1011] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[ 80.925883] [ T312] nvmet: Created nvm controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349.
[ 80.940866] [ T1018] nvme nvme5: creating 4 I/O queues.
[ 80.942243] [ T1018] nvme nvme5: mapped 4/0/0 default/read/poll queues.
[ 80.942689] [ T1018] ------------[ cut here ]------------
[ 80.942879] [ T1018] DEBUG_LOCKS_WARN_ON(lock->magic != lock)
[ 80.942882] [ T1018] WARNING: kernel/locking/mutex.c:625 at __mutex_lock+0x1008/0x2510, CPU#0: nvme/1018
...
(See '/home/shin/Blktests/blktests/results/nodev/md/001.dmesg' for the entire message)
[2] dmesg during md/001
2026-10-17T22:40:06Z {detected_level="unknown"} [ 2568.828164] run blktests md/001 at 2026-10-17 22:40:06.674885632
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.162185] brd: module loaded
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.587993] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.629198] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.848587] nvmet: Created nvm controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349.
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.875225] nvme nvme2: creating 4 I/O queues.
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.878288] nvme nvme2: mapped 4/0/0 default/read/poll queues.
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.883211] ------------[ cut here ]------------
2026-10-17T22:40:07Z {detected_level="debug"} [ 2569.883660] DEBUG_LOCKS_WARN_ON(lock->magic != lock)
2026-10-17T22:40:07Z {detected_level="warn"} [ 2569.883666] WARNING: kernel/locking/mutex.c:625 at __mutex_lock+0xb73/0x2350, CPU#2: nvme/55686
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.887533] Modules linked in: nvme_tcp(E) nvme_fabrics(E) nvmet_tcp(E) nvmet(E) brd(E) raid1(E) dm_dust(E) loop(E) xfs(E) tun(E) nvme(E) iTCO_wdt(E) intel_rapl_msr(E) intel_pmc_bxt(E) intel_rapl_common(E) pcspkr(E) i2c_i801(E) i2c_smbus(E) nvme_core(E) nvme_keyring(E) virtio_balloon(E) nvme_auth(E) virtio_scsi(E) lpc_ich(E) bochs(E) serio_raw(E) qemu_fw_cfg(E) vmw_vsock_virtio_transport(E) vmw_vsock_virtio_transport_common(E) vsock(E)
2026-10-17T22:40:07Z {detected_level="debug"} [ 2569.887693] Unloaded tainted modules: nvme_tcp(E):1 nvmet(E):63 nvme_fabrics(E):63 nvme_loop(E):63 loop(E):14 ublk_drv(E):2 null_blk(E):37 scsi_debug(E):53 amd_atl(E):1 edac_mce_amd(E):1 bpf_preload(E):1 [last unloaded: scsi_debug(E)]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.899624] CPU: 2 UID: 0 PID: 55686 Comm: nvme Tainted: G E 7.3.0-rc7 #1 PREEMPT(lazy)
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.902229] Tainted: [E]=UNSIGNED_MODULE
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.903507] Hardware name: KubeVirt None/RHEL, BIOS 1.16.3-4.el9 04/01/2014
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.905532] RIP: 0010:__mutex_lock+0xb7a/0x2350
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.907336] Code: 38 d0 7c 08 84 d2 0f 85 52 17 00 00 44 8b 05 8d af 8d 02 45 85 c0 0f 85 0a f6 ff ff 48 8d 3d ad cc 8e 02 48 c7 c6 c0 fd 2f 85 <67> 48 0f b9 3a e9 f2 f5 ff ff 4d 8d a7 e0 0e 00 00 4c 89 e7 e8 3d
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.912320] RSP: 0018:ffff8881438c7370 EFLAGS: 00010246
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.914185] RAX: 0000000000000003 RBX: ffff8881face6460 RCX: 1ffffffff0f35008
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.916368] RDX: 0000000000000000 RSI: ffffffff852ffdc0 RDI: ffffffff879b9d70
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.918675] RBP: ffff8881438c74e0 R08: 0000000000000000 R09: fffffbfff0f35008
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.920803] R10: ffffffff879a8047 R11: ffff88812d120008 R12: 0000000000000000
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.923039] R13: 0000000000000000 R14: dffffc0000000000 R15: ffff88812d3a0000
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.925258] FS: 00007fd431b6b640(0000) GS:ffff8882ac2ea000(0000) knlGS:0000000000000000
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.927640] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.929481] CR2: 00007f50088d0c1c CR3: 000000010d0d4006 CR4: 0000000000770ef0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.932053] PKRU: 55555554
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.933507] Call Trace:
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.934813] <TASK>
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.936625] ? nvme_tcp_init_request+0xc8/0x300 [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.938805] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.940675] ? __alloc_frozen_pages_noprof+0x543/0x7c0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.942464] ? __pfx___mutex_lock+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.944123] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.945814] ? alloc_pages_node_noprof+0xae/0x150
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.947474] ? nvme_tcp_init_request+0xc8/0x300 [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.949268] nvme_tcp_init_request+0xc8/0x300 [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.951141] blk_mq_alloc_rqs+0x3fc/0x6f0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.952747] blk_mq_alloc_map_and_rqs+0x168/0x320
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.954450] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.956260] blk_mq_alloc_set_map_and_rqs+0x1be/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.958088] blk_mq_alloc_tag_set+0x59a/0xc40
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.959710] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.961321] ? dev_printk_emit+0xbe/0xf3
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.962736] nvme_alloc_io_tag_set+0x37d/0x6a0 [nvme_core]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.964446] ? __pfx_nvme_alloc_io_tag_set+0x10/0x10 [nvme_core]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.966345] ? percpu_ref_put_many.constprop.0+0x7f/0x1b0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.968109] ? __nvme_submit_sync_cmd+0x1e2/0x320 [nvme_core]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.970040] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.971714] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="info"} [ 2569.973339] ? _dev_info+0xe6/0x11a
2026-10-17T22:40:07Z {detected_level="info"} [ 2569.974760] ? __pfx__dev_info+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.976153] ? __pfx_nvme_set_queue_count+0x10/0x10 [nvme_core]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.977938] ? lock_acquire.part.0+0xc7/0x240
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.979582] nvme_tcp_configure_io_queues.cold+0x27f/0x8ee [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.981436] ? __pfx_nvme_enable_ctrl+0x10/0x10 [nvme_core]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.983132] ? lockdep_hardirqs_on_prepare.part.0+0x9b/0x140
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.984762] ? __pfx_nvme_tcp_configure_io_queues+0x10/0x10 [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.986525] nvme_tcp_setup_ctrl+0x281/0x5f0 [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.988113] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.989665] nvme_tcp_create_ctrl+0xf6/0x120 [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.991236] nvmf_create_ctrl+0x34e/0x920 [nvme_fabrics]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.992832] nvmf_dev_write+0xd5/0x180 [nvme_fabrics]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.994357] vfs_write+0x1c7/0x1060
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.995622] ? __pfx_vfs_write+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.996900] ksys_write+0xf9/0x1d0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.998093] ? __pfx_ksys_write+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.999349] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.000692] ? lockdep_hardirqs_on_prepare.part.0+0x9b/0x140
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.002249] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.003675] ? trace_hardirqs_on+0x18/0x160
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.005046] do_syscall_64+0xef/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.006137] ? kmem_cache_free+0x143/0x600
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.007405] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.008722] ? do_sys_openat2+0x107/0x180
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.009946] ? __pfx_do_sys_openat2+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.011180] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.012539] ? fput_close_sync+0xe5/0x1a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.013752] ? __pfx_fput_close_sync+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.015062] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.016455] ? __x64_sys_openat+0x108/0x1d0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.017722] ? __pfx___x64_sys_openat+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.019017] ? trace_hardirqs_on_prepare+0x119/0x150
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.020393] ? do_syscall_64+0x1d7/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.021623] ? do_syscall_64+0x1b9/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.022876] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.024209] ? trace_hardirqs_on_prepare+0x119/0x150
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.025581] ? do_syscall_64+0x1b9/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.026867] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.028225] ? trace_hardirqs_on_prepare+0x119/0x150
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.029718] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.031169] ? do_syscall_64+0x1d7/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.032408] ? do_syscall_64+0x1b9/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.033704] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.035099] ? trace_hardirqs_on_prepare+0x119/0x150
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.036605] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.037956] ? do_syscall_64+0x1d7/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.039181] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.040575] ? __lock_release.isra.0+0x5b/0x180
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.041947] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.043389] ? do_user_addr_fault+0x8d3/0xe80
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.044575] ? irqentry_exit+0xb8/0x810
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.045690] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.046990] ? do_syscall_64+0x2d/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.048067] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.049302] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.050545] ? do_syscall_64+0x5c/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.051649] entry_SYSCALL_64_after_hwframe+0x76/0x7e
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.052916] RIP: 0033:0x7fd431c3d56e
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.053948] Code: 4d 89 d8 e8 d4 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 <c9> c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.057937] RSP: 002b:00007ffdaa469710 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.059733] RAX: ffffffffffffffda RBX: 00005609fa418740 RCX: 00007fd431c3d56e
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.061425] RDX: 00000000000000cf RSI: 00005609fa418740 RDI: 0000000000000003
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.063121] RBP: 00007ffdaa469720 R08: 0000000000000000 R09: 0000000000000000
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.064826] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000003
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.066523] R13: 00005609fa4175b0 R14: 00000000000000cf R15: 00005609fa4175b0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.068235] </TASK>
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.069130] irq event stamp: 11409
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.070225] hardirqs last enabled at (11409): [<ffffffff850e58f5>] _raw_spin_unlock_irqrestore+0x45/0x60
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.072350] hardirqs last disabled at (11408): [<ffffffff817db728>] __raw_spin_lock_irqsave+0x58/0x60
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.074391] softirqs last enabled at (11372): [<ffffffff815ffc88>] handle_softirqs+0x638/0x880
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.076357] softirqs last disabled at (11365): [<ffffffff8160003f>] __irq_exit_rcu+0x15f/0x290
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.078305] ---[ end trace 0000000000000000 ]---
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.084017] nvme nvme2: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420, hostnqn: nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.400433] md127: array will not be assembled in old kernels that lack configurable LBS support (<= 6.18)
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.401014] md/raid1:md127: active with 1 out of 2 mirrors
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.404015] md127: detected capacity change from 0 to 2093056
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.559964] md127: detected capacity change from 2093056 to 0
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.560710] md: md127 stopped.
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.632174] nvme nvme2: Removing ctrl: NQN "blktests-subsystem-1"
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2571.118121] brd: module unloaded
2026-10-17T22:40:09Z {detected_level="debug"} [ 2572.039779] scsi_debug:sdebug_driver_probe: scsi_debug: trim poll_queues to 0. poll_q/nr_hw = (0/1)
2026-10-17T22:40:09Z {detected_level="debug"} [ 2572.040354] scsi host66: scsi_debug: version 0191 [20210520]
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.040354] dev_size_mb=8, opts=0x0, submit_queues=1, statistics=0
2026-10-17T22:40:09Z {detected_level="debug"} [ 2572.046424] scsi 66:0:0:0: Direct-Access Linux scsi_debug 0191 PQ: 0 ANSI: 7
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.053271] scsi 66:0:0:0: Power-on or device reset occurred
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.063350] sd 66:0:0:0: [sda] 16384 512-byte logical blocks: (8.39 MB/8.00 MiB)
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.063603] sd 66:0:0:0: Attached scsi generic sg1 type 0
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.064965] sd 66:0:0:0: [sda] Write Protect is off
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.065470] sd 66:0:0:0: [sda] Mode Sense: 73 00 10 08
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.068116] sd 66:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.075669] sd 66:0:0:0: [sda] permanent stream count = 5
2026-10-17T22:40:10Z {detected_level="unknown"} [ 2572.080471] sd 66:0:0:0: [sda] Preferred minimum I/O size 512 bytes
2026-10-17T22:40:10Z {detected_level="unknown"} [ 2572.082032] sd 66:0:0:0: [sda] Optimal transfer size 524288 bytes
2026-10-17T22:40:10Z {detected_level="unknown"} [ 2572.131009] sd 66:0:0:0: [sda] Attached SCSI disk
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-10-18 12:26 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-09-22 20:07 [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 1/3] nvme-tcp: store and use the caller's network namespace Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 2/3] nvme-tcp: refactor I/O queue setup path Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 3/3] nvme-tcp: parallelize I/O queue allocation and startup Surabhi Gogte
2026-10-18 12:25 ` Shin'ichiro Kawasaki
2026-10-13 15:33 ` [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Keith Busch
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®