mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Surabhi Gogte <sgogte@purestorage.com>
To: Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@kernel.dk>,
	Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>
Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org,
	mkhalfella@purestorage.com, randyj@purestorage.com,
	adailey@purestorage.com, Surabhi Gogte <sgogte@purestorage.com>
Subject: [PATCH v3 0/2] nvme-rdma: parallelize I/O queue setup
Date: Thu, 25 Jun 2026 15:27:20 -0600	[thread overview]
Message-ID: <20260625212722.1302344-1-sgogte@purestorage.com> (raw)

This patch series parallelizes nvme-rdma connection and reconnection by
setting up I/O queues in parallel instead of sequentially. Allocation and
startup of each queue are combined into a single per-queue async work
item, so per-queue connection latency overlaps across all queues. This
matters most on high-core-count hosts with many I/O queues, where serial
setup dominates connect time.

Patch 1 is a preparatory refactor: nvme_rdma_alloc_queue() takes a queue
pointer so allocation and startup can be folded into a single async
worker.

Patch 2 contains the implementation for async setup of I/O queues.

Testing on a 64-core host with 64 I/O queues shows nvme-rdma connection
time reduced from ~1.4s to 416ms.

Signed-off-by: Surabhi Gogte <sgogte@purestorage.com>
---
Changes from v2->v3:
- Split the series into two patches: extract the nvme_rdma_alloc_queue()
  refactor into a separate preparatory patch.
- Replace the atomic error flag in struct nvme_rdma_ctrl with a per-work
  nvme_rdma_setup_ctx { queue, err } struct.
- Fix formatting changes regarding line overflow indentation and nesting.

Changes from v1->v2:
- Remove separate workqueue and use the async API instead.

Previous versions:
v1: https://lore.kernel.org/all/20260529001354.1003640-1-sgogte@purestorage.com/
v2: https://lore.kernel.org/all/20260604195321.2232838-1-sgogte@purestorage.com/

Surabhi Gogte (2):
  nvme-rdma: refactor nvme_rdma_alloc_queue() to take a queue pointer
  nvme-rdma: parallelize I/O queue allocation and startup

 drivers/nvme/host/rdma.c | 135 ++++++++++++++++++++++++---------------
 1 file changed, 82 insertions(+), 53 deletions(-)

--
2.54.0


             reply	other threads:[~2026-06-25 21:27 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-25 21:27 Surabhi Gogte [this message]
2026-06-25 21:27 ` [PATCH v3 1/2] nvme-rdma: refactor nvme_rdma_alloc_queue() to take a queue pointer Surabhi Gogte
2026-06-26  7:34   ` Christoph Hellwig
2026-06-25 21:27 ` [PATCH v3 2/2] nvme-rdma: parallelize I/O queue allocation and startup Surabhi Gogte
2026-06-26  7:40   ` Christoph Hellwig
2026-06-27  4:26     ` Surabhi Gogte (she/her)
2026-07-01 11:08 ` [PATCH v3 0/2] nvme-rdma: parallelize I/O queue setup Hannes Reinecke
2026-07-01 23:22   ` Surabhi Gogte (she/her)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260625212722.1302344-1-sgogte@purestorage.com \
    --to=sgogte@purestorage.com \
    --cc=adailey@purestorage.com \
    --cc=axboe@kernel.dk \
    --cc=hch@lst.de \
    --cc=kbusch@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=mkhalfella@purestorage.com \
    --cc=randyj@purestorage.com \
    --cc=sagi@grimberg.me \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®