From: Saravanan D <saravanand@crusoe.ai>
To: linux-nvme@lists.infradead.org
Cc: kbusch@kernel.org, hch@lst.de, sagi@grimberg.me, axboe@kernel.dk,
nilay@linux.ibm.com, dwagner@suse.de,
linux-kernel@vger.kernel.org, iyamahata@crusoe.ai,
kiyer@crusoe.ai, ganbalagane@crusoe.ai, sj@kernel.org,
Saravanan D <saravanand@crusoe.ai>
Subject: [PATCH v4 0/2] nvme-tcp: allow setting per-queue io_cpu through sysfs
Date: Sun, 27 Sep 2026 00:09:23 -0700 [thread overview]
Message-ID: <20260927070925.47209-1-saravanand@crusoe.ai> (raw)
On hosts that partition their CPUs after connect time, the connect
time io_cpu selection can land a queue's socket work on CPUs owned by
a different workload. This series lets a control plane set each
queue's io_cpu directly through sysfs.
The first patch adds a small common queue info that a transport embeds
in its queue structure, backing a transport neutral
/sys/class/nvme/nvmeX/queues/<qid>/
directory under the controller device. The second patch has nvme-tcp
expose io_cpu and managed attributes there.
Changes since v3 [1]:
- Split into two patches and introduced the common queue info that
Sagi proposed [2], named nvme_queue_info since the PCI driver
already uses nvme_queue, as Nilay pointed out.
- Renamed the tcp_queues directory to queues so the ABI does not bake
in the transport, as suggested by Nilay [3].
- Added the read-only managed attribute, 1 while io_cpu is driver
managed and 0 once the user assigned it, as suggested by Nilay.
- io_cpu now accepts -1 or the string "unbound" to leave the socket
work unbound, instead of restoring the connect time selection, as
suggested by Sagi.
- A user assignment is tracked in the common queue info flags and
still persists across reconnects, following Sagi's earlier guidance
that io_cpu should not change on reconnect.
- The queue directories register once per controller lifetime and
persist across reconnects, and the io_cpu accounting moved from the
queue lock to a controller level lock since a write can now arrive
while a queue is torn down.
- The io_cpu accounting now also covers a pin to a queue whose connect
time pick did not bind a cpu, and the queue teardown clears any
accounting left by a write racing the teardown.
[1] https://lore.kernel.org/linux-nvme/20260909211432.6741-1-saravanand@crusoe.ai/
[2] https://lore.kernel.org/linux-nvme/13c160d0-0657-47ce-9a16-58d348267ab0@grimberg.me/
[3] https://lore.kernel.org/linux-nvme/8dfee10f-89f3-463b-8d16-587873c3394c@linux.ibm.com/
Saravanan D (2):
nvme: add per-queue sysfs directories
nvme-tcp: allow setting per-queue io_cpu through sysfs
Documentation/ABI/stable/sysfs-nvme | 21 ++++
drivers/nvme/host/core.c | 2 +
drivers/nvme/host/nvme.h | 23 +++++
drivers/nvme/host/sysfs.c | 42 ++++++++
drivers/nvme/host/tcp.c | 153 +++++++++++++++++++++++++++-
5 files changed, 240 insertions(+), 1 deletion(-)
base-commit: 2ee54f01f07c0307deaf90ca8691a4643ae0357b
--
2.55.0
next reply other threads:[~2026-09-27 7:09 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-27 7:09 Saravanan D [this message]
2026-09-27 7:09 ` [PATCH v4 1/2] nvme: add per-queue sysfs directories Saravanan D
2026-10-08 9:59 ` Nilay Shroff
2026-09-27 7:09 ` [PATCH v4 2/2] nvme-tcp: allow setting per-queue io_cpu through sysfs Saravanan D
2026-10-08 9:45 ` Nilay Shroff
2026-10-09 2:01 ` Saravanan D
2026-10-11 20:27 ` Nilay Shroff
2026-10-17 2:34 ` Saravanan D
2026-10-08 5:39 ` [PATCH v4 0/2] " Saravanan D
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260927070925.47209-1-saravanand@crusoe.ai \
--to=saravanand@crusoe.ai \
--cc=axboe@kernel.dk \
--cc=dwagner@suse.de \
--cc=ganbalagane@crusoe.ai \
--cc=hch@lst.de \
--cc=iyamahata@crusoe.ai \
--cc=kbusch@kernel.org \
--cc=kiyer@crusoe.ai \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=nilay@linux.ibm.com \
--cc=sagi@grimberg.me \
--cc=sj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®