[PATCH v5 0/2] nvme-tcp: allow setting per-queue io_cpu through sysfs

[PATCH v5 0/2] nvme-tcp: allow setting per-queue io_cpu through sysfs

[Saravanan D]

On hosts that partition their CPUs after connect time, the connect
time io_cpu selection can land a queue's socket work on CPUs owned by
a different workload. This series lets a control plane set each
queue's io_cpu directly through sysfs.

The first patch adds core helpers for a transport neutral

  /sys/class/nvme/nvmeX/queues/<qid>/

directory per queue under the controller device. The second patch has
nvme-tcp expose io_cpu and managed attributes there.

Changes since v4 [1]:
- Rebased onto nvme-7.4, which sets up the I/O queues in parallel
  (61b0ed3be81b) and serializes the io_cpu accounting with
  nvme_tcp_cpu_queues_lock. The per controller io_cpu_lock is gone,
  and the io_cpu writes take nvme_tcp_cpu_queues_lock instead.
- A queue's directory now follows the queue's allocation, added in
  nvme_tcp_alloc_queue() and removed in nvme_tcp_free_queue(), so the
  directories match the queue count after a reconnect (Nilay). The
  core allocates each directory and its release frees only that
  allocation, which drops the transport's reference counting and
  completion (Nilay) and fixes the use after free, the controller
  leak and the teardown hang found by the Sashiko review of v4 [2].
- The transport creates the queues directory once before the parallel
  queue setup, and the kerneldoc states which calls may run
  concurrently (Nilay).
- The user assignment flag moved from the core into the nvme-tcp
  queue flags (Nilay).
- io_cpu is read through nvme_tcp_queue_io_cpu() and written through
  nvme_tcp_queue_change_io_cpu(), which carries __must_hold() for the
  context analysis (Nilay).
- The sysfs store collapsed to a single path and no longer checks
  NVME_TCP_Q_ALLOCATED, since the directory only exists while the
  queue is allocated (Nilay).
- The queue stop now clears NVME_TCP_Q_IO_CPU_SET under
  nvme_tcp_cpu_queues_lock, since a store may move io_cpu between the
  clear and the decrement.
- nvme_tcp_free_queue() drops NVME_TCP_Q_IO_CPU_SET, which also fixes
  a ledger count leaked by every failed I/O queue connect.
- io_cpu parses a decimal cpu number, and the ABI entry describes the
  directory lifetime, persistence across CPU offline and the behavior
  with wq_unbound.

[1] https://lore.kernel.org/linux-nvme/20260927070925.47209-1-saravanand@crusoe.ai/
[2] https://sashiko.dev/#/patchset/20260927070925.47209-1-saravanand%40crusoe.ai

Saravanan D (2):
  nvme: add per-queue sysfs directories
  nvme-tcp: allow setting per-queue io_cpu through sysfs

 Documentation/ABI/stable/sysfs-nvme |  29 +++++
 drivers/nvme/host/core.c            |   6 +
 drivers/nvme/host/nvme.h            |  23 ++++
 drivers/nvme/host/sysfs.c           |  88 ++++++++++++++
 drivers/nvme/host/tcp.c             | 173 ++++++++++++++++++++++++----
 5 files changed, 299 insertions(+), 20 deletions(-)


base-commit: 2e8e30b3cf9f165e1a1d079331543cb7f2ad0e0c
-- 
2.55.0

[PATCH v5 1/2] nvme: add per-queue sysfs directories

[Saravanan D]

Per I/O queue transport state, such as the nvme-tcp io_cpu, has
nowhere to live in sysfs, since the core has no directory for
individual queues.

Add helpers that create the controller's queues directory and add or
remove one directory per queue

  /sys/class/nvme/nvmeX/queues/<qid>/

carrying attributes provided by the transport. The transport creates
the queues directory from its controller setup path, adds a queue's
directory when it allocates the queue and removes it when it frees
the queue, so the directories follow the queues across reconnects and
changes in the queue count. The core allocates each queue directory
and its release frees only that allocation. Removal waits for running
attribute callbacks, so the transport may tear the queue down once it
returns.

The queues directory holds a reference on the controller device, so
nvme_uninit_ctrl() deletes it after the transport has removed the
queue directories.

Suggested-by: Sagi Grimberg <sagi@grimberg.me>
Link: https://lore.kernel.org/linux-nvme/13c160d0-0657-47ce-9a16-58d348267ab0@grimberg.me/
Assisted-by: Claude:claude-opus-5-5 [Claude Code]
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
---
 drivers/nvme/host/core.c  |  6 +++
 drivers/nvme/host/nvme.h  | 23 ++++++++++
 drivers/nvme/host/sysfs.c | 88 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 117 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 531f9bb14bab..41e1c9d0ee58 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -5153,6 +5153,12 @@ void nvme_uninit_ctrl(struct nvme_ctrl *ctrl)
 {
 	nvme_stop_keep_alive(ctrl);
 	nvme_hwmon_exit(ctrl);
+	/* transports remove their queue directories before this point */
+	if (ctrl->queues_kobj) {
+		kobject_del(ctrl->queues_kobj);
+		kobject_put(ctrl->queues_kobj);
+		ctrl->queues_kobj = NULL;
+	}
 	nvme_fault_inject_fini(&ctrl->fault_inject);
 	dev_pm_qos_hide_latency_tolerance(ctrl->device);
 	cdev_device_del(&ctrl->cdev, ctrl->device);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index a8ef62b8e34b..d198214d7f8b 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -339,6 +339,28 @@ enum nvme_ctrl_flags {
 	NVME_CTRL_FROZEN		= 6,
 };
 
+/*
+ * Per-queue directory /sys/class/nvme/nvmeX/queues/<qid>/, allocated by the
+ * core and carrying attributes provided by the transport.
+ */
+struct nvme_queue_info {
+	struct kobject		kobj;
+	void			*data;
+};
+
+static inline void *nvme_queue_info_data(struct kobject *kobj)
+{
+	return container_of(kobj, struct nvme_queue_info, kobj)->data;
+}
+
+struct nvme_ctrl;
+
+int nvme_queue_dir_create(struct nvme_ctrl *ctrl);
+struct nvme_queue_info *
+nvme_add_queue_info(struct nvme_ctrl *ctrl, unsigned int qid,
+		    const struct attribute_group **groups, void *data);
+void nvme_remove_queue_info(struct nvme_queue_info *qinfo);
+
 struct nvme_ctrl {
 	bool comp_seen;
 	bool identified;
@@ -360,6 +382,7 @@ struct nvme_ctrl {
 	struct srcu_struct srcu;
 	struct device ctrl_device;
 	struct device *device;	/* char device */
+	struct kobject *queues_kobj;
 #ifdef CONFIG_NVME_HWMON
 	struct device *hwmon_device;
 #endif
diff --git a/drivers/nvme/host/sysfs.c b/drivers/nvme/host/sysfs.c
index 5224597a32f1..cf62ff905a35 100644
--- a/drivers/nvme/host/sysfs.c
+++ b/drivers/nvme/host/sysfs.c
@@ -1313,3 +1313,91 @@ const struct attribute_group *nvme_subsys_attrs_groups[] = {
 	&nvme_subsys_attrs_group,
 	NULL,
 };
+
+static void nvme_queue_info_release(struct kobject *kobj)
+{
+	kfree(container_of(kobj, struct nvme_queue_info, kobj));
+}
+
+static const struct kobj_type nvme_queue_info_ktype = {
+	.sysfs_ops	= &kobj_sysfs_ops,
+	.release	= nvme_queue_info_release,
+};
+
+/**
+ * nvme_queue_dir_create - create the controller's queues directory
+ * @ctrl: controller
+ *
+ * Creates /sys/class/nvme/nvmeX/queues/ if it does not exist yet. The
+ * transport calls this from its controller setup path, which must not run
+ * it concurrently for the same controller. nvme_uninit_ctrl() deletes the
+ * directory.
+ */
+int nvme_queue_dir_create(struct nvme_ctrl *ctrl)
+{
+	if (ctrl->queues_kobj)
+		return 0;
+
+	ctrl->queues_kobj = kobject_create_and_add("queues",
+						   &ctrl->device->kobj);
+	if (!ctrl->queues_kobj)
+		return -ENOMEM;
+	return 0;
+}
+EXPORT_SYMBOL_GPL(nvme_queue_dir_create);
+
+/**
+ * nvme_add_queue_info - add a directory for one queue
+ * @ctrl: controller, whose queues directory must exist
+ * @qid: queue id, used as the directory name
+ * @groups: attributes provided by the transport
+ * @data: transport data, returned by nvme_queue_info_data()
+ *
+ * Adds /sys/class/nvme/nvmeX/queues/<qid>/. Calls for different queues of
+ * the same controller may run concurrently. Returns the queue info, to be
+ * passed to nvme_remove_queue_info(), or an ERR_PTR().
+ */
+struct nvme_queue_info *
+nvme_add_queue_info(struct nvme_ctrl *ctrl, unsigned int qid,
+		    const struct attribute_group **groups, void *data)
+{
+	struct nvme_queue_info *qinfo;
+	int ret;
+
+	if (WARN_ON_ONCE(!ctrl->queues_kobj))
+		return ERR_PTR(-EINVAL);
+
+	qinfo = kzalloc_obj(*qinfo);
+	if (!qinfo)
+		return ERR_PTR(-ENOMEM);
+	qinfo->data = data;
+
+	ret = kobject_init_and_add(&qinfo->kobj, &nvme_queue_info_ktype,
+				   ctrl->queues_kobj, "%u", qid);
+	if (ret)
+		goto out_put;
+
+	ret = sysfs_create_groups(&qinfo->kobj, groups);
+	if (ret)
+		goto out_put;
+	return qinfo;
+
+out_put:
+	kobject_put(&qinfo->kobj);
+	return ERR_PTR(ret);
+}
+EXPORT_SYMBOL_GPL(nvme_add_queue_info);
+
+/**
+ * nvme_remove_queue_info - remove a queue directory
+ * @qinfo: queue info returned by nvme_add_queue_info()
+ *
+ * Waits for running attribute callbacks, so none run once this returns
+ * and the transport data may be torn down.
+ */
+void nvme_remove_queue_info(struct nvme_queue_info *qinfo)
+{
+	kobject_del(&qinfo->kobj);
+	kobject_put(&qinfo->kobj);
+}
+EXPORT_SYMBOL_GPL(nvme_remove_queue_info);
-- 
2.55.0

[PATCH v5 2/2] nvme-tcp: allow setting per-queue io_cpu through sysfs

[Saravanan D]

nvme_tcp_set_queue_io_cpu() picks each queue's io_cpu at connect time
as the least loaded CPU in the queue's blk-mq map group, and all socket
work runs there for the connection's lifetime. This decision falls
short when the host partitions its CPUs after connect time. On a 384
cpu multi tenant host with 128 queue controllers, blk-mq folds three
CPUs into every map group, some groups straddle two tenants' cpusets,
and 9% of nvme_tcp_io_work executions ran outside the submitting VM's
cpuset, seen by the neighbor as steal time.

Expose the io_cpu in each I/O queue's directory

  /sys/class/nvme/nvmeX/queues/<qid>/io_cpu

so a control plane that owns CPU placement can set it directly instead
of relying on the driver's heuristic. The attribute accepts a decimal
cpu number, -1 or "unbound". A user assignment, unbound included, is
marked in the queue flags, persists across reconnects and is never
reselected by the driver. The read-only managed attribute reports 1
while io_cpu is driver managed and 0 once the user assigned it. The
accounting of nvme_tcp_cpu_queues applies regardless of who set the
io_cpu.

A queue's directory is added when the queue is allocated and removed
when it is freed, before the queue is torn down, so a sysfs store
never runs on a freed queue. The queues directory is created before
the I/O queues are set up in parallel. All io_cpu writes, from sysfs
and from the driver, serialize with the accounting under
nvme_tcp_cpu_queues_lock, while the I/O paths read io_cpu locklessly.
The queue stop now clears NVME_TCP_Q_IO_CPU_SET under that lock, since
a sysfs store may move io_cpu in between.

nvme_tcp_free_queue() now clears NVME_TCP_Q_IO_CPU_SET and drops its
count, because a queue can still be counted when it is freed. A sysfs
store may have run after the queue stop, or the connect may have
failed, and the setup error path frees such a queue without calling
nvme_tcp_stop_queue(), where the count is normally dropped.

Suggested-by: Sagi Grimberg <sagi@grimberg.me>
Link: https://lore.kernel.org/linux-nvme/13c160d0-0657-47ce-9a16-58d348267ab0@grimberg.me/
Assisted-by: Claude:claude-opus-5-5 [Claude Code]
Signed-off-by: Saravanan D <saravanand@crusoe.ai>
---
 Documentation/ABI/stable/sysfs-nvme |  29 +++++
 drivers/nvme/host/tcp.c             | 173 ++++++++++++++++++++++++----
 2 files changed, 182 insertions(+), 20 deletions(-)

diff --git a/Documentation/ABI/stable/sysfs-nvme b/Documentation/ABI/stable/sysfs-nvme
index 9291595f8102..4160589f7524 100644
--- a/Documentation/ABI/stable/sysfs-nvme
+++ b/Documentation/ABI/stable/sysfs-nvme
@@ -503,3 +503,32 @@ Contact:	Hannes Reinecke <hare@suse.de>
 Description:
 		Shows the subsystem type. Possible values: "discovery",
 		"nvm", "reserved".
+
+What:		/sys/class/nvme/nvmeX/queues/<qid>/io_cpu
+What:		/sys/class/nvme/nvmeX/queues/<qid>/managed
+Date:		October 2026
+KernelVersion:	7.4
+Contact:	Saravanan D <saravanand@crusoe.ai>
+Description:
+		Per I/O queue directories, populated by the transport
+		backing the controller. A directory exists while its
+		queue is allocated, so the directories follow the queue
+		count and are absent while the controller reconnects.
+		NVMe over TCP exposes:
+
+		io_cpu: (RW) The CPU that runs the socket work for I/O
+		queue <qid>, selected by the driver at connect time.
+		Writing a decimal online CPU number overrides the
+		selection, and writing -1 or "unbound" leaves the socket
+		work unbound. A written value persists across reconnects
+		for the life of the controller, also when its CPU goes
+		offline. Unbound socket work runs on the CPU that queued
+		it. When the nvme_tcp module is loaded with wq_unbound
+		set, the socket work runs on an unbound workqueue and
+		how closely it follows io_cpu depends on the affinity
+		scope of the nvme_tcp_wq workqueue. Reads return the
+		current CPU, or -1 when the queue is unbound.
+
+		managed: (RO) Shows who owns the CPU placement. 1 while
+		io_cpu is driver managed, 0 once the user assigned it,
+		for the life of the controller.
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index ec678ed07af8..699997c0e8e2 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -95,6 +95,7 @@ enum nvme_tcp_queue_flags {
 	NVME_TCP_Q_LIVE		= 1,
 	NVME_TCP_Q_POLLING	= 2,
 	NVME_TCP_Q_IO_CPU_SET	= 3,
+	NVME_TCP_Q_IO_CPU_USER	= 4,
 };
 
 enum nvme_tcp_recv_state {
@@ -142,6 +143,7 @@ struct nvme_tcp_queue {
 	struct completion       tls_complete;
 	int                     tls_err;
 	struct page_frag_cache	pf_cache;
+	struct nvme_queue_info	*qinfo;
 
 	void (*state_change)(struct sock *);
 	void (*data_ready)(struct sock *);
@@ -230,6 +232,19 @@ static inline int nvme_tcp_queue_id(struct nvme_tcp_queue *queue)
 	return queue - queue->ctrl->queues;
 }
 
+/* the I/O paths read io_cpu locklessly, writers hold the accounting lock */
+static inline int nvme_tcp_queue_io_cpu(struct nvme_tcp_queue *queue)
+{
+	return READ_ONCE(queue->io_cpu);
+}
+
+static inline void nvme_tcp_queue_change_io_cpu(struct nvme_tcp_queue *queue,
+						int cpu)
+	__must_hold(&nvme_tcp_cpu_queues_lock)
+{
+	WRITE_ONCE(queue->io_cpu, cpu);
+}
+
 static inline bool nvme_tcp_recv_pdu_supported(enum nvme_tcp_pdu_type type)
 {
 	switch (type) {
@@ -427,7 +442,7 @@ static inline void nvme_tcp_queue_request(struct nvme_tcp_request *req,
 	 * lockdep reports circular locking via elevator_lock.  Defer TLS
 	 * sends to the io workqueue instead of inline from this path.
 	 */
-	if (queue->io_cpu == raw_smp_processor_id() &&
+	if (nvme_tcp_queue_io_cpu(queue) == raw_smp_processor_id() &&
 	    !nvme_tcp_queue_tls(queue) &&
 	    empty && mutex_trylock(&queue->send_mutex)) {
 		nvme_tcp_send_all(queue);
@@ -435,7 +450,8 @@ static inline void nvme_tcp_queue_request(struct nvme_tcp_request *req,
 	}
 
 	if (last && nvme_tcp_queue_has_pending(queue))
-		queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue->io_work);
+		queue_work_on(nvme_tcp_queue_io_cpu(queue), nvme_tcp_wq,
+			      &queue->io_work);
 }
 
 static void nvme_tcp_process_req_list(struct nvme_tcp_queue *queue)
@@ -844,7 +860,8 @@ static int nvme_tcp_handle_r2t(struct nvme_tcp_queue *queue,
 	nvme_tcp_setup_h2c_data_pdu(req);
 
 	llist_add(&req->lentry, &queue->req_list);
-	queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue->io_work);
+	queue_work_on(nvme_tcp_queue_io_cpu(queue), nvme_tcp_wq,
+		      &queue->io_work);
 
 	return 0;
 }
@@ -1131,7 +1148,8 @@ static void nvme_tcp_data_ready(struct sock *sk)
 	queue = sk->sk_user_data;
 	if (likely(queue && queue->rd_enabled) &&
 	    !test_bit(NVME_TCP_Q_POLLING, &queue->flags))
-		queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue->io_work);
+		queue_work_on(nvme_tcp_queue_io_cpu(queue), nvme_tcp_wq,
+			      &queue->io_work);
 	read_unlock_bh(&sk->sk_callback_lock);
 }
 
@@ -1146,7 +1164,8 @@ static void nvme_tcp_write_space(struct sock *sk)
 		/* Ensure pending TLS partial records are retried */
 		if (nvme_tcp_queue_tls(queue))
 			queue->write_space(sk);
-		queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue->io_work);
+		queue_work_on(nvme_tcp_queue_io_cpu(queue), nvme_tcp_wq,
+			      &queue->io_work);
 	}
 	read_unlock_bh(&sk->sk_callback_lock);
 }
@@ -1469,7 +1488,8 @@ static void nvme_tcp_io_work(struct work_struct *w)
 
 	} while (!time_after(jiffies, deadline)); /* quota is exhausted */
 
-	queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue->io_work);
+	queue_work_on(nvme_tcp_queue_io_cpu(queue), nvme_tcp_wq,
+		      &queue->io_work);
 }
 
 static void nvme_tcp_free_async_req(struct nvme_tcp_ctrl *ctrl)
@@ -1503,9 +1523,22 @@ static void nvme_tcp_free_queue(struct nvme_ctrl *nctrl, int qid)
 	struct nvme_tcp_queue *queue = &ctrl->queues[qid];
 	unsigned int noio_flag;
 
-	if (!test_and_clear_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
+	if (!test_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
 		return;
 
+	/* no io_cpu store runs once the directory is removed */
+	if (queue->qinfo) {
+		nvme_remove_queue_info(queue->qinfo);
+		queue->qinfo = NULL;
+	}
+	clear_bit(NVME_TCP_Q_ALLOCATED, &queue->flags);
+
+	/* a failed connect or a store after the stop left it counted */
+	spin_lock(&nvme_tcp_cpu_queues_lock);
+	if (test_and_clear_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags))
+		nvme_tcp_cpu_queues[nvme_tcp_queue_io_cpu(queue)]--;
+	spin_unlock(&nvme_tcp_cpu_queues_lock);
+
 	page_frag_cache_drain(&queue->pf_cache);
 
 	/**
@@ -1725,8 +1758,18 @@ static void nvme_tcp_set_queue_io_cpu(struct nvme_tcp_queue *queue)
 	unsigned int *mq_map = NULL;
 	int cpu, min_queues = INT_MAX, io_cpu;
 
+	spin_lock(&nvme_tcp_cpu_queues_lock);
+	/* a user assigned io_cpu is kept across reconnects */
+	if (test_bit(NVME_TCP_Q_IO_CPU_USER, &queue->flags)) {
+		io_cpu = nvme_tcp_queue_io_cpu(queue);
+		if (io_cpu != WORK_CPU_UNBOUND &&
+		    !test_and_set_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags))
+			nvme_tcp_cpu_queues[io_cpu]++;
+		goto out_unlock;
+	}
+
 	if (wq_unbound)
-		goto out;
+		goto out_unlock;
 
 	if (nvme_tcp_default_queue(queue))
 		mq_map = set->map[HCTX_TYPE_DEFAULT].mq_map;
@@ -1736,10 +1779,9 @@ static void nvme_tcp_set_queue_io_cpu(struct nvme_tcp_queue *queue)
 		mq_map = set->map[HCTX_TYPE_POLL].mq_map;
 
 	if (WARN_ON(!mq_map))
-		goto out;
+		goto out_unlock;
 
 	/* Search for the least used cpu from the mq_map */
-	spin_lock(&nvme_tcp_cpu_queues_lock);
 	io_cpu = WORK_CPU_UNBOUND;
 	for_each_online_cpu(cpu) {
 		int num_queues = nvme_tcp_cpu_queues[cpu];
@@ -1752,14 +1794,14 @@ static void nvme_tcp_set_queue_io_cpu(struct nvme_tcp_queue *queue)
 		}
 	}
 	if (io_cpu != WORK_CPU_UNBOUND) {
-		queue->io_cpu = io_cpu;
+		nvme_tcp_queue_change_io_cpu(queue, io_cpu);
 		nvme_tcp_cpu_queues[io_cpu]++;
 		set_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags);
 	}
+out_unlock:
 	spin_unlock(&nvme_tcp_cpu_queues_lock);
-out:
 	dev_dbg(ctrl->ctrl.device, "queue %d: using cpu %d\n",
-		qid, queue->io_cpu);
+		qid, nvme_tcp_queue_io_cpu(queue));
 }
 
 static void nvme_tcp_tls_done(void *data, int status, key_serial_t pskid)
@@ -1847,6 +1889,85 @@ static int nvme_tcp_start_tls(struct nvme_ctrl *nctrl,
 	return ret;
 }
 
+static ssize_t io_cpu_show(struct kobject *kobj, struct kobj_attribute *attr,
+			   char *buf)
+{
+	struct nvme_tcp_queue *queue = nvme_queue_info_data(kobj);
+	int io_cpu = nvme_tcp_queue_io_cpu(queue);
+
+	return sysfs_emit(buf, "%d\n",
+			  io_cpu == WORK_CPU_UNBOUND ? -1 : io_cpu);
+}
+
+static ssize_t io_cpu_store(struct kobject *kobj, struct kobj_attribute *attr,
+			    const char *buf, size_t count)
+{
+	struct nvme_tcp_queue *queue = nvme_queue_info_data(kobj);
+	int cpu, old, ret;
+
+	if (sysfs_streq(buf, "unbound")) {
+		cpu = WORK_CPU_UNBOUND;
+	} else {
+		ret = kstrtoint(buf, 10, &cpu);
+		if (ret)
+			return ret;
+		if (cpu == -1)
+			cpu = WORK_CPU_UNBOUND;
+		else if ((unsigned int)cpu >= nr_cpu_ids || !cpu_online(cpu))
+			return -EINVAL;
+	}
+
+	spin_lock(&nvme_tcp_cpu_queues_lock);
+	old = nvme_tcp_queue_io_cpu(queue);
+	nvme_tcp_queue_change_io_cpu(queue, cpu);
+	set_bit(NVME_TCP_Q_IO_CPU_USER, &queue->flags);
+	if (test_and_clear_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags))
+		nvme_tcp_cpu_queues[old]--;
+	if (cpu != WORK_CPU_UNBOUND) {
+		nvme_tcp_cpu_queues[cpu]++;
+		set_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags);
+	}
+	spin_unlock(&nvme_tcp_cpu_queues_lock);
+
+	return count;
+}
+
+static ssize_t managed_show(struct kobject *kobj, struct kobj_attribute *attr,
+			    char *buf)
+{
+	struct nvme_tcp_queue *queue = nvme_queue_info_data(kobj);
+
+	return sysfs_emit(buf, "%d\n",
+			  !test_bit(NVME_TCP_Q_IO_CPU_USER, &queue->flags));
+}
+
+static struct kobj_attribute nvme_tcp_io_cpu_attr =
+	__ATTR(io_cpu, 0644, io_cpu_show, io_cpu_store);
+static struct kobj_attribute nvme_tcp_managed_attr =
+	__ATTR_RO(managed);
+
+static struct attribute *nvme_tcp_queue_attrs[] = {
+	&nvme_tcp_io_cpu_attr.attr,
+	&nvme_tcp_managed_attr.attr,
+	NULL,
+};
+ATTRIBUTE_GROUPS(nvme_tcp_queue);
+
+static void nvme_tcp_add_queue_sysfs(struct nvme_tcp_queue *queue)
+{
+	struct nvme_ctrl *nctrl = &queue->ctrl->ctrl;
+	struct nvme_queue_info *qinfo;
+
+	qinfo = nvme_add_queue_info(nctrl, nvme_tcp_queue_id(queue),
+				    nvme_tcp_queue_groups, queue);
+	if (IS_ERR(qinfo)) {
+		dev_warn(nctrl->device, "queue %d: no sysfs directory: %ld\n",
+			 nvme_tcp_queue_id(queue), PTR_ERR(qinfo));
+		return;
+	}
+	queue->qinfo = qinfo;
+}
+
 static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 				key_serial_t pskid)
 {
@@ -1917,7 +2038,11 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 
 	queue->sock->sk->sk_allocation = GFP_ATOMIC;
 	queue->sock->sk->sk_use_task_frag = false;
-	queue->io_cpu = WORK_CPU_UNBOUND;
+	/* no store races here, the lock is for the write helper */
+	spin_lock(&nvme_tcp_cpu_queues_lock);
+	if (!test_bit(NVME_TCP_Q_IO_CPU_USER, &queue->flags))
+		nvme_tcp_queue_change_io_cpu(queue, WORK_CPU_UNBOUND);
+	spin_unlock(&nvme_tcp_cpu_queues_lock);
 	queue->request = NULL;
 	queue->data_remaining = 0;
 	queue->ddgst_remaining = 0;
@@ -1985,6 +2110,9 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
 
 	set_bit(NVME_TCP_Q_ALLOCATED, &queue->flags);
 
+	if (qid)
+		nvme_tcp_add_queue_sysfs(queue);
+
 	return 0;
 
 err_init_connect:
@@ -2033,11 +2161,10 @@ static void nvme_tcp_stop_queue_nowait(struct nvme_ctrl *nctrl, int qid)
 	if (!test_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
 		return;
 
-	if (test_and_clear_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags)) {
-		spin_lock(&nvme_tcp_cpu_queues_lock);
-		nvme_tcp_cpu_queues[queue->io_cpu]--;
-		spin_unlock(&nvme_tcp_cpu_queues_lock);
-	}
+	spin_lock(&nvme_tcp_cpu_queues_lock);
+	if (test_and_clear_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags))
+		nvme_tcp_cpu_queues[nvme_tcp_queue_io_cpu(queue)]--;
+	spin_unlock(&nvme_tcp_cpu_queues_lock);
 
 	mutex_lock(&queue->queue_lock);
 	if (test_and_clear_bit(NVME_TCP_Q_LIVE, &queue->flags))
@@ -2306,6 +2433,11 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
 	if (ret)
 		return ret;
 
+	/* the parallel queue setup below adds the queue directories */
+	ret = nvme_queue_dir_create(ctrl);
+	if (ret)
+		return ret;
+
 	if (new) {
 		ret = nvme_alloc_io_tag_set(ctrl, &to_tcp_ctrl(ctrl)->tag_set,
 				&nvme_tcp_mq_ops,
@@ -2893,7 +3025,8 @@ static void nvme_tcp_commit_rqs(struct blk_mq_hw_ctx *hctx)
 	struct nvme_tcp_queue *queue = hctx->driver_data;
 
 	if (!llist_empty(&queue->req_list))
-		queue_work_on(queue->io_cpu, nvme_tcp_wq, &queue->io_work);
+		queue_work_on(nvme_tcp_queue_io_cpu(queue), nvme_tcp_wq,
+			      &queue->io_work);
 }
 
 static blk_status_t nvme_tcp_queue_rq(struct blk_mq_hw_ctx *hctx,
-- 
2.55.0

Re: [PATCH v5 2/2] nvme-tcp: allow setting per-queue io_cpu through sysfs

[Daniel Wagner]

On Sat, Oct 17, 2026 at 06:00:59PM -0700, Saravanan D wrote:
> nvme_tcp_set_queue_io_cpu() picks each queue's io_cpu at connect time
> as the least loaded CPU in the queue's blk-mq map group, and all socket
> work runs there for the connection's lifetime. This decision falls
> short when the host partitions its CPUs after connect time. On a 384
> cpu multi tenant host with 128 queue controllers, blk-mq folds three
> CPUs into every map group, some groups straddle two tenants' cpusets,
> and 9% of nvme_tcp_io_work executions ran outside the submitting VM's
> cpuset, seen by the neighbor as steal time.
> 
> Expose the io_cpu in each I/O queue's directory
> 
>   /sys/class/nvme/nvmeX/queues/<qid>/io_cpu
> 
> so a control plane that owns CPU placement can set it directly instead
> of relying on the driver's heuristic. The attribute accepts a decimal
> cpu number, -1 or "unbound". A user assignment, unbound included, is
> marked in the queue flags, persists across reconnects and is never
> reselected by the driver. The read-only managed attribute reports 1
> while io_cpu is driver managed and 0 once the user assigned it. The
> accounting of nvme_tcp_cpu_queues applies regardless of who set the
> io_cpu.
> 
> A queue's directory is added when the queue is allocated and removed
> when it is freed, before the queue is torn down, so a sysfs store
> never runs on a freed queue. The queues directory is created before
> the I/O queues are set up in parallel. All io_cpu writes, from sysfs
> and from the driver, serialize with the accounting under
> nvme_tcp_cpu_queues_lock, while the I/O paths read io_cpu locklessly.
> The queue stop now clears NVME_TCP_Q_IO_CPU_SET under that lock, since
> a sysfs store may move io_cpu in between.
> 
> nvme_tcp_free_queue() now clears NVME_TCP_Q_IO_CPU_SET and drops its
> count, because a queue can still be counted when it is freed. A sysfs
> store may have run after the queue stop, or the connect may have
> failed, and the setup error path frees such a queue without calling
> nvme_tcp_stop_queue(), where the count is normally dropped.
 
This looks good, just trying to wrap my head around with isolcpu (yeah,
doing a bit of work in that area makes me a paranoid).

I think it make sense to allow that the user can change queue CPU
pinning during runtime even with isolcpu enabled. And then the question
is should this be connected to cgroups?

Re: [PATCH v5 2/2] nvme-tcp: allow setting per-queue io_cpu through sysfs

[Hannes Reinecke]

On 10/20/26 5:28 PM, Daniel Wagner wrote:
> On Sat, Oct 17, 2026 at 06:00:59PM -0700, Saravanan D wrote:
>> nvme_tcp_set_queue_io_cpu() picks each queue's io_cpu at connect time
>> as the least loaded CPU in the queue's blk-mq map group, and all socket
>> work runs there for the connection's lifetime. This decision falls
>> short when the host partitions its CPUs after connect time. On a 384
>> cpu multi tenant host with 128 queue controllers, blk-mq folds three
>> CPUs into every map group, some groups straddle two tenants' cpusets,
>> and 9% of nvme_tcp_io_work executions ran outside the submitting VM's
>> cpuset, seen by the neighbor as steal time.
>>
>> Expose the io_cpu in each I/O queue's directory
>>
>>    /sys/class/nvme/nvmeX/queues/<qid>/io_cpu
>>
>> so a control plane that owns CPU placement can set it directly instead
>> of relying on the driver's heuristic. The attribute accepts a decimal
>> cpu number, -1 or "unbound". A user assignment, unbound included, is
>> marked in the queue flags, persists across reconnects and is never
>> reselected by the driver. The read-only managed attribute reports 1
>> while io_cpu is driver managed and 0 once the user assigned it. The
>> accounting of nvme_tcp_cpu_queues applies regardless of who set the
>> io_cpu.
>>
>> A queue's directory is added when the queue is allocated and removed
>> when it is freed, before the queue is torn down, so a sysfs store
>> never runs on a freed queue. The queues directory is created before
>> the I/O queues are set up in parallel. All io_cpu writes, from sysfs
>> and from the driver, serialize with the accounting under
>> nvme_tcp_cpu_queues_lock, while the I/O paths read io_cpu locklessly.
>> The queue stop now clears NVME_TCP_Q_IO_CPU_SET under that lock, since
>> a sysfs store may move io_cpu in between.
>>
>> nvme_tcp_free_queue() now clears NVME_TCP_Q_IO_CPU_SET and drops its
>> count, because a queue can still be counted when it is freed. A sysfs
>> store may have run after the queue stop, or the connect may have
>> failed, and the setup error path frees such a queue without calling
>> nvme_tcp_stop_queue(), where the count is normally dropped.
>   
> This looks good, just trying to wrap my head around with isolcpu (yeah,
> doing a bit of work in that area makes me a paranoid).
> 
> I think it make sense to allow that the user can change queue CPU
> pinning during runtime even with isolcpu enabled. And then the question
> is should this be connected to cgroups?

Not sure if cgroups is the right concept here; cgroups are user/
application centric whereas nvme-tcp is pretty much queue/cpu centric.
IE after all the cgroup calculations are done.
Don't think cgroups are a good fit here.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                  Kernel Storage Architect
hare@suse.de                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich

Re: [PATCH v5 2/2] nvme-tcp: allow setting per-queue io_cpu through sysfs

[Daniel Wagner]

On Wed, Oct 21, 2026 at 09:21:09AM +0200, Hannes Reinecke wrote:
> > I think it make sense to allow that the user can change queue CPU
> > pinning during runtime even with isolcpu enabled. And then the question
> > is should this be connected to cgroups?
> 
> Not sure if cgroups is the right concept here; cgroups are user/
> application centric whereas nvme-tcp is pretty much queue/cpu centric.
> IE after all the cgroup calculations are done.
> Don't think cgroups are a good fit here.

There was a lot of discussion during Linux Plumbers on how to configure
a system for the isolcpus case at runtime. From what I understand, the
preferred interface for this is cgroups. Hence, pinning the CPU for this
work is likely a target for it, though I don't think there is a need to
figure it out right now. I just wanted to highlight this point.

However, to make this truly user-friendly, I think nvme-cli should be
extended so it is aware of this and can configure it directly. I've
merged Nilay's --nr-io-queues patches, so this is the natural next
feature nvme-cli should support; see commit:

e170d6985797 ("news: document the use of --nr-io-queues during nvme connect")

I would also really like to have a blktest for this as well.

Re: [PATCH v5 2/2] nvme-tcp: allow setting per-queue io_cpu through sysfs

[Saravanan D]

On Wed, Oct 21, 2026 at 10:19:37AM +0200, Daniel Wagner wrote:
[...]
> There was a lot of discussion during Linux Plumbers on how to configure
> a system for the isolcpus case at runtime. From what I understand, the
> preferred interface for this is cgroups. Hence, pinning the CPU for this
> work is likely a target for it, though I don't think there is a need to
> figure it out right now. I just wanted to highlight this point.

Thanks for the pointer. I see the two as layers. cgroups decide which
CPUs a tenant owns. io_cpu then places a queue's socket work inside
that set. Today our control plane writes it, and a cgroup aware tool
could write it later. So io_cpu would not need to change if the
isolcpus configuration moves to cgroups.

On your earlier isolcpus point, the store only requires the cpu to be
online, so a queue can be pinned to an isolated cpu at runtime, for
example one that belongs to the VM issuing the I/O, even though blk-mq
leaves isolated cpus out of the hardware context masks.

> However, to make this truly user-friendly, I think nvme-cli should be
> extended so it is aware of this and can configure it directly. I've
> merged Nilay's --nr-io-queues patches, so this is the natural next
> feature nvme-cli should support; see commit:
>
> e170d6985797 ("news: document the use of --nr-io-queues during nvme connect")

Agreed. Once the ABI settles I'll add nvme-cli support to show and set
io_cpu per queue, following the --nr-io-queues work you merged.

> I would also really like to have a blktest for this as well.

I'll write one and post it alongside v6. It will use nvmet tcp over
loopback and check that

  - io_cpu reads and writes work and invalid values are rejected
  - an assignment survives a controller reset
  - the queue directories follow the queue count as it shrinks and grows
  - deleting the controller while io_cpu is written is safe

Thanks,
Saravanan D.

Re: [PATCH v5 2/2] nvme-tcp: allow setting per-queue io_cpu through sysfs

[Hannes Reinecke]

On 10/18/26 3:00 AM, Saravanan D wrote:
> nvme_tcp_set_queue_io_cpu() picks each queue's io_cpu at connect time
> as the least loaded CPU in the queue's blk-mq map group, and all socket
> work runs there for the connection's lifetime. This decision falls
> short when the host partitions its CPUs after connect time. On a 384
> cpu multi tenant host with 128 queue controllers, blk-mq folds three
> CPUs into every map group, some groups straddle two tenants' cpusets,
> and 9% of nvme_tcp_io_work executions ran outside the submitting VM's
> cpuset, seen by the neighbor as steal time.
> 
> Expose the io_cpu in each I/O queue's directory
> 
>    /sys/class/nvme/nvmeX/queues/<qid>/io_cpu
> 
> so a control plane that owns CPU placement can set it directly instead
> of relying on the driver's heuristic. The attribute accepts a decimal
> cpu number, -1 or "unbound". A user assignment, unbound included, is
> marked in the queue flags, persists across reconnects and is never
> reselected by the driver. The read-only managed attribute reports 1
> while io_cpu is driver managed and 0 once the user assigned it. The
> accounting of nvme_tcp_cpu_queues applies regardless of who set the
> io_cpu.
> 
> A queue's directory is added when the queue is allocated and removed
> when it is freed, before the queue is torn down, so a sysfs store
> never runs on a freed queue. The queues directory is created before
> the I/O queues are set up in parallel. All io_cpu writes, from sysfs
> and from the driver, serialize with the accounting under
> nvme_tcp_cpu_queues_lock, while the I/O paths read io_cpu locklessly.
> The queue stop now clears NVME_TCP_Q_IO_CPU_SET under that lock, since
> a sysfs store may move io_cpu in between.
> 
> nvme_tcp_free_queue() now clears NVME_TCP_Q_IO_CPU_SET and drops its
> count, because a queue can still be counted when it is freed. A sysfs
> store may have run after the queue stop, or the connect may have
> failed, and the setup error path frees such a queue without calling
> nvme_tcp_stop_queue(), where the count is normally dropped.
> 
> Suggested-by: Sagi Grimberg <sagi@grimberg.me>
> Link: https://lore.kernel.org/linux-nvme/13c160d0-0657-47ce-9a16-58d348267ab0@grimberg.me/
> Assisted-by: Claude:claude-opus-5-5 [Claude Code]
> Signed-off-by: Saravanan D <saravanand@crusoe.ai>
> ---
>   Documentation/ABI/stable/sysfs-nvme |  29 +++++
>   drivers/nvme/host/tcp.c             | 173 ++++++++++++++++++++++++----
>   2 files changed, 182 insertions(+), 20 deletions(-)
> 
I think the 'managed' sysfs attribute needs to be per controller, as it
would be extremely awkward to allow some queues to be managed from the
admin and some queues automatically placed.

And I think you need to check for interactions with the existing 
'wq_unbound' module option; both claim to do similar things so we
would need to clarify the details here.

And should be we able to specify a cpu _set_ here?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                  Kernel Storage Architect
hare@suse.de                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich

Re: [PATCH v5 2/2] nvme-tcp: allow setting per-queue io_cpu through sysfs

[Saravanan D]

On 10/21/26 12:28 AM, Hannes Reinecke wrote:
[...]
> I think the 'managed' sysfs attribute needs to be per controller, as it
> would be extremely awkward to allow some queues to be managed from the
> admin and some queues automatically placed.

The mix is intentional, and it is the case this series is for. On a
host that partitions its CPUs between tenants, a queue only needs a
user assigned io_cpu when its blk-mq map group straddles two tenants'
cpusets. On our 384 cpu hosts with 128 queue controllers that is one
or two queues per controller per VM, and the driver's least loaded
selection already places the others well. With ownership per
controller, the control plane would have to take over every queue of
the controller to fix those one or two, and reimplement the least
loaded selection for all the others.

Per queue granularity is also where Sagi steered the series after the
earlier per controller connect option, since the CPU partitioning
changes after connect time and only some queues are affected [1].
Sagi, does the per queue model still match what you had in mind?

> And I think you need to check for interactions with the existing
> 'wq_unbound' module option; both claim to do similar things so we
> would need to clarify the details here.

They are independent. wq_unbound picks the workqueue type for all of
nvme-tcp at module load, and when it is set the driver never selects an
io_cpu, so every driver managed queue stays unbound. io_cpu places the
work of a single queue. When both are used, a written io_cpu selects the
worker pool for that cpu, and how closely the work follows it depends on
the affinity_scope of nvme_tcp_wq. To keep the work on exactly that cpu,
nvme_tcp_wq needs affinity_scope=cpu and affinity_strict=1, and the cpu
has to be in its cpumask. The ABI entry in this patch documents that,
and I can expand the text in v6 if it reads ambiguous.

> And should be we able to specify a cpu _set_ here?

For this series I would keep a single cpu. io_work is one work item
queued with queue_work_on(), which takes a single cpu, so honoring a
set would need a separate unbound workqueue per queue with its own
cpumask. A single cpu inside the owning tenant's cpuset covers the
case above, and a cpumask could be added later as a separate attribute
without changing io_cpu.

[1] https://lore.kernel.org/linux-nvme/13c160d0-0657-47ce-9a16-58d348267ab0@grimberg.me/

Thanks,
Saravanan D.

Re: [PATCH v5 2/2] nvme-tcp: allow setting per-queue io_cpu through sysfs

[Saravanan D]

On Sun, Oct 18, 2026 at 01:15:14AM +0000, sashiko-bot@kernel.org wrote:
[...]
> [Severity: Medium]
> In nvme_tcp_setup_queue_async(), nvme_tcp_alloc_queue() runs before
> nvme_tcp_start_queue(), where nvme_tcp_set_queue_io_cpu() is called:
>
> nvme_tcp_setup_queue_async()
>   nvme_tcp_alloc_queue()
>     if (!test_bit(NVME_TCP_Q_IO_CPU_USER, &queue->flags))
>       nvme_tcp_queue_change_io_cpu(queue, WORK_CPU_UNBOUND);
>     ...
>     nvme_tcp_add_queue_sysfs(queue);
>   nvme_tcp_start_queue()
>     if (idx) {
>       nvme_tcp_set_queue_io_cpu(queue);
>       ret = nvmf_connect_io_queue(nctrl, idx);
>     }
>
> Because nvme_tcp_add_queue_sysfs() publishes the sysfs attributes in
> nvme_tcp_alloc_queue() before nvme_tcp_set_queue_io_cpu() runs in
> nvme_tcp_start_queue(), can a userspace control plane reading io_cpu and
> managed as soon as the queue directory appears observe io_cpu as -1
> (unbound) with managed=1 even when wq_unbound is false?

Agreed. v6 adds the queue directory in nvme_tcp_start_queue(), right
after nvme_tcp_set_queue_io_cpu(), instead of at the end of
nvme_tcp_alloc_queue(), so io_cpu and managed only become visible once
the selection has run. The directory is still removed in
nvme_tcp_free_queue(), which the setup error path calls for every
allocated queue, so a failed connect does not leave it behind.

> Also, during parallel reconnects in nvme_tcp_setup_io_queues(), a queue
> with NVME_TCP_Q_IO_CPU_USER set does not re-increment
> nvme_tcp_cpu_queues[io_cpu] until it reaches nvme_tcp_set_queue_io_cpu()
> after completing the blocking TCP, TLS, and ICReq handshakes in
> nvme_tcp_alloc_queue(). If a driver-managed queue finishes its handshake
> first, could nvme_tcp_set_queue_io_cpu() pick a CPU without accounting for
> the user-assigned queue's load?

This window exists for every queue, not only user assigned ones,
since a queue is counted only once its handshakes complete and it
reaches its own selection. The selection is best effort, and the user
assigned queue keeps its io_cpu regardless, so I'd rather not add
accounting to nvme_tcp_alloc_queue() and its error path for it.

Thanks,
Saravanan D.