[PATCH] nvme: wait for SRCU readers before splicing the namespace list

[PATCH] nvme: wait for SRCU readers before splicing the namespace list

[Qiliang Yuan]

Readers walk ctrl->namespaces under ctrl->srcu, and some of them sleep
there, e.g. nvme_wait_freeze() waiting for I/O to drain.

nvme_remove_namespaces() moves the list onto its on-stack ns_list with
list_splice_init_rcu(), which only waits for RCU readers before
relinking the entries. A reader still on the last namespace then follows
its next pointer to ns_list instead of ctrl->namespaces and treats the
stack as a namespace. Deleting a controller while a Format NVM
passthrough waits for a delayed read on the last namespace gives:

  BUG: KASAN: stack-out-of-bounds in nvme_wait_freeze+0x102/0x120
  Read of size 8 at addr ff110001d2f3fc38 by task nvme/4413
  Call Trace:
   nvme_wait_freeze
   nvme_passthru_start
   nvme_submit_user_cmd
   nvme_user_cmd
   nvme_dev_ioctl
  BUG: unable to handle page fault for address: ffffffff8af04850

Open-code the splice and wait for SRCU readers instead. This also takes
the RCU grace period out of every controller deletion:

  64 tcp controllers, 1 namespace each   before        after
  nvme disconnect-all (ms)               3543 / 3474   3221 / 3146
  nvme_remove_namespaces() avg (ms)      39.6 / 38.9   34.4 / 33.7

Fixes: be647e2c76b2 ("nvme: use srcu for iterating namespace list")
Cc: stable@vger.kernel.org
Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
---
 drivers/nvme/host/core.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index a7a4d09644c13..8f032b3edde9a 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -4754,10 +4754,25 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 	/* this is a no-op when called from the controller reset handler */
 	nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING_NOIO);
 
+	/*
+	 * Readers walk the list under ctrl->srcu, so hide the namespaces from
+	 * new readers and wait for the current ones before moving them over.
+	 * list_splice_init_rcu() can't be used as it only waits for RCU
+	 * readers.
+	 */
 	mutex_lock(&ctrl->namespaces_lock);
-	list_splice_init_rcu(&ctrl->namespaces, &ns_list, synchronize_rcu);
+	if (!list_empty(&ctrl->namespaces)) {
+		struct list_head *first = ctrl->namespaces.next;
+		struct list_head *last = ctrl->namespaces.prev;
+
+		INIT_LIST_HEAD_RCU(&ctrl->namespaces);
+		synchronize_srcu(&ctrl->srcu);
+		ns_list.next = first;
+		ns_list.prev = last;
+		first->prev = &ns_list;
+		last->next = &ns_list;
+	}
 	mutex_unlock(&ctrl->namespaces_lock);
-	synchronize_srcu(&ctrl->srcu);
 
 	list_for_each_entry_safe(ns, next, &ns_list, list)
 		nvme_ns_remove(ns);

---
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
change-id: 20261102-bug-nvme-remove-ns-srcu-93d8a75a9b1b

Best regards,
-- 
Qiliang Yuan <odys.yuan@gmail.com>

Re: [PATCH] nvme: wait for SRCU readers before splicing the namespace list

[Christoph Hellwig]

On Tue, Nov 03, 2026 at 01:30:35AM -0500, Qiliang Yuan wrote:
> Readers walk ctrl->namespaces under ctrl->srcu, and some of them sleep
> there, e.g. nvme_wait_freeze() waiting for I/O to drain.
> 
> nvme_remove_namespaces() moves the list onto its on-stack ns_list with
> list_splice_init_rcu(), which only waits for RCU readers before
> relinking the entries. A reader still on the last namespace then follows
> its next pointer to ns_list instead of ctrl->namespaces and treats the
> stack as a namespace. Deleting a controller while a Format NVM
> passthrough waits for a delayed read on the last namespace gives:
> 
>   BUG: KASAN: stack-out-of-bounds in nvme_wait_freeze+0x102/0x120
>   Read of size 8 at addr ff110001d2f3fc38 by task nvme/4413
>   Call Trace:
>    nvme_wait_freeze
>    nvme_passthru_start
>    nvme_submit_user_cmd
>    nvme_user_cmd
>    nvme_dev_ioctl
>   BUG: unable to handle page fault for address: ffffffff8af04850
> 
> Open-code the splice and wait for SRCU readers instead. This also takes
> the RCU grace period out of every controller deletion:

This feels like a design issue with list_splice_init_rcu or rather
using list_splice_init_rcu with SRCU.  Do we need to encapsulate
and verify this logic in a list_splice_init_srcu so that other
potential users won't get it wrong again?

> 
>   64 tcp controllers, 1 namespace each   before        after
>   nvme disconnect-all (ms)               3543 / 3474   3221 / 3146
>   nvme_remove_namespaces() avg (ms)      39.6 / 38.9   34.4 / 33.7
> 
> Fixes: be647e2c76b2 ("nvme: use srcu for iterating namespace list")
> Cc: stable@vger.kernel.org
> Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
> ---
>  drivers/nvme/host/core.c | 19 +++++++++++++++++--
>  1 file changed, 17 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index a7a4d09644c13..8f032b3edde9a 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -4754,10 +4754,25 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
>  	/* this is a no-op when called from the controller reset handler */
>  	nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING_NOIO);
>  
> +	/*
> +	 * Readers walk the list under ctrl->srcu, so hide the namespaces from
> +	 * new readers and wait for the current ones before moving them over.
> +	 * list_splice_init_rcu() can't be used as it only waits for RCU
> +	 * readers.
> +	 */
>  	mutex_lock(&ctrl->namespaces_lock);
> -	list_splice_init_rcu(&ctrl->namespaces, &ns_list, synchronize_rcu);
> +	if (!list_empty(&ctrl->namespaces)) {
> +		struct list_head *first = ctrl->namespaces.next;
> +		struct list_head *last = ctrl->namespaces.prev;
> +
> +		INIT_LIST_HEAD_RCU(&ctrl->namespaces);
> +		synchronize_srcu(&ctrl->srcu);
> +		ns_list.next = first;
> +		ns_list.prev = last;
> +		first->prev = &ns_list;
> +		last->next = &ns_list;
> +	}
>  	mutex_unlock(&ctrl->namespaces_lock);
> -	synchronize_srcu(&ctrl->srcu);
>  
>  	list_for_each_entry_safe(ns, next, &ns_list, list)
>  		nvme_ns_remove(ns);
> 
> ---
> base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
> change-id: 20261102-bug-nvme-remove-ns-srcu-93d8a75a9b1b
> 
> Best regards,
> -- 
> Qiliang Yuan <odys.yuan@gmail.com>
---end quoted text---

Re: [PATCH] nvme: wait for SRCU readers before splicing the namespace list

[Qiliang Yuan]

On Tue, Nov 03, 2026 at 09:14:18AM +0100, Christoph Hellwig wrote:
> This feels like a design issue with list_splice_init_rcu or rather
> using list_splice_init_rcu with SRCU.  Do we need to encapsulate
> and verify this logic in a list_splice_init_srcu so that other
> potential users won't get it wrong again?

Makes sense. nvme is the only caller that pairs it with SRCU today; the
others (btrfs, apparmor, ima, smack) protect their lists with plain RCU.

I'll add list_splice_init_srcu(), which takes the srcu_struct to wait
on, as a separate patch next to list_splice_init_rcu(), and switch nvme
to it in v2.

Thanks,
Qiliang