mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Robert Lin <robelin@nvidia.com>
To: Jens Axboe <axboe@kernel.dk>
Cc: <linux-block@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	Robert Lin <robelin@nvidia.com>
Subject: [PATCH] blk-mq: Always rebuild the queue map in blk_mq_update_nr_hw_queues()
Date: Wed, 11 Nov 2026 11:21:54 +0800	[thread overview]
Message-ID: <20261111032154.49207-1-robelin@nvidia.com> (raw)

__blk_mq_update_nr_hw_queues() returns early for a single-map tag set
when the number of hardware queues is unchanged, on the assumption that
the map can then not have changed either. That does not hold for
drivers whose ->map_queues() follows managed IRQ affinity.

NVMe re-allocates its MSI-X vectors on every controller reset, including
suspend/resume, and the affinity masks are recomputed from the CPU
topology at that time. If CPUs went offline after the vectors were first
allocated, the pairing of vectors and CPUs can change while the number
of queues stays the same: the topology masks of offline CPUs collapse to
a single CPU, which changes how group_cpus_evenly() orders the groups.

That ordering depends on the CPU topology since commit 89802ca36c96
("lib/group_cpus: make group CPU cluster aware"), merged in v7.0-rc1,
which groups CPUs by cluster.

The stale CPU -> hctx map then sends I/O from a CPU to a queue whose
interrupt is affine only to offline CPUs. That managed interrupt is shut
down, the completion interrupt is never delivered, and the request only
completes through the I/O timeout ("completion polled").

Observed on Jetson Orin (6 present CPUs, 6 NVMe I/O queues) after
offlining CPU4 and CPU5 shortly after boot. After a controller reset the
vectors for queues 1 and 2 are affine to CPU4 and CPU5 while blk-mq still
maps CPU0 and CPU1 to them, so direct reads submitted from CPU0 and CPU1
take 23-30 seconds. The same state can be reached without suspend by
offlining CPU4 and CPU5 and writing 1 to reset_controller.

The queue count alone cannot tell whether the mapping changed, so drop
the early return and always go through the full update, which freezes
the queues and rebuilds the map. Tag sets with more than one map
already take this path unconditionally. The extra cost is a queue freeze
and hctx re-allocation when the queue count is unchanged, which only
happens on controller reset/reconnect and similar slow paths.

Link: https://lore.kernel.org/lkml/20260113022958.3379650-1-wangyang.guo@intel.com/
Signed-off-by: Robert Lin <robelin@nvidia.com>
---
 block/blk-mq.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 061c8ef4484a..cbedd8f8ed61 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -5132,8 +5132,6 @@ static void __blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set,
 		nr_hw_queues = nr_cpu_ids;
 	if (nr_hw_queues < 1)
 		return;
-	if (set->nr_maps == 1 && nr_hw_queues == set->nr_hw_queues)
-		return;
 
 	memflags = memalloc_noio_save();
 
-- 
2.34.1


                 reply	other threads:[~2026-11-11  3:22 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261111032154.49207-1-robelin@nvidia.com \
    --to=robelin@nvidia.com \
    --cc=axboe@kernel.dk \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®