* [PATCH v16 0/9] blk: honor isolcpus configuration
@ 2026-09-10 16:42 Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 1/9] scsi: aacraid: use block layer helpers to calculate num of queues Aaron Tomlin
` (9 more replies)
0 siblings, 10 replies; 11+ messages in thread
From: Aaron Tomlin @ 2026-09-10 16:42 UTC (permalink / raw)
To: axboe, tglx, aacraid, James.Bottomley, mkp, frederic, bigeasy
Cc: atomlin, ionut.nechita, corbet, vincent.guittot, mingo, peterz,
radu, akpm, steve, sean, chjohnst, neelx, mproche, nick.lange,
marco.crivellari, rishil1999, linux-doc, linux-block, linux-scsi,
linux-kernel
Hi Thomas, Jens, Sebastian, Frederic,
This is version 16 of the series enabling multiqueue devices and managed
interrupts to adhere to CPU isolation constraints. The series has been
rebased on v7.3-rc1-708-g893e11787f78.
Background and Problem
======================
Modern multiqueue storage controllers (e.g. NVMe devices) typically
allocate as many hardware queues and interrupt vectors as there are CPUs in
the system. When CPU isolation is requested via isolcpus, the kernel's
current mechanism "isolcpus=managed_irq" relies on post-allocation affinity
steering inside kernel/irq/manage.c. However, because drivers allocate
queues and vectors before affinity steering takes place, the number of
vectors frequently matches or exceeds the total CPU count. In such
scenarios, the post-allocation steering logic cannot avoid assigning
managed interrupts to isolated CPUs, effectively defeating CPU isolation
for real-time and latency-critical workloads.
Proposed solution and architecture
==================================
This series introduces a new CPU isolation feature,
"isolcpus=managed_irq_strict", designed to protect isolated cores from the
disruptive hardware interrupts generated by high-performance multiqueue
devices.
To prevent I/O stalls, the block layer is additionally hardened to reject
hotplug requests that attempt to offline a housekeeping CPU if it is the
last remaining CPU actively serving an online isolated core.
The complex "top-down" mask plumbing introduced in earlier revisions
(version 12), which modified struct irq_affinity and expanded block layer
APIs, has been abandoned. It is replaced by a centralised approach: direct
isolation querying via housekeeping_cpumask(HK_TYPE_MANAGED_IRQ_STRICT)
within the genirq/affinity subsystem. This architectural simplification
successfully decouples core changes from driver-specific implementations.
In version 15, this was addressed under the previously named
"isolcpus=io_queue". During subsequent review, Sebastian Andrzej Siewior
pointed out that the existing "isolcpus=managed_irq" implementation is
fundamentally broken on modern multiqueue hardware, and raised the question
of whether we should introduce another flag at all, or instead fix
"managed_irq" directly in place. As Sebastian observed, if
backward-compatibility concerns prevent modifying the legacy behaviour, a
"managed_irq_strict" designation represents the logical counterpart.
To make progress while keeping backwards compatibility completely safe,
this iteration adopts option 2: we drop the "io_queue" naming in favour of
"isolcpus=managed_irq_strict". It acts as the strict, pre-allocation
counterpart to "managed_irq", constraining both block layer queue mappings
and generic interrupt affinity masks exclusively to housekeeping CPUs.
Furthermore, if both options are supplied, "managed_irq_strict" takes
precedence.
Maintainer guidance requested
=============================
We would appreciate guidance from Frederic Weisbecker regarding preference
on this front:
1. Do you prefer keeping "managed_irq_strict" as an explicit
opt-in (as implemented in this series) to preserve historical
behaviour for existing users of "managed_irq"?
2. Or would you prefer upgrading the semantics of "managed_irq"
directly in place without introducing a new parameter, on the
grounds that the legacy post-allocation behaviour is already
broken for modern multiqueue hardware?
Please let me know your thoughts.
Changes since v15:
- Migrated feature naming from "isolcpus=io_queue" (HK_TYPE_IO_QUEUE) to
"isolcpus=managed_irq_strict" (HK_TYPE_MANAGED_IRQ_STRICT), establishing
it as the strict, pre-allocation counterpart to "managed_irq"
- Specifying "managed_irq_strict" automatically enables
HK_FLAG_MANAGED_IRQ so that dependent subsystems continue to function
- Cleaned up commit bodies across the series, stripping informal
scratchpads and adding Co-developed-by attribution where appropriate
- Added new patch 4/9 ("sched/isolation: Prevent out-of-bounds read in
isolcpus= boot parameter parser"). Incorporated prerequisite bugfix
resolving an out-of-bounds read in housekeeping_isolcpus_setup() when
sub-parameters lack trailing commas
- Defined HK_TYPE_MANAGED_IRQ_STRICT and HK_FLAG_MANAGED_IRQ_STRICT.
Ensured HK_FLAG_MANAGED_IRQ is set when managed_irq_strict is enabled
- Decoupled queue mapping from dynamic cpu_online_mask; eliminated the
blk_mq_validate() check and online cpumask snapshotting to avoid TOCTOU
races during concurrent CPU hotplug events
- Added cpuhp_tasks_frozen check to ensure suspend, hibernation, and
resume transitions are not aborted
- Added blk_mq_map_queue_type() check so CPU offline validation only
evaluates hardware contexts actively mapped to the CPU being offlined
- Enhanced warning message to report the offending CPU ID and queue
number
- Prioritised affd->calc_sets in irq_calc_affinity_vectors() so multi-set
drivers scale vector allocations with requested set sizes to prevent
cross-queue contention, while vector spreading remains strictly
constrained to housekeeping cores
- Documented "managed_irq_strict" in kernel-parameters.txt, detailing its
role in blk-mq queue distribution and genirq affinity masking, as well
as its superset relationship with legacy "managed_irq"
- Link to v15: https://lore.kernel.org/lkml/20260521232956.553287-1-atomlin@atomlin.com/
Changes since v14:
- Fixed a division-by-zero by ensuring group_mask_cpus_evenly() safely
frees its allocations and returns NULL instead of an empty array if the
provided mask yields zero groups.
- Fixed a device probe -ENOSPC regression in blk_mq_num_queues(). If the
housekeeping mask intersection evaluated to 0 (e.g., against a localised
NUMA node), min_not_zero() would erroneously return the absolute maximum
hardware queues. The result is now safely clamped to a minimum of 1.
- Added a mapping verification check to prevent unrelated housekeeping
CPUs from aborting the global hotplug offline sequence.
- Aligned the pr_warn format specifier with the unsigned int declaration
of hctx->queue_num in blk_mq_hctx_can_offline_hk_cpu().
- Link to v14: https://lore.kernel.org/lkml/20260520215030.496803-1-atomlin@atomlin.com/
Changes since v13:
- Removed ineffective data_race() annotations around mask and
cpu_present_mask pointers. Wrapping the pointers failed to suppress
KCSAN warnings for the underlying inline bitmap memory accesses.
- Fixed a silent validation bypass in blk_mq_map_hw_queues() caused by
overlapping IRQ affinity masks by removing the short-circuiting
optimisation and evaluating the active_hctx bitmap in a secondary pass.
- Restored topology-aware multi-queue fallback by correctly routing
missing IRQ affinity masks to the map_software path instead of the naive
map-all fallback.
- Dropped hctx->queue->disk->disk_name from warning to avoid a UAF.
- Fixed an isolation leak where excess allocated hardware queues were
improperly padded with irq_default_affinity. Because these queues are
marked as managed, they bypassed user-space IRQ balancing; they are now
safely padded with the housekeeping mask.
- Enforced the housekeeping vector cap prior to evaluating driver-provided
calc_sets() callbacks, preventing modern multi-queue drivers from
bypassing the cap and wasting memory on dead queues.
- Introduced a safety net to the vector calculation to prevent fatal
-ENOSPC device probe aborts on heavily isolated systems where the
housekeeping CPU count is lower than the device's structural minimum.
- Removed an inaccurate claim stating that the io_queue isolation flag
takes precedence over managed_irq. Both flags are parsed, evaluated, and
enforced entirely independently by their respective subsystems.
- Link to v13: https://lore.kernel.org/lkml/20260513005509.135966-1-atomlin@atomlin.com/
Changes since v12:
- Resolved TOCTOU race conditions against CPU hotplug events in
blk_mq_map_queues() and group_mask_cpus_evenly() by taking lockless
snapshots of the online CPU mask prior to algorithmic evaluation.
- Migrated the active_hctx tracking to a dynamically sized bitmap
(bitmap_zalloc), resolving a critical out-of-bounds memory write that
occurred when hardware queues exceeded the system CPU count.
- Wrapped the disk pointer fetch in blk_mq_hctx_can_offline_hk_cpu() with
READ_ONCE() to prevent a TOCTOU NULL pointer dereference against
concurrent device teardowns.
- Introduced bitmap_empty() checks to prevent the mapping logic from
routing unassigned CPUs into unallocated memory when all mapped CPUs are
offline, safely forcing a fallback mapping instead.
- Implemented a native two-stage distribution logic in
group_mask_cpus_evenly() that first prioritises physically present CPUs
to prevent I/O starvation before distributing remaining vectors to
non-present CPUs for hotplug safety.
- Restricted the maximum number of allocated vectors in
irq_calc_affinity_vectors() to the weight of the housekeeping mask,
preventing drivers from wasting memory on dead hardware queues that
physically cannot be routed.
- Added padding logic using irq_default_affinity for sets where isolation
constraints yield fewer masks than requested vectors, preserving the 1:1
hardware queue mapping sequence for subsequent sets.
- Fixed a logic flaw that prematurely rejected valid offline requests by
manually iterating over cpu_online_mask and reverse-mapping to
accurately detect isolated CPUs, properly permitting the offlining of
non-housekeeping CPUs.
- Corrected an absolute versus relative queue index calculation bug in
blk_mq_map_queues() that was overwriting loop iterations, by iterating
directly over the generated masks.
- Replaced scoped __free cleanups with traditional goto unwinding in the
block layer to align with subsystem styling guidelines.
- Refined the io_queue kernel command-line parameter documentation for
better clarity and precision.
- Link to v12: https://lore.kernel.org/lkml/20260422185215.100929-1-atomlin@atomlin.com/
Changes since v11:
- Removed duplicate paragraph from the commit message in patch 11
(Marco Crivellari)
- Ensure ZERO_SIZE_PTR is not returned by group_mask_cpus_evenly()
(Marco Crivellari)
- Link to v11: https://lore.kernel.org/lkml/20260416192942.1243421-1-atomlin@atomlin.com/
Changes since v10:
- Completely rewrote the isolcpus=io_queue documentation in
Documentation/admin-guide/kernel-parameters.txt to clarify its exclusive
application to managed IRQs, queue allocation limits, vector exhaustion
prevention, and hardware interrupt routing (Ming Lei)
- Fixed a stack frame bloat issue by avoiding the on-stack declaration of
struct cpumask (Waiman Long)
- Link to v10: https://lore.kernel.org/linux-nvme/20260401222312.772334-1-atomlin@atomlin.com/
Changes since v9:
- Fixed a page fault regression encountered when initialising secondary
queue maps (e.g. NVMe poll queues). Restored the qmap->queue_offset to
the mq_map assignment to ensure CPUs are strictly mapped to absolute
hardware indices (Keith Busch)
- Corrected the active_hctx tracker to utilise relative queue indices,
preventing out-of-bounds mask assignments
- Fixed the blk_mq_validate() sanity check to properly evaluate absolute
queue indices against the offset-adjusted loop index
- Corrected typographical errors within block/blk-mq-cpumap.c
(Keith Busch)
- Clarified the commit message regarding the removal of the !SMP fallback
code, explicitly noting that the core scheduler now mandates SMP
unconditionally (Sebastian Andrzej Siewior)
- Added missing "Signed-off-by:" tags to properly record the patch series
chain of custody
- Link to v9: https://lore.kernel.org/lkml/20260330221047.630206-1-atomlin@atomlin.com/
Changes since v8:
- Added "Reviewed-by:" tags
- Introduced irq_spread_hk_filter() to safely restrict managed IRQ
affinity to housekeeping CPUs (Thomas Gleixner)
- Removed the unsafe global static variable blk_hk_online_mask from
blk-mq-cpumap.c and blk-mq.c. blk_mq_online_queue_affinity() now returns
a stable pointer, delegating safe intersection to the callers to prevent
concurrent modification races (Thomas Gleixner, Hannes Reinecke)
- Resolved BUG: kernel NULL pointer dereference in __blk_mq_all_tag_iter
reported by the kernel test robot during cpuhotplug rcutorture stress
testing
- Link to v8: https://lore.kernel.org/lkml/20250905-isolcpus-io-queues-v8-0-885984c5daca@kernel.org/
Changes since v7:
- Added commit 524f5eea4bbe ("lib/group_cpus: remove !SMP code")
- Merged the new mapping logic directly into the existing function to
avoid special casing
- Refined the group_mask_cpus_evenly() implementation with the following
updates:
- Corrected the function name typo (changed group_masks_cpus_evenly to
group_mask_cpus_evenly)
- Updated the documentation comment to accurately reflect the function's
behavior
- Renamed the cpu_mask argument to mask for consistency
- Added a new patch for aacraid to include the missing number of queues
calculation
- Restricted updates to only affect SCSI drivers that support
PCI_IRQ_AFFINITY and do not utilise nvme-fabrics
- Removed the __free cleanup attribute usage for cpumask_var_t allocations
due to compatibility issues
- Updated the documentation to explicitly highlight the limitations
surrounding CPU offlining
- Collected accumulated Reviewed-by and Acked-by tags
- Link to v7: https://patch.msgid.link/20250702-isolcpus-io-queues-v7-0-557aa7eacce4@kernel.org
Changes since v6:
- Sent out the first part of the series independently:
https://lore.kernel.org/all/20250617-isolcpus-queue-counters-v1-0-13923686b54b@kernel.org/
- Added comprehensive kernel command-line documentation
- Added validation logic to ensure the resulting CPU-to-queue mapping is
fully operational
- Rewrote the isolcpus mapping code to properly account for active
hardware contexts (hctx)
- Introduced blk_mq_map_hk_irq_queues, which utilizes the mask retrieved
from irq_get_affinity()
- Refactored blk_mq_map_hk_queues to require the caller to explicitly test
for HK_TYPE_MANAGED_IRQ
- Link to v6: https://patch.msgid.link/20250424-isolcpus-io-queues-v6-0-9a53a870ca1f@kernel.org
Changes since v5:
- Reintroduced the io_queue type for the isolcpus kernel parameter
- Prevented the offlining of a housekeeping CPU if an isolated CPU is
still present, upgrading this behavior from a simple warning to a hard
restriction
- Link to v5: https://lore.kernel.org/r/20250110-isolcpus-io-queues-v5-0-0e4f118680b0@kernel.org
Changes since v4:
- Rebased the series onto the latest for-6.14/block branch.
- Updated the documentation regarding the managed_irq parameters
- Reworded the commit message for "blk-mq: issue warning when offlining
hctx with online isolcpus" for better clarity
- Split the input and output parameters in the patch "lib/group_cpus: let
group_cpu_evenly return number of groups"
- Dropped the patch "sched/isolation: document HK_TYPE housekeeping
option"
- Link to v4: https://lore.kernel.org/r/20241217-isolcpus-io-queues-v4-0-5d355fbb1e14@kernel.org
Changes since v3:
- Added the patch "blk-mq: issue warning when offlining hctx with online
isolcpus"
- Fixed the check in group_cpus_evenly(); the condition now properly uses
housekeeping_enabled() instead of cpumask_weight(), as the latter always
returns a valid mask
- Dropped the Fixes: tag from "lib/group_cpus.c: honor housekeeping config
when grouping CPUs"
- Fixed an overlong line warning in the patch "scsi: use block layer
helpers to calculate num of queues"
- Dropped the patch "sched/isolation: Add io_queue housekeeping option" in
favor of simply documenting the housekeeping hk_type enum
- Added the patch "lib/group_cpus: let group_cpu_evenly return number of
groups"
- Collected accumulated Reviewed-by and Acked-by tags
- Split the patchset by moving foundational changes into a separate
preparation series:
https://lore.kernel.org/linux-nvme/20241202-refactor-blk-affinity-helpers-v6-0-27211e9c2cd5@kernel.org/
- Link to v3: https://lore.kernel.org/r/20240806-isolcpus-io-queues-v3-0-da0eecfeaf8b@suse.de
Changes since v2:
- Integrated patches from Ming Lei
(https://lore.kernel.org/all/20210709081005.421340-1-ming.lei@redhat.com/):
"virtio: add APIs for retrieving vq affinity" and "blk-mq: introduce
blk_mq_dev_map_queues"
- Replaced all instances of blk_mq_pci_map_queues and
blk_mq_virtio_map_queues with the new unified blk_mq_dev_map_queues
- Updated and expanded the helper functions used for calculating the
number of queues
- Added the CPU-to-hctx mapping function specifically to support the
isolcpus=io_queue parameter
- Documented the hk_type enum and the newly introduced isolcpus=io_queue
parameter
- Added the patch "scsi: pm8001: do not overwrite PCI queue mapping"
- Link to v2: https://lore.kernel.org/r/20240627-isolcpus-io-queues-v2-0-26a32e3c4f75@suse.de
Changes since v1:
- Updated the feature documentation for clarity and completeness
- Split the blk/nvme-pci patch into smaller, logical commits
- Dropped the HK_TYPE_IO_QUEUE macro in favor of reusing
HK_TYPE_MANAGED_IRQ
- Link to v1: https://lore.kernel.org/r/20240621-isolcpus-io-queues-v1-0-8b169bf41083@suse.de
Aaron Tomlin (2):
sched/isolation: Prevent out-of-bounds read in isolcpus= boot
parameter parser
genirq/affinity: Restrict managed IRQ affinity to housekeeping CPUs
Daniel Wagner (7):
scsi: aacraid: use block layer helpers to calculate num of queues
lib/group_cpus: remove dead !SMP code
lib/group_cpus: Add group_mask_cpus_evenly()
isolation: Introduce managed_irq_strict isolcpus type
blk-mq: use hk cpus only when isolcpus=managed_irq_strict is enabled
blk-mq: prevent offlining hk CPUs with associated online isolated CPUs
docs: add managed_irq_strict flag to isolcpus
.../admin-guide/kernel-parameters.txt | 37 +++-
Documentation/core-api/housekeeping.rst | 6 +-
block/blk-mq-cpumap.c | 163 ++++++++++++++++--
block/blk-mq.c | 63 +++++++
drivers/scsi/aacraid/comminit.c | 3 +-
include/linux/group_cpus.h | 3 +
include/linux/sched/isolation.h | 1 +
kernel/irq/affinity.c | 29 +++-
kernel/sched/isolation.c | 18 ++
lib/group_cpus.c | 112 ++++++++++--
10 files changed, 395 insertions(+), 40 deletions(-)
--
2.55.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v16 1/9] scsi: aacraid: use block layer helpers to calculate num of queues
2026-09-10 16:42 [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
@ 2026-09-10 16:42 ` Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 2/9] lib/group_cpus: remove dead !SMP code Aaron Tomlin
` (8 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Aaron Tomlin @ 2026-09-10 16:42 UTC (permalink / raw)
To: axboe, tglx, aacraid, James.Bottomley, mkp, frederic, bigeasy
Cc: atomlin, ionut.nechita, corbet, vincent.guittot, mingo, peterz,
radu, akpm, steve, sean, chjohnst, neelx, mproche, nick.lange,
marco.crivellari, rishil1999, linux-doc, linux-block, linux-scsi,
linux-kernel
From: Daniel Wagner <wagi@kernel.org>
The calculation of the upper limit for queues does not depend solely on
the number of online CPUs; for example, the isolcpus kernel
command-line option must also be considered.
To account for this, the block layer provides a helper function to
retrieve the maximum number of queues. Use it to set an appropriate
upper queue number limit.
This patch brings aacraid in line with the API migration initiated for
other SCSI drivers in commit 94970cfb5f10 ("scsi: use block layer
helpers to calculate num of queues").
Signed-off-by: Daniel Wagner <wagi@kernel.org>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
drivers/scsi/aacraid/comminit.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/drivers/scsi/aacraid/comminit.c b/drivers/scsi/aacraid/comminit.c
index 9bd3f5b868bc..ec165b57182d 100644
--- a/drivers/scsi/aacraid/comminit.c
+++ b/drivers/scsi/aacraid/comminit.c
@@ -469,8 +469,7 @@ void aac_define_int_mode(struct aac_dev *dev)
}
/* Don't bother allocating more MSI-X vectors than cpus */
- msi_count = min(dev->max_msix,
- (unsigned int)num_online_cpus());
+ msi_count = blk_mq_num_online_queues(dev->max_msix);
dev->max_msix = msi_count;
--
2.55.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v16 2/9] lib/group_cpus: remove dead !SMP code
2026-09-10 16:42 [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 1/9] scsi: aacraid: use block layer helpers to calculate num of queues Aaron Tomlin
@ 2026-09-10 16:42 ` Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 3/9] lib/group_cpus: Add group_mask_cpus_evenly() Aaron Tomlin
` (7 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Aaron Tomlin @ 2026-09-10 16:42 UTC (permalink / raw)
To: axboe, tglx, aacraid, James.Bottomley, mkp, frederic, bigeasy
Cc: atomlin, ionut.nechita, corbet, vincent.guittot, mingo, peterz,
radu, akpm, steve, sean, chjohnst, neelx, mproche, nick.lange,
marco.crivellari, rishil1999, linux-doc, linux-block, linux-scsi,
linux-kernel
From: Daniel Wagner <wagi@kernel.org>
The core scheduler recently transitioned to compiling SMP data
structures unconditionally to reduce code complexity - see commit
cac5cefbade9 ("sched/smp: Make SMP unconditional").
In alignment with this philosophy of reducing dual-path maintenance,
this patch removes the #ifdef CONFIG_SMP guards and the dedicated !SMP
fallback logic here.
While the !SMP path provided a slightly simpler execution flow for
uniprocessor kernels (avoiding SMP-specific overhead), maintaining these
separate code paths adds unnecessary complexity and testing burden.
Removing these guards simplifies the codebase by standardizing entirely
on the SMP logic, which safely resolves to single-CPU operations on UP
configurations.
Signed-off-by: Daniel Wagner <wagi@kernel.org>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
lib/group_cpus.c | 20 --------------------
1 file changed, 20 deletions(-)
diff --git a/lib/group_cpus.c b/lib/group_cpus.c
index e6e18d7a49bb..b8d54398f88a 100644
--- a/lib/group_cpus.c
+++ b/lib/group_cpus.c
@@ -9,8 +9,6 @@
#include <linux/sort.h>
#include <linux/group_cpus.h>
-#ifdef CONFIG_SMP
-
static void grp_spread_init_one(struct cpumask *irqmsk, struct cpumask *nmsk,
unsigned int cpus_per_grp)
{
@@ -564,22 +562,4 @@ struct cpumask *group_cpus_evenly(unsigned int numgrps, unsigned int *nummasks)
*nummasks = min(nr_present + nr_others, numgrps);
return masks;
}
-#else /* CONFIG_SMP */
-struct cpumask *group_cpus_evenly(unsigned int numgrps, unsigned int *nummasks)
-{
- struct cpumask *masks;
-
- if (numgrps == 0)
- return NULL;
-
- masks = kzalloc_objs(*masks, numgrps);
- if (!masks)
- return NULL;
-
- /* assign all CPUs(cpu 0) to the 1st group only */
- cpumask_copy(&masks[0], cpu_possible_mask);
- *nummasks = 1;
- return masks;
-}
-#endif /* CONFIG_SMP */
EXPORT_SYMBOL_GPL(group_cpus_evenly);
--
2.55.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v16 3/9] lib/group_cpus: Add group_mask_cpus_evenly()
2026-09-10 16:42 [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 1/9] scsi: aacraid: use block layer helpers to calculate num of queues Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 2/9] lib/group_cpus: remove dead !SMP code Aaron Tomlin
@ 2026-09-10 16:42 ` Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 4/9] sched/isolation: Prevent out-of-bounds read in isolcpus= boot parameter parser Aaron Tomlin
` (6 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Aaron Tomlin @ 2026-09-10 16:42 UTC (permalink / raw)
To: axboe, tglx, aacraid, James.Bottomley, mkp, frederic, bigeasy
Cc: atomlin, ionut.nechita, corbet, vincent.guittot, mingo, peterz,
radu, akpm, steve, sean, chjohnst, neelx, mproche, nick.lange,
marco.crivellari, rishil1999, linux-doc, linux-block, linux-scsi,
linux-kernel
From: Daniel Wagner <wagi@kernel.org>
This commit introduces group_mask_cpus_evenly(), which allows callers to
distribute a specific CPU mask evenly across groups. It serves as a bounded
version of group_cpus_evenly().
While group_cpus_evenly() operates on the global cpu_possible_mask,
group_mask_cpus_evenly() confines the distribution strictly within the
boundaries of the caller-provided mask. It preserves the kernel's native
two-stage spreading logic-first prioritising CPUs that are physically
present (cpu_present_mask) to prevent I/O starvation, and then distributing
any remaining vectors to non-present CPUs to maintain hotplug safety.
Signed-off-by: Daniel Wagner <wagi@kernel.org>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Co-developed-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
include/linux/group_cpus.h | 3 +
lib/group_cpus.c | 110 +++++++++++++++++++++++++++++++++++++
2 files changed, 113 insertions(+)
diff --git a/include/linux/group_cpus.h b/include/linux/group_cpus.h
index 9d4e5ab6c314..defab4123a82 100644
--- a/include/linux/group_cpus.h
+++ b/include/linux/group_cpus.h
@@ -10,5 +10,8 @@
#include <linux/cpu.h>
struct cpumask *group_cpus_evenly(unsigned int numgrps, unsigned int *nummasks);
+struct cpumask *group_mask_cpus_evenly(unsigned int numgrps,
+ const struct cpumask *mask,
+ unsigned int *nummasks);
#endif
diff --git a/lib/group_cpus.c b/lib/group_cpus.c
index b8d54398f88a..75bd082e00bf 100644
--- a/lib/group_cpus.c
+++ b/lib/group_cpus.c
@@ -563,3 +563,113 @@ struct cpumask *group_cpus_evenly(unsigned int numgrps, unsigned int *nummasks)
return masks;
}
EXPORT_SYMBOL_GPL(group_cpus_evenly);
+
+/**
+ * group_mask_cpus_evenly - Group all CPUs evenly per NUMA/CPU locality
+ * @numgrps: number of cpumasks to create
+ * @mask: CPUs to consider for the grouping
+ * @nummasks: number of initialized cpumasks
+ *
+ * Return: cpumask array if successful, NULL otherwise. Only the CPUs
+ * marked in the mask will be considered for the grouping. And each
+ * element includes CPUs assigned to this group. nummasks contains the
+ * number of initialized masks which can be less than numgrps.
+ *
+ * Try to put close CPUs from viewpoint of CPU and NUMA locality into
+ * the same group.
+ *
+ * We guarantee in the resulting grouping that all CPUs specified in the
+ * provided mask are covered, and no same CPU is assigned to multiple
+ * groups.
+ */
+struct cpumask *group_mask_cpus_evenly(unsigned int numgrps,
+ const struct cpumask *mask,
+ unsigned int *nummasks)
+{
+ unsigned int curgrp = 0, nr_present = 0, nr_others = 0;
+ cpumask_var_t *node_to_cpumask;
+ cpumask_var_t nmsk, local_mask, npresmsk;
+ int ret = -ENOMEM;
+ struct cpumask *masks = NULL;
+
+ if (numgrps == 0)
+ return NULL;
+
+ if (!zalloc_cpumask_var(&nmsk, GFP_KERNEL))
+ return NULL;
+
+ if (!zalloc_cpumask_var(&local_mask, GFP_KERNEL))
+ goto fail_nmsk;
+
+ if (!zalloc_cpumask_var(&npresmsk, GFP_KERNEL))
+ goto fail_local_mask;
+
+ node_to_cpumask = alloc_node_to_cpumask();
+ if (!node_to_cpumask)
+ goto fail_npresmsk;
+
+ masks = kzalloc_objs(*masks, numgrps);
+ if (!masks)
+ goto fail_node_to_cpumask;
+
+ build_node_to_cpumask(node_to_cpumask);
+
+ /*
+ * Create a stable snapshot of the mask. The grouping algorithm
+ * requires the CPU count to remain constant across its multiple
+ * passes. This prevents allocation failures if the caller passes a
+ * dynamic mask (e.g., cpu_online_mask) that changes concurrently.
+ */
+ cpumask_copy(local_mask, mask);
+
+ /*
+ * Grouping present CPUs first. We intersect the provided mask with
+ * cpu_present_mask to ensure that we prioritise physically
+ * available CPUs for the initial distribution.
+ */
+ cpumask_and(npresmsk, local_mask, cpu_present_mask);
+ ret = __group_cpus_evenly(curgrp, numgrps, node_to_cpumask,
+ npresmsk, nmsk, masks);
+ if (ret < 0)
+ goto fail_node_to_cpumask;
+ nr_present = ret;
+
+ /*
+ * Allocate non-present CPUs starting from the next group to be
+ * handled. If the grouping of present CPUs already exhausted the
+ * group space, assign the non-present CPUs to the already
+ * allocated out groups.
+ */
+ if (nr_present >= numgrps)
+ curgrp = 0;
+ else
+ curgrp = nr_present;
+ cpumask_andnot(npresmsk, local_mask, npresmsk);
+ ret = __group_cpus_evenly(curgrp, numgrps, node_to_cpumask,
+ npresmsk, nmsk, masks);
+ if (ret >= 0)
+ nr_others = ret;
+
+fail_node_to_cpumask:
+ free_node_to_cpumask(node_to_cpumask);
+
+fail_npresmsk:
+ free_cpumask_var(npresmsk);
+
+fail_local_mask:
+ free_cpumask_var(local_mask);
+
+fail_nmsk:
+ free_cpumask_var(nmsk);
+ if (ret < 0) {
+ kfree(masks);
+ return NULL;
+ }
+ *nummasks = min(nr_present + nr_others, numgrps);
+ if (*nummasks == 0) {
+ kfree(masks);
+ return NULL;
+ }
+ return masks;
+}
+EXPORT_SYMBOL_GPL(group_mask_cpus_evenly);
--
2.55.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v16 4/9] sched/isolation: Prevent out-of-bounds read in isolcpus= boot parameter parser
2026-09-10 16:42 [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
` (2 preceding siblings ...)
2026-09-10 16:42 ` [PATCH v16 3/9] lib/group_cpus: Add group_mask_cpus_evenly() Aaron Tomlin
@ 2026-09-10 16:42 ` Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 5/9] isolation: Introduce managed_irq_strict isolcpus type Aaron Tomlin
` (5 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Aaron Tomlin @ 2026-09-10 16:42 UTC (permalink / raw)
To: axboe, tglx, aacraid, James.Bottomley, mkp, frederic, bigeasy
Cc: atomlin, ionut.nechita, corbet, vincent.guittot, mingo, peterz,
radu, akpm, steve, sean, chjohnst, neelx, mproche, nick.lange,
marco.crivellari, rishil1999, linux-doc, linux-block, linux-scsi,
linux-kernel
The "isolcpus=" boot parameter parser in housekeeping_isolcpus_setup()
contains an out-of-bounds memory read bug when handling unterminated
flags.
When parsing the boot parameter string, the logic expects flags to be
comma-separated. If a user passes an unrecognised or legitimate flag
at the very end of the string without a trailing comma (e.g.,
"isolcpus=unknown"), the strict strncmp() checks will fail.
The execution then falls through to a fallback for loop designed to
skip the unknown sub-parameter. This inner loop consumes characters until
it encounters either a comma or the NULL terminator ('\0'). When the loop
terminates due to hitting the end of the string, the str pointer rests
exactly on the NULL terminator.
However, immediately following this inner loop, the code unconditionally
executes str++. This advances the pointer past the end of the string
and into uninitialised memory. The outer while (isalpha(*str)) loop
subsequently evaluates this out-of-bounds memory. If the adjacent byte
happens to be alphabetical, the parser will continue reading garbage
data, potentially leading to undefined behavior or boot anomalies.
Fix this by adding a bounds check immediately before the pointer
increment. This ensures the parsing loop cleanly terminates when
reaching the end of the boot parameter string.
Reported-by: sashiko-bot@kernel.org
Fixes: 3662daf023500 ("sched/isolation: Allow "isolcpus=" to skip unknown sub-parameters")
Cc: stable@vger.kernel.org
Reviewed-by: Valentin Schneider <vschneid@redhat.com>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
kernel/sched/isolation.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 156025ef81b7..9f02e9264a3f 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -369,6 +369,8 @@ static int __init housekeeping_isolcpus_setup(char *str)
}
pr_info("isolcpus: Skipped unknown flag %.*s\n", len, par);
+ if (!*str)
+ break;
str++;
}
--
2.55.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v16 5/9] isolation: Introduce managed_irq_strict isolcpus type
2026-09-10 16:42 [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
` (3 preceding siblings ...)
2026-09-10 16:42 ` [PATCH v16 4/9] sched/isolation: Prevent out-of-bounds read in isolcpus= boot parameter parser Aaron Tomlin
@ 2026-09-10 16:42 ` Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 6/9] blk-mq: use hk cpus only when isolcpus=managed_irq_strict is enabled Aaron Tomlin
` (4 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Aaron Tomlin @ 2026-09-10 16:42 UTC (permalink / raw)
To: axboe, tglx, aacraid, James.Bottomley, mkp, frederic, bigeasy
Cc: atomlin, ionut.nechita, corbet, vincent.guittot, mingo, peterz,
radu, akpm, steve, sean, chjohnst, neelx, mproche, nick.lange,
marco.crivellari, rishil1999, linux-doc, linux-block, linux-scsi,
linux-kernel
From: Daniel Wagner <wagi@kernel.org>
Multiqueue drivers spread I/O queues and managed interrupts across all
CPUs for optimal performance. However, these drivers are not aware of
CPU isolation requirements and will distribute queues and interrupt
vectors without considering the isolcpus configuration.
The existing isolcpus=managed_irq parameter attempts post-allocation
affinity steering, but this is best-effort and fails on modern multiqueue
storage where the number of queues meets or exceeds the CPU count.
Introduce a new isolcpus flag, "managed_irq_strict", that allows users
to strictly define which CPUs should have multiqueue hardware queues
and managed interrupts assigned. This enforces pre-allocation queue
bounding and mask exclusion, preventing managed hardware interrupts
from ever targeting isolated cores.
Because managed_irq_strict is a strict superset of managed_irq, specifying
managed_irq_strict automatically enables HK_FLAG_MANAGED_IRQ as well. This
ensures subsystems that rely on managed interrupt isolation (e.g. Hyper-V
VMBus channel distribution) continue to function seamlessly.
Signed-off-by: Daniel Wagner <wagi@kernel.org>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
include/linux/sched/isolation.h | 1 +
kernel/sched/isolation.c | 16 ++++++++++++++++
2 files changed, 17 insertions(+)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index cf0fd03dd7a2..bda3003ce86f 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -18,6 +18,7 @@ enum hk_type {
HK_TYPE_MANAGED_IRQ,
/* Inverse of boot-time nohz_full= or isolcpus=nohz arguments */
HK_TYPE_KERNEL_NOISE,
+ HK_TYPE_MANAGED_IRQ_STRICT,
HK_TYPE_MAX,
/*
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 9f02e9264a3f..7e9e78a18fb9 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -17,6 +17,7 @@ enum hk_flags {
HK_FLAG_DOMAIN = BIT(HK_TYPE_DOMAIN),
HK_FLAG_MANAGED_IRQ = BIT(HK_TYPE_MANAGED_IRQ),
HK_FLAG_KERNEL_NOISE = BIT(HK_TYPE_KERNEL_NOISE),
+ HK_FLAG_MANAGED_IRQ_STRICT = BIT(HK_TYPE_MANAGED_IRQ_STRICT),
};
DEFINE_STATIC_KEY_FALSE(housekeeping_overridden);
@@ -354,6 +355,12 @@ static int __init housekeeping_isolcpus_setup(char *str)
continue;
}
+ if (!strncmp(str, "managed_irq_strict,", 19)) {
+ str += 19;
+ flags |= HK_FLAG_MANAGED_IRQ_STRICT;
+ continue;
+ }
+
/*
* Skip unknown sub-parameter and validate that it is not
* containing an invalid character.
@@ -374,6 +381,15 @@ static int __init housekeeping_isolcpus_setup(char *str)
str++;
}
+ /*
+ * managed_irq_strict is a strict superset of managed_irq.
+ * Ensure that HK_FLAG_MANAGED_IRQ is also set so that subsystems
+ * relying on managed interrupt isolation (e.g. Hyper-V VMBus channel
+ * distribution and CPU hotplug IRQ restoration) continue to function.
+ */
+ if (flags & HK_FLAG_MANAGED_IRQ_STRICT)
+ flags |= HK_FLAG_MANAGED_IRQ;
+
/* Default behaviour for isolcpus without flags */
if (!flags)
flags |= HK_FLAG_DOMAIN | HK_FLAG_DOMAIN_BOOT;
--
2.55.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v16 6/9] blk-mq: use hk cpus only when isolcpus=managed_irq_strict is enabled
2026-09-10 16:42 [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
` (4 preceding siblings ...)
2026-09-10 16:42 ` [PATCH v16 5/9] isolation: Introduce managed_irq_strict isolcpus type Aaron Tomlin
@ 2026-09-10 16:42 ` Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 7/9] blk-mq: prevent offlining hk CPUs with associated online isolated CPUs Aaron Tomlin
` (3 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Aaron Tomlin @ 2026-09-10 16:42 UTC (permalink / raw)
To: axboe, tglx, aacraid, James.Bottomley, mkp, frederic, bigeasy
Cc: atomlin, ionut.nechita, corbet, vincent.guittot, mingo, peterz,
radu, akpm, steve, sean, chjohnst, neelx, mproche, nick.lange,
marco.crivellari, rishil1999, linux-doc, linux-block, linux-scsi,
linux-kernel
From: Daniel Wagner <wagi@kernel.org>
Extend the capabilities of the generic CPU to hardware queue (hctx)
mapping code, so it maps housekeeping CPUs and isolated CPUs to the
hardware queues evenly.
Example mapping result:
16 online CPUs
isolcpus=managed_irq_strict,2-3,6-7,12-13
Queue mapping:
hctx0: default 0 2
hctx1: default 1 3
hctx2: default 4 6
hctx3: default 5 7
hctx4: default 8 12
hctx5: default 9 13
hctx6: default 10
hctx7: default 11
hctx8: default 14
hctx9: default 15
IRQ mapping:
irq 42 affinity 0 effective 0 nvme0q0
irq 43 affinity 0 effective 0 nvme0q1
irq 44 affinity 1 effective 1 nvme0q2
irq 45 affinity 4 effective 4 nvme0q3
irq 46 affinity 5 effective 5 nvme0q4
irq 47 affinity 8 effective 8 nvme0q5
irq 48 affinity 9 effective 9 nvme0q6
irq 49 affinity 10 effective 10 nvme0q7
irq 50 affinity 11 effective 11 nvme0q8
irq 51 affinity 14 effective 14 nvme0q9
irq 52 affinity 15 effective 15 nvme0q10
In this scenario, the system has 16 online CPUs with 6 isolated cores
(2, 3, 6, 7, 12, and 13) and 10 housekeeping cores (0, 1, 4, 5, 8, 9,
10, 11, 14, and 15):
1. Queue allocation and ownership
Rather than allocating 16 hardware queues, the block layer
allocates only 10 hardware contexts (hctx0 to hctx9),
corresponding strictly to the 10 housekeeping CPUs. The 6
isolated CPUs do not own dedicated hardware queues; instead,
they are mapped across the existing active housekeeping queues
(e.g. isolated CPU 2 shares hctx0 with CPU 0). This ensures
tasks running on isolated CPUs can still issue I/O without
restriction.
2. Interrupt routing
All device interrupts, including the NVMe admin queue (irq 42)
and the 10 I/O completion queues (irq 43 to 52), target
housekeeping CPUs exclusively. When a task on isolated CPU 2
issues I/O via hctx0, the resulting completion interrupt (irq
43) fires on housekeeping CPU 0. Consequently, isolated CPUs are
never interrupted by device hardware, guaranteeing zero latency
disturbance for isolated workloads.
A corner case is when the number of online CPUs and present CPUs
differ and the driver asks for less queues than online CPUs, e.g.
8 online CPUs, 16 possible CPUs
isolcpus=managed_irq_strict,2-3,6-7,12-13
virtio_blk.num_request_queues=2
Queue mapping:
hctx0: default 0 1 2 3 4 5 6 7 8 12 13
hctx1: default 9 10 11 14 15
IRQ mapping
irq 27 affinity 0 effective 0 virtio0-config
irq 28 affinity 0-1,4-5,8 effective 5 virtio0-req.0
irq 29 affinity 9-11,14-15 effective 0 virtio0-req.1
This corner case demonstrates behaviour when hardware queue counts are
constrained (only 2 request queues) on a system with CPU hotplug (8
CPUs online out of 16 possible):
1. Coarse queue grouping
Because the driver requests only 2 queues, the 10 possible
housekeeping CPUs are partitioned into two groups: hctx0
receives CPUs 0-1, 4-5, and 8, while hctx1 receives CPUs 9-11
and 14-15. All isolated CPUs (both online cores 2-3, 6-7 and
offline cores 12-13) are mapped to hctx0 to share submission
capacity without allocating excess queues.
2. Isolation and hotplug protection in interrupt affinity
Although hctx0 serves both housekeeping and isolated CPUs, the
resulting interrupt affinity mask for irq 28 (affinity
0-1,4-5,8) strictly includes only the housekeeping cores,
completely excluding isolated cores 2-3, 6-7, and 12-13.
Furthermore, for hctx1 (irq 29), whose assigned housekeeping
CPUs (9-11, 14-15) are currently offline, the kernel routes the
effective interrupt to an available online housekeeping core
(CPU 0), guaranteeing that interrupts never spill onto isolated
cores under any hotplug state.
Noteworthy is that for the normal/default configuration (without
isolcpus=) the mapping will change for systems which have non
hyperthreading CPUs. The main assignment loop will completely rely that
group_mask_cpus_evenly to do the right thing. The old code would
distribute the CPUs linearly over the hardware context:
queue mapping for /dev/nvme0n1
hctx0: default 0 8
hctx1: default 1 9
hctx2: default 2 10
hctx3: default 3 11
hctx4: default 4 12
hctx5: default 5 13
hctx6: default 6 14
hctx7: default 7 15
The assign each hardware context the map generated by the
group_mask_cpus_evenly function:
queue mapping for /dev/nvme0n1
hctx0: default 0 1
hctx1: default 2 3
hctx2: default 4 5
hctx3: default 6 7
hctx4: default 8 9
hctx5: default 10 11
hctx6: default 12 13
hctx7: default 14 15
In case of hyperthreading CPUs, the resulting map stays the same.
Signed-off-by: Daniel Wagner <wagi@kernel.org>
Co-developed-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
block/blk-mq-cpumap.c | 163 +++++++++++++++++++++++++++++++++++++-----
1 file changed, 145 insertions(+), 18 deletions(-)
diff --git a/block/blk-mq-cpumap.c b/block/blk-mq-cpumap.c
index 705da074ad6c..cec5b26bd57c 100644
--- a/block/blk-mq-cpumap.c
+++ b/block/blk-mq-cpumap.c
@@ -22,8 +22,15 @@ static unsigned int blk_mq_num_queues(const struct cpumask *mask,
{
unsigned int num;
- num = cpumask_weight(mask);
- return min_not_zero(num, max_queues);
+ if (housekeeping_enabled(HK_TYPE_MANAGED_IRQ_STRICT))
+ num = cpumask_weight_and(mask, housekeeping_cpumask(HK_TYPE_MANAGED_IRQ_STRICT));
+ else
+ num = cpumask_weight(mask);
+ /*
+ * Ensure that a count of zero does not inadvertently result in
+ * allocating the maximum number of queues.
+ */
+ return min_not_zero(num ?: 1U, max_queues);
}
/**
@@ -33,7 +40,8 @@ static unsigned int blk_mq_num_queues(const struct cpumask *mask,
* ignored.
*
* Calculates the number of queues to be used for a multiqueue
- * device based on the number of possible CPUs.
+ * device based on the number of possible CPUs. This helper
+ * takes isolcpus settings into account.
*/
unsigned int blk_mq_num_possible_queues(unsigned int max_queues)
{
@@ -48,7 +56,8 @@ EXPORT_SYMBOL_GPL(blk_mq_num_possible_queues);
* ignored.
*
* Calculates the number of queues to be used for a multiqueue
- * device based on the number of online CPUs.
+ * device based on the number of online CPUs. This helper
+ * takes isolcpus settings into account.
*/
unsigned int blk_mq_num_online_queues(unsigned int max_queues)
{
@@ -56,23 +65,81 @@ unsigned int blk_mq_num_online_queues(unsigned int max_queues)
}
EXPORT_SYMBOL_GPL(blk_mq_num_online_queues);
+static void blk_mq_map_fallback(struct blk_mq_queue_map *qmap)
+{
+ unsigned int cpu;
+
+ /*
+ * Map all CPUs to the first hctx of this specific map, respecting
+ * the map's boundaries so secondary maps do not route into the default map.
+ */
+ for_each_possible_cpu(cpu)
+ qmap->mq_map[cpu] = qmap->queue_offset;
+}
+
void blk_mq_map_queues(struct blk_mq_queue_map *qmap)
{
- const struct cpumask *masks;
+ struct cpumask *masks;
+ const struct cpumask *constraint;
unsigned int queue, cpu, nr_masks;
+ unsigned long *active_hctx;
- masks = group_cpus_evenly(qmap->nr_queues, &nr_masks);
- if (!masks) {
- for_each_possible_cpu(cpu)
- qmap->mq_map[cpu] = qmap->queue_offset;
- return;
- }
+ active_hctx = bitmap_zalloc(qmap->nr_queues, GFP_KERNEL);
+ if (!active_hctx)
+ goto fallback;
- for (queue = 0; queue < qmap->nr_queues; queue++) {
- for_each_cpu(cpu, &masks[queue % nr_masks])
+ if (housekeeping_enabled(HK_TYPE_MANAGED_IRQ_STRICT))
+ constraint = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ_STRICT);
+ else
+ constraint = cpu_possible_mask;
+
+ /* Map CPUs to the hardware contexts (hctx) */
+ masks = group_mask_cpus_evenly(qmap->nr_queues, constraint, &nr_masks);
+ if (!masks)
+ goto free_fallback_hctx;
+
+ /*
+ * Iterate directly over the generated CPU masks.
+ * Calculate the final, highest hardware queue index that maps to this
+ * mask. This skips all intermediate overwrites and safely evaluates
+ * active_hctx only for queues that survive the mapping.
+ */
+ for (unsigned int idx = 0; idx < nr_masks; idx++) {
+ queue = qmap->nr_queues - 1 -
+ ((qmap->nr_queues - 1 - idx) % nr_masks);
+
+ for_each_cpu(cpu, &masks[idx])
qmap->mq_map[cpu] = qmap->queue_offset + queue;
+
+ __set_bit(queue, active_hctx);
+ }
+
+ /*
+ * If the active_hctx bitmap is empty, attempting to route unassigned
+ * CPUs will map them out-of-bounds. Fall back instead.
+ */
+ if (bitmap_empty(active_hctx, qmap->nr_queues))
+ goto free_fallback;
+
+ /* Map any unassigned CPU evenly to the hardware contexts (hctx) */
+ queue = find_first_bit(active_hctx, qmap->nr_queues);
+ for_each_cpu_andnot(cpu, cpu_possible_mask, constraint) {
+ qmap->mq_map[cpu] = qmap->queue_offset + queue;
+ queue = find_next_bit_wrap(active_hctx, qmap->nr_queues, queue + 1);
}
+
+ kfree(masks);
+ bitmap_free(active_hctx);
+
+ return;
+
+free_fallback:
kfree(masks);
+free_fallback_hctx:
+ bitmap_free(active_hctx);
+
+fallback:
+ blk_mq_map_fallback(qmap);
}
EXPORT_SYMBOL_GPL(blk_mq_map_queues);
@@ -109,24 +176,84 @@ void blk_mq_map_hw_queues(struct blk_mq_queue_map *qmap,
struct device *dev, unsigned int offset)
{
- const struct cpumask *mask;
+ cpumask_var_t mask;
+ const struct cpumask *constraint;
+ unsigned long *active_hctx;
unsigned int queue, cpu;
if (!dev->bus->irq_get_affinity)
+ goto map_software;
+
+ active_hctx = bitmap_zalloc(qmap->nr_queues, GFP_KERNEL);
+ if (!active_hctx)
goto fallback;
+ if (!zalloc_cpumask_var(&mask, GFP_KERNEL)) {
+ bitmap_free(active_hctx);
+ goto fallback;
+ }
+
+ if (housekeeping_enabled(HK_TYPE_MANAGED_IRQ_STRICT))
+ constraint = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ_STRICT);
+ else
+ constraint = cpu_possible_mask;
+
+ /* Map CPUs to the hardware contexts (hctx) */
for (queue = 0; queue < qmap->nr_queues; queue++) {
- mask = dev->bus->irq_get_affinity(dev, queue + offset);
- if (!mask)
- goto fallback;
+ const struct cpumask *affinity_mask;
+
+ affinity_mask = dev->bus->irq_get_affinity(dev, offset + queue);
+ if (!affinity_mask)
+ goto free_map_software;
- for_each_cpu(cpu, mask)
+ for_each_cpu(cpu, affinity_mask) {
qmap->mq_map[cpu] = qmap->queue_offset + queue;
+ cpumask_set_cpu(cpu, mask);
+ }
}
+ /*
+ * Evaluate active_hctx after mapping to handle overlapping masks.
+ * This ensures queues that were overwritten do not falsely pass validation.
+ */
+ for_each_cpu(cpu, mask) {
+ if (cpumask_test_cpu(cpu, constraint)) {
+ queue = qmap->mq_map[cpu] - qmap->queue_offset;
+ __set_bit(queue, active_hctx);
+ }
+ }
+
+ /*
+ * If no assigned CPU matches the constraint, the active_hctx
+ * bitmap will be empty. Fall back instead of routing out of bounds.
+ */
+ if (bitmap_empty(active_hctx, qmap->nr_queues))
+ goto free_fallback;
+
+ /* Map any unassigned CPU evenly to the hardware contexts (hctx) */
+ queue = find_first_bit(active_hctx, qmap->nr_queues);
+ for_each_cpu_andnot(cpu, cpu_possible_mask, mask) {
+ qmap->mq_map[cpu] = qmap->queue_offset + queue;
+ queue = find_next_bit_wrap(active_hctx, qmap->nr_queues, queue + 1);
+ }
+
+ bitmap_free(active_hctx);
+ free_cpumask_var(mask);
+
return;
+free_fallback:
+ bitmap_free(active_hctx);
+ free_cpumask_var(mask);
+
fallback:
+ blk_mq_map_fallback(qmap);
+ return;
+
+free_map_software:
+ free_cpumask_var(mask);
+ bitmap_free(active_hctx);
+map_software:
blk_mq_map_queues(qmap);
}
EXPORT_SYMBOL_GPL(blk_mq_map_hw_queues);
--
2.55.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v16 7/9] blk-mq: prevent offlining hk CPUs with associated online isolated CPUs
2026-09-10 16:42 [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
` (5 preceding siblings ...)
2026-09-10 16:42 ` [PATCH v16 6/9] blk-mq: use hk cpus only when isolcpus=managed_irq_strict is enabled Aaron Tomlin
@ 2026-09-10 16:42 ` Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 8/9] genirq/affinity: Restrict managed IRQ affinity to housekeeping CPUs Aaron Tomlin
` (2 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Aaron Tomlin @ 2026-09-10 16:42 UTC (permalink / raw)
To: axboe, tglx, aacraid, James.Bottomley, mkp, frederic, bigeasy
Cc: atomlin, ionut.nechita, corbet, vincent.guittot, mingo, peterz,
radu, akpm, steve, sean, chjohnst, neelx, mproche, nick.lange,
marco.crivellari, rishil1999, linux-doc, linux-block, linux-scsi,
linux-kernel
From: Daniel Wagner <wagi@kernel.org>
When isolcpus=managed_irq_strict is enabled and the last housekeeping CPU
for a given hctx goes offline, no CPU would be left to handle I/O.
To prevent I/O stalls, disallow offlining housekeeping CPUs that are
still serving isolated CPUs.
Signed-off-by: Daniel Wagner <wagi@kernel.org>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Co-developed-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
block/blk-mq.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 63 insertions(+)
diff --git a/block/blk-mq.c b/block/blk-mq.c
index a26a11c73ee3..6e903e1310e9 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -3690,6 +3690,64 @@ static bool blk_mq_hctx_has_requests(struct blk_mq_hw_ctx *hctx)
return data.has_rq;
}
+static bool blk_mq_hctx_can_offline_hk_cpu(struct blk_mq_hw_ctx *hctx,
+ unsigned int this_cpu)
+{
+ const struct cpumask *hk_mask = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ_STRICT);
+ int cpu, fallback_isolated_cpu = -1;
+
+ /*
+ * During system suspend or hibernation, tasks are frozen and all
+ * CPUs are being taken offline. Allow offlining to proceed so
+ * suspend is not aborted.
+ */
+ if (cpuhp_tasks_frozen)
+ return true;
+
+ /*
+ * If the CPU being offlined is not a housekeeping CPU,
+ * offlining it will not strand isolated CPUs. Allow it.
+ */
+ if (!cpumask_test_cpu(this_cpu, hk_mask))
+ return true;
+ /*
+ * If this CPU is not mapped to this specific hardware context,
+ * offlining it will not affect the context's I/O routing. Allow it.
+ */
+ if (blk_mq_map_queue_type(hctx->queue, hctx->type, this_cpu) != hctx)
+ return true;
+ /*
+ * Iterate over all online CPUs and manually check their mapping.
+ * We cannot use hctx->cpumask here because blk_mq_map_swqueue()
+ * intentionally strips isolated CPUs from it to prevent kworker
+ * routing.
+ */
+ for_each_online_cpu(cpu) {
+ struct blk_mq_hw_ctx *h;
+
+ if (cpu == this_cpu)
+ continue;
+
+ h = blk_mq_map_queue_type(hctx->queue, hctx->type, cpu);
+ if (h != hctx)
+ continue;
+
+ if (cpumask_test_cpu(cpu, hk_mask))
+ return true;
+
+ if (fallback_isolated_cpu == -1)
+ fallback_isolated_cpu = cpu;
+ }
+
+ if (fallback_isolated_cpu != -1) {
+ pr_warn("blk-mq: cannot offline CPU %u: online isolated CPU %d is still mapped to hctx%u\n",
+ this_cpu, fallback_isolated_cpu, hctx->queue_num);
+ return false;
+ }
+
+ return true;
+}
+
static bool blk_mq_hctx_has_online_cpu(struct blk_mq_hw_ctx *hctx,
unsigned int this_cpu)
{
@@ -3722,6 +3780,11 @@ static int blk_mq_hctx_notify_offline(unsigned int cpu, struct hlist_node *node)
struct blk_mq_hw_ctx, cpuhp_online);
int ret = 0;
+ if (housekeeping_enabled(HK_TYPE_MANAGED_IRQ_STRICT)) {
+ if (!blk_mq_hctx_can_offline_hk_cpu(hctx, cpu))
+ return -EINVAL;
+ }
+
if (!hctx->nr_ctx || blk_mq_hctx_has_online_cpu(hctx, cpu))
return 0;
--
2.55.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v16 8/9] genirq/affinity: Restrict managed IRQ affinity to housekeeping CPUs
2026-09-10 16:42 [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
` (6 preceding siblings ...)
2026-09-10 16:42 ` [PATCH v16 7/9] blk-mq: prevent offlining hk CPUs with associated online isolated CPUs Aaron Tomlin
@ 2026-09-10 16:42 ` Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 9/9] docs: add managed_irq_strict flag to isolcpus Aaron Tomlin
2026-09-10 18:26 ` [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
9 siblings, 0 replies; 11+ messages in thread
From: Aaron Tomlin @ 2026-09-10 16:42 UTC (permalink / raw)
To: axboe, tglx, aacraid, James.Bottomley, mkp, frederic, bigeasy
Cc: atomlin, ionut.nechita, corbet, vincent.guittot, mingo, peterz,
radu, akpm, steve, sean, chjohnst, neelx, mproche, nick.lange,
marco.crivellari, rishil1999, linux-doc, linux-block, linux-scsi,
linux-kernel
At present, the managed interrupt spreading algorithm distributes vectors
across all available CPUs within a given node or system. On systems
employing CPU isolation (e.g. "isolcpus=managed_irq_strict"), this
behaviour defeats the primary purpose of isolation by routing hardware
interrupts (such as NVMe completion queues) directly to isolated cores.
Update irq_create_affinity_masks() to respect the housekeeping CPU mask.
By passing the HK_TYPE_MANAGED_IRQ_STRICT mask directly to the
topological distribution function (group_mask_cpus_evenly()), we ensure
that managed interrupts are kept strictly off isolated CPUs.
This patch additionally addresses the architectural constraints of
restricted vector distribution:
1. Vector limits and multi-set scaling
Updated irq_calc_affinity_vectors() to bound the maximum number
of allocated vectors to the weight of the housekeeping mask for
single-set drivers. For drivers providing a calc_sets()
callback, vector calculations continue to scale with the
driver's requested set sizes (maxvec - resv), preventing
unnecessary queue contention across distinct functional sets
while irq_create_affinity_masks() guarantees that all allocated
vectors remain strictly restricted to housekeeping CPUs.
2. Multi-set alignment and leak prevention
When isolation constraints result in fewer available masks than
requested vectors for a given set, the remaining vector slots
are padded with the housekeeping mask. This replaces the
historical irq_default_affinity padding, ensuring excess managed
queues do not leak interrupts onto isolated CPUs.
3. Minimum vector safety net
To prevent fatal -ENOSPC device probe aborts on heavily isolated
systems (where the housekeeping CPU count might be lower than a
device's structural minimum), the final vector calculation is
safeguarded to never drop below minvec. Queues will safely share
the available housekeeping CPUs instead of failing the probe.
4. Zero overhead
The housekeeping mask is conditionally assigned via a direct
pointer, completely avoiding temporary mask allocations (e.g.
alloc_cpumask_var) and bitwise operations when CPU isolation is
disabled. This guarantees zero performance or memory overhead
for standard configurations.
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
kernel/irq/affinity.c | 29 ++++++++++++++++++++++-------
1 file changed, 22 insertions(+), 7 deletions(-)
diff --git a/kernel/irq/affinity.c b/kernel/irq/affinity.c
index 78f2418a8925..7796882a567a 100644
--- a/kernel/irq/affinity.c
+++ b/kernel/irq/affinity.c
@@ -8,6 +8,7 @@
#include <linux/slab.h>
#include <linux/cpu.h>
#include <linux/group_cpus.h>
+#include <linux/sched/isolation.h>
static void default_calc_sets(struct irq_affinity *affd, unsigned int affvecs)
{
@@ -25,8 +26,10 @@ static void default_calc_sets(struct irq_affinity *affd, unsigned int affvecs)
struct irq_affinity_desc *
irq_create_affinity_masks(unsigned int nvecs, struct irq_affinity *affd)
{
- unsigned int affvecs, curvec, usedvecs, i;
+ unsigned int affvecs, curvec, usedvecs, i, j;
struct irq_affinity_desc *masks = NULL;
+ const struct cpumask *hk_mask = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ_STRICT);
+ bool hk_enabled = housekeeping_enabled(HK_TYPE_MANAGED_IRQ_STRICT);
/*
* Determine the number of vectors which need interrupt affinities
@@ -70,19 +73,29 @@ irq_create_affinity_masks(unsigned int nvecs, struct irq_affinity *affd)
*/
for (i = 0, usedvecs = 0; i < affd->nr_sets; i++) {
unsigned int nr_masks, this_vecs = affd->set_size[i];
- struct cpumask *result = group_cpus_evenly(this_vecs, &nr_masks);
+ struct cpumask *result;
+ const struct cpumask *mask;
+ if (hk_enabled)
+ mask = hk_mask;
+ else
+ mask = cpu_possible_mask;
+
+ result = group_mask_cpus_evenly(this_vecs, mask,
+ &nr_masks);
if (!result) {
kfree(masks);
return NULL;
}
-
- for (int j = 0; j < nr_masks; j++)
+ for (j = 0; j < nr_masks; j++)
cpumask_copy(&masks[curvec + j].mask, &result[j]);
+ for (j = nr_masks; j < this_vecs; j++)
+ cpumask_copy(&masks[curvec + j].mask, mask);
+
kfree(result);
- curvec += nr_masks;
- usedvecs += nr_masks;
+ curvec += this_vecs;
+ usedvecs += this_vecs;
}
/* Fill out vectors at the end that don't need affinity */
@@ -117,8 +130,10 @@ unsigned int irq_calc_affinity_vectors(unsigned int minvec, unsigned int maxvec,
if (affd->calc_sets)
set_vecs = maxvec - resv;
+ else if (housekeeping_enabled(HK_TYPE_MANAGED_IRQ_STRICT))
+ set_vecs = cpumask_weight(housekeeping_cpumask(HK_TYPE_MANAGED_IRQ_STRICT));
else
set_vecs = cpumask_weight(cpu_possible_mask);
- return resv + min(set_vecs, maxvec - resv);
+ return max(minvec, resv + min(set_vecs, maxvec - resv));
}
--
2.55.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v16 9/9] docs: add managed_irq_strict flag to isolcpus
2026-09-10 16:42 [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
` (7 preceding siblings ...)
2026-09-10 16:42 ` [PATCH v16 8/9] genirq/affinity: Restrict managed IRQ affinity to housekeeping CPUs Aaron Tomlin
@ 2026-09-10 16:42 ` Aaron Tomlin
2026-09-10 18:26 ` [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
9 siblings, 0 replies; 11+ messages in thread
From: Aaron Tomlin @ 2026-09-10 16:42 UTC (permalink / raw)
To: axboe, tglx, aacraid, James.Bottomley, mkp, frederic, bigeasy
Cc: atomlin, ionut.nechita, corbet, vincent.guittot, mingo, peterz,
radu, akpm, steve, sean, chjohnst, neelx, mproche, nick.lange,
marco.crivellari, rishil1999, linux-doc, linux-block, linux-scsi,
linux-kernel
From: Daniel Wagner <wagi@kernel.org>
The managed_irq_strict flag informs multiqueue device drivers where to
place hardware queues and managed interrupt vectors. Document this new
flag in the isolcpus command-line argument description, describe its
relationship and precedence with respect to the legacy managed_irq flag,
and document HK_TYPE_MANAGED_IRQ_STRICT in the core housekeeping guide.
Signed-off-by: Daniel Wagner <wagi@kernel.org>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Co-developed-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
.../admin-guide/kernel-parameters.txt | 37 ++++++++++++++++++-
Documentation/core-api/housekeeping.rst | 6 ++-
2 files changed, 41 insertions(+), 2 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 68647ff4bdd2..eb608e5139a6 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2822,7 +2822,6 @@ Kernel parameters
"number of CPUs in system - 1".
managed_irq
-
Isolate from being targeted by managed interrupts
which have an interrupt mask containing isolated
CPUs. The affinity of managed interrupts is
@@ -2845,6 +2844,42 @@ Kernel parameters
housekeeping CPUs has no influence on those
queues.
+ managed_irq_strict
+ Strictly restrict multiqueue hardware queue
+ allocation and managed interrupt affinity to
+ housekeeping CPUs. Unlike the best-effort
+ managed_irq flag, this option guarantees that
+ managed interrupts and multiqueue block device
+ queues are never created for or mapped to isolated
+ CPUs.
+
+ In the block layer (blk-mq), device queue allocation
+ and CPU-to-queue mappings are restricted to
+ housekeeping CPUs. For devices using managed
+ interrupts, hardware completion interrupts are routed
+ exclusively to housekeeping cores, shielding isolated
+ CPUs from I/O interruptions even if they initiated the
+ request.
+
+ If both managed_irq and managed_irq_strict are
+ specified, managed_irq_strict takes precedence.
+
+ Note: Using managed_irq_strict restricts the number
+ of allocated hardware queues to match the number of
+ housekeeping CPUs for single-set drivers. This
+ prevents MSI-X vector exhaustion and forces isolated
+ CPUs to share submission queues.
+
+ Note: Offlining housekeeping CPUs which serve
+ isolated CPUs will fail. The isolated CPUs must
+ be offlined before offlining the housekeeping
+ CPUs.
+
+ Note: When I/O is submitted by an application on
+ an isolated CPU, the hardware completion
+ interrupt is handled entirely by a housekeeping
+ CPU.
+
The format of <cpu-list> is described above.
iucv= [HW,NET]
diff --git a/Documentation/core-api/housekeeping.rst b/Documentation/core-api/housekeeping.rst
index ccb0a88b9cb3..6d08c71e6c0e 100644
--- a/Documentation/core-api/housekeeping.rst
+++ b/Documentation/core-api/housekeeping.rst
@@ -23,7 +23,7 @@ CPU. It is simply a CPU that can execute housekeeping work. There must
always be at least one online housekeeping CPU at any time. The CPUs that
are not isolated are automatically assigned as housekeeping.
-Housekeeping is currently divided in four features described
+Housekeeping is currently divided in five features described
by the ``enum hk_type type``:
1. HK_TYPE_DOMAIN matches the work moved away by scheduler domain
@@ -44,6 +44,10 @@ by the ``enum hk_type type``:
to HK_TYPE_DOMAIN except it ignores the isolation performed by
cpusets.
+5. HK_TYPE_MANAGED_IRQ_STRICT matches the multiqueue block devices and
+ managed IRQs strictly restricted to housekeeping CPUs through
+ ``isolcpus=managed_irq_strict``.
+
Housekeeping cpumasks
=================================
--
2.55.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v16 0/9] blk: honor isolcpus configuration
2026-09-10 16:42 [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
` (8 preceding siblings ...)
2026-09-10 16:42 ` [PATCH v16 9/9] docs: add managed_irq_strict flag to isolcpus Aaron Tomlin
@ 2026-09-10 18:26 ` Aaron Tomlin
9 siblings, 0 replies; 11+ messages in thread
From: Aaron Tomlin @ 2026-09-10 18:26 UTC (permalink / raw)
To: axboe, tglx, aacraid, James.Bottomley, mkp, frederic, bigeasy
Cc: wagi, martin.petersen, hare, ionut.nechita, corbet,
vincent.guittot, mingo, peterz, radu, akpm, steve, sean,
chjohnst, neelx, mproche, nick.lange, marco.crivellari,
rishil1999, linux-doc, linux-block, linux-scsi, linux-kernel
On Thu, Sep 10, 2026 at 12:42:28PM -0400, Aaron Tomlin wrote:
> Hi Thomas, Jens, Sebastian, Frederic,
Adding Daniel, Martin, and Hannes on Cc.
> This is version 16 of the series enabling multiqueue devices and managed
> interrupts to adhere to CPU isolation constraints. The series has been
> rebased on v7.3-rc1-708-g893e11787f78.
>
> Background and Problem
> ======================
>
> Modern multiqueue storage controllers (e.g. NVMe devices) typically
> allocate as many hardware queues and interrupt vectors as there are CPUs in
> the system. When CPU isolation is requested via isolcpus, the kernel's
> current mechanism "isolcpus=managed_irq" relies on post-allocation affinity
> steering inside kernel/irq/manage.c. However, because drivers allocate
> queues and vectors before affinity steering takes place, the number of
> vectors frequently matches or exceeds the total CPU count. In such
> scenarios, the post-allocation steering logic cannot avoid assigning
> managed interrupts to isolated CPUs, effectively defeating CPU isolation
> for real-time and latency-critical workloads.
>
> Proposed solution and architecture
> ==================================
>
> This series introduces a new CPU isolation feature,
> "isolcpus=managed_irq_strict", designed to protect isolated cores from the
> disruptive hardware interrupts generated by high-performance multiqueue
> devices.
>
> To prevent I/O stalls, the block layer is additionally hardened to reject
> hotplug requests that attempt to offline a housekeeping CPU if it is the
> last remaining CPU actively serving an online isolated core.
>
> The complex "top-down" mask plumbing introduced in earlier revisions
> (version 12), which modified struct irq_affinity and expanded block layer
> APIs, has been abandoned. It is replaced by a centralised approach: direct
> isolation querying via housekeeping_cpumask(HK_TYPE_MANAGED_IRQ_STRICT)
> within the genirq/affinity subsystem. This architectural simplification
> successfully decouples core changes from driver-specific implementations.
>
>
> In version 15, this was addressed under the previously named
> "isolcpus=io_queue". During subsequent review, Sebastian Andrzej Siewior
> pointed out that the existing "isolcpus=managed_irq" implementation is
> fundamentally broken on modern multiqueue hardware, and raised the question
> of whether we should introduce another flag at all, or instead fix
> "managed_irq" directly in place. As Sebastian observed, if
> backward-compatibility concerns prevent modifying the legacy behaviour, a
> "managed_irq_strict" designation represents the logical counterpart.
>
> To make progress while keeping backwards compatibility completely safe,
> this iteration adopts option 2: we drop the "io_queue" naming in favour of
> "isolcpus=managed_irq_strict". It acts as the strict, pre-allocation
> counterpart to "managed_irq", constraining both block layer queue mappings
> and generic interrupt affinity masks exclusively to housekeeping CPUs.
> Furthermore, if both options are supplied, "managed_irq_strict" takes
> precedence.
>
> Maintainer guidance requested
> =============================
>
> We would appreciate guidance from Frederic Weisbecker regarding preference
> on this front:
> 1. Do you prefer keeping "managed_irq_strict" as an explicit
> opt-in (as implemented in this series) to preserve historical
> behaviour for existing users of "managed_irq"?
>
> 2. Or would you prefer upgrading the semantics of "managed_irq"
> directly in place without introducing a new parameter, on the
> grounds that the legacy post-allocation behaviour is already
> broken for modern multiqueue hardware?
>
>
> Please let me know your thoughts.
>
> Changes since v15:
>
> - Migrated feature naming from "isolcpus=io_queue" (HK_TYPE_IO_QUEUE) to
> "isolcpus=managed_irq_strict" (HK_TYPE_MANAGED_IRQ_STRICT), establishing
> it as the strict, pre-allocation counterpart to "managed_irq"
>
> - Specifying "managed_irq_strict" automatically enables
> HK_FLAG_MANAGED_IRQ so that dependent subsystems continue to function
>
> - Cleaned up commit bodies across the series, stripping informal
> scratchpads and adding Co-developed-by attribution where appropriate
>
> - Added new patch 4/9 ("sched/isolation: Prevent out-of-bounds read in
> isolcpus= boot parameter parser"). Incorporated prerequisite bugfix
> resolving an out-of-bounds read in housekeeping_isolcpus_setup() when
> sub-parameters lack trailing commas
>
> - Defined HK_TYPE_MANAGED_IRQ_STRICT and HK_FLAG_MANAGED_IRQ_STRICT.
> Ensured HK_FLAG_MANAGED_IRQ is set when managed_irq_strict is enabled
>
> - Decoupled queue mapping from dynamic cpu_online_mask; eliminated the
> blk_mq_validate() check and online cpumask snapshotting to avoid TOCTOU
> races during concurrent CPU hotplug events
>
> - Added cpuhp_tasks_frozen check to ensure suspend, hibernation, and
> resume transitions are not aborted
>
> - Added blk_mq_map_queue_type() check so CPU offline validation only
> evaluates hardware contexts actively mapped to the CPU being offlined
>
> - Enhanced warning message to report the offending CPU ID and queue
> number
>
> - Prioritised affd->calc_sets in irq_calc_affinity_vectors() so multi-set
> drivers scale vector allocations with requested set sizes to prevent
> cross-queue contention, while vector spreading remains strictly
> constrained to housekeeping cores
>
> - Documented "managed_irq_strict" in kernel-parameters.txt, detailing its
> role in blk-mq queue distribution and genirq affinity masking, as well
> as its superset relationship with legacy "managed_irq"
>
> - Link to v15: https://lore.kernel.org/lkml/20260521232956.553287-1-atomlin@atomlin.com/
>
> Changes since v14:
>
> - Fixed a division-by-zero by ensuring group_mask_cpus_evenly() safely
> frees its allocations and returns NULL instead of an empty array if the
> provided mask yields zero groups.
>
> - Fixed a device probe -ENOSPC regression in blk_mq_num_queues(). If the
> housekeeping mask intersection evaluated to 0 (e.g., against a localised
> NUMA node), min_not_zero() would erroneously return the absolute maximum
> hardware queues. The result is now safely clamped to a minimum of 1.
>
> - Added a mapping verification check to prevent unrelated housekeeping
> CPUs from aborting the global hotplug offline sequence.
>
> - Aligned the pr_warn format specifier with the unsigned int declaration
> of hctx->queue_num in blk_mq_hctx_can_offline_hk_cpu().
>
> - Link to v14: https://lore.kernel.org/lkml/20260520215030.496803-1-atomlin@atomlin.com/
>
> Changes since v13:
>
> - Removed ineffective data_race() annotations around mask and
> cpu_present_mask pointers. Wrapping the pointers failed to suppress
> KCSAN warnings for the underlying inline bitmap memory accesses.
>
> - Fixed a silent validation bypass in blk_mq_map_hw_queues() caused by
> overlapping IRQ affinity masks by removing the short-circuiting
> optimisation and evaluating the active_hctx bitmap in a secondary pass.
>
> - Restored topology-aware multi-queue fallback by correctly routing
> missing IRQ affinity masks to the map_software path instead of the naive
> map-all fallback.
>
> - Dropped hctx->queue->disk->disk_name from warning to avoid a UAF.
>
> - Fixed an isolation leak where excess allocated hardware queues were
> improperly padded with irq_default_affinity. Because these queues are
> marked as managed, they bypassed user-space IRQ balancing; they are now
> safely padded with the housekeeping mask.
>
> - Enforced the housekeeping vector cap prior to evaluating driver-provided
> calc_sets() callbacks, preventing modern multi-queue drivers from
> bypassing the cap and wasting memory on dead queues.
>
> - Introduced a safety net to the vector calculation to prevent fatal
> -ENOSPC device probe aborts on heavily isolated systems where the
> housekeeping CPU count is lower than the device's structural minimum.
>
> - Removed an inaccurate claim stating that the io_queue isolation flag
> takes precedence over managed_irq. Both flags are parsed, evaluated, and
> enforced entirely independently by their respective subsystems.
>
> - Link to v13: https://lore.kernel.org/lkml/20260513005509.135966-1-atomlin@atomlin.com/
>
> Changes since v12:
>
> - Resolved TOCTOU race conditions against CPU hotplug events in
> blk_mq_map_queues() and group_mask_cpus_evenly() by taking lockless
> snapshots of the online CPU mask prior to algorithmic evaluation.
>
> - Migrated the active_hctx tracking to a dynamically sized bitmap
> (bitmap_zalloc), resolving a critical out-of-bounds memory write that
> occurred when hardware queues exceeded the system CPU count.
>
> - Wrapped the disk pointer fetch in blk_mq_hctx_can_offline_hk_cpu() with
> READ_ONCE() to prevent a TOCTOU NULL pointer dereference against
> concurrent device teardowns.
>
> - Introduced bitmap_empty() checks to prevent the mapping logic from
> routing unassigned CPUs into unallocated memory when all mapped CPUs are
> offline, safely forcing a fallback mapping instead.
>
> - Implemented a native two-stage distribution logic in
> group_mask_cpus_evenly() that first prioritises physically present CPUs
> to prevent I/O starvation before distributing remaining vectors to
> non-present CPUs for hotplug safety.
>
> - Restricted the maximum number of allocated vectors in
> irq_calc_affinity_vectors() to the weight of the housekeeping mask,
> preventing drivers from wasting memory on dead hardware queues that
> physically cannot be routed.
>
> - Added padding logic using irq_default_affinity for sets where isolation
> constraints yield fewer masks than requested vectors, preserving the 1:1
> hardware queue mapping sequence for subsequent sets.
>
> - Fixed a logic flaw that prematurely rejected valid offline requests by
> manually iterating over cpu_online_mask and reverse-mapping to
> accurately detect isolated CPUs, properly permitting the offlining of
> non-housekeeping CPUs.
>
> - Corrected an absolute versus relative queue index calculation bug in
> blk_mq_map_queues() that was overwriting loop iterations, by iterating
> directly over the generated masks.
>
> - Replaced scoped __free cleanups with traditional goto unwinding in the
> block layer to align with subsystem styling guidelines.
>
> - Refined the io_queue kernel command-line parameter documentation for
> better clarity and precision.
>
> - Link to v12: https://lore.kernel.org/lkml/20260422185215.100929-1-atomlin@atomlin.com/
>
> Changes since v11:
>
> - Removed duplicate paragraph from the commit message in patch 11
> (Marco Crivellari)
>
> - Ensure ZERO_SIZE_PTR is not returned by group_mask_cpus_evenly()
> (Marco Crivellari)
>
> - Link to v11: https://lore.kernel.org/lkml/20260416192942.1243421-1-atomlin@atomlin.com/
>
> Changes since v10:
>
> - Completely rewrote the isolcpus=io_queue documentation in
> Documentation/admin-guide/kernel-parameters.txt to clarify its exclusive
> application to managed IRQs, queue allocation limits, vector exhaustion
> prevention, and hardware interrupt routing (Ming Lei)
>
> - Fixed a stack frame bloat issue by avoiding the on-stack declaration of
> struct cpumask (Waiman Long)
>
> - Link to v10: https://lore.kernel.org/linux-nvme/20260401222312.772334-1-atomlin@atomlin.com/
>
> Changes since v9:
>
> - Fixed a page fault regression encountered when initialising secondary
> queue maps (e.g. NVMe poll queues). Restored the qmap->queue_offset to
> the mq_map assignment to ensure CPUs are strictly mapped to absolute
> hardware indices (Keith Busch)
>
> - Corrected the active_hctx tracker to utilise relative queue indices,
> preventing out-of-bounds mask assignments
>
> - Fixed the blk_mq_validate() sanity check to properly evaluate absolute
> queue indices against the offset-adjusted loop index
>
> - Corrected typographical errors within block/blk-mq-cpumap.c
> (Keith Busch)
>
> - Clarified the commit message regarding the removal of the !SMP fallback
> code, explicitly noting that the core scheduler now mandates SMP
> unconditionally (Sebastian Andrzej Siewior)
>
> - Added missing "Signed-off-by:" tags to properly record the patch series
> chain of custody
>
> - Link to v9: https://lore.kernel.org/lkml/20260330221047.630206-1-atomlin@atomlin.com/
>
> Changes since v8:
>
> - Added "Reviewed-by:" tags
>
> - Introduced irq_spread_hk_filter() to safely restrict managed IRQ
> affinity to housekeeping CPUs (Thomas Gleixner)
>
> - Removed the unsafe global static variable blk_hk_online_mask from
> blk-mq-cpumap.c and blk-mq.c. blk_mq_online_queue_affinity() now returns
> a stable pointer, delegating safe intersection to the callers to prevent
> concurrent modification races (Thomas Gleixner, Hannes Reinecke)
>
> - Resolved BUG: kernel NULL pointer dereference in __blk_mq_all_tag_iter
> reported by the kernel test robot during cpuhotplug rcutorture stress
> testing
>
> - Link to v8: https://lore.kernel.org/lkml/20250905-isolcpus-io-queues-v8-0-885984c5daca@kernel.org/
>
> Changes since v7:
>
> - Added commit 524f5eea4bbe ("lib/group_cpus: remove !SMP code")
>
> - Merged the new mapping logic directly into the existing function to
> avoid special casing
>
> - Refined the group_mask_cpus_evenly() implementation with the following
> updates:
>
> - Corrected the function name typo (changed group_masks_cpus_evenly to
> group_mask_cpus_evenly)
>
> - Updated the documentation comment to accurately reflect the function's
> behavior
>
> - Renamed the cpu_mask argument to mask for consistency
>
> - Added a new patch for aacraid to include the missing number of queues
> calculation
>
> - Restricted updates to only affect SCSI drivers that support
> PCI_IRQ_AFFINITY and do not utilise nvme-fabrics
>
> - Removed the __free cleanup attribute usage for cpumask_var_t allocations
> due to compatibility issues
>
> - Updated the documentation to explicitly highlight the limitations
> surrounding CPU offlining
>
> - Collected accumulated Reviewed-by and Acked-by tags
>
> - Link to v7: https://patch.msgid.link/20250702-isolcpus-io-queues-v7-0-557aa7eacce4@kernel.org
>
> Changes since v6:
>
> - Sent out the first part of the series independently:
> https://lore.kernel.org/all/20250617-isolcpus-queue-counters-v1-0-13923686b54b@kernel.org/
>
> - Added comprehensive kernel command-line documentation
>
> - Added validation logic to ensure the resulting CPU-to-queue mapping is
> fully operational
>
> - Rewrote the isolcpus mapping code to properly account for active
> hardware contexts (hctx)
>
> - Introduced blk_mq_map_hk_irq_queues, which utilizes the mask retrieved
> from irq_get_affinity()
>
> - Refactored blk_mq_map_hk_queues to require the caller to explicitly test
> for HK_TYPE_MANAGED_IRQ
>
> - Link to v6: https://patch.msgid.link/20250424-isolcpus-io-queues-v6-0-9a53a870ca1f@kernel.org
>
> Changes since v5:
>
> - Reintroduced the io_queue type for the isolcpus kernel parameter
>
> - Prevented the offlining of a housekeeping CPU if an isolated CPU is
> still present, upgrading this behavior from a simple warning to a hard
> restriction
>
> - Link to v5: https://lore.kernel.org/r/20250110-isolcpus-io-queues-v5-0-0e4f118680b0@kernel.org
>
> Changes since v4:
>
> - Rebased the series onto the latest for-6.14/block branch.
>
> - Updated the documentation regarding the managed_irq parameters
>
> - Reworded the commit message for "blk-mq: issue warning when offlining
> hctx with online isolcpus" for better clarity
>
> - Split the input and output parameters in the patch "lib/group_cpus: let
> group_cpu_evenly return number of groups"
>
> - Dropped the patch "sched/isolation: document HK_TYPE housekeeping
> option"
>
> - Link to v4: https://lore.kernel.org/r/20241217-isolcpus-io-queues-v4-0-5d355fbb1e14@kernel.org
>
> Changes since v3:
>
> - Added the patch "blk-mq: issue warning when offlining hctx with online
> isolcpus"
>
> - Fixed the check in group_cpus_evenly(); the condition now properly uses
> housekeeping_enabled() instead of cpumask_weight(), as the latter always
> returns a valid mask
>
> - Dropped the Fixes: tag from "lib/group_cpus.c: honor housekeeping config
> when grouping CPUs"
>
> - Fixed an overlong line warning in the patch "scsi: use block layer
> helpers to calculate num of queues"
>
> - Dropped the patch "sched/isolation: Add io_queue housekeeping option" in
> favor of simply documenting the housekeeping hk_type enum
>
> - Added the patch "lib/group_cpus: let group_cpu_evenly return number of
> groups"
>
> - Collected accumulated Reviewed-by and Acked-by tags
>
> - Split the patchset by moving foundational changes into a separate
> preparation series:
> https://lore.kernel.org/linux-nvme/20241202-refactor-blk-affinity-helpers-v6-0-27211e9c2cd5@kernel.org/
>
> - Link to v3: https://lore.kernel.org/r/20240806-isolcpus-io-queues-v3-0-da0eecfeaf8b@suse.de
>
> Changes since v2:
>
> - Integrated patches from Ming Lei
> (https://lore.kernel.org/all/20210709081005.421340-1-ming.lei@redhat.com/):
> "virtio: add APIs for retrieving vq affinity" and "blk-mq: introduce
> blk_mq_dev_map_queues"
>
> - Replaced all instances of blk_mq_pci_map_queues and
> blk_mq_virtio_map_queues with the new unified blk_mq_dev_map_queues
>
> - Updated and expanded the helper functions used for calculating the
> number of queues
>
> - Added the CPU-to-hctx mapping function specifically to support the
> isolcpus=io_queue parameter
>
> - Documented the hk_type enum and the newly introduced isolcpus=io_queue
> parameter
>
> - Added the patch "scsi: pm8001: do not overwrite PCI queue mapping"
>
> - Link to v2: https://lore.kernel.org/r/20240627-isolcpus-io-queues-v2-0-26a32e3c4f75@suse.de
>
> Changes since v1:
>
> - Updated the feature documentation for clarity and completeness
>
> - Split the blk/nvme-pci patch into smaller, logical commits
>
> - Dropped the HK_TYPE_IO_QUEUE macro in favor of reusing
> HK_TYPE_MANAGED_IRQ
>
> - Link to v1: https://lore.kernel.org/r/20240621-isolcpus-io-queues-v1-0-8b169bf41083@suse.de
>
> Aaron Tomlin (2):
> sched/isolation: Prevent out-of-bounds read in isolcpus= boot
> parameter parser
> genirq/affinity: Restrict managed IRQ affinity to housekeeping CPUs
>
> Daniel Wagner (7):
> scsi: aacraid: use block layer helpers to calculate num of queues
> lib/group_cpus: remove dead !SMP code
> lib/group_cpus: Add group_mask_cpus_evenly()
> isolation: Introduce managed_irq_strict isolcpus type
> blk-mq: use hk cpus only when isolcpus=managed_irq_strict is enabled
> blk-mq: prevent offlining hk CPUs with associated online isolated CPUs
> docs: add managed_irq_strict flag to isolcpus
>
> .../admin-guide/kernel-parameters.txt | 37 +++-
> Documentation/core-api/housekeeping.rst | 6 +-
> block/blk-mq-cpumap.c | 163 ++++++++++++++++--
> block/blk-mq.c | 63 +++++++
> drivers/scsi/aacraid/comminit.c | 3 +-
> include/linux/group_cpus.h | 3 +
> include/linux/sched/isolation.h | 1 +
> kernel/irq/affinity.c | 29 +++-
> kernel/sched/isolation.c | 18 ++
> lib/group_cpus.c | 112 ++++++++++--
> 10 files changed, 395 insertions(+), 40 deletions(-)
>
> --
> 2.55.0
>
--
Aaron Tomlin
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2026-09-10 18:26 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-09-10 16:42 [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 1/9] scsi: aacraid: use block layer helpers to calculate num of queues Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 2/9] lib/group_cpus: remove dead !SMP code Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 3/9] lib/group_cpus: Add group_mask_cpus_evenly() Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 4/9] sched/isolation: Prevent out-of-bounds read in isolcpus= boot parameter parser Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 5/9] isolation: Introduce managed_irq_strict isolcpus type Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 6/9] blk-mq: use hk cpus only when isolcpus=managed_irq_strict is enabled Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 7/9] blk-mq: prevent offlining hk CPUs with associated online isolated CPUs Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 8/9] genirq/affinity: Restrict managed IRQ affinity to housekeeping CPUs Aaron Tomlin
2026-09-10 16:42 ` [PATCH v16 9/9] docs: add managed_irq_strict flag to isolcpus Aaron Tomlin
2026-09-10 18:26 ` [PATCH v16 0/9] blk: honor isolcpus configuration Aaron Tomlin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®