Re: [PATCH v2] lib/group_cpus: rotate extra groups to avoid IRQ stacking

[Naman Jain <namjain@linux.microsoft.com>]

On 8/26/2026 7:31 AM, Michael Kelley wrote:
> From: Naman Jain <namjain@linux.microsoft.com> Sent: Sunday, August 9, 2026 11:22 PM
> 
> [snip]
> 
>> @@ -510,6 +633,8 @@ struct cpumask *group_cpus_evenly(unsigned int numgrps, unsigned int *nummasks)
>>   	if (!masks)
>>   		goto fail_node_to_cpumask;
>>
>> +	spread_offset = (unsigned int)atomic_fetch_inc(&group_spread_cnt);
>> +
>>   	build_node_to_cpumask(node_to_cpumask);
>>
> 
> One additional observation:  In my testing, group_cpus_evenly() is
> often called with numgrps set to 1. This happens in the block "loop"
> devices (drivers/block/loop.c) and for the NVMe admin queue. In
> these cases, the spread_offset is never used, but group_spread_cnt
> gets incremented anyway. Incrementing for NVMe admin queues
> tends to dirty the spreading for multiple NVMe devices with the
> same configuration because it is usually interleaved with the
> spreading of the main NVMe I/O queues.
> 
> To improve this, I changed the above code to this:
> 
> +       if (numgrps == 1)
> +               spread_offset = 0;
> +       else
> +               spread_offset = (unsigned int)atomic_fetch_inc(&group_spread_cnt);
> 
> With this change, my configuration #1 (Azure L48s v2 VM) is noticeably
> better.  All CPUs in NUMA node 1 have either 3 or 4 IRQs assigned. NUMA
> node 0 ranges from 3 to 5 IRQs, but that's partly because the NUMA
> nodes themselves aren't balanced, as previously discussed. With your
> change to apply group_spread_cnt to the NUMA nodes, and my change
> above, my config #1 is likely to work out very near optimal. Of course,
> there's no guarantee that some other device won't increment
> group_spread_cnt and dirty things, but for the typical case it probably
> works very well.
> 

Thanks for the suggestion, I tried this and it works fine.

> This change to skip incrementing group_spread_cnt when numgrps == 1
> doesn't help my arm64 configs. I'm still thinking about ways to do better
> when there aren't any clusters. I have an idea that I'm experimenting
> with, but it may be a few more days before I reach any conclusions.
> 
> Michael

Acked.

Regards,
Naman