[PATCH v2] arch_topology: Make cluster topology span at least SMT CPUs

[PATCH v2] arch_topology: Make cluster topology span at least SMT CPUs

[Yicong Yang]

From: Yicong Yang <yangyicong@hisilicon.com>

Currently cpu_clustergroup_mask() will return CPU mask if cluster span more
or the same CPUs as cpu_coregroup_mask(). This will result topology borken
on non-Cluster SMT machines when building with CONFIG_SCHED_CLUSTER=y.

Test with:
qemu-system-aarch64 -enable-kvm -machine virt \
 -net none \
 -cpu host \
 -bios ./QEMU_EFI.fd \
 -m 2G \
 -smp 48,sockets=2,cores=12,threads=2 \
 -kernel $Image \
 -initrd $Rootfs \
 -nographic \
 -append "rdinit=init console=ttyAMA0 sched_verbose loglevel=8"

We'll get below error:
[    3.084568] BUG: arch topology borken
[    3.084570]      the SMT domain not a subset of the CLS domain

Since cluster is a level higher than SMT, fix this by making cluster
spans at least SMT CPUs.

Cc: Sudeep Holla <sudeep.holla@arm.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Ionela Voinescu <ionela.voinescu@arm.com>
Fixes: bfcc4397435d ("arch_topology: Limit span of cpu_clustergroup_mask()")
Signed-off-by: Yicong Yang <yangyicong@hisilicon.com>
Reviewed-by: Sudeep Holla <sudeep.holla@arm.com>
Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com>
---
Change since v1:
- mention the kernel config CONFIG_SCHED_CLUSTER=y, per Ionela
Link:https://lore.kernel.org/lkml/20220823073044.58697-1-yangyicong@huawei.com/

 drivers/base/arch_topology.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index 0424b59b695e..0056a1273275 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -724,7 +724,7 @@ const struct cpumask *cpu_clustergroup_mask(int cpu)
 	 */
 	if (cpumask_subset(cpu_coregroup_mask(cpu),
 			   &cpu_topology[cpu].cluster_sibling))
-		return get_cpu_mask(cpu);
+		return cpu_smt_mask(cpu);
 
 	return &cpu_topology[cpu].cluster_sibling;
 }
-- 
2.24.0

Re: [PATCH v2] arch_topology: Make cluster topology span at least SMT CPUs

[Greg KH]

On Thu, Aug 25, 2022 at 05:20:07PM +0800, Yicong Yang wrote:
> From: Yicong Yang <yangyicong@hisilicon.com>
> 
> Currently cpu_clustergroup_mask() will return CPU mask if cluster span more
> or the same CPUs as cpu_coregroup_mask(). This will result topology borken
> on non-Cluster SMT machines when building with CONFIG_SCHED_CLUSTER=y.
> 
> Test with:
> qemu-system-aarch64 -enable-kvm -machine virt \
>  -net none \
>  -cpu host \
>  -bios ./QEMU_EFI.fd \
>  -m 2G \
>  -smp 48,sockets=2,cores=12,threads=2 \
>  -kernel $Image \
>  -initrd $Rootfs \
>  -nographic \
>  -append "rdinit=init console=ttyAMA0 sched_verbose loglevel=8"
> 
> We'll get below error:
> [    3.084568] BUG: arch topology borken
> [    3.084570]      the SMT domain not a subset of the CLS domain
> 
> Since cluster is a level higher than SMT, fix this by making cluster
> spans at least SMT CPUs.
> 
> Cc: Sudeep Holla <sudeep.holla@arm.com>
> Cc: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Ionela Voinescu <ionela.voinescu@arm.com>
> Fixes: bfcc4397435d ("arch_topology: Limit span of cpu_clustergroup_mask()")
> Signed-off-by: Yicong Yang <yangyicong@hisilicon.com>
> Reviewed-by: Sudeep Holla <sudeep.holla@arm.com>
> Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com>
> ---
> Change since v1:
> - mention the kernel config CONFIG_SCHED_CLUSTER=y, per Ionela
> Link:https://lore.kernel.org/lkml/20220823073044.58697-1-yangyicong@huawei.com/
> 
>  drivers/base/arch_topology.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
> index 0424b59b695e..0056a1273275 100644
> --- a/drivers/base/arch_topology.c
> +++ b/drivers/base/arch_topology.c
> @@ -724,7 +724,7 @@ const struct cpumask *cpu_clustergroup_mask(int cpu)
>  	 */
>  	if (cpumask_subset(cpu_coregroup_mask(cpu),
>  			   &cpu_topology[cpu].cluster_sibling))
> -		return get_cpu_mask(cpu);
> +		return cpu_smt_mask(cpu);

As the kernel-test robot reported, this breaks the build on some
architectures, so I'll go and revert it from my tree now.

thanks,

greg k-h

Re: [PATCH v2] arch_topology: Make cluster topology span at least SMT CPUs

[Yicong Yang]

On 2022/9/3 14:00, Greg KH wrote:
> On Thu, Aug 25, 2022 at 05:20:07PM +0800, Yicong Yang wrote:
>> From: Yicong Yang <yangyicong@hisilicon.com>
>>
>> Currently cpu_clustergroup_mask() will return CPU mask if cluster span more
>> or the same CPUs as cpu_coregroup_mask(). This will result topology borken
>> on non-Cluster SMT machines when building with CONFIG_SCHED_CLUSTER=y.
>>
>> Test with:
>> qemu-system-aarch64 -enable-kvm -machine virt \
>>  -net none \
>>  -cpu host \
>>  -bios ./QEMU_EFI.fd \
>>  -m 2G \
>>  -smp 48,sockets=2,cores=12,threads=2 \
>>  -kernel $Image \
>>  -initrd $Rootfs \
>>  -nographic \
>>  -append "rdinit=init console=ttyAMA0 sched_verbose loglevel=8"
>>
>> We'll get below error:
>> [    3.084568] BUG: arch topology borken
>> [    3.084570]      the SMT domain not a subset of the CLS domain
>>
>> Since cluster is a level higher than SMT, fix this by making cluster
>> spans at least SMT CPUs.
>>
>> Cc: Sudeep Holla <sudeep.holla@arm.com>
>> Cc: Vincent Guittot <vincent.guittot@linaro.org>
>> Cc: Ionela Voinescu <ionela.voinescu@arm.com>
>> Fixes: bfcc4397435d ("arch_topology: Limit span of cpu_clustergroup_mask()")
>> Signed-off-by: Yicong Yang <yangyicong@hisilicon.com>
>> Reviewed-by: Sudeep Holla <sudeep.holla@arm.com>
>> Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com>
>> ---
>> Change since v1:
>> - mention the kernel config CONFIG_SCHED_CLUSTER=y, per Ionela
>> Link:https://lore.kernel.org/lkml/20220823073044.58697-1-yangyicong@huawei.com/
>>
>>  drivers/base/arch_topology.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
>> index 0424b59b695e..0056a1273275 100644
>> --- a/drivers/base/arch_topology.c
>> +++ b/drivers/base/arch_topology.c
>> @@ -724,7 +724,7 @@ const struct cpumask *cpu_clustergroup_mask(int cpu)
>>  	 */
>>  	if (cpumask_subset(cpu_coregroup_mask(cpu),
>>  			   &cpu_topology[cpu].cluster_sibling))
>> -		return get_cpu_mask(cpu);
>> +		return cpu_smt_mask(cpu);
> 
> As the kernel-test robot reported, this breaks the build on some
> architectures, so I'll go and revert it from my tree now.
> 

Sorry for the regression.

The build error is because cpu_smt_mask() is defined only when CONFIG_SCHED_SMT=y
so we'll fail when CONFIG_SCHED_SMT=n. Should use topology_sibling_cpumask() to
get SMT mask here. Will respin this.

Thanks.