[PATCH 1/2] ARM: dts: imx6q: add pmu interrupt-affinity

[PATCH 1/2] ARM: dts: imx6q: add pmu interrupt-affinity

[Stefan Agner]

Explicitly specify interrupt affinity to avoid HW perfevents
need to guess. This avoids the following error upon boot:
  hw perfevents: no interrupt-affinity property for /pmu, guessing.

Specifying all four CPUs shows no aversive effects on i.MX 6Dual
SoCs.

Signed-off-by: Stefan Agner <stefan@agner.ch>
---
 arch/arm/boot/dts/imx6q.dtsi | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
index 8381d24eff7d..d2c1977c8b16 100644
--- a/arch/arm/boot/dts/imx6q.dtsi
+++ b/arch/arm/boot/dts/imx6q.dtsi
@@ -537,6 +537,13 @@
 			<0x28 0x0000000c>; /* DCIC2_MUX_CTL */
 };
 
+&pmu {
+	interrupt-affinity = <&{/cpus/cpu@0}>,
+			     <&{/cpus/cpu@1}>,
+			     <&{/cpus/cpu@2}>,
+			     <&{/cpus/cpu@3}>;
+};
+
 &vpu {
 	compatible = "fsl,imx6q-vpu", "cnm,coda960";
 };
-- 
2.20.1

[PATCH 2/2] ARM: dts: imx6dl: add pmu interrupt-affinity

[Stefan Agner]

Explicitly specify interrupt affinity to avoid HW perfevents
need to guess. This avoids the following error upon boot:
  hw perfevents: no interrupt-affinity property for /pmu, guessing.

Specifying all four CPUs shows no aversive effects on i.MX 6Solo
SoCs.

Signed-off-by: Stefan Agner <stefan@agner.ch>
---
 arch/arm/boot/dts/imx6dl-tx6s-8034.dts | 4 ++++
 arch/arm/boot/dts/imx6dl-tx6s-8035.dts | 4 ++++
 arch/arm/boot/dts/imx6dl.dtsi          | 5 +++++
 3 files changed, 13 insertions(+)

diff --git a/arch/arm/boot/dts/imx6dl-tx6s-8034.dts b/arch/arm/boot/dts/imx6dl-tx6s-8034.dts
index 9eb2ef17339c..f1606f63d687 100644
--- a/arch/arm/boot/dts/imx6dl-tx6s-8034.dts
+++ b/arch/arm/boot/dts/imx6dl-tx6s-8034.dts
@@ -68,3 +68,7 @@
 		MX6QDL_PAD_SD3_CMD__GPIO7_IO02		0x170b0 /* SD1 CD */
 	>;
 };
+
+&pmu {
+	interrupt-affinity = <&{/cpus/cpu@0}>;
+};
diff --git a/arch/arm/boot/dts/imx6dl-tx6s-8035.dts b/arch/arm/boot/dts/imx6dl-tx6s-8035.dts
index a5532ecc18c5..b3bf1f80aff0 100644
--- a/arch/arm/boot/dts/imx6dl-tx6s-8035.dts
+++ b/arch/arm/boot/dts/imx6dl-tx6s-8035.dts
@@ -84,3 +84,7 @@
 		>;
 	};
 };
+
+&pmu {
+	interrupt-affinity = <&{/cpus/cpu@0}>;
+};
diff --git a/arch/arm/boot/dts/imx6dl.dtsi b/arch/arm/boot/dts/imx6dl.dtsi
index f0607eb41df4..3c8b533b3039 100644
--- a/arch/arm/boot/dts/imx6dl.dtsi
+++ b/arch/arm/boot/dts/imx6dl.dtsi
@@ -386,6 +386,11 @@
 			<0x28 0x0000000c>; /* DCIC2_MUX_CTL */
 };
 
+&pmu {
+	interrupt-affinity = <&{/cpus/cpu@0}>,
+			     <&{/cpus/cpu@1}>;
+};
+
 &vpu {
 	compatible = "fsl,imx6dl-vpu", "cnm,coda960";
 };
-- 
2.20.1

Re: [PATCH 1/2] ARM: dts: imx6q: add pmu interrupt-affinity

[Lucas Stach]

Am Freitag, den 18.01.2019, 14:59 +0100 schrieb Stefan Agner:
> Explicitly specify interrupt affinity to avoid HW perfevents
> need to guess. This avoids the following error upon boot:
>   hw perfevents: no interrupt-affinity property for /pmu, guessing.
> 
But then it isn't correct either AFAICS. On i.MX6 all the PMU IRQs are
ORed together into a single SPI, instead of each core dealing with its
own PPI. So pretending that there are more IRQs with affinity to each
core isn't the right thing to do, no?

Regards,
Lucas

> Specifying all four CPUs shows no aversive effects on i.MX 6Dual
> SoCs.
> 
> > Signed-off-by: Stefan Agner <stefan@agner.ch>
> ---
>  arch/arm/boot/dts/imx6q.dtsi | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
> index 8381d24eff7d..d2c1977c8b16 100644
> --- a/arch/arm/boot/dts/imx6q.dtsi
> +++ b/arch/arm/boot/dts/imx6q.dtsi
> @@ -537,6 +537,13 @@
> >  			<0x28 0x0000000c>; /* DCIC2_MUX_CTL */
>  };
>  
> +&pmu {
> > > +	interrupt-affinity = <&{/cpus/cpu@0}>,
> > > +			     <&{/cpus/cpu@1}>,
> > > +			     <&{/cpus/cpu@2}>,
> > > +			     <&{/cpus/cpu@3}>;
> +};
> +
>  &vpu {
> >  	compatible = "fsl,imx6q-vpu", "cnm,coda960";
>  };

Re: [PATCH 1/2] ARM: dts: imx6q: add pmu interrupt-affinity

[Stefan Agner]

On 18.01.2019 15:12, Lucas Stach wrote:
> Am Freitag, den 18.01.2019, 14:59 +0100 schrieb Stefan Agner:
>> Explicitly specify interrupt affinity to avoid HW perfevents
>> need to guess. This avoids the following error upon boot:
>>   hw perfevents: no interrupt-affinity property for /pmu, guessing.
>>
> But then it isn't correct either AFAICS. On i.MX6 all the PMU IRQs are
> ORed together into a single SPI, instead of each core dealing with its
> own PPI. So pretending that there are more IRQs with affinity to each
> core isn't the right thing to do, no?
> 

Oh I see, we only have a single interrupt in the i.MX 6 case.

I agree, this patches are wrong.

Hm, but why does hw perf then think it needs to guess? Doesn't seem hard
to guess right if there is only one choice...

We probably need to do something like this?

--- a/drivers/perf/arm_pmu_platform.c
+++ b/drivers/perf/arm_pmu_platform.c
@@ -122,7 +122,8 @@ static int pmu_parse_irqs(struct arm_pmu *pmu)
                        return pmu_parse_percpu_irq(pmu, irq);
        }
 
-       if (nr_cpu_ids != 1 && !pmu_has_irq_affinity(pdev->dev.of_node))
{
+       if ((nr_cpu_ids != 1 || num_irqs != 1) &&
+           !pmu_has_irq_affinity(pdev->dev.of_node)) {
                pr_warn("no interrupt-affinity property for %pOF,
guessing.\n",
                        pdev->dev.of_node);
        }

--
Stefan



> Regards,
> Lucas
> 
>> Specifying all four CPUs shows no aversive effects on i.MX 6Dual
>> SoCs.
>>
>> > Signed-off-by: Stefan Agner <stefan@agner.ch>
>> ---
>>  arch/arm/boot/dts/imx6q.dtsi | 7 +++++++
>>  1 file changed, 7 insertions(+)
>>
>> diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
>> index 8381d24eff7d..d2c1977c8b16 100644
>> --- a/arch/arm/boot/dts/imx6q.dtsi
>> +++ b/arch/arm/boot/dts/imx6q.dtsi
>> @@ -537,6 +537,13 @@
>> >  			<0x28 0x0000000c>; /* DCIC2_MUX_CTL */
>>  };
>>  
>> +&pmu {
>> > > +	interrupt-affinity = <&{/cpus/cpu@0}>,
>> > > +			     <&{/cpus/cpu@1}>,
>> > > +			     <&{/cpus/cpu@2}>,
>> > > +			     <&{/cpus/cpu@3}>;
>> +};
>> +
>>  &vpu {
>> >  	compatible = "fsl,imx6q-vpu", "cnm,coda960";
>>  };

Re: [PATCH 1/2] ARM: dts: imx6q: add pmu interrupt-affinity

[Stefan Agner]

[adding Will/Mark]

On 18.01.2019 16:41, Stefan Agner wrote:
> On 18.01.2019 15:12, Lucas Stach wrote:
>> Am Freitag, den 18.01.2019, 14:59 +0100 schrieb Stefan Agner:
>>> Explicitly specify interrupt affinity to avoid HW perfevents
>>> need to guess. This avoids the following error upon boot:
>>>   hw perfevents: no interrupt-affinity property for /pmu, guessing.
>>>
>> But then it isn't correct either AFAICS. On i.MX6 all the PMU IRQs are
>> ORed together into a single SPI, instead of each core dealing with its
>> own PPI. So pretending that there are more IRQs with affinity to each
>> core isn't the right thing to do, no?
>>
> 
> Oh I see, we only have a single interrupt in the i.MX 6 case.
> 
> I agree, this patches are wrong.
> 
> Hm, but why does hw perf then think it needs to guess? Doesn't seem hard
> to guess right if there is only one choice...
> 
> We probably need to do something like this?
> 
> --- a/drivers/perf/arm_pmu_platform.c
> +++ b/drivers/perf/arm_pmu_platform.c
> @@ -122,7 +122,8 @@ static int pmu_parse_irqs(struct arm_pmu *pmu)
>                         return pmu_parse_percpu_irq(pmu, irq);
>         }
>  
> -       if (nr_cpu_ids != 1 && !pmu_has_irq_affinity(pdev->dev.of_node))
> {
> +       if ((nr_cpu_ids != 1 || num_irqs != 1) &&
> +           !pmu_has_irq_affinity(pdev->dev.of_node)) {
>                 pr_warn("no interrupt-affinity property for %pOF,
> guessing.\n",
>                         pdev->dev.of_node);
>         }

Yeah, I see, it was on Friday... if anything, it should be:

-       if (nr_cpu_ids != 1 && !pmu_has_irq_affinity(pdev->dev.of_node))
{
+       if (nr_cpu_ids != 1 && num_irqs != 1 &&
+           !pmu_has_irq_affinity(pdev->dev.of_node)) {

However, I realized that arm_pmu_platform actually currently only
assigns the one IRQ to the first CPU. This leads to perf not working if
a process is scheduled on another CPU then the first.

I could reproduce this. PID 763 is a long running task.

root@colibri-imx6-05051054:~# taskset -p 0x1 763
pid 763's current affinity mask: 3
pid 763's new affinity mask: 1


root@colibri-imx6-05051054:~# perf stat -e cpu-cycles -p 763
^C
 Performance counter stats for process id '763':

           7581021      cpu-cycles

       1.222248490 seconds time elapsed


root@colibri-imx6-05051054:~# taskset -p 0x2 763
pid 763's current affinity mask: 1
pid 763's new affinity mask: 2


root@colibri-imx6-05051054:~# perf stat -e cpu-cycles -p 763
^C
 Performance counter stats for process id '763':

     <not counted>      cpu-cycles                                      
             (0.00%)

       1.050253575 seconds time elapsed

I guess we need to tell the PMU driver that a single IRQ should be used
for all CPUs?

--
Stefan



> 
> 
>> Regards,
>> Lucas
>>
>>> Specifying all four CPUs shows no aversive effects on i.MX 6Dual
>>> SoCs.
>>>
>>> > Signed-off-by: Stefan Agner <stefan@agner.ch>
>>> ---
>>>  arch/arm/boot/dts/imx6q.dtsi | 7 +++++++
>>>  1 file changed, 7 insertions(+)
>>>
>>> diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
>>> index 8381d24eff7d..d2c1977c8b16 100644
>>> --- a/arch/arm/boot/dts/imx6q.dtsi
>>> +++ b/arch/arm/boot/dts/imx6q.dtsi
>>> @@ -537,6 +537,13 @@
>>> >  			<0x28 0x0000000c>; /* DCIC2_MUX_CTL */
>>>  };
>>>  
>>> +&pmu {
>>> > > +	interrupt-affinity = <&{/cpus/cpu@0}>,
>>> > > +			     <&{/cpus/cpu@1}>,
>>> > > +			     <&{/cpus/cpu@2}>,
>>> > > +			     <&{/cpus/cpu@3}>;
>>> +};
>>> +
>>>  &vpu {
>>> >  	compatible = "fsl,imx6q-vpu", "cnm,coda960";
>>>  };