From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752354AbdEJKbt (ORCPT ); Wed, 10 May 2017 06:31:49 -0400 Received: from mx08-00252a01.pphosted.com ([91.207.212.211]:44381 "EHLO mx08-00252a01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751301AbdEJKbs (ORCPT ); Wed, 10 May 2017 06:31:48 -0400 Subject: Re: [PATCH] irq_bcm2836: Send event when onlining sleeping cores To: Marc Zyngier References: <87ziemm0fb.fsf@eliezer.anholt.net> <1e871525-3298-f16d-4f95-a44bc78068b5@arm.com> <87o9v1c37r.fsf@eliezer.anholt.net> <7f6f5e05-07b1-e64d-401c-a189623c22a0@arm.com> <3b9a0ec6-fc9e-5cd0-e40a-e91b5926f53e@raspberrypi.org> <93a69cf4-cbf6-e0a7-b7e3-d0e53040efe4@arm.com> <187815f0-1316-ee87-883d-11304064f8f9@raspberrypi.org> <06fad93c-a850-a99a-ae60-7306f9179032@arm.com> <87tw4thyzu.fsf@on-the-bus.cambridge.arm.com> <907820fc-c100-dad4-cf4f-95aa330667c5@arm.com> Cc: Eric Anholt , Thomas Gleixner , Jason Cooper , Florian Fainelli , Ray Jui , Scott Branden , bcm-kernel-feedback-list@broadcom.com, linux-kernel@vger.kernel.org, linux-rpi-kernel@lists.infradead.org From: Phil Elwell Message-ID: <1bbd3527-3328-b9c0-e9e0-d37092850a91@raspberrypi.org> Date: Wed, 10 May 2017 11:31:46 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <907820fc-c100-dad4-cf4f-95aa330667c5@arm.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-05-10_07:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_spam_notspam policy=outbound_spam score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705100073 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/05/2017 11:09, Marc Zyngier wrote: > On 10/05/17 10:05, Phil Elwell wrote: >> On 10/05/2017 09:55, Marc Zyngier wrote: >>> On Wed, May 10 2017 at 9:27:10 am BST, Phil Elwell wrote: >>>> On 10/05/2017 08:42, Marc Zyngier wrote: >>>>> On 09/05/17 20:02, Phil Elwell wrote: >>>>>> On 09/05/2017 19:53, Marc Zyngier wrote: >>>>>>> On 09/05/17 19:52, Phil Elwell wrote: >>>>>>>> On 09/05/2017 19:14, Marc Zyngier wrote: >>>>>>>>> On 09/05/17 19:08, Eric Anholt wrote: >>>>>>>>>> Marc Zyngier writes: >>>>>>>>>> >>>>>>>>>>> On 09/05/17 17:59, Eric Anholt wrote: >>>>>>>>>>>> Phil Elwell writes: >>>>>>>>>>>> >>>>>>>>>>>>> In order to reduce power consumption and bus traffic, it is sensible >>>>>>>>>>>>> for secondary cores to enter a low-power idle state when waiting to >>>>>>>>>>>>> be started. The wfe instruction causes a core to wait until an event >>>>>>>>>>>>> or interrupt arrives before continuing to the next instruction. >>>>>>>>>>>>> The sev instruction sends a wakeup event to the other cores, so call >>>>>>>>>>>>> it from bcm2836_smp_boot_secondary, the function that wakes up the >>>>>>>>>>>>> waiting cores during booting. >>>>>>>>>>>>> >>>>>>>>>>>>> It is harmless to use this patch without the corresponding change >>>>>>>>>>>>> adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated >>>>>>>>>>>>> and this patch is not applied then the other cores will sleep forever. >>>>>>>>>>>>> >>>>>>>>>>>>> See: https://github.com/raspberrypi/linux/issues/1989 >>>>>>>>>>>>> >>>>>>>>>>>>> Signed-off-by: Phil Elwell >>>>>>>>>>>>> --- >>>>>>>>>>>>> drivers/irqchip/irq-bcm2836.c | 3 +++ >>>>>>>>>>>>> 1 file changed, 3 insertions(+) >>>>>>>>>>>>> >>>>>>>>>>>>> diff --git a/drivers/irqchip/irq-bcm2836.c b/drivers/irqchip/irq-bcm2836.c >>>>>>>>>>>>> index e10597c..6dccdf9 100644 >>>>>>>>>>>>> --- a/drivers/irqchip/irq-bcm2836.c >>>>>>>>>>>>> +++ b/drivers/irqchip/irq-bcm2836.c >>>>>>>>>>>>> @@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu, >>>>>>>>>>>>> writel(secondary_startup_phys, >>>>>>>>>>>>> intc.base + LOCAL_MAILBOX3_SET0 + 16 * cpu); >>>>>>>>>>>>> >>>>>>>>>>>>> + dsb(sy); /* Ensure write has completed before waking the other CPUs */ >>>>>>>>>>>>> + sev(); >>>>>>>>>>>>> + >>>>>>>>>>>>> return 0; >>>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> This is also the behavior that the standard arm64 spin-table >>>>>>>>>>>> method has, >>>>>>>>>>>> which we unfortunately can't quite use. >>>>>>>>>>> >>>>>>>>>>> And why is that so? Why do you have to reinvent the wheel (and hide the >>>>>>>>>>> cloned wheel in an interrupt controller driver)? >>>>>>>>>>> >>>>>>>>>>> That doesn't seem right to me. >>>>>>>>>> >>>>>>>>>> The armv8 stubs (firmware-supplied code in the low page that do the >>>>>>>>>> spinning) do actually implement arm64's spin-table method. It's the >>>>>>>>>> armv7 stubs that use these registers in the irqchip instead of plain >>>>>>>>>> addresses in system memory. >>>>>>>>> >>>>>>>>> Let's put ARMv7 aside for the time being. If your firmware already >>>>>>>>> implements spin-tables, why don't you simply use that at least on arm64? >>>>>>>> >>>>>>>> We do. >>>>>>> >>>>>>> Obviously not the way it is intended if you have to duplicate the core >>>>>>> architectural code in the interrupt controller driver, which couldn't >>>>>>> care less. >>>>>> >>>>>> If we were using this method on arm64 then the other cores would not start up >>>>>> because armstub8.S has always included a wfe. Nothing in the commit mentions >>>>>> arm64 - this is an ARCH=arm fix. >>>>> >>>>> Thanks for the clarification, which you could have added to the commit >>>>> message. >>>>> >>>>> The question still remains: why do we have CPU bring-up code in an >>>>> interrupt controller, instead of having it in the architecture code? >>>>> >>>>> The RPi-2 is the *only* platform to have its SMP bringup code outside of >>>>> arch/arm, so the first course of action would be to move that code where >>>>> it belongs. >>>> >>>> You were CC'd on the commit (41f4988cc287e5f836d3f6620c9f900bc9b560e9) that >>>> introduced bcm2836_smp_boot_secondary - it seems strange to start objecting >>>> now. >>> >>> Well, I'm far from being perfect. If I had noticed it, I'd have NACKed >>> it. >>> >>>> Yes, I think it is odd that it didn't go into arch/arm/mach-bcm, but in >>>> the interests of making changes in small, independent steps, do you have a >>>> problem with this commit? >>> >>> On its own, no. I'm just not keen on adding more unrelated stuff to this >>> file, so let's start with dealing with the original bug, and you can >>> then add this fix on top. >> >> That's an interesting use of the word "bug". From Wikipedia: >> >> "A software bug is an error, flaw, failure or fault in a computer program or >> system that causes it to produce an incorrect or unexpected result, or to >> behave in unintended ways." > > Whatever. Should I call it "pile of crap dumped in unsuitable locations" > instead? What does Wikipedia says about it? > >> Although your concerns are valid, the faults you are objecting to are not causing >> a malfunction of any kind. If we were to update the RPi firmware before this >> patch was merged then upstream users would be left with one wheel on their wagon. > > And that'd be your problem, not mine. Look, you can argue around this > all day, or you can fix this mess. Your choice. Is that the opinion of all here? Phil