From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C4EBC04EBF for ; Mon, 3 Dec 2018 10:38:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3274A2082F for ; Mon, 3 Dec 2018 10:38:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3274A2082F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726314AbeLCKir (ORCPT ); Mon, 3 Dec 2018 05:38:47 -0500 Received: from foss.arm.com ([217.140.101.70]:34064 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725907AbeLCKir (ORCPT ); Mon, 3 Dec 2018 05:38:47 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4802880D; Mon, 3 Dec 2018 02:38:18 -0800 (PST) Received: from [10.1.197.36] (e112298-lin.cambridge.arm.com [10.1.197.36]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 922B03F59C; Mon, 3 Dec 2018 02:38:16 -0800 (PST) Subject: Re: [PATCH v6 08/24] arm64: Unmask PMR before going idle To: Mark Rutland Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, daniel.thompson@linaro.org, joel@joelfernandes.org, marc.zyngier@arm.com, christoffer.dall@arm.com, james.morse@arm.com, catalin.marinas@arm.com, will.deacon@arm.com References: <1542023835-21446-1-git-send-email-julien.thierry@arm.com> <1542023835-21446-9-git-send-email-julien.thierry@arm.com> <20181129174436.avqjydyzvv6ubnjd@lakrids.cambridge.arm.com> <9f157e6c-531f-8316-c326-27ff013a489e@arm.com> <20181130133719.zw7wcklqayiebgap@lakrids.cambridge.arm.com> From: Julien Thierry Message-ID: <8a4cdfa8-14ad-cd06-60fc-8395a689915e@arm.com> Date: Mon, 3 Dec 2018 10:38:14 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <20181130133719.zw7wcklqayiebgap@lakrids.cambridge.arm.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 30/11/18 13:37, Mark Rutland wrote: > On Fri, Nov 30, 2018 at 10:55:47AM +0000, Julien Thierry wrote: >> On 29/11/18 17:44, Mark Rutland wrote: >>> On Mon, Nov 12, 2018 at 11:56:59AM +0000, Julien Thierry wrote: > >>>> + mov x2, #GIC_PRIO_IRQON >>>> + msr_s SYS_ICC_PMR_EL1, x2 // unmask PMR >>>> dsb sy // WFI may enter a low-power mode >>> >>> Is the DSB SY sufficient and necessary to synchronise the update of >>> SYS_ICC_PMR_EL1? We don't need an ISB too? >> >> DSB SY is necessary when we unmask interrupts to make sure that the >> redistributor sees the update to PMR before we do WFI. My understanding >> is that the resdistributor is free to stop forwarding interrupts to the >> CPU interface if from its point of view those interrupts don't have a >> high enough priority. >> >> As for the ISB, I don't think we need one because writes to PMR are >> self-synchronizing, so the write to PMR should be seen before DSB SY and >> wfi. > > Having looked at ARM IHI 0069D, 8.1.6 "Observability of the effects of > accesses to the GIC registers", I think I agree. > > My specific concern was that a CPU might complete the DSB before the > MSR, but I think it's clear per the GIC spec it's clear that an ISB is > not expected between the MSR and DSB, even if that's unusual. > >>>> wfi >>>> + msr_s SYS_ICC_PMR_EL1, x1 // restore PMR >>> >>> Likewise, we don't need any barriers here before we poke DAIF? >> >> Here we don't need DSB SY because the value being restored is either: >> - GIC_PRIO_IRQON which is the same as the current value, the >> redistributor is already aware of it. >> - GIC_PRIO_IRQOFF and the self-synchronization of PMR ensures that no >> interrupts with priorities lower than the value of PMR can be taken >> (this does not require to be seen by the redistributor). >> >> For the ISB, I have this small doubt about whether it is needed between >> WFI and MSR PMR. But there is this bit in the ARM ARM section D12.1.3 >> "General behavior of accesses to the AArch64 System registers", >> subsection "Synchronization requirements for AArch64 System registers": >> >> "Direct writes using the instructions in Table D11-2 on page D11-2660 >> require synchronization before software can rely on the effects of >> changes to the System registers to affect instructions appearing in >> program order after the direct write to the System register. Direct >> writes to these registers are not allowed to affect any instructions >> appearing in program order before the direct write." >> >> ICC_PMR_EL1 is part of the mentioned table. > > I think that's a defect in the ARM ARM, given it disagrees with the GIC > spec. > >> And reordering the direct write to PMR before the WFI would definitely >> affect the WFI instruction, so my interpretation is that this would >> not be allowed by the architecture. So I don't think we need the ISB >> either, but my understanding could be wrong. > > We already assume that a DSB can't be re-ordered w.r.t. the WFI, so as > long as the DSB can't complete before the MSR, I think we're good. > >>> >>>> + msr daif, x0 // restore I bit >>>> ret >>>> ENDPROC(cpu_do_idle) >>> >>> If we build without CONFIG_ARM64_PSEUDO_NMI surely we don't want to emit >>> the alternative? >>> >>> How about we move this to C, and have something like the below? >>> >>> For the !CONFIG_ARM64_PSEUDO_NMI case it generates identical assembly to the >>> existing cpu_do_idle(). Note that I've assumed we don't need barriers, which >>> (as above) I'm not certain of. >>> >>> Thanks, >>> Mark. >>> >>> ---->8---- >>> diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c >>> index 7f1628effe6d..ccd2ad8c5e2f 100644 >>> --- a/arch/arm64/kernel/process.c >>> +++ b/arch/arm64/kernel/process.c >>> @@ -73,6 +73,40 @@ EXPORT_SYMBOL_GPL(pm_power_off); >>> >>> void (*arm_pm_restart)(enum reboot_mode reboot_mode, const char *cmd); >>> >>> +static inline void __cpu_do_idle(void) >>> +{ >>> + /* WFI may enter a low-power mode */ >>> + dsb(sy); >>> + wfi(); >>> +} >>> + >>> +/* >>> + * When using priority masking we need to take extra care, etc. >>> + */ >>> +static inline void __cpu_do_idle_irqprio(void) >>> +{ >>> + unsigned long flags = arch_local_irq_save(); >> >> The issue with this is that in patch 10, arch_local_irq_* functions >> toggle PMR rather than PSR.I. >> >> I could use local_daif_mask but I don't think disabling debug and async >> is good. Otherwise and can do a small bit of inline assembly and have >> something like: > > Can we factor out the existing arch_local_irq_save() somehow? > I'm not sure I understand what you're suggesting. Using individual accessors for PMR and PSR.I instead of arch_local_irq_save? I am just a bit concerned about having too many functions to play with either (especially since most of the time those functions end up being single use). Thanks, -- Julien Thierry