From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED, USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14B18C43387 for ; Tue, 8 Jan 2019 16:45:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DFF262087F for ; Tue, 8 Jan 2019 16:45:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728794AbfAHQpS (ORCPT ); Tue, 8 Jan 2019 11:45:18 -0500 Received: from foss.arm.com ([217.140.101.70]:56050 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728075AbfAHQpS (ORCPT ); Tue, 8 Jan 2019 11:45:18 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BD04AA78; Tue, 8 Jan 2019 08:45:17 -0800 (PST) Received: from e103592.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 832BC3F5AF; Tue, 8 Jan 2019 08:45:15 -0800 (PST) Date: Tue, 8 Jan 2019 16:45:13 +0000 From: Dave Martin To: Marc Zyngier Cc: Julien Thierry , mark.rutland@arm.com, daniel.thompson@linaro.org, Ard Biesheuvel , catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, Oleg Nesterov , joel@joelfernandes.org, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH v8 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking Message-ID: <20190108164510.GB5840@e103592.cambridge.arm.com> References: <1546956464-48825-1-git-send-email-julien.thierry@arm.com> <1546956464-48825-13-git-send-email-julien.thierry@arm.com> <20190108153957.GA5840@e103592.cambridge.arm.com> <8cae8016-d2c7-3c86-9832-f4278d42ea21@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <8cae8016-d2c7-3c86-9832-f4278d42ea21@arm.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jan 08, 2019 at 03:51:18PM +0000, Marc Zyngier wrote: > On 08/01/2019 15:40, Dave Martin wrote: > > On Tue, Jan 08, 2019 at 02:07:30PM +0000, Julien Thierry wrote: > >> Instead disabling interrupts by setting the PSR.I bit, use a priority > >> higher than the one used for interrupts to mask them via PMR. > >> > >> When using PMR to disable interrupts, the value of PMR will be used > >> instead of PSR.[DAIF] for the irqflags. > >> > >> Signed-off-by: Julien Thierry > >> Suggested-by: Daniel Thompson > >> Cc: Catalin Marinas > >> Cc: Will Deacon > >> Cc: Ard Biesheuvel > >> Cc: Oleg Nesterov > >> --- > >> arch/arm64/include/asm/efi.h | 11 ++++ > >> arch/arm64/include/asm/irqflags.h | 123 +++++++++++++++++++++++++++++--------- > >> 2 files changed, 106 insertions(+), 28 deletions(-) > > > > [...] > > > >> diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h > >> index 24692ed..fa3b06f 100644 > >> --- a/arch/arm64/include/asm/irqflags.h > >> +++ b/arch/arm64/include/asm/irqflags.h > >> @@ -18,7 +18,9 @@ > > > > [...] > > > >> static inline void arch_local_irq_enable(void) > >> { > >> - asm volatile( > >> - "msr daifclr, #2 // arch_local_irq_enable" > >> - : > >> + unsigned long unmasked = GIC_PRIO_IRQON; > >> + > >> + asm volatile(ALTERNATIVE( > >> + "msr daifclr, #2 // arch_local_irq_enable\n" > >> + "nop", > >> + "msr_s " __stringify(SYS_ICC_PMR_EL1) ",%0\n" > >> + "dsb sy", > > > > I'm still not convinced these dsbs are needed. > > > > Without the dsb, we are probably not guaranteed to take a pending > > interrupt _immediately_ on unmasking, but I'm not sure that's a > > problem. > > > > What goes wrong if we omit them? > > Then the GIC doesn't know it can now deliver interrupts of a lower > priority. Only a dsb can guarantee that the GIC's view of PMR will get > updated. > > See 9.1.6 (Observability of the effects of accesses to the GIC > registers), which states: > > > Architectural execution of a DSB instruction guarantees that > — The last value written to ICC_PMR_EL1 or GICC_PMR is observed by the > associated Redistributor. > > > So yes, DSB is required. But it says neither what is means for the PMR write to be "observed by the redistributor", nor whether the DSB is required for the redistributor to observe the write at all. (So, is an implementation allowed to cached in the CPU interface indefinitely until forcibly flushed to the redistributor by a DSB, and in any case can the write's reaching the distributor in finite time or not have any effect that we care about in this case?). My reason for querying this is that temporary local masking of classes of interrupts seems an obvious use case for the PMR, and the DSB requirement flies rather in the face of this. Have we seen hardware where interrupts may stall forever upstream of the CPU interface after a PMR write, until a dsb is executed by the CPU? If so that is sad, but I guess we have to live with it. Also, is it ever important in Linux that a pending interrupt be taken immediately upon unmasking (and how do we know that said interrupt is pending)? If not, we don't care precisely when such interrupts are pended to the PE, just that such an interrupt cannot be taken before the PMR write that unmasks it. It would be insane for the self- synchronization of PMR writes to lack this guarantee (and a DSB after the PMR write would do no good anyway in that case). Happy to be put right -- I'm doubtless showing my ignorance here! Cheers ---Dave