From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752484AbeCWJTv (ORCPT ); Fri, 23 Mar 2018 05:19:51 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:47300 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752031AbeCWJTt (ORCPT ); Fri, 23 Mar 2018 05:19:49 -0400 Date: Fri, 23 Mar 2018 09:19:35 +0000 Message-ID: <86o9jf174o.wl-marc.zyngier@arm.com> From: Marc Zyngier To: shankerd@codeaurora.org Cc: linux-kernel , linux-arm-kernel , Mark Rutland , Thomas Gleixner , Jason Cooper , Vikram Sethi Subject: Re: [PATCH v3] irqchip/gic-v3: Ensure GICR_CTLR.EnableLPI=0 is observed before enabling In-Reply-To: <8087e009-4eca-ab92-cbca-23ccfb3fafac@codeaurora.org> References: <1521683929-15644-1-git-send-email-shankerd@codeaurora.org> <8087e009-4eca-ab92-cbca-23ccfb3fafac@codeaurora.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.8 EasyPG/1.0.0 Emacs/25.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Organization: ARM Ltd MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 22 Mar 2018 19:41:09 +0000, Shanker Donthineni wrote: > > Hi Marc, > > On 03/22/2018 10:51 AM, Marc Zyngier wrote: > > On 22/03/18 01:58, Shanker Donthineni wrote: > >> The definition of the GICR_CTLR.RWP control bit was expanded to indicate > >> status of changing GICR_CTLR.EnableLPI from 1 to 0 is being in progress > >> or completed. Software must observe GICR_CTLR.RWP==0 after clearing > >> GICR_CTLR.EnableLPI from 1 to 0 and before writing GICR_PENDBASER and/or > >> GICR_PROPBASER, otherwise behavior is UNPREDICTABLE. > >> > >> Signed-off-by: Shanker Donthineni > >> --- > >> Changes since v2: > >> -Revert readl_relaxed_poll() usage since it's not usable in GICv3 probe(). > >> -Changes to pr_xxx messages. > >> > >> Changes since v1: > >> -Moved LPI disable code to a seperate function as Marc suggested. > >> -Mark's suggestion to use readl_relaxed_poll_timeout() helper functions. > >> > >> drivers/irqchip/irq-gic-v3-its.c | 75 +++++++++++++++++++++++++++++++------- > >> include/linux/irqchip/arm-gic-v3.h | 1 + > >> 2 files changed, 62 insertions(+), 14 deletions(-) > >> > >> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c > >> index 2cbb19c..c1e8a8e 100644 > >> --- a/drivers/irqchip/irq-gic-v3-its.c > >> +++ b/drivers/irqchip/irq-gic-v3-its.c > >> @@ -33,6 +33,7 @@ > >> #include > >> #include > >> #include > >> +#include > >> > >> #include > >> #include > > > > This hunk doesn't apply to my -next branch, but I don't think it is > > actually required either... > > > > I'll try to drop "#include " in next patch if USEC_PER_SEC > included by other header files or rebase to -next branch. > > >> @@ -1875,16 +1876,6 @@ static void its_cpu_init_lpis(void) > >> gic_data_rdist()->pend_page = pend_page; > >> } > >> > >> - /* Disable LPIs */ > >> - val = readl_relaxed(rbase + GICR_CTLR); > >> - val &= ~GICR_CTLR_ENABLE_LPIS; > >> - writel_relaxed(val, rbase + GICR_CTLR); > >> - > >> - /* > >> - * Make sure any change to the table is observable by the GIC. > >> - */ > >> - dsb(sy); > >> - > >> /* set PROPBASE */ > >> val = (page_to_phys(gic_rdists->prop_page) | > >> GICR_PROPBASER_InnerShareable | > >> @@ -3287,13 +3278,69 @@ static bool gic_rdists_supports_plpis(void) > >> return !!(gic_read_typer(gic_data_rdist_rd_base() + GICR_TYPER) & GICR_TYPER_PLPIS); > >> } > >> > >> +static int redist_disable_lpis(void) > >> +{ > >> + void __iomem *rbase = gic_data_rdist_rd_base(); > >> + u64 timeout = USEC_PER_SEC; > >> + u64 val; > >> + > >> + if (!gic_rdists_supports_plpis()) { > >> + pr_info("CPU%d: LPIs not supported\n", smp_processor_id()); > >> + return -ENXIO; > >> + } > >> + > >> + val = readl_relaxed(rbase + GICR_CTLR); > >> + if (!(val & GICR_CTLR_ENABLE_LPIS)) > >> + return 0; > >> + > >> + pr_warn("CPU%d: Booted with LPIs enabled, memory probably corrupted\n", > >> + smp_processor_id()); > >> + add_taint(TAINT_CRAP, LOCKDEP_STILL_OK); > >> + > >> + /* Disable LPIs */ > >> + val &= ~GICR_CTLR_ENABLE_LPIS; > >> + writel_relaxed(val, rbase + GICR_CTLR); > >> + > >> + /* Make sure any change to GICR_CTLR is observable by the GIC */ > >> + dsb(sy); > >> + > >> + /** > >> + * Software must observe RWP==0 after clearing GICR_CTLR.EnableLPIs > >> + * from 1 to 0 before programming GICR_PEND{PROP}BASER registers. > >> + * Bail out the driver probe() in case of timeout. > >> + */ > >> + while (readl_relaxed(rbase + GICR_CTLR) & GICR_CTLR_RWP) { > >> + if (!timeout) { > >> + pr_err("CPU%d: Failed to observe RWP==0 after disabling LPIs\n", > > > > I think you can simplify the message with something like: > > > > "Time-out disabling LPIs\n" > > > > Nobody apart from you and I really want to know about RWP... > > > > I'll change. > > >> + smp_processor_id()); > >> + return -ETIMEDOUT; > >> + } > >> + udelay(1); > >> + timeout--; > >> + } > >> + > >> + /** > >> + * After it has been written to 1, it is IMPLEMENTATION DEFINED whether > >> + * the bit GICR_CTLR.EnableLPI becomes RES1 or can be cleared to 0. > >> + * Bail out the driver probe() on systems where it's RES1. > >> + */ > >> + if (readl_relaxed(rbase + GICR_CTLR) & GICR_CTLR_ENABLE_LPIS) { > >> + pr_err("CPU%d: Failed to disable LPIs\n", smp_processor_id()); > >> + return -EBUSY; > >> + } > >> + > >> + return 0; > >> +} > >> + > >> int its_cpu_init(void) > >> { > >> if (!list_empty(&its_nodes)) { > >> - if (!gic_rdists_supports_plpis()) { > >> - pr_info("CPU%d: LPIs not supported\n", smp_processor_id()); > >> - return -ENXIO; > >> - } > >> + int ret; > >> + > >> + ret = redist_disable_lpis(); > >> + if (ret) > >> + return ret; > > > > Just realised that this is totally broken. > > > > Why do we have this in the loop? Checking the LPI support for each ITS > > was admittedly braindead (we only need to check it once per CPU), but > > now trying to disable the LPIs each time we encounter an ITS is going to > > make it go crazy and taint the kernel for no reason. > > > > Sorry, I didn't quite understand suggestions you're recommending. I don't > see any loop here, it just checks the ITS_LIST_EMPTY. > > The function its_cpu_init() is being called for each CPU coming online. > We're trying to disable GICR LPI before calling its_cpu_init_lpis() and > its_cpu_init_collection(). Newly added function redist_disable_lpis() > will be called only once per CPU but not per each ITS hardware instance. > Is something I'm missing here? No you're not. I just got confused with my own patches and completely misread yours. Sorry about that. I'll apply the patch directly with the above changes. Thanks, M. -- Jazz is not dead, it just smell funny.