From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D4AFC433EF for ; Mon, 2 May 2022 15:34:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1385869AbiEBPhp (ORCPT ); Mon, 2 May 2022 11:37:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44066 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1358130AbiEBPhm (ORCPT ); Mon, 2 May 2022 11:37:42 -0400 Received: from mail.pcs.gmbh (mail.pcs.gmbh [89.27.162.52]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B285BDF81; Mon, 2 May 2022 08:34:09 -0700 (PDT) Received: from mail.csna.de (mail.csna.de [89.27.162.50]) by mail.pcs.gmbh with ESMTPA ; Mon, 2 May 2022 17:34:03 +0200 Received: from EXCHANGE2019.pcs.ditec.de (mail.pcs.com [89.27.162.5]) by mail.csna.de with ESMTPA ; Mon, 2 May 2022 17:34:03 +0200 Received: from EXCHANGE2019.pcs.ditec.de (192.168.8.214) by EXCHANGE2019.pcs.ditec.de (192.168.8.214) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22; Mon, 2 May 2022 17:34:03 +0200 Received: from lxtpfaff.pcs.ditec.de (192.168.9.96) by EXCHANGE2019.pcs.ditec.de (192.168.8.214) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.22 via Frontend Transport; Mon, 2 May 2022 17:34:03 +0200 Date: Mon, 2 May 2022 17:34:03 +0200 From: Thomas Pfaff To: Thomas Gleixner CC: , , Marc Zyngier , Lukas Wunner Subject: Re: [PATCH v3] irq/core: synchronize irq_thread startup In-Reply-To: <87mtg0m2jb.ffs@tglx> Message-ID: References: <552fe7b4-9224-b183-bb87-a8f36d335690@pcs.com> <87mtg0m2jb.ffs@tglx> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" X-KSE-ServerInfo: EXCHANGE2019.pcs.ditec.de, 9 X-KSE-AntiSpam-Interceptor-Info: white sender email list X-KSE-AttachmentFiltering-Interceptor-Info: protection disabled X-KSE-Antivirus-Interceptor-Info: scan successful X-KSE-Antivirus-Info: Clean, bases: 02.05.2022 14:16:00 X-KSE-BulkMessagesFiltering-Scan-Result: protection disabled Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 2 May 2022, Thomas Gleixner wrote: > But free_irq() stopped issuing synchronize_irq() with commit > 519cc8652b3a ("genirq: Synchronize only with single thread on > free_irq()"). And that turns out to be the root cause of the problem. > I should have caught that back then, but in hindsight .... > > While the proposed patch works, I think the real solution is to ensure > that both the hardware interrupt _and_ the interrupt threads which are > associated to the removed action are in quiescent state. This should > catch the case you observed. I can confirm that your patch works. And it also explains why I never had this issue on a 4.4 kernel with realtime patch ... > --- > Subject: genirq: Quiesce interrupt threads in free_irq() > From: Thomas Gleixner > Date: Mon, 02 May 2022 15:40:25 +0200 > > Fill void... > > Fixes: 519cc8652b3a ("genirq: Synchronize only with single thread on free_irq()") > Signed-off-by: Thomas Gleixner > --- > kernel/irq/manage.c | 25 +++++++++++++++++++++---- > 1 file changed, 21 insertions(+), 4 deletions(-) > > --- a/kernel/irq/manage.c > +++ b/kernel/irq/manage.c > @@ -1914,6 +1914,22 @@ static struct irqaction *__free_irq(stru > */ > __synchronize_hardirq(desc, true); > > + /* > + * Wait for associated interrupt threads to complete. This cannot > + * use synchronize_irq() due to interrupt sharing in the PCIe > + * layer. See 519cc8652b3a ("genirq: Synchronize only with single > + * thread on free_irq()") for further explanation. > + */ > + if (action->thread) { > + unsigned int thread_mask = action->thread_mask; > + > + if (action->secondary) > + thread_mask |= action->secondary->thread_mask; > + > + wait_event(desc->wait_for_threads, > + !(atomic_read(&desc->threads_active) & thread_mask)); > + } > + > #ifdef CONFIG_DEBUG_SHIRQ > /* > * It's a shared IRQ -- the driver ought to be prepared for an IRQ > @@ -1931,10 +1947,11 @@ static struct irqaction *__free_irq(stru > #endif > > /* > - * The action has already been removed above, but the thread writes > - * its oneshot mask bit when it completes. Though request_mutex is > - * held across this which prevents __setup_irq() from handing out > - * the same bit to a newly requested action. > + * The action has already been removed above and both the hardware > + * interrupt and the associated threads have been synchronized, > + * which means they are in quiescent state. request_mutex is still > + * held which prevents __setup_irq() from handing out action's > + * thread_mask to a newly requested action. > */ > if (action->thread) { > kthread_stop(action->thread); > > > Tested-by: Thomas Pfaff Thank you, Thomas