From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BFC4C433E1 for ; Wed, 26 Aug 2020 21:47:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 20E6D2087D for ; Wed, 26 Aug 2020 21:47:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726825AbgHZVra convert rfc822-to-8bit (ORCPT ); Wed, 26 Aug 2020 17:47:30 -0400 Received: from eu-smtp-delivery-151.mimecast.com ([185.58.86.151]:27651 "EHLO eu-smtp-delivery-151.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726753AbgHZVr2 (ORCPT ); Wed, 26 Aug 2020 17:47:28 -0400 Received: from AcuMS.aculab.com (156.67.243.126 [156.67.243.126]) (Using TLS) by relay.mimecast.com with ESMTP id uk-mta-257-g12td2zPN3C-gOdetsRJcQ-1; Wed, 26 Aug 2020 22:47:25 +0100 X-MC-Unique: g12td2zPN3C-gOdetsRJcQ-1 Received: from AcuMS.Aculab.com (fd9f:af1c:a25b:0:43c:695e:880f:8750) by AcuMS.aculab.com (fd9f:af1c:a25b:0:43c:695e:880f:8750) with Microsoft SMTP Server (TLS) id 15.0.1347.2; Wed, 26 Aug 2020 22:47:24 +0100 Received: from AcuMS.Aculab.com ([fe80::43c:695e:880f:8750]) by AcuMS.aculab.com ([fe80::43c:695e:880f:8750%12]) with mapi id 15.00.1347.000; Wed, 26 Aug 2020 22:47:24 +0100 From: David Laight To: 'Thomas Gleixner' , 'Alexander Graf' , 'X86 ML' CC: 'Andy Lutomirski' , 'LKML' , 'Andrew Cooper' , "'Paul E. McKenney'" , 'Alexandre Chartre' , 'Frederic Weisbecker' , 'Paolo Bonzini' , 'Sean Christopherson' , 'Masami Hiramatsu' , 'Petr Mladek' , 'Steven Rostedt' , 'Joel Fernandes' , 'Boris Ostrovsky' , 'Juergen Gross' , "'Mathieu Desnoyers'" , 'Josh Poimboeuf' , 'Will Deacon' , 'Tom Lendacky' , 'Wei Liu' , 'Michael Kelley' , 'Jason Chen CJ' , "'Zhao Yakui'" , "'Peter Zijlstra (Intel)'" , 'Avi Kivity' , "'Herrenschmidt, Benjamin'" , "'robketr@amazon.de'" , "'amos@scylladb.com'" , 'Brian Gerst' , "'stable@vger.kernel.org'" , 'Alex bykov' Subject: RE: x86/irq: Unbreak interrupt affinity setting Thread-Topic: x86/irq: Unbreak interrupt affinity setting Thread-Index: AQHWe+aKb+AhwM2rPkq6/MK3Hcp5nKlK5iEAgAAGXqA= Date: Wed, 26 Aug 2020 21:47:23 +0000 Message-ID: <42ae8716e425495c964ae7372bd7ff52@AcuMS.aculab.com> References: <20200826115357.3049-1-graf@amazon.com> <87k0xlv5w5.fsf@nanos.tec.linutronix.de> <87blixuuny.fsf@nanos.tec.linutronix.de> <873649utm4.fsf@nanos.tec.linutronix.de> <87wo1ltaxz.fsf@nanos.tec.linutronix.de> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.202.205.107] MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=C51A453 smtp.mailfrom=david.laight@aculab.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: aculab.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: David Laight > Sent: 26 August 2020 22:37 > > From: Thomas Gleixner > > Sent: 26 August 2020 21:22 > ... > > Moving interrupts on x86 happens in several steps. A new vector on a > > different CPU is allocated and the relevant interrupt source is > > reprogrammed to that. But that's racy and there might be an interrupt > > already in flight to the old vector. So the old vector is preserved until > > the first interrupt arrives on the new vector and the new target CPU. Once > > that happens the old vector is cleaned up, but this cleanup still depends > > on the vector number being stored in pt_regs::orig_ax, which is now -1. > > I suspect that it is much more 'racy' than that for PCI-X interrupts. > On the hardware side there is an interrupt disable bit, and address > and a value. > To raise an interrupt the hardware must write the value to the address. > > If the cpu needs to move an interrupt both the address and value > need changing, but the cpu wont write the address and value using > the same TLP, so the hardware could potentially write a value to > the wrong address. > Worse than that, the hardware could easily only look at the address > and value in the clocks after checking the interrupt is enabled. > So masking the interrupt immediately prior to changing the vector > info may not be enough. > > It is likely that a read-back of the mask before updating the vector > is enough. But not enough to assume you won't receive an interrupt after reading back that interrupts are masked. (I've implemented the hardware side for an fpga ...) David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)