From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932737AbdBVO2X (ORCPT ); Wed, 22 Feb 2017 09:28:23 -0500 Received: from mail.kernel.org ([198.145.29.136]:48472 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932444AbdBVO2P (ORCPT ); Wed, 22 Feb 2017 09:28:15 -0500 Date: Wed, 22 Feb 2017 08:28:10 -0600 From: Bjorn Helgaas To: Boris Ostrovsky Cc: Juergen Gross , Dan Streetman , Konrad Rzeszutek Wilk , Stefano Stabellini , Dan Streetman , Bjorn Helgaas , xen-devel@lists.xenproject.org, linux-kernel , linux-pci@vger.kernel.org Subject: Re: [PATCH] xen: do not re-use pirq number cached in pci device msi msg data Message-ID: <20170222142810.GA28359@bhelgaas-glaptop.roam.corp.google.com> References: <04f2a09f-59be-a720-bc98-4afb53171790@oracle.com> <20170113200751.20125-1-ddstreet@ieee.org> <20170113223035.GE23066@char.us.oracle.com> <0c2b4d6a-93e7-90a2-ffaf-fa423fb022d5@suse.com> <72f8b025-e858-5e54-d900-50fbec46fea6@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <72f8b025-e858-5e54-d900-50fbec46fea6@oracle.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Feb 21, 2017 at 10:58:39AM -0500, Boris Ostrovsky wrote: > On 02/21/2017 10:45 AM, Juergen Gross wrote: > > On 21/02/17 16:31, Dan Streetman wrote: > >> On Fri, Jan 13, 2017 at 5:30 PM, Konrad Rzeszutek Wilk > >> wrote: > >>> On Fri, Jan 13, 2017 at 03:07:51PM -0500, Dan Streetman wrote: > >>>> Revert the main part of commit: > >>>> af42b8d12f8a ("xen: fix MSI setup and teardown for PV on HVM guests") > >>>> > >>>> That commit introduced reading the pci device's msi message data to see > >>>> if a pirq was previously configured for the device's msi/msix, and re-use > >>>> that pirq. At the time, that was the correct behavior. However, a > >>>> later change to Qemu caused it to call into the Xen hypervisor to unmap > >>>> all pirqs for a pci device, when the pci device disables its MSI/MSIX > >>>> vectors; specifically the Qemu commit: > >>>> c976437c7dba9c7444fb41df45468968aaa326ad > >>>> ("qemu-xen: free all the pirqs for msi/msix when driver unload") > >>>> > >>>> Once Qemu added this pirq unmapping, it was no longer correct for the > >>>> kernel to re-use the pirq number cached in the pci device msi message > >>>> data. All Qemu releases since 2.1.0 contain the patch that unmaps the > >>>> pirqs when the pci device disables its MSI/MSIX vectors. > >>>> > >>>> This bug is causing failures to initialize multiple NVMe controllers > >>>> under Xen, because the NVMe driver sets up a single MSIX vector for > >>>> each controller (concurrently), and then after using that to talk to > >>>> the controller for some configuration data, it disables the single MSIX > >>>> vector and re-configures all the MSIX vectors it needs. So the MSIX > >>>> setup code tries to re-use the cached pirq from the first vector > >>>> for each controller, but the hypervisor has already given away that > >>>> pirq to another controller, and its initialization fails. > >>>> > >>>> This is discussed in more detail at: > >>>> https://lists.xen.org/archives/html/xen-devel/2017-01/msg00447.html > >>>> > >>>> Fixes: af42b8d12f8a ("xen: fix MSI setup and teardown for PV on HVM guests") > >>>> Signed-off-by: Dan Streetman > >>> Acked-by: Konrad Rzeszutek Wilk > >> This doesn't seem to be applied yet, is it still waiting on another > >> ack? Or maybe I'm looking at the wrong git tree... > > Am I wrong or shouldn't this go through the PCI tree? Konrad? > > Konrad is away this week but since pull request for Xen tree just went > out we should probably wait until rc1 anyway (unless something big comes > up before that). I assume this should go via the Xen or x86 tree, since that's how most arch/x86/pci/xen.c patches have been handled, including af42b8d12f8a. If you think otherwise, let me know.