From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751673AbdFIV7b (ORCPT ); Fri, 9 Jun 2017 17:59:31 -0400 Received: from mx1.redhat.com ([209.132.183.28]:48244 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751554AbdFIV7a (ORCPT ); Fri, 9 Jun 2017 17:59:30 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 8B4A6C059721 Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=alex.williamson@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 8B4A6C059721 Subject: [PATCH 0/7] vfio: Fix release ordering races and use driver_override From: Alex Williamson To: kvm@vger.kernel.org Cc: eric.auger@redhat.com, alex.williamson@redhat.com, linux-kernel@vger.kernel.org Date: Fri, 09 Jun 2017 15:59:25 -0600 Message-ID: <20170609215816.31986.89321.stgit@gimli.home> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Fri, 09 Jun 2017 21:59:29 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org VM hotplug testing reveals a number of races in the vfio device, group, container shutdown path, some attributed to libvirt's ask/take unplug behavior and some long standing with groups potentially composed of multiple devices, where each device can be independently bound to drivers. Libvirt's ask/take behavior is a result of the asynchronous nature of PCI hotplug, libvirt registers a hot-unplug request (ask), which is acknowledged almost immediately and then proceeds to try to unbind the device from the vfio bus driver (take). This sets us off on racing paths where we allow the device to be released from the group much like would happen in groups with multiple devices, while the group and container are torn down separately. These races are addressed in the first 3 patches of this series. The long standing issue with removing devices from in-use groups is that we feel that the system is compromised if we allow user and host devices within the same non-isolated group. This triggers a BUG_ON when we detect this condition after the rogue driver binding. Since that code was put in place we've added driver_override support for all of the physical buses supported by vfio, giving us a way to block binding to such compromising drivers. We finally enable that in the latter 4 patches of this series, minding that we need to allow re-binding to non-compromising drivers, and also noting that a small synchronization stall is effective in eliminating the need for this blocking in the more common singleton device group case. Reviews, comments, and acks appreciated. Thanks, Alex --- Alex Williamson (7): vfio: Fix group release deadlock kvm-vfio: Decouple only when we match a group vfio: New external user group/file match iommu: Add driver-not-bound notification vfio: Create interface for vfio bus drivers to register vfio: Register pci, platform, amba, and mdev bus drivers vfio: Use driver_override to avert binding to compromising drivers drivers/iommu/iommu.c | 2 drivers/vfio/mdev/vfio_mdev.c | 13 ++ drivers/vfio/pci/vfio_pci.c | 7 + drivers/vfio/platform/vfio_amba.c | 24 +++ drivers/vfio/platform/vfio_platform.c | 24 +++ drivers/vfio/vfio.c | 252 ++++++++++++++++++++++++++++++++- include/linux/iommu.h | 1 include/linux/vfio.h | 5 + virt/kvm/vfio.c | 33 +++- 9 files changed, 338 insertions(+), 23 deletions(-)