From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4869F427A1B for ; Wed, 10 Jun 2026 15:19:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781104748; cv=none; b=stZo+FoTxhTQ7XO9H+v8pIN6RUvq89zOUk02ziLKcCuVCEs7LPVAYKmiKwdUxWXk0HuyLhkJk6UhapYU729Z7uxYCIbPNWCv4jzTF+8/UoSjGpx+eYFe6xbPApOTD/T25cFflg5oFD6sNHkR443OiYasOhc0a6xVRM2zm3Saz0M= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781104748; c=relaxed/simple; bh=/cTHa/At87E8vPajzqVShlG51CB8kJlWKuB7Y9ueVDM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=h/TfL92eChDUH+9I0O7PqPHlz4l4lnKZ9NgkX+Ev9/r1cC3uCFjbNP3xXE76WHgdys/V8osTuPp3DYTwHFM/YgYIL7r5Yq71EYLRKT0XtKcav/uwUpTGosV8MVVjL7XWzK/E2skZipeOgnvAX+IslW98PayIF8Vur4Mk43AUyiM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--praan.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=oKyYUQbl; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--praan.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="oKyYUQbl" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-842278a630dso8560550b3a.2 for ; Wed, 10 Jun 2026 08:19:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1781104746; x=1781709546; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=FVXdMltEQgkHjALmtUIidgf93/BpfptYJnT+4mfBNsk=; b=oKyYUQblygdXod00T4j84V5eJ46oNJuNUwGCx0QsC3/V8WIjp0pUqrId9UZExp0XaG A21mu9oUBBysgG3gbBKDgasv5do3VO+crY1hmrB/VkXFFJnsVCUxbigiGG/20FwHnCSz 0impCZIK1AsGrOIg8Ot0eZcnr+O53ivnVpFXLs7SlYgs6RP84q2x+Wy+A6jtuO/HWxPs JtPcmN/mqMnmZpjSxCd/Xg+OuQhMtoVH8uSBvKPtVhEJ60El1kQvd4qDAFJ7hE3roaIM kwdDqAyiaOLWLtaqmq4IYGsMZu/dqH8R9w4fJR31JMst/wDxEIAoBmuXWVZZ+qwjgAsW dm6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781104746; x=1781709546; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FVXdMltEQgkHjALmtUIidgf93/BpfptYJnT+4mfBNsk=; b=aPJFk0FCFAYJS3UB7ZCWZwIyh+j8d8pxP90rCHOEY3T6gdBnkbCFnfmGDkR8RevrkD qKYT28ZunXLIVIJqlwY24x/D+yAmrEZsL04jj6zSYopdiywOlYaWqzto5J+eH1U2Lp8f bcgayb//d2tn6NnozQplA2ilP83VWf1kCvnvC3QVdtXbjMV5St+A1ppsK/u9rgRCzEWn WfCRogycB8j1p/TTnlp/aKbLkDFJkW83yXan6TCFd2HyqGw10xGFGtqvgdlkbGo/MhOb FgLsGgtMMCqT9wE5Ua4rUbZG/kUYTGVYz/Qh5szgNwK5jURUI4EWcpBlWmIeGEWw6Zs3 7WZw== X-Forwarded-Encrypted: i=1; AFNElJ994PR3MKj0iW2K5x+0jr2Xp/HN+wPle8Qb1YTEnTrXxeB86mPUyzHw+SdcV5Njf+Hpu4Tx/RqQelaPL0s=@vger.kernel.org X-Gm-Message-State: AOJu0YycsFko2gxKuS/9joTYK30a76m5L38c8x2t9qq7S8K4Nipx+QkM j2gHZI2axmqhFUSwRDrQFtko2AMR6cjwtN6Prf1lEokP85JDwSUrMjKYULeOF1lXfaVMYgjDtiJ BZQ== X-Received: from pfmb16.prod.google.com ([2002:a05:6a00:1150:b0:842:7045:f89b]) (user=praan job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:1302:b0:82c:9897:70e6 with SMTP id d2e1a72fcca58-842b0f9c4abmr26940329b3a.34.1781104746343; Wed, 10 Jun 2026 08:19:06 -0700 (PDT) Date: Wed, 10 Jun 2026 15:18:53 +0000 In-Reply-To: <20260610151853.3608948-1-praan@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260610151853.3608948-1-praan@google.com> X-Mailer: git-send-email 2.54.0.1099.g489fc7bff1-goog Message-ID: <20260610151853.3608948-6-praan@google.com> Subject: [RFC PATCH 5/5] PCI/P2PDMA: Introduce a helper to release P2P resources From: Pranjal Shrivastava To: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Bjorn Helgaas , Logan Gunthorpe , Alex Williamson , Jason Gunthorpe , Kevin Tian , Pranjal Shrivastava , Ankit Agrawal , Matt Evans , Vivek Kasireddy , Leon Romanovsky , Shivaji Kant , Samiullah Khawaja Content-Type: text/plain; charset="UTF-8" Introduce pci_p2pdma_remove_resource() to allow manual teardown of a device's P2P DMA pool. The new API enables exclusive owners of a device, such as vfio-pci, to cleanly release P2P resources during session closure or hardware reset. Update vfio-pci to call this function during vfio_pci_zap_bars(), ensuring that any BARs registered with ZONE_DEVICE P2P are released. Signed-off-by: Pranjal Shrivastava --- drivers/pci/p2pdma.c | 38 ++++++++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_core.c | 5 +++++ include/linux/pci-p2pdma.h | 1 + 3 files changed, 44 insertions(+) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index adb17a4f6939..2a48ffefa01c 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -26,6 +26,7 @@ struct pci_p2pdma { bool p2pmem_published; struct xarray map_types; struct p2pdma_provider mem[PCI_STD_NUM_BARS]; + struct pci_p2pdma_pagemap *pagemaps[PCI_STD_NUM_BARS]; }; struct pci_p2pdma_pagemap { @@ -453,6 +454,8 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, if (error) goto pages_free; + p2pdma->pagemaps[bar] = p2p_pgmap; + pci_info(pdev, "added peer-to-peer DMA memory %#llx-%#llx\n", pgmap->range.start, pgmap->range.end); @@ -466,6 +469,41 @@ int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, } EXPORT_SYMBOL_GPL(pci_p2pdma_add_resource); +/** + * pci_p2pdma_remove_resource - remove all p2p memory for a device + * @pdev: the device to remove the memory from + * + * Tear down the entire p2p DMA pool for the device. Zap any existinga + * userspace mappings of the p2pmem/allocate file + */ +void pci_p2pdma_remove_resource(struct pci_dev *pdev) +{ + struct pci_p2pdma *p2pdma; + int i; + + p2pdma = rcu_dereference_protected(pdev->p2pdma, 1); + if (!p2pdma || !p2pdma->pool) + return; + + for (i = 0; i < PCI_STD_NUM_BARS; i++) { + if (p2pdma->pagemaps[i]) { + devm_release_action(&pdev->dev, pci_p2pdma_unmap_mappings, + p2pdma->pagemaps[i]); + devm_memunmap_pages(&pdev->dev, &p2pdma->pagemaps[i]->pgmap); + devm_kfree(&pdev->dev, p2pdma->pagemaps[i]); + p2pdma->pagemaps[i] = NULL; + } + } + + gen_pool_destroy(p2pdma->pool); + p2pdma->pool = NULL; + + sysfs_remove_group(&pdev->dev.kobj, &p2pmem_group); + + pci_info(pdev, "removed all peer-to-peer DMA memory\n"); +} +EXPORT_SYMBOL_GPL(pci_p2pdma_remove_resource); + /* * Note this function returns the parent PCI device with a * reference taken. It is the caller's responsibility to drop diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 7913b8916df9..7b58cb344408 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1704,6 +1704,11 @@ static void vfio_pci_zap_bars(struct vfio_pci_core_device *vdev) loff_t len = end - start; unmap_mapping_range(core_vdev->inode->i_mapping, start, len, true); + + if (vdev->p2p_registered_bars) { + pci_p2pdma_remove_resource(vdev->pdev); + vdev->p2p_registered_bars = 0; + } } void vfio_pci_zap_and_down_write_memory_lock(struct vfio_pci_core_device *vdev) diff --git a/include/linux/pci-p2pdma.h b/include/linux/pci-p2pdma.h index 873de20a2247..14ee2e59a43e 100644 --- a/include/linux/pci-p2pdma.h +++ b/include/linux/pci-p2pdma.h @@ -72,6 +72,7 @@ int pcim_p2pdma_init(struct pci_dev *pdev); struct p2pdma_provider *pcim_p2pdma_provider(struct pci_dev *pdev, int bar); int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, u64 offset); +void pci_p2pdma_remove_resource(struct pci_dev *pdev); int pci_p2pdma_distance_many(struct pci_dev *provider, struct device **clients, int num_clients, bool verbose); struct pci_dev *pci_p2pmem_find_many(struct device **clients, int num_clients); -- 2.54.0.1099.g489fc7bff1-goog