[PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Pavel Popov]

VMware and Hyper-V expose passthrough devices to guest VMs without an
explicit host bridge device. Consequently, host_bridge_whitelist()
cannot automatically identify the underlying host bridge structure,
causing Peer-to-Peer DMA (P2PDMA) requests to be rejected unless the
device IDs are explicitly listed in pci_p2pdma_whitelist[]. To enable
P2PDMA support in virtualized environments on Intel hosts, this series
introduces hypervisor_supports_p2pdma().

Although alternative P2PDMA mechanisms are in development [1], their
adoption timeline in hypervisors remains uncertain. This solution
addresses the immediate need to support both new and existing
hypervisor deployments.

The patches are organized by hypervisor for clarity. While the primary
focus is VMware, a corresponding Hyper-V patch addresses the same
pattern and is included for maintainer consideration.

[1] https://lore.kernel.org/r/20260812-hmat-p2p-v1-0-75ac41380585@nvidia.com

Pavel Popov (2):
  PCI/P2PDMA: Allow P2PDMA in VMware guests on Intel hosts
  PCI/P2PDMA: Allow P2PDMA in Hyper-V guests on Intel hosts

 drivers/pci/p2pdma.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)


base-commit: 6c377d19d4a5116d9bec5203aa3c6c11523e7898
-- 
2.53.0

[PATCH 1/2] PCI/P2PDMA: Allow P2PDMA in VMware guests on Intel hosts

[Pavel Popov]

Peer-to-Peer DMA (P2PDMA) between two devices requires either a shared
upstream bridge or a host bridge matching pci_p2pdma_whitelist[]. In
VMware guests, passthrough devices appear directly on virtual root
buses without host bridge devices. Consequently, P2PDMA checks
evaluate endpoint Device IDs against the host bridge whitelist,
causing P2PDMA requests to fail regardless of host support.

This affects discrete devices such as GPUs, NICs and NVMe drives.

The P2PDMA host bridge check does not work in a VMware guest. Bypass it on
Intel hosts under VMware, starting with Skylake-SP, the first Xeon
Scalable generation. Xeon Scalable introduced UPI and is whitelisted for
P2PDMA without restrictions. AMD Zen and newer are already covered by
cpu_supports_p2pdma(), so the bypass is implemented for Intel hosts only.

Tested with the Intel Level Zero P2P benchmark (ze_peer) on Arc Pro B60
cards in a Linux guest on VMware ESXi 9.0 (Emerald Rapids host).

Reviewed-by: Jim Chow <jim.chow@broadcom.com>
Signed-off-by: Pavel Popov <pavel.e.popov@intel.com>
---
 drivers/pci/p2pdma.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c
index 9334eb314663..d96f44d0738a 100644
--- a/drivers/pci/p2pdma.c
+++ b/drivers/pci/p2pdma.c
@@ -21,6 +21,12 @@
 #include <linux/seq_buf.h>
 #include <linux/xarray.h>
 
+#ifdef CONFIG_X86
+#include <asm/cpu_device_id.h>
+#include <asm/hypervisor.h>
+#include <asm/intel-family.h>
+#endif
+
 struct pci_p2pdma {
 	struct gen_pool *pool;
 	bool p2pmem_published;
@@ -532,6 +538,19 @@ static bool cpu_supports_p2pdma(void)
 	return false;
 }
 
+static bool hypervisor_supports_p2pdma(void)
+{
+#ifdef CONFIG_X86
+	/* Hypervisor hides host topology, allow p2pdma on Skylake and newer */
+	if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL &&
+	    boot_cpu_data.x86_vfm >= INTEL_SKYLAKE_X &&
+	    hypervisor_is_type(X86_HYPER_VMWARE))
+		return true;
+#endif
+
+	return false;
+}
+
 static const struct pci_p2pdma_whitelist_entry {
 	unsigned short vendor;
 	int device;
@@ -781,6 +800,7 @@ calc_map_type_and_dist(struct pci_dev *provider, struct pci_dev *client,
 
 map_through_host_bridge:
 	if (!cpu_supports_p2pdma() &&
+	    !hypervisor_supports_p2pdma() &&
 	    !host_bridge_whitelist(provider, client, acs_redirects)) {
 		if (verbose)
 			pci_warn(client, "cannot be used for peer-to-peer DMA as the client and provider (%s) do not share an upstream bridge or whitelisted host bridge\n",
-- 
2.53.0

Re: [PATCH 1/2] PCI/P2PDMA: Allow P2PDMA in VMware guests on Intel hosts

[Logan Gunthorpe]

On 2026-10-09 9:40 a.m., Pavel Popov wrote:
> Peer-to-Peer DMA (P2PDMA) between two devices requires either a shared
> upstream bridge or a host bridge matching pci_p2pdma_whitelist[]. In
> VMware guests, passthrough devices appear directly on virtual root
> buses without host bridge devices. Consequently, P2PDMA checks
> evaluate endpoint Device IDs against the host bridge whitelist,
> causing P2PDMA requests to fail regardless of host support.
> 
> This affects discrete devices such as GPUs, NICs and NVMe drives.
> 
> The P2PDMA host bridge check does not work in a VMware guest. Bypass it on
> Intel hosts under VMware, starting with Skylake-SP, the first Xeon
> Scalable generation. Xeon Scalable introduced UPI and is whitelisted for
> P2PDMA without restrictions. AMD Zen and newer are already covered by
> cpu_supports_p2pdma(), so the bypass is implemented for Intel hosts only.
> 
> Tested with the Intel Level Zero P2P benchmark (ze_peer) on Arc Pro B60
> cards in a Linux guest on VMware ESXi 9.0 (Emerald Rapids host).
> 
> Reviewed-by: Jim Chow <jim.chow@broadcom.com>
> Signed-off-by: Pavel Popov <pavel.e.popov@intel.com>
> ---
>  drivers/pci/p2pdma.c | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
> 
> diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c
> index 9334eb314663..d96f44d0738a 100644
> --- a/drivers/pci/p2pdma.c
> +++ b/drivers/pci/p2pdma.c
> @@ -21,6 +21,12 @@
>  #include <linux/seq_buf.h>
>  #include <linux/xarray.h>
>  
> +#ifdef CONFIG_X86
> +#include <asm/cpu_device_id.h>
> +#include <asm/hypervisor.h>
> +#include <asm/intel-family.h>
> +#endif
> +
>  struct pci_p2pdma {
>  	struct gen_pool *pool;
>  	bool p2pmem_published;
> @@ -532,6 +538,19 @@ static bool cpu_supports_p2pdma(void)
>  	return false;
>  }
>  
> +static bool hypervisor_supports_p2pdma(void)
> +{
> +#ifdef CONFIG_X86
> +	/* Hypervisor hides host topology, allow p2pdma on Skylake and newer */
> +	if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL &&
> +	    boot_cpu_data.x86_vfm >= INTEL_SKYLAKE_X &&
> +	    hypervisor_is_type(X86_HYPER_VMWARE))
> +		return true;
> +#endif
> +
> +	return false;
> +}

The changes seem fine, but one minor nit from me. Can we avoid putting
the #ifdefs inside the function?

Something like:

#ifdef CONFIG_X86
static bool hypervisor_supports_p2pdma(void)
{
...
}
#else /* !CONFIG_X86 */
static bool hypervisor_supports_p2pdma(void)
{
	return false;
}
#endif /* CONFIG_X86 */


Other than that, the core concept seems fine to me:

Reviewed-by: Logan Gunthorpe <logang@deltatee.com>

Thanks!

Logan

RE: [PATCH 1/2] PCI/P2PDMA: Allow P2PDMA in VMware guests on Intel hosts

[Popov, Pavel E]

> The changes seem fine, but one minor nit from me. Can we avoid putting
> the #ifdefs inside the function?

Sure, will do in v2. Thanks for the review.

[PATCH 2/2] PCI/P2PDMA: Allow P2PDMA in Hyper-V guests on Intel hosts

[Pavel Popov]

Peer-to-Peer DMA (P2PDMA) between two devices requires either a shared
upstream bridge or a host bridge matching pci_p2pdma_whitelist[]. In
Hyper-V guests, each passthrough device appears in a PCI domain of its
own without a host bridge device. Consequently, P2PDMA checks evaluate
endpoint Device IDs against the host bridge whitelist, causing P2PDMA
requests to fail regardless of host support.

This affects discrete devices such as GPUs, NICs and NVMe drives.

The P2PDMA host bridge check does not work in a Hyper-V guest. Bypass it
on Intel hosts under Hyper-V, starting with Skylake-SP, the first Xeon
Scalable generation. Xeon Scalable introduced UPI and is whitelisted for
P2PDMA without restrictions. AMD Zen and newer are already covered by
cpu_supports_p2pdma(), so the bypass is implemented for Intel hosts only.

Tested with the Intel Level Zero P2P benchmark (ze_peer) on Arc Pro B60
cards in a Linux guest on Windows Server 2025 (Emerald Rapids host).

Reviewed-by: Jim Chow <jim.chow@broadcom.com>
Signed-off-by: Pavel Popov <pavel.e.popov@intel.com>
---
 drivers/pci/p2pdma.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c
index d96f44d0738a..76fadfbc1814 100644
--- a/drivers/pci/p2pdma.c
+++ b/drivers/pci/p2pdma.c
@@ -544,7 +544,8 @@ static bool hypervisor_supports_p2pdma(void)
 	/* Hypervisor hides host topology, allow p2pdma on Skylake and newer */
 	if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL &&
 	    boot_cpu_data.x86_vfm >= INTEL_SKYLAKE_X &&
-	    hypervisor_is_type(X86_HYPER_VMWARE))
+	    (hypervisor_is_type(X86_HYPER_VMWARE) ||
+	     hypervisor_is_type(X86_HYPER_MS_HYPERV)))
 		return true;
 #endif
 
-- 
2.53.0

Re: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Bjorn Helgaas]

On Fri, Oct 09, 2026 at 08:40:32AM -0700, Pavel Popov wrote:
> VMware and Hyper-V expose passthrough devices to guest VMs without an
> explicit host bridge device. Consequently, host_bridge_whitelist()
> cannot automatically identify the underlying host bridge structure,
> causing Peer-to-Peer DMA (P2PDMA) requests to be rejected unless the
> device IDs are explicitly listed in pci_p2pdma_whitelist[]. To enable
> P2PDMA support in virtualized environments on Intel hosts, this series
> introduces hypervisor_supports_p2pdma().
> 
> Although alternative P2PDMA mechanisms are in development [1], their
> adoption timeline in hypervisors remains uncertain. This solution
> addresses the immediate need to support both new and existing
> hypervisor deployments.
> 
> The patches are organized by hypervisor for clarity. While the primary
> focus is VMware, a corresponding Hyper-V patch addresses the same
> pattern and is included for maintainer consideration.
> 
> [1] https://lore.kernel.org/r/20260812-hmat-p2p-v1-0-75ac41380585@nvidia.com
> 
> Pavel Popov (2):
>   PCI/P2PDMA: Allow P2PDMA in VMware guests on Intel hosts
>   PCI/P2PDMA: Allow P2PDMA in Hyper-V guests on Intel hosts
> 
>  drivers/pci/p2pdma.c | 21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)

Applied with the #ifdef tweak Logan suggested to pci/p2pdma for v7.4,
thanks!

FWIW, it's better if responses like Jim's Reviewed-by actually appear
on the mailing list.  I don't see anything in lore, so I assume it
must have been private review.

Re: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Leon Romanovsky]

On Fri, Oct 09, 2026 at 11:58:33AM -0500, Bjorn Helgaas wrote:
> On Fri, Oct 09, 2026 at 08:40:32AM -0700, Pavel Popov wrote:
> > VMware and Hyper-V expose passthrough devices to guest VMs without an
> > explicit host bridge device. Consequently, host_bridge_whitelist()
> > cannot automatically identify the underlying host bridge structure,
> > causing Peer-to-Peer DMA (P2PDMA) requests to be rejected unless the
> > device IDs are explicitly listed in pci_p2pdma_whitelist[]. To enable
> > P2PDMA support in virtualized environments on Intel hosts, this series
> > introduces hypervisor_supports_p2pdma().
> > 
> > Although alternative P2PDMA mechanisms are in development [1], their
> > adoption timeline in hypervisors remains uncertain. This solution
> > addresses the immediate need to support both new and existing
> > hypervisor deployments.
> > 
> > The patches are organized by hypervisor for clarity. While the primary
> > focus is VMware, a corresponding Hyper-V patch addresses the same
> > pattern and is included for maintainer consideration.
> > 
> > [1] https://lore.kernel.org/r/20260812-hmat-p2p-v1-0-75ac41380585@nvidia.com
> > 
> > Pavel Popov (2):
> >   PCI/P2PDMA: Allow P2PDMA in VMware guests on Intel hosts
> >   PCI/P2PDMA: Allow P2PDMA in Hyper-V guests on Intel hosts
> > 
> >  drivers/pci/p2pdma.c | 21 +++++++++++++++++++++
> >  1 file changed, 21 insertions(+)
> 
> Applied with the #ifdef tweak Logan suggested to pci/p2pdma for v7.4,
> thanks!

Bjorn,

I disagree with this decision for several reasons:

1. Note where `hypervisor_supports_p2pdma()` is called: before
   `host_bridge_whitelist()`. This means the code ignores the hypervisor
   topology. The claim that the VM has a virtual bridge, causing
   `pci_p2pdma_whitelist()` to fail, describes exactly how P2P is expected
   to work today.

2. We are not developing an alternative solution. This is the right
   solution, and there is broad agreement that it is the only reliable
   way to enable P2P in VMs, for ALL emulation software stacks.

3. This problem has existed and been known for at least the past eight
   years, since Logan upstreamed P2P support. The claim "we need it now
   and ASAP" is not valid at all.

4. The proposed hack does not solve the P2P-in-VM problem; it only makes it work
   in some random cases. An HMAT-based solution will still be needed, even on systems
   that use this hack.

Let's get the HMAT solution merged sooner rather than later. To make that happen,
we need a coordinated effort across the industry, not hacks to work around the
problem.

> 
> FWIW, it's better if responses like Jim's Reviewed-by actually appear
> on the mailing list.  I don't see anything in lore, so I assume it
> must have been private review.

Unfortunately, this patch series was not given enough time for review.

Thanks

> 

RE: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Popov, Pavel E]

> 1. Note where `hypervisor_supports_p2pdma()` is called: before
>    `host_bridge_whitelist()`. This means the code ignores the hypervisor
>    topology. The claim that the VM has a virtual bridge, causing
>    `pci_p2pdma_whitelist()` to fail, describes exactly how P2P is expected
>    to work today.

It does not ignore the topology. The check is reached only after the
common-upstream-bridge walk and the ACS checks have failed, at the point
where the host bridge allowlist is consulted. It skips that lookup on
the matched hypervisors, the same way cpu_supports_p2pdma() already
skips it for AMD.

The lookup cannot work there because VMware and Hyper-V expose no host
bridge device at all on the root buses carrying passthrough devices, so
pci_host_bridge_dev() returns the passthrough endpoint itself. That is
described in the commit messages.


> 2. We are not developing an alternative solution. This is the right
>    solution, and there is broad agreement that it is the only reliable
>    way to enable P2P in VMs, for ALL emulation software stacks.

Agreed, and the cover letter says so. Our understanding is that HMAT
becomes the primary source of P2PDMA information and the allowlist
remains as a fallback; the hypervisor check is part of that fallback,
not a competing path. When the HMAT lookup lands it takes precedence in
the same decision path, and this check only matters where no HMAT is
exposed. The adoption timeline of HMAT in hypervisors is uncertain, and
guests on existing hypervisor releases will not get it at all, so the
fallback is needed for both new and existing deployments.


> 3. This problem has existed and been known for at least the past eight
>    years, since Logan upstreamed P2P support. The claim "we need it now
>    and ASAP" is not valid at all.

The problem is old, the demand is not. Passing several accelerators
through to one guest and expecting them to talk to each other is a
recent requirement, and today P2PDMA does not work at all for
passthrough devices in VMware or Hyper-V guests on Intel hosts. The
series does not claim urgency beyond that: a working path exists now,
HMAT does not yet, and the two do not conflict.


> 4. The proposed hack does not solve the P2P-in-VM problem; it only makes it work
>    in some random cases. An HMAT-based solution will still be needed, even on systems
>    that use this hack.

It makes it work on Xeon Scalable hosts under VMware and Hyper-V, which
is where passthrough accelerators are deployed today. Nothing more is
claimed. I agree HMAT is still needed on top for latency, bandwidth and
ordering attributes.

Re: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Leon Romanovsky]

On Fri, Oct 09, 2026 at 06:21:49PM +0000, Popov, Pavel E wrote:
> > 1. Note where `hypervisor_supports_p2pdma()` is called: before
> >    `host_bridge_whitelist()`. This means the code ignores the hypervisor
> >    topology. The claim that the VM has a virtual bridge, causing
> >    `pci_p2pdma_whitelist()` to fail, describes exactly how P2P is expected
> >    to work today.
> 
> It does not ignore the topology. The check is reached only after the
> common-upstream-bridge walk and the ACS checks have failed, at the point
> where the host bridge allowlist is consulted.

Since the request comes from a VM, we always take the "skip" path,
regardless of the topology.

<...>

> 
> > 2. We are not developing an alternative solution. This is the right
> >    solution, and there is broad agreement that it is the only reliable
> >    way to enable P2P in VMs, for ALL emulation software stacks.
> 
> Agreed, and the cover letter says so. Our understanding is that HMAT
> becomes the primary source of P2PDMA information and the allowlist
> remains as a fallback; the hypervisor check is part of that fallback,
> not a competing path. When the HMAT lookup lands it takes precedence in
> the same decision path, and this check only matters where no HMAT is
> exposed. The adoption timeline of HMAT in hypervisors is uncertain, and
> guests on existing hypervisor releases will not get it at all, so the
> fallback is needed for both new and existing deployments.

I won't worry for hypervisors, they have enough brilliant developers to
take the upstream code to their codebase.

> 
> 
> > 3. This problem has existed and been known for at least the past eight
> >    years, since Logan upstreamed P2P support. The claim "we need it now
> >    and ASAP" is not valid at all.
> 
> The problem is old, the demand is not. Passing several accelerators
> through to one guest and expecting them to talk to each other is a
> recent requirement

This is not correct, at least for mlx5 devices. The need was already recognized
in 2020 in commit 90da7dc8206a ("RDMA/mlx5: Support dma-buf based userspace
memory region").

> and today P2PDMA does not work at all for
> passthrough devices in VMware or Hyper-V guests on Intel hosts. The
> series does not claim urgency beyond that: a working path exists now,
> HMAT does not yet, and the two do not conflict.
> 
> 
> > 4. The proposed hack does not solve the P2P-in-VM problem; it only makes it work
> >    in some random cases. An HMAT-based solution will still be needed, even on systems
> >    that use this hack.
> 
> It makes it work on Xeon Scalable hosts under VMware and Hyper-V, which
> is where passthrough accelerators are deployed today.

This is a very Intel-centric view. The vast majority of systems probably run on ARM
and use QEMU.

> Nothing more is claimed. I agree HMAT is still needed on top for latency, bandwidth and
> ordering attributes.
> 

The primary goal of HMAT is to eliminate the need for whitelists and to
simulate a P2P route for the VM that accounts for the hypervisor topology.
Latency and bandwidth are secondary considerations.

Thanks

RE: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Popov, Pavel E]

Leon,

Understood on all points, and I agree HMAT is the proper fix. We discussed
it with Lukas Wunner as well. Exposing HMAT is on the hypervisor side; Jim
Chow from VMware can speak to their plans. Even once HMAT lands in Linux,
it only helps guests after the hypervisors add support for it.

The key point for us is that today there is no path at all for
passthrough devices on VMware or Hyper-V, and that is what this patch
addresses. We reviewed it with VMware and aligned on this approach for the
guest in the meantime.

Is there anything you would recommend to make the series acceptable as an
interim? Bjorn, Logan, your view would help here as well.

Re: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Leon Romanovsky]

On Fri, Oct 09, 2026 at 07:13:31PM +0000, Popov, Pavel E wrote:
> Leon,
> 
> Understood on all points, and I agree HMAT is the proper fix. We discussed
> it with Lukas Wunner as well. Exposing HMAT is on the hypervisor side; Jim
> Chow from VMware can speak to their plans. Even once HMAT lands in Linux,
> it only helps guests after the hypervisors add support for it.
> 
> The key point for us is that today there is no path at all for
> passthrough devices on VMware or Hyper-V, and that is what this patch
> addresses. We reviewed it with VMware and aligned on this approach for the
> guest in the meantime.

This does not change the fact that the patch causes unpredictable behavior for
P2P devices in the guest. Sometimes it works; other times, devices that
previously worked may be classified as P2P-capable and will stop.

> 
> Is there anything you would recommend to make the series acceptable as an
> interim?

Help Lukas and Mahesh succeed in their work.

Thanks

RE: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Popov, Pavel E]

To be clear about what is at stake: this series is what enables P2PDMA
in VMware and Hyper-V guests on Intel hosts. The assumption that the
hypervisor can simply be fixed to emit HMAT is Linux-centric. It holds
for KVM/QEMU, which is open source and where anyone can add HMAT support.
ESXi and Hyper-V are proprietary, so only VMware and Microsoft can add
it, and until they do, this series is the only path for guests on those
hypervisors.

> This does not change the fact that the patch causes unpredictable
> behavior for P2P devices in the guest. Sometimes it works; other times,
> devices that previously worked may be classified as P2P-capable and
> will stop.

Nothing "previously worked" here. Before this patch, P2PDMA was refused
for every passthrough device pair in VMware and Hyper-V guests on Intel
hosts, except Intel QAT/DSA/IAA, whose own Device IDs happen to be in
the allowlist. Those keep working unchanged, so there is no working
configuration that could regress. And the check is deterministic: it
depends on the CPU model and the hypervisor type, nothing else. The same
guest on the same host gets the same answer every time.

> Help Lukas and Mahesh succeed in their work.

I'd like to, but the blocker for guests is on the hypervisor side. A
guest only benefits from HMAT once ESXi or Hyper-V emits it, and that is
VMware's and Microsoft's code, outside the control of Intel or the kernel
community. The kernel cannot make them ship it; what it can do is provide
this fallback. When your HMAT-based detection lands, it should be
consulted first and this check only when no HMAT is present, so on any
hypervisor that emits HMAT your code is what runs and this one is never
reached.

> -----Original Message-----
> From: Leon Romanovsky <leon@kernel.org>
> Sent: Saturday, October 10, 2026 10:47 PM
> To: Popov, Pavel E <pavel.e.popov@intel.com>
> Cc: Bjorn Helgaas <helgaas@kernel.org>; Jim Chow <jim.chow@broadcom.com>;
> Bjorn Helgaas <bhelgaas@google.com>; Logan Gunthorpe
> <logang@deltatee.com>; Radu Rugina <radu.rugina@broadcom.com>; Alexey
> Makhalov <alexey.makhalov@broadcom.com>; Wei Liu <wei.liu@kernel.org>;
> Michael Kelley <mhklinux@outlook.com>; Lukas Wunner <lukas@wunner.de>;
> Nathan Ciobanu <nathan.d.ciobanu@linux.intel.com>; bcm-kernel-feedback-
> list@broadcom.com; virtualization@lists.linux.dev; linux-hyperv@vger.kernel.org;
> linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V
> guests on Intel hosts
> 
> On Fri, Oct 09, 2026 at 07:13:31PM +0000, Popov, Pavel E wrote:
> > Leon,
> >
> > Understood on all points, and I agree HMAT is the proper fix. We
> > discussed it with Lukas Wunner as well. Exposing HMAT is on the
> > hypervisor side; Jim Chow from VMware can speak to their plans. Even
> > once HMAT lands in Linux, it only helps guests after the hypervisors add support
> for it.
> >
> > The key point for us is that today there is no path at all for
> > passthrough devices on VMware or Hyper-V, and that is what this patch
> > addresses. We reviewed it with VMware and aligned on this approach for
> > the guest in the meantime.
> 
> This does not change the fact that the patch causes unpredictable behavior for
> P2P devices in the guest. Sometimes it works; other times, devices that previously
> worked may be classified as P2P-capable and will stop.
> 
> >
> > Is there anything you would recommend to make the series acceptable as
> > an interim?
> 
> Help Lukas and Mahesh succeed in their work.
> 
> Thanks

Re: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Leon Romanovsky]

On Mon, Oct 12, 2026 at 02:26:43PM +0000, Popov, Pavel E wrote:
> To be clear about what is at stake: this series is what enables P2PDMA
> in VMware and Hyper-V guests on Intel hosts. The assumption that the
> hypervisor can simply be fixed to emit HMAT is Linux-centric. It holds
> for KVM/QEMU, which is open source and where anyone can add HMAT support.
> ESXi and Hyper-V are proprietary, so only VMware and Microsoft can add
> it, and until they do, this series is the only path for guests on those
> hypervisors.

I'm not following your reasoning here. In the cover letter, you say that
VMware reviewed your work. If VMware wants to provide P2P support to its
users, it can add the necessary support itself. As for Microsoft, there's
no need to worry; it can handle the ACPI changes just fine.

> 
> > This does not change the fact that the patch causes unpredictable
> > behavior for P2P devices in the guest. Sometimes it works; other times,
> > devices that previously worked may be classified as P2P-capable and
> > will stop.
> 
> Nothing "previously worked" here. Before this patch, P2PDMA was refused
> for every passthrough device pair in VMware and Hyper-V guests on Intel
> hosts, except Intel QAT/DSA/IAA, whose own Device IDs happen to be in
> the allowlist. Those keep working unchanged, so there is no working
> configuration that could regress. And the check is deterministic: it
> depends on the CPU model and the hypervisor type, nothing else. The same
> guest on the same host gets the same answer every time.

Of course not. P2P operation depends on the hypervisor topology, which the
proposed code does not account for.

> 
> > Help Lukas and Mahesh succeed in their work.
> 
> I'd like to, but the blocker for guests is on the hypervisor side. A
> guest only benefits from HMAT once ESXi or Hyper-V emits it, and that is
> VMware's and Microsoft's code, outside the control of Intel or the kernel
> community. The kernel cannot make them ship it; what it can do is provide
> this fallback. When your HMAT-based detection lands, it should be
> consulted first and this check only when no HMAT is present, so on any
> hypervisor that emits HMAT your code is what runs and this one is never
> reached.

The kernel community has no obligation to accommodate users of proprietary
products who are unwilling to invest in a proper solution.

Please keep these hacks out of tree.

Thanks

RE: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Popov, Pavel E]

> I'm not following your reasoning here. In the cover letter, you say that
> VMware reviewed your work. If VMware wants to provide P2P support to its
> users, it can add the necessary support itself. As for Microsoft, there's
> no need to worry; it can handle the ACPI changes just fine.

The cover letter sentence on this, "their adoption timeline in
hypervisors remains uncertain", was written with VMware's input; Jim,
Radu and Alexey are on CC and can speak for ESXi. But whatever VMware
and Microsoft do in future releases, guests on already shipped ESXi and
Hyper-V versions are not going to get HMAT, and those stay in production
for years. On them, passthrough GPUs, NICs and NVMe drives have no P2PDMA
at all without a kernel-side fallback.

> Of course not. P2P operation depends on the hypervisor topology, which
> the proposed code does not account for.

The guest topology is what the code already evaluates, unchanged: the
common upstream bridge walk and the ACS checks run first, and the new
check is reached only after they fail, right where cpu_supports_p2pdma()
already bypasses the host bridge lookup for AMD Zen. What the guest sees
is a flat topology:

VMware, four GPUs:

  -+-[0000:00]-+-00.0  Intel 440BX/ZX/DX Host bridge
   |           +-07.x  PIIX4 ISA/IDE/ACPI, VMCI, SVGA
   +-[0000:02]-+-00.0  VMware PVSCSI
   |           +-01.0  VMware VMXNET3
   |           ...
   \-[0000:03]-+-00.0  Intel GPU
               +-01.0  Intel GPU
               +-02.0  Intel GPU
               \-03.0  Intel GPU

Hyper-V, four GPUs:

  -[2fc0:00]---00.0  Intel GPU
  -[542b:00]---00.0  Intel GPU
  -[5c71:00]---00.0  Intel GPU
  -[9624:00]---00.0  Intel GPU

The passthrough devices sit directly on root buses with no host bridge
device and no root ports; on Hyper-V each is alone in its own PCI
domain. There is no upstream bridge to share, no ACS to evaluate, and
the host bridge lookup lands on the endpoint itself. The topology
carries no routing information, so the check falls back to the CPU model
and hypervisor type. HMAT would provide what the topology cannot; when
your HMAT-based detection lands it should be consulted first, so this
check is never reached where HMAT exists.

> -----Original Message-----
> From: Leon Romanovsky <leon@kernel.org>
> Sent: Monday, October 12, 2026 9:24 AM
> To: Popov, Pavel E <pavel.e.popov@intel.com>
> Cc: Bjorn Helgaas <helgaas@kernel.org>; Jim Chow <jim.chow@broadcom.com>;
> Bjorn Helgaas <bhelgaas@google.com>; Logan Gunthorpe
> <logang@deltatee.com>; Radu Rugina <radu.rugina@broadcom.com>; Alexey
> Makhalov <alexey.makhalov@broadcom.com>; Wei Liu <wei.liu@kernel.org>;
> Michael Kelley <mhklinux@outlook.com>; Lukas Wunner <lukas@wunner.de>;
> Nathan Ciobanu <nathan.d.ciobanu@linux.intel.com>; bcm-kernel-feedback-
> list@broadcom.com; virtualization@lists.linux.dev; linux-hyperv@vger.kernel.org;
> linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V
> guests on Intel hosts
> 
> On Mon, Oct 12, 2026 at 02:26:43PM +0000, Popov, Pavel E wrote:
> > To be clear about what is at stake: this series is what enables P2PDMA
> > in VMware and Hyper-V guests on Intel hosts. The assumption that the
> > hypervisor can simply be fixed to emit HMAT is Linux-centric. It holds
> > for KVM/QEMU, which is open source and where anyone can add HMAT
> support.
> > ESXi and Hyper-V are proprietary, so only VMware and Microsoft can add
> > it, and until they do, this series is the only path for guests on
> > those hypervisors.
> 
> I'm not following your reasoning here. In the cover letter, you say that VMware
> reviewed your work. If VMware wants to provide P2P support to its users, it can
> add the necessary support itself. As for Microsoft, there's no need to worry; it can
> handle the ACPI changes just fine.
> 
> >
> > > This does not change the fact that the patch causes unpredictable
> > > behavior for P2P devices in the guest. Sometimes it works; other
> > > times, devices that previously worked may be classified as
> > > P2P-capable and will stop.
> >
> > Nothing "previously worked" here. Before this patch, P2PDMA was
> > refused for every passthrough device pair in VMware and Hyper-V guests
> > on Intel hosts, except Intel QAT/DSA/IAA, whose own Device IDs happen
> > to be in the allowlist. Those keep working unchanged, so there is no
> > working configuration that could regress. And the check is
> > deterministic: it depends on the CPU model and the hypervisor type,
> > nothing else. The same guest on the same host gets the same answer every
> time.
> 
> Of course not. P2P operation depends on the hypervisor topology, which the
> proposed code does not account for.
> 
> >
> > > Help Lukas and Mahesh succeed in their work.
> >
> > I'd like to, but the blocker for guests is on the hypervisor side. A
> > guest only benefits from HMAT once ESXi or Hyper-V emits it, and that
> > is VMware's and Microsoft's code, outside the control of Intel or the
> > kernel community. The kernel cannot make them ship it; what it can do
> > is provide this fallback. When your HMAT-based detection lands, it
> > should be consulted first and this check only when no HMAT is present,
> > so on any hypervisor that emits HMAT your code is what runs and this
> > one is never reached.
> 
> The kernel community has no obligation to accommodate users of proprietary
> products who are unwilling to invest in a proper solution.
> 
> Please keep these hacks out of tree.
> 
> Thanks

Re: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Leon Romanovsky]

On Mon, Oct 12, 2026 at 04:45:08PM +0000, Popov, Pavel E wrote:
> > I'm not following your reasoning here. In the cover letter, you say that
> > VMware reviewed your work. If VMware wants to provide P2P support to its
> > users, it can add the necessary support itself. As for Microsoft, there's
> > no need to worry; it can handle the ACPI changes just fine.
> 
> The cover letter sentence on this, "their adoption timeline in
> hypervisors remains uncertain", was written with VMware's input; Jim,
> Radu and Alexey are on CC and can speak for ESXi. But whatever VMware
> and Microsoft do in future releases, guests on already shipped ESXi and
> Hyper-V versions are not going to get HMAT, and those stay in production
> for years. On them, passthrough GPUs, NICs and NVMe drives have no P2PDMA
> at all without a kernel-side fallback.
> 
> > Of course not. P2P operation depends on the hypervisor topology, which
> > the proposed code does not account for.
> 
> The guest topology is what the code already evaluates, unchanged: the
> common upstream bridge walk and the ACS checks run first, and the new
> check is reached only after they fail, right where cpu_supports_p2pdma()
> already bypasses the host bridge lookup for AMD Zen. What the guest sees
> is a flat topology:
> 
> VMware, four GPUs:
> 
>   -+-[0000:00]-+-00.0  Intel 440BX/ZX/DX Host bridge
>    |           +-07.x  PIIX4 ISA/IDE/ACPI, VMCI, SVGA
>    +-[0000:02]-+-00.0  VMware PVSCSI
>    |           +-01.0  VMware VMXNET3
>    |           ...
>    \-[0000:03]-+-00.0  Intel GPU
>                +-01.0  Intel GPU
>                +-02.0  Intel GPU
>                \-03.0  Intel GPU
> 
> Hyper-V, four GPUs:
> 
>   -[2fc0:00]---00.0  Intel GPU
>   -[542b:00]---00.0  Intel GPU
>   -[5c71:00]---00.0  Intel GPU
>   -[9624:00]---00.0  Intel GPU
> 
> The passthrough devices sit directly on root buses with no host bridge
> device and no root ports; on Hyper-V each is alone in its own PCI
> domain. There is no upstream bridge to share, no ACS to evaluate, and
> the host bridge lookup lands on the endpoint itself. The topology
> carries no routing information, so the check falls back to the CPU model
> and hypervisor type. HMAT would provide what the topology cannot; when
> your HMAT-based detection lands it should be consulted first, so this
> check is never reached where HMAT exists.

Please read all of my responses. I was explicit about why the check inside
the guest VM produces random results. However, your responses continue
to focus on the guest, which is not the problem.

In the meantime, add:
Nacked-by: Leon Romanovsky <leon@kernel.org>

to any patches that do not fix the hypervisor first.

Thanks

RE: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Popov, Pavel E]

> Please read all of my responses. I was explicit about why the check
> inside the guest VM produces random results. However, your responses
> continue to focus on the guest, which is not the problem.

Fair, my earlier replies were about the guest. Taking the host-side
points in turn:

1. Host topology decides P2P, the guest cannot see it, and the patch
   says "yes" for every pair regardless.

   The topology is accounted for where the knowledge is: on the host.
   Which devices land in one VM is decided by the admin, from devices
   VMware has certified for passthrough in the Compatibility Guide,
   and on vSphere 8+ by Device Groups, which hands devices sharing a
   switch or interconnect to a VM as a topology-aware group. Hyper-V
   has no equivalent grouping; DDA is plain passthrough, each device
   assigned to the VM explicitly by the admin.

   The blanket "yes" is not new either. On bare metal the flags = 0
   whitelist entries say "yes" for any pair of Xeon Scalable root
   buses, across sockets, with no knowledge of BIOS P2P settings. And
   cpu_supports_p2pdma() already bypasses the host bridge check for
   AMD Zen on CPUID alone, in guests included. This series gives Intel
   hosts what the tree already does for AMD; if that is unacceptable
   in principle, it is unacceptable for AMD guests today.

2. Regression: consumers that work today via a system-memory fallback
   will take the P2P path and break on hosts that cannot route it.

   A wrong "yes" fails safely: without host-side P2P the request is
   stopped at the host IOMMU, nothing is corrupted, the same exposure
   an AMD guest has today. An unconditional "no" is worse: it blocks
   P2P on every host that does route it, and the guest's answer
   changes nothing about whether the host routes it.

   Xeon Scalable routes P2P between any two root ports, which the
   whitelist already encodes. The only hypervisor-side requirement is
   mapping the peer's MMIO into the VM's IOMMU domain, and that is
   what passthrough P2P support means: on ESXi it was an explicit
   per-VM setting for years and is the default since 9.0; Hyper-V does
   it for devices assigned to the same VM, as tested on Windows Server
   2025. The decision is made on the host; the guest only has to stop
   vetoing it.

3. The hypervisor must describe it, HMAT is the way, NAK to anything
   that does not fix the hypervisor first.

   Agreed HMAT is the right source and should take precedence: consult
   it first, this check only where no HMAT exists. The disagreement is
   on "first". Guests on already shipped ESXi and Hyper-V releases
   will never get HMAT, and those stay in production for years; the
   cover letter's note on the adoption timeline was written with
   VMware's input, and Jim can speak to it. Until a hypervisor emits
   HMAT, this fallback is the only path that works.

Thanks,
Pavel

Re: [PATCH 0/2] PCI/P2PDMA: Allow P2PDMA in VMware and Hyper-V guests on Intel hosts

[Bjorn Helgaas]

On Fri, Oct 09, 2026 at 09:08:47PM +0300, Leon Romanovsky wrote:
> On Fri, Oct 09, 2026 at 11:58:33AM -0500, Bjorn Helgaas wrote:
> > On Fri, Oct 09, 2026 at 08:40:32AM -0700, Pavel Popov wrote:
> > > VMware and Hyper-V expose passthrough devices to guest VMs without an
> > > explicit host bridge device. Consequently, host_bridge_whitelist()
> > > cannot automatically identify the underlying host bridge structure,
> > > causing Peer-to-Peer DMA (P2PDMA) requests to be rejected unless the
> > > device IDs are explicitly listed in pci_p2pdma_whitelist[]. To enable
> > > P2PDMA support in virtualized environments on Intel hosts, this series
> > > introduces hypervisor_supports_p2pdma().
> > > 
> > > Although alternative P2PDMA mechanisms are in development [1], their
> > > adoption timeline in hypervisors remains uncertain. This solution
> > > addresses the immediate need to support both new and existing
> > > hypervisor deployments.
> > > 
> > > The patches are organized by hypervisor for clarity. While the primary
> > > focus is VMware, a corresponding Hyper-V patch addresses the same
> > > pattern and is included for maintainer consideration.
> > > 
> > > [1] https://lore.kernel.org/r/20260812-hmat-p2p-v1-0-75ac41380585@nvidia.com
> > > 
> > > Pavel Popov (2):
> > >   PCI/P2PDMA: Allow P2PDMA in VMware guests on Intel hosts
> > >   PCI/P2PDMA: Allow P2PDMA in Hyper-V guests on Intel hosts
> > > 
> > >  drivers/pci/p2pdma.c | 21 +++++++++++++++++++++
> > >  1 file changed, 21 insertions(+)
> > 
> > Applied with the #ifdef tweak Logan suggested to pci/p2pdma for v7.4,
> > thanks!
> 
> Bjorn,
> 
> I disagree with this decision for several reasons:
> 
> 1. Note where `hypervisor_supports_p2pdma()` is called: before
>    `host_bridge_whitelist()`. This means the code ignores the hypervisor
>    topology. The claim that the VM has a virtual bridge, causing
>    `pci_p2pdma_whitelist()` to fail, describes exactly how P2P is expected
>    to work today.
> 
> 2. We are not developing an alternative solution. This is the right
>    solution, and there is broad agreement that it is the only reliable
>    way to enable P2P in VMs, for ALL emulation software stacks.
> 
> 3. This problem has existed and been known for at least the past eight
>    years, since Logan upstreamed P2P support. The claim "we need it now
>    and ASAP" is not valid at all.
> 
> 4. The proposed hack does not solve the P2P-in-VM problem; it only makes it work
>    in some random cases. An HMAT-based solution will still be needed, even on systems
>    that use this hack.
> 
> Let's get the HMAT solution merged sooner rather than later. To make that happen,
> we need a coordinated effort across the industry, not hacks to work around the
> problem.

OK, I'll drop this while we discuss it.