[PATCH v4] x86/pci-dma: add a SWIOTLB_ANY flag to lift the low mem limitation

[PATCH v4] x86/pci-dma: add a SWIOTLB_ANY flag to lift the low mem limitation

[Jun Miao]

When high-speed NICs or multi-GPU setups are passed through into confidential
VMs, the SWIOTLB bounce buffer becomes the critical path between private and
shared memory. Restricting it to low memory limits throughput and fails to
scale for larger workloads.

AMD SEV-SNP and Intel TDX guests run in a TEE where the hypervisor is untrusted.
DMA-capable devices require bounce buffers to mediate between encrypted private
memory and unencrypted shared memory. Confining these buffers to low memory (<4GB)
unnecessarily caps their size and degrades performance.

Power SVM already supports this; x86 does not. See commit 8ba2ed1be9
("swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction").

[ aakarsh: completely trim down/rewrite changelog ]

Tested-by: Aakarsh Jain <aakarsh.jain@oss.qualcomm.com>
Suggested-by: Borislav Petkov <bp@alien8.de>
Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: Aakarsh Jain <aakarsh.jain@oss.qualcomm.com>
Signed-off-by: Jun Miao <jun.miao@intel.com>
---

v1 -> v2:
- Updated commit message and description.
- Add Reviewed and Tested.
V1 Latest Feedback : https://lists.openwall.net/linux-kernel/2026/02/11/483

v2 -> v3:
- We can alloc 4GB with the dynamic swiotlb, rather than 1GB.
  1G is not correct. So change the commit log.

v3 -> v4:
- Not only TDX-specific but all encrypted guests include SEV.
- SEV-SNP guest passed the test with the help of Aakarsh.
  Tested-by: Aakarsh Jain <aakarsh.jain@oss.qualcomm.com>

- Add "Acked-by: Marek Szyprowski"
- Explain the usage case in the commit log following Boris`s suggestion.

---
 arch/x86/kernel/pci-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index 6267363e0189..73b9320c4a7d 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -61,7 +61,7 @@ static void __init pci_swiotlb_detect(void)
 	 */
 	if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) {
 		x86_swiotlb_enable = true;
-		x86_swiotlb_flags |= SWIOTLB_FORCE;
+		x86_swiotlb_flags |= SWIOTLB_ANY | SWIOTLB_FORCE;
 	}
 }
 #else
-- 
2.47.1

RE: [PATCH v4] x86/pci-dma: add a SWIOTLB_ANY flag to lift the low mem limitation

[Michael Kelley]

From: Jun Miao <jun.miao@intel.com> Sent: Wednesday, June 24, 2026 6:26 PM
> 
> When high-speed NICs or multi-GPU setups are passed through into confidential
> VMs, the SWIOTLB bounce buffer becomes the critical path between private and
> shared memory. Restricting it to low memory limits throughput and fails to
> scale for larger workloads.
> 
> AMD SEV-SNP and Intel TDX guests run in a TEE where the hypervisor is untrusted.
> DMA-capable devices require bounce buffers to mediate between encrypted private
> memory and unencrypted shared memory. Confining these buffers to low memory
> (<4GB) unnecessarily caps their size and degrades performance.
> 
> Power SVM already supports this; x86 does not. See commit 8ba2ed1be9
> ("swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction").
> 
> [ aakarsh: completely trim down/rewrite changelog ]
> 
> Tested-by: Aakarsh Jain <aakarsh.jain@oss.qualcomm.com>
> Suggested-by: Borislav Petkov <bp@alien8.de>
> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
> Reviewed-by: Aakarsh Jain <aakarsh.jain@oss.qualcomm.com>
> Signed-off-by: Jun Miao <jun.miao@intel.com>

I tested this change in Intel TDX and AMD SEV-SNP VMs in the
Azure cloud, which are running on Hyper-V and with a paravisor.
The SEV-SNP VM operates in vTOM mode.

I was able to allocate a 4 GiB swiotlb, whereas prior to this patch
specifying 4 GiB resulted in a memory allocation failure and a
downgrade to 2 GiB. All bounce buffered DMA operations from
the Hyper-V synthetic SCSI and NIC controllers worked correctly,
as did DMA operations from a PCI pass-thru NVMe controller in
the TDX VM.

Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Tested-by: Michael Kelley <mhklinux@outlook.com>

> ---
> 
> v1 -> v2:
> - Updated commit message and description.
> - Add Reviewed and Tested.
> V1 Latest Feedback : https://lists.openwall.net/linux-kernel/2026/02/11/483
> 
> v2 -> v3:
> - We can alloc 4GB with the dynamic swiotlb, rather than 1GB.
>   1G is not correct. So change the commit log.
> 
> v3 -> v4:
> - Not only TDX-specific but all encrypted guests include SEV.
> - SEV-SNP guest passed the test with the help of Aakarsh.
>   Tested-by: Aakarsh Jain <aakarsh.jain@oss.qualcomm.com>
> 
> - Add "Acked-by: Marek Szyprowski"
> - Explain the usage case in the commit log following Boris`s suggestion.
> 
> ---
>  arch/x86/kernel/pci-dma.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
> index 6267363e0189..73b9320c4a7d 100644
> --- a/arch/x86/kernel/pci-dma.c
> +++ b/arch/x86/kernel/pci-dma.c
> @@ -61,7 +61,7 @@ static void __init pci_swiotlb_detect(void)
>  	 */
>  	if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) {
>  		x86_swiotlb_enable = true;
> -		x86_swiotlb_flags |= SWIOTLB_FORCE;
> +		x86_swiotlb_flags |= SWIOTLB_ANY | SWIOTLB_FORCE;
>  	}
>  }
>  #else
> --
> 2.47.1
> 

Re: [PATCH v4] x86/pci-dma: add a SWIOTLB_ANY flag to lift the low mem limitation

[Borislav Petkov]

On Thu, Jun 25, 2026 at 09:26:16AM +0800, Jun Miao wrote:
> When high-speed NICs or multi-GPU setups are passed through into confidential
> VMs, the SWIOTLB bounce buffer becomes the critical path between private and
> shared memory. Restricting it to low memory limits throughput and fails to
> scale for larger workloads.
> 
> AMD SEV-SNP and Intel TDX guests run in a TEE where the hypervisor is untrusted.
> DMA-capable devices require bounce buffers to mediate between encrypted private
> memory and unencrypted shared memory. Confining these buffers to low memory (<4GB)
> unnecessarily caps their size and degrades performance.
> 
> Power SVM already supports this; x86 does not. See commit 8ba2ed1be9
> ("swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction").
> 
> [ aakarsh: completely trim down/rewrite changelog ]
> 
> Tested-by: Aakarsh Jain <aakarsh.jain@oss.qualcomm.com>
> Suggested-by: Borislav Petkov <bp@alien8.de>
> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
> Reviewed-by: Aakarsh Jain <aakarsh.jain@oss.qualcomm.com>
> Signed-off-by: Jun Miao <jun.miao@intel.com>
> ---

Sashiko has a question:

https://lore.kernel.org/r/20260625012616.2992535-1-jun.miao@intel.com


-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

RE: [PATCH v4] x86/pci-dma: add a SWIOTLB_ANY flag to lift the low mem limitation

[Miao, Jun]

>On Thu, Jun 25, 2026 at 09:26:16AM +0800, Jun Miao wrote:
>> When high-speed NICs or multi-GPU setups are passed through into
>> confidential VMs, the SWIOTLB bounce buffer becomes the critical path
>> between private and shared memory. Restricting it to low memory limits
>> throughput and fails to scale for larger workloads.
>>
>> AMD SEV-SNP and Intel TDX guests run in a TEE where the hypervisor is
>untrusted.
>> DMA-capable devices require bounce buffers to mediate between
>> encrypted private memory and unencrypted shared memory. Confining
>> these buffers to low memory (<4GB) unnecessarily caps their size and
>degrades performance.
>>
>> Power SVM already supports this; x86 does not. See commit 8ba2ed1be9
>> ("swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction").
>>
>> [ aakarsh: completely trim down/rewrite changelog ]
>>
>> Tested-by: Aakarsh Jain <aakarsh.jain@oss.qualcomm.com>
>> Suggested-by: Borislav Petkov <bp@alien8.de>
>> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
>> Reviewed-by: Aakarsh Jain <aakarsh.jain@oss.qualcomm.com>
>> Signed-off-by: Jun Miao <jun.miao@intel.com>
>> ---
>
>Sashiko has a question:
>
>https://lore.kernel.org/r/20260625012616.2992535-1-jun.miao@intel.com
>
>
Sashiko  said “Does x86 lack DMA-mask filtering during pool search?”:

Conclusion:  yes, it does (at least in the current common SWIOTLB search path).
Therefore, after enabling SWIOTLB_ANY, dma-direct with 32-bit DMA-mask devices is indeed more
likely to hit the case where a high-address bounce buffer is allocated first and then rejected by dma_capable().

I plan to modify SWIOTLB directly: in the direct DMA path, I’ll filter candidate pool/slot addresses by dma_capable() 
during the search phase, so we avoid the case where a high address is allocated first and then fails without retrying 
a low-memory buffer. At the same time, I’ll keep the IOMMU/Xen paths unchanged.

Warm regards
Jun Miao
>--
>Regards/Gruss,
>    Boris.
>
>https://people.kernel.org/tglx/notes-about-netiquette