Re: [PATCH 2/2] iommu/amd: Don't allocate kdump device table from DMA32

["Kalra, Ashish" <ashish.kalra@amd.com>]

On 10/2/2026 8:31 AM, Matt Fleming wrote:
> [You don't often get email from matt@readmodwrite.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
> 
> From: Matt Fleming <mfleming@cloudflare.com>
> 
> Booting a kdump kernel with crashkernel=X,high crashkernel=0,low on an
> AMD system fails the per-segment device table allocation because there
> is no memory below 4G:
> 
>   swapper/0: page allocation failure: order:9, mode:0x40104(GFP_DMA32|__GFP_ZERO|__GFP_COMP)
>   iommu_alloc_pages_node_sz+0x7f/0x150
>   alloc_pci_segment+0x4bd/0x650
>   iommu_go_to_state+0x213d/0x34e0
>   amd_iommu_prepare+0x39/0xa0
>   irq_remapping_prepare+0x83/0xd0
>   enable_IR_x2apic+0x61/0x370
> 
> On our E810 systems, the kdump kernel booted but ice MSI-X interrupts
> never fired and the TX queues timed out.
> 
> Commit b336781b82cc ("iommu/amd: Allocate memory below 4G for dev table
> if translation pre-enabled") allocated device tables below 4G. At the
> time, the kdump kernel programmed a copy of the old table into the base
> register with the IOMMU still enabled. Keeping the table below 4G means
> it doesn't matter that the hardware splits that write into two 32-bit
> halves.

Probably, expand/explain this a bit:

The 64-bit base register is not updated atomically - the
write lands as two 32-bit halves - so mid-update the live IOMMU can
briefly latch a half-written, invalid table pointer. Allocating the
table below 4G keeps the upper 32 bits zero, so the register stays valid
throughout the update and the running IOMMU never sees a bad pointer.

> 
> Since commit 38e5f33ee359 ("iommu/amd: Reuse device table for kdump")
> the kdump kernel reuses the previous kernel's table without writing the
> base register, so its newly allocated table is discarded when reuse
> succeeds. If reuse fails, the previous patch programs the new table only
> after disabling the IOMMU. Either way, the kdump kernel's table doesn't
> need to be below 4G.

Since dropping GFP_DMA32 is only safe on top of the reuse conversion, i.e.,
on top of 38e5f33ee359, so it should not be backported without it.

Probably, consider added a Cc:stable tag without Fixes: tag, so it doesn't
read as if 38e5f33 was buggy

Cc: stable@vger.kernel.org # depends on 38e5f33ee359
Signed-off-by: Matt Fleming mfleming@cloudflare.com

Reviewed-by: Ashish Kalra ashish.kalra@amd.com

> 
> The previous kernel still allocates its table from DMA32, so the kdump
> kernel's check that the old table is below 4G still works. Drop
> GFP_DMA32 for kdump kernels only.
> 
> Signed-off-by: Matt Fleming <mfleming@cloudflare.com>
> ---
>  drivers/iommu/amd/init.c | 15 +++++++++++++--
>  1 file changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
> index 0572e1a674f0..16c26d75f46f 100644
> --- a/drivers/iommu/amd/init.c
> +++ b/drivers/iommu/amd/init.c
> @@ -648,8 +648,19 @@ static int __init find_last_devid_acpi(struct acpi_table_header *table, u16 pci_
>  /* Allocate per PCI segment device table */
>  static inline int __init alloc_dev_table(struct amd_iommu_pci_seg *pci_seg)
>  {
> -       pci_seg->dev_table = iommu_alloc_pages_sz(GFP_KERNEL | GFP_DMA32,
> -                                                 pci_seg->dev_table_size);
> +       gfp_t gfp = GFP_KERNEL;
> +
> +       /*
> +        * Keep normal kernel device tables below 4G because a later kdump
> +        * kernel only trusts and reuses old tables below that limit. A kdump
> +        * kernel's new table is either discarded when reuse succeeds or
> +        * programmed after the IOMMU is disabled, so the table does not
> +        * need DMA32 memory (which the crashkernel may not have).
> +        */
> +       if (!is_kdump_kernel())
> +               gfp |= GFP_DMA32;
> +
> +       pci_seg->dev_table = iommu_alloc_pages_sz(gfp, pci_seg->dev_table_size);
>         if (!pci_seg->dev_table)
>                 return -ENOMEM;
> 
> --
> 2.43.0
>