Re: [PATCH v3 8/8] PCI/quirks: Avoid certain address on Genoa systems

["Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>]

[-- Attachment #1: Type: text/plain, Size: 5581 bytes --]

On Tue, 13 Oct 2026, Mario Limonciello wrote:
> On 10/13/26 12:00, Mario Limonciello wrote:
> > On 10/13/26 08:09, Ilpo Järvinen wrote:
> > > While testing the resource placement changes, tests hit a case where
> > > igb fails to probe when BAR 0 is placed at 0x9c000000:
> > > 
> > >     90000000-9cffffff : PCI Bus 0000:a0
> > >    -  90000000-902fffff : PCI Bus 0000:a1
> > >    -    90000000-900fffff : 0000:a1:00.0
> > >    -      90000000-900fffff : igb
> > >    -    90100000-901fffff : 0000:a1:00.0
> > >    -    90200000-90203fff : 0000:a1:00.0
> > >    -      90200000-90203fff : igb
> > >    +  9be00000-9c0fffff : PCI Bus 0000:a1
> > >    +    9be00000-9befffff : 0000:a1:00.0
> > >    +    9bf00000-9bf03fff : 0000:a1:00.0
> > >    +    9c000000-9c0fffff : 0000:a1:00.0
> > >       9c100000-9c17ffff : amd_iommu
> > >       9c180000-9c1803ff : IOAPIC 8
> > > 
> > >    -       Region 0: Memory at 90000000 (32-bit, non-prefetchable)
> > > [size=1M]
> > >    -       Region 3: Memory at 90200000 (32-bit, non-prefetchable)
> > > [size=16K]
> > >    -       Expansion ROM at 90100000 [disabled] [size=1M]
> > >    +       Region 0: Memory at 9c000000 (32-bit, non-prefetchable)
> > > [size=1M]
> > >    +       Region 3: Memory at 9bf00000 (32-bit, non-prefetchable)
> > > [size=16K]
> > >    +       Expansion ROM at 9be00000 [disabled] [size=1M]
> > > 
> > > igb 0000:a1:00.0 0000:a1:00.0 (uninitialized): PCIe link lost
> > > ------------[ cut here ]------------
> > > igb: Failed to read reg 0x18!
> > > WARNING: drivers/net/ethernet/intel/igb/igb_main.c:724 at
> > > igb_rd32.cold+0x3c/0x4f [igb], CPU#32: kworker/32:1/706
> > > ...
> > >   igb_get_invariants_82575+0xff/0xf00 [igb]
> > >   igb_probe+0x3c8/0x1190 [igb]
> > >   local_pci_probe+0x3b/0x80
> > > 
> > > It turns out there is a 64kB iomem black hole at 9c000000 that returns
> > > ~0 and this is where igb's BAR 0 resides. If another BAR of the same
> > > card is placed into that address, it is similarly black holed.
> > > 
> > > Mark the problematic 64kB range reserved using a quirk bound to the
> > > bridge found in the problematic system.
> > > 
> > > Closes: https://bugzilla.kernel.org/show_bug.cgi?id=222074
> > > Suggested-by: Mario Limonciello <mario.limonciello@amd.com>
> > > Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
> > > ---
> > > Cc: Borislav Petkov <bp@alien8.de>
> > > 
> > > As an alternative to this, I've also tested reverting the commit
> > > 07eab0901ede ("efi/x86: Remove EfiMemoryMappedIO from E820 map") and
> > > that also resolves the issue by retaining "9c000000-9cffffff : Reserved"
> > > entry in the resource tree.
> > > 
> > > ---
> > >   arch/x86/pci/fixup.c | 36 ++++++++++++++++++++++++++++++++++++
> > >   1 file changed, 36 insertions(+)
> > > 
> > > diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c
> > > index b301c6c8df75..cc463bf492b6 100644
> > > --- a/arch/x86/pci/fixup.c
> > > +++ b/arch/x86/pci/fixup.c
> > > @@ -822,6 +822,42 @@ DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1571,
> > > pci_amd_enable_64bit_bar);
> > >   DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x15b1,
> > > pci_amd_enable_64bit_bar);
> > >   DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1601,
> > > pci_amd_enable_64bit_bar);
> > > +/*
> > > + * TODO: Remove when/if root cause is found.
> > > + *
> > > + * Genoa appears to have a 64kB iomem black hole starting at 0x9c000000
> > > + * address for which all reads return ~0 blocking an overlapping BAR from
> > > + * working.
> > > + *
> > > + * Work around the problem by reserving the space prior to making any
> > > iomem
> > > + * allocations that could overlap with the black hole.
> > > + */
> > > +static struct resource genoa_black_hole =
> > > DEFINE_RES_MEM_NAMED(0x9c000000,
> > > +                                   SZ_64K,
> > > +                                   "reserved");
> > > +
> > > +static void genoa_iomem_black_hole(struct pci_dev *dev)
> > > +{
> > > +    struct resource *r;
> > > +
> > > +    pci_bus_for_each_resource(dev->bus, r) {
> > > +        if (!r || !r->flags || !resource_assigned(r))
> > > +            continue;
> > > +
> > > +        if (!__resource_contains_unbound(r, &genoa_black_hole))
> > > +            continue;
> > > +
> > > +        if (resource_assigned(&genoa_black_hole))
> > > +            pci_dbg(dev, "iomem black hole workaround already
> > > applied\n");
> > > +        else if (!insert_resource(r, &genoa_black_hole))
> > > +            pci_info(dev, "iomem black hole workaround enabled\n");
> > > +        else
> > > +            pci_dbg(dev, "iomem black hole workaround add failed\n");
> > > +        return;
> > > +    }
> > > +}
> > > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, 0x14ab,
> > > genoa_iomem_black_hole);
> > > +
> > >   #define RS690_LOWER_TOP_OF_DRAM2    0x30
> > >   #define RS690_LOWER_TOP_OF_DRAM2_VALID    0x1
> > >   #define RS690_UPPER_TOP_OF_DRAM2    0x31
> > 
> > I guess it's good we know the root cause is 07eab0901ede. If we're going
> > down the path of making holes in that we may well put the hole in
> > efi_remove_e820_mmio() like this (attached).
> 
> I don't really like that though.  If Genoa is affected I would expect plenty
> of other machines are too they just haven't been found yet.

That's what I'm worried as well. My own testing can only cover very little 
ground here.

> Can we roll back 07eab0901ede and add quirks for the busted machines instead?


-- 
 i.