RE: [RFC PATCH] PCI: Work around Pericom PI7C9X3G606GPC Port 4 BAR erratum

["Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>]

[-- Attachment #1: Type: text/plain, Size: 9788 bytes --]

On Fri, 24 Jul 2026, JC Chen[陳饒靜] wrote:

> 
> Hi,
> 
>  
> 
> Attached are kernel log "dmesg.log" and lspci "lspci.log" output.
> 
>  
> 
> I guess without this patch, MSI-X from the downstream port doesn't work correctly?
> 
> => YES
> 
>  
> 
> By "mirror", I guess you mean you want the same value in BAR 0 of both the switch
> upstream port and the downstream port?
> 
> => Yes, please use the same value in BAR 0 of both the switch upstream and downstream
> ports.
> 
> I don't know what that even means, because the upstream port BAR 0 can't be inside
> its memory window, so reads should never reach the downstream port.
>
> => Yes, you are correct that the upstream port BAR 0 can't be inside its memory
> window, so reads should never reach the downstream port. When CDEP function is
> disabled, BAR0 of P4 is read as all zeros. If root complex send out memory address
> 0x7F000 or 0x7F080 to P4, it will hit P4’s MSI-X Table or PBA address range, switch
> will response UR and then cause SOC reports timeout. To avoid this issue, the patch
> write P0’s BAR0 to P4’s BAR0.

Hi,

I have problem of understanding why touching that downstream BAR helps 
because the bridge windows on 0000:17:00.0 are disabled.

-- 
 i.

> 
>  
> 
> Thanks,
> 
> JC
> 
>  
> 
> -----Original Message-----
> From: Bjorn Helgaas <helgaas@kernel.org>
> Sent: Friday, July 24, 2026 4:54 AM
> To: Nirmoy Das <nirmoyd@nvidia.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>; linux-pci@vger.kernel.org;
> linux-kernel@vger.kernel.org; JC Chen[陳饒靜] <jc_chen@diodes.com>; Ilpo Järvinen
> <ilpo.jarvinen@linux.intel.com>
> Subject: Re: [RFC PATCH] PCI: Work around Pericom PI7C9X3G606GPC Port 4 BAR erratum
> 
>  
> 
> [You don't often get email from helgaas@kernel.org. Learn why this is important at
> https://aka.ms/LearnAboutSenderIdentification ]
> 
>  
> 
> [+cc Ilpo, this sounds really weird from a resource perspective]
> 
>  
> 
> On Thu, Jul 23, 2026 at 09:10:00AM -0700, Nirmoy Das wrote:
> 
> > The Pericom PI7C9X3G606GPC PCIe switch has an erratum where downstream
> 
> > Port 4 retains a default MSI-X table and PBA decode when BAR 0 is zero.
> 
> > Memory reads that match this window are dropped with an Unsupported
> 
> > Request completion, which may cause the SoC to report a timeout.
> 
>  
> 
> Help me understand what's going on here.  Can you share the dmesg log of enumeration
> and resource assignment and the "lspci -v" output for the whole switch (both upstream
> and downstream ports) without this patch?
> 
>  
> 
> I guess without this patch, MSI-X from the downstream port doesn't work correctly?
> 
>  
> 
> > The workaround is to make Port 4 BAR 0 mirror BAR 0 of the immediate
> 
> > upstream port. Firmware may establish this at boot, but PCI resource
> 
> > assignment can move upstream BAR 0 without updating Port 4.
> 
>  
> 
> By "mirror", I guess you mean you want the same value in BAR 0 of both the switch
> upstream port and the downstream port?  I don't know what that even means, because
> the upstream port BAR 0 can't be inside its memory window, so reads should never
> reach the downstream port.
> 
>  
> 
> > For a 64-bit BAR, also mirror BAR 1 while memory decoding is disabled,
> 
> > matching the PCI core update sequence. Port 4 BAR 0 may read back as
> 
> > zero after a successful write, so do not use readback to validate the update.
> 
> > 
> 
> > Signed-off-by: Nirmoy Das <nirmoyd@nvidia.com>
> 
> > ---
> 
> >  drivers/pci/quirks.c | 92
> 
> > ++++++++++++++++++++++++++++++++++++++++++++
> 
> >  1 file changed, 92 insertions(+)
> 
> > 
> 
> > diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index
> 
> > b09f27f7846fc..fa8098d77adb5 100644
> 
> > --- a/drivers/pci/quirks.c
> 
> > +++ b/drivers/pci/quirks.c
> 
> > @@ -6264,6 +6264,98 @@ DECLARE_PCI_FIXUP_ENABLE(PCI_VENDOR_ID_PERICOM,
> 
> > 0xb404,  DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_PERICOM, 0xb404,
> 
> >                        pci_fixup_pericom_acs_store_forward);
> 
> > 
> 
> > +#define PCI_DEVICE_ID_PERICOM_PI7C9X3G606GPC 0xc008
> 
> > +
> 
> > +/*
> 
> > + * Pericom PI7C9X3G606GPC switch erratum E15 -
> 
> > + * Downstream Port 4 BAR 0 must mirror the immediate upstream port
> 
> > +BAR 0
> 
> > + *
> 
> > + * Port 4 uses BAR 0 for its MSI-X table and PBA. Firmware may
> 
> > +program this
> 
> > + * mirror at boot, but Linux resource assignment can move the
> 
> > +upstream BAR
> 
> > + * and leave Port 4 with a stale mirror.
> 
> > + *
> 
> > + * Diodes confirmed Tile0/P4 appears to Linux as device 4, function 0
> 
> > +on the
> 
> > + * bus below the upstream port. Match that downstream function and
> 
> > +re-apply
> 
> > + * the mirror after resource assignment and early resume.
> 
> > + */
> 
> > +static void pci_fixup_pericom_pi7c9x3g606gpc_bar0_mirror(struct
> 
> > +pci_dev *pdev) {
> 
> > +     struct pci_dev *upstream;
> 
> > +     bool bar0_64, disable_mem;
> 
> > +     u16 cmd = 0;
> 
> > +     u32 bar = 0, bar1 = 0, upstream_bar = 0, upstream_bar1 = 0;
> 
> > +
> 
> > +     if (pci_pcie_type(pdev) != PCI_EXP_TYPE_DOWNSTREAM)
> 
> > +             return;
> 
> > +
> 
> > +     if (PCI_SLOT(pdev->devfn) != 4 || PCI_FUNC(pdev->devfn))
> 
> > +             return;
> 
> > +
> 
> > +     upstream = pci_upstream_bridge(pdev);
> 
> > +     if (!upstream || upstream->vendor != PCI_VENDOR_ID_PERICOM ||
> 
> > +         upstream->device != PCI_DEVICE_ID_PERICOM_PI7C9X3G606GPC ||
> 
> > +         pci_pcie_type(upstream) != PCI_EXP_TYPE_UPSTREAM)
> 
> > +             return;
> 
> > +
> 
> > +     pci_read_config_dword(upstream, PCI_BASE_ADDRESS_0, &upstream_bar);
> 
> > +     if (upstream_bar & PCI_BASE_ADDRESS_SPACE_IO)
> 
> > +             return;
> 
> > +
> 
> > +     bar0_64 = (upstream_bar & PCI_BASE_ADDRESS_MEM_TYPE_MASK) ==
> 
> > +               PCI_BASE_ADDRESS_MEM_TYPE_64;
> 
> > +     if (bar0_64)
> 
> > +             pci_read_config_dword(upstream, PCI_BASE_ADDRESS_1,
> 
> > +                                   &upstream_bar1);
> 
> > +
> 
> > +     if (!(upstream_bar & PCI_BASE_ADDRESS_MEM_MASK) &&
> 
> > +         (!bar0_64 || !upstream_bar1)) {
> 
> > +             pci_warn(pdev, "skipping PI7C9X3G606GPC BAR 0 mirror workaround
> because upstream BAR 0 is unassigned\n");
> 
> > +             return;
> 
> > +     }
> 
> > +
> 
> > +     pci_read_config_dword(pdev, PCI_BASE_ADDRESS_0, &bar);
> 
> > +     if (bar0_64) {
> 
> > +             pci_read_config_dword(pdev, PCI_BASE_ADDRESS_1, &bar1);
> 
> > +             if (bar == upstream_bar && bar1 == upstream_bar1)
> 
> > +                     return;
> 
> > +     } else {
> 
> > +             if (bar == upstream_bar)
> 
> > +                     return;
> 
> > +     }
> 
> > +
> 
> > +     /*
> 
> > +      * Port 4 BAR 0 may read back as zero even after a successful write.
> 
> > +      * If BAR 0 is configured as 64-bit, BAR 1 is the upper half.
> 
> > +      * Disable memory decoding while updating both dwords, matching PCI
> 
> > +      * core's 64-bit BAR update sequence.
> 
> > +      */
> 
> > +     disable_mem = bar0_64 && !pdev->mmio_always_on;
> 
> > +     if (disable_mem) {
> 
> > +             pci_read_config_word(pdev, PCI_COMMAND, &cmd);
> 
> > +             pci_write_config_word(pdev, PCI_COMMAND,
> 
> > +                                   cmd & ~PCI_COMMAND_MEMORY);
> 
> > +     }
> 
> > +
> 
> > +     pci_write_config_dword(pdev, PCI_BASE_ADDRESS_0, upstream_bar);
> 
> > +     if (bar0_64)
> 
> > +             pci_write_config_dword(pdev, PCI_BASE_ADDRESS_1, upstream_bar1);
> 
> > +     if (disable_mem)
> 
> > +             pci_write_config_word(pdev, PCI_COMMAND, cmd);
> 
> > +
> 
> > +     if (bar0_64)
> 
> > +             pci_info(pdev, "wrote upstream BAR 0/1 %#x/%#x to Port 4 BAR 0/1 for
> PI7C9X3G606GPC BAR 0 mirror workaround\n",
> 
> > +                      upstream_bar, upstream_bar1);
> 
> > +     else
> 
> > +             pci_info(pdev, "wrote upstream BAR 0 %#x to Port 4 BAR 0 for
> PI7C9X3G606GPC BAR 0 mirror workaround\n",
> 
> > +                      upstream_bar);
> 
> > +}
> 
> > +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_PERICOM,
> 
> > +                     PCI_DEVICE_ID_PERICOM_PI7C9X3G606GPC,
> 
> > +                     pci_fixup_pericom_pi7c9x3g606gpc_bar0_mirror);
> 
> > +DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_PERICOM,
> 
> > +                            PCI_DEVICE_ID_PERICOM_PI7C9X3G606GPC,
> 
> > +                           
> 
> > +pci_fixup_pericom_pi7c9x3g606gpc_bar0_mirror);
> 
> > +
> 
> >  static void nvidia_ion_ahci_fixup(struct pci_dev *pdev)  {
> 
> >       pdev->dev_flags |= PCI_DEV_FLAGS_HAS_MSI_MASKING;
> 
> > 
> 
> > base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482
> 
> > --
> 
> > 2.43.0
> 
> > 
> 
> 
> 

-- 
 i.