[RFC PATCH] PCI: Tolerate non-prefetchable 64-bit BARs in prefetchable windows

[RFC PATCH] PCI: Tolerate non-prefetchable 64-bit BARs in prefetchable windows

[Ard Biesheuvel]

From: Ard Biesheuvel <ardb@kernel.org>

The prefetchable vs. non-prefetchable distinction is a relic of
conventional PCI, to denote from which regions PCI-PCI bridges were
permitted to perform speculative readahead.

For software compatibility reasons, PCI Express inherited the Type 1
header and models PCIe root ports as PCI-PCI bridges. However, this
readahead behavior does not exist in PCIe, and so this distinction has
mostly become meaningless on the bridge level.

As per the PCIe r6.3 ECN "Removing Prefetchable Terminology", the
'prefetchable' designation has been removed from the specification
entirely, on the basis that it is obsolete, and is being abused to
inform memory mapping attributes and other device/BAR level properties
that it was never intended for.

Given the limited range for non-prefetchable windows in the Type 1
header, and the fact that the distinction no longer exists for PCIe,
resource allocation performed by firmware may result in non-prefetchable
64-bit BARs being allocated inside prefetchable bridge windows.

Linux rejects such allocations ("can't claim; no compatible bridge
window") when it encounters them, but will usually fail to produce an
alternative allocation, given that firmware wouldn't have placed them
there in the first place if there was sufficient space in the
non-prefetchable window.

So at the very least, let's not reject such allocations when they were
made by the firmware.

Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: "Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
Link: https://github.com/tianocore/edk2/issues/13104

 drivers/pci/pci.c   | 3 ++-
 include/linux/pci.h | 4 ++--
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index b2879a6be5f8..e33eb9f3a139 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -761,7 +761,8 @@ struct resource *pci_find_parent_resource(const struct pci_dev *dev,
 			 * not, the allocator made a mistake.
 			 */
 			if (r->flags & IORESOURCE_PREFETCH &&
-			    !(res->flags & IORESOURCE_PREFETCH))
+			    !(res->flags & IORESOURCE_PREFETCH) &&
+			    !pci_is_pcie(dev))
 				return NULL;
 
 			/*
diff --git a/include/linux/pci.h b/include/linux/pci.h
index d31a8d107b1e..1cbb4b6c02c3 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -2684,7 +2684,7 @@ static inline void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool probe) { }
  * need to calculate PCIe capability offset from raw device for some
  * reasons, please use pci_find_capability() instead.
  */
-static inline int pci_pcie_cap(struct pci_dev *dev)
+static inline int pci_pcie_cap(const struct pci_dev *dev)
 {
 	return dev->pcie_cap;
 }
@@ -2695,7 +2695,7 @@ static inline int pci_pcie_cap(struct pci_dev *dev)
  *
  * Returns: true if the PCI device is PCI Express capable, false otherwise.
  */
-static inline bool pci_is_pcie(struct pci_dev *dev)
+static inline bool pci_is_pcie(const struct pci_dev *dev)
 {
 	return pci_pcie_cap(dev);
 }
-- 
2.55.0.1003.g10538fe699-goog

Re: [RFC PATCH] PCI: Tolerate non-prefetchable 64-bit BARs in prefetchable windows

[Ilpo Järvinen]

[-- Attachment #1: Type: text/plain, Size: 3919 bytes --]

On Thu, 10 Sep 2026, Ard Biesheuvel wrote:

> From: Ard Biesheuvel <ardb@kernel.org>
> 
> The prefetchable vs. non-prefetchable distinction is a relic of
> conventional PCI, to denote from which regions PCI-PCI bridges were
> permitted to perform speculative readahead.
> 
> For software compatibility reasons, PCI Express inherited the Type 1
> header and models PCIe root ports as PCI-PCI bridges. However, this
> readahead behavior does not exist in PCIe, and so this distinction has
> mostly become meaningless on the bridge level.
> 
> As per the PCIe r6.3 ECN "Removing Prefetchable Terminology", the
> 'prefetchable' designation has been removed from the specification
> entirely, on the basis that it is obsolete, and is being abused to
> inform memory mapping attributes and other device/BAR level properties
> that it was never intended for.
> 
> Given the limited range for non-prefetchable windows in the Type 1
> header, and the fact that the distinction no longer exists for PCIe,
> resource allocation performed by firmware may result in non-prefetchable
> 64-bit BARs being allocated inside prefetchable bridge windows.
> 
> Linux rejects such allocations ("can't claim; no compatible bridge
> window") when it encounters them, but will usually fail to produce an
> alternative allocation, given that firmware wouldn't have placed them
> there in the first place if there was sufficient space in the
> non-prefetchable window.
> 
> So at the very least, let's not reject such allocations when they were
> made by the firmware.
> 
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: "Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> ---
> Link: https://github.com/tianocore/edk2/issues/13104
> 
>  drivers/pci/pci.c   | 3 ++-
>  include/linux/pci.h | 4 ++--
>  2 files changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index b2879a6be5f8..e33eb9f3a139 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -761,7 +761,8 @@ struct resource *pci_find_parent_resource(const struct pci_dev *dev,
>  			 * not, the allocator made a mistake.
>  			 */
>  			if (r->flags & IORESOURCE_PREFETCH &&
> -			    !(res->flags & IORESOURCE_PREFETCH))
> +			    !(res->flags & IORESOURCE_PREFETCH) &&
> +			    !pci_is_pcie(dev))
>  				return NULL;
>  
>  			/*

I was more thinking along the lines of always setting IORESOURCE_PREFETCH 
for 64-bit BARs on PCIe devices but I've not had time to look at that/test 
how many things would break as a result. ...It would seem much simpler 
solution to differentiate PCI from PCIe while keeping the existing logic 
without adding similar pci_is_pcie() checks everywhere.

> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index d31a8d107b1e..1cbb4b6c02c3 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -2684,7 +2684,7 @@ static inline void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool probe) { }
>   * need to calculate PCIe capability offset from raw device for some
>   * reasons, please use pci_find_capability() instead.
>   */
> -static inline int pci_pcie_cap(struct pci_dev *dev)
> +static inline int pci_pcie_cap(const struct pci_dev *dev)
>  {
>  	return dev->pcie_cap;
>  }
> @@ -2695,7 +2695,7 @@ static inline int pci_pcie_cap(struct pci_dev *dev)
>   *
>   * Returns: true if the PCI device is PCI Express capable, false otherwise.
>   */
> -static inline bool pci_is_pcie(struct pci_dev *dev)
> +static inline bool pci_is_pcie(const struct pci_dev *dev)
>  {
>  	return pci_pcie_cap(dev);
>  }
> 

This const conversion should be done in a separate preparatory patch (if 
needed in the end, though IMO it wouldn't hurt to have it as const 
anyway even if it's not necessary for the 64-bit/pref change in the end).

-- 
 i.

Re: [RFC PATCH] PCI: Tolerate non-prefetchable 64-bit BARs in prefetchable windows

[Ard Biesheuvel]

On Thu, 10 Sep 2026, at 19:31, Ilpo Järvinen wrote:
> On Thu, 10 Sep 2026, Ard Biesheuvel wrote:
>
>> From: Ard Biesheuvel <ardb@kernel.org>
>> 
>> The prefetchable vs. non-prefetchable distinction is a relic of
>> conventional PCI, to denote from which regions PCI-PCI bridges were
>> permitted to perform speculative readahead.
>> 
>> For software compatibility reasons, PCI Express inherited the Type 1
>> header and models PCIe root ports as PCI-PCI bridges. However, this
>> readahead behavior does not exist in PCIe, and so this distinction has
>> mostly become meaningless on the bridge level.
>> 
>> As per the PCIe r6.3 ECN "Removing Prefetchable Terminology", the
>> 'prefetchable' designation has been removed from the specification
>> entirely, on the basis that it is obsolete, and is being abused to
>> inform memory mapping attributes and other device/BAR level properties
>> that it was never intended for.
>> 
>> Given the limited range for non-prefetchable windows in the Type 1
>> header, and the fact that the distinction no longer exists for PCIe,
>> resource allocation performed by firmware may result in non-prefetchable
>> 64-bit BARs being allocated inside prefetchable bridge windows.
>> 
>> Linux rejects such allocations ("can't claim; no compatible bridge
>> window") when it encounters them, but will usually fail to produce an
>> alternative allocation, given that firmware wouldn't have placed them
>> there in the first place if there was sufficient space in the
>> non-prefetchable window.
>> 
>> So at the very least, let's not reject such allocations when they were
>> made by the firmware.
>> 
>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>> Cc: "Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>
>> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
>> ---
>> Link: https://github.com/tianocore/edk2/issues/13104
>> 
>>  drivers/pci/pci.c   | 3 ++-
>>  include/linux/pci.h | 4 ++--
>>  2 files changed, 4 insertions(+), 3 deletions(-)
>> 
>> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
>> index b2879a6be5f8..e33eb9f3a139 100644
>> --- a/drivers/pci/pci.c
>> +++ b/drivers/pci/pci.c
>> @@ -761,7 +761,8 @@ struct resource *pci_find_parent_resource(const struct pci_dev *dev,
>>  			 * not, the allocator made a mistake.
>>  			 */
>>  			if (r->flags & IORESOURCE_PREFETCH &&
>> -			    !(res->flags & IORESOURCE_PREFETCH))
>> +			    !(res->flags & IORESOURCE_PREFETCH) &&
>> +			    !pci_is_pcie(dev))
>>  				return NULL;
>>  
>>  			/*
>
> I was more thinking along the lines of always setting IORESOURCE_PREFETCH 
> for 64-bit BARs on PCIe devices but I've not had time to look at that/test 
> how many things would break as a result. ...It would seem much simpler 
> solution to differentiate PCI from PCIe while keeping the existing logic 
> without adding similar pci_is_pcie() checks everywhere.
>

I agree that the code changes would be much simpler.

However, would this impact the PCI metadata observed by all consumers,
including userspace, and drivers that may expect a certain BAR layout,
and/or base decisions about memory attributes on this?

In particular, I am concerned about non-prefetchable BARs that actually
have side effects on read, being mapped with WC (or Normal-NC on arm64)
semantics, where the interconnect may widen, combine or reorder accesses.


>> diff --git a/include/linux/pci.h b/include/linux/pci.h
>> index d31a8d107b1e..1cbb4b6c02c3 100644
>> --- a/include/linux/pci.h
>> +++ b/include/linux/pci.h
>> @@ -2684,7 +2684,7 @@ static inline void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool probe) { }
>>   * need to calculate PCIe capability offset from raw device for some
>>   * reasons, please use pci_find_capability() instead.
>>   */
>> -static inline int pci_pcie_cap(struct pci_dev *dev)
>> +static inline int pci_pcie_cap(const struct pci_dev *dev)
>>  {
>>  	return dev->pcie_cap;
>>  }
>> @@ -2695,7 +2695,7 @@ static inline int pci_pcie_cap(struct pci_dev *dev)
>>   *
>>   * Returns: true if the PCI device is PCI Express capable, false otherwise.
>>   */
>> -static inline bool pci_is_pcie(struct pci_dev *dev)
>> +static inline bool pci_is_pcie(const struct pci_dev *dev)
>>  {
>>  	return pci_pcie_cap(dev);
>>  }
>> 
>
> This const conversion should be done in a separate preparatory patch (if 
> needed in the end, though IMO it wouldn't hurt to have it as const 
> anyway even if it's not necessary for the 64-bit/pref change in the end).
>

Of course. But this is just an RFC to get the discussion going, so I didn't
bother at this point.

Re: [RFC PATCH] PCI: Tolerate non-prefetchable 64-bit BARs in prefetchable windows

[Ilpo Järvinen]

[-- Attachment #1: Type: text/plain, Size: 4646 bytes --]

On Thu, 10 Sep 2026, Ard Biesheuvel wrote:

> 
> On Thu, 10 Sep 2026, at 19:31, Ilpo Järvinen wrote:
> > On Thu, 10 Sep 2026, Ard Biesheuvel wrote:
> >
> >> From: Ard Biesheuvel <ardb@kernel.org>
> >> 
> >> The prefetchable vs. non-prefetchable distinction is a relic of
> >> conventional PCI, to denote from which regions PCI-PCI bridges were
> >> permitted to perform speculative readahead.
> >> 
> >> For software compatibility reasons, PCI Express inherited the Type 1
> >> header and models PCIe root ports as PCI-PCI bridges. However, this
> >> readahead behavior does not exist in PCIe, and so this distinction has
> >> mostly become meaningless on the bridge level.
> >> 
> >> As per the PCIe r6.3 ECN "Removing Prefetchable Terminology", the
> >> 'prefetchable' designation has been removed from the specification
> >> entirely, on the basis that it is obsolete, and is being abused to
> >> inform memory mapping attributes and other device/BAR level properties
> >> that it was never intended for.
> >> 
> >> Given the limited range for non-prefetchable windows in the Type 1
> >> header, and the fact that the distinction no longer exists for PCIe,
> >> resource allocation performed by firmware may result in non-prefetchable
> >> 64-bit BARs being allocated inside prefetchable bridge windows.
> >> 
> >> Linux rejects such allocations ("can't claim; no compatible bridge
> >> window") when it encounters them, but will usually fail to produce an
> >> alternative allocation, given that firmware wouldn't have placed them
> >> there in the first place if there was sufficient space in the
> >> non-prefetchable window.
> >> 
> >> So at the very least, let's not reject such allocations when they were
> >> made by the firmware.
> >> 
> >> Cc: Bjorn Helgaas <bhelgaas@google.com>
> >> Cc: "Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>
> >> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> >> ---
> >> Link: https://github.com/tianocore/edk2/issues/13104
> >> 
> >>  drivers/pci/pci.c   | 3 ++-
> >>  include/linux/pci.h | 4 ++--
> >>  2 files changed, 4 insertions(+), 3 deletions(-)
> >> 
> >> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> >> index b2879a6be5f8..e33eb9f3a139 100644
> >> --- a/drivers/pci/pci.c
> >> +++ b/drivers/pci/pci.c
> >> @@ -761,7 +761,8 @@ struct resource *pci_find_parent_resource(const struct pci_dev *dev,
> >>  			 * not, the allocator made a mistake.
> >>  			 */
> >>  			if (r->flags & IORESOURCE_PREFETCH &&
> >> -			    !(res->flags & IORESOURCE_PREFETCH))
> >> +			    !(res->flags & IORESOURCE_PREFETCH) &&
> >> +			    !pci_is_pcie(dev))
> >>  				return NULL;
> >>  
> >>  			/*
> >
> > I was more thinking along the lines of always setting IORESOURCE_PREFETCH 
> > for 64-bit BARs on PCIe devices but I've not had time to look at that/test 
> > how many things would break as a result. ...It would seem much simpler 
> > solution to differentiate PCI from PCIe while keeping the existing logic 
> > without adding similar pci_is_pcie() checks everywhere.
> >
> 
> I agree that the code changes would be much simpler.
> 
> However, would this impact the PCI metadata observed by all consumers,
> including userspace,

Yes, it will impact userspace. But quoting you from above:

"is being abused to inform memory mapping attributes and other device/BAR 
level properties that it was never intended for."

What does the userspace then do with the information? Does it qualify 
under "it was never intended for"?

> and drivers that may expect a certain BAR layout,

??? Would that even be spec compliant??

> and/or base decisions about memory attributes on this?

The point is to consider them 64-bit window eligible so yes, kernel would 
definitely be basing decision on that but that's intentional.

> In particular, I am concerned about non-prefetchable BARs that actually
> have side effects on read, being mapped with WC (or Normal-NC on arm64)
> semantics, where the interconnect may widen, combine or reorder accesses.

So on a more concrete terms, you're referring to the check in 
proc_bus_pci_mmap()? And the one in __pci_resource_attr_is_visible() + 
pci_dev_resource_wc_is_visible()? I suppose that wouldn't work then.

So if just setting IORESOURCE_PREFETCH is not workable, how about adding a 
getter for res->flags which adds IORESOURCE_PREFETCH into the returned 
flags if it's PCIe device and (in the end) use the raw value only in those 
places that actually care about wc distinction. What I don't want to see 
us adding that pci_is_pcie() everywhere.

-- 
 i.

Re: [RFC PATCH] PCI: Tolerate non-prefetchable 64-bit BARs in prefetchable windows

[Ard Biesheuvel]

On Fri, 11 Sep 2026, at 11:27, Ilpo Järvinen wrote:
> On Thu, 10 Sep 2026, Ard Biesheuvel wrote:
>
>> 
>> On Thu, 10 Sep 2026, at 19:31, Ilpo Järvinen wrote:
>> > On Thu, 10 Sep 2026, Ard Biesheuvel wrote:
>> >
>> >> From: Ard Biesheuvel <ardb@kernel.org>
>> >> 
>> >> The prefetchable vs. non-prefetchable distinction is a relic of
>> >> conventional PCI, to denote from which regions PCI-PCI bridges were
>> >> permitted to perform speculative readahead.
>> >> 
>> >> For software compatibility reasons, PCI Express inherited the Type 1
>> >> header and models PCIe root ports as PCI-PCI bridges. However, this
>> >> readahead behavior does not exist in PCIe, and so this distinction has
>> >> mostly become meaningless on the bridge level.
>> >> 
>> >> As per the PCIe r6.3 ECN "Removing Prefetchable Terminology", the
>> >> 'prefetchable' designation has been removed from the specification
>> >> entirely, on the basis that it is obsolete, and is being abused to
>> >> inform memory mapping attributes and other device/BAR level properties
>> >> that it was never intended for.
>> >> 
>> >> Given the limited range for non-prefetchable windows in the Type 1
>> >> header, and the fact that the distinction no longer exists for PCIe,
>> >> resource allocation performed by firmware may result in non-prefetchable
>> >> 64-bit BARs being allocated inside prefetchable bridge windows.
>> >> 
>> >> Linux rejects such allocations ("can't claim; no compatible bridge
>> >> window") when it encounters them, but will usually fail to produce an
>> >> alternative allocation, given that firmware wouldn't have placed them
>> >> there in the first place if there was sufficient space in the
>> >> non-prefetchable window.
>> >> 
>> >> So at the very least, let's not reject such allocations when they were
>> >> made by the firmware.
>> >> 
>> >> Cc: Bjorn Helgaas <bhelgaas@google.com>
>> >> Cc: "Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>
>> >> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
>> >> ---
>> >> Link: https://github.com/tianocore/edk2/issues/13104
>> >> 
>> >>  drivers/pci/pci.c   | 3 ++-
>> >>  include/linux/pci.h | 4 ++--
>> >>  2 files changed, 4 insertions(+), 3 deletions(-)
>> >> 
>> >> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
>> >> index b2879a6be5f8..e33eb9f3a139 100644
>> >> --- a/drivers/pci/pci.c
>> >> +++ b/drivers/pci/pci.c
>> >> @@ -761,7 +761,8 @@ struct resource *pci_find_parent_resource(const struct pci_dev *dev,
>> >>  			 * not, the allocator made a mistake.
>> >>  			 */
>> >>  			if (r->flags & IORESOURCE_PREFETCH &&
>> >> -			    !(res->flags & IORESOURCE_PREFETCH))
>> >> +			    !(res->flags & IORESOURCE_PREFETCH) &&
>> >> +			    !pci_is_pcie(dev))
>> >>  				return NULL;
>> >>  
>> >>  			/*
>> >
>> > I was more thinking along the lines of always setting IORESOURCE_PREFETCH 
>> > for 64-bit BARs on PCIe devices but I've not had time to look at that/test 
>> > how many things would break as a result. ...It would seem much simpler 
>> > solution to differentiate PCI from PCIe while keeping the existing logic 
>> > without adding similar pci_is_pcie() checks everywhere.
>> >
>> 
>> I agree that the code changes would be much simpler.
>> 
>> However, would this impact the PCI metadata observed by all consumers,
>> including userspace,
>
> Yes, it will impact userspace. But quoting you from above:
>
> "is being abused to inform memory mapping attributes and other device/BAR 
> level properties that it was never intended for."
>
> What does the userspace then do with the information? Does it qualify 
> under "it was never intended for"?
>

Perhaps, but that does not mean we are allowed to break it now.

More specifically, would the output of 'lspci' change as a result?

>> and drivers that may expect a certain BAR layout,
>
> ??? Would that even be spec compliant??
>

Again, maybe not, but if something that works fine today stops
working because the PCI subsystem started lying to the driver about
how the PCIe device describes itself, we'll be on the hook to fix it.

>> and/or base decisions about memory attributes on this?
>
> The point is to consider them 64-bit window eligible so yes, kernel would 
> definitely be basing decision on that but that's intentional.
>

That would mean that a driver may decide to use ioremap_wc() rather than
ioremap() to map a non-prefetchable BAR that we decided to misrepresent
as a prefetchable one. Even if the (pseudo-)PCI-PCI bridge will not do
any readahead, the CPU or interconnect may behave very differently as a
result, and touch BAR regions that the driver never accessed explicitly.

>> In particular, I am concerned about non-prefetchable BARs that actually
>> have side effects on read, being mapped with WC (or Normal-NC on arm64)
>> semantics, where the interconnect may widen, combine or reorder accesses.
>
> So on a more concrete terms, you're referring to the check in 
> proc_bus_pci_mmap()? And the one in __pci_resource_attr_is_visible() + 
> pci_dev_resource_wc_is_visible()? I suppose that wouldn't work then.
>

No, I am referring to the hundreds of ioremap() and ioremap_wc() calls
under drivers. Maybe none of them are affected but who knows.

> So if just setting IORESOURCE_PREFETCH is not workable, how about adding a 
> getter for res->flags which adds IORESOURCE_PREFETCH into the returned 
> flags if it's PCIe device and (in the end) use the raw value only in those 
> places that actually care about wc distinction. What I don't want to see 
> us adding that pci_is_pcie() everywhere.
>

Maybe add another IORESOURCE_PREFETCH_xxx flag that indicates that the
resource may be placed in a prefetchable bridge window? We'd only have
to set it in a single place (when probing the BAR), and we can add
support for it piecemeal in the validation and allocation logic.

Re: [RFC PATCH] PCI: Tolerate non-prefetchable 64-bit BARs in prefetchable windows

[Lorenzo Pieralisi]

On Fri, Sep 11, 2026 at 12:27:25PM +0300, Ilpo Järvinen wrote:
> On Thu, 10 Sep 2026, Ard Biesheuvel wrote:
> 
> > 
> > On Thu, 10 Sep 2026, at 19:31, Ilpo Järvinen wrote:
> > > On Thu, 10 Sep 2026, Ard Biesheuvel wrote:
> > >
> > >> From: Ard Biesheuvel <ardb@kernel.org>
> > >> 
> > >> The prefetchable vs. non-prefetchable distinction is a relic of
> > >> conventional PCI, to denote from which regions PCI-PCI bridges were
> > >> permitted to perform speculative readahead.
> > >> 
> > >> For software compatibility reasons, PCI Express inherited the Type 1
> > >> header and models PCIe root ports as PCI-PCI bridges. However, this
> > >> readahead behavior does not exist in PCIe, and so this distinction has
> > >> mostly become meaningless on the bridge level.
> > >> 
> > >> As per the PCIe r6.3 ECN "Removing Prefetchable Terminology", the
> > >> 'prefetchable' designation has been removed from the specification
> > >> entirely, on the basis that it is obsolete, and is being abused to
> > >> inform memory mapping attributes and other device/BAR level properties
> > >> that it was never intended for.
> > >> 
> > >> Given the limited range for non-prefetchable windows in the Type 1
> > >> header, and the fact that the distinction no longer exists for PCIe,
> > >> resource allocation performed by firmware may result in non-prefetchable
> > >> 64-bit BARs being allocated inside prefetchable bridge windows.
> > >> 
> > >> Linux rejects such allocations ("can't claim; no compatible bridge
> > >> window") when it encounters them, but will usually fail to produce an
> > >> alternative allocation, given that firmware wouldn't have placed them
> > >> there in the first place if there was sufficient space in the
> > >> non-prefetchable window.
> > >> 
> > >> So at the very least, let's not reject such allocations when they were
> > >> made by the firmware.
> > >> 
> > >> Cc: Bjorn Helgaas <bhelgaas@google.com>
> > >> Cc: "Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>
> > >> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> > >> ---
> > >> Link: https://github.com/tianocore/edk2/issues/13104
> > >> 
> > >>  drivers/pci/pci.c   | 3 ++-
> > >>  include/linux/pci.h | 4 ++--
> > >>  2 files changed, 4 insertions(+), 3 deletions(-)
> > >> 
> > >> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > >> index b2879a6be5f8..e33eb9f3a139 100644
> > >> --- a/drivers/pci/pci.c
> > >> +++ b/drivers/pci/pci.c
> > >> @@ -761,7 +761,8 @@ struct resource *pci_find_parent_resource(const struct pci_dev *dev,
> > >>  			 * not, the allocator made a mistake.
> > >>  			 */
> > >>  			if (r->flags & IORESOURCE_PREFETCH &&
> > >> -			    !(res->flags & IORESOURCE_PREFETCH))
> > >> +			    !(res->flags & IORESOURCE_PREFETCH) &&
> > >> +			    !pci_is_pcie(dev))
> > >>  				return NULL;
> > >>  
> > >>  			/*
> > >
> > > I was more thinking along the lines of always setting IORESOURCE_PREFETCH 
> > > for 64-bit BARs on PCIe devices but I've not had time to look at that/test 
> > > how many things would break as a result. ...It would seem much simpler 
> > > solution to differentiate PCI from PCIe while keeping the existing logic 
> > > without adding similar pci_is_pcie() checks everywhere.
> > >
> > 
> > I agree that the code changes would be much simpler.
> > 
> > However, would this impact the PCI metadata observed by all consumers,
> > including userspace,
> 
> Yes, it will impact userspace. But quoting you from above:
> 
> "is being abused to inform memory mapping attributes and other device/BAR 
> level properties that it was never intended for."
> 
> What does the userspace then do with the information? Does it qualify 
> under "it was never intended for"?

The issue is the resulting mmap in the kernel- we can end up mapping
MMIO regions that don't tolerate side-effects with NormalNC attributes on arm64
(that's what WC is on arm64 and in turn WC is detected in places - eg sysfs -
through IORESOURCE_PREFETCH), which can end up in mayhem because those mappings
allow eg speculative reads.

Hopefully, this is not an issue in kernel drivers because drivers *should* know
what a BAR is and should ignore the resource flag as far as mem mappings are
concerned.

> > and drivers that may expect a certain BAR layout,
> 
> ??? Would that even be spec compliant??
> 
> > and/or base decisions about memory attributes on this?
> 
> The point is to consider them 64-bit window eligible so yes, kernel would 
> definitely be basing decision on that but that's intentional.
> 
> > In particular, I am concerned about non-prefetchable BARs that actually
> > have side effects on read, being mapped with WC (or Normal-NC on arm64)
> > semantics, where the interconnect may widen, combine or reorder accesses.
> 
> So on a more concrete terms, you're referring to the check in 
> proc_bus_pci_mmap()? And the one in __pci_resource_attr_is_visible() + 
> pci_dev_resource_wc_is_visible()? I suppose that wouldn't work then.

Indeed, I don't want to speak for Ard but that's exactly a couple of places
where things can go haywire. Then there is PCI drivers even though I don't
think we should have a problem there but as Ard replied, who knows, see above.

Lorenzo

> So if just setting IORESOURCE_PREFETCH is not workable, how about adding a 
> getter for res->flags which adds IORESOURCE_PREFETCH into the returned 
> flags if it's PCIe device and (in the end) use the raw value only in those 
> places that actually care about wc distinction. What I don't want to see 
> us adding that pci_is_pcie() everywhere.
> 
> -- 
>  i.