[PATCH v3 0/2] PCI: Allow non-prefetchable BARs in prefetchable windows

[PATCH v3 0/2] PCI: Allow non-prefetchable BARs in prefetchable windows

[Ard Biesheuvel]

From: Ard Biesheuvel <ardb@kernel.org>

(v2 never made it to linux-pci@ - please disregard it if you did get it)

The non-prefetchable window of a PCI-to-PCI bridge can only decode
32-bit addresses, and so even 64-bit non-prefetchable BARs of devices
below a bridge compete for the scarce MMIO space below 4 GB.

The PCIe spec notes that marking such a BAR prefetchable still permits
correct operation as long as the entire path from the host to the device
is PCIe (PCIe r7.0, sec 7.5.1.2.1). The same applies when the OS places
a non-prefetchable BAR in a prefetchable bridge window, and so this
series permits that unless there is a bridge on the path to the root bus
that is not a PCIe Root Port or Switch Port, allowing 64-bit
non-prefetchable BARs to be placed above 4 GB.

So add a new bus flag PCI_BUS_FLAGS_NO_NP_BARS_IN_P_WINDOWS, and wire it
up so that it is only set on buses where this relaxed placement is not
permitted. Then, use the flag to to decide whether to treat 64-bit NP
BARs as prefetchable only for the purpose of resource allocation (but
not for mapping the MMIO region or other reporting its properties)

Patch #1 is a preparatory refactor of pci_bus_alloc_resource() with no
functional change, and patch #2 implements the relaxation.

(based on v7.3-rc4)

Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: "Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>
Cc: Lorenzo Pieralisi <lpieralisi@kernel.org>
Link: https://lore.kernel.org/linux-pci/20260910143440.3865663-2-ardb+git@google.com/

Ard Biesheuvel (2):
  PCI: Add __pci_bus_alloc_resource() to allocate with explicit flags
  PCI: Allow 64-bit non-prefetchable BARs in prefetchable windows

 drivers/pci/bus.c       | 70 ++++++++++++++------
 drivers/pci/pci.c       | 43 +++++++++++-
 drivers/pci/pci.h       |  7 ++
 drivers/pci/probe.c     | 35 ++++++++++
 drivers/pci/setup-bus.c | 23 +++++--
 drivers/pci/setup-res.c | 30 ++++++---
 include/linux/pci.h     |  1 +
 7 files changed, 169 insertions(+), 40 deletions(-)

-- 
2.56.0.rc1.315.gc6ed9934b7-goog

[PATCH v3 1/2] PCI: Add __pci_bus_alloc_resource() to allocate with explicit flags

[Ard Biesheuvel]

From: Ard Biesheuvel <ardb@kernel.org>

pci_bus_alloc_resource() decides which bus resources a resource may be
allocated from based on the flags of that resource: the type bits
selected by @type_mask must match, a non-prefetchable resource is never
allocated from a prefetchable bus resource, and only a 64-bit resource
may be placed above 4 GB.

In preparation for placing some non-prefetchable BARs in prefetchable
bridge windows, factor out __pci_bus_alloc_resource(), which takes the
flags to match against the bus resources as a separate argument, and
implement pci_bus_alloc_resource() on top of it by passing the flags of
the resource itself.

No functional change intended.

Assisted-by: LLM
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 drivers/pci/bus.c | 70 ++++++++++++++------
 drivers/pci/pci.h |  6 ++
 2 files changed, 54 insertions(+), 22 deletions(-)

diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index 655ed53436d3..65a54d6e049e 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -188,9 +188,9 @@ static void pci_clip_resource_to_region(struct pci_bus *bus,
 }
 
 static int pci_bus_alloc_from_region(struct pci_bus *bus, struct resource *res,
-		resource_size_t size, resource_size_t align,
-		resource_size_t min, unsigned long type_mask,
-		resource_alignf alignf,
+		unsigned long flags, resource_size_t size,
+		resource_size_t align, resource_size_t min,
+		unsigned long type_mask, resource_alignf alignf,
 		void *alignf_data,
 		struct pci_bus_region *region)
 {
@@ -210,13 +210,13 @@ static int pci_bus_alloc_from_region(struct pci_bus *bus, struct resource *res,
 			continue;
 
 		/* type_mask must match */
-		if ((res->flags ^ r->flags) & type_mask)
+		if ((flags ^ r->flags) & type_mask)
 			continue;
 
 		/* We cannot allocate a non-prefetching resource
 		   from a pre-fetching area */
 		if ((r->flags & IORESOURCE_PREFETCH) &&
-		    !(res->flags & IORESOURCE_PREFETCH))
+		    !(flags & IORESOURCE_PREFETCH))
 			continue;
 
 		avail = *r;
@@ -247,9 +247,10 @@ static int pci_bus_alloc_from_region(struct pci_bus *bus, struct resource *res,
 }
 
 /**
- * pci_bus_alloc_resource - allocate a resource from a parent bus
+ * __pci_bus_alloc_resource - allocate a resource from a parent bus
  * @bus: PCI bus
  * @res: resource to allocate
+ * @flags: resource flags to match against the bus resources
  * @size: size of resource to allocate
  * @align: alignment of resource to allocate
  * @min: minimum /proc/iomem address to allocate
@@ -257,36 +258,61 @@ static int pci_bus_alloc_from_region(struct pci_bus *bus, struct resource *res,
  * @alignf: resource alignment function
  * @alignf_data: data argument for resource alignment function
  *
- * Given the PCI bus a device resides on, the size, minimum address,
- * alignment and type, try to find an acceptable resource allocation
- * for a specific device resource.
+ * Like pci_bus_alloc_resource(), but match the bus resources against @flags
+ * rather than @res->flags. This permits allocating a non-prefetchable
+ * resource from a prefetchable bus resource by passing @flags with
+ * IORESOURCE_PREFETCH set, which the caller must have established to be safe.
  */
-int pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res,
-		resource_size_t size, resource_size_t align,
-		resource_size_t min, unsigned long type_mask,
-		resource_alignf alignf,
-		void *alignf_data)
+int __pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res,
+			     unsigned long flags, resource_size_t size,
+			     resource_size_t align, resource_size_t min,
+			     unsigned long type_mask, resource_alignf alignf,
+			     void *alignf_data)
 {
 #ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
 	int rc;
 
-	if (res->flags & IORESOURCE_MEM_64) {
-		rc = pci_bus_alloc_from_region(bus, res, size, align, min,
-					       type_mask, alignf, alignf_data,
-					       &pci_high);
+	if (flags & IORESOURCE_MEM_64) {
+		rc = pci_bus_alloc_from_region(bus, res, flags, size, align,
+					       min, type_mask, alignf,
+					       alignf_data, &pci_high);
 		if (rc == 0)
 			return 0;
 
-		return pci_bus_alloc_from_region(bus, res, size, align, min,
-						 type_mask, alignf, alignf_data,
-						 &pci_64_bit);
+		return pci_bus_alloc_from_region(bus, res, flags, size, align,
+						 min, type_mask, alignf,
+						 alignf_data, &pci_64_bit);
 	}
 #endif
 
-	return pci_bus_alloc_from_region(bus, res, size, align, min,
+	return pci_bus_alloc_from_region(bus, res, flags, size, align, min,
 					 type_mask, alignf, alignf_data,
 					 &pci_32_bit);
 }
+
+/**
+ * pci_bus_alloc_resource - allocate a resource from a parent bus
+ * @bus: PCI bus
+ * @res: resource to allocate
+ * @size: size of resource to allocate
+ * @align: alignment of resource to allocate
+ * @min: minimum /proc/iomem address to allocate
+ * @type_mask: IORESOURCE_* type flags
+ * @alignf: resource alignment function
+ * @alignf_data: data argument for resource alignment function
+ *
+ * Given the PCI bus a device resides on, the size, minimum address,
+ * alignment and type, try to find an acceptable resource allocation
+ * for a specific device resource.
+ */
+int pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res,
+			   resource_size_t size, resource_size_t align,
+			   resource_size_t min, unsigned long type_mask,
+			   resource_alignf alignf, void *alignf_data)
+{
+	return __pci_bus_alloc_resource(bus, res, res->flags, size, align, min,
+					type_mask, alignf, alignf_data);
+}
 EXPORT_SYMBOL(pci_bus_alloc_resource);
 
 /*
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index ba3c3fddddc2..8297cfb5dcd5 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -567,6 +567,12 @@ static inline int pci_resource_num(const struct pci_dev *dev,
 	return resno;
 }
 
+int __pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res,
+			     unsigned long flags, resource_size_t size,
+			     resource_size_t align, resource_size_t min,
+			     unsigned long type_mask, resource_alignf alignf,
+			     void *alignf_data);
+
 void pbus_validate_busn(struct pci_bus *bus);
 struct resource *pbus_select_window(struct pci_bus *bus,
 				    const struct resource *res);
-- 
2.56.0.rc1.315.gc6ed9934b7-goog

[PATCH v3 2/2] PCI: Allow 64-bit non-prefetchable BARs in prefetchable windows

[Ard Biesheuvel]

From: Ard Biesheuvel <ardb@kernel.org>

The non-prefetchable memory window of a PCI-to-PCI bridge can only
decode 32-bit addresses, and so non-prefetchable BARs of devices below a
bridge can only be allocated from the part of the host bridge memory
space below 4 GB. This is the case even for 64-bit BARs, which could
easily be placed above 4 GB if there was a bridge window to put them in,
and on many platforms, 32-bit addressable MMIO space is scarce.

The PCIe spec addresses this in the implementation note "Additional
Guidance on the Prefetchable Bit in Memory Space BARs" (PCIe r7.0, sec
7.5.1.2.1): on PCIe, setting the Prefetchable bit of a BAR still permits
correct operation even if the range has read side effects or cannot
tolerate write merging, as long as the entire path from the host to the
device is PCIe, given that PCIe Memory Reads always carry an explicit
length, and PCIe Switches never prefetch or merge writes. The same
reasoning applies when it is the OS that places a non-prefetchable BAR
in a prefetchable bridge window: PCIe Root Ports and Switch Ports
forward requests that hit either window in exactly the same way. Hence,
the prefetchable window of a PCIe Root Port or Switch Port, which may be
64-bit, can serve as a 64-bit window for non-prefetchable BARs too. [0]

So add pci_bus_placement_flags(), which returns the flags of a resource
on a given bus with IORESOURCE_PREFETCH set if it is a 64-bit
non-prefetchable memory resource, the bus is not a root bus, all bridges
between the bus and the root bus are PCIe Root Ports or PCIe Switch
Ports, and the host bridge has no prefetchable memory window, but does
have one that extends above 4 GB in PCI bus address space. [1]

Evaluate the conditions on the bridges and the host bridge only once
per bus, when it is added, and record the result in a new bus flag,
PCI_BUS_FLAGS_NO_NP_BARS_IN_P_WINDOWS: pci_register_host_bridge() sets
it on the root bus if the host bridge does not qualify, child buses
inherit it, and pci_alloc_child_bus() sets it on the secondary bus of
any bridge that is not a PCIe Root Port or Switch Port.

Use pci_bus_placement_flags() wherever the resource allocator decides
which bridge window a device resource (including an SR-IOV VF BAR)
belongs in:

- in pbus_select_window_for_type(), which is used when sizing bridge
  windows, and when releasing them to retry failed assignments or to
  resize a BAR;
- when allocating the resource in __pci_assign_resource(), by passing
  its result to __pci_bus_alloc_resource();
- when claiming a resource assigned by firmware, in
  pci_find_parent_resource();
- when deciding which assigned resources to release after a failed
  assignment, and which failures are relevant to a resized BAR.

As a result, an eligible 64-bit non-prefetchable BAR is handled exactly
like a 64-bit prefetchable BAR: it is placed in the prefetchable window
of the upstream bridge, which may be above 4 GB, and allocation falls
back to the non-prefetchable window if the prefetchable one has no
space. [2]

32-bit BARs are not affected, and neither are bridge windows, as only
prefetchable bridge windows can be 64-bit. Devices below conventional
PCI or CardBus bridges, below PCIe to PCI/PCI-X bridges (in either
direction), or below host bridges that have a prefetchable window or no
window above 4 GB are not affected either, and neither are devices on a
root bus, as there is no prefetchable window for their BARs to go to.

[0] This reasoning does not extend to the host bridge, though: how it
    treats the windows that firmware describes as prefetchable is
    platform specific. For instance, the V3 Semiconductor V360EPC
    (pci-v3-semi) enables prefetching for its prefetchable window, the
    MPC52xx uses Memory Read Multiple for it, and Freescale PCI/PCIe
    host bridges (fsl_pci) enable relaxed ordering for it. However, if
    the host bridge has no prefetchable windows at all (as appears to be
    the case on many x86 PCs), all prefetchable bridge windows are
    carved out of its non-prefetchable windows, and so the host bridge
    does not treat them any differently.

[1] Placing non-prefetchable BARs in prefetchable bridge windows only
    helps if one of those non-prefetchable host bridge windows extends
    above 4 GB. Otherwise, the prefetchable bridge windows end up below
    4 GB as well, and BARs would merely move between two bridge windows
    that are carved out of the same 32-bit space.

[2] Note that this only concerns where a BAR is placed. How it is mapped
    is decided by the driver and by the attributes of the BAR itself
    (e.g., pci_iomap_wc() and the sysfs resource<N>_wc files only honour
    IORESOURCE_PREFETCH on the BAR), and this change does not modify the
    flags of any BAR.)

Assisted-by: LLM
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
Tested on QEMU arm64 'virt' (DT, all resources assigned by Linux) with
a qemu-xhci below a Root Port, an NVMe below a Switch, a qemu-xhci
below a PCIe-to-PCI bridge and another qemu-xhci on the root bus: the
64-bit non-prefetchable BARs of the first two (and of the PCIe-to-PCI
bridge itself) move from the 32-bit non-prefetchable windows into the
64-bit prefetchable windows above 4 GB, while the others stay where
they were. When the 64-bit host bridge window is marked prefetchable
in the DT, or removed from it, all resources are assigned exactly as
without this patch. Both drivers work, also after hot removal and
rescan of the endpoints and of the Switch. Also tested on QEMU x86_64
q35 with SeaBIOS, which assigns all resources itself: the firmware
assignment is claimed as before, and after hot removal and rescan, the
eligible BARs are placed in the prefetchable windows.
---
 drivers/pci/pci.c       | 43 +++++++++++++++++++-
 drivers/pci/pci.h       |  1 +
 drivers/pci/probe.c     | 35 ++++++++++++++++
 drivers/pci/setup-bus.c | 23 ++++++++---
 drivers/pci/setup-res.c | 30 +++++++++-----
 include/linux/pci.h     |  1 +
 6 files changed, 115 insertions(+), 18 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index b2879a6be5f8..51e96611d734 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -736,6 +736,43 @@ static bool pci_dev_config_accessible(struct pci_dev *dev, char *msg)
 	return true;
 }
 
+/**
+ * pci_bus_placement_flags - Get the flags to use for placing a resource
+ * @bus: PCI bus of the device that owns the resource
+ * @flags: Resource flags
+ *
+ * The non-prefetchable window of a PCI-to-PCI bridge can only decode 32-bit
+ * addresses, so 64-bit non-prefetchable BARs of devices below a bridge have to
+ * compete for space below 4GB. However, the PCIe spec notes that setting the
+ * Prefetchable bit of a BAR permits correct operation even if the range has
+ * read side effects or cannot tolerate write merging, as long as the entire
+ * path from the host to the device is PCIe: PCIe Memory Reads always carry an
+ * explicit length, and PCIe Switches never prefetch or merge writes (PCIe
+ * r7.0, sec 7.5.1.2.1, Implementation Note "Additional Guidance on the
+ * Prefetchable Bit in Memory Space BARs").
+ *
+ * So if all bridges between @bus and the root bus are PCIe Root Ports or PCIe
+ * Switch Ports, handle 64-bit non-prefetchable resources on @bus like 64-bit
+ * prefetchable ones, so that they can be placed in the prefetchable window of
+ * the bridge above @bus, which may be above 4GB. Other bridges, such as
+ * conventional PCI bridges, may prefetch from their prefetchable window.
+ *
+ * Return: @flags, with IORESOURCE_PREFETCH set if a resource with @flags on
+ * @bus may be placed in a prefetchable bridge window.
+ */
+unsigned long pci_bus_placement_flags(struct pci_bus *bus, unsigned long flags)
+{
+	if ((flags & (IORESOURCE_TYPE_BITS | IORESOURCE_PREFETCH |
+		      IORESOURCE_MEM_64)) != (IORESOURCE_MEM | IORESOURCE_MEM_64))
+		return flags;
+
+	if (pci_is_root_bus(bus) ||
+	    (bus->bus_flags & PCI_BUS_FLAGS_NO_NP_BARS_IN_P_WINDOWS))
+		return flags;
+
+	return flags | IORESOURCE_PREFETCH;
+}
+
 /**
  * pci_find_parent_resource - return resource region of parent bus of given
  *			      region
@@ -749,6 +786,7 @@ struct resource *pci_find_parent_resource(const struct pci_dev *dev,
 					  struct resource *res)
 {
 	const struct pci_bus *bus = dev->bus;
+	unsigned long flags = pci_bus_placement_flags(dev->bus, res->flags);
 	struct resource *r;
 
 	pci_bus_for_each_resource(bus, r) {
@@ -758,10 +796,11 @@ struct resource *pci_find_parent_resource(const struct pci_dev *dev,
 
 			/*
 			 * If the window is prefetchable but the BAR is
-			 * not, the allocator made a mistake.
+			 * not (and may not be treated as such), the
+			 * allocator made a mistake.
 			 */
 			if (r->flags & IORESOURCE_PREFETCH &&
-			    !(res->flags & IORESOURCE_PREFETCH))
+			    !(flags & IORESOURCE_PREFETCH))
 				return NULL;
 
 			/*
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 8297cfb5dcd5..989b9c1f1580 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -567,6 +567,7 @@ static inline int pci_resource_num(const struct pci_dev *dev,
 	return resno;
 }
 
+unsigned long pci_bus_placement_flags(struct pci_bus *bus, unsigned long flags);
 int __pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res,
 			     unsigned long flags, resource_size_t size,
 			     resource_size_t align, resource_size_t min,
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 27008e2ea5af..8f00ea3321c0 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -990,6 +990,32 @@ static bool pci_preserve_config(struct pci_host_bridge *host_bridge)
 	return false;
 }
 
+/*
+ * Return true if all memory windows of @bridge are non-prefetchable, and at
+ * least one of them extends above 4GB in PCI bus address space (see
+ * pci_bus_placement_flags()).
+ */
+static bool pci_host_np_only_with_high_window(struct pci_host_bridge *bridge)
+{
+	struct resource_entry *window;
+	bool high = false;
+
+	resource_list_for_each_entry(window, &bridge->windows) {
+		struct resource *res = window->res;
+
+		if (resource_type(res) != IORESOURCE_MEM)
+			continue;
+
+		if (res->flags & IORESOURCE_PREFETCH)
+			return false;
+
+		if (upper_32_bits(res->end - window->offset))
+			high = true;
+	}
+
+	return high;
+}
+
 static int pci_register_host_bridge(struct pci_host_bridge *bridge)
 {
 	struct device *parent = bridge->dev.parent;
@@ -1137,6 +1163,9 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge)
 		dev_info(&bus->dev, "root bus resource %pR%s\n", res, addr);
 	}
 
+	if (!pci_host_np_only_with_high_window(bridge))
+		bus->bus_flags |= PCI_BUS_FLAGS_NO_NP_BARS_IN_P_WINDOWS;
+
 	of_pci_make_host_bridge_node(bridge);
 
 	down_write(&pci_bus_sem);
@@ -1256,6 +1285,12 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent,
 		pci_info(child, "extended config space not accessible\n");
 	}
 
+	if (!pci_is_pcie(bridge) ||
+	    (pci_pcie_type(bridge) != PCI_EXP_TYPE_ROOT_PORT &&
+	     pci_pcie_type(bridge) != PCI_EXP_TYPE_UPSTREAM &&
+	     pci_pcie_type(bridge) != PCI_EXP_TYPE_DOWNSTREAM))
+		child->bus_flags |= PCI_BUS_FLAGS_NO_NP_BARS_IN_P_WINDOWS;
+
 	/* Set up default resource pointers and names */
 	for (i = 0; i < PCI_BRIDGE_RESOURCE_NUM; i++) {
 		child->resource[i] = &bridge->resource[PCI_BRIDGE_RESOURCES+i];
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index e8c94aa1d3c1..64826d8504c7 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -184,7 +184,9 @@ static struct resource *find_bus_resource_of_type(struct pci_bus *bus,
  *
  * For memory resources, the selection is done as follows:
  *
- * Any non-prefetchable resource is put into the non-prefetchable window.
+ * Any non-prefetchable resource is put into the non-prefetchable window,
+ * except for 64-bit ones that may be placed like prefetchable resources on
+ * @bus (see pci_bus_placement_flags()).
  *
  * If there is no prefetchable MMIO window, put all memory resources into the
  * non-prefetchable window.
@@ -203,6 +205,7 @@ static struct resource *pbus_select_window_for_type(struct pci_bus *bus,
 	int iores_type = type & IORESOURCE_TYPE_BITS;	/* w/o 64bit & pref */
 	struct resource *mmio, *mmio_pref, *win;
 
+	type = pci_bus_placement_flags(bus, type);
 	type &= PCI_RES_TYPE_MASK;			/* with 64bit & pref */
 
 	if ((iores_type != IORESOURCE_IO) && (iores_type != IORESOURCE_MEM))
@@ -261,7 +264,9 @@ static struct resource *pbus_select_window_for_type(struct pci_bus *bus,
  *
  * For memory resources, the selection is done as follows:
  *
- * Any non-prefetchable resource is put into the non-prefetchable window.
+ * Any non-prefetchable resource is put into the non-prefetchable window,
+ * except for 64-bit ones that may be placed like prefetchable resources on
+ * @bus (see pci_bus_placement_flags()).
  *
  * If there is no prefetchable MMIO window, put all memory resources into the
  * non-prefetchable window.
@@ -520,8 +525,11 @@ static unsigned long pci_fail_res_type_mask(struct list_head *fail_head)
 	unsigned long mask = 0;
 
 	/* Check failed type */
-	list_for_each_entry(fail_res, fail_head, list)
-		mask |= fail_res->flags;
+	list_for_each_entry(fail_res, fail_head, list) {
+		struct pci_bus *bus = fail_res->dev->bus;
+
+		mask |= pci_bus_placement_flags(bus, fail_res->flags);
+	}
 
 	/*
 	 * One pref failed resource will set IORESOURCE_MEM, as we can
@@ -564,8 +572,11 @@ static bool pci_required_resource_failed(struct list_head *fail_head,
 
 	list_for_each_entry(fail_res, fail_head, list) {
 		int idx = pci_resource_num(fail_res->dev, fail_res->res);
+		unsigned long flags;
 
-		if (type && (fail_res->flags & PCI_RES_TYPE_MASK) != type)
+		flags = pci_bus_placement_flags(fail_res->dev->bus,
+						fail_res->flags);
+		if (type && (flags & PCI_RES_TYPE_MASK) != type)
 			continue;
 
 		if (!pci_resource_is_optional(fail_res->dev, idx))
@@ -2308,7 +2319,7 @@ EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
 static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *res,
 					  struct list_head *saved)
 {
-	unsigned long type = res->flags;
+	unsigned long type = pci_bus_placement_flags(bus, res->flags);
 	struct pci_dev_resource *dev_res;
 	struct pci_dev *bridge = NULL;
 	LIST_HEAD(add_list);
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 376f09630a4a..78b7df0eee94 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -315,11 +315,20 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 		int resno, resource_size_t size, resource_size_t align)
 {
 	struct resource *res = pci_resource_n(dev, resno);
+	unsigned long flags;
 	resource_size_t min;
 	int ret;
 
 	min = (res->flags & IORESOURCE_IO) ? PCIBIOS_MIN_IO : PCIBIOS_MIN_MEM;
 
+	/*
+	 * A 64-bit non-prefetchable BAR may be placed as if it were
+	 * prefetchable (see pci_bus_placement_flags()), in which case the
+	 * upstream bridge window was sized accordingly, so use the same flags
+	 * here.
+	 */
+	flags = pci_bus_placement_flags(dev->bus, res->flags);
+
 	/*
 	 * First, try exact prefetching match.  Even if a 64-bit
 	 * prefetchable bridge window is below 4GB, we can't put a 32-bit
@@ -327,9 +336,9 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 	 * 64-bit window will contain no 32-bit resources.  If we assign
 	 * things differently than they were sized, not everything will fit.
 	 */
-	ret = pci_bus_alloc_resource(bus, res, size, align, min,
-				     IORESOURCE_PREFETCH | IORESOURCE_MEM_64,
-				     pcibios_align_resource, dev);
+	ret = __pci_bus_alloc_resource(bus, res, flags, size, align, min,
+				       IORESOURCE_PREFETCH | IORESOURCE_MEM_64,
+				       pcibios_align_resource, dev);
 	if (ret == 0)
 		return 0;
 
@@ -337,11 +346,11 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 	 * If the prefetchable window is only 32 bits wide, we can put
 	 * 64-bit prefetchable resources in it.
 	 */
-	if ((res->flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64)) ==
+	if ((flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64)) ==
 	     (IORESOURCE_PREFETCH | IORESOURCE_MEM_64)) {
-		ret = pci_bus_alloc_resource(bus, res, size, align, min,
-					     IORESOURCE_PREFETCH,
-					     pcibios_align_resource, dev);
+		ret = __pci_bus_alloc_resource(bus, res, flags, size, align,
+					       min, IORESOURCE_PREFETCH,
+					       pcibios_align_resource, dev);
 		if (ret == 0)
 			return 0;
 	}
@@ -352,9 +361,10 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 	 * non-prefetchable, the first call already tried the only possibility
 	 * so we don't need to try again.
 	 */
-	if (res->flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64))
-		ret = pci_bus_alloc_resource(bus, res, size, align, min, 0,
-					     pcibios_align_resource, dev);
+	if (flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64))
+		ret = __pci_bus_alloc_resource(bus, res, flags, size, align,
+					       min, 0, pcibios_align_resource,
+					       dev);
 
 	return ret;
 }
diff --git a/include/linux/pci.h b/include/linux/pci.h
index d31a8d107b1e..ba22caba51f1 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -279,6 +279,7 @@ enum pci_bus_flags {
 	PCI_BUS_FLAGS_NO_MMRBC	= (__force pci_bus_flags_t) 2,
 	PCI_BUS_FLAGS_NO_AERSID	= (__force pci_bus_flags_t) 4,
 	PCI_BUS_FLAGS_NO_EXTCFG	= (__force pci_bus_flags_t) 8,
+	PCI_BUS_FLAGS_NO_NP_BARS_IN_P_WINDOWS = (__force pci_bus_flags_t) 16,
 };
 
 /* Values from Link Status register, PCIe r3.1, sec 7.8.8 */
-- 
2.56.0.rc1.315.gc6ed9934b7-goog

Re: [PATCH v3 2/2] PCI: Allow 64-bit non-prefetchable BARs in prefetchable windows

[Ilpo Järvinen]

On Wed, 30 Sep 2026, Ard Biesheuvel wrote:

> From: Ard Biesheuvel <ardb@kernel.org>
> 
> The non-prefetchable memory window of a PCI-to-PCI bridge can only
> decode 32-bit addresses, and so non-prefetchable BARs of devices below a
> bridge can only be allocated from the part of the host bridge memory
> space below 4 GB. This is the case even for 64-bit BARs, which could
> easily be placed above 4 GB if there was a bridge window to put them in,
> and on many platforms, 32-bit addressable MMIO space is scarce.
> 
> The PCIe spec addresses this in the implementation note "Additional
> Guidance on the Prefetchable Bit in Memory Space BARs" (PCIe r7.0, sec
> 7.5.1.2.1): on PCIe, setting the Prefetchable bit of a BAR still permits
> correct operation even if the range has read side effects or cannot
> tolerate write merging, as long as the entire path from the host to the
> device is PCIe, given that PCIe Memory Reads always carry an explicit
> length, and PCIe Switches never prefetch or merge writes. The same
> reasoning applies when it is the OS that places a non-prefetchable BAR
> in a prefetchable bridge window: PCIe Root Ports and Switch Ports
> forward requests that hit either window in exactly the same way. Hence,
> the prefetchable window of a PCIe Root Port or Switch Port, which may be
> 64-bit, can serve as a 64-bit window for non-prefetchable BARs too. [0]
> 
> So add pci_bus_placement_flags(), which returns the flags of a resource
> on a given bus with IORESOURCE_PREFETCH set if it is a 64-bit
> non-prefetchable memory resource, the bus is not a root bus, all bridges
> between the bus and the root bus are PCIe Root Ports or PCIe Switch
> Ports, and the host bridge has no prefetchable memory window, but does
> have one that extends above 4 GB in PCI bus address space. [1]
> 
> Evaluate the conditions on the bridges and the host bridge only once
> per bus, when it is added, and record the result in a new bus flag,
> PCI_BUS_FLAGS_NO_NP_BARS_IN_P_WINDOWS: pci_register_host_bridge() sets
> it on the root bus if the host bridge does not qualify, child buses
> inherit it, and pci_alloc_child_bus() sets it on the secondary bus of
> any bridge that is not a PCIe Root Port or Switch Port.
> 
> Use pci_bus_placement_flags() wherever the resource allocator decides
> which bridge window a device resource (including an SR-IOV VF BAR)
> belongs in:
> 
> - in pbus_select_window_for_type(), which is used when sizing bridge
>   windows, and when releasing them to retry failed assignments or to
>   resize a BAR;
> - when allocating the resource in __pci_assign_resource(), by passing
>   its result to __pci_bus_alloc_resource();
> - when claiming a resource assigned by firmware, in
>   pci_find_parent_resource();
> - when deciding which assigned resources to release after a failed
>   assignment, and which failures are relevant to a resized BAR.
> 
> As a result, an eligible 64-bit non-prefetchable BAR is handled exactly
> like a 64-bit prefetchable BAR: it is placed in the prefetchable window
> of the upstream bridge, which may be above 4 GB, and allocation falls
> back to the non-prefetchable window if the prefetchable one has no
> space. [2]
> 
> 32-bit BARs are not affected, and neither are bridge windows, as only
> prefetchable bridge windows can be 64-bit. Devices below conventional
> PCI or CardBus bridges, below PCIe to PCI/PCI-X bridges (in either
> direction), or below host bridges that have a prefetchable window or no
> window above 4 GB are not affected either, and neither are devices on a
> root bus, as there is no prefetchable window for their BARs to go to.
> 
> [0] This reasoning does not extend to the host bridge, though: how it
>     treats the windows that firmware describes as prefetchable is
>     platform specific. For instance, the V3 Semiconductor V360EPC
>     (pci-v3-semi) enables prefetching for its prefetchable window, the
>     MPC52xx uses Memory Read Multiple for it, and Freescale PCI/PCIe
>     host bridges (fsl_pci) enable relaxed ordering for it. However, if
>     the host bridge has no prefetchable windows at all (as appears to be
>     the case on many x86 PCs), all prefetchable bridge windows are
>     carved out of its non-prefetchable windows, and so the host bridge
>     does not treat them any differently.
> 
> [1] Placing non-prefetchable BARs in prefetchable bridge windows only
>     helps if one of those non-prefetchable host bridge windows extends
>     above 4 GB. Otherwise, the prefetchable bridge windows end up below
>     4 GB as well, and BARs would merely move between two bridge windows
>     that are carved out of the same 32-bit space.
> 
> [2] Note that this only concerns where a BAR is placed. How it is mapped
>     is decided by the driver and by the attributes of the BAR itself
>     (e.g., pci_iomap_wc() and the sysfs resource<N>_wc files only honour
>     IORESOURCE_PREFETCH on the BAR), and this change does not modify the
>     flags of any BAR.)
> 
> Assisted-by: LLM
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> ---
> Tested on QEMU arm64 'virt' (DT, all resources assigned by Linux) with
> a qemu-xhci below a Root Port, an NVMe below a Switch, a qemu-xhci
> below a PCIe-to-PCI bridge and another qemu-xhci on the root bus: the
> 64-bit non-prefetchable BARs of the first two (and of the PCIe-to-PCI
> bridge itself) move from the 32-bit non-prefetchable windows into the
> 64-bit prefetchable windows above 4 GB, while the others stay where
> they were. When the 64-bit host bridge window is marked prefetchable
> in the DT, or removed from it, all resources are assigned exactly as
> without this patch. Both drivers work, also after hot removal and
> rescan of the endpoints and of the Switch. Also tested on QEMU x86_64
> q35 with SeaBIOS, which assigns all resources itself: the firmware
> assignment is claimed as before, and after hot removal and rescan, the
> eligible BARs are placed in the prefetchable windows.
> ---
>  drivers/pci/pci.c       | 43 +++++++++++++++++++-
>  drivers/pci/pci.h       |  1 +
>  drivers/pci/probe.c     | 35 ++++++++++++++++
>  drivers/pci/setup-bus.c | 23 ++++++++---
>  drivers/pci/setup-res.c | 30 +++++++++-----
>  include/linux/pci.h     |  1 +
>  6 files changed, 115 insertions(+), 18 deletions(-)
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index b2879a6be5f8..51e96611d734 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -736,6 +736,43 @@ static bool pci_dev_config_accessible(struct pci_dev *dev, char *msg)
>  	return true;
>  }
>  
> +/**
> + * pci_bus_placement_flags - Get the flags to use for placing a resource
> + * @bus: PCI bus of the device that owns the resource
> + * @flags: Resource flags
> + *
> + * The non-prefetchable window of a PCI-to-PCI bridge can only decode 32-bit
> + * addresses, so 64-bit non-prefetchable BARs of devices below a bridge have to
> + * compete for space below 4GB. However, the PCIe spec notes that setting the
> + * Prefetchable bit of a BAR permits correct operation even if the range has
> + * read side effects or cannot tolerate write merging, as long as the entire
> + * path from the host to the device is PCIe: PCIe Memory Reads always carry an
> + * explicit length, and PCIe Switches never prefetch or merge writes (PCIe
> + * r7.0, sec 7.5.1.2.1, Implementation Note "Additional Guidance on the
> + * Prefetchable Bit in Memory Space BARs").
> + *
> + * So if all bridges between @bus and the root bus are PCIe Root Ports or PCIe
> + * Switch Ports, handle 64-bit non-prefetchable resources on @bus like 64-bit
> + * prefetchable ones, so that they can be placed in the prefetchable window of
> + * the bridge above @bus, which may be above 4GB. Other bridges, such as
> + * conventional PCI bridges, may prefetch from their prefetchable window.
> + *
> + * Return: @flags, with IORESOURCE_PREFETCH set if a resource with @flags on
> + * @bus may be placed in a prefetchable bridge window.
> + */
> +unsigned long pci_bus_placement_flags(struct pci_bus *bus, unsigned long flags)
> +{
> +	if ((flags & (IORESOURCE_TYPE_BITS | IORESOURCE_PREFETCH |
> +		      IORESOURCE_MEM_64)) != (IORESOURCE_MEM | IORESOURCE_MEM_64))

PCI_RES_TYPE_MASK

> +		return flags;
> +
> +	if (pci_is_root_bus(bus) ||
> +	    (bus->bus_flags & PCI_BUS_FLAGS_NO_NP_BARS_IN_P_WINDOWS))
> +		return flags;
> +
> +	return flags | IORESOURCE_PREFETCH;
> +}
> +
>  /**
>   * pci_find_parent_resource - return resource region of parent bus of given
>   *			      region
> @@ -749,6 +786,7 @@ struct resource *pci_find_parent_resource(const struct pci_dev *dev,
>  					  struct resource *res)
>  {
>  	const struct pci_bus *bus = dev->bus;
> +	unsigned long flags = pci_bus_placement_flags(dev->bus, res->flags);
>  	struct resource *r;
>  
>  	pci_bus_for_each_resource(bus, r) {
> @@ -758,10 +796,11 @@ struct resource *pci_find_parent_resource(const struct pci_dev *dev,
>  
>  			/*
>  			 * If the window is prefetchable but the BAR is
> -			 * not, the allocator made a mistake.
> +			 * not (and may not be treated as such), the
> +			 * allocator made a mistake.
>  			 */
>  			if (r->flags & IORESOURCE_PREFETCH &&
> -			    !(res->flags & IORESOURCE_PREFETCH))
> +			    !(flags & IORESOURCE_PREFETCH))
>  				return NULL;
>  
>  			/*
> diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
> index 8297cfb5dcd5..989b9c1f1580 100644
> --- a/drivers/pci/pci.h
> +++ b/drivers/pci/pci.h
> @@ -567,6 +567,7 @@ static inline int pci_resource_num(const struct pci_dev *dev,
>  	return resno;
>  }
>  
> +unsigned long pci_bus_placement_flags(struct pci_bus *bus, unsigned long flags);
>  int __pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res,
>  			     unsigned long flags, resource_size_t size,
>  			     resource_size_t align, resource_size_t min,
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 27008e2ea5af..8f00ea3321c0 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -990,6 +990,32 @@ static bool pci_preserve_config(struct pci_host_bridge *host_bridge)
>  	return false;
>  }
>  
> +/*
> + * Return true if all memory windows of @bridge are non-prefetchable, and at
> + * least one of them extends above 4GB in PCI bus address space (see
> + * pci_bus_placement_flags()).
> + */
> +static bool pci_host_np_only_with_high_window(struct pci_host_bridge *bridge)
> +{
> +	struct resource_entry *window;
> +	bool high = false;
> +
> +	resource_list_for_each_entry(window, &bridge->windows) {
> +		struct resource *res = window->res;
> +
> +		if (resource_type(res) != IORESOURCE_MEM)
> +			continue;
> +
> +		if (res->flags & IORESOURCE_PREFETCH)
> +			return false;
> +
> +		if (upper_32_bits(res->end - window->offset))

Should this use pcibios_resource_to_bus() for consistency with the rest of 
the code?

> +			high = true;
> +	}
> +
> +	return high;
> +}
> +
>  static int pci_register_host_bridge(struct pci_host_bridge *bridge)
>  {
>  	struct device *parent = bridge->dev.parent;
> @@ -1137,6 +1163,9 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge)
>  		dev_info(&bus->dev, "root bus resource %pR%s\n", res, addr);
>  	}
>  
> +	if (!pci_host_np_only_with_high_window(bridge))
> +		bus->bus_flags |= PCI_BUS_FLAGS_NO_NP_BARS_IN_P_WINDOWS;

Could this be simplified, e.g. to PCI_BUS_FLAGS_PREF_WIN_ANY_BAR.

> +
>  	of_pci_make_host_bridge_node(bridge);
>  
>  	down_write(&pci_bus_sem);
> @@ -1256,6 +1285,12 @@ static struct pci_bus *pci_alloc_child_bus(struct pci_bus *parent,
>  		pci_info(child, "extended config space not accessible\n");
>  	}
>  
> +	if (!pci_is_pcie(bridge) ||
> +	    (pci_pcie_type(bridge) != PCI_EXP_TYPE_ROOT_PORT &&
> +	     pci_pcie_type(bridge) != PCI_EXP_TYPE_UPSTREAM &&
> +	     pci_pcie_type(bridge) != PCI_EXP_TYPE_DOWNSTREAM))
> +		child->bus_flags |= PCI_BUS_FLAGS_NO_NP_BARS_IN_P_WINDOWS;
> +
>  	/* Set up default resource pointers and names */
>  	for (i = 0; i < PCI_BRIDGE_RESOURCE_NUM; i++) {
>  		child->resource[i] = &bridge->resource[PCI_BRIDGE_RESOURCES+i];
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index e8c94aa1d3c1..64826d8504c7 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -184,7 +184,9 @@ static struct resource *find_bus_resource_of_type(struct pci_bus *bus,
>   *
>   * For memory resources, the selection is done as follows:
>   *
> - * Any non-prefetchable resource is put into the non-prefetchable window.
> + * Any non-prefetchable resource is put into the non-prefetchable window,
> + * except for 64-bit ones that may be placed like prefetchable resources on
> + * @bus (see pci_bus_placement_flags()).
>   *
>   * If there is no prefetchable MMIO window, put all memory resources into the
>   * non-prefetchable window.
> @@ -203,6 +205,7 @@ static struct resource *pbus_select_window_for_type(struct pci_bus *bus,
>  	int iores_type = type & IORESOURCE_TYPE_BITS;	/* w/o 64bit & pref */
>  	struct resource *mmio, *mmio_pref, *win;
>  
> +	type = pci_bus_placement_flags(bus, type);
>  	type &= PCI_RES_TYPE_MASK;			/* with 64bit & pref */
>  
>  	if ((iores_type != IORESOURCE_IO) && (iores_type != IORESOURCE_MEM))
> @@ -261,7 +264,9 @@ static struct resource *pbus_select_window_for_type(struct pci_bus *bus,
>   *
>   * For memory resources, the selection is done as follows:
>   *
> - * Any non-prefetchable resource is put into the non-prefetchable window.
> + * Any non-prefetchable resource is put into the non-prefetchable window,
> + * except for 64-bit ones that may be placed like prefetchable resources on
> + * @bus (see pci_bus_placement_flags()).
>   *
>   * If there is no prefetchable MMIO window, put all memory resources into the
>   * non-prefetchable window.
> @@ -520,8 +525,11 @@ static unsigned long pci_fail_res_type_mask(struct list_head *fail_head)
>  	unsigned long mask = 0;
>  
>  	/* Check failed type */
> -	list_for_each_entry(fail_res, fail_head, list)
> -		mask |= fail_res->flags;
> +	list_for_each_entry(fail_res, fail_head, list) {
> +		struct pci_bus *bus = fail_res->dev->bus;
> +
> +		mask |= pci_bus_placement_flags(bus, fail_res->flags);
> +	}
>  
>  	/*
>  	 * One pref failed resource will set IORESOURCE_MEM, as we can
> @@ -564,8 +572,11 @@ static bool pci_required_resource_failed(struct list_head *fail_head,
>  
>  	list_for_each_entry(fail_res, fail_head, list) {
>  		int idx = pci_resource_num(fail_res->dev, fail_res->res);
> +		unsigned long flags;
>  
> -		if (type && (fail_res->flags & PCI_RES_TYPE_MASK) != type)
> +		flags = pci_bus_placement_flags(fail_res->dev->bus,
> +						fail_res->flags);
> +		if (type && (flags & PCI_RES_TYPE_MASK) != type)
>  			continue;
>  
>  		if (!pci_resource_is_optional(fail_res->dev, idx))
> @@ -2308,7 +2319,7 @@ EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
>  static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *res,
>  					  struct list_head *saved)
>  {
> -	unsigned long type = res->flags;
> +	unsigned long type = pci_bus_placement_flags(bus, res->flags);
>  	struct pci_dev_resource *dev_res;
>  	struct pci_dev *bridge = NULL;
>  	LIST_HEAD(add_list);
> diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
> index 376f09630a4a..78b7df0eee94 100644
> --- a/drivers/pci/setup-res.c
> +++ b/drivers/pci/setup-res.c
> @@ -315,11 +315,20 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
>  		int resno, resource_size_t size, resource_size_t align)
>  {
>  	struct resource *res = pci_resource_n(dev, resno);
> +	unsigned long flags;
>  	resource_size_t min;
>  	int ret;
>  
>  	min = (res->flags & IORESOURCE_IO) ? PCIBIOS_MIN_IO : PCIBIOS_MIN_MEM;
>  
> +	/*
> +	 * A 64-bit non-prefetchable BAR may be placed as if it were
> +	 * prefetchable (see pci_bus_placement_flags()), in which case the
> +	 * upstream bridge window was sized accordingly, so use the same flags
> +	 * here.
> +	 */
> +	flags = pci_bus_placement_flags(dev->bus, res->flags);
> +
>  	/*
>  	 * First, try exact prefetching match.  Even if a 64-bit
>  	 * prefetchable bridge window is below 4GB, we can't put a 32-bit
> @@ -327,9 +336,9 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
>  	 * 64-bit window will contain no 32-bit resources.  If we assign
>  	 * things differently than they were sized, not everything will fit.
>  	 */
> -	ret = pci_bus_alloc_resource(bus, res, size, align, min,
> -				     IORESOURCE_PREFETCH | IORESOURCE_MEM_64,
> -				     pcibios_align_resource, dev);
> +	ret = __pci_bus_alloc_resource(bus, res, flags, size, align, min,
> +				       IORESOURCE_PREFETCH | IORESOURCE_MEM_64,
> +				       pcibios_align_resource, dev);
>  	if (ret == 0)
>  		return 0;
>  
> @@ -337,11 +346,11 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
>  	 * If the prefetchable window is only 32 bits wide, we can put
>  	 * 64-bit prefetchable resources in it.
>  	 */
> -	if ((res->flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64)) ==
> +	if ((flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64)) ==
>  	     (IORESOURCE_PREFETCH | IORESOURCE_MEM_64)) {
> -		ret = pci_bus_alloc_resource(bus, res, size, align, min,
> -					     IORESOURCE_PREFETCH,
> -					     pcibios_align_resource, dev);
> +		ret = __pci_bus_alloc_resource(bus, res, flags, size, align,
> +					       min, IORESOURCE_PREFETCH,
> +					       pcibios_align_resource, dev);
>  		if (ret == 0)
>  			return 0;
>  	}
> @@ -352,9 +361,10 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
>  	 * non-prefetchable, the first call already tried the only possibility
>  	 * so we don't need to try again.
>  	 */
> -	if (res->flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64))
> -		ret = pci_bus_alloc_resource(bus, res, size, align, min, 0,
> -					     pcibios_align_resource, dev);
> +	if (flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64))
> +		ret = __pci_bus_alloc_resource(bus, res, flags, size, align,
> +					       min, 0, pcibios_align_resource,
> +					       dev);
>  
>  	return ret;
>  }
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index d31a8d107b1e..ba22caba51f1 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -279,6 +279,7 @@ enum pci_bus_flags {
>  	PCI_BUS_FLAGS_NO_MMRBC	= (__force pci_bus_flags_t) 2,
>  	PCI_BUS_FLAGS_NO_AERSID	= (__force pci_bus_flags_t) 4,
>  	PCI_BUS_FLAGS_NO_EXTCFG	= (__force pci_bus_flags_t) 8,
> +	PCI_BUS_FLAGS_NO_NP_BARS_IN_P_WINDOWS = (__force pci_bus_flags_t) 16,
>  };
>  
>  /* Values from Link Status register, PCIe r3.1, sec 7.8.8 */
> 

-- 
 i.

Re: [PATCH v3 0/2] PCI: Allow non-prefetchable BARs in prefetchable windows

[Ilpo Järvinen]

[-- Attachment #1: Type: text/plain, Size: 7859 bytes --]

On Wed, 30 Sep 2026, Ard Biesheuvel wrote:

> From: Ard Biesheuvel <ardb@kernel.org>
> 
> (v2 never made it to linux-pci@ - please disregard it if you did get it)
> 
> The non-prefetchable window of a PCI-to-PCI bridge can only decode
> 32-bit addresses, and so even 64-bit non-prefetchable BARs of devices
> below a bridge compete for the scarce MMIO space below 4 GB.
> 
> The PCIe spec notes that marking such a BAR prefetchable still permits
> correct operation as long as the entire path from the host to the device
> is PCIe (PCIe r7.0, sec 7.5.1.2.1). The same applies when the OS places
> a non-prefetchable BAR in a prefetchable bridge window, and so this
> series permits that unless there is a bridge on the path to the root bus
> that is not a PCIe Root Port or Switch Port, allowing 64-bit
> non-prefetchable BARs to be placed above 4 GB.
> 
> So add a new bus flag PCI_BUS_FLAGS_NO_NP_BARS_IN_P_WINDOWS, and wire it
> up so that it is only set on buses where this relaxed placement is not
> permitted. Then, use the flag to to decide whether to treat 64-bit NP
> BARs as prefetchable only for the purpose of resource allocation (but
> not for mapping the MMIO region or other reporting its properties)
> 
> Patch #1 is a preparatory refactor of pci_bus_alloc_resource() with no
> functional change, and patch #2 implements the relaxation.
> 
> (based on v7.3-rc4)
> 
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: "Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>
> Cc: Lorenzo Pieralisi <lpieralisi@kernel.org>
> Link: https://lore.kernel.org/linux-pci/20260910143440.3865663-2-ardb+git@google.com/
> 
> Ard Biesheuvel (2):
>   PCI: Add __pci_bus_alloc_resource() to allocate with explicit flags
>   PCI: Allow 64-bit non-prefetchable BARs in prefetchable windows
> 
>  drivers/pci/bus.c       | 70 ++++++++++++++------
>  drivers/pci/pci.c       | 43 +++++++++++-
>  drivers/pci/pci.h       |  7 ++
>  drivers/pci/probe.c     | 35 ++++++++++
>  drivers/pci/setup-bus.c | 23 +++++--
>  drivers/pci/setup-res.c | 30 ++++++---
>  include/linux/pci.h     |  1 +
>  7 files changed, 169 insertions(+), 40 deletions(-)

Hi,

Nice. I gave this series a spin.

Without pci=realloc, there weren't any changes in iomem (which is probably 
okay result).

With pci=realloc, there seems to be some successful moves from 32-bit -> 
64-bit but something goes wrong with it in some other cases.

I looked into the simplest case with problems. VF BARs only vanished 
from 32-bit window, which I suppose is part of the correct behavior, but 
never reappeared on 64-bit side.

Here's the timestampless diff for convenience:

--- /dev/fd/63	2026-09-30 16:01:19.364780770 +0300
+++ /dev/fd/62	2026-09-30 16:01:19.364780770 +0300
@@ -72,21 +72,45 @@
 pci 0000:d7:00.0: bridge window [mem 0xee800000-0xeebfffff]: releasing
 pci 0000:d8:00.1: ROM [mem size 0x00080000 pref]: resource restored
 pci 0000:d8:00.0: ROM [mem size 0x00080000 pref]: resource restored
-pci 0000:d7:00.0: bridge window [mem size 0x00000000] to [bus d8-d9] add_size 500000 add_align 100000
-pci 0000:d7:00.0: bridge window [mem 0xeed00000-0xef1fffff]: assigned
-pci 0000:d8:00.0: ROM [mem 0xeed00000-0xeed7ffff pref]: assigned
-pci 0000:d8:00.1: ROM [mem 0xeed80000-0xeedfffff pref]: assigned
-pci 0000:d8:00.0: VF BAR 0 [mem 0xeee00000-0xeeefffff 64bit]: assigned
-pci 0000:d8:00.0: VF BAR 3 [mem 0xeef00000-0xeeffffff 64bit]: assigned
-pci 0000:d8:00.1: VF BAR 0 [mem 0xef000000-0xef0fffff 64bit]: assigned
-pci 0000:d8:00.1: VF BAR 3 [mem 0xef100000-0xef1fffff 64bit]: assigned
+pci 0000:d7:00.0: bridge window [mem size 0x00000000] to [bus d8-d9] add_size 100000 add_align 100000
+pci 0000:d7:00.0: bridge window [mem 0xee800000-0xee8fffff]: assigned
+pci 0000:d8:00.0: ROM [mem 0xee800000-0xee87ffff pref]: assigned
+pci 0000:d8:00.1: ROM [mem 0xee880000-0xee8fffff pref]: assigned
+pci 0000:d8:00.0: VF BAR 0 [mem size 0x00100000 64bit]: can't assign; no space
+pci 0000:d8:00.0: VF BAR 0 [mem size 0x00100000 64bit]: failed to assign
+pci 0000:d8:00.0: VF BAR 3 [mem size 0x00100000 64bit]: can't assign; no space
+pci 0000:d8:00.0: VF BAR 3 [mem size 0x00100000 64bit]: failed to assign
+pci 0000:d8:00.1: VF BAR 0 [mem size 0x00100000 64bit]: can't assign; no space
+pci 0000:d8:00.1: VF BAR 0 [mem size 0x00100000 64bit]: failed to assign
+pci 0000:d8:00.1: VF BAR 3 [mem size 0x00100000 64bit]: can't assign; no space
+pci 0000:d8:00.1: VF BAR 3 [mem size 0x00100000 64bit]: failed to assign
+pci 0000:d8:00.0: ROM [mem 0xee800000-0xee87ffff pref]: releasing
+pci 0000:d8:00.0: ROM [mem size 0x00080000 pref]: resource restored
+pci 0000:d8:00.1: ROM [mem 0xee880000-0xee8fffff pref]: releasing
+pci 0000:d8:00.1: ROM [mem size 0x00080000 pref]: resource restored
+pci 0000:d8:00.0: VF BAR 0 [mem 0x00000000-0x000fffff 64bit]: resource restored
+pci 0000:d8:00.0: VF BAR 3 [mem 0x00000000-0x000fffff 64bit]: resource restored
+pci 0000:d8:00.1: VF BAR 0 [mem 0x00000000-0x000fffff 64bit]: resource restored
+pci 0000:d8:00.1: VF BAR 3 [mem 0x00000000-0x000fffff 64bit]: resource restored
+pci 0000:d8:00.1: VF BAR 3 [mem 0x00000000-0x000fffff 64bit]: resource restored
+pci 0000:d8:00.1: VF BAR 0 [mem 0x00000000-0x000fffff 64bit]: resource restored
+pci 0000:d8:00.0: VF BAR 3 [mem 0x00000000-0x000fffff 64bit]: resource restored
+pci 0000:d8:00.0: VF BAR 0 [mem 0x00000000-0x000fffff 64bit]: resource restored
+pci 0000:d8:00.1: ROM [mem size 0x00080000 pref]: resource restored
+pci 0000:d8:00.0: ROM [mem size 0x00080000 pref]: resource restored
+pci 0000:d8:00.1: ROM [mem 0xee800000-0xee87ffff pref]: assigned
+pci 0000:d8:00.0: ROM [mem 0xee880000-0xee8fffff pref]: assigned
+pci 0000:d8:00.0: VF BAR 0 [mem 0x00000000-0x000fffff 64bit]: resetting resource
+pci 0000:d8:00.0: VF BAR 3 [mem 0x00000000-0x000fffff 64bit]: resetting resource
+pci 0000:d8:00.1: VF BAR 0 [mem 0x00000000-0x000fffff 64bit]: resetting resource
+pci 0000:d8:00.1: VF BAR 3 [mem 0x00000000-0x000fffff 64bit]: resetting resource
 pci 0000:d7:00.0: PCI bridge to [bus d8-d9]
-pci 0000:d7:00.0:   bridge window [mem 0xeed00000-0xef1fffff]
+pci 0000:d7:00.0:   bridge window [mem 0xee800000-0xee8fffff]
 pci 0000:d7:00.0:   bridge window [mem 0x207fff400000-0x207fffcfffff 64bit pref]
 pci_bus 0000:d7: resource 4 [io  0xf000-0xffff window]
 pci_bus 0000:d7: resource 5 [mem 0xee800000-0xfbffffff window]
 pci_bus 0000:d7: resource 6 [mem 0x207000000000-0x207fffffffff window]
-pci_bus 0000:d8: resource 1 [mem 0xeed00000-0xef1fffff]
+pci_bus 0000:d8: resource 1 [mem 0xee800000-0xee8fffff]
 pci_bus 0000:d8: resource 2 [mem 0x207fff400000-0x207fffcfffff 64bit pref]
 pci 0000:d7:05.0: disabled boot interrupts on device [8086:2034]
 pci 0000:d8:00.0: CLS mismatch (128 != 32), using 64 bytes

The problems could be related to pci=realloc not touching already assigned 
bridge windows in many cases. pbus_size_mem() has resource_assigned() 
check early so it won't size those bridge windows again. I've some 
unfinished patch to remove that restriction but have been busy solving the 
recent set of regressions.

Because VF BARs are optional resources, their failures are considered 
acceptable so together with not trying to resize those windows, no 
failures are detected by the fitting algorithm.

There are a few cases that are more messy than the one above. I'll try to 
later look if they are fine or not. Annoyingly VF BARs are left with the 
original base address when they fail, so they appear in lspci with what 
looks a valid address but the corresponding resource is not in the 
resource tree so it takes time to decipher if everything is properly 
assigned or not (when things move around in iomem diffing it won't 
easily tell it either).


-- 
 i.

[-- Attachment #2: Type: application/vnd.lotus-freelance, Size: 7611 bytes --]

[-- Attachment #3: Type: text/plain, Size: 9914 bytes --]

[    9.677904] PCI host bridge to bus 0000:d7
[    9.677915] pci_bus 0000:d7: root bus resource [io  0xf000-0xffff window]
[    9.677918] pci_bus 0000:d7: root bus resource [mem 0xee800000-0xfbffffff window]
[    9.677921] pci_bus 0000:d7: root bus resource [mem 0x207000000000-0x207fffffffff window]
[    9.677923] pci_bus 0000:d7: root bus resource [bus d7-ff]
[    9.677927] pci_bus 0000:d7: scanning bus
[    9.677965] pci 0000:d7:00.0: [8086:203f] type 01 class 0x060400 PCIe Root Port
[    9.677986] pci 0000:d7:00.0: PCI bridge to [bus d8-d9]
[    9.677991] pci 0000:d7:00.0:   bridge window [mem 0xee800000-0xeebfffff]
[    9.678000] pci 0000:d7:00.0:   bridge window [mem 0x207fff400000-0x207fffcfffff 64bit pref]
[    9.678008] pci 0000:d7:00.0: enabling Extended Tags
[    9.678074] pci 0000:d7:00.0: PME# supported from D0 D3hot D3cold
[    9.678078] pci 0000:d7:00.0: PME# disabled
[    9.679704] pci 0000:d7:05.0: [8086:2034] type 00 class 0x088000 PCIe Root Complex Integrated Endpoint
[    9.680113] pci 0000:d7:05.2: [8086:2035] type 00 class 0x088000 PCIe Root Complex Integrated Endpoint
[    9.680514] pci 0000:d7:05.4: [8086:2036] type 00 class 0x080020 PCIe Root Complex Integrated Endpoint
[    9.680543] pci 0000:d7:05.4: BAR 0 [mem 0xeec80000-0xeec80fff]
[    9.680951] pci_bus 0000:d7: fixups for bus
[    9.680954] pci 0000:d7:00.0: scanning [bus d8-d9] behind bridge, pass 0
[    9.681275] pci_bus 0000:d8: dev 00, created physical slot 9
[    9.681755] acpiphp_glue: found ACPI PCI Hotplug slot 9 at PCI 0000:d8:00
[    9.681914] pci_bus 0000:d8: scanning bus
[    9.681970] pci 0000:d8:00.0: [8086:1563] type 00 class 0x020000 PCIe Endpoint
[    9.682014] pci 0000:d8:00.0: BAR 0 [mem 0x207fff800000-0x207fffbfffff 64bit pref]
[    9.682020] pci 0000:d8:00.0: BAR 4 [mem 0x207fffc04000-0x207fffc07fff 64bit pref]
[    9.682023] pci 0000:d8:00.0: ROM [mem 0xfff80000-0xffffffff pref]
[    9.682138] pci 0000:d8:00.0: PME# supported from D0 D3hot
[    9.682142] pci 0000:d8:00.0: PME# disabled
[    9.682188] pci 0000:d8:00.0: VF BAR 0 [mem 0xee800000-0xee803fff 64bit]
[    9.682191] pci 0000:d8:00.0: VF BAR 0 [mem 0xee800000-0xee8fffff 64bit]: contains BAR 0 for 64 VFs
[    9.682201] pci 0000:d8:00.0: VF BAR 3 [mem 0xee900000-0xee903fff 64bit]
[    9.682202] pci 0000:d8:00.0: VF BAR 3 [mem 0xee900000-0xee9fffff 64bit]: contains BAR 3 for 64 VFs
[    9.683220] pci 0000:d8:00.1: [8086:1563] type 00 class 0x020000 PCIe Endpoint
[    9.683268] pci 0000:d8:00.1: BAR 0 [mem 0x207fff400000-0x207fff7fffff 64bit pref]
[    9.683274] pci 0000:d8:00.1: BAR 4 [mem 0x207fffc00000-0x207fffc03fff 64bit pref]
[    9.683277] pci 0000:d8:00.1: ROM [mem 0xfff80000-0xffffffff pref]
[    9.683382] pci 0000:d8:00.1: PME# supported from D0 D3hot
[    9.683386] pci 0000:d8:00.1: PME# disabled
[    9.683427] pci 0000:d8:00.1: VF BAR 0 [mem 0xeea00000-0xeea03fff 64bit]
[    9.683429] pci 0000:d8:00.1: VF BAR 0 [mem 0xeea00000-0xeeafffff 64bit]: contains BAR 0 for 64 VFs
[    9.683434] pci 0000:d8:00.1: VF BAR 3 [mem 0xeeb00000-0xeeb03fff 64bit]
[    9.683436] pci 0000:d8:00.1: VF BAR 3 [mem 0xeeb00000-0xeebfffff 64bit]: contains BAR 3 for 64 VFs
[    9.684452] pci_bus 0000:d8: fixups for bus
[    9.684453] pci 0000:d7:00.0: PCI bridge to [bus d8-d9]
[    9.684462] pci_bus 0000:d8: bus scan returning with max=d8
[    9.684467] pci 0000:d7:00.0: scanning [bus d8-d9] behind bridge, pass 1
[    9.684472] pci_bus 0000:d7: bus scan returning with max=d9
[    9.684473] pci_bus 0000:d7: on NUMA node 1
[    9.713581] pci 0000:d7:00.0: bridge window [mem 0xee800000-0xeebfffff]: claiming
[    9.713582] pci 0000:d7:00.0: bridge window [mem 0x207fff400000-0x207fffcfffff 64bit pref]: claiming
[    9.713890] pci 0000:d8:00.0: BAR 0 [mem 0x207fff800000-0x207fffbfffff 64bit pref]: claiming
[    9.713892] pci 0000:d8:00.0: BAR 4 [mem 0x207fffc04000-0x207fffc07fff 64bit pref]: claiming
[    9.713894] pci 0000:d8:00.0: VF BAR 0 [mem 0xee800000-0xee8fffff 64bit]: claiming
[    9.713896] pci 0000:d8:00.0: VF BAR 3 [mem 0xee900000-0xee9fffff 64bit]: claiming
[    9.713903] pci 0000:d8:00.1: BAR 0 [mem 0x207fff400000-0x207fff7fffff 64bit pref]: claiming
[    9.713904] pci 0000:d8:00.1: BAR 4 [mem 0x207fffc00000-0x207fffc03fff 64bit pref]: claiming
[    9.713906] pci 0000:d8:00.1: VF BAR 0 [mem 0xeea00000-0xeeafffff 64bit]: claiming
[    9.713908] pci 0000:d8:00.1: VF BAR 3 [mem 0xeeb00000-0xeebfffff 64bit]: claiming
[    9.713913] pci 0000:d7:05.4: BAR 0 [mem 0xeec80000-0xeec80fff]: claiming
[    9.872901] pci 0000:d8:00.0: ROM [mem 0xfff80000-0xffffffff pref]: can't claim; no compatible bridge window
[    9.872905] pci 0000:d8:00.1: ROM [mem 0xfff80000-0xffffffff pref]: can't claim; no compatible bridge window
[    9.874174] pci_bus 0000:d7: max bus depth: 1 pci_try_num: 2
[    9.874181] pci 0000:d8:00.0: ROM [mem size 0x00080000 pref]: can't assign; no space
[    9.874198] pci 0000:d8:00.0: ROM [mem size 0x00080000 pref]: failed to assign
[    9.874200] pci 0000:d8:00.1: ROM [mem size 0x00080000 pref]: can't assign; no space
[    9.874202] pci 0000:d8:00.1: ROM [mem size 0x00080000 pref]: failed to assign
[    9.874206] pci 0000:d8:00.0: ROM [mem size 0x00080000 pref]: resetting resource
[    9.874209] pci 0000:d8:00.1: ROM [mem size 0x00080000 pref]: resetting resource
[    9.874212] pci 0000:d7:00.0: PCI bridge to [bus d8-d9]
[    9.874216] pci 0000:d7:00.0:   bridge window [mem 0xee800000-0xeebfffff]
[    9.874219] pci 0000:d7:00.0:   bridge window [mem 0x207fff400000-0x207fffcfffff 64bit pref]
[    9.874232] pci 0000:d7:00.0: bridge window [mem 0xee800000-0xeebfffff]: releasing
[    9.874235] pci 0000:d8:00.1: ROM [mem size 0x00080000 pref]: resource restored
[    9.874236] pci 0000:d8:00.0: ROM [mem size 0x00080000 pref]: resource restored
[    9.874241] pci 0000:d7:00.0: bridge window [mem size 0x00000000] to [bus d8-d9] add_size 100000 add_align 100000
[    9.874248] pci 0000:d7:00.0: bridge window [mem 0xee800000-0xee8fffff]: assigned
[    9.874259] pci 0000:d8:00.0: ROM [mem 0xee800000-0xee87ffff pref]: assigned
[    9.874261] pci 0000:d8:00.1: ROM [mem 0xee880000-0xee8fffff pref]: assigned
[    9.874266] pci 0000:d8:00.0: VF BAR 0 [mem size 0x00100000 64bit]: can't assign; no space
[    9.874268] pci 0000:d8:00.0: VF BAR 0 [mem size 0x00100000 64bit]: failed to assign
[    9.874273] pci 0000:d8:00.0: VF BAR 3 [mem size 0x00100000 64bit]: can't assign; no space
[    9.874274] pci 0000:d8:00.0: VF BAR 3 [mem size 0x00100000 64bit]: failed to assign
[    9.874279] pci 0000:d8:00.1: VF BAR 0 [mem size 0x00100000 64bit]: can't assign; no space
[    9.874280] pci 0000:d8:00.1: VF BAR 0 [mem size 0x00100000 64bit]: failed to assign
[    9.874284] pci 0000:d8:00.1: VF BAR 3 [mem size 0x00100000 64bit]: can't assign; no space
[    9.874286] pci 0000:d8:00.1: VF BAR 3 [mem size 0x00100000 64bit]: failed to assign
[    9.874294] pci 0000:d8:00.0: ROM [mem 0xee800000-0xee87ffff pref]: releasing
[    9.874296] pci 0000:d8:00.0: ROM [mem size 0x00080000 pref]: resource restored
[    9.874297] pci 0000:d8:00.1: ROM [mem 0xee880000-0xee8fffff pref]: releasing
[    9.874299] pci 0000:d8:00.1: ROM [mem size 0x00080000 pref]: resource restored
[    9.874301] pci 0000:d8:00.0: VF BAR 0 [mem 0x00000000-0x000fffff 64bit]: resource restored
[    9.874302] pci 0000:d8:00.0: VF BAR 3 [mem 0x00000000-0x000fffff 64bit]: resource restored
[    9.874304] pci 0000:d8:00.1: VF BAR 0 [mem 0x00000000-0x000fffff 64bit]: resource restored
[    9.874305] pci 0000:d8:00.1: VF BAR 3 [mem 0x00000000-0x000fffff 64bit]: resource restored
[    9.874307] pci 0000:d8:00.1: VF BAR 3 [mem 0x00000000-0x000fffff 64bit]: resource restored
[    9.874308] pci 0000:d8:00.1: VF BAR 0 [mem 0x00000000-0x000fffff 64bit]: resource restored
[    9.874310] pci 0000:d8:00.0: VF BAR 3 [mem 0x00000000-0x000fffff 64bit]: resource restored
[    9.874311] pci 0000:d8:00.0: VF BAR 0 [mem 0x00000000-0x000fffff 64bit]: resource restored
[    9.874313] pci 0000:d8:00.1: ROM [mem size 0x00080000 pref]: resource restored
[    9.874315] pci 0000:d8:00.0: ROM [mem size 0x00080000 pref]: resource restored
[    9.874318] pci 0000:d8:00.1: ROM [mem 0xee800000-0xee87ffff pref]: assigned
[    9.874320] pci 0000:d8:00.0: ROM [mem 0xee880000-0xee8fffff pref]: assigned
[    9.874323] pci 0000:d8:00.0: VF BAR 0 [mem 0x00000000-0x000fffff 64bit]: resetting resource
[    9.874325] pci 0000:d8:00.0: VF BAR 3 [mem 0x00000000-0x000fffff 64bit]: resetting resource
[    9.874327] pci 0000:d8:00.1: VF BAR 0 [mem 0x00000000-0x000fffff 64bit]: resetting resource
[    9.874329] pci 0000:d8:00.1: VF BAR 3 [mem 0x00000000-0x000fffff 64bit]: resetting resource
[    9.874332] pci 0000:d7:00.0: PCI bridge to [bus d8-d9]
[    9.874336] pci 0000:d7:00.0:   bridge window [mem 0xee800000-0xee8fffff]
[    9.874339] pci 0000:d7:00.0:   bridge window [mem 0x207fff400000-0x207fffcfffff 64bit pref]
[    9.874344] pci_bus 0000:d7: resource 4 [io  0xf000-0xffff window]
[    9.874346] pci_bus 0000:d7: resource 5 [mem 0xee800000-0xfbffffff window]
[    9.874348] pci_bus 0000:d7: resource 6 [mem 0x207000000000-0x207fffffffff window]
[    9.874350] pci_bus 0000:d8: resource 1 [mem 0xee800000-0xee8fffff]
[    9.874352] pci_bus 0000:d8: resource 2 [mem 0x207fff400000-0x207fffcfffff 64bit pref]
[    9.876327] pci 0000:d7:05.0: disabled boot interrupts on device [8086:2034]
[    9.876341] pci 0000:d8:00.0: CLS mismatch (128 != 32), using 64 bytes
[   10.076194] pcieport 0000:d7:00.0: vgaarb: pci_notify
[   10.076212] pcieport 0000:d7:00.0: runtime IRQ mapping not provided by arch
[   10.077347] pcieport 0000:d7:00.0: PME: Signaling with IRQ 27
[   10.077620] pcieport 0000:d7:00.0: bwctrl: enabled with IRQ 27
[   10.077965] pcieport 0000:d7:00.0: vgaarb: pci_notify

[-- Attachment #4: Type: application/vnd.lotus-freelance, Size: 4985 bytes --]

[-- Attachment #5: Type: text/plain, Size: 4969 bytes --]

00000000-00000fff : Reserved
00001000-0009dfff : System RAM
0009e000-0009efff : Reserved
0009f000-0009ffff : ACPI Non-volatile Storage
000a0000-000fffff : Reserved
  000a0000-000bffff : PCI Bus 0000:00
  000c4000-000c7fff : PCI Bus 0000:00
  000f0000-000fffff : System ROM
00100000-62c88fff : System RAM
62c89000-64488fff : Reserved
  63452018-63452067 : APEI ERST
  63452070-63452077 : APEI ERST
  63452078-63454017 : APEI ERST
64489000-65aacfff : ACPI Non-volatile Storage
65aad000-663fdfff : ACPI Tables
663fe000-6f7fffff : System RAM
6f800000-7fffffff : Reserved
80000000-8fffffff : PCI ECAM 0000 [bus 00-ff]
90000000-9d7fffff : PCI Bus 0000:00
  91000000-920fffff : PCI Bus 0000:02
    91000000-920fffff : PCI Bus 0000:03
      91000000-91ffffff : 0000:03:00.0
        91000000-912fffff : efifb
      92000000-9201ffff : 0000:03:00.0
  92100000-921fffff : PCI Bus 0000:01
    92100000-9217ffff : 0000:01:00.0
      92100000-9217ffff : igb
    92180000-92183fff : 0000:01:00.0
      92180000-92183fff : igb
  92200000-9227ffff : 0000:00:17.0
    92200000-9227ffff : ahci
  92280000-922fffff : 0000:00:11.5
    92280000-922fffff : ahci
  92300000-92303fff : 0000:00:1f.2
  92304000-92305fff : 0000:00:17.0
    92304000-92305fff : ahci
  92306000-92307fff : 0000:00:11.5
    92306000-92307fff : ahci
  92309000-923090ff : 0000:00:17.0
    92309000-923090ff : ahci
  9230a000-9230a0ff : 0000:00:11.5
    9230a000-9230a0ff : ahci
  9230b000-9230bfff : 0000:00:05.4
9d800000-aaffffff : PCI Bus 0000:16
  9d800000-9d800fff : 0000:16:05.4
ab000000-b87fffff : PCI Bus 0000:39
  ab000000-ab000fff : 0000:39:05.4
b8800000-c5ffffff : PCI Bus 0000:5d
  b8800000-b8800fff : 0000:5d:05.4
c6000000-d37fffff : PCI Bus 0000:80
  c6000000-c6000fff : 0000:80:05.4
d3800000-e0ffffff : PCI Bus 0000:84
  d3800000-d3800fff : 0000:84:05.4
e1000000-ee7fffff : PCI Bus 0000:ad
  e1000000-e1000fff : 0000:ad:05.4
ee800000-fbffffff : PCI Bus 0000:d7
  ee800000-ee8fffff : PCI Bus 0000:d8
    ee800000-ee87ffff : 0000:d8:00.1
    ee880000-ee8fffff : 0000:d8:00.0
  eec80000-eec80fff : 0000:d7:05.4
fd000000-fdabffff : PNP0C02:01
fdad0000-fdadffff : PNP0C02:01
fdb00000-fdffffff : PNP0C02:01
  fdc6000c-fdc6000f : iTCO_wdt
fe000000-fe010fff : Reserved
  fe000000-fe00ffff : PNP0C02:01
  fe010000-fe010fff : PCI Bus 0000:00
    fe010000-fe010fff : 0000:00:1f.5
fe011000-fe01ffff : PNP0C02:01
fe036000-fe03bfff : PNP0C02:01
fe03d000-fe3fffff : PNP0C02:01
fe410000-fe7fffff : PNP0C02:01
fec00000-fecfffff : PNP0003:00
  fec00000-fec003ff : IOAPIC 0
  fec01000-fec013ff : IOAPIC 1
  fec08000-fec083ff : IOAPIC 2
  fec10000-fec103ff : IOAPIC 3
  fec18000-fec183ff : IOAPIC 4
  fec20000-fec203ff : IOAPIC 5
  fec28000-fec283ff : IOAPIC 6
  fec30000-fec303ff : IOAPIC 7
  fec38000-fec383ff : IOAPIC 8
fed00000-fed003ff : HPET 0
  fed00000-fed003ff : PNP0103:00
fed40000-fed47fff : PNP0C02:00
ff000000-ffffffff : PNP0C02:00
100000000-87fffffff : System RAM
  6e2000000-6e325aef7 : Kernel code
  6e3400000-6e4117fff : Kernel rodata
  6e4200000-6e45e5c3f : Kernel data
  6e4f6b000-6e5c25fff : Kernel bss
200000000000-200fffffffff : PCI Bus 0000:00
  200ffff00000-200ffff0ffff : 0000:00:14.0
    200ffff00000-200ffff0ffff : xhci-hcd
  200ffff10000-200ffff13fff : 0000:00:04.7
  200ffff14000-200ffff17fff : 0000:00:04.6
  200ffff18000-200ffff1bfff : 0000:00:04.5
  200ffff1c000-200ffff1ffff : 0000:00:04.4
  200ffff20000-200ffff23fff : 0000:00:04.3
  200ffff24000-200ffff27fff : 0000:00:04.2
  200ffff28000-200ffff2bfff : 0000:00:04.1
  200ffff2c000-200ffff2ffff : 0000:00:04.0
  200ffff30000-200ffff300ff : 0000:00:1f.4
    200ffff30000-200ffff300ff : i801_smbus
  200ffff31000-200ffff31fff : 0000:00:16.4
  200ffff32000-200ffff32fff : 0000:00:16.1
  200ffff33000-200ffff33fff : 0000:00:16.0
  200ffff34000-200ffff34fff : 0000:00:14.2
201000000000-201fffffffff : PCI Bus 0000:16
202000000000-202fffffffff : PCI Bus 0000:39
203000000000-203fffffffff : PCI Bus 0000:5d
204000000000-204fffffffff : PCI Bus 0000:80
  204ffff00000-204ffff03fff : 0000:80:04.7
  204ffff04000-204ffff07fff : 0000:80:04.6
  204ffff08000-204ffff0bfff : 0000:80:04.5
  204ffff0c000-204ffff0ffff : 0000:80:04.4
  204ffff10000-204ffff13fff : 0000:80:04.3
  204ffff14000-204ffff17fff : 0000:80:04.2
  204ffff18000-204ffff1bfff : 0000:80:04.1
  204ffff1c000-204ffff1ffff : 0000:80:04.0
205000000000-205fffffffff : PCI Bus 0000:84
206000000000-206fffffffff : PCI Bus 0000:ad
207000000000-207fffffffff : PCI Bus 0000:d7
  207fff400000-207fffcfffff : PCI Bus 0000:d8
    207fff400000-207fff7fffff : 0000:d8:00.1
      207fff400000-207fff7fffff : ixgbe
    207fff800000-207fffbfffff : 0000:d8:00.0
      207fff800000-207fffbfffff : ixgbe
    207fffc00000-207fffc03fff : 0000:d8:00.1
      207fffc00000-207fffc03fff : ixgbe
    207fffc04000-207fffc07fff : 0000:d8:00.0
      207fffc04000-207fffc07fff : ixgbe