From: "Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>
To: linux-pci@vger.kernel.org, "Bjorn Helgaas" <bhelgaas@google.com>,
"Lorenzo Pieralisi" <lorenzo.pieralisi@arm.com>,
"Rob Herring" <robh@kernel.org>,
"Krzysztof Wilczyński" <kw@linux.com>,
"Rafael J. Wysocki" <rafael@kernel.org>,
"Len Brown" <lenb@kernel.org>,
"Christian König" <christian.koenig@amd.com>,
"Alex Williamson" <alex@shazbot.org>,
linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: "Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>, stable@vger.kernel.org
Subject: [PATCH 05/13] PCI: Serialize bridge window sizing and resource assignment
Date: Mon, 16 Nov 2026 17:23:50 +0200 [thread overview]
Message-ID: <20261116152403.2672-6-ilpo.jarvinen@linux.intel.com> (raw)
In-Reply-To: <20261116152403.2672-1-ilpo.jarvinen@linux.intel.com>
Bridge window sizing and resource assignment are started from multiple
entry points: rescan and hotplug under pci_rescan_remove_lock, and BAR
resize from sysfs or driver probe under device_lock() only. The entry
points hold pci_bus_sem only for read, so nothing prevents two sizing
and assignment passes from running concurrently on the same bridge
windows and BARs. E.g., a rescan may assign a BAR that a concurrent BAR
resize has released, or restore a resource the resize has already
assigned.
Looping BAR resize through sysfs against a rescan of the parent bus on a
system with a BMG GPU triggers the add_list sanity check. The rescan
sizes the Downstream Port's prefetchable window while the resize assigns
it, and the sizing values overwrite the assigned window. The resize then
programs these values into the Downstream Port, outside the root bus
resources and overlapping System RAM [mem 0x100000000-0x4075ffffff]. The
rescan skips assigning the window as it looks assigned, which leaves its
add_list entry unused:
pci_bus 0000:b7: root bus resource [mem 0xd6000000-0xddfbffff window]
pci_bus 0000:b7: root bus resource [mem 0x2e0000000000-0x2effffffffff window]
pcieport 0000:b9:01.0: bridge window [mem 0x800000000-0x1000ffffff 64bit pref] to [bus ba] add_size 4000000 add_align 800000000
pcieport 0000:b9:01.0: PCI bridge to [bus ba]
pcieport 0000:b9:01.0: bridge window [mem 0xdda00000-0xddbfffff]
pcieport 0000:b9:01.0: bridge window [mem 0x800000000-0x1000ffffff 64bit pref]
add_list && !list_empty(add_list)
WARNING: drivers/pci/setup-bus.c:1546 at pci_assign_unassigned_bus_resources+0x9e/0x130, CPU#58: srf8-resize-rac/2861
CPU: 58 UID: 0 PID: 2861 Comm: srf8-resize-rac Tainted: G S 7.4.0-rc1-g6628e7d4c6b2 #59 PREEMPT(full)
RIP: 0010:pci_assign_unassigned_bus_resources+0x9e/0x130
Call Trace:
pci_rescan_bus+0x20/0x40
dev_rescan_store+0x6e/0x90
Other trigger vectors are possible as well. Hotplug takes the same locks
as the sysfs rescan. BAR resize walks the bridge windows up to the Root
Port to make room for the resized BAR, and then sizes and assigns the
whole hierarchy below the Root Port. Thus, hotplug anywhere below the
same Root Port can race with the resize, e.g., when connecting a
Thunderbolt device. Instead of the sysfs resize, a driver can resize a
BAR during probe, e.g., after unbind and bind.
pci_rescan_remove_lock cannot be taken under device_lock() because the
device removal path takes them in the opposite order.
Add pci_res_setup_lock and hold it across sizing and assignment in all
entry points, including acpiphp which calls the internal sizing and
assignment functions directly. Assert the lock is held in the
functions that modify resources deeper in the call chains. In
pci_do_resource_release_and_resize(), take the lock before selecting the
bridge window and setting the BAR size as both depend on or change
resource state.
The race was also reproduced in QEMU, which found more symptoms.
Looping BAR resize through sysfs against a rescan of the parent bus on
a guest with 512M of RAM, using a synthetic host bridge with a Root Port
and an Endpoint with a Resizable BAR, triggers the following failures:
1) Both the rescan and the resize release the Root Port's prefetchable
window. Both find it assigned, but the rescan releases it first, and
the resize then dereferences the NULL parent in __release_resource():
pcieport 10000:00:00.0: bridge window [mem 0x940000000-0x97fffffff 64bit pref]: releasing
pcieport 10000:00:00.0: bridge window [mem size 0x40000000 64bit pref]: releasing
BUG: kernel NULL pointer dereference, address: 0000000000000038
Oops: Oops: 0000 [#1] SMP
CPU: 0 UID: 0 PID: 49 Comm: init Not tainted 7.4.0-rc1-00001-g4aa3acaaeb08 #1 PREEMPTLAZY
RIP: 0010:__release_resource+0x8/0x70
Call Trace:
release_resource+0x22/0x40
pci_release_resource+0x54/0x90
pci_do_resource_release_and_resize+0x1f3/0x3d0
__resource_resize_store+0xf0/0x180
2) The rescan restores a resource that has already been assigned:
WARNING: drivers/pci/setup-bus.c:136 at pci_dev_res_restore+0x32/0x90, CPU#0: init/1
CPU: 0 UID: 0 PID: 1 Comm: init Not tainted 7.4.0-rc1-dirty #6 PREEMPTLAZY
RIP: 0010:pci_dev_res_restore+0x32/0x90
Call Trace:
__assign_resources_sorted+0x2db/0x570
__pci_bus_assign_resources_one+0x57/0x190
__pci_bus_assign_resources_one+0x12d/0x190
__pci_bus_assign_resources+0xd/0x30
pci_assign_unassigned_bus_resources+0x6f/0x80
pci_rescan_bus+0x1a/0x30
dev_rescan_store+0x46/0x60
3) The rescan assigns the BAR while the resize is assigning it, failing
the resize:
pci 10000:01:00.0: BAR 0 [mem 0x900000000-0x90fffffff 64bit pref]: releasing
pcieport 10000:00:00.0: bridge window [mem 0x900000000-0x90fffffff 64bit pref]: releasing
pci 10000:01:00.0: BAR 0 [mem size 0x40000000 64bit pref]: can't assign; no space
pcieport 10000:00:00.0: bridge window [mem 0x900000000-0x93fffffff 64bit pref]: assigned
pci 10000:01:00.0: BAR 0 [mem size 0x40000000 64bit pref]: failed to assign
pci 10000:01:00.0: BAR 0 [mem 0x900000000-0x93fffffff 64bit pref]: releasing
pci 10000:01:00.0: BAR 0 [mem 0x900000000-0x93fffffff 64bit pref]: assigned
pci 10000:01:00.0: BAR 0 [mem size 0x40000000 64bit pref]: resource restored
pci 10000:01:00.0: BAR 0 [mem 0x900000000-0x90fffffff 64bit pref]: old value restored
pci 10000:01:00.0: Failed to resize BAR 0: -28
4) While one pass is sorting a released bridge window for assignment,
the other pass assigns it, which clears IORESOURCE_STARTALIGN. The
window then has zero alignment:
pci 10000:01:00.0: BAR 0 [mem 0x900000000-0x93fffffff 64bit pref]: releasing
pcieport 10000:00:00.0: bridge window [mem 0x900000000-0x93fffffff 64bit pref]: releasing
pcieport 10000:00:00.0: bridge window [mem 0x900000000-0x90fffffff 64bit pref]: assigned
pcieport 10000:00:00.0: bridge window [mem 0x900000000-0x90fffffff 64bit pref]: alignment must not be zero
pci 10000:01:00.0: BAR 0 [mem 0x900000000-0x90fffffff 64bit pref]: assigned
5) BAR 0 loses its flags and the Root Port's prefetchable window gets
programmed over RAM, after which every BAR resize fails with -EINVAL:
pci 10000:01:00.0: BAR 0 [mem size 0x10000000 64bit pref]: can't assign; no space
resource: remaining [mem 0x000000001ffdf000-0x000000001fffffff] available
resource: avoiding allocation from e820 entry [mem 0x1ffdf000-0x1fffffff]
pci 10000:01:00.0: BAR 0 [mem size 0x10000000 64bit pref]: failed to assign
pci 10000:01:00.0: BAR 0 [mem size 0x10000000 64bit pref]: can't assign; no space
pci 10000:01:00.0: BAR 0 [mem size 0x10000000 64bit pref]: resetting resource
pci 10000:01:00.0: BAR 0 [??? 0x00000000 flags 0x0]: failed to assign
pci 10000:01:00.0: BAR 0 [??? 0x00000000 flags 0x0]: resetting resource
pcieport 10000:00:00.0: PCI bridge to [bus 01]
pcieport 10000:00:00.0: bridge window [mem 0x20000000-0x200fffff]
pcieport 10000:00:00.0: bridge window [mem 0x10000000-0x1fffffff 64bit pref]
pci 10000:01:00.0: Failed to resize BAR 0: -22
Fixes: 8bb705e3e79d ("PCI: Add pci_resize_resource() for resizing BARs")
Fixes: 91fa127794ac ("PCI: Expose PCIe Resizable BAR support via sysfs")
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Cc: stable@vger.kernel.org
---
Bjorn,
It's up to you if you want to keep all those qemu reproduced cases in the
changelog. You usually seem to prefer the log messages for easier googling
in case somebody happens to hit the issue so I added them all.
I couldn't get the rest to trigger on a real host but that could be due to
serial output that was too slow to keep up with the test loops which might
have affected timing.
drivers/pci/hotplug/acpiphp_glue.c | 1 +
drivers/pci/pci.h | 1 +
drivers/pci/setup-bus.c | 40 ++++++++++++++++++++++++++++++
drivers/pci/setup-cardbus.c | 3 +++
4 files changed, 45 insertions(+)
diff --git a/drivers/pci/hotplug/acpiphp_glue.c b/drivers/pci/hotplug/acpiphp_glue.c
index 632344f34cae..d3987be044e3 100644
--- a/drivers/pci/hotplug/acpiphp_glue.c
+++ b/drivers/pci/hotplug/acpiphp_glue.c
@@ -502,6 +502,7 @@ static void enable_slot(struct acpiphp_slot *slot, bool bridge)
acpiphp_rescan_slot(slot);
max = acpiphp_max_busnr(bus);
+ guard(mutex)(&pci_res_setup_lock);
for (pass = 0; pass < 2; pass++) {
for_each_pci_bridge(dev, bus) {
if (PCI_SLOT(dev->devfn) != slot->device)
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 77efa4c5c960..660a83b0045a 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -124,6 +124,7 @@ unsigned char pcie_get_link_speed(unsigned int speed);
extern bool pci_early_dump;
extern struct mutex pci_rescan_remove_lock;
+extern struct mutex pci_res_setup_lock;
bool pcie_cap_has_lnkctl(const struct pci_dev *dev);
bool pcie_cap_has_lnkctl2(const struct pci_dev *dev);
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 5b2d870ca13d..314c649c648f 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -26,6 +26,7 @@
#include <linux/ioport.h>
#include <linux/cache.h>
#include <linux/limits.h>
+#include <linux/lockdep.h>
#include <linux/sizes.h>
#include <linux/slab.h>
#include <linux/acpi.h>
@@ -38,6 +39,17 @@
unsigned int pci_flags;
EXPORT_SYMBOL_GPL(pci_flags);
+/*
+ * Serializes bridge window sizing, resource fitting, and assignment.
+ * pci_rescan_remove_lock cannot be used for this because BAR resize and
+ * some drivers run these operations under device_lock() during probe.
+ *
+ * Context:
+ * - Nests inside pci_rescan_remove_lock and device_lock().
+ * - Nests outside pci_bus_sem.
+ */
+DEFINE_MUTEX(pci_res_setup_lock);
+
struct pci_dev_resource {
struct list_head list;
struct resource *res;
@@ -133,6 +145,8 @@ static void pci_dev_res_restore(struct pci_dev_resource *dev_res)
int idx = pci_resource_num(dev, res);
const char *res_name = pci_resource_name(dev, idx);
+ lockdep_assert_held(&pci_res_setup_lock);
+
if (WARN_ON_ONCE(resource_assigned(res)))
return;
@@ -611,6 +625,8 @@ static void __assign_resources_sorted(struct list_head *head,
unsigned long fail_type;
resource_size_t align;
+ lockdep_assert_held(&pci_res_setup_lock);
+
if (!realloc_head)
realloc_head = &dummy_head;
@@ -893,6 +909,8 @@ static void __pci_setup_bridge(struct pci_bus *bus, unsigned long type)
{
struct pci_dev *bridge = bus->self;
+ lockdep_assert_held(&pci_res_setup_lock);
+
pci_info(bridge, "PCI bridge to %pR\n", &bus->busn_res);
if (type & IORESOURCE_IO)
@@ -973,6 +991,8 @@ static void pci_bridge_check_ranges(struct pci_bus *bus)
struct pci_dev *bridge = bus->self;
struct resource *b_res;
+ lockdep_assert_held(&pci_res_setup_lock);
+
b_res = &bridge->resource[PCI_BRIDGE_MEM_WINDOW];
b_res->flags |= IORESOURCE_MEM;
@@ -1077,6 +1097,8 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t add_size,
resource_size_t children_add_size = 0;
resource_size_t min_align, align;
+ lockdep_assert_held(&pci_res_setup_lock);
+
if (!b_res)
return;
@@ -1286,6 +1308,8 @@ static void pbus_size_mem(struct pci_bus *bus, struct resource *b_res,
resource_size_t children_add_size = 0;
resource_size_t add_align = 0;
+ lockdep_assert_held(&pci_res_setup_lock);
+
if (!b_res)
return;
@@ -1457,6 +1481,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
void pci_bus_size_bridges(struct pci_bus *bus)
{
+ guard(mutex)(&pci_res_setup_lock);
guard(rwsem_read)(&pci_bus_sem);
__pci_bus_size_bridges(bus, NULL);
}
@@ -1486,6 +1511,8 @@ static void pdev_assign_fixed_resources(struct pci_dev *dev)
{
struct resource *r;
+ lockdep_assert_held(&pci_res_setup_lock);
+
pci_dev_for_each_resource(dev, r) {
struct pci_bus *b;
@@ -1550,6 +1577,7 @@ void __pci_bus_assign_resources(const struct pci_bus *bus,
void pci_bus_assign_resources(const struct pci_bus *bus)
{
+ guard(mutex)(&pci_res_setup_lock);
guard(rwsem_read)(&pci_bus_sem);
__pci_bus_assign_resources(bus, NULL, NULL);
}
@@ -1663,6 +1691,8 @@ static void pci_bridge_release_resources(struct pci_bus *bus,
struct pci_dev *dev = bus->self;
int idx, ret;
+ lockdep_assert_held(&pci_res_setup_lock);
+
if (!resource_assigned(b_win))
return;
@@ -1860,6 +1890,8 @@ static void adjust_bridge_window(struct pci_dev *bridge, struct resource *res,
resource_size_t add_size, size = resource_size(res);
struct pci_dev_resource *dev_res;
+ lockdep_assert_held(&pci_res_setup_lock);
+
if (resource_assigned(res))
return;
@@ -2192,6 +2224,7 @@ void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus)
int pci_try_num = 1;
enum enable_type enable_local;
+ guard(mutex)(&pci_res_setup_lock);
guard(rwsem_read)(&pci_bus_sem);
/* Don't realloc if asked to do so */
@@ -2270,6 +2303,7 @@ static void pci_bridge_size_and_assign_resources(struct pci_dev *bridge)
int tried_times = 0;
LIST_HEAD(fail_head);
+ guard(mutex)(&pci_res_setup_lock);
guard(rwsem_read)(&pci_bus_sem);
while (1) {
@@ -2328,6 +2362,8 @@ static int pbus_reassign_bridge_resources(struct pci_bus *bus, struct resource *
unsigned int i;
int ret = 0;
+ lockdep_assert_held(&pci_res_setup_lock);
+
while (!pci_is_root_bus(bus)) {
bridge = bus->self;
res = pbus_select_window(bus, res);
@@ -2398,6 +2434,8 @@ int pci_do_resource_release_and_resize(struct pci_dev *pdev, int resno, int size
unsigned int i;
int old, ret;
+ guard(mutex)(&pci_res_setup_lock);
+
b_win = pbus_select_window(bus, res);
if (!b_win)
return -EINVAL;
@@ -2495,6 +2533,7 @@ void pci_assign_unassigned_bus_resources(struct pci_bus *bus)
/* List of resources that want additional resources */
LIST_HEAD(add_list);
+ guard(mutex)(&pci_res_setup_lock);
guard(rwsem_read)(&pci_bus_sem);
for_each_pci_bridge(dev, bus)
if (pci_has_subordinate(dev))
@@ -2505,6 +2544,7 @@ EXPORT_SYMBOL_GPL(pci_assign_unassigned_bus_resources);
void pci_bus_size_and_assign_resources(struct pci_bus *bus)
{
+ guard(mutex)(&pci_res_setup_lock);
guard(rwsem_read)(&pci_bus_sem);
__pci_bus_size_bridges(bus, NULL);
__pci_bus_assign_resources(bus, NULL, NULL);
diff --git a/drivers/pci/setup-cardbus.c b/drivers/pci/setup-cardbus.c
index dcb8e2b3f73b..6a473a1694b9 100644
--- a/drivers/pci/setup-cardbus.c
+++ b/drivers/pci/setup-cardbus.c
@@ -6,6 +6,7 @@
#include <linux/bitfield.h>
#include <linux/errno.h>
#include <linux/ioport.h>
+#include <linux/lockdep.h>
#include <linux/pci.h>
#include <linux/sizes.h>
#include <linux/sprintf.h>
@@ -39,6 +40,8 @@ int pci_bus_size_cardbus_bridge(struct pci_bus *bus,
resource_size_t b_res_3_size = pci_cardbus_mem_size * 2;
u16 ctrl;
+ lockdep_assert_held(&pci_res_setup_lock);
+
b_res = &bridge->resource[PCI_CB_BRIDGE_IO_0_WINDOW];
if (resource_assigned(b_res))
goto handle_b_res_1;
--
2.47.3
next prev parent reply other threads:[~2026-11-16 15:25 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 15:23 [PATCH 00/13] " Ilpo Järvinen
2026-11-16 15:23 ` [PATCH 01/13] PCI: Add pci_bus_size_and_assign_resources() Ilpo Järvinen
2026-11-16 15:23 ` [PATCH 02/13] alpha/PCI: Convert to pci_bus_size_and_assign_resources() Ilpo Järvinen
2026-11-16 15:23 ` [PATCH 03/13] PCI: Use guard(&pci_bus_sem) in pci_do_resource_release_and_resize() Ilpo Järvinen
2026-11-16 15:23 ` [PATCH 04/13] PCI: Hold pci_bus_sem across sizing and assignment entrypoints Ilpo Järvinen
2026-11-16 15:23 ` Ilpo Järvinen [this message]
2026-11-16 15:23 ` [PATCH 06/13] i82875p_edac: Convert to pci_assign_unassigned_bus_resources() Ilpo Järvinen
2026-11-16 15:40 ` Borislav Petkov
2026-11-16 15:23 ` [PATCH 07/13] platform/x86: asus-wmi: " Ilpo Järvinen
2026-11-16 15:23 ` [PATCH 08/13] platform/x86: eeepc-laptop: " Ilpo Järvinen
2026-11-16 15:23 ` [PATCH 09/13] parisc/PCI: " Ilpo Järvinen
2026-11-16 15:23 ` [PATCH 10/13] PCI: hotplug: octep: " Ilpo Järvinen
2026-11-16 15:23 ` [PATCH 11/13] PCI: ftpci100: " Ilpo Järvinen
2026-11-16 15:23 ` [PATCH 12/13] PCI: hv: " Ilpo Järvinen
2026-11-16 15:23 ` [PATCH 13/13] PCI: Remove pci_bus_assign_resources() Ilpo Järvinen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261116152403.2672-6-ilpo.jarvinen@linux.intel.com \
--to=ilpo.jarvinen@linux.intel.com \
--cc=alex@shazbot.org \
--cc=bhelgaas@google.com \
--cc=christian.koenig@amd.com \
--cc=kw@linux.com \
--cc=lenb@kernel.org \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=lorenzo.pieralisi@arm.com \
--cc=rafael@kernel.org \
--cc=robh@kernel.org \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®