* [PATCH v6 0/1] PCI/IOV: Make pci_lock_rescan_remove() reentrant and protect sriov_add_vfs/sriov_del_vfs
@ 2026-03-06 8:21 Ionut Nechita (Wind River)
2026-03-06 8:21 ` [PATCH v6 1/1] " Ionut Nechita (Wind River)
0 siblings, 1 reply; 3+ messages in thread
From: Ionut Nechita (Wind River) @ 2026-03-06 8:21 UTC (permalink / raw)
To: linux-pci, bhelgaas
Cc: helgaas, sebott, schnelle, bblock, alifm, julianr, dtatulea,
mani, lukas, kbusch, ionut_n2001, sunlightlinux, linux-kernel,
stable, intel-xe, Ionut Nechita (Wind River)
Hi Bjorn,
This is v6 of the fix for the SR-IOV race between driver .remove()
and concurrent hotplug events (particularly on s390).
Changes since v5 (Mar 3):
- Reworked based on Lukas Wunner's suggestion: instead of introducing
separate pci_lock_rescan_remove_reentrant() /
pci_unlock_rescan_remove_reentrant() helpers, make the existing
pci_lock_rescan_remove() / pci_unlock_rescan_remove() themselves
reentrant using owner tracking and a depth counter
- No new API: callers simply use pci_lock/unlock_rescan_remove()
without needing to track any return value
- No changes to include/linux/pci.h
- Rebased on linux-next (20260305)
Changes since v4 (Feb 28):
- Replaced local pci_rescan_remove_owner variable with
mutex_get_owner() to check lock ownership, as suggested by
Manivannan Sadhasivam and agreed by Benjamin Block
- Removed owner tracking from pci_lock_rescan_remove() and
pci_unlock_rescan_remove() - they are now unchanged from upstream
- Rebased on linux-next (20260302)
Changes since v3 (Feb 25):
- Rebased on linux-next (next-20260227)
- Declared pci_rescan_remove_owner as const pointer
(const struct task_struct *) to make clear it is not meant to
modify the task (Benjamin Block)
- Added Reviewed-by and Tested-by from Benjamin Block (IBM)
Changes since v2 (Feb 19):
- Rebased on linux-next (next-20260225)
- Added Tested-by from Dragos Tatulea (NVIDIA)
- No code changes from v2
Changes since v1 (Feb 14):
- Renamed from pci_lock_rescan_remove_nested() to
pci_lock_rescan_remove_reentrant() to avoid confusion with
mutex_lock_nested() lockdep annotations (Benjamin Block)
- Added pci_unlock_rescan_remove_reentrant(const bool locked) helper
to avoid open-coding conditional unlock at each call site
(Benjamin Block)
- Moved declarations from drivers/pci/pci.h to include/linux/pci.h
alongside existing lock/unlock declarations (Benjamin Block)
- Simplified callers: removed negation of return value and manual
conditional unlock in favor of the paired lock/unlock helpers
The problem: on s390, platform-generated hot-unplug events for VFs
can race with sriov_del_vfs() when a PF driver is being unloaded.
The platform event handler takes pci_rescan_remove_lock, but
sriov_del_vfs() does not, leading to double removal and list
corruption. We cannot use a plain mutex_lock() because
sriov_del_vfs() may be called from paths that already hold the
lock (deadlock), and mutex_trylock() cannot distinguish self from
other holders.
The fix makes pci_lock_rescan_remove() reentrant using owner tracking
and a depth counter: if the current task already holds the lock, the
counter is incremented; pci_unlock_rescan_remove() decrements the
counter and only releases the mutex when it reaches zero. This keeps
the existing API unchanged while providing correct serialization.
Link: https://lore.kernel.org/linux-pci/20260214193235.262219-3-ionut.nechita@windriver.com/ [v1]
Link: https://lore.kernel.org/linux-pci/20260219212648.82606-1-ionut.nechita@windriver.com/ [v2]
Link: https://lore.kernel.org/linux-pci/20260225202434.18737-1-ionut.nechita@windriver.com/ [v3]
Link: https://lore.kernel.org/linux-pci/20260228120138.51197-2-ionut.nechita@windriver.com/ [v4]
Link: https://lore.kernel.org/linux-pci/20260303080903.28693-1-ionut.nechita@windriver.com/ [v5]
Ionut Nechita (Wind River) (1):
PCI/IOV: Make pci_lock_rescan_remove() reentrant and protect
sriov_add_vfs/sriov_del_vfs
drivers/pci/iov.c | 5 +++++
drivers/pci/probe.c | 13 ++++++++++++-
2 files changed, 17 insertions(+), 1 deletion(-)
base-commit: 3f9cd19e764b782706dbaacc69e502099cb014ba
--
2.53.0
^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH v6 1/1] PCI/IOV: Make pci_lock_rescan_remove() reentrant and protect sriov_add_vfs/sriov_del_vfs
2026-03-06 8:21 [PATCH v6 0/1] PCI/IOV: Make pci_lock_rescan_remove() reentrant and protect sriov_add_vfs/sriov_del_vfs Ionut Nechita (Wind River)
@ 2026-03-06 8:21 ` Ionut Nechita (Wind River)
2026-03-06 15:26 ` Benjamin Block
0 siblings, 1 reply; 3+ messages in thread
From: Ionut Nechita (Wind River) @ 2026-03-06 8:21 UTC (permalink / raw)
To: linux-pci, bhelgaas
Cc: helgaas, sebott, schnelle, bblock, alifm, julianr, dtatulea,
mani, lukas, kbusch, ionut_n2001, sunlightlinux, linux-kernel,
stable, intel-xe, Ionut Nechita (Wind River)
After reverting commit 05703271c3cd ("PCI/IOV: Add PCI rescan-remove
locking when enabling/disabling SR-IOV") and moving the lock to
sriov_numvfs_store(), the path through driver .remove() (e.g. rmmod,
or manual unbind) that calls pci_disable_sriov() directly remains
unprotected against concurrent hotplug events. This affects any SR-IOV
capable driver that calls pci_disable_sriov() from its .remove()
callback (i40e, ice, mlx5, bnxt, etc.).
On s390, platform-generated hot-unplug events for VFs can race with
sriov_del_vfs() when a PF driver is being unloaded. The platform event
handler takes pci_rescan_remove_lock, but sriov_del_vfs() does not,
leading to double removal and list corruption.
We cannot use a plain mutex_lock() here because sriov_del_vfs() may also
be called from paths that already hold pci_rescan_remove_lock (e.g.
remove_store -> pci_stop_and_remove_bus_device_locked, or
sriov_numvfs_store with the lock taken by the previous patch). Using
mutex_lock() in those cases would deadlock.
Make pci_lock_rescan_remove() itself reentrant using owner tracking and
a depth counter, as suggested by Lukas Wunner, since these recursive
locking scenarios exist elsewhere in the PCI subsystem:
- If the lock is not held: acquires the mutex, sets owner to current
task, and initializes the depth counter to 1.
- If the lock is already held by the current task: increments the
depth counter and returns without re-acquiring, avoiding deadlock.
- If the lock is held by another task: blocks until the lock is
released, providing complete serialization.
pci_unlock_rescan_remove() decrements the depth counter and only
releases the mutex when it reaches zero.
This approach keeps the API unchanged: callers simply pair lock/unlock
calls without needing to track any return value or use separate
reentrant variants.
Add pci_lock_rescan_remove()/pci_unlock_rescan_remove() calls to
sriov_add_vfs() and sriov_del_vfs() to protect VF addition and
removal against concurrent hotplug events.
Fixes: 18f9e9d150fc ("PCI/IOV: Factor out sriov_add_vfs()")
Cc: stable@vger.kernel.org
Suggested-by: Lukas Wunner <lukas@wunner.de>
Tested-by: Dragos Tatulea <dtatulea@nvidia.com>
Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
Tested-by: Benjamin Block <bblock@linux.ibm.com>
Signed-off-by: Ionut Nechita <ionut_n2001@yahoo.com>
Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>
---
drivers/pci/iov.c | 5 +++++
drivers/pci/probe.c | 13 ++++++++++++-
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index 91ac4e37ecb9c..aba2fb90759cd 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -633,15 +633,18 @@ static int sriov_add_vfs(struct pci_dev *dev, u16 num_vfs)
if (dev->no_vf_scan)
return 0;
+ pci_lock_rescan_remove();
for (i = 0; i < num_vfs; i++) {
rc = pci_iov_add_virtfn(dev, i);
if (rc)
goto failed;
}
+ pci_unlock_rescan_remove();
return 0;
failed:
while (i--)
pci_iov_remove_virtfn(dev, i);
+ pci_unlock_rescan_remove();
return rc;
}
@@ -766,8 +769,10 @@ static void sriov_del_vfs(struct pci_dev *dev)
struct pci_sriov *iov = dev->sriov;
int i;
+ pci_lock_rescan_remove();
for (i = 0; i < iov->num_VFs; i++)
pci_iov_remove_virtfn(dev, i);
+ pci_unlock_rescan_remove();
}
static void sriov_disable(struct pci_dev *dev)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index bccc7a4bdd794..c7efb8e1010d3 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -3509,16 +3509,27 @@ EXPORT_SYMBOL_GPL(pci_rescan_bus);
* routines should always be executed under this mutex.
*/
DEFINE_MUTEX(pci_rescan_remove_lock);
+static struct task_struct *pci_rescan_remove_owner;
+static unsigned int pci_rescan_remove_count;
void pci_lock_rescan_remove(void)
{
+ if (pci_rescan_remove_owner == current) {
+ pci_rescan_remove_count++;
+ return;
+ }
mutex_lock(&pci_rescan_remove_lock);
+ pci_rescan_remove_owner = current;
+ pci_rescan_remove_count = 1;
}
EXPORT_SYMBOL_GPL(pci_lock_rescan_remove);
void pci_unlock_rescan_remove(void)
{
- mutex_unlock(&pci_rescan_remove_lock);
+ if (--pci_rescan_remove_count == 0) {
+ pci_rescan_remove_owner = NULL;
+ mutex_unlock(&pci_rescan_remove_lock);
+ }
}
EXPORT_SYMBOL_GPL(pci_unlock_rescan_remove);
--
2.53.0
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH v6 1/1] PCI/IOV: Make pci_lock_rescan_remove() reentrant and protect sriov_add_vfs/sriov_del_vfs
2026-03-06 8:21 ` [PATCH v6 1/1] " Ionut Nechita (Wind River)
@ 2026-03-06 15:26 ` Benjamin Block
0 siblings, 0 replies; 3+ messages in thread
From: Benjamin Block @ 2026-03-06 15:26 UTC (permalink / raw)
To: Ionut Nechita (Wind River)
Cc: linux-pci, bhelgaas, helgaas, sebott, schnelle, alifm, julianr,
dtatulea, mani, lukas, kbusch, ionut_n2001, sunlightlinux,
linux-kernel, stable, intel-xe
Hey Ionut,
On Fri, Mar 06, 2026 at 10:21:08AM +0200, Ionut Nechita (Wind River) wrote:
--8<--
> Fixes: 18f9e9d150fc ("PCI/IOV: Factor out sriov_add_vfs()")
> Cc: stable@vger.kernel.org
> Suggested-by: Lukas Wunner <lukas@wunner.de>
> Tested-by: Dragos Tatulea <dtatulea@nvidia.com>
> Reviewed-by: Benjamin Block <bblock@linux.ibm.com>
> Tested-by: Benjamin Block <bblock@linux.ibm.com>
I'm not sure whether you are aware of this, but generally, once you make
substantially changes to a patch, it is common practice to remove the previous
Tested-by and/or Reviewed-by tags again:
https://docs.kernel.org/process/submitting-patches.html#reviewer-s-statement-of-oversight
(3. paragraph).
It's a bit of a judgement call what level of change is necessary to be
"substantial", but anyway, I thought I'd mention it :)
> Signed-off-by: Ionut Nechita <ionut_n2001@yahoo.com>
> Signed-off-by: Ionut Nechita <ionut.nechita@windriver.com>
> ---
--8<--
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index bccc7a4bdd794..c7efb8e1010d3 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -3509,16 +3509,27 @@ EXPORT_SYMBOL_GPL(pci_rescan_bus);
> * routines should always be executed under this mutex.
> */
> DEFINE_MUTEX(pci_rescan_remove_lock);
> +static struct task_struct *pci_rescan_remove_owner;
> +static unsigned int pci_rescan_remove_count;
>
> void pci_lock_rescan_remove(void)
> {
> + if (pci_rescan_remove_owner == current) {
> + pci_rescan_remove_count++;
> + return;
> + }
> mutex_lock(&pci_rescan_remove_lock);
> + pci_rescan_remove_owner = current;
> + pci_rescan_remove_count = 1;
> }
> EXPORT_SYMBOL_GPL(pci_lock_rescan_remove);
>
> void pci_unlock_rescan_remove(void)
> {
> - mutex_unlock(&pci_rescan_remove_lock);
> + if (--pci_rescan_remove_count == 0) {
> + pci_rescan_remove_owner = NULL;
> + mutex_unlock(&pci_rescan_remove_lock);
> + }
> }
> EXPORT_SYMBOL_GPL(pci_unlock_rescan_remove);
I'm not sure Lukas meant to revert to a local cache for the task_struct, just
that the counter variable itself doesn't need to be an atomic.
How the counting is done itself is probably very much a taste question. I've
run tests in my test lab with this change:
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 6686cee98afc..28c0384424db 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -3510,27 +3510,23 @@ EXPORT_SYMBOL_GPL(pci_rescan_bus);
*/
DEFINE_MUTEX(pci_rescan_remove_lock);
EXPORT_SYMBOL_GPL(pci_rescan_remove_lock);
-static struct task_struct *pci_rescan_remove_owner;
-static unsigned int pci_rescan_remove_count;
+static size_t pci_rescan_remove_reentrant_count = 0;
void pci_lock_rescan_remove(void)
{
- if (pci_rescan_remove_owner == current) {
- pci_rescan_remove_count++;
- return;
- }
- mutex_lock(&pci_rescan_remove_lock);
- pci_rescan_remove_owner = current;
- pci_rescan_remove_count = 1;
+ if (mutex_get_owner(&pci_rescan_remove_lock) == (unsigned long)current)
+ pci_rescan_remove_reentrant_count++;
+ else
+ mutex_lock(&pci_rescan_remove_lock);
}
EXPORT_SYMBOL_GPL(pci_lock_rescan_remove);
void pci_unlock_rescan_remove(void)
{
- if (--pci_rescan_remove_count == 0) {
- pci_rescan_remove_owner = NULL;
+ if (pci_rescan_remove_reentrant_count > 0)
+ pci_rescan_remove_reentrant_count--;
+ else
mutex_unlock(&pci_rescan_remove_lock);
- }
}
EXPORT_SYMBOL_GPL(pci_unlock_rescan_remove);
But like I said, the counting comes down to about the same as with your patch
(as far as I can tell), just that it also uses the mutex_get_owner() call.
--
Best Regards, Benjamin Block / Linux on IBM Z Kernel Development
IBM Deutschland Research & Development GmbH / https://www.ibm.com/privacy
Vors. Aufs.-R.: Wolfgang Wendt / Geschäftsführung: David Faller
Sitz der Ges.: Ehningen / Registergericht: AmtsG Stuttgart, HRB 243294
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-03-06 15:27 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-03-06 8:21 [PATCH v6 0/1] PCI/IOV: Make pci_lock_rescan_remove() reentrant and protect sriov_add_vfs/sriov_del_vfs Ionut Nechita (Wind River)
2026-03-06 8:21 ` [PATCH v6 1/1] " Ionut Nechita (Wind River)
2026-03-06 15:26 ` Benjamin Block
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®