* [PATCH 1/3] usb: core: Fix use-after-free in anchor wakeup protocol
2026-10-22 3:53 [PATCH 0/3] usb: core: fix three use-after-free windows in the USB anchor protocol Hongyu Xie
@ 2026-10-22 3:53 ` Hongyu Xie
2026-10-22 3:53 ` [PATCH 2/3] usb: core: Close anchor capture-to-increment race in URB giveback Hongyu Xie
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Hongyu Xie @ 2026-10-22 3:53 UTC (permalink / raw)
To: Greg KH, Alan Stern; +Cc: linux-usb, Oliver Neukum, Hans de Goede, linux-kernel
The API contract of usb_wait_anchor_empty_timeout() and of the anchor
cleanup helpers is that when they return successfully (non-zero, for
usb_wait_anchor_empty_timeout()), no URB giveback is in flight for the
anchor anymore, so drivers may free the memory holding the anchor
(xpad, for example, embeds irq_out_anchor in its private structure and
kfrees it in xpad_disconnect()).
This contract is broken by two windows in which the waiter can observe
"anchor list empty and suspend_wakeups == 0" while the URB giveback
path is still about to access anchor->wait:
1. usb_anchor_resume_wakeups() decrements suspend_wakeups, which
makes the wait condition true, and only afterwards calls
wake_up(&anchor->wait), without any synchronization in between.
A waiter returns, the driver frees the anchor, and the late
wake_up() then touches the freed wait queue head.
2. __usb_unanchor_urb() wakes up waiters while holding anchor->lock,
but usb_wait_anchor_empty_timeout() evaluated the same condition
without the lock, so it could observe the emptied urb_list in the
middle of that critical section and return before the wake_up()
(and the spin_unlock()) had run. Reading the two condition fields
locklessly could likewise observe a stale suspend_wakeups value
after the giveback's unanchor on weakly ordered systems.
Complete the locking protocol that commit fbc299437c06 ("usb: core:
Solve race condition in anchor cleanup functions") established for the
kill/poison/scuttle helpers: usb_wait_anchor_empty_timeout() now
evaluates the condition under anchor->lock (a small self-locking
wrapper around the lockless check), and usb_anchor_resume_wakeups()
performs the suspend_wakeups decrement together with the wakeup check
and the wake_up() within a single anchor->lock critical section.
With the decrement and the wake_up() atomic with respect to the locked
condition check, a waiter can only observe the condition true after
every giveback that has incremented suspend_wakeups has finished
accessing the anchor, so the guarantee that the wakeup is delayed
until the completion handler has run also extends to the anchor memory
itself.
The race has been reported by syzbot repeatedly since 2019, most
recently as:
BUG: KASAN: slab-use-after-free in register_lock_class+0x8d1/0x980 kernel/locking/lockdep.c:1333
[...]
__wake_up_common_lock+0x25/0x1e0 kernel/sched/wait.c:105
__usb_hcd_giveback_urb+0x3ac/0x530 drivers/usb/core/hcd.c:1653
Freed by task 7828:
[...]
kfree+0x14a/0x380 mm/slub.c:4390
xpad_disconnect+0x359/0x490 drivers/input/joystick/xpad.c:2194
and:
BUG: KASAN: use-after-free in register_lock_class+0xe6a/0x1120 kernel/locking/lockdep.c:1336
[...]
__wake_up_common_lock+0xb4/0x130 kernel/sched/wait.c:136
usb_anchor_resume_wakeups drivers/usb/core/urb.c:958 [inline]
usb_anchor_resume_wakeups+0xbe/0xe0 drivers/usb/core/urb.c:951
__usb_hcd_giveback_urb+0x2df/0x5c0 drivers/usb/core/hcd.c:1674
Freed by task 3889:
[...]
kfree+0x173/0x390 mm/slab.c:3786
xpad_disconnect+0x1cb/0x530 drivers/input/joystick/xpad.c:1905
Both reports lack a reproducer; the analysis was verified by walking
the two stacks against the code, not at runtime. All in-tree callers
of usb_wait_anchor_empty_timeout() (and every driver which frees an
anchor after the anchor cleanup helpers) are equally affected; xpad
is merely the one syzbot found.
Fixes: 6ec4147e7bdb ("usb-anchor: Delay usb_wait_anchor_empty_timeout wake up till completion is done")
Fixes: 51a2f077c44e ("USB: introduce usb_anchor")
Reported-by: syzbot+7f5e98b052c766f47f68@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/r/00000000000090170a0614431f47@google.com
Reported-by: syzbot+59d3714b44d79d02f944@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/r/00000000000038435e05e9f94cd4@google.com
Cc: stable@vger.kernel.org
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Oliver Neukum <oliver@neukum.org>
Cc: Hans de Goede <hansg@kernel.org>
Signed-off-by: Hongyu Xie <xiehongyu1@kylinos.cn>
---
drivers/usb/core/urb.c | 51 +++++++++++++++++++++++++++++++++++++-----
1 file changed, 45 insertions(+), 6 deletions(-)
diff --git a/drivers/usb/core/urb.c b/drivers/usb/core/urb.c
index c06b44ca507b..3afd184c9b32 100644
--- a/drivers/usb/core/urb.c
+++ b/drivers/usb/core/urb.c
@@ -139,19 +139,39 @@ void usb_anchor_urb(struct urb *urb, struct usb_anchor *anchor)
}
EXPORT_SYMBOL_GPL(usb_anchor_urb);
-static int usb_anchor_check_wakeup(struct usb_anchor *anchor)
+/* Callers must hold anchor->lock */
+static int __usb_anchor_check_wakeup(struct usb_anchor *anchor)
{
return atomic_read(&anchor->suspend_wakeups) == 0 &&
list_empty(&anchor->urb_list);
}
+/*
+ * Same check as __usb_anchor_check_wakeup(), but takes anchor->lock. It is
+ * meant for waiters which do not hold the lock: the critical section
+ * serializes with the URB giveback path, so once the check succeeds no
+ * giveback that has incremented suspend_wakeups will touch the anchor
+ * anymore and the caller may free the memory holding the anchor.
+ */
+static int usb_anchor_check_wakeup(struct usb_anchor *anchor)
+{
+ unsigned long flags;
+ int wakeup;
+
+ spin_lock_irqsave(&anchor->lock, flags);
+ wakeup = __usb_anchor_check_wakeup(anchor);
+ spin_unlock_irqrestore(&anchor->lock, flags);
+
+ return wakeup;
+}
+
/* Callers must hold anchor->lock */
static void __usb_unanchor_urb(struct urb *urb, struct usb_anchor *anchor)
{
urb->anchor = NULL;
list_del(&urb->anchor_list);
usb_put_urb(urb);
- if (usb_anchor_check_wakeup(anchor))
+ if (__usb_anchor_check_wakeup(anchor))
wake_up(&anchor->wait);
}
@@ -827,7 +847,7 @@ void usb_kill_anchored_urbs(struct usb_anchor *anchor)
usb_put_urb(victim);
spin_lock_irq(&anchor->lock);
}
- surely_empty = usb_anchor_check_wakeup(anchor);
+ surely_empty = __usb_anchor_check_wakeup(anchor);
spin_unlock_irq(&anchor->lock);
cpu_relax();
@@ -866,7 +886,7 @@ void usb_poison_anchored_urbs(struct usb_anchor *anchor)
usb_put_urb(victim);
spin_lock_irq(&anchor->lock);
}
- surely_empty = usb_anchor_check_wakeup(anchor);
+ surely_empty = __usb_anchor_check_wakeup(anchor);
spin_unlock_irq(&anchor->lock);
cpu_relax();
@@ -919,12 +939,23 @@ EXPORT_SYMBOL_GPL(usb_anchor_suspend_wakeups);
*/
void usb_anchor_resume_wakeups(struct usb_anchor *anchor)
{
+ unsigned long flags;
+
if (!anchor)
return;
+ /*
+ * The decrement, the wakeup check and the wake_up() must happen
+ * under anchor->lock: usb_wait_anchor_empty_timeout() waiters
+ * evaluate the same condition under the lock and may free the
+ * anchor once they observe it true, so this function must not
+ * touch anchor->wait after that.
+ */
+ spin_lock_irqsave(&anchor->lock, flags);
atomic_dec(&anchor->suspend_wakeups);
- if (usb_anchor_check_wakeup(anchor))
+ if (__usb_anchor_check_wakeup(anchor))
wake_up(&anchor->wait);
+ spin_unlock_irqrestore(&anchor->lock, flags);
}
EXPORT_SYMBOL_GPL(usb_anchor_resume_wakeups);
@@ -941,6 +972,14 @@ EXPORT_SYMBOL_GPL(usb_anchor_resume_wakeups);
int usb_wait_anchor_empty_timeout(struct usb_anchor *anchor,
unsigned int timeout)
{
+ /*
+ * The condition must be evaluated under anchor->lock: when this
+ * function returns non-zero, every URB giveback that has incremented
+ * suspend_wakeups has finished with the anchor, so the caller may
+ * free the anchor, which is typically embedded in the driver's
+ * private structure. A zero return (timeout) means a giveback may
+ * still be in flight and gives no such guarantee.
+ */
return wait_event_timeout(anchor->wait,
usb_anchor_check_wakeup(anchor),
msecs_to_jiffies(timeout));
@@ -997,7 +1036,7 @@ void usb_scuttle_anchored_urbs(struct usb_anchor *anchor)
struct urb, anchor_list);
__usb_unanchor_urb(victim, anchor);
}
- surely_empty = usb_anchor_check_wakeup(anchor);
+ surely_empty = __usb_anchor_check_wakeup(anchor);
spin_unlock_irqrestore(&anchor->lock, flags);
cpu_relax();
--
2.32.0
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 2/3] usb: core: Close anchor capture-to-increment race in URB giveback
2026-10-22 3:53 [PATCH 0/3] usb: core: fix three use-after-free windows in the USB anchor protocol Hongyu Xie
2026-10-22 3:53 ` [PATCH 1/3] usb: core: Fix use-after-free in anchor wakeup protocol Hongyu Xie
@ 2026-10-22 3:53 ` Hongyu Xie
2026-10-22 3:53 ` [PATCH 3/3] usb: core: Close residual anchor read-to-lock window with RCU Hongyu Xie
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Hongyu Xie @ 2026-10-22 3:53 UTC (permalink / raw)
To: Greg KH, Alan Stern; +Cc: linux-usb, Oliver Neukum, Hans de Goede, linux-kernel
__usb_hcd_giveback_urb() captures urb->anchor into a local variable at
the top of the function, but increments the anchor's suspend_wakeups
counter only after unmapping the URB for DMA and running the usbmon
hooks. Nothing pins the anchor during that window. If the URB is
concurrently unanchored by another thread -- a driver calling
usb_unanchor_urb() on its own in-flight URB, or usb_get_from_anchor()
or usb_scuttle_anchored_urbs() taking it -- the anchor list can become
empty while suspend_wakeups is still zero. A
usb_wait_anchor_empty_timeout() waiter then observes the wait condition
true, returns, and the driver may free the anchor (anchors are
typically embedded in driver-private structures); the giveback's
atomic_inc() on the captured pointer, and the later decrement and
wake_up() in usb_anchor_resume_wakeups(), operate on freed memory.
The previous fix ("usb: core: Fix use-after-free in anchor wakeup
protocol") made the waiter's condition check and the decrement/wakeup
side hold anchor->lock, but its guarantee was deliberately scoped to
givebacks that had already incremented suspend_wakeups: the
capture-to-increment gap remained open.
Fold the pointer capture, the suspend_wakeups increment and the
unanchor into a single anchor->lock critical section which re-verifies
that urb->anchor still names the captured anchor, so a giveback that
lost the race against a concurrent unanchor performs no anchor work at
all. From the verified increment onward, a non-zero suspend_wakeups
keeps waiters from freeing the anchor until
usb_anchor_resume_wakeups() has finished with it.
What remains open is the few instructions between reading urb->anchor
and acquiring anchor->lock inside the new helper: a concurrent
unanchor plus waiter return plus anchor free would all have to fit
into that gap for the lock acquisition itself to touch freed memory.
This is the same residual exposure usb_unanchor_urb() has had since
usb_anchor was introduced; closing it entirely would require a
lifetime protocol change (reference count or RCU) for anchors, which
is out of scope for a bugfix.
No syzbot report has been attributed to this interleaving; it was
found by extending the analysis of the previous fix.
Fixes: 6ec4147e7bdb ("usb-anchor: Delay usb_wait_anchor_empty_timeout wake up till completion is done")
Cc: stable@vger.kernel.org
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Oliver Neukum <oliver@neukum.org>
Cc: Hans de Goede <hansg@kernel.org>
Signed-off-by: Hongyu Xie <xiehongyu1@kylinos.cn>
---
drivers/usb/core/hcd.c | 10 ++++++---
drivers/usb/core/urb.c | 47 ++++++++++++++++++++++++++++++++++++++++++
drivers/usb/core/usb.h | 1 +
3 files changed, 55 insertions(+), 3 deletions(-)
diff --git a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c
index 8e8a5f59b319..f221425fec0a 100644
--- a/drivers/usb/core/hcd.c
+++ b/drivers/usb/core/hcd.c
@@ -1630,7 +1630,7 @@ int usb_hcd_unlink_urb (struct urb *urb, int status)
static void __usb_hcd_giveback_urb(struct urb *urb)
{
struct usb_hcd *hcd = bus_to_hcd(urb->dev->bus);
- struct usb_anchor *anchor = urb->anchor;
+ struct usb_anchor *anchor;
int status = urb->unlinked;
urb->hcpriv = NULL;
@@ -1641,8 +1641,12 @@ static void __usb_hcd_giveback_urb(struct urb *urb)
unmap_urb_for_dma(hcd, urb);
usbmon_urb_complete(&hcd->self, urb, status);
- usb_anchor_suspend_wakeups(anchor);
- usb_unanchor_urb(urb);
+ /*
+ * Capturing the anchor, suspending its wakeups and unanchoring
+ * must be atomic with each other; see
+ * usb_unanchor_urb_and_suspend_wakeups().
+ */
+ anchor = usb_unanchor_urb_and_suspend_wakeups(urb);
if (likely(status == 0))
usb_led_activity(USB_LED_EVENT_HOST);
diff --git a/drivers/usb/core/urb.c b/drivers/usb/core/urb.c
index 3afd184c9b32..c78f72bc7cc0 100644
--- a/drivers/usb/core/urb.c
+++ b/drivers/usb/core/urb.c
@@ -14,6 +14,8 @@
#include <linux/usb/hcd.h>
#include <linux/scatterlist.h>
+#include "usb.h"
+
#define to_urb(d) container_of(d, struct urb, kref)
@@ -205,6 +207,51 @@ void usb_unanchor_urb(struct urb *urb)
}
EXPORT_SYMBOL_GPL(usb_unanchor_urb);
+/*
+ * usb_unanchor_urb_and_suspend_wakeups - atomically unanchor @urb and
+ * increment the suspend_wakeups counter of its anchor.
+ * @urb: the URB to unanchor
+ *
+ * Used by the URB giveback path, which must capture urb->anchor and
+ * increment anchor->suspend_wakeups atomically with the removal of
+ * @urb from anchor->urb_list. Otherwise a concurrent unanchor of
+ * @urb (a driver calling usb_unanchor_urb() on its own in-flight URB,
+ * or usb_get_from_anchor() or usb_scuttle_anchored_urbs() taking it)
+ * can leave the anchor with an empty urb_list while suspend_wakeups
+ * is still zero, so a usb_wait_anchor_empty_timeout() waiter observes
+ * the wait condition true, returns, and the driver may free the
+ * anchor before the giveback gets to touch it.
+ *
+ * The verification of urb->anchor under anchor->lock makes this safe
+ * against racing re-anchoring, because "urb->anchor == anchor" and
+ * "@urb is on anchor->urb_list" are changed together only while
+ * holding anchor->lock.
+ *
+ * Return: the anchor, which the caller must later pass to
+ * usb_anchor_resume_wakeups(), or NULL if @urb is not anchored
+ * anymore (a concurrent unanchor won the race).
+ */
+struct usb_anchor *usb_unanchor_urb_and_suspend_wakeups(struct urb *urb)
+{
+ struct usb_anchor *anchor;
+ unsigned long flags;
+
+ anchor = urb->anchor;
+ if (!anchor)
+ return NULL;
+
+ spin_lock_irqsave(&anchor->lock, flags);
+ if (likely(anchor == urb->anchor)) {
+ atomic_inc(&anchor->suspend_wakeups);
+ __usb_unanchor_urb(urb, anchor);
+ } else {
+ anchor = NULL;
+ }
+ spin_unlock_irqrestore(&anchor->lock, flags);
+
+ return anchor;
+}
+
/*-------------------------------------------------------------------*/
static const int pipetypes[4] = {
diff --git a/drivers/usb/core/usb.h b/drivers/usb/core/usb.h
index a9b37aeb515b..362bfda497cc 100644
--- a/drivers/usb/core/usb.h
+++ b/drivers/usb/core/usb.h
@@ -20,6 +20,7 @@ extern int usb_create_ep_devs(struct device *parent,
struct usb_host_endpoint *endpoint,
struct usb_device *udev);
extern void usb_remove_ep_devs(struct usb_host_endpoint *endpoint);
+extern struct usb_anchor *usb_unanchor_urb_and_suspend_wakeups(struct urb *urb);
extern void usb_enable_endpoint(struct usb_device *dev,
struct usb_host_endpoint *ep, bool reset_toggle);
--
2.32.0
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 3/3] usb: core: Close residual anchor read-to-lock window with RCU
2026-10-22 3:53 [PATCH 0/3] usb: core: fix three use-after-free windows in the USB anchor protocol Hongyu Xie
2026-10-22 3:53 ` [PATCH 1/3] usb: core: Fix use-after-free in anchor wakeup protocol Hongyu Xie
2026-10-22 3:53 ` [PATCH 2/3] usb: core: Close anchor capture-to-increment race in URB giveback Hongyu Xie
@ 2026-10-22 3:53 ` Hongyu Xie
2026-11-03 2:20 ` [PATCH v2 0/3] usb: core: fix three use-after-free windows in the USB anchor protocol Hongyu Xie
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Hongyu Xie @ 2026-10-22 3:53 UTC (permalink / raw)
To: Greg KH, Alan Stern; +Cc: linux-usb, Oliver Neukum, Hans de Goede, linux-kernel
usb_unanchor_urb() and the URB giveback read urb->anchor and only then
acquire anchor->lock. Between the read and the lock acquisition the
anchor can already have been freed: a concurrent unanchor of the last
URB empties the anchor, a usb_wait_anchor_empty_timeout() waiter
observes the wait condition true, returns, and the driver frees the
memory holding the anchor; the spin_lock() then touches freed memory.
This gap is as old as usb_unanchor_urb() itself and was deliberately
left open by the capture-to-increment fix, whose guarantee is scoped to
givebacks that have already incremented suspend_wakeups.
A plain reference count cannot close this gap: the first dereference of
the freshly read anchor pointer would have to be the refcount increment
itself, so the pin would be taken on memory that may already have been
freed. Protecting the pointer read requires deferred reclamation, and
because anchors are embedded in driver-private structures and freed
with plain kfree() right after a successful wait, the deferral has to
live on the side usbcore controls: the return of the wait.
Convert the anchor wakeup protocol into a lifetime protocol:
- The two lockless urb->anchor reads (usb_unanchor_urb() and
usb_unanchor_urb_and_suspend_wakeups()) run inside RCU read-side
critical sections, and urb->anchor is published with
rcu_assign_pointer().
- The wait condition is evaluated under anchor->lock together with
taking a suspend_wakeups pin in the same critical section, so at
most one thread at a time owns the "anchor is unused" observation.
- Every function whose return lets the caller free the anchor --
usb_wait_anchor_empty_timeout() on a non-zero return,
usb_kill_anchored_urbs(), usb_poison_anchored_urbs() and
usb_scuttle_anchored_urbs() -- holds that pin across one
synchronize_rcu() before dropping it and returning. These are
teardown guarantees: provided the caller stops anchoring new URBs,
readers that read urb->anchor before the grace period have
acquired and released anchor->lock by the time the caller may free
the anchor, and readers after it find urb->anchor == NULL and
never touch the anchor at all. (A concurrent usb_anchor_urb()
would re-publish urb->anchor; the protocol does not police the
caller's own anchoring.)
usb_wait_anchor_empty_timeout() keeps its documented contract: a zero
timeout still performs a single condition evaluation without entering
the wait queue -- a successful evaluation still waits one grace period
before returning -- the deadline is fixed at entry so any grace period
waited is accounted against the caller's timeout budget, and a
successful wait returns 1 rather than the remaining jiffies, with the
anchor state re-checked after the grace period.
The giveback's long window across the completion handler keeps the
suspend_wakeups pin it already takes. No fields are added to struct
usb_anchor or struct urb and no driver changes are required; the cost
is one RCU grace period per successful teardown exit -- including
teardown of an already-empty anchor, which previously returned without
one -- so usb_kill_anchored_urbs(), usb_poison_anchored_urbs() and
usb_scuttle_anchored_urbs() must now be called from a context that can
sleep (all in-tree callers already do). The per-URB hot paths gain
only rcu_read_lock()/rcu_dereference() around the lockless anchor
reads and rcu_assign_pointer() for the anchor pointer updates.
Fixes: 51a2f077c44e ("USB: introduce usb_anchor")
Cc: stable@vger.kernel.org
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Oliver Neukum <oliver@neukum.org>
Cc: Hans de Goede <hansg@kernel.org>
Signed-off-by: Hongyu Xie <xiehongyu1@kylinos.cn>
---
drivers/usb/core/urb.c | 255 +++++++++++++++++++++++++++++++----------
include/linux/usb.h | 7 +-
2 files changed, 202 insertions(+), 60 deletions(-)
diff --git a/drivers/usb/core/urb.c b/drivers/usb/core/urb.c
index c78f72bc7cc0..4fc6ec758294 100644
--- a/drivers/usb/core/urb.c
+++ b/drivers/usb/core/urb.c
@@ -9,6 +9,7 @@
#include <linux/slab.h>
#include <linux/log2.h>
#include <linux/kmsan.h>
+#include <linux/rcupdate.h>
#include <linux/usb.h>
#include <linux/wait.h>
#include <linux/usb/hcd.h>
@@ -132,7 +133,7 @@ void usb_anchor_urb(struct urb *urb, struct usb_anchor *anchor)
spin_lock_irqsave(&anchor->lock, flags);
usb_get_urb(urb);
list_add_tail(&urb->anchor_list, &anchor->urb_list);
- urb->anchor = anchor;
+ rcu_assign_pointer(urb->anchor, anchor);
if (unlikely(anchor->poisoned))
atomic_inc(&urb->reject);
@@ -149,28 +150,71 @@ static int __usb_anchor_check_wakeup(struct usb_anchor *anchor)
}
/*
- * Same check as __usb_anchor_check_wakeup(), but takes anchor->lock. It is
- * meant for waiters which do not hold the lock: the critical section
- * serializes with the URB giveback path, so once the check succeeds no
- * giveback that has incremented suspend_wakeups will touch the anchor
- * anymore and the caller may free the memory holding the anchor.
+ * Wait-condition side of the anchor lifetime protocol: evaluate the
+ * wakeup condition under anchor->lock and, if it holds, take a pin
+ * (increment suspend_wakeups) inside the same critical section. While
+ * the pin is held the condition is false for every other thread, so at
+ * most one waiter at a time owns the "anchor is unused" observation and
+ * may carry out the grace-period exit before any other waiter proceeds.
+ *
+ * The caller must pair a true return with usb_anchor_finish_wait(),
+ * which drops the pin.
*/
-static int usb_anchor_check_wakeup(struct usb_anchor *anchor)
+static int usb_anchor_pin_if_wakeup(struct usb_anchor *anchor)
{
unsigned long flags;
int wakeup;
spin_lock_irqsave(&anchor->lock, flags);
- wakeup = __usb_anchor_check_wakeup(anchor);
+ if (__usb_anchor_check_wakeup(anchor)) {
+ atomic_inc(&anchor->suspend_wakeups);
+ wakeup = 1;
+ } else {
+ wakeup = 0;
+ }
spin_unlock_irqrestore(&anchor->lock, flags);
return wakeup;
}
+/*
+ * Drop a pin taken by usb_anchor_pin_if_wakeup() after letting one RCU
+ * grace period elapse. The grace period flushes readers that have read
+ * urb->anchor but not yet acquired anchor->lock (see usb_unanchor_urb()
+ * and usb_unanchor_urb_and_suspend_wakeups()): once it completes, no
+ * reader that started before it can still touch the anchor, so waking
+ * other waiters and returning to the caller -- which may then free the
+ * memory holding the anchor -- is safe. Readers that start after the
+ * grace period see urb->anchor == NULL and never touch the anchor at
+ * all, provided that no concurrent usb_anchor_urb() re-publishes the
+ * anchor: the safe-to-free guarantees of this protocol are teardown
+ * guarantees and assume the caller stops anchoring new URBs.
+ *
+ * Must not be called with anchor->lock held.
+ *
+ * Returns true if the anchor is unused once the pin is dropped.
+ */
+static bool usb_anchor_finish_wait(struct usb_anchor *anchor)
+{
+ unsigned long flags;
+ bool unused;
+
+ synchronize_rcu();
+
+ spin_lock_irqsave(&anchor->lock, flags);
+ atomic_dec(&anchor->suspend_wakeups);
+ unused = __usb_anchor_check_wakeup(anchor);
+ if (unused)
+ wake_up(&anchor->wait);
+ spin_unlock_irqrestore(&anchor->lock, flags);
+
+ return unused;
+}
+
/* Callers must hold anchor->lock */
static void __usb_unanchor_urb(struct urb *urb, struct usb_anchor *anchor)
{
- urb->anchor = NULL;
+ rcu_assign_pointer(urb->anchor, NULL);
list_del(&urb->anchor_list);
usb_put_urb(urb);
if (__usb_anchor_check_wakeup(anchor))
@@ -191,19 +235,28 @@ void usb_unanchor_urb(struct urb *urb)
if (!urb)
return;
- anchor = urb->anchor;
- if (!anchor)
- return;
-
- spin_lock_irqsave(&anchor->lock, flags);
/*
- * At this point, we could be competing with another thread which
- * has the same intention. To protect the urb from being unanchored
- * twice, only the winner of the race gets the job.
+ * The anchor pointer is read locklessly: a concurrent unanchor can
+ * empty the anchor and let a waiter return and free it before this
+ * thread manages to acquire anchor->lock. The RCU read-side
+ * critical section keeps the anchor memory alive across that gap;
+ * anchor lifetime gates (see usb_anchor_finish_wait()) wait for a
+ * grace period before any of them lets the caller free the anchor.
*/
- if (likely(anchor == urb->anchor))
- __usb_unanchor_urb(urb, anchor);
- spin_unlock_irqrestore(&anchor->lock, flags);
+ rcu_read_lock();
+ anchor = rcu_dereference(urb->anchor);
+ if (anchor) {
+ spin_lock_irqsave(&anchor->lock, flags);
+ /*
+ * At this point, we could be competing with another thread which
+ * has the same intention. To protect the urb from being unanchored
+ * twice, only the winner of the race gets the job.
+ */
+ if (likely(anchor == rcu_dereference(urb->anchor)))
+ __usb_unanchor_urb(urb, anchor);
+ spin_unlock_irqrestore(&anchor->lock, flags);
+ }
+ rcu_read_unlock();
}
EXPORT_SYMBOL_GPL(usb_unanchor_urb);
@@ -227,6 +280,13 @@ EXPORT_SYMBOL_GPL(usb_unanchor_urb);
* "@urb is on anchor->urb_list" are changed together only while
* holding anchor->lock.
*
+ * Like usb_unanchor_urb(), the initial urb->anchor read is protected by
+ * an RCU read-side critical section, so a concurrent unanchor cannot
+ * make the anchor vanish between the read and the lock acquisition:
+ * an anchor is only freed after a grace period that flushes readers
+ * of the anchoring being torn down. Anchoring new URBs concurrently
+ * would re-publish urb->anchor and is outside that teardown contract.
+ *
* Return: the anchor, which the caller must later pass to
* usb_anchor_resume_wakeups(), or NULL if @urb is not anchored
* anymore (a concurrent unanchor won the race).
@@ -236,18 +296,19 @@ struct usb_anchor *usb_unanchor_urb_and_suspend_wakeups(struct urb *urb)
struct usb_anchor *anchor;
unsigned long flags;
- anchor = urb->anchor;
- if (!anchor)
- return NULL;
-
- spin_lock_irqsave(&anchor->lock, flags);
- if (likely(anchor == urb->anchor)) {
- atomic_inc(&anchor->suspend_wakeups);
- __usb_unanchor_urb(urb, anchor);
- } else {
- anchor = NULL;
+ rcu_read_lock();
+ anchor = rcu_dereference(urb->anchor);
+ if (anchor) {
+ spin_lock_irqsave(&anchor->lock, flags);
+ if (likely(anchor == rcu_dereference(urb->anchor))) {
+ atomic_inc(&anchor->suspend_wakeups);
+ __usb_unanchor_urb(urb, anchor);
+ } else {
+ anchor = NULL;
+ }
+ spin_unlock_irqrestore(&anchor->lock, flags);
}
- spin_unlock_irqrestore(&anchor->lock, flags);
+ rcu_read_unlock();
return anchor;
}
@@ -873,15 +934,19 @@ EXPORT_SYMBOL_GPL(usb_block_urb);
* with guarantee that no completer callbacks will take place from the
* anchor after this function returns.
*
+ * It is safe to free the anchor (or the structure embedding it) once
+ * this function returns, provided that no new URBs are anchored to it
+ * concurrently. This function may sleep (it waits one RCU grace
+ * period on the success path).
+ *
* This routine should not be called by a driver after its disconnect
* method has returned.
*/
void usb_kill_anchored_urbs(struct usb_anchor *anchor)
{
struct urb *victim;
- int surely_empty;
- do {
+ for (;;) {
spin_lock_irq(&anchor->lock);
while (!list_empty(&anchor->urb_list)) {
victim = list_entry(anchor->urb_list.prev,
@@ -894,11 +959,24 @@ void usb_kill_anchored_urbs(struct usb_anchor *anchor)
usb_put_urb(victim);
spin_lock_irq(&anchor->lock);
}
- surely_empty = __usb_anchor_check_wakeup(anchor);
- spin_unlock_irq(&anchor->lock);
- cpu_relax();
- } while (!surely_empty);
+ if (__usb_anchor_check_wakeup(anchor)) {
+ /*
+ * Pin the anchor across one grace period before
+ * declaring it unused: a thread that has read
+ * urb->anchor but has not yet acquired anchor->lock
+ * must not touch the anchor after the caller frees
+ * it.
+ */
+ atomic_inc(&anchor->suspend_wakeups);
+ spin_unlock_irq(&anchor->lock);
+ if (usb_anchor_finish_wait(anchor))
+ return;
+ } else {
+ spin_unlock_irq(&anchor->lock);
+ cpu_relax();
+ }
+ }
}
EXPORT_SYMBOL_GPL(usb_kill_anchored_urbs);
@@ -911,15 +989,19 @@ EXPORT_SYMBOL_GPL(usb_kill_anchored_urbs);
* from the back of the queue. Newly added URBs will also be
* poisoned
*
+ * Like usb_kill_anchored_urbs(), it is safe to free the anchor once
+ * this function returns, provided that no new URBs are anchored to it
+ * concurrently. This function may sleep (it waits one RCU grace
+ * period on the success path).
+ *
* This routine should not be called by a driver after its disconnect
* method has returned.
*/
void usb_poison_anchored_urbs(struct usb_anchor *anchor)
{
struct urb *victim;
- int surely_empty;
- do {
+ for (;;) {
spin_lock_irq(&anchor->lock);
anchor->poisoned = 1;
while (!list_empty(&anchor->urb_list)) {
@@ -933,11 +1015,18 @@ void usb_poison_anchored_urbs(struct usb_anchor *anchor)
usb_put_urb(victim);
spin_lock_irq(&anchor->lock);
}
- surely_empty = __usb_anchor_check_wakeup(anchor);
- spin_unlock_irq(&anchor->lock);
- cpu_relax();
- } while (!surely_empty);
+ if (__usb_anchor_check_wakeup(anchor)) {
+ /* Same lifetime exit as usb_kill_anchored_urbs(). */
+ atomic_inc(&anchor->suspend_wakeups);
+ spin_unlock_irq(&anchor->lock);
+ if (usb_anchor_finish_wait(anchor))
+ return;
+ } else {
+ spin_unlock_irq(&anchor->lock);
+ cpu_relax();
+ }
+ }
}
EXPORT_SYMBOL_GPL(usb_poison_anchored_urbs);
@@ -967,8 +1056,11 @@ EXPORT_SYMBOL_GPL(usb_unpoison_anchored_urbs);
* @anchor: the anchor you want to suspend wakeups on
*
* Call this to stop the last urb being unanchored from waking up any
- * usb_wait_anchor_empty_timeout waiters. This is used in the hcd urb give-
- * back path to delay waking up until after the completion handler has run.
+ * usb_wait_anchor_empty_timeout waiters. The hcd urb give-back path,
+ * which delays waking up until after the completion handler has run,
+ * uses usb_unanchor_urb_and_suspend_wakeups() for that nowadays; this
+ * export remains for external users pairing it with
+ * usb_anchor_resume_wakeups().
*/
void usb_anchor_suspend_wakeups(struct usb_anchor *anchor)
{
@@ -1014,22 +1106,51 @@ EXPORT_SYMBOL_GPL(usb_anchor_resume_wakeups);
* Call this is you want to be sure all an anchor's
* URBs have finished
*
+ * A non-zero return also makes it safe to free the anchor (or the
+ * structure embedding it), provided that no new URBs are anchored to
+ * it concurrently. This function may sleep; in particular, a
+ * successful wait -- even with a zero timeout -- waits one RCU grace
+ * period before returning.
+ *
* Return: Non-zero if the anchor became unused. Zero on timeout.
*/
int usb_wait_anchor_empty_timeout(struct usb_anchor *anchor,
unsigned int timeout)
{
+ unsigned long deadline = jiffies + msecs_to_jiffies(timeout);
+ long ret;
+
/*
- * The condition must be evaluated under anchor->lock: when this
- * function returns non-zero, every URB giveback that has incremented
- * suspend_wakeups has finished with the anchor, so the caller may
- * free the anchor, which is typically embedded in the driver's
+ * The condition is evaluated and pinned under anchor->lock. When
+ * this function returns non-zero, every URB giveback that has
+ * incremented suspend_wakeups has finished with the anchor and one
+ * RCU grace period has flushed any reader that was still between
+ * reading urb->anchor and acquiring anchor->lock, so the caller
+ * may free the anchor -- provided that no new URBs are anchored
+ * concurrently -- which is typically embedded in the driver's
* private structure. A zero return (timeout) means a giveback may
* still be in flight and gives no such guarantee.
*/
- return wait_event_timeout(anchor->wait,
- usb_anchor_check_wakeup(anchor),
- msecs_to_jiffies(timeout));
+ for (;;) {
+ /*
+ * Clamped to zero so that a timeout of zero keeps its
+ * historical meaning of a single condition evaluation
+ * without entering the wait queue (a successful
+ * evaluation still waits one grace period before
+ * returning), and so the macro never sees a negative
+ * timeout after the deadline passes.
+ */
+ ret = wait_event_timeout(anchor->wait,
+ usb_anchor_pin_if_wakeup(anchor),
+ max_t(long, (long)(deadline - jiffies), 0));
+ if (!ret)
+ return 0;
+
+ if (usb_anchor_finish_wait(anchor))
+ return 1;
+ if ((long)(deadline - jiffies) <= 0)
+ return 0;
+ }
}
EXPORT_SYMBOL_GPL(usb_wait_anchor_empty_timeout);
@@ -1040,6 +1161,11 @@ EXPORT_SYMBOL_GPL(usb_wait_anchor_empty_timeout);
* This will take the oldest urb from an anchor,
* unanchor and return it
*
+ * A %NULL return does not mean the anchor is quiescent: a URB giveback
+ * may still be between its suspend_wakeups increment and its wakeup
+ * resume. Call usb_wait_anchor_empty_timeout() and check for a
+ * non-zero return before freeing the anchor.
+ *
* Return: The oldest urb from @anchor, or %NULL if @anchor has no
* urbs associated with it.
*/
@@ -1069,25 +1195,36 @@ EXPORT_SYMBOL_GPL(usb_get_from_anchor);
* @anchor: the anchor whose urbs you want to unanchor
*
* use this to get rid of all an anchor's urbs
+ *
+ * Like usb_kill_anchored_urbs(), it is safe to free the anchor once
+ * this function returns, provided that no new URBs are anchored to it
+ * concurrently. This function may sleep (it waits one RCU grace
+ * period on the success path).
*/
void usb_scuttle_anchored_urbs(struct usb_anchor *anchor)
{
struct urb *victim;
unsigned long flags;
- int surely_empty;
- do {
+ for (;;) {
spin_lock_irqsave(&anchor->lock, flags);
while (!list_empty(&anchor->urb_list)) {
victim = list_entry(anchor->urb_list.prev,
struct urb, anchor_list);
__usb_unanchor_urb(victim, anchor);
}
- surely_empty = __usb_anchor_check_wakeup(anchor);
- spin_unlock_irqrestore(&anchor->lock, flags);
- cpu_relax();
- } while (!surely_empty);
+ if (__usb_anchor_check_wakeup(anchor)) {
+ /* Same lifetime exit as usb_kill_anchored_urbs(). */
+ atomic_inc(&anchor->suspend_wakeups);
+ spin_unlock_irqrestore(&anchor->lock, flags);
+ if (usb_anchor_finish_wait(anchor))
+ return;
+ } else {
+ spin_unlock_irqrestore(&anchor->lock, flags);
+ cpu_relax();
+ }
+ }
}
EXPORT_SYMBOL_GPL(usb_scuttle_anchored_urbs);
@@ -1096,6 +1233,8 @@ EXPORT_SYMBOL_GPL(usb_scuttle_anchored_urbs);
* usb_anchor_empty - is an anchor empty
* @anchor: the anchor you want to query
*
+ * This is an unsynchronized peek; it gives no lifetime guarantee.
+ *
* Return: 1 if the anchor has no urbs associated with it.
*/
int usb_anchor_empty(struct usb_anchor *anchor)
diff --git a/include/linux/usb.h b/include/linux/usb.h
index 49ab8dbb885f..5adc502b4868 100644
--- a/include/linux/usb.h
+++ b/include/linux/usb.h
@@ -1445,7 +1445,10 @@ typedef void (*usb_complete_t)(struct urb *);
* struct urb - USB Request Block
* @urb_list: For use by current owner of the URB.
* @anchor_list: membership in the list of an anchor
- * @anchor: to anchor URBs to a common mooring
+ * @anchor: to anchor URBs to a common mooring; usbcore publishes it
+ * with rcu_assign_pointer() and consumes it under
+ * rcu_read_lock() with rcu_dereference(), see
+ * drivers/usb/core/urb.c for the anchor lifetime protocol
* @ep: Points to the endpoint's data structure. Will eventually
* replace @pipe.
* @pipe: Holds endpoint number, direction, type, and more.
@@ -1640,7 +1643,7 @@ struct urb {
struct list_head urb_list; /* list head for use by the urb's
* current owner */
struct list_head anchor_list; /* the URB may be anchored */
- struct usb_anchor *anchor;
+ struct usb_anchor __rcu *anchor;
struct usb_device *dev; /* (in) pointer to associated device */
struct usb_host_endpoint *ep; /* (internal) pointer to endpoint */
unsigned int pipe; /* (in) pipe information */
--
2.32.0
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH v2 0/3] usb: core: fix three use-after-free windows in the USB anchor protocol
2026-10-22 3:53 [PATCH 0/3] usb: core: fix three use-after-free windows in the USB anchor protocol Hongyu Xie
` (2 preceding siblings ...)
2026-10-22 3:53 ` [PATCH 3/3] usb: core: Close residual anchor read-to-lock window with RCU Hongyu Xie
@ 2026-11-03 2:20 ` Hongyu Xie
2026-11-03 2:20 ` [PATCH v2 1/3] usb: core: Fix use-after-free in anchor wakeup protocol Hongyu Xie
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Hongyu Xie @ 2026-11-03 2:20 UTC (permalink / raw)
To: Greg KH, Alan Stern; +Cc: linux-usb, Oliver Neukum, Hans de Goede, linux-kernel
Anchors are usually embedded in driver-private structures, and
several anchor API returns -- a non-zero
usb_wait_anchor_empty_timeout(), usb_kill_anchored_urbs(),
usb_poison_anchored_urbs() and usb_scuttle_anchored_urbs() --
license the caller to free that memory. The wakeup protocol backing
this license has three windows in which a URB giveback or unanchor
still touches the anchor after the caller has been told it is
unused:
1. The waiter's condition check and the giveback's
suspend_wakeups decrement plus wake_up() were not serialized,
so the waiter could return while a giveback was still in
flight. Reported by syzbot repeatedly since 2019; patch 1
references the two reports this analysis is built on.
2. __usb_hcd_giveback_urb() captured urb->anchor at the top of
the function but incremented suspend_wakeups only after DMA
unmapping and the usbmon hooks. A concurrent unanchor could
empty the anchor with suspend_wakeups still zero, so the waiter
returned and freed the anchor underneath the pending increment.
No syzbot report is attributed to this interleaving; it was
found by extending the analysis of patch 1.
3. usb_unanchor_urb() and the giveback read urb->anchor and only
then acquire anchor->lock; between the two the anchor can
already have been freed. The gap is as old as
usb_unanchor_urb() itself and cannot be closed with a
reference count: the first dereference of the freshly read
pointer would have to be the increment itself.
Patch 1 completes the locking protocol for the waiter/wakeup side.
Patch 2 folds the giveback's pointer capture, the increment and the
unanchor into a single anchor->lock critical section. Patch 3
turns the wakeup protocol into a lifetime protocol: the lockless
anchor reads run in RCU read-side critical sections, urb->anchor is
published with rcu_assign_pointer(), and every return that lets the
caller free the anchor first waits one synchronize_rcu() and
touches nothing on the anchor after it.
Patch 3 changes contracts, stated up front:
- usb_kill_anchored_urbs(), usb_poison_anchored_urbs() and
usb_scuttle_anchored_urbs() may now sleep even on an empty
anchor: the empty-anchor retry blocks on the anchor's wait queue
instead of busy-waiting. usb_scuttle_anchored_urbs() is the
only contract change among the three (the others already slept
transitively through usb_kill_urb()/usb_poison_urb()), and all
three carry a might_sleep() so an atomic-context caller is
caught at the call site with debugging configurations enabled.
All in-tree callers already sleep.
- usb_wait_anchor_empty_timeout() returns 1 on success instead of
the remaining jiffies, and a successful wait -- including one
with a zero timeout -- pays one RCU grace period. The condition
itself is unchanged from the wakeup-protocol fix.
- The safe-to-free license is per caller: a caller that frees on
its own successful return must first wait for any other
in-flight teardown calls to return. The kerneldoc of all four
teardown returns states this proviso alongside the no-new-URBs
one.
- The cost is one grace period per successful teardown exit. On
x86-64 real hardware, 100 consecutive successful waits on an
empty anchor cost 3.8 us per call on the base kernel and 3.9 us
with this series (+4.3%); per call site, a usbtmc synchronous
read converges on a kill and a scuttle and pays two grace
periods, btusb pays one per anchor it stops, and zd1211rw one
per command batch. It is small next to the URB I/O these paths
bracket, but it is the price of the free-after-return guarantee.
Why the retry loop must not busy-wait: on !PREEMPT_RCU kernels
whose scheduler tick reports no quiescent state for a spinning
process context (the v6.6 stable tree, for one; the 7.3 base of
this series adds process context only with CONFIG_PREEMPT_COUNT),
a busy retry stalls every grace period that still awaits that
CPU's quiescent state for as long as it spins, and under real-time
scheduling it can starve the very completion path it is waiting
for. On PREEMPT_RCU builds the tick reports from process context
and the window self-heals, which is why the v1 test kernels never
showed it. Current mainline x86 can no longer build a !PREEMPT_RCU
kernel at all (since 7dadeaa6e851); the exposure is the stable
backports of this series and older kernels.
The sleeping retry keeps the re-drain behavior of the busy loop it
replaces: usb_anchor_urb() wakes the anchor's wait queue (gated by
waitqueue_active(), so the per-submission hot path stays off the
wakeup machinery when no drain is sleeping; both the wakeup and the
sleepers' condition evaluation run under anchor->lock, so the gate
cannot lose a wakeup), and the wait also ends -- without declaring
the anchor unused -- when a URB is anchored after the drain, so the
drain restarts and tears the late arrival down too. A
late-anchored URB cannot strand the teardown.
No fields are added to struct usb_anchor or struct urb, and no
driver changes are needed.
Testing:
- A QEMU/KASAN attack harness reproduced each window separately
(deterministic arming for the two giveback windows, a forced
free inside the read-to-lock gap for the third) and verified
that each patch closes its own window with zero reports under
identical pressure. The v2 kernel reruns the full matrix with
zero reports and keeps the zero-timeout contract (empty anchor,
timeout 0, returns 1).
- The livelock boundary is driven directly: a waiter thread bound
to CPU0 calls usb_wait_anchor_empty_timeout() on a shared anchor
while a killer thread bound to CPU1 calls
usb_kill_anchored_urbs(), round after round, on v6.6 (the
stable-backport representative, where PREEMPT_NONE x86 is still
selectable) with PREEMPT_NONE and no PREEMPT_DYNAMIC. The v1
kernel wedges in the first waiter-first round -- kill rounds
stop completing and RCU stall warnings follow -- while the v2
kernel completes every round; both complete every round on
preemptible configs.
- The late-anchor liveness of the sleeping retry is probed
directly: a scuttle thread is parked in the empty-anchor wait
behind a manual suspend_wakeups pin, a never-submitted URB is
anchored after it sleeps, and the pin is dropped. The
intermediate v2 kernel (before the re-drain fix) never returns,
while the final v2 kernel wakes, drains the late arrival and
returns.
- The full stack also ran on real x86-64 hardware with KASAN
kernels built from the same base, on two machines. The base
kernel hits every probe: both deterministic windows fire within
100 iterations (hundreds of probe arm hits), and the natural
free-race variants hit within 2000 iterations with zero window
widening -- one of them with no arming at all on the second
machine. The kernel with all three patches runs the identical
workload with zero reports (the attack still executes there,
its probes arm hundreds of times too, only the windows are
gone), and completes the kill-in-parallel-with-wait rounds
(10/10) and the loser-path late-anchored-URB drain on real
hardware as well.
checkpatch: 0 errors, 0 warnings for all three patches.
Signed-off-by: Hongyu Xie <xiehongyu1@kylinos.cn>
---
Changes in v2 (patches 2 and 3; patch 1 is unchanged):
- Patch 2's usb_unanchor_urb_and_suspend_wakeups() race-loser
path cleared the local anchor variable before the shared
unlock, so the documented lose-the-race return unlocked
&NULL->lock -- a near-NULL fault with the real anchor left
locked. v2 unlocks, in each branch, the anchor that branch
locked. (Found by our own review of the v1 series.)
- The kill/poison/scuttle empty-anchor retry waits on the
anchor's wait queue and takes the same grace-period exit as
usb_wait_anchor_empty_timeout() instead of busy-waiting,
removing the self-deadlocking busy loop on !PREEMPT_RCU
kernels (review feedback on v1).
- The retry keeps the re-drain behavior of the busy loop:
usb_anchor_urb() wakes the anchor's wait queue and the
teardown wait ends on a late-anchored URB, so the drain
restarts for it instead of stranding a teardown that would
previously have killed it (review feedback on the v2 draft).
The wake is gated by waitqueue_active() so the per-submission
hot path stays off the wakeup machinery when no drain is
sleeping.
- The teardown functions no longer take suspend_wakeups pins of
their own: the pins of an earlier v2 draft let a concurrent
kill/poison/scuttle make a zero- or short-timeout wait on an
empty anchor spuriously return 0 (review feedback on v1). The
wait condition is again the giveback-only check, each teardown
exit waits its one grace period and touches nothing after it,
and concurrent teardown calls remain governed by the per-caller
proviso, as in the baseline.
- The safe-to-free kerneldoc of the four teardown returns now
also excludes concurrent in-flight teardown calls on the same
anchor, states the completion-callback recursion ban, and
notes that a manually held suspend_wakeups pin holds up a
teardown until released; kill/poison/scuttle carry
might_sleep() (for kill and poison it covers the empty-anchor
entry path); in-code comments spell out that
usb_anchor_check_wakeup()'s success does not by itself license
freeing, and why the waitqueue_active() gate is safe under
anchor->lock.
- The anchor documentation now matches the new contract: the
deallocation section of
Documentation/driver-api/usb/anchors.rst states that freeing
requires a successful teardown return with no concurrent
anchoring or in-flight teardown, and the wait section
describes the 0/1 return values and the one-grace-period
overrun.
- Cover letter: per-call-site grace-period cost enumeration, the
livelock test evidence, and the two-machine real-hardware
results.
Documentation/driver-api/usb/anchors.rst | 23 +-
drivers/usb/core/hcd.c | 10 +-
drivers/usb/core/urb.c | 388 ++++++++++++++++++++++++++++---
drivers/usb/core/usb.h | 1 +
include/linux/usb.h | 7 +-
5 files changed, 382 insertions(+), 47 deletions(-)
--
2.39.5 (Apple Git-150)
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH v2 1/3] usb: core: Fix use-after-free in anchor wakeup protocol
2026-10-22 3:53 [PATCH 0/3] usb: core: fix three use-after-free windows in the USB anchor protocol Hongyu Xie
` (3 preceding siblings ...)
2026-11-03 2:20 ` [PATCH v2 0/3] usb: core: fix three use-after-free windows in the USB anchor protocol Hongyu Xie
@ 2026-11-03 2:20 ` Hongyu Xie
2026-11-03 2:20 ` [PATCH v2 2/3] usb: core: Close anchor capture-to-increment race in URB giveback Hongyu Xie
2026-11-03 2:20 ` [PATCH v2 3/3] usb: core: Close residual anchor read-to-lock window with RCU Hongyu Xie
6 siblings, 0 replies; 8+ messages in thread
From: Hongyu Xie @ 2026-11-03 2:20 UTC (permalink / raw)
To: Greg KH, Alan Stern; +Cc: linux-usb, Oliver Neukum, Hans de Goede, linux-kernel
usb_wait_anchor_empty_timeout() and the anchor cleanup helpers
promise that once they return successfully (non-zero, for
usb_wait_anchor_empty_timeout()), no URB giveback is in flight for
the anchor any more, so the driver may free the memory holding it
(xpad, for example, embeds irq_out_anchor in its private structure
and kfrees it in xpad_disconnect()).
Two windows break this promise: the waiter can observe "anchor
list empty and suspend_wakeups == 0" while a giveback is still
about to access anchor->wait.
1. usb_anchor_resume_wakeups() decrements suspend_wakeups -- which
makes the wait condition true -- and only then calls
wake_up(&anchor->wait), with nothing in between serializing the
two. A waiter returns, the driver frees the anchor, and the
late wake_up() touches the freed wait queue head.
2. __usb_unanchor_urb() wakes waiters while holding anchor->lock,
but usb_wait_anchor_empty_timeout() evaluated the same condition
without the lock, so it could observe the emptied urb_list in
the middle of that critical section and return before the
wake_up() and the spin_unlock() had run. Reading the two
condition fields locklessly could likewise observe a stale
suspend_wakeups value after the giveback's unanchor on weakly
ordered systems.
Complete the locking protocol that commit fbc299437c06 ("usb:
core: Solve race condition in anchor cleanup functions")
established for the kill/poison/scuttle helpers:
usb_wait_anchor_empty_timeout() now evaluates the condition under
anchor->lock (a self-locking wrapper around the lockless check),
and usb_anchor_resume_wakeups() performs the suspend_wakeups
decrement, the wakeup check and the wake_up() within a single
anchor->lock critical section.
With the decrement and the wake_up() atomic with respect to the
locked condition check, a waiter can only observe the condition
true after every giveback that has incremented suspend_wakeups has
stopped accessing the anchor: the guarantee that the wakeup is
delayed until the completion handler has run now extends to the
anchor memory itself.
syzbot has reported this race repeatedly since 2019, most
recently as:
BUG: KASAN: slab-use-after-free in register_lock_class+0x8d1/0x980 kernel/locking/lockdep.c:1333
[...]
__wake_up_common_lock+0x25/0x1e0 kernel/sched/wait.c:105
__usb_hcd_giveback_urb+0x3ac/0x530 drivers/usb/core/hcd.c:1653
Freed by task 7828:
[...]
kfree+0x14a/0x380 mm/slub.c:4390
xpad_disconnect+0x359/0x490 drivers/input/joystick/xpad.c:2194
and:
BUG: KASAN: use-after-free in register_lock_class+0xe6a/0x1120 kernel/locking/lockdep.c:1336
[...]
__wake_up_common_lock+0xb4/0x130 kernel/sched/wait.c:136
usb_anchor_resume_wakeups drivers/usb/core/urb.c:958 [inline]
usb_anchor_resume_wakeups+0xbe/0xe0 drivers/usb/core/urb.c:951
__usb_hcd_giveback_urb+0x2df/0x5c0 drivers/usb/core/hcd.c:1674
Freed by task 3889:
[...]
kfree+0x173/0x390 mm/slab.c:3786
xpad_disconnect+0x1cb/0x530 drivers/input/joystick/xpad.c:1905
Both reports lack a reproducer; the analysis was verified
by walking the two stacks against the code, not at runtime. All
in-tree callers of usb_wait_anchor_empty_timeout() (and every
driver which frees an anchor after the anchor cleanup helpers) are
equally affected; xpad is merely the one syzbot found.
Fixes: 6ec4147e7bdb ("usb-anchor: Delay usb_wait_anchor_empty_timeout wake up till completion is done")
Fixes: 51a2f077c44e ("USB: introduce usb_anchor")
Reported-by: syzbot+7f5e98b052c766f47f68@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/r/00000000000090170a0614431f47@google.com
Reported-by: syzbot+59d3714b44d79d02f944@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/r/00000000000038435e05e9f94cd4@google.com
Cc: stable@vger.kernel.org
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Oliver Neukum <oliver@neukum.org>
Cc: Hans de Goede <hansg@kernel.org>
Signed-off-by: Hongyu Xie <xiehongyu1@kylinos.cn>
---
drivers/usb/core/urb.c | 51 +++++++++++++++++++++++++++++++++++++-----
1 file changed, 45 insertions(+), 6 deletions(-)
diff --git a/drivers/usb/core/urb.c b/drivers/usb/core/urb.c
index c06b44ca507b..3afd184c9b32 100644
--- a/drivers/usb/core/urb.c
+++ b/drivers/usb/core/urb.c
@@ -139,19 +139,39 @@ void usb_anchor_urb(struct urb *urb, struct usb_anchor *anchor)
}
EXPORT_SYMBOL_GPL(usb_anchor_urb);
-static int usb_anchor_check_wakeup(struct usb_anchor *anchor)
+/* Callers must hold anchor->lock */
+static int __usb_anchor_check_wakeup(struct usb_anchor *anchor)
{
return atomic_read(&anchor->suspend_wakeups) == 0 &&
list_empty(&anchor->urb_list);
}
+/*
+ * Same check as __usb_anchor_check_wakeup(), but takes anchor->lock. It is
+ * meant for waiters which do not hold the lock: the critical section
+ * serializes with the URB giveback path, so once the check succeeds no
+ * giveback that has incremented suspend_wakeups will touch the anchor
+ * anymore and the caller may free the memory holding the anchor.
+ */
+static int usb_anchor_check_wakeup(struct usb_anchor *anchor)
+{
+ unsigned long flags;
+ int wakeup;
+
+ spin_lock_irqsave(&anchor->lock, flags);
+ wakeup = __usb_anchor_check_wakeup(anchor);
+ spin_unlock_irqrestore(&anchor->lock, flags);
+
+ return wakeup;
+}
+
/* Callers must hold anchor->lock */
static void __usb_unanchor_urb(struct urb *urb, struct usb_anchor *anchor)
{
urb->anchor = NULL;
list_del(&urb->anchor_list);
usb_put_urb(urb);
- if (usb_anchor_check_wakeup(anchor))
+ if (__usb_anchor_check_wakeup(anchor))
wake_up(&anchor->wait);
}
@@ -827,7 +847,7 @@ void usb_kill_anchored_urbs(struct usb_anchor *anchor)
usb_put_urb(victim);
spin_lock_irq(&anchor->lock);
}
- surely_empty = usb_anchor_check_wakeup(anchor);
+ surely_empty = __usb_anchor_check_wakeup(anchor);
spin_unlock_irq(&anchor->lock);
cpu_relax();
@@ -866,7 +886,7 @@ void usb_poison_anchored_urbs(struct usb_anchor *anchor)
usb_put_urb(victim);
spin_lock_irq(&anchor->lock);
}
- surely_empty = usb_anchor_check_wakeup(anchor);
+ surely_empty = __usb_anchor_check_wakeup(anchor);
spin_unlock_irq(&anchor->lock);
cpu_relax();
@@ -919,12 +939,23 @@ EXPORT_SYMBOL_GPL(usb_anchor_suspend_wakeups);
*/
void usb_anchor_resume_wakeups(struct usb_anchor *anchor)
{
+ unsigned long flags;
+
if (!anchor)
return;
+ /*
+ * The decrement, the wakeup check and the wake_up() must happen
+ * under anchor->lock: usb_wait_anchor_empty_timeout() waiters
+ * evaluate the same condition under the lock and may free the
+ * anchor once they observe it true, so this function must not
+ * touch anchor->wait after that.
+ */
+ spin_lock_irqsave(&anchor->lock, flags);
atomic_dec(&anchor->suspend_wakeups);
- if (usb_anchor_check_wakeup(anchor))
+ if (__usb_anchor_check_wakeup(anchor))
wake_up(&anchor->wait);
+ spin_unlock_irqrestore(&anchor->lock, flags);
}
EXPORT_SYMBOL_GPL(usb_anchor_resume_wakeups);
@@ -941,6 +972,14 @@ EXPORT_SYMBOL_GPL(usb_anchor_resume_wakeups);
int usb_wait_anchor_empty_timeout(struct usb_anchor *anchor,
unsigned int timeout)
{
+ /*
+ * The condition must be evaluated under anchor->lock: when this
+ * function returns non-zero, every URB giveback that has incremented
+ * suspend_wakeups has finished with the anchor, so the caller may
+ * free the anchor, which is typically embedded in the driver's
+ * private structure. A zero return (timeout) means a giveback may
+ * still be in flight and gives no such guarantee.
+ */
return wait_event_timeout(anchor->wait,
usb_anchor_check_wakeup(anchor),
msecs_to_jiffies(timeout));
@@ -997,7 +1036,7 @@ void usb_scuttle_anchored_urbs(struct usb_anchor *anchor)
struct urb, anchor_list);
__usb_unanchor_urb(victim, anchor);
}
- surely_empty = usb_anchor_check_wakeup(anchor);
+ surely_empty = __usb_anchor_check_wakeup(anchor);
spin_unlock_irqrestore(&anchor->lock, flags);
cpu_relax();
base-commit: 22430ae5d90ab288b0ee2ad99ae941f4a666b694
--
2.32.0
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH v2 2/3] usb: core: Close anchor capture-to-increment race in URB giveback
2026-10-22 3:53 [PATCH 0/3] usb: core: fix three use-after-free windows in the USB anchor protocol Hongyu Xie
` (4 preceding siblings ...)
2026-11-03 2:20 ` [PATCH v2 1/3] usb: core: Fix use-after-free in anchor wakeup protocol Hongyu Xie
@ 2026-11-03 2:20 ` Hongyu Xie
2026-11-03 2:20 ` [PATCH v2 3/3] usb: core: Close residual anchor read-to-lock window with RCU Hongyu Xie
6 siblings, 0 replies; 8+ messages in thread
From: Hongyu Xie @ 2026-11-03 2:20 UTC (permalink / raw)
To: Greg KH, Alan Stern; +Cc: linux-usb, Oliver Neukum, Hans de Goede, linux-kernel
__usb_hcd_giveback_urb() captures urb->anchor into a local
variable at the top of the function, but increments the anchor's
suspend_wakeups counter only after unmapping the URB for DMA and
running the usbmon hooks. Nothing pins the anchor during that
window. If the URB is concurrently unanchored by another thread --
a driver calling usb_unanchor_urb() on its own in-flight URB, or
usb_get_from_anchor() or usb_scuttle_anchored_urbs() taking it --
the anchor list can become empty while suspend_wakeups is still
zero. A usb_wait_anchor_empty_timeout() waiter then observes the
condition true, returns, and the driver frees the anchor (anchors
are typically embedded in driver-private structures); the giveback's
atomic_inc() on the captured pointer, and the later decrement and
wake_up() in usb_anchor_resume_wakeups(), operate on freed memory.
The previous fix ("usb: core: Fix use-after-free in anchor wakeup
protocol") made both the waiter's condition check and the wakeup
side hold anchor->lock, but its guarantee was deliberately scoped
to givebacks that had already incremented suspend_wakeups: the
capture-to-increment gap remained open.
Fold the pointer capture, the increment and the unanchor into a
single anchor->lock critical section which re-verifies that
urb->anchor still names the captured anchor, so a giveback that
lost the race to a concurrent unanchor performs no anchor work at
all. From the verified increment onward, a non-zero
suspend_wakeups keeps waiters from freeing the anchor until
usb_anchor_resume_wakeups() has finished with it.
What remains open is the few instructions between reading
urb->anchor and acquiring anchor->lock inside the new helper: a
concurrent unanchor plus waiter return plus anchor free would all
have to fit into that gap for the lock acquisition itself to touch
freed memory. This is the same residual exposure
usb_unanchor_urb() has had since usb_anchor was introduced; closing
it entirely would require a lifetime protocol change (reference
count or RCU) for anchors, which is out of scope for a bugfix.
No syzbot report has been attributed to this interleaving; it was
found by extending the analysis of the previous fix.
Fixes: 6ec4147e7bdb ("usb-anchor: Delay usb_wait_anchor_empty_timeout wake up till completion is done")
Cc: stable@vger.kernel.org
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Oliver Neukum <oliver@neukum.org>
Cc: Hans de Goede <hansg@kernel.org>
Signed-off-by: Hongyu Xie <xiehongyu1@kylinos.cn>
---
drivers/usb/core/hcd.c | 10 ++++++---
drivers/usb/core/urb.c | 48 ++++++++++++++++++++++++++++++++++++++++++
drivers/usb/core/usb.h | 1 +
3 files changed, 56 insertions(+), 3 deletions(-)
diff --git a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c
index 8e8a5f59b319..f221425fec0a 100644
--- a/drivers/usb/core/hcd.c
+++ b/drivers/usb/core/hcd.c
@@ -1630,7 +1630,7 @@ int usb_hcd_unlink_urb (struct urb *urb, int status)
static void __usb_hcd_giveback_urb(struct urb *urb)
{
struct usb_hcd *hcd = bus_to_hcd(urb->dev->bus);
- struct usb_anchor *anchor = urb->anchor;
+ struct usb_anchor *anchor;
int status = urb->unlinked;
urb->hcpriv = NULL;
@@ -1641,8 +1641,12 @@ static void __usb_hcd_giveback_urb(struct urb *urb)
unmap_urb_for_dma(hcd, urb);
usbmon_urb_complete(&hcd->self, urb, status);
- usb_anchor_suspend_wakeups(anchor);
- usb_unanchor_urb(urb);
+ /*
+ * Capturing the anchor, suspending its wakeups and unanchoring
+ * must be atomic with each other; see
+ * usb_unanchor_urb_and_suspend_wakeups().
+ */
+ anchor = usb_unanchor_urb_and_suspend_wakeups(urb);
if (likely(status == 0))
usb_led_activity(USB_LED_EVENT_HOST);
diff --git a/drivers/usb/core/urb.c b/drivers/usb/core/urb.c
index 3afd184c9b32..22312892e528 100644
--- a/drivers/usb/core/urb.c
+++ b/drivers/usb/core/urb.c
@@ -14,6 +14,8 @@
#include <linux/usb/hcd.h>
#include <linux/scatterlist.h>
+#include "usb.h"
+
#define to_urb(d) container_of(d, struct urb, kref)
@@ -205,6 +207,52 @@ void usb_unanchor_urb(struct urb *urb)
}
EXPORT_SYMBOL_GPL(usb_unanchor_urb);
+/*
+ * usb_unanchor_urb_and_suspend_wakeups - atomically unanchor @urb and
+ * increment the suspend_wakeups counter of its anchor.
+ * @urb: the URB to unanchor
+ *
+ * Used by the URB giveback path, which must capture urb->anchor and
+ * increment anchor->suspend_wakeups atomically with the removal of
+ * @urb from anchor->urb_list. Otherwise a concurrent unanchor of
+ * @urb (a driver calling usb_unanchor_urb() on its own in-flight URB,
+ * or usb_get_from_anchor() or usb_scuttle_anchored_urbs() taking it)
+ * can leave the anchor with an empty urb_list while suspend_wakeups
+ * is still zero, so a usb_wait_anchor_empty_timeout() waiter observes
+ * the wait condition true, returns, and the driver may free the
+ * anchor before the giveback gets to touch it.
+ *
+ * The verification of urb->anchor under anchor->lock makes this safe
+ * against racing re-anchoring, because "urb->anchor == anchor" and
+ * "@urb is on anchor->urb_list" are changed together only while
+ * holding anchor->lock.
+ *
+ * Return: the anchor, which the caller must later pass to
+ * usb_anchor_resume_wakeups(), or NULL if @urb is not anchored
+ * anymore (a concurrent unanchor won the race).
+ */
+struct usb_anchor *usb_unanchor_urb_and_suspend_wakeups(struct urb *urb)
+{
+ struct usb_anchor *anchor;
+ unsigned long flags;
+
+ anchor = urb->anchor;
+ if (!anchor)
+ return NULL;
+
+ spin_lock_irqsave(&anchor->lock, flags);
+ if (likely(anchor == urb->anchor)) {
+ atomic_inc(&anchor->suspend_wakeups);
+ __usb_unanchor_urb(urb, anchor);
+ spin_unlock_irqrestore(&anchor->lock, flags);
+ } else {
+ spin_unlock_irqrestore(&anchor->lock, flags);
+ anchor = NULL;
+ }
+
+ return anchor;
+}
+
/*-------------------------------------------------------------------*/
static const int pipetypes[4] = {
diff --git a/drivers/usb/core/usb.h b/drivers/usb/core/usb.h
index a9b37aeb515b..362bfda497cc 100644
--- a/drivers/usb/core/usb.h
+++ b/drivers/usb/core/usb.h
@@ -20,6 +20,7 @@ extern int usb_create_ep_devs(struct device *parent,
struct usb_host_endpoint *endpoint,
struct usb_device *udev);
extern void usb_remove_ep_devs(struct usb_host_endpoint *endpoint);
+extern struct usb_anchor *usb_unanchor_urb_and_suspend_wakeups(struct urb *urb);
extern void usb_enable_endpoint(struct usb_device *dev,
struct usb_host_endpoint *ep, bool reset_toggle);
--
2.32.0
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH v2 3/3] usb: core: Close residual anchor read-to-lock window with RCU
2026-10-22 3:53 [PATCH 0/3] usb: core: fix three use-after-free windows in the USB anchor protocol Hongyu Xie
` (5 preceding siblings ...)
2026-11-03 2:20 ` [PATCH v2 2/3] usb: core: Close anchor capture-to-increment race in URB giveback Hongyu Xie
@ 2026-11-03 2:20 ` Hongyu Xie
6 siblings, 0 replies; 8+ messages in thread
From: Hongyu Xie @ 2026-11-03 2:20 UTC (permalink / raw)
To: Greg KH, Alan Stern; +Cc: linux-usb, Oliver Neukum, Hans de Goede, linux-kernel
usb_unanchor_urb() and the URB giveback read urb->anchor and only
then acquire anchor->lock. Between the read and the lock
acquisition the anchor can already have been freed: a concurrent
unanchor of the last URB empties the anchor, a
usb_wait_anchor_empty_timeout() waiter observes the condition true,
returns, and the driver frees the memory holding the anchor; the
spin_lock() then touches freed memory. This gap is as old as
usb_unanchor_urb() itself and was deliberately left open by the
capture-to-increment fix, whose guarantee is scoped to givebacks
that have already incremented suspend_wakeups.
A plain reference count cannot close it: the first dereference of
the freshly read anchor pointer would have to be the refcount
increment itself, so the pin would be taken on memory that may
already have been freed. Protecting the pointer read requires
deferred reclamation, and because anchors are embedded in
driver-private structures and freed with plain kfree() right after
a successful wait, the deferral has to live on the side usbcore
controls: the return of the wait.
Convert the anchor wakeup protocol into a lifetime protocol:
- The lockless urb->anchor reads (usb_unanchor_urb() and
usb_unanchor_urb_and_suspend_wakeups()) run inside RCU
read-side critical sections, and urb->anchor is published with
rcu_assign_pointer().
- Every function whose return lets the caller free the anchor --
usb_wait_anchor_empty_timeout() on a non-zero return,
usb_kill_anchored_urbs(), usb_poison_anchored_urbs() and
usb_scuttle_anchored_urbs() -- waits one synchronize_rcu()
before returning, and touches nothing on the anchor after it.
These are teardown guarantees: provided the caller stops
anchoring new URBs, readers that read urb->anchor before the
grace period have acquired and released anchor->lock by the
time the caller may free the anchor, and readers after it find
urb->anchor == NULL and never touch the anchor at all. (A
concurrent usb_anchor_urb() would re-publish urb->anchor; the
protocol does not police the caller's own anchoring.) The
license is per caller: another teardown call may still be
sleeping on the anchor's wait queue, so a caller that frees on
its own success must first let other in-flight teardown calls
return. And as with any wait on the anchor, these calls must
not be made from the completion callback of a URB on the same
anchor: the giveback holds its suspend_wakeups pin until after
the callback returns, so such a call would wait for itself.
The empty-anchor retry of kill/poison/scuttle waits on the
anchor's wait queue and takes the same grace-period exit as
usb_wait_anchor_empty_timeout() instead of busy-waiting. On
!PREEMPT_RCU kernels whose scheduler tick reports no quiescent
state for a spinning process context, a busy retry keeps this CPU
from reporting a quiescent state and stalls every grace period
that still awaits this CPU's quiescent state -- a concurrent
teardown's grace-period exit included -- and under real-time
scheduling it can starve the very completion path it is waiting
for. The retry keeps the re-drain behavior of the busy loop it
replaces: usb_anchor_urb() wakes the anchor's wait queue, and the
wait also ends -- without declaring the anchor unused -- when a
URB is anchored after the drain, so the loop restarts and tears
the late arrival down too.
usb_wait_anchor_empty_timeout() keeps its documented contract: a
zero timeout still performs a single condition evaluation without
entering the wait queue -- a successful evaluation still waits one
grace period before returning -- and a successful wait returns 1
rather than the remaining jiffies. The wait may overrun the
caller's timeout by up to one grace period, because the grace
period is waited after the condition is observed true: a non-zero
return must carry the safe-to-free guarantee.
The giveback's long window across the completion handler keeps
the suspend_wakeups pin it already takes. No fields are added to
struct usb_anchor or struct urb and no driver changes are
required. The cost is one RCU grace period per successful
teardown exit -- including teardown of an already-empty anchor,
which previously returned without one -- which cannot be elided
for a caller that frees the anchor without giving up the
safe-to-free guarantee of the non-zero return; callers that keep
the anchor alive across teardowns pay it without needing it
(usbtmc waits on its submission anchor, btusb stops five anchors
in a row, and a usbtmc synchronous read converges on a kill and a
scuttle, paying two grace periods per read).
usb_kill_anchored_urbs(), usb_poison_anchored_urbs() and
usb_scuttle_anchored_urbs() must now be called from a context
that can sleep (all in-tree callers already do). Only
usb_scuttle_anchored_urbs() changes contract, because the other
two already slept transitively through usb_kill_urb() and
usb_poison_urb(). All three carry a might_sleep() -- for kill
and poison it covers the empty-anchor entry path, which returns
through the grace-period exit without ever reaching
usb_kill_urb() or usb_poison_urb() -- so that an atomic-context
caller is caught at the call site with debugging configurations
enabled. The per-URB hot paths gain rcu_read_lock()/
rcu_dereference() around the lockless anchor reads,
rcu_assign_pointer() for the anchor pointer updates, and one
waitqueue_active()-gated wakeup in usb_anchor_urb(), which lets a
teardown drain sleeping on the anchor notice a late-anchored URB
without putting an unconditional wake_up() under the lock on
every submission.
The deallocation and usb_wait_anchor_empty_timeout() sections of
Documentation/driver-api/usb/anchors.rst are updated to describe
the grace-period teardown guarantee and the new wait return
semantics.
Fixes: 51a2f077c44e ("USB: introduce usb_anchor")
Cc: stable@vger.kernel.org
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Oliver Neukum <oliver@neukum.org>
Cc: Hans de Goede <hansg@kernel.org>
Signed-off-by: Hongyu Xie <xiehongyu1@kylinos.cn>
---
Documentation/driver-api/usb/anchors.rst | 23 +-
drivers/usb/core/urb.c | 329 +++++++++++++++++++----
include/linux/usb.h | 7 +-
3 files changed, 301 insertions(+), 58 deletions(-)
diff --git a/Documentation/driver-api/usb/anchors.rst b/Documentation/driver-api/usb/anchors.rst
index 5a93d171e76c..1715a8a5aecd 100644
--- a/Documentation/driver-api/usb/anchors.rst
+++ b/Documentation/driver-api/usb/anchors.rst
@@ -22,8 +22,15 @@ initialise the data structure.
Deallocation
============
-Once it has no more URBs associated with it, the anchor can be
-freed with normal memory management operations.
+Once it has no more URBs associated with it, and one of
+usb_wait_anchor_empty_timeout(), usb_kill_anchored_urbs(),
+usb_poison_anchored_urbs() or usb_scuttle_anchored_urbs() has returned
+successfully, the anchor can be freed with normal memory management
+operations. The successful return drains readers that may still hold
+the anchor pointer by waiting one RCU grace period. This holds only
+for the caller's own teardown: the caller must have stopped anchoring
+new URBs, and must not free the anchor while another call on it is
+still in flight.
Association and disassociation of URBs with anchors
===================================================
@@ -48,14 +55,20 @@ This way no data can be reordered.
:c:func:`usb_scuttle_anchored_urbs`
-----------------------------------
-All URBs of an anchor are unanchored en masse.
+All URBs of an anchor are unanchored en masse. This function may
+sleep; an empty anchor is torn down through the same grace-period
+exit as a non-empty one.
:c:func:`usb_wait_anchor_empty_timeout`
---------------------------------------
This function waits for all URBs associated with an anchor to finish
-or a timeout, whichever comes first. Its return value will tell you
-whether the timeout was reached.
+or a timeout, whichever comes first. It returns 0 if the timeout was
+reached and 1 if the anchor drained. A successful wait -- even one
+whose condition was already true, with a zero timeout -- waits one RCU
+grace period before returning, so it may overrun the caller's timeout
+by up to one grace period; this is what makes it safe for the caller
+to free the anchor on a non-zero return.
:c:func:`usb_anchor_empty`
--------------------------
diff --git a/drivers/usb/core/urb.c b/drivers/usb/core/urb.c
index 22312892e528..ea3b0736e80e 100644
--- a/drivers/usb/core/urb.c
+++ b/drivers/usb/core/urb.c
@@ -9,6 +9,7 @@
#include <linux/slab.h>
#include <linux/log2.h>
#include <linux/kmsan.h>
+#include <linux/rcupdate.h>
#include <linux/usb.h>
#include <linux/wait.h>
#include <linux/usb/hcd.h>
@@ -132,11 +133,34 @@ void usb_anchor_urb(struct urb *urb, struct usb_anchor *anchor)
spin_lock_irqsave(&anchor->lock, flags);
usb_get_urb(urb);
list_add_tail(&urb->anchor_list, &anchor->urb_list);
- urb->anchor = anchor;
+ rcu_assign_pointer(urb->anchor, anchor);
if (unlikely(anchor->poisoned))
atomic_inc(&urb->reject);
+ /*
+ * Wake a teardown drain sleeping on this anchor: the late URB
+ * makes the drain's wait condition true again, so the wait
+ * ends with the anchor not unused and the drain restarts to
+ * take care of the late arrival. The waitqueue_active()
+ * gate keeps the common no-drain submission off the wakeup
+ * path; it cannot lose this wakeup, because a drain that
+ * enqueues only after this critical section re-checks the
+ * list under anchor->lock and restarts on its own (see
+ * usb_anchor_teardown_progress()).
+ *
+ * The gate deviates from the two usages wait.h sanctions for
+ * waitqueue_active() (the waitqueue lock, or an explicit
+ * smp_mb()): here it is evaluated inside the same
+ * anchor->lock critical section as the list_add_tail() above,
+ * and the drains evaluate their wait condition under that
+ * lock too, so the store and the gate load cannot be
+ * reordered apart. Hoisting the gate out of the lock would
+ * silently lose wakeups.
+ */
+ if (waitqueue_active(&anchor->wait))
+ wake_up(&anchor->wait);
+
spin_unlock_irqrestore(&anchor->lock, flags);
}
EXPORT_SYMBOL_GPL(usb_anchor_urb);
@@ -153,7 +177,11 @@ static int __usb_anchor_check_wakeup(struct usb_anchor *anchor)
* meant for waiters which do not hold the lock: the critical section
* serializes with the URB giveback path, so once the check succeeds no
* giveback that has incremented suspend_wakeups will touch the anchor
- * anymore and the caller may free the memory holding the anchor.
+ * anymore. This does not by itself make freeing safe: the caller must
+ * still wait one RCU grace period for readers that are between reading
+ * urb->anchor and acquiring anchor->lock, and must not anchor new URBs
+ * concurrently; see usb_wait_anchor_empty_timeout() for the full
+ * contract.
*/
static int usb_anchor_check_wakeup(struct usb_anchor *anchor)
{
@@ -167,10 +195,47 @@ static int usb_anchor_check_wakeup(struct usb_anchor *anchor)
return wakeup;
}
+/*
+ * Wait-condition helper for the teardown drain loops: the wait ends
+ * either with the anchor observed unused under anchor->lock (*unused
+ * is set and the caller takes the grace-period exit), or -- without
+ * that observation -- when a URB was anchored after the drain, so the
+ * caller returns to the top of its loop and drains the late arrival.
+ * usb_anchor_urb() wakes the anchor's wait queue for the second case,
+ * and the lockless list read is only a fast path: a stale non-empty
+ * entry merely restarts the drain, and a stale empty one is caught by
+ * the locked re-check below, so neither direction can lose the
+ * restart.
+ */
+static bool usb_anchor_teardown_progress(struct usb_anchor *anchor,
+ bool *unused)
+{
+ unsigned long flags;
+ bool ret;
+
+ if (!list_empty_careful(&anchor->urb_list)) {
+ *unused = false;
+ return true;
+ }
+
+ spin_lock_irqsave(&anchor->lock, flags);
+ if (!list_empty(&anchor->urb_list)) {
+ /* A late URB landed: restart the drain for it. */
+ *unused = false;
+ ret = true;
+ } else {
+ *unused = __usb_anchor_check_wakeup(anchor);
+ ret = *unused;
+ }
+ spin_unlock_irqrestore(&anchor->lock, flags);
+
+ return ret;
+}
+
/* Callers must hold anchor->lock */
static void __usb_unanchor_urb(struct urb *urb, struct usb_anchor *anchor)
{
- urb->anchor = NULL;
+ rcu_assign_pointer(urb->anchor, NULL);
list_del(&urb->anchor_list);
usb_put_urb(urb);
if (__usb_anchor_check_wakeup(anchor))
@@ -191,19 +256,28 @@ void usb_unanchor_urb(struct urb *urb)
if (!urb)
return;
- anchor = urb->anchor;
- if (!anchor)
- return;
-
- spin_lock_irqsave(&anchor->lock, flags);
/*
- * At this point, we could be competing with another thread which
- * has the same intention. To protect the urb from being unanchored
- * twice, only the winner of the race gets the job.
+ * The anchor pointer is read locklessly: a concurrent unanchor can
+ * empty the anchor and let a waiter return and free it before this
+ * thread manages to acquire anchor->lock. The RCU read-side
+ * critical section keeps the anchor memory alive across that gap;
+ * every anchor teardown exit waits for a grace period before it
+ * lets the caller free the anchor.
*/
- if (likely(anchor == urb->anchor))
- __usb_unanchor_urb(urb, anchor);
- spin_unlock_irqrestore(&anchor->lock, flags);
+ rcu_read_lock();
+ anchor = rcu_dereference(urb->anchor);
+ if (anchor) {
+ spin_lock_irqsave(&anchor->lock, flags);
+ /*
+ * At this point, we could be competing with another thread which
+ * has the same intention. To protect the urb from being unanchored
+ * twice, only the winner of the race gets the job.
+ */
+ if (likely(anchor == rcu_dereference(urb->anchor)))
+ __usb_unanchor_urb(urb, anchor);
+ spin_unlock_irqrestore(&anchor->lock, flags);
+ }
+ rcu_read_unlock();
}
EXPORT_SYMBOL_GPL(usb_unanchor_urb);
@@ -227,6 +301,13 @@ EXPORT_SYMBOL_GPL(usb_unanchor_urb);
* "@urb is on anchor->urb_list" are changed together only while
* holding anchor->lock.
*
+ * Like usb_unanchor_urb(), the initial urb->anchor read is protected by
+ * an RCU read-side critical section, so a concurrent unanchor cannot
+ * make the anchor vanish between the read and the lock acquisition:
+ * an anchor is only freed after a grace period that flushes readers
+ * of the anchoring being torn down. Anchoring new URBs concurrently
+ * would re-publish urb->anchor and is outside that teardown contract.
+ *
* Return: the anchor, which the caller must later pass to
* usb_anchor_resume_wakeups(), or NULL if @urb is not anchored
* anymore (a concurrent unanchor won the race).
@@ -236,19 +317,20 @@ struct usb_anchor *usb_unanchor_urb_and_suspend_wakeups(struct urb *urb)
struct usb_anchor *anchor;
unsigned long flags;
- anchor = urb->anchor;
- if (!anchor)
- return NULL;
-
- spin_lock_irqsave(&anchor->lock, flags);
- if (likely(anchor == urb->anchor)) {
- atomic_inc(&anchor->suspend_wakeups);
- __usb_unanchor_urb(urb, anchor);
- spin_unlock_irqrestore(&anchor->lock, flags);
- } else {
- spin_unlock_irqrestore(&anchor->lock, flags);
- anchor = NULL;
+ rcu_read_lock();
+ anchor = rcu_dereference(urb->anchor);
+ if (anchor) {
+ spin_lock_irqsave(&anchor->lock, flags);
+ if (likely(anchor == rcu_dereference(urb->anchor))) {
+ atomic_inc(&anchor->suspend_wakeups);
+ __usb_unanchor_urb(urb, anchor);
+ spin_unlock_irqrestore(&anchor->lock, flags);
+ } else {
+ spin_unlock_irqrestore(&anchor->lock, flags);
+ anchor = NULL;
+ }
}
+ rcu_read_unlock();
return anchor;
}
@@ -874,15 +956,30 @@ EXPORT_SYMBOL_GPL(usb_block_urb);
* with guarantee that no completer callbacks will take place from the
* anchor after this function returns.
*
+ * It is safe to free the anchor (or the structure embedding it) once
+ * this function returns, provided that no new URBs are anchored to it
+ * concurrently and no other teardown call (a kill, poison, scuttle or
+ * wait on the same anchor) is still in flight: such a call may be
+ * sleeping on the anchor's wait queue, which lives in the memory to
+ * be freed. This function may sleep (it waits one RCU grace period
+ * on the success path).
+ *
+ * This function must not be called from the completion callback of a
+ * URB on @anchor: the callback runs while the giveback still holds
+ * the anchor's suspend_wakeups pin, so the drain would wait for
+ * itself.
+ *
* This routine should not be called by a driver after its disconnect
* method has returned.
*/
void usb_kill_anchored_urbs(struct usb_anchor *anchor)
{
struct urb *victim;
- int surely_empty;
+ bool unused;
- do {
+ might_sleep();
+
+ for (;;) {
spin_lock_irq(&anchor->lock);
while (!list_empty(&anchor->urb_list)) {
victim = list_entry(anchor->urb_list.prev,
@@ -895,11 +992,47 @@ void usb_kill_anchored_urbs(struct usb_anchor *anchor)
usb_put_urb(victim);
spin_lock_irq(&anchor->lock);
}
- surely_empty = __usb_anchor_check_wakeup(anchor);
+
+ if (__usb_anchor_check_wakeup(anchor)) {
+ spin_unlock_irq(&anchor->lock);
+ /*
+ * One grace period before declaring the anchor
+ * unused and returning: a thread that has read
+ * urb->anchor but has not yet acquired anchor->lock
+ * must not touch the anchor after the caller frees
+ * it. Nothing in this path may touch the anchor
+ * after this point.
+ */
+ synchronize_rcu();
+ return;
+ }
spin_unlock_irq(&anchor->lock);
- cpu_relax();
- } while (!surely_empty);
+ /*
+ * The anchor is empty but a giveback still holds a
+ * suspend_wakeups pin. Busy-waiting for it would
+ * keep this CPU from reporting a quiescent state on
+ * !PREEMPT_RCU kernels whose scheduler tick reports
+ * none for a spinning process context, stalling every
+ * grace period that still awaits this CPU's quiescent
+ * state -- a concurrent teardown's grace-period exit
+ * included -- and under real-time scheduling it can
+ * starve the very completion path it is waiting for.
+ * Wait for the wakeup protocol to signal the state
+ * change and take the same grace-period exit as
+ * usb_wait_anchor_empty_timeout(). The wait also
+ * ends without an unused observation when a URB is
+ * anchored after the drain, so the loop restarts and
+ * kills it, like the busy loop this replaced did.
+ */
+ wait_event(anchor->wait,
+ usb_anchor_teardown_progress(anchor, &unused));
+ if (unused) {
+ /* Same lifetime exit as above. */
+ synchronize_rcu();
+ return;
+ }
+ }
}
EXPORT_SYMBOL_GPL(usb_kill_anchored_urbs);
@@ -912,15 +1045,27 @@ EXPORT_SYMBOL_GPL(usb_kill_anchored_urbs);
* from the back of the queue. Newly added URBs will also be
* poisoned
*
+ * Like usb_kill_anchored_urbs(), it is safe to free the anchor once
+ * this function returns, provided that no new URBs are anchored to it
+ * concurrently and no other teardown call on the same anchor is still
+ * in flight. This function may sleep (it waits one RCU grace period
+ * on the success path).
+ *
+ * Like usb_kill_anchored_urbs(), it must not be called from the
+ * completion callback of a URB on @anchor: the drain would wait for
+ * the pin its own giveback holds.
+ *
* This routine should not be called by a driver after its disconnect
* method has returned.
*/
void usb_poison_anchored_urbs(struct usb_anchor *anchor)
{
struct urb *victim;
- int surely_empty;
+ bool unused;
+
+ might_sleep();
- do {
+ for (;;) {
spin_lock_irq(&anchor->lock);
anchor->poisoned = 1;
while (!list_empty(&anchor->urb_list)) {
@@ -934,11 +1079,24 @@ void usb_poison_anchored_urbs(struct usb_anchor *anchor)
usb_put_urb(victim);
spin_lock_irq(&anchor->lock);
}
- surely_empty = __usb_anchor_check_wakeup(anchor);
+
+ if (__usb_anchor_check_wakeup(anchor)) {
+ spin_unlock_irq(&anchor->lock);
+ /* Same lifetime exit as usb_kill_anchored_urbs(). */
+ synchronize_rcu();
+ return;
+ }
spin_unlock_irq(&anchor->lock);
- cpu_relax();
- } while (!surely_empty);
+ /* Same sleepable retry as usb_kill_anchored_urbs(). */
+ wait_event(anchor->wait,
+ usb_anchor_teardown_progress(anchor, &unused));
+ if (unused) {
+ /* Same lifetime exit as usb_kill_anchored_urbs(). */
+ synchronize_rcu();
+ return;
+ }
+ }
}
EXPORT_SYMBOL_GPL(usb_poison_anchored_urbs);
@@ -968,8 +1126,15 @@ EXPORT_SYMBOL_GPL(usb_unpoison_anchored_urbs);
* @anchor: the anchor you want to suspend wakeups on
*
* Call this to stop the last urb being unanchored from waking up any
- * usb_wait_anchor_empty_timeout waiters. This is used in the hcd urb give-
- * back path to delay waking up until after the completion handler has run.
+ * usb_wait_anchor_empty_timeout waiters. The hcd urb give-back path,
+ * which delays waking up until after the completion handler has run,
+ * uses usb_unanchor_urb_and_suspend_wakeups() for that nowadays; this
+ * export remains for external users pairing it with
+ * usb_anchor_resume_wakeups().
+ *
+ * Note that a manually held pin also holds up anchor teardown: a
+ * kill, poison, scuttle or wait call on the same anchor will not
+ * return while the pin is outstanding.
*/
void usb_anchor_suspend_wakeups(struct usb_anchor *anchor)
{
@@ -1008,29 +1173,59 @@ void usb_anchor_resume_wakeups(struct usb_anchor *anchor)
EXPORT_SYMBOL_GPL(usb_anchor_resume_wakeups);
/**
- * usb_wait_anchor_empty_timeout - wait for an anchor to be unused
+ * usb_wait_anchor_empty_timeout - wait for an anchor to become unused
* @anchor: the anchor you want to become unused
* @timeout: how long you are willing to wait in milliseconds
*
* Call this is you want to be sure all an anchor's
* URBs have finished
*
+ * A non-zero return also makes it safe to free the anchor (or the
+ * structure embedding it), provided that no new URBs are anchored to
+ * it concurrently and no other teardown call on the same anchor is
+ * still in flight (it may be sleeping on the anchor's wait queue).
+ * This function may sleep; in particular, a successful wait -- even
+ * with a zero timeout -- waits one RCU grace period before returning.
+ * It must not be called from the completion callback of a URB on
+ * @anchor: the wait cannot end before the giveback releases its
+ * suspend_wakeups pin, which happens only after the callback returns.
+ *
* Return: Non-zero if the anchor became unused. Zero on timeout.
*/
int usb_wait_anchor_empty_timeout(struct usb_anchor *anchor,
unsigned int timeout)
{
+ unsigned long deadline = jiffies + msecs_to_jiffies(timeout);
+ long ret;
+
/*
- * The condition must be evaluated under anchor->lock: when this
- * function returns non-zero, every URB giveback that has incremented
- * suspend_wakeups has finished with the anchor, so the caller may
- * free the anchor, which is typically embedded in the driver's
+ * The condition is evaluated under anchor->lock. When
+ * this function returns non-zero, every URB giveback that has
+ * incremented suspend_wakeups has finished with the anchor and one
+ * RCU grace period has flushed any reader that was still between
+ * reading urb->anchor and acquiring anchor->lock, so the caller
+ * may free the anchor -- provided that no new URBs are anchored
+ * concurrently -- which is typically embedded in the driver's
* private structure. A zero return (timeout) means a giveback may
* still be in flight and gives no such guarantee.
*/
- return wait_event_timeout(anchor->wait,
- usb_anchor_check_wakeup(anchor),
- msecs_to_jiffies(timeout));
+ /*
+ * Clamped to zero so that a timeout of zero keeps its
+ * historical meaning of a single condition evaluation
+ * without entering the wait queue (a successful
+ * evaluation still waits one grace period before
+ * returning), and so the macro never sees a negative
+ * timeout after the deadline passes.
+ */
+ ret = wait_event_timeout(anchor->wait,
+ usb_anchor_check_wakeup(anchor),
+ max_t(long, (long)(deadline - jiffies), 0));
+ if (!ret)
+ return 0;
+
+ /* Same lifetime exit as the teardown drains. */
+ synchronize_rcu();
+ return 1;
}
EXPORT_SYMBOL_GPL(usb_wait_anchor_empty_timeout);
@@ -1041,6 +1236,11 @@ EXPORT_SYMBOL_GPL(usb_wait_anchor_empty_timeout);
* This will take the oldest urb from an anchor,
* unanchor and return it
*
+ * A %NULL return does not mean the anchor is quiescent: a URB giveback
+ * may still be between its suspend_wakeups increment and its wakeup
+ * resume. Call usb_wait_anchor_empty_timeout() and check for a
+ * non-zero return before freeing the anchor.
+ *
* Return: The oldest urb from @anchor, or %NULL if @anchor has no
* urbs associated with it.
*/
@@ -1070,25 +1270,50 @@ EXPORT_SYMBOL_GPL(usb_get_from_anchor);
* @anchor: the anchor whose urbs you want to unanchor
*
* use this to get rid of all an anchor's urbs
+ *
+ * Like usb_kill_anchored_urbs(), it is safe to free the anchor once
+ * this function returns, provided that no new URBs are anchored to it
+ * concurrently and no other teardown call on the same anchor is still
+ * in flight. This function may sleep (it waits one RCU grace period
+ * on the success path).
+ *
+ * Like usb_kill_anchored_urbs(), it must not be called from the
+ * completion callback of a URB on @anchor: the drain would wait for
+ * the pin its own giveback holds.
*/
void usb_scuttle_anchored_urbs(struct usb_anchor *anchor)
{
struct urb *victim;
unsigned long flags;
- int surely_empty;
+ bool unused;
- do {
+ might_sleep();
+
+ for (;;) {
spin_lock_irqsave(&anchor->lock, flags);
while (!list_empty(&anchor->urb_list)) {
victim = list_entry(anchor->urb_list.prev,
struct urb, anchor_list);
__usb_unanchor_urb(victim, anchor);
}
- surely_empty = __usb_anchor_check_wakeup(anchor);
+
+ if (__usb_anchor_check_wakeup(anchor)) {
+ spin_unlock_irqrestore(&anchor->lock, flags);
+ /* Same lifetime exit as usb_kill_anchored_urbs(). */
+ synchronize_rcu();
+ return;
+ }
spin_unlock_irqrestore(&anchor->lock, flags);
- cpu_relax();
- } while (!surely_empty);
+ /* Same sleepable retry as usb_kill_anchored_urbs(). */
+ wait_event(anchor->wait,
+ usb_anchor_teardown_progress(anchor, &unused));
+ if (unused) {
+ /* Same lifetime exit as usb_kill_anchored_urbs(). */
+ synchronize_rcu();
+ return;
+ }
+ }
}
EXPORT_SYMBOL_GPL(usb_scuttle_anchored_urbs);
@@ -1097,6 +1322,8 @@ EXPORT_SYMBOL_GPL(usb_scuttle_anchored_urbs);
* usb_anchor_empty - is an anchor empty
* @anchor: the anchor you want to query
*
+ * This is an unsynchronized peek; it gives no lifetime guarantee.
+ *
* Return: 1 if the anchor has no urbs associated with it.
*/
int usb_anchor_empty(struct usb_anchor *anchor)
diff --git a/include/linux/usb.h b/include/linux/usb.h
index 49ab8dbb885f..5adc502b4868 100644
--- a/include/linux/usb.h
+++ b/include/linux/usb.h
@@ -1445,7 +1445,10 @@ typedef void (*usb_complete_t)(struct urb *);
* struct urb - USB Request Block
* @urb_list: For use by current owner of the URB.
* @anchor_list: membership in the list of an anchor
- * @anchor: to anchor URBs to a common mooring
+ * @anchor: to anchor URBs to a common mooring; usbcore publishes it
+ * with rcu_assign_pointer() and consumes it under
+ * rcu_read_lock() with rcu_dereference(), see
+ * drivers/usb/core/urb.c for the anchor lifetime protocol
* @ep: Points to the endpoint's data structure. Will eventually
* replace @pipe.
* @pipe: Holds endpoint number, direction, type, and more.
@@ -1640,7 +1643,7 @@ struct urb {
struct list_head urb_list; /* list head for use by the urb's
* current owner */
struct list_head anchor_list; /* the URB may be anchored */
- struct usb_anchor *anchor;
+ struct usb_anchor __rcu *anchor;
struct usb_device *dev; /* (in) pointer to associated device */
struct usb_host_endpoint *ep; /* (internal) pointer to endpoint */
unsigned int pipe; /* (in) pipe information */
--
2.32.0
^ permalink raw reply [flat|nested] 8+ messages in thread