* [PATCH 0/2] dm thin: stop waiting for grace periods when activating and removing thins
@ 2026-11-16 5:12 Qiliang Yuan
2026-11-16 5:12 ` [PATCH 1/2] dm thin: drop the grace period from thin_ctr() Qiliang Yuan
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Qiliang Yuan @ 2026-11-16 5:12 UTC (permalink / raw)
To: Alasdair Kergon, Mike Snitzer, Mikulas Patocka, Benjamin Marzinski
Cc: dm-devel, linux-kernel, Qiliang Yuan
Activating a thin device waits for an RCU grace period in thin_ctr(),
and removing one waits for another in thin_dtr(). Tools that create and
remove thin snapshots for every container, such as the containerd
devmapper snapshotter, pay for both each time.
Patch 1 drops the grace period from thin_ctr(), which isn't needed.
Patch 2 lets the active_thins walkers skip thins whose last reference is
gone, so that thin_dtr() can free the thin with kfree_rcu() instead of
waiting. Activating a thin device gets 51% faster and removing one 15%.
Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
Qiliang Yuan (2):
dm thin: drop the grace period from thin_ctr()
dm thin: free thins with kfree_rcu() instead of waiting in thin_dtr()
drivers/md/dm-thin.c | 37 ++++++++++++++++++-------------------
1 file changed, 18 insertions(+), 19 deletions(-)
---
base-commit: 23c62304dde29abe92013d85910d03a77dada35d
change-id: 20261114-bug-dm-thin-rcu-gp-d29d4b74a0b8
Best regards,
--
Qiliang Yuan <qiliang.yuan@linux.dev>
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/2] dm thin: drop the grace period from thin_ctr()
2026-11-16 5:12 [PATCH 0/2] dm thin: stop waiting for grace periods when activating and removing thins Qiliang Yuan
@ 2026-11-16 5:12 ` Qiliang Yuan
2026-11-18 0:51 ` Benjamin Marzinski
2026-11-16 5:12 ` [PATCH 2/2] dm thin: free thins with kfree_rcu() instead of waiting in thin_dtr() Qiliang Yuan
2026-11-19 5:24 ` [PATCH 0/2] dm thin: stop waiting for grace periods when activating and removing thins Benjamin Marzinski
2 siblings, 1 reply; 7+ messages in thread
From: Qiliang Yuan @ 2026-11-16 5:12 UTC (permalink / raw)
To: Alasdair Kergon, Mike Snitzer, Mikulas Patocka, Benjamin Marzinski
Cc: dm-devel, linux-kernel, Qiliang Yuan
thin_ctr() adds the new thin to pool->active_thins with
list_add_tail_rcu() and then calls synchronize_rcu(), so that a
wake_worker() call can't miss the bios of a thin that isn't visible yet.
The grace period doesn't make the entry any more visible. A worker that
walks the list after list_add_tail_rcu() finds the thin, and the thin
gets no bios before its table is resumed. A walk already in progress may
miss it, but the wake_worker() for the first bio queues the worker
again. Still, activating a thin device waits for a full grace period,
which takes about half of the activation.
Remove the synchronize_rcu() call.
Activating 100 thin devices of an idle pool with dmsetup create takes
per device (mean of five rounds, min-max of the rounds in parentheses):
before (ms) after (ms) change
15.4 (14.9-15.9) 7.6 (7.3-7.8) -51%
Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
drivers/md/dm-thin.c | 7 -------
1 file changed, 7 deletions(-)
diff --git a/drivers/md/dm-thin.c b/drivers/md/dm-thin.c
index 358ed77ffb2be..d5ff937eeacba 100644
--- a/drivers/md/dm-thin.c
+++ b/drivers/md/dm-thin.c
@@ -4289,13 +4289,6 @@ static int thin_ctr(struct dm_target *ti, unsigned int argc, char **argv)
init_completion(&tc->can_destroy);
list_add_tail_rcu(&tc->list, &tc->pool->active_thins);
spin_unlock_irq(&tc->pool->lock);
- /*
- * This synchronize_rcu() call is needed here otherwise we risk a
- * wake_worker() call finding no bios to process (because the newly
- * added tc isn't yet visible). So this reduces latency since we
- * aren't then dependent on the periodic commit to wake_worker().
- */
- synchronize_rcu();
dm_put(pool_md);
--
2.43.0
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/2] dm thin: free thins with kfree_rcu() instead of waiting in thin_dtr()
2026-11-16 5:12 [PATCH 0/2] dm thin: stop waiting for grace periods when activating and removing thins Qiliang Yuan
2026-11-16 5:12 ` [PATCH 1/2] dm thin: drop the grace period from thin_ctr() Qiliang Yuan
@ 2026-11-16 5:12 ` Qiliang Yuan
2026-11-18 1:13 ` Benjamin Marzinski
2026-11-19 5:16 ` [PATCH v2] " Benjamin Marzinski
2026-11-19 5:24 ` [PATCH 0/2] dm thin: stop waiting for grace periods when activating and removing thins Benjamin Marzinski
2 siblings, 2 replies; 7+ messages in thread
From: Qiliang Yuan @ 2026-11-16 5:12 UTC (permalink / raw)
To: Alasdair Kergon, Mike Snitzer, Mikulas Patocka, Benjamin Marzinski
Cc: dm-devel, linux-kernel, Qiliang Yuan
get_first_thin() and get_next_thin() find a thin on pool->active_thins
under rcu_read_lock() and take a reference on it. thin_dtr() removes the
thin from the list and waits for a grace period before dropping its own
reference, so that no walker takes one after the last is gone.
Removing a thin device thus waits for a full grace period in thin_dtr(),
before it waits for the references that are still held.
Take the references with refcount_inc_not_zero() and skip the thins
whose last reference is gone, so that thin_dtr() only waits for the
references already taken. Free the thin with kfree_rcu(), as
requeue_bios() and error_retry_list_with_code() may still look at it
under rcu_read_lock().
Removing 100 thin devices of an idle pool with dmsetup remove takes per
device (mean of five rounds, min-max of the rounds in parentheses). The
queue freeze in del_gendisk() waits for a grace period of its own, which
the one in thin_dtr() used to overlap, so only part of it is saved:
before (ms) after (ms) change
16.9 (15.8-17.3) 14.4 (13.8-15.3) -15%
Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
drivers/md/dm-thin.c | 30 ++++++++++++++++++------------
1 file changed, 18 insertions(+), 12 deletions(-)
diff --git a/drivers/md/dm-thin.c b/drivers/md/dm-thin.c
index d5ff937eeacba..8386215101eb2 100644
--- a/drivers/md/dm-thin.c
+++ b/drivers/md/dm-thin.c
@@ -360,6 +360,7 @@ struct thin_c {
*/
refcount_t refcount;
struct completion can_destroy;
+ struct rcu_head rcu;
};
/*----------------------------------------------------------------*/
@@ -2317,25 +2318,29 @@ static void process_thin_deferred_cells(struct thin_c *tc)
} while (!list_empty(&cells));
}
-static void thin_get(struct thin_c *tc);
+static bool thin_tryget(struct thin_c *tc);
static void thin_put(struct thin_c *tc);
/*
* We can't hold rcu_read_lock() around code that can block. So we
* find a thin with the rcu lock held; bump a refcount; then drop
- * the lock.
+ * the lock. Skip the thins whose refcount thin_dtr() already dropped
+ * to zero.
*/
static struct thin_c *get_first_thin(struct pool *pool)
{
- struct thin_c *tc = NULL;
+ struct thin_c *tc;
rcu_read_lock();
- tc = list_first_or_null_rcu(&pool->active_thins, struct thin_c, list);
- if (tc)
- thin_get(tc);
+ list_for_each_entry_rcu(tc, &pool->active_thins, list) {
+ if (thin_tryget(tc)) {
+ rcu_read_unlock();
+ return tc;
+ }
+ }
rcu_read_unlock();
- return tc;
+ return NULL;
}
static struct thin_c *get_next_thin(struct pool *pool, struct thin_c *tc)
@@ -2344,7 +2349,8 @@ static struct thin_c *get_next_thin(struct pool *pool, struct thin_c *tc)
rcu_read_lock();
list_for_each_entry_continue_rcu(tc, &pool->active_thins, list) {
- thin_get(tc);
+ if (!thin_tryget(tc))
+ continue;
thin_put(old_tc);
rcu_read_unlock();
return tc;
@@ -4128,9 +4134,9 @@ static struct target_type pool_target = {
* Thin target methods
*--------------------------------------------------------------
*/
-static void thin_get(struct thin_c *tc)
+static bool thin_tryget(struct thin_c *tc)
{
- refcount_inc(&tc->refcount);
+ return refcount_inc_not_zero(&tc->refcount);
}
static void thin_put(struct thin_c *tc)
@@ -4146,7 +4152,6 @@ static void thin_dtr(struct dm_target *ti)
spin_lock_irq(&tc->pool->lock);
list_del_rcu(&tc->list);
spin_unlock_irq(&tc->pool->lock);
- synchronize_rcu();
thin_put(tc);
wait_for_completion(&tc->can_destroy);
@@ -4158,7 +4163,8 @@ static void thin_dtr(struct dm_target *ti)
dm_put_device(ti, tc->pool_dev);
if (tc->origin_dev)
dm_put_device(ti, tc->origin_dev);
- kfree(tc);
+ /* Let RCU readers of active_thins finish with tc before freeing it */
+ kfree_rcu(tc, rcu);
mutex_unlock(&dm_thin_pool_table.mutex);
}
--
2.43.0
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/2] dm thin: drop the grace period from thin_ctr()
2026-11-16 5:12 ` [PATCH 1/2] dm thin: drop the grace period from thin_ctr() Qiliang Yuan
@ 2026-11-18 0:51 ` Benjamin Marzinski
0 siblings, 0 replies; 7+ messages in thread
From: Benjamin Marzinski @ 2026-11-18 0:51 UTC (permalink / raw)
To: Qiliang Yuan
Cc: Alasdair Kergon, Mike Snitzer, Mikulas Patocka, dm-devel, linux-kernel
On Mon, Nov 16, 2026 at 12:12:03AM -0500, Qiliang Yuan wrote:
> thin_ctr() adds the new thin to pool->active_thins with
> list_add_tail_rcu() and then calls synchronize_rcu(), so that a
> wake_worker() call can't miss the bios of a thin that isn't visible yet.
>
> The grace period doesn't make the entry any more visible. A worker that
> walks the list after list_add_tail_rcu() finds the thin, and the thin
> gets no bios before its table is resumed. A walk already in progress may
> miss it, but the wake_worker() for the first bio queues the worker
> again. Still, activating a thin device waits for a full grace period,
> which takes about half of the activation.
>
> Remove the synchronize_rcu() call.
>
> Activating 100 thin devices of an idle pool with dmsetup create takes
> per device (mean of five rounds, min-max of the rounds in parentheses):
>
> before (ms) after (ms) change
> 15.4 (14.9-15.9) 7.6 (7.3-7.8) -51%
>
> Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
Reviewed-by: Benjamin Marzinski <bmarzins@redhat.com>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 2/2] dm thin: free thins with kfree_rcu() instead of waiting in thin_dtr()
2026-11-16 5:12 ` [PATCH 2/2] dm thin: free thins with kfree_rcu() instead of waiting in thin_dtr() Qiliang Yuan
@ 2026-11-18 1:13 ` Benjamin Marzinski
2026-11-19 5:16 ` [PATCH v2] " Benjamin Marzinski
1 sibling, 0 replies; 7+ messages in thread
From: Benjamin Marzinski @ 2026-11-18 1:13 UTC (permalink / raw)
To: Qiliang Yuan
Cc: Alasdair Kergon, Mike Snitzer, Mikulas Patocka, dm-devel, linux-kernel
On Mon, Nov 16, 2026 at 12:12:04AM -0500, Qiliang Yuan wrote:
> get_first_thin() and get_next_thin() find a thin on pool->active_thins
> under rcu_read_lock() and take a reference on it. thin_dtr() removes the
> thin from the list and waits for a grace period before dropping its own
> reference, so that no walker takes one after the last is gone.
>
> Removing a thin device thus waits for a full grace period in thin_dtr(),
> before it waits for the references that are still held.
>
> Take the references with refcount_inc_not_zero() and skip the thins
> whose last reference is gone, so that thin_dtr() only waits for the
> references already taken. Free the thin with kfree_rcu(), as
> requeue_bios() and error_retry_list_with_code() may still look at it
> under rcu_read_lock().
>
> Removing 100 thin devices of an idle pool with dmsetup remove takes per
> device (mean of five rounds, min-max of the rounds in parentheses). The
> queue freeze in del_gendisk() waits for a grace period of its own, which
> the one in thin_dtr() used to overlap, so only part of it is saved:
>
> before (ms) after (ms) change
> 16.9 (15.8-17.3) 14.4 (13.8-15.3) -15%
>
> Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
So, this isn't directly a problem with your patch, but in reviewing it,
I noticed a bug in the interaction between thin_dtr() and
get_next_thin(). thin_dtr() can take tc off the active_thins list, while
it's being held by a worker. The refcount keeps it from being deleted.
But once tc is removed from the list, there's nothing that guarantees
that tc->list.next is still meaningful. If "dmsetup remove" is called on
the thin device that is next in the active_thins list, that thin_dtr()
call can free the thin pointed to by the original tc->list.next pointer.
So when the original thin_dtr() calls get_next_thin(), it will access
freed memory.
I think the solution is for thin_dtr to mark tc as dead somehow, instead
of removing it from active_thins where we currently do. Once the
refcount drops to 0, we can remove it, wait for a grace period, and free
it.
Or am I missing something subtle that keeps this issue from happening?
-Ben
> ---
> drivers/md/dm-thin.c | 30 ++++++++++++++++++------------
> 1 file changed, 18 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/md/dm-thin.c b/drivers/md/dm-thin.c
> index d5ff937eeacba..8386215101eb2 100644
> --- a/drivers/md/dm-thin.c
> +++ b/drivers/md/dm-thin.c
> @@ -360,6 +360,7 @@ struct thin_c {
> */
> refcount_t refcount;
> struct completion can_destroy;
> + struct rcu_head rcu;
> };
>
> /*----------------------------------------------------------------*/
> @@ -2317,25 +2318,29 @@ static void process_thin_deferred_cells(struct thin_c *tc)
> } while (!list_empty(&cells));
> }
>
> -static void thin_get(struct thin_c *tc);
> +static bool thin_tryget(struct thin_c *tc);
> static void thin_put(struct thin_c *tc);
>
> /*
> * We can't hold rcu_read_lock() around code that can block. So we
> * find a thin with the rcu lock held; bump a refcount; then drop
> - * the lock.
> + * the lock. Skip the thins whose refcount thin_dtr() already dropped
> + * to zero.
> */
> static struct thin_c *get_first_thin(struct pool *pool)
> {
> - struct thin_c *tc = NULL;
> + struct thin_c *tc;
>
> rcu_read_lock();
> - tc = list_first_or_null_rcu(&pool->active_thins, struct thin_c, list);
> - if (tc)
> - thin_get(tc);
> + list_for_each_entry_rcu(tc, &pool->active_thins, list) {
> + if (thin_tryget(tc)) {
> + rcu_read_unlock();
> + return tc;
> + }
> + }
> rcu_read_unlock();
>
> - return tc;
> + return NULL;
> }
>
> static struct thin_c *get_next_thin(struct pool *pool, struct thin_c *tc)
> @@ -2344,7 +2349,8 @@ static struct thin_c *get_next_thin(struct pool *pool, struct thin_c *tc)
>
> rcu_read_lock();
> list_for_each_entry_continue_rcu(tc, &pool->active_thins, list) {
> - thin_get(tc);
> + if (!thin_tryget(tc))
> + continue;
> thin_put(old_tc);
> rcu_read_unlock();
> return tc;
> @@ -4128,9 +4134,9 @@ static struct target_type pool_target = {
> * Thin target methods
> *--------------------------------------------------------------
> */
> -static void thin_get(struct thin_c *tc)
> +static bool thin_tryget(struct thin_c *tc)
> {
> - refcount_inc(&tc->refcount);
> + return refcount_inc_not_zero(&tc->refcount);
> }
>
> static void thin_put(struct thin_c *tc)
> @@ -4146,7 +4152,6 @@ static void thin_dtr(struct dm_target *ti)
> spin_lock_irq(&tc->pool->lock);
> list_del_rcu(&tc->list);
> spin_unlock_irq(&tc->pool->lock);
> - synchronize_rcu();
>
> thin_put(tc);
> wait_for_completion(&tc->can_destroy);
> @@ -4158,7 +4163,8 @@ static void thin_dtr(struct dm_target *ti)
> dm_put_device(ti, tc->pool_dev);
> if (tc->origin_dev)
> dm_put_device(ti, tc->origin_dev);
> - kfree(tc);
> + /* Let RCU readers of active_thins finish with tc before freeing it */
> + kfree_rcu(tc, rcu);
>
> mutex_unlock(&dm_thin_pool_table.mutex);
> }
>
> --
> 2.43.0
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v2] dm thin: free thins with kfree_rcu() instead of waiting in thin_dtr()
2026-11-16 5:12 ` [PATCH 2/2] dm thin: free thins with kfree_rcu() instead of waiting in thin_dtr() Qiliang Yuan
2026-11-18 1:13 ` Benjamin Marzinski
@ 2026-11-19 5:16 ` Benjamin Marzinski
1 sibling, 0 replies; 7+ messages in thread
From: Benjamin Marzinski @ 2026-11-19 5:16 UTC (permalink / raw)
To: Qiliang Yuan, Alasdair Kergon, Mike Snitzer, Mikulas Patocka
Cc: dm-devel, linux-kernel
From: Qiliang Yuan <qiliang.yuan@linux.dev>
get_first_thin() and get_next_thin() find a thin on pool->active_thins
under rcu_read_lock() and take a reference on it. thin_dtr() removes the
thin from the list and waits for a grace period before dropping its own
reference, so that no walker takes one after the last is gone. Removing
a thin device thus waits for a full grace period in thin_dtr(), before
it waits for the references that are still held.
Additionally, thin_dtr() can take a thin off the active_thins list,
while it's being held by a worker. The refcount keeps it from being
deleted. But once the thin is removed from the list, there's nothing
that guarantees that tc->list.next is still meaningful. If "dmsetup
remove" is called on the thin device that is next in the active_thins
list, that thin_dtr() call can free the thin pointed to by the original
tc->list.next pointer. So when the worker calls get_next_thin(), it
will access freed memory.
Instead of removing thins from the active_thins list at the start of
thin_dtr(), mark them as dying. This keeps active_thins walkers from
grabbing them, while thin_dtr() waits for the refcount to drop to zero.
It's racy, but no more than removing the thin from the list was.
Also, take the references with refcount_inc_not_zero() and skip the
thins who are dying or whose last reference is gone, so that thin_dtr()
only waits for the references already taken. Free the thin with
kfree_rcu(), as requeue_bios() and error_retry_list_with_code() may
still look at it under rcu_read_lock().
Removing 100 thin devices of an idle pool with dmsetup remove takes per
device (mean of five rounds, min-max of the rounds in parentheses). The
queue freeze in del_gendisk() waits for a grace period of its own, which
the one in thin_dtr() used to overlap, so only part of it is saved:
before (ms) after (ms) change
16.9 (15.8-17.3) 14.4 (13.8-15.3) -15%
Fixes: 17181fb7a0c3 ("dm thin: fix a race in thin_dtr")
Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
Co-developed-by: Benjamin Marzinski <bmarzins@redhat.com>
Signed-off-by: Benjamin Marzinski <bmarzins@redhat.com>
---
Qiliang, does this version work for you? Could you rerun the timing?
I'm squashed my changes together with your patch, because my changes
depend on your patch to actually fix 17181fb7a0c3. I just moved the list
removal till after refcount drops to zero (which depends on your
refcount_inc_not_zero changes to be safe), and added a dying variable to
keep active_thins walkers from grabbing the thin, while thin_dtr() is
waiting for the existing holders to drop it (just like what removeing it
from the list did).
If people want this split into a fix and a feature patch, that's fine.
However, the feature patch would just be swapping the synchronize_rcu()
for kfree_rcu(), which seemed to me kind of pointless to move to a
separate patch.
drivers/md/dm-thin.c | 54 +++++++++++++++++++++++++++++---------------
1 file changed, 36 insertions(+), 18 deletions(-)
diff --git a/drivers/md/dm-thin.c b/drivers/md/dm-thin.c
index bdd250d1eb2e..e695fc356c4d 100644
--- a/drivers/md/dm-thin.c
+++ b/drivers/md/dm-thin.c
@@ -355,11 +355,17 @@ struct thin_c {
struct rb_root sort_bio_list; /* sorted list of deferred bios */
/*
- * Ensures the thin is not destroyed until the worker has finished
- * iterating the active_thins list.
+ * Ensures the thin stays on the active_thins list until the workers
+ * have moved past it while iterating the list.
*/
refcount_t refcount;
struct completion can_destroy;
+ struct rcu_head rcu;
+ /*
+ * Keeps new active_thin walkers from grabbing the table while
+ * thin_dtr() is waiting for the existing holders to finish.
+ */
+ bool dying;
};
/*----------------------------------------------------------------*/
@@ -652,8 +658,11 @@ static void error_retry_list_with_code(struct pool *pool, blk_status_t error)
struct thin_c *tc;
rcu_read_lock();
- list_for_each_entry_rcu(tc, &pool->active_thins, list)
+ list_for_each_entry_rcu(tc, &pool->active_thins, list) {
+ if (READ_ONCE(tc->dying))
+ continue;
error_thin_bio_list(tc, &tc->retry_on_resume_list, error);
+ }
rcu_read_unlock();
}
@@ -2318,25 +2327,29 @@ static void process_thin_deferred_cells(struct thin_c *tc)
} while (!list_empty(&cells));
}
-static void thin_get(struct thin_c *tc);
+static bool thin_tryget(struct thin_c *tc);
static void thin_put(struct thin_c *tc);
/*
* We can't hold rcu_read_lock() around code that can block. So we
* find a thin with the rcu lock held; bump a refcount; then drop
- * the lock.
+ * the lock. Skip the thins marked as dying or whose refcount thin_dtr()
+ * already dropped to zero.
*/
static struct thin_c *get_first_thin(struct pool *pool)
{
- struct thin_c *tc = NULL;
+ struct thin_c *tc;
rcu_read_lock();
- tc = list_first_or_null_rcu(&pool->active_thins, struct thin_c, list);
- if (tc)
- thin_get(tc);
+ list_for_each_entry_rcu(tc, &pool->active_thins, list) {
+ if (thin_tryget(tc)) {
+ rcu_read_unlock();
+ return tc;
+ }
+ }
rcu_read_unlock();
- return tc;
+ return NULL;
}
static struct thin_c *get_next_thin(struct pool *pool, struct thin_c *tc)
@@ -2345,7 +2358,8 @@ static struct thin_c *get_next_thin(struct pool *pool, struct thin_c *tc)
rcu_read_lock();
list_for_each_entry_continue_rcu(tc, &pool->active_thins, list) {
- thin_get(tc);
+ if (!thin_tryget(tc))
+ continue;
thin_put(old_tc);
rcu_read_unlock();
return tc;
@@ -2817,6 +2831,8 @@ static void requeue_bios(struct pool *pool)
rcu_read_lock();
list_for_each_entry_rcu(tc, &pool->active_thins, list) {
+ if (READ_ONCE(tc->dying))
+ continue;
spin_lock_irq(&tc->lock);
bio_list_merge(&tc->deferred_bio_list, &tc->retry_on_resume_list);
bio_list_init(&tc->retry_on_resume_list);
@@ -4131,9 +4147,9 @@ static struct target_type pool_target = {
* Thin target methods
*--------------------------------------------------------------
*/
-static void thin_get(struct thin_c *tc)
+static bool thin_tryget(struct thin_c *tc)
{
- refcount_inc(&tc->refcount);
+ return !READ_ONCE(tc->dying) && refcount_inc_not_zero(&tc->refcount);
}
static void thin_put(struct thin_c *tc)
@@ -4146,14 +4162,15 @@ static void thin_dtr(struct dm_target *ti)
{
struct thin_c *tc = ti->private;
- spin_lock_irq(&tc->pool->lock);
- list_del_rcu(&tc->list);
- spin_unlock_irq(&tc->pool->lock);
- synchronize_rcu();
+ WRITE_ONCE(tc->dying, true);
thin_put(tc);
wait_for_completion(&tc->can_destroy);
+ spin_lock_irq(&tc->pool->lock);
+ list_del_rcu(&tc->list);
+ spin_unlock_irq(&tc->pool->lock);
+
mutex_lock(&dm_thin_pool_table.mutex);
__pool_dec(tc->pool);
@@ -4161,7 +4178,8 @@ static void thin_dtr(struct dm_target *ti)
dm_put_device(ti, tc->pool_dev);
if (tc->origin_dev)
dm_put_device(ti, tc->origin_dev);
- kfree(tc);
+ /* Let RCU readers of active_thins finish with tc before freeing it */
+ kfree_rcu(tc, rcu);
mutex_unlock(&dm_thin_pool_table.mutex);
}
--
2.56.0
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 0/2] dm thin: stop waiting for grace periods when activating and removing thins
2026-11-16 5:12 [PATCH 0/2] dm thin: stop waiting for grace periods when activating and removing thins Qiliang Yuan
2026-11-16 5:12 ` [PATCH 1/2] dm thin: drop the grace period from thin_ctr() Qiliang Yuan
2026-11-16 5:12 ` [PATCH 2/2] dm thin: free thins with kfree_rcu() instead of waiting in thin_dtr() Qiliang Yuan
@ 2026-11-19 5:24 ` Benjamin Marzinski
2 siblings, 0 replies; 7+ messages in thread
From: Benjamin Marzinski @ 2026-11-19 5:24 UTC (permalink / raw)
To: Qiliang Yuan
Cc: Alasdair Kergon, Mike Snitzer, Mikulas Patocka, dm-devel, linux-kernel
On Mon, Nov 16, 2026 at 12:12:02AM -0500, Qiliang Yuan wrote:
> Activating a thin device waits for an RCU grace period in thin_ctr(),
> and removing one waits for another in thin_dtr(). Tools that create and
> remove thin snapshots for every container, such as the containerd
> devmapper snapshotter, pay for both each time.
>
> Patch 1 drops the grace period from thin_ctr(), which isn't needed.
> Patch 2 lets the active_thins walkers skip thins whose last reference is
> gone, so that thin_dtr() can free the thin with kfree_rcu() instead of
> waiting. Activating a thin device gets 51% faster and removing one 15%.
>
> Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
I posted an alternate version of your [PATCH 2/2] that fixes the issue
I mentioned. Can you verify that it works for you?
I squashed it with your patch, since it your original patch did most of
the work of fixing the issue, and it seemed easier to just have one
patch to backport to fix it. I could refactor it into two patches: one
the fixes the bug, and a second that just swaps the synchronize_rcu()
for kfree_rcu(), if people would rather not mix the bug fix and feature.
-Ben
> ---
> Qiliang Yuan (2):
> dm thin: drop the grace period from thin_ctr()
> dm thin: free thins with kfree_rcu() instead of waiting in thin_dtr()
>
> drivers/md/dm-thin.c | 37 ++++++++++++++++++-------------------
> 1 file changed, 18 insertions(+), 19 deletions(-)
> ---
> base-commit: 23c62304dde29abe92013d85910d03a77dada35d
> change-id: 20261114-bug-dm-thin-rcu-gp-d29d4b74a0b8
>
> Best regards,
> --
> Qiliang Yuan <qiliang.yuan@linux.dev>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2026-11-19 5:24 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-16 5:12 [PATCH 0/2] dm thin: stop waiting for grace periods when activating and removing thins Qiliang Yuan
2026-11-16 5:12 ` [PATCH 1/2] dm thin: drop the grace period from thin_ctr() Qiliang Yuan
2026-11-18 0:51 ` Benjamin Marzinski
2026-11-16 5:12 ` [PATCH 2/2] dm thin: free thins with kfree_rcu() instead of waiting in thin_dtr() Qiliang Yuan
2026-11-18 1:13 ` Benjamin Marzinski
2026-11-19 5:16 ` [PATCH v2] " Benjamin Marzinski
2026-11-19 5:24 ` [PATCH 0/2] dm thin: stop waiting for grace periods when activating and removing thins Benjamin Marzinski
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®