[PATCH] rbtree: clarify documentation of rbtree_postorder_for_each_entry_safe()

[PATCH] rbtree: clarify documentation of rbtree_postorder_for_each_entry_safe()

[Cody P Schafer]

Noticed that commit a20135ffbc44 ("writeback: don't drain
bdi_writeback_congested on bdi destruction") added a usage of
rbtree_postorder_for_each_entry_safe() in mm/backing-dev.c which appears
to try to rb_erase() elements from an rbtree while iterating over it
using rbtree_postorder_for_each_entry_safe().

Doing this will cause random nodes to be missed by the iteration because
rb_erase() may rebalance the tree, changing the ordering that we're
trying to iterate over.

The previous documentation for rbtree_postorder_for_each_entry_safe()
wasn't clear that this wasn't allowed, it was taken from the docs for
list_for_each_entry_safe(), where erasing isn't a problem due to
list_del() not reordering.

Explicitly warn developers about this potential pit-fall.

Note that I haven't fixed the actual issue that (it appears) the commit
referenced above introduced (not familiar enough with that code).

In general (and in this case), the patterns to follow are:
 - switch to rb_first() + rb_erase(), don't use
   rbtree_postorder_for_each_entry_safe().
 - keep the postorder iteration and don't rb_erase() at all. Instead
   just clear the fields of rb_node & cgwb_congested_tree as required by
   other users of those structures.

CC: Tejun Heo <tj@kernel.org>
Signed-off-by: Cody P Schafer <dev@codyps.com>
---
 include/linux/rbtree.h | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/include/linux/rbtree.h b/include/linux/rbtree.h
index 830c499..39de3df 100644
--- a/include/linux/rbtree.h
+++ b/include/linux/rbtree.h
@@ -101,13 +101,21 @@ static inline void rb_link_node_rcu(struct rb_node *node, struct rb_node *parent
 	})
 
 /**
- * rbtree_postorder_for_each_entry_safe - iterate over rb_root in post order of
- * given type safe against removal of rb_node entry
+ * rbtree_postorder_for_each_entry_safe - iterate in post-order over rb_root of
+ * given type allowing the backing memory of @pos to be invalidated
  *
  * @pos:	the 'type *' to use as a loop cursor.
  * @n:		another 'type *' to use as temporary storage
  * @root:	'rb_root *' of the rbtree.
  * @field:	the name of the rb_node field within 'type'.
+ *
+ * This function provides a similar guarantee as list_for_each_entry_safe() and
+ * allows the iteration to continue independent of changes to @pos by the body
+ * of the loop.
+ *
+ * Note, however, that it cannot handle other modifications that re-order the
+ * rbtree it is iterating over. This includes calling rb_erase() on @pos, as
+ * rb_erase() may rebalance the tree, causing us to miss some nodes.
  */
 #define rbtree_postorder_for_each_entry_safe(pos, n, root, field) \
 	for (pos = rb_entry_safe(rb_first_postorder(root), typeof(*pos), field); \
-- 
2.4.9

Re: [PATCH] rbtree: clarify documentation of rbtree_postorder_for_each_entry_safe()

[Tejun Heo]

Hello,

On Sun, Oct 04, 2015 at 06:32:50PM -0400, Cody P Schafer wrote:
> Doing this will cause random nodes to be missed by the iteration because
> rb_erase() may rebalance the tree, changing the ordering that we're
> trying to iterate over.
> 
> The previous documentation for rbtree_postorder_for_each_entry_safe()
> wasn't clear that this wasn't allowed, it was taken from the docs for
> list_for_each_entry_safe(), where erasing isn't a problem due to
> list_del() not reordering.

Ugh... that's a misleading name for an iterator if it doesn't allow
removal of elements during iteration.

> Explicitly warn developers about this potential pit-fall.
> 
> Note that I haven't fixed the actual issue that (it appears) the commit
> referenced above introduced (not familiar enough with that code).
> 
> In general (and in this case), the patterns to follow are:
>  - switch to rb_first() + rb_erase(), don't use
>    rbtree_postorder_for_each_entry_safe().

I'll update it to a while loop on rb_first().

Thanks.

-- 
tejun

[PATCH block/for-linus] writeback: remove broken rbtree_postorder_for_each_entry_safe() usage in cgwb_bdi_destroy()

[Tejun Heo]

a20135ffbc44 ("writeback: don't drain bdi_writeback_congested on bdi
destruction") added rbtree_postorder_for_each_entry_safe() which is
used to remove all entries; however, according to Cody, the iterator
isn't safe against operations which may rebalance the tree.  Fix it by
switching to repeatedly removing rb_first() until empty.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Cody P Schafer <dev@codyps.com>
Fixes: a20135ffbc44 ("writeback: don't drain bdi_writeback_congested on bdi destruction")
Link: http://lkml.kernel.org/g/1443997973-1700-1-git-send-email-dev@codyps.com
---
 mm/backing-dev.c |   10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

--- a/mm/backing-dev.c
+++ b/mm/backing-dev.c
@@ -681,7 +681,7 @@ static int cgwb_bdi_init(struct backing_
 static void cgwb_bdi_destroy(struct backing_dev_info *bdi)
 {
 	struct radix_tree_iter iter;
-	struct bdi_writeback_congested *congested, *congested_n;
+	struct rb_node *rbn;
 	void **slot;
 
 	WARN_ON(test_bit(WB_registered, &bdi->wb.state));
@@ -691,9 +691,11 @@ static void cgwb_bdi_destroy(struct back
 	radix_tree_for_each_slot(slot, &bdi->cgwb_tree, &iter, 0)
 		cgwb_kill(*slot);
 
-	rbtree_postorder_for_each_entry_safe(congested, congested_n,
-					&bdi->cgwb_congested_tree, rb_node) {
-		rb_erase(&congested->rb_node, &bdi->cgwb_congested_tree);
+	while ((rbn = rb_first(&bdi->cgwb_congested_tree))) {
+		struct bdi_writeback_congested *congested =
+			rb_entry(rbn, struct bdi_writeback_congested, rb_node);
+
+		rb_erase(rbn, &bdi->cgwb_congested_tree);
 		congested->bdi = NULL;	/* mark @congested unlinked */
 	}