[PATCH v2] futex: Fix private hash use-after-free on resize

[Chris Mason <mason@kernel.org>]

poll_state_synchronize_rcu(mm->futex.phash.batches) is used by
futex_ref_drop() to check that a grace period has passed since the
current hash was published. This relies on batches referencing a grace
period which started after the hash pointer was assigned.

__futex_pivot_hash() sets mmph->batches before it replaces mmph->hash:

    scoped_guard(rcu) {
            mmph->batches = get_state_synchronize_rcu();
            rcu_assign_pointer(mmph->hash, new);
    }

The scoped_guard(rcu) doesn't stop new grace periods from starting, and
if one starts between those two assignments, futex_ref_drop() can move
forward while a reader still holds a pointer to the old hash.

Fix things by setting mmph->batches after assigning mmph->hash.  The
scoped_guard(rcu) isn't needed, so let's drop that as well.

Fixes: 56180dd20c19 ("futex: Use RCU-based per-CPU reference counting instead of rcuref_t")
Assisted-by: kres
Signed-off-by: Chris Mason <mason@kernel.org>
---
 kernel/futex/core.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/kernel/futex/core.c b/kernel/futex/core.c
index a061f54b606d..095f9fe440e3 100644
--- a/kernel/futex/core.c
+++ b/kernel/futex/core.c
@@ -213,10 +213,12 @@ static bool __futex_pivot_hash(struct mm_struct *mm, struct futex_private_hash *
 		futex_rehash_private(fph, new);
 	}
 	new->state = FR_PERCPU;
-	scoped_guard(rcu) {
-		mmph->batches = get_state_synchronize_rcu();
-		rcu_assign_pointer(mmph->hash, new);
-	}
+	rcu_assign_pointer(mmph->hash, new);
+	/*
+	 * mmph->batches must reference a grace period which started after
+	 * mmph->hash was assigned. See futex_ref_drop().
+	 */
+	mmph->batches = get_state_synchronize_rcu();
 	kvfree_rcu(fph, rcu);
 	return true;
 }
-- 
2.54.0