From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0D52299927 for ; Tue, 18 Aug 2026 01:10:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787015455; cv=none; b=ZEKG6cozdB0SOBOksLcCH86PVxT/u0qHtpj1mIUp9bvj8/YFZQ+q2zqaorfL5+GAmmWjVksIajB16GSnb9XAyYSp7OrlxBcH5gLrkOtXrXVxsKvIZKtpr0RV6WjtsizS+28vtgwxVPGWdWgaPIeuU+xw65h61KXQpMOcY1J6kFU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787015455; c=relaxed/simple; bh=iWMOA58o72jzvpNVMYPrTgfgR2fBhOwcN3kWxKE6O2E=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Oj4NkxSVoQ2jx8Z35knkjn02+iz0CK3urHLoOTzmcir0w6bo8QS+D/1veWwFdiMjSx8Zs5OXE+L0ueGLZZOTN+/Dpft0VNr92uMcNbV67A0QkgdxMryrLZ+/IAOTRXNRijuWf6PZ7zcGtDt3hRfztST0t0iKLvHSusrQ/b5WHLs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=H1jvE9Nj; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="H1jvE9Nj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6DF201F00A3A; Tue, 18 Aug 2026 01:10:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1787015453; bh=WYE8mVBAUnVw3EiKMfL7fAGEvqr53qS2duLVSsgBptI=; h=From:To:Subject:Date:In-Reply-To:References; b=H1jvE9NjF4HR2vmCaNjcNqLPsBSBqB5Iwe3kISuAvcX+vN64Dq5KnbULGN9org+8n 8kq3Dpiwoy8U/rc05gZj1FgifRidXtO2D2uG/yTsaAFm5eFw6uV4UQqc8iyJ2pNJs4 dZadHVFzlyLm4q7NFGWMzh3Sv6swuxyrugL3pn3zyy3IoBFr54Z3WE+5k/DoDGEEiw G6DUkYEI6fwYegxBMc6jPuy3/x66yWOqDhgy/0FRmMxnZma+CMgRCWFFyd+LVegCzX H7cZ2k8Q1ymWyPNTc/6wszeR/OlUT7s4CG0fRNz4JeNsiW9bdwlF6/HTVBlwrO5MrG MsO7vnz3hZVxg== From: Chris Mason To: peterz@infradead.org, tglx@kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH] futex: sample poll cookie after publishing new hash Date: Mon, 17 Aug 2026 18:01:42 -0700 Message-ID: <20260818011036.1138213-2-mason@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260818011036.1138213-1-mason@kernel.org> References: <20260818011036.1138213-1-mason@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit futex_ref_drop() may only skip its grace period when one has already elapsed since the current private hash was published: kernel/futex/core.c:futex_ref_drop if (poll_state_synchronize_rcu(mm->futex.phash.batches)) { /* * There was a grace-period, we can begin now. */ __futex_ref_atomic_begin(fph); return; } The cookie it polls is sampled one statement before that publication: kernel/futex/core.c:__futex_pivot_hash new->state = FR_PERCPU; scoped_guard(rcu) { mmph->batches = get_state_synchronize_rcu(); rcu_assign_pointer(mmph->hash, new); } kvfree_rcu(fph, rcu); get_state_synchronize_rcu() anchors its guarantee at the snapshot, so the cookie is cleared by the first grace period that starts from there on, including one starting between the two stores which never waited for a reader that loaded the old hash after it began. CPU 0 (resize) CPU 1 (futex_hash) ============== ================== __futex_pivot_hash() batches = get_state_...() grace period starts guard(rcu) fph = old hash rcu_assign_pointer(hash, new) kvfree_rcu(old hash) futex_hash_allocate() futex_ref_drop(new hash) poll_state_...() -> true __futex_ref_atomic_begin() atomic = LONG_MAX futex_ref_get(old hash) -> true spin_lock(&fph->queues[i].lock) The reference count lives in the mm and has just been biased for the new generation, so the stalled reader pins and then locks the retired hash that is already queued for free, and its later put is charged against the live generation. Fix by sampling the cookie after rcu_assign_pointer() publishes the new hash. Drop the surrounding scoped_guard(rcu) while at it: it only delayed completion of the prematurely anchored grace-period and serves no purpose once the cookie is sampled after publication. The writer side is serialized by mm->futex.phash.lock and neither rcu_assign_pointer() nor kvfree_rcu() requires a read-side section. Fixes: 56180dd20c19 ("futex: Use RCU-based per-CPU reference counting instead of rcuref_t") Assisted-by: kres:claude-opus-5 Signed-off-by: Chris Mason --- kernel/futex/core.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/kernel/futex/core.c b/kernel/futex/core.c index 128c5752f225..05619eb8329c 100644 --- a/kernel/futex/core.c +++ b/kernel/futex/core.c @@ -209,10 +209,14 @@ static bool __futex_pivot_hash(struct mm_struct *mm, struct futex_private_hash * futex_rehash_private(fph, new); } new->state = FR_PERCPU; - scoped_guard(rcu) { - mmph->batches = get_state_synchronize_rcu(); - rcu_assign_pointer(mmph->hash, new); - } + rcu_assign_pointer(mmph->hash, new); + /* + * Pairs with futex_ref_drop(): ->batches must be sampled at or after + * the rcu_assign_pointer() above, so any grace-period satisfying + * poll_state_synchronize_rcu() provably started once no reader could + * still load the retired fph. Both stores are done under mmph->lock. + */ + mmph->batches = get_state_synchronize_rcu(); kvfree_rcu(fph, rcu); return true; } -- 2.53.0-Meta