From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from canpmsgout02.his.huawei.com (canpmsgout02.his.huawei.com [113.46.200.217]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 332A02C21F1 for ; Fri, 17 Jul 2026 08:29:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.217 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784276980; cv=none; b=ZknbsFWShb6saQRs98jpZFM3i3GZ+ARXj8yRuUGMocnI4Q+GiE0qkI6f6kimFEp+YgI5vR315L4TRDxnKlQVXhiD2tPiYWgZHPsw5ySR1a1kSNey4rRot1m1r5Sch6vnil7ut3ThnA075DO0TE7tcPJ3TvMykVKCYo9xHMfy9o4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784276980; c=relaxed/simple; bh=ZI0w0+1z+x+mzRRRottpyATo5Dkfvy8uIdueXNSGTP8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=tB+dKPwuldSetFCIx+v9nMZA5HW+CI05oPzUJ7qG5ulsM8XKEG85pfSeoOKAd0ptQmadBjNO+KK8F5ZoEF+9lLNXxL7zTo8FZXPepsGrvOJAYd8mPqoED2Ez+vpgRmhjqoXj830Po8B5V6FhhUQG/dCQzBTgP8/1bW1vHNxcmYk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=aAyA03yN; arc=none smtp.client-ip=113.46.200.217 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="aAyA03yN" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=hvnDD1vqyrPPNTfEtgXotmMB/W/dCik1E2632t2GFIw=; b=aAyA03yN6UzncjGGV1bM6ZqXsseA1WFivlCgMWEDzxo4fngvT+LMwnGJNTuNHY0tig+X/svl7 w8Ab9aXVucXM+MAERtKYTfyoU4+5qpvTS4AGLUvnMC6kKpBoNTiubWkgVMyci9DnH8MbjOOo4tr 5701e3Q56IfqKmEV0i6Sm0o= Received: from mail.maildlp.com (unknown [172.19.162.140]) by canpmsgout02.his.huawei.com (SkyGuard) with ESMTPS id 4h1jWw3Xl0zcb3h; Fri, 17 Jul 2026 16:20:12 +0800 (CST) Received: from dggpemr500006.china.huawei.com (unknown [7.185.36.185]) by mail.maildlp.com (Postfix) with ESMTPS id 14DA52012A; Fri, 17 Jul 2026 16:29:32 +0800 (CST) Received: from localhost.localdomain (10.50.85.180) by dggpemr500006.china.huawei.com (7.185.36.185) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Fri, 17 Jul 2026 16:29:31 +0800 From: Yao Kai To: CC: , , , , , , , Subject: [PATCH 1/2] futex/requeue: Fix rtmutex schedule preparation for requeue PI Date: Fri, 17 Jul 2026 16:49:21 +0800 Message-ID: <20260717084922.4153317-2-yaokai34@huawei.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260717084922.4153317-1-yaokai34@huawei.com> References: <20260717084922.4153317-1-yaokai34@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: kwepems500002.china.huawei.com (7.221.188.17) To dggpemr500006.china.huawei.com (7.185.36.185) A waiter requeued onto a PI futex can reach rt_mutex_wait_proxy_lock() without rtmutex schedule preparation: WARNING: CPU: 0 PID: 293 at kernel/sched/core.c:7606 RIP: rt_mutex_schedule+0x43/0x50 Call Trace: rt_mutex_slowlock_block.constprop.0+0x5b/0x320 rt_mutex_wait_proxy_lock+0x3e/0x80 futex_wait_requeue_pi+0x3ba/0x590 do_futex+0x171/0x1f0 rt_mutex_schedule() requires current->sched_rt_mutex to be set. Normally, rt_mutex_pre_schedule() sets it and submits pending work before the waiter is enqueued. With requeue PI, another task can enqueue the waiter after its futex_q becomes visible: waiter requeue task ------ ------------ futex_wait_requeue_pi() futex_wait_setup() futex_queue(&q) futex_requeue() rt_mutex_start_proxy_lock() enqueue rt_waiter install pi_blocked_on requeue_futex() plist_del(&q->list) futex_do_wait() plist_node_empty(&q->list) skip schedule() plist_add(&q->list) futex_requeue_pi_complete() IN_PROGRESS -> DONE futex_requeue_pi_wakeup_sync() // DONE rt_mutex_wait_proxy_lock() rt_mutex_schedule() futex_do_wait() mistakes the temporary removal for a wakeup and skips schedule(). The waiter consequently enters rt_mutex_schedule() with current->sched_rt_mutex clear. Its block plug remains unflushed, and workqueue or io-wq users are not notified that the worker is going to sleep. rt_mutex_pre_schedule() cannot be used directly. Calling it before q is published would set current->sched_rt_mutex while futex_do_wait() can still call regular schedule(). Calling it after DONE would flush the block plug after current->pi_blocked_on is installed, which can recurse into rtmutex waiter setup. Split the preparation instead. Flush the plug before futex_wait_setup() publishes q. After futex_requeue_pi_wakeup_sync() returns DONE, enter the rtmutex scheduling state and notify workqueue and io-wq users without flushing the plug again. Split sched_submit_work() to support this ordering and leave the scheduling state after the waiter has acquired the lock or has been removed. Fixes: d14f9e930b90 ("locking/rtmutex: Use rt_mutex specific scheduler helpers") Cc: stable@vger.kernel.org Signed-off-by: Yao Kai --- include/linux/sched/rt.h | 2 ++ kernel/futex/requeue.c | 15 +++++++++++++- kernel/sched/core.c | 45 +++++++++++++++++++++++++++++++++------- 3 files changed, 53 insertions(+), 9 deletions(-) diff --git a/include/linux/sched/rt.h b/include/linux/sched/rt.h index 4e3338103654..3d8cbdbfe14b 100644 --- a/include/linux/sched/rt.h +++ b/include/linux/sched/rt.h @@ -51,7 +51,9 @@ static inline bool rt_or_dl_task_policy(struct task_struct *tsk) } #ifdef CONFIG_RT_MUTEXES +void rt_mutex_pre_schedule_flush_plug(void); extern void rt_mutex_pre_schedule(void); +void rt_mutex_pre_schedule_pi_blocked(void); extern void rt_mutex_schedule(void); extern void rt_mutex_post_schedule(void); diff --git a/kernel/futex/requeue.c b/kernel/futex/requeue.c index 79823ad13683..abc652b5b2dd 100644 --- a/kernel/futex/requeue.c +++ b/kernel/futex/requeue.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-or-later #include +#include #include #include "futex.h" @@ -820,6 +821,13 @@ int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags, q.rt_waiter = &rt_waiter; q.requeue_pi_key = &key2; + /* + * Once q is visible, requeue can enqueue rt_waiter and install + * current->pi_blocked_on on our behalf. Flush plugged I/O before + * that can happen. + */ + rt_mutex_pre_schedule_flush_plug(); + /* * Prepare to wait on uaddr. On success, it holds hb->lock and q * is initialized. @@ -865,6 +873,11 @@ int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags, case Q_REQUEUE_PI_DONE: /* Requeue completed. Current is 'pi_blocked_on' the rtmutex */ pi_mutex = &q.pi_state->pi_mutex; + /* + * The rt_waiter is already enqueued and the plug is flushed. + * Enter schedule state and notify worker users. + */ + rt_mutex_pre_schedule_pi_blocked(); ret = rt_mutex_wait_proxy_lock(pi_mutex, to, &rt_waiter); /* @@ -875,6 +888,7 @@ int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags, futex_q_lockptr_lock(&q); debug_rt_mutex_free_waiter(&rt_waiter); + rt_mutex_post_schedule(); /* * Fixup the pi_state owner and possibly acquire the lock if we * haven't already. @@ -915,4 +929,3 @@ int futex_wait_requeue_pi(u32 __user *uaddr, unsigned int flags, } return ret; } - diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 96226707c2f6..f481e49e16b8 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -7257,17 +7257,12 @@ void __noreturn do_task_dead(void) cpu_relax(); } -static inline void sched_submit_work(struct task_struct *tsk) +static DEFINE_WAIT_OVERRIDE_MAP(sched_map, LD_WAIT_CONFIG); + +static inline void sched_submit_work_notifiers(struct task_struct *tsk) { - static DEFINE_WAIT_OVERRIDE_MAP(sched_map, LD_WAIT_CONFIG); unsigned int task_flags; - /* - * Establish LD_WAIT_CONFIG context to ensure none of the code called - * will use a blocking primitive -- which would lead to recursion. - */ - lock_map_acquire_try(&sched_map); - task_flags = tsk->flags; /* * If a worker goes to sleep, notify and ask workqueue whether it @@ -7277,7 +7272,10 @@ static inline void sched_submit_work(struct task_struct *tsk) wq_worker_sleeping(tsk); else if (task_flags & PF_IO_WORKER) io_wq_worker_sleeping(tsk); +} +static inline void sched_submit_work_plug(struct task_struct *tsk) +{ /* * spinlock and rwlock must not flush block requests. This will * deadlock if the callback attempts to acquire a lock which is @@ -7290,6 +7288,18 @@ static inline void sched_submit_work(struct task_struct *tsk) * make sure to submit it to avoid deadlocks. */ blk_flush_plug(tsk->plug, true); +} + +static inline void sched_submit_work(struct task_struct *tsk) +{ + /* + * Establish LD_WAIT_CONFIG context to ensure none of the code called + * will use a blocking primitive -- which would lead to recursion. + */ + lock_map_acquire_try(&sched_map); + + sched_submit_work_notifiers(tsk); + sched_submit_work_plug(tsk); lock_map_release(&sched_map); } @@ -7595,12 +7605,31 @@ const struct sched_class *__setscheduler_class(int policy, int prio) */ #define fetch_and_set(x, v) ({ int _x = (x); (x) = (v); _x; }) +void rt_mutex_pre_schedule_flush_plug(void) +{ + lockdep_assert(!current->pi_blocked_on); + lock_map_acquire_try(&sched_map); + sched_submit_work_plug(current); + lock_map_release(&sched_map); +} + void rt_mutex_pre_schedule(void) { lockdep_assert(!fetch_and_set(current->sched_rt_mutex, 1)); sched_submit_work(current); } +void rt_mutex_pre_schedule_pi_blocked(void) +{ + lockdep_assert(current->pi_blocked_on); + lockdep_assert(!fetch_and_set(current->sched_rt_mutex, 1)); + + /* The plug was flushed before current->pi_blocked_on was installed. */ + lock_map_acquire_try(&sched_map); + sched_submit_work_notifiers(current); + lock_map_release(&sched_map); +} + void rt_mutex_schedule(void) { lockdep_assert(current->sched_rt_mutex); -- 2.43.0