From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from PH0PR06CU001.outbound.protection.outlook.com (mail-westus3azon11011068.outbound.protection.outlook.com [40.107.208.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 043C336923B for ; Sun, 3 May 2026 18:43:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.208.68 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777833797; cv=fail; b=np2oxYm1Qkh6wuQAQLYaor9VQp5V6ED1F4M7SkLxc9tIseF/XqFH/GqWLBIU2lsNthLBx3qqMOKD9y1kUdSnLyKwkV060U1lZeLA8M1QxZsDtGcTe+/kps7lkqD4u572fs2/1ICaUM5wTv00JQZPigAuN3RIVfjqS0oxoSWhQbA= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777833797; c=relaxed/simple; bh=JB1e9SYEzuvuA1l9yPbPZbUhrV53rzqHeAKMx3wjmQE=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=T8/kmcbAeCPpT5NlhNC5QqDl6u9E1eG8c8mlQyYBrvONe2/StL1H0Qz1Bz2wnqzLWMNzoeTeAnH0Tq9z0m/x40VT+Qh2Afqq1kD51+aZzu1J3qaGzGxWVs8zta63dndVbDVDtXUnRgYy6yHfKB2K/JPctVv70IUSoL+f/JS5mHY= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=tLi6KaKU; arc=fail smtp.client-ip=40.107.208.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="tLi6KaKU" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=YmBWL6e8mGW7HbfTAa+pjN2N9HlMPJSrk8PoOMP5K4YVdhEI0/106uFzAECAeDX3Yqw/LqSYOn7seSQpkPrr6chWTNNyx3apDFHUZ2ZyjfNKpnCS+FyS0emsgOMMnApaCHRDPz52J3akSyQRyTix+e5P7sPDqXtnpJbadsrRwmHmrJudczhrIl5Q74w5djuGQJwdvJImMgts6DIb+6SXCyiodaVuAqhFrL4uAxb/9R06x9qZoNE2MfJs7k0sA9e2FOuWvBb95stbGONInV6yARUMHxdlQUqFqdCSvzehAOiI8knMaKFRz5yjXFVXCBWyRaWa3971OubOhrV2IkAjJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0h+DJyw/VdsR0KQtMYR2Jirx98uB2ZG2VSaeRgm0dik=; b=Yj85pvznS+tIfc54p7f2f7NP9pYayPvXSeuM2Z31pSkj2DdFBd5uJWWJate1DfSacxC0cCyRDP0zemYF8IiSVa5LtQfzH7M0blZYjeXYbLq5cWPzcd14FH4bDoYYodm4+ZfGUwFdY8V/xuTdia0AeA4Zk4jT+l8DEPPGx+W+4jlBxiFQneZ2BslQHlnmcXNROWzRmAKmXVIDGp52/DvBCdu5Oqroaeb0TFqJvFq3xmZkSs2Dbk1mzehvMvyPu1NWFNBghDg5byVglovXUiMQisqIqzWF75qxrdHG2sN9k5/l0Q3mr4XP8rDx6xWGEYPhM2dDcZShdxlgfFxLHWmFLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=google.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0h+DJyw/VdsR0KQtMYR2Jirx98uB2ZG2VSaeRgm0dik=; b=tLi6KaKUimq1epJHBaIg5XOR3j+HA6GS9rj14HGKS+eeCUh0qQmzNkowX/pq0+BkSNDe4QpbaPtueemQQirqsbj7jzpZE+W9T5r4Oi92ayIp4napl5r/vcJm1ohypc33Btho7PVw78pLzBVfTPOryGHs878SGkPdSwBicSSMEXM= Received: from SJ0PR03CA0192.namprd03.prod.outlook.com (2603:10b6:a03:2ef::17) by SN7PR12MB7203.namprd12.prod.outlook.com (2603:10b6:806:2aa::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.25; Sun, 3 May 2026 18:43:08 +0000 Received: from SJ5PEPF000001F7.namprd05.prod.outlook.com (2603:10b6:a03:2ef:cafe::a2) by SJ0PR03CA0192.outlook.office365.com (2603:10b6:a03:2ef::17) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9870.25 via Frontend Transport; Sun, 3 May 2026 18:43:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by SJ5PEPF000001F7.mail.protection.outlook.com (10.167.242.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9891.9 via Frontend Transport; Sun, 3 May 2026 18:43:07 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.2562.17; Sun, 3 May 2026 13:43:07 -0500 Received: from satlexmb07.amd.com (10.181.42.216) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 3 May 2026 13:43:06 -0500 Received: from [172.31.184.125] (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Sun, 3 May 2026 13:43:00 -0500 Message-ID: Date: Mon, 4 May 2026 00:12:59 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 1/2] sched: proxy-exec: Close race causing workqueue work being delayed To: John Stultz , Peter Zijlstra CC: LKML , Vineeth Pillai , Sonam Sanju , "Sean Christopherson" , Kunwu Chan , "Tejun Heo" , Joel Fernandes , Qais Yousef , Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Valentin Schneider , Steven Rostedt , Will Deacon , Waiman Long , Boqun Feng , "Paul E. McKenney" , Metin Kaya , Xuewen Yan , Thomas Gleixner , Daniel Lezcano , "Suleiman Souhlal" , kuyo chang , hupu , References: <20260430215103.2978955-1-jstultz@google.com> <20260430215103.2978955-2-jstultz@google.com> <20260501132143.GC1026330@noisy.programming.kicks-ass.net> <63c830c3-fe6d-4822-81db-9fdd1597282e@amd.com> <20260501185900.GF1026330@noisy.programming.kicks-ass.net> Content-Language: en-US From: K Prateek Nayak In-Reply-To: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit Received-SPF: None (SATLEXMB04.amd.com: kprateek.nayak@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ5PEPF000001F7:EE_|SN7PR12MB7203:EE_ X-MS-Office365-Filtering-Correlation-Id: 3f41cdf7-57ee-410b-9da0-08dea943d1c5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|30052699003|82310400026|376014|7416014|36860700016|1800799024|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: gzNJ8mdky8W8bYUTAZl++f5NUokX16p0IE6nTM6oBUqUV5kOPzU6Z0h+OUcBdn/syEnNGYqAhZsKmC4PwW3tkEa47nt8xUG3XO+nHC9Uor0DyPec85+6N4gz58FBaPlZEF5FzDfmIXU7Lubh60AW2SPFyuHAhS56m4VdpGVtsaXIgvQLLYXWLYPyO39qmXibhNOZoL2j2b7ksP4KebdyjK4FaUCRcWLkrubS5xTyvc/HBm35c1AkDjjJn13/9XVG5BHOvOMZLVSQpdL7d5I0s0+zRmwofKlYa6zQ47Cg1OzXh8LudOX1AzX+xxv/1tiFER4CGc/DTJOt+KRh+hhAEueHwJgZyskbezfnVIRkQ7O8jrPPQNSNIQh6HDZBYgXdlDry4b2Fq2y1MbJtJF98R+/XMfITP90yTbdC/0R+skEQdOX6EjztcudDVw6sSzrn2V7cXPDcHDw8er5WzCt8/TJv0XWsS3ZA5S8lwvXnhh0lU44hNH3WgzysSTGWNEjDJapJBdVRocyhGmrZ0/0Ek2zsV0mhTjokuYLiPXGQdVRd/CvhEnylaaYC+oXOdj0K06B5V1XvEn8Ca/x551lprhY8d9dbSIJG7D00ZrWR0KyGYNGMpVLMNLiIjiMRiKGeiGbbebCHijGlz8ZdsB+8FT6TkNqa8u6YAfVd4FssCvyQ9kWPxwUJKPFqdiwJ9ijTAA6c0Mqwwfrl46xrDxbRyLqP2gm+SwLRobwcg08N8ho= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(30052699003)(82310400026)(376014)(7416014)(36860700016)(1800799024)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: wZUvWPT7EV1Uyslrwmi6MLYcPncXWVIyd2Ruu3sWuGy8RVvcS4B/jlRO4w1EZf9GARmr5QSIUuSOyAm69uwtdFq5nVqtD7lI940CrQ5zNtmfFQ9/vd4v+0EsHagu7Cvwav0DbypDF2qYv9AtkdYJmY406rrsNjJWwWyUXUvyiZPa3vIcurR5aESv3o+l4WFqlAkZFu6sDHuHb/9eLR5jSHkqNbk2a1Mi1WBiF4Nx0+wzufngnK1OBQH07hhSpx/D9KA6GD0wcglw/r7ekpLlk4nJFqNBCDMxoi0iJXvnmxyhtC30DTnmjDnc7WyWtOsvIEk8zn+JihHc/A11Ger1ZCWaALb8cWlIrm1f6RpqWyjqVsvshRP2Zb2EwkCOuxa5E47FZIwaRstHKMeWvAZzW1OX4JWMvToOa3PnQ8vGlLfCb0d0MHO6kcLPPK5Oi7+u X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 May 2026 18:43:07.9619 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3f41cdf7-57ee-410b-9da0-08dea943d1c5 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ5PEPF000001F7.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7203 Hello folks, On 5/2/2026 3:56 AM, John Stultz wrote: > On Fri, May 1, 2026 at 11:59 AM Peter Zijlstra wrote: >> On Fri, May 01, 2026 at 09:25:29PM +0530, K Prateek Nayak wrote: >> >>>> @@ -3685,6 +3691,7 @@ ttwu_stat(struct task_struct *p, int cpu, int wake_flags) >>>> */ >>>> static inline void ttwu_do_wakeup(struct task_struct *p) >>>> { >>>> + p->is_blocked = 0; >>> >>> I don't think it is this simple at the moment because the proxy bits in >>> __schedule() still have to handle PROXY_WAKING and once we clear it here >>> task will no longer go through proxy_needs_return() path. >>> >>> Clearing of ->is_blocked has to be done at the same point where >>> ->blocked_on is cleared although they are set separately. >> >> Argh. Its all a convoluted mess. AFAICT this all goes away when we make >> ttwu() do the return migration properly. And then it does work. >> >> So we're now in the situation that things are a bit of a mess, and we >> need to make a bigger mess, only to then instantly remove it all again >> when we clean up :/ > > Apologies! I don't want to make you grumpy coming back from being ill > (hope you're feeling better!). > >> Can't we simply mark PROXY_EXEc broken for a cycle? Its not like the >> upstream version has been very functional anyway. > > This issue has been present for awhile (since it is really around the > proxy deactivation path taking action in the preempt case). I just > reproduced it with the early chunk of PROXY_EXEC logic that was in > v6.18. So I don't think it's super urgent as the proxy-exec code > upstream isn't complete (and behind CONFIG_EXPERIMENTAL). > > So let me take a swing at integrating your approach into the next > chunk of patches, and hopefully they can be ready for the next merge > window. So when looking at all of this, I realized we probably don't need PROXY_WAKING anymore if we have the "is_blocked" state in task_struct. The owner can simply clear the blocked_on and move along and the waiter's "is_blocked" state will handle the sched bits. (p->is_blocked && !p->blocked_on) can then be interpreted as PROXY_WAKING and that task should explore return migration in find_proxy_task(). Would something like below be more amenable from a backport standpoint instead of marking the config broken? (Lightly tested; Based on tip:sched/core) diff --git a/include/linux/sched.h b/include/linux/sched.h index 8ec3b6d7d718b..7be5e1faf56a1 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -846,7 +846,11 @@ struct task_struct { struct alloc_tag *alloc_tag; #endif - int on_cpu; + u8 on_cpu; + u8 on_rq; + u8 is_blocked; + u8 __pad; + struct __call_single_node wake_entry; unsigned int wakee_flips; unsigned long wakee_flip_decay_ts; @@ -861,7 +865,6 @@ struct task_struct { */ int recent_used_cpu; int wake_cpu; - int on_rq; int prio; int static_prio; @@ -2181,19 +2184,10 @@ extern int __cond_resched_rwlock_write(rwlock_t *lock) __must_hold(lock); #ifndef CONFIG_PREEMPT_RT -/* - * With proxy exec, if a task has been proxy-migrated, it may be a donor - * on a cpu that it can't actually run on. Thus we need a special state - * to denote that the task is being woken, but that it needs to be - * evaluated for return-migration before it is run. So if the task is - * blocked_on PROXY_WAKING, return migrate it before running it. - */ -#define PROXY_WAKING ((struct mutex *)(-1L)) - static inline struct mutex *__get_task_blocked_on(struct task_struct *p) { lockdep_assert_held_once(&p->blocked_lock); - return p->blocked_on == PROXY_WAKING ? NULL : p->blocked_on; + return p->blocked_on; } static inline void __set_task_blocked_on(struct task_struct *p, struct mutex *m) @@ -2221,7 +2215,7 @@ static inline void __clear_task_blocked_on(struct task_struct *p, struct mutex * * blocked_on relationships, but make sure we are not * clearing the relationship with a different lock. */ - WARN_ON_ONCE(m && p->blocked_on && p->blocked_on != m && p->blocked_on != PROXY_WAKING); + WARN_ON_ONCE(m && p->blocked_on && p->blocked_on != m); p->blocked_on = NULL; } @@ -2231,34 +2225,6 @@ static inline void clear_task_blocked_on(struct task_struct *p, struct mutex *m) __clear_task_blocked_on(p, m); } -static inline void __set_task_blocked_on_waking(struct task_struct *p, struct mutex *m) -{ - /* Currently we serialize blocked_on under the task::blocked_lock */ - lockdep_assert_held_once(&p->blocked_lock); - - if (!sched_proxy_exec()) { - __clear_task_blocked_on(p, m); - return; - } - - /* Don't set PROXY_WAKING if blocked_on was already cleared */ - if (!p->blocked_on) - return; - /* - * There may be cases where we set PROXY_WAKING on tasks that were - * already set to waking, but make sure we are not changing - * the relationship with a different lock. - */ - WARN_ON_ONCE(m && p->blocked_on != m && p->blocked_on != PROXY_WAKING); - p->blocked_on = PROXY_WAKING; -} - -static inline void set_task_blocked_on_waking(struct task_struct *p, struct mutex *m) -{ - guard(raw_spinlock_irqsave)(&p->blocked_lock); - __set_task_blocked_on_waking(p, m); -} - #else static inline void __clear_task_blocked_on(struct task_struct *p, struct rt_mutex *m) { @@ -2267,14 +2233,6 @@ static inline void __clear_task_blocked_on(struct task_struct *p, struct rt_mute static inline void clear_task_blocked_on(struct task_struct *p, struct rt_mutex *m) { } - -static inline void __set_task_blocked_on_waking(struct task_struct *p, struct rt_mutex *m) -{ -} - -static inline void set_task_blocked_on_waking(struct task_struct *p, struct rt_mutex *m) -{ -} #endif /* !CONFIG_PREEMPT_RT */ static __always_inline bool need_resched(void) diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c index 7d359647156df..4aa79bcab08c7 100644 --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -983,7 +983,7 @@ static noinline void __sched __mutex_unlock_slowpath(struct mutex *lock, unsigne next = waiter->task; debug_mutex_wake_waiter(lock, waiter); - set_task_blocked_on_waking(next, lock); + clear_task_blocked_on(next, lock); wake_q_add(&wake_q, next); } diff --git a/kernel/locking/ww_mutex.h b/kernel/locking/ww_mutex.h index 5cd9dfa4b31e6..522fe045eb1b2 100644 --- a/kernel/locking/ww_mutex.h +++ b/kernel/locking/ww_mutex.h @@ -285,11 +285,11 @@ __ww_mutex_die(struct MUTEX *lock, struct MUTEX_WAITER *waiter, debug_mutex_wake_waiter(lock, waiter); #endif /* - * When waking up the task to die, be sure to set the - * blocked_on to PROXY_WAKING. Otherwise we can see - * circular blocked_on relationships that can't resolve. + * When waking up the task to die, be sure to clear the + * blocked_on. Otherwise we can see circular blocked_on + * relationships that can't resolve. */ - set_task_blocked_on_waking(waiter->task, lock); + clear_task_blocked_on(waiter->task, lock); wake_q_add(wake_q, waiter->task); } @@ -340,14 +340,14 @@ static bool __ww_mutex_wound(struct MUTEX *lock, if (owner != current) { /* * When waking up the task to wound, be sure to set the - * blocked_on to PROXY_WAKING. Otherwise we can see - * circular blocked_on relationships that can't resolve. + * clear blocked_on. Otherwise we can see circular + * blocked_on relationships that can't resolve. * * NOTE: We pass NULL here instead of lock, because we * are waking the mutex owner, who may be currently * blocked on a different mutex. */ - set_task_blocked_on_waking(owner, NULL); + clear_task_blocked_on(owner, NULL); wake_q_add(wake_q, owner); } return true; diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 49cd5d2171613..d33398a03a1c2 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6495,6 +6495,9 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) #endif /* !CONFIG_SCHED_CORE */ +static inline void sched_set_task_is_blocked(struct task_struct *p); +static inline void sched_clear_task_is_blocked(struct task_struct *p); + /* * Constants for the sched_mode argument of __schedule(). * @@ -6523,7 +6526,18 @@ static bool try_to_block_task(struct rq *rq, struct task_struct *p, if (signal_pending_state(task_state, p)) { WRITE_ONCE(p->__state, TASK_RUNNING); *task_state_p = TASK_RUNNING; - set_task_blocked_on_waking(p, NULL); + + /* + * Clear blocked_on relation if we were planning to + * retain the task as proxy donor since it is runnable + * again as a result of pending signal. + * + * Since only the running task can set the blocked_on + * relation for itself, do not unnecessarily grab the + * blocked_lock if blocked_on is not set. + */ + if (!should_block) + clear_task_blocked_on(p, NULL); return false; } @@ -6535,8 +6549,10 @@ static bool try_to_block_task(struct rq *rq, struct task_struct *p, * blocked on a mutex, and we want to keep it on the runqueue * to be selectable for proxy-execution. */ - if (!should_block) + if (!should_block) { + sched_set_task_is_blocked(p); return false; + } p->sched_contributes_to_load = (task_state & TASK_UNINTERRUPTIBLE) && @@ -6562,6 +6578,27 @@ static bool try_to_block_task(struct rq *rq, struct task_struct *p, } #ifdef CONFIG_SCHED_PROXY_EXEC +static inline void sched_set_task_is_blocked(struct task_struct *p) +{ + if (!sched_proxy_exec()) + return; + + p->is_blocked = 1; +} + +static inline void sched_clear_task_is_blocked(struct task_struct *p) +{ + p->is_blocked = 0; +} + +static inline bool task_should_block(struct task_struct *p) +{ + if (!sched_proxy_exec()) + return true; + + return !p->blocked_on; +} + static inline void proxy_set_task_cpu(struct task_struct *p, int cpu) { unsigned int wake_cpu; @@ -6602,6 +6639,7 @@ static bool proxy_deactivate(struct rq *rq, struct task_struct *donor) * need to be changed from next *before* we deactivate. */ proxy_resched_idle(rq); + sched_clear_task_is_blocked(donor); return try_to_block_task(rq, donor, &state, true); } @@ -6732,7 +6770,7 @@ static void proxy_force_return(struct rq *rq, struct rq_flags *rf, cpu = select_task_rq(p, p->wake_cpu, &wake_flag); set_task_cpu(p, cpu); target_rq = cpu_rq(cpu); - clear_task_blocked_on(p, NULL); + sched_clear_task_is_blocked(p); } if (target_rq) @@ -6765,15 +6803,16 @@ find_proxy_task(struct rq *rq, struct task_struct *donor, struct rq_flags *rf) bool curr_in_chain = false; int this_cpu = cpu_of(rq); struct task_struct *p; - struct mutex *mutex; int owner_cpu; /* Follow blocked_on chain. */ - for (p = donor; (mutex = p->blocked_on); p = owner) { - /* if its PROXY_WAKING, do return migration or run if current */ - if (mutex == PROXY_WAKING) { + for (p = donor; task_is_blocked(p); p = owner) { + struct mutex *mutex = p->blocked_on; + + /* If task is no longer blocked, do return migration or run if current */ + if (!mutex) { if (task_current(rq, p)) { - clear_task_blocked_on(p, PROXY_WAKING); + sched_clear_task_is_blocked(p); return p; } goto force_return; @@ -6807,8 +6846,9 @@ find_proxy_task(struct rq *rq, struct task_struct *donor, struct rq_flags *rf) * and return p (if it is current and safe to * just run on this rq), or return-migrate the task. */ + __clear_task_blocked_on(p, mutex); if (task_current(rq, p)) { - __clear_task_blocked_on(p, NULL); + sched_clear_task_is_blocked(p); return p; } goto force_return; @@ -6902,6 +6942,14 @@ find_proxy_task(struct rq *rq, struct task_struct *donor, struct rq_flags *rf) return NULL; } #else /* SCHED_PROXY_EXEC */ +static inline void sched_set_task_is_blocked(struct task_struct *p) {} +static inline void sched_clear_task_is_blocked(struct task_struct *p) {} + +static inline bool task_should_block(struct task_struct *p) +{ + return true; +} + static struct task_struct * find_proxy_task(struct rq *rq, struct task_struct *donor, struct rq_flags *rf) { @@ -7044,7 +7092,7 @@ static void __sched notrace __schedule(int sched_mode) struct task_struct *prev_donor = rq->donor; rq_set_donor(rq, next); - if (unlikely(next->blocked_on)) { + if (unlikely(task_is_blocked(next))) { next = find_proxy_task(rq, next, &rf); if (!next) { zap_balance_callbacks(rq); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index c95584191d58f..5c1085f260ad4 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2390,7 +2390,7 @@ static inline bool task_is_blocked(struct task_struct *p) if (!sched_proxy_exec()) return false; - return !!p->blocked_on; + return !!p->is_blocked; } static inline int task_on_cpu(struct rq *rq, struct task_struct *p) --- It could be split as introduction of new state + removal of PROXY_WAKING for easier review. I'll let you two decide if it is worthwhile of not. -- Thanks and Regards, Prateek