From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from BN1PR04CU002.outbound.protection.outlook.com (mail-eastus2azon11010047.outbound.protection.outlook.com [52.101.56.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 66DD12EEE93 for ; Fri, 17 Jul 2026 11:17:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.56.47 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784287030; cv=fail; b=Zw9V0j1V/2FEnsjkeJDVJ3be6oE7DUchIvXyvoS8FL+e5P533zZDOXQ3NeZT+TScTcD7bJ/x2A7lWjzHzbU+1QXP1tNgSAiJvEw5usDPHvYwC//g3NXib23XTZNOq586fyWYdnXrwdegUyytzYdSfwBpNAefeLDCUEFgXN6gqDQ= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784287030; c=relaxed/simple; bh=+5KVOf4fR8BdFfrxH4LQ2mWkxcb2dmR8g8EX1yp+/YY=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=e7qiOnIirZskp4B6ZTCwliHVuC8cbKUTEbulKTKGM51cW4rFn7k+E6dAMOVNQT8mkxDyU3r5ed4celucn9xCnTaoDAZ9twTjC9NQ7tx1lz4M59zRoZjk07joobcJ+Rp87x2dDC+9Sskc7RY4Q5qZ6adGRweMKyGyebs8x1QEs08= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=BuJaTs+O; arc=fail smtp.client-ip=52.101.56.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="BuJaTs+O" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=REOWB42TLiBJriAVp6P/MV7SnO51UlkSprg5GM9Ty7fGfa3Xj10PrFfEaT0Xd/jILgFy02NlUXypz4X3t5XRuKPgL9Zj45zLu1bO+C20AjXHqBEDnc9mlaw/qeszPcKq27rC480mvnfei98k+LbGlxs/Iw97/Q6BhHQuAR0huMpjYmIoh7dneg4d2YYaMvNKg1gn8B1fi9himvuEHYDfebjY+1dg+HPG16T9jqyJytiLR9p16S9ns8/qWpEa0Z/7kBRpjGlpsHUhvsz3mFWka8EbbvvN7ljCNfVokQ/raTTFPWc+7aPYIhlwY7lLVyVh1z/9tQ6hSjwgebDOmlLQug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UFjzCKhyrAA+nimRDsZl0rtJQZGbuT3JVI9WQ3UgPhE=; b=IpZG2r+qYibrAtyh0aH2si3LkBpguhGDmvLlBqsYvAO4AFmEkxA+V8qNl7IEu25An41+o/46rsj7gQJxqxfjM+9fgJ8coz1Boj8eC7wk1mq/dpBP3Y45M8/U3RaEjRtW2weSAark89WYKBOW4YuRWsnWqf13Q5IvVz9nAokt48giDLw7wsfiSQ7M1PfxJ6h/OQK2aHlb6/ek0tU85rd8KCUwflnateE/NCDLNwGEeokcC6OBcqq4N+tb4Qp/a3rTmvl4ANpsq0UgPv1uIir1n+47cCWGWiItxO7jJYrHHR0Yw9W9CM/DOpCuoHthO2wfh0R7Sa5V2F6ZtZmhjJ4zyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UFjzCKhyrAA+nimRDsZl0rtJQZGbuT3JVI9WQ3UgPhE=; b=BuJaTs+OYLB1mnXH7IC+XMK1puR7qfOvVic7zmix+hFWeho85JOz6jCmIoJfLTAiLk9uffgMkdJ4kL5yttitWzK6J/gTLimj2DLI+BEp0jiZaBKb4vrc1MhRblpm5bsomexscblysKN1y9s+CK+RKIbZ+lP/U6P5cJ9xk8i1q3I= Received: from BN0PR10CA0012.namprd10.prod.outlook.com (2603:10b6:408:143::10) by DM4PR12MB7576.namprd12.prod.outlook.com (2603:10b6:8:10c::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.202.18; Fri, 17 Jul 2026 11:17:04 +0000 Received: from BN2PEPF00004FBE.namprd04.prod.outlook.com (2603:10b6:408:143:cafe::96) by BN0PR10CA0012.outlook.office365.com (2603:10b6:408:143::10) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.223.12 via Frontend Transport; Fri, 17 Jul 2026 11:17:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by BN2PEPF00004FBE.mail.protection.outlook.com (10.167.243.184) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.245.5 via Frontend Transport; Fri, 17 Jul 2026 11:17:03 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Fri, 17 Jul 2026 06:16:56 -0500 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , John Stultz , CC: Dietmar Eggemann , Valentin Schneider , Steven Rostedt , Ben Segall , Joel Fernandes , Qais Yousef , Zimuzo Ezeozue , Will Deacon , Waiman Long , Boqun Feng , "Paul E. McKenney" , Metin Kaya , Xuewen Yan , K Prateek Nayak , Thomas Gleixner , "Daniel Lezcano" , Suleiman Souhlal , kuyo chang , hupu , Vasily Gorbik , Subject: [RFC PATCH 2/3] sched/core: Fix forceidle when lock owner has mismatchd cookie Date: Fri, 17 Jul 2026 11:16:00 +0000 Message-ID: <20260717111601.6017-3-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260717111601.6017-1-kprateek.nayak@amd.com> References: <20260717111601.6017-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF00004FBE:EE_|DM4PR12MB7576:EE_ X-MS-Office365-Filtering-Correlation-Id: 9d906380-3def-4b43-a84c-08dee3f4ee1d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|23010399003|376014|7416014|1800799024|82310400026|56012099006|4143699003|18002099003|22082099003|11063799006|10067099003; X-Microsoft-Antispam-Message-Info: +ZPgU1OQ0rwJrb3hVSfc8waFv92ooDlnVvbqdX1ZcTYKrbp38ilLhYpnfsKDr1zVLs+Co1qSj2S8H1oh+mURtRNQUyOR1zElBeQ9oQXALQMXSd1/+IRdFvnvKUM8fO9YVtTkpj2rUvhSSw1244nPoyP1xZPH3zeTKMTX+xtOWnFcajhllkTml5v+G5S9Y1FUX+dTKBsE8ijkqwrYDPAzMibcv2L+zMxZt1Iu+ZEcixP/m5EKE1p1zQ8hN8/ZswNiNNUEvbRlZCO7MlE8p/TfDF0cMP75hF1m3ZypcyZRSrtbqtf0BrSuPMm0ki4roJg9CvkMteiYmCjs93cTwml4s95Pg82zsB8u1oqjuY1NPyRBQg3rjziTkh0TWvxwnje+EkhSp0pT7GNRguspkHS1/uF2xTeTngHo7g2/2UoxQlQulOYJOwZFr76bmnK4/Ukyontm0PD4DvMJ1AECqjEv6gYPvW3KKkKW0C+tx4McVSXFQLwNZaSqmuu9G0x9z3p4tnfTsgzWzeg1jLl2N9Z/GMbJ7ZEvKSQwr30cp8/4OlLtv5vJc3qhbnR83RG2LwYQeo2GI9g3+TvSjyqHCYZHzWOAa6XwjA4otZvBpF7zQf7lKV1eax8Xr8X5wYm0wQoiinJcViQPLzIUIcyPEdhq3ytkKwBxJyfwE7RkTWlHiKuKPuIT+aEDGpUVJzT5hNp7+WtNaemvH3yPBQpaFXoIfw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700016)(23010399003)(376014)(7416014)(1800799024)(82310400026)(56012099006)(4143699003)(18002099003)(22082099003)(11063799006)(10067099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: hO2dlruuWrfwVjDfYs/d3GryLl8IyUo7Wn3AEayWJJCZiEIv7l4QwReiJdYoH282R1q6ulLrfJPQk6tOhkI+e5NBiqeealVGKwY3ryFug8GhFEhGzjhEpuON14XxyERAfgRvhLNZg/OhqemPcKq5xJsVGlWfk/m5ePyiqH36EWgd0/kRhW16YcglewJZTeAjRJZwvIYzpBOiCZzuw4xZavTC7bhh0ywfPbbPxQxpssHoMjDEhxQgp94JVbn14BbpCl6ftXBrwQ6VGATep4KtlnyITs/zGlSfFVbyZ0xhmljk3p6YzajAO87VObvdNBCiovtD9wQIb9Z+iimcdeatv/uWCGlLZj8anLmy5Nx7IhHlDBQMRtKRtHoSHLQv9zZDGSdMYL6RiXPDts29Tz7Q3KouSps0JUUxUkav3p234nh6Ly+X+MRipRzS/ZCFsjW6 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jul 2026 11:17:03.9233 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9d906380-3def-4b43-a84c-08dee3f4ee1d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF00004FBE.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7576 Move Vasily's fix from 7cd4088f70e0 ("sched/core: Don't proxy-exec unmatched cookie lock owners") into a separate helper and fix the forceidle stats, balance callback queuing when rq is not the one dictating the core cookie. The helper will be extended in subsequent commits to allow for re-pick with cookie from the lock owner if rq is the pick leader. Signed-off-by: K Prateek Nayak --- kernel/sched/core.c | 43 ++++++++++++++++++++++++++++++++++++------- 1 file changed, 36 insertions(+), 7 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 31fa6bcee616..ed11410f1416 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6598,6 +6598,8 @@ __pick_next_task(struct rq *rq, struct rq_flags *rf) BUG(); /* The idle class should always have a runnable task. */ } +static void proxy_deactivate(struct rq *rq, struct task_struct *donor); + #ifdef CONFIG_SCHED_CORE static inline bool is_task_rq_idle(struct task_struct *t) { @@ -7100,6 +7102,23 @@ static inline void sched_core_cpu_dying(unsigned int cpu) rq->core = rq; } +static struct task_struct * +sched_core_swap_pick(struct rq *rq, struct task_struct *next) +{ + if (!rq->core_pick_leader) { + if (!rq->core->core_forceidle_count++) + rq->core->core_forceidle_seq++; + + proxy_resched_idle(rq); + queue_core_balance(rq); + return rq->idle; + } + + clear_task_blocked_on(rq->donor, NULL); + proxy_deactivate(rq, rq->donor); + return RETRY_TASK; +} + #else /* !CONFIG_SCHED_CORE: */ static inline void sched_core_cpu_starting(unsigned int cpu) {} @@ -7113,6 +7132,18 @@ pick_next_task(struct rq *rq, struct rq_flags *rf) return __pick_next_task(rq, rf); } +static struct task_struct * +sched_core_swap_pick(struct rq *rq, struct task_struct *next) +{ + /* + * Caller should always check !sched_cpu_cookie_match(rq, next) + * which is always false for !CONFIG_SCHED_CORE and execution + * should never reach here. + */ + BUG_ON(); + return next; +} + #endif /* !CONFIG_SCHED_CORE */ /* @@ -7512,13 +7543,6 @@ find_proxy_task(struct rq *rq, struct task_struct *donor, struct rq_flags *rf) } WARN_ON_ONCE(owner && !owner->on_rq); - if (owner && !sched_cpu_cookie_match(rq, owner)) { - if (curr_in_chain) - return proxy_resched_idle(rq); - p = donor; /* Deactivate the donor, not the runnable owner */ - clear_task_blocked_on(p, NULL); - goto deactivate; - } return owner; deactivate: @@ -7686,6 +7710,11 @@ static void __sched notrace __schedule(int sched_mode) zap_balance_callbacks(rq); goto keep_resched; } + if (!sched_cpu_cookie_match(rq, next)) { + next = sched_core_swap_pick(rq, next); + if (next == RETRY_TASK) + goto pick_again; + } trace_sched_found_proxy_task(rq->donor, next, cpu); } if (rq->donor == prev_donor && prev != next) { -- 2.43.0