From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011029.outbound.protection.outlook.com [40.93.194.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 89B0E3B3883 for ; Thu, 28 May 2026 09:48:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.29 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779961735; cv=fail; b=qXoLpIytmZESk5t9XobMEHn9SX7iIq7GKDTdKmSAVp/sUmqKzR8km2QM23xoMUo3vFWTzDt/8qlHq03D7Gtizd1b7bE8Mv82SHiezJYOFm9uhhyUIRxbObIE5BIBJj/j7DH2WvPnXqJsFHc8bTK5JgxZ3UREXHP94s/oBYLWwIU= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779961735; c=relaxed/simple; bh=DH/Jyak+P3TQ+VrWH8gTkSoAkb9icixpFWS97PQcOVI=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=VtwLJIDlnXF5Ybv6oshoXkhKV0ZgWmX/kUp4egiwNP+qsbij9RMTXAYUCzbU2QD/RyzTr4RXnnyGrL1i5MGF99INlyo6Kj2rykgUD8EYFOf7+S5mt6Sjq9MDwkbHC5dvuXh322CnDasrALHHE3iZoVjli0ADnyZpirQeQVOw3aQ= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=j9LQpQXc; arc=fail smtp.client-ip=40.93.194.29 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="j9LQpQXc" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Sufmg6fLGNQLJem1fbfxSSaPkC1CKJ2c26x0guAJFmpdLMK2w6VOs51EyfFS+gpUKxOfztGgv4RPEDumSpenay05s6vqUbSJt0N3E6azNBnTnXhRNb3j9lqAOSlpdp+yLfLOgn7fgKDaMe5uCW8MLTojkdXCqqN1KCtsm5VbLCwjNjtcRGMQV9D9Jo9hS+za6IIEQNKre94vSJMg+5mhvwcB0AqWN5cCkQ/NHDKDnbX8G5LsdNGLsheKSyK02JwcSf43HBS+weZmtq/bLJZcTrdyPIe3bQ1GzQ5r8ja1xAKxsGw/HesO3CvF2Xb33NrjKpq2DfSLnaj2oArYvYVdKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eyXdYveg+a+2NHFqF8nfOBd9VHhxdBN8c2bBIOvb1D4=; b=vPi5Gs13RxpjY7L81anYnz4natcwCbe6sn4QXdeUUXCiR1btCMjiXYq2I5zAdwb1Lcc6o5nuaXL5mH2A4C9Z67M5/q0jUd2hZWnBhp68z0YlHzVLo1DiXpknPnmf+Pmj8uSpHe7UUuLxxGX+rtVSt7VOcJrURpNnida2TwlFTtqBO+mcpwAnY3kzvhKnaHQWY00dkx+/nm8ZgrfsEBGURk33UWFUeh9txYRq/29X3+Q5F6cpzcdZpuJv46ZbdIwdcjhEndh7iUM2yTNqcxNfLkFuSAMBD2rsLxTvibC2/mKK7kbdwwvGbKLbbEZe+cMItivBwxGfTapJE4kK4hFHUA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eyXdYveg+a+2NHFqF8nfOBd9VHhxdBN8c2bBIOvb1D4=; b=j9LQpQXcNlQSDTqNpRohZn3SJ1YYbJ6n77veStuuQT0abzrOOjp/+/vV8eyPAa9Bs4dtTwFpt2sN1MnBUXe3IWREBc4/pMsDv2E9wbkc7NR0HqIEmLAkV6rdTXnQrl40c/ZEw6fWxTWL4C7EO8rUEO+/AyIwg1FMoEPKIj2V6Vw= Received: from MN2PR08CA0018.namprd08.prod.outlook.com (2603:10b6:208:239::23) by CY5PR12MB6624.namprd12.prod.outlook.com (2603:10b6:930:40::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.71.14; Thu, 28 May 2026 09:48:48 +0000 Received: from BN2PEPF00004FBD.namprd04.prod.outlook.com (2603:10b6:208:239:cafe::66) by MN2PR08CA0018.outlook.office365.com (2603:10b6:208:239::23) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.71.13 via Frontend Transport; Thu, 28 May 2026 09:48:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by BN2PEPF00004FBD.mail.protection.outlook.com (10.167.243.183) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.71.7 via Frontend Transport; Thu, 28 May 2026 09:48:47 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Thu, 28 May 2026 04:48:43 -0500 From: K Prateek Nayak To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot CC: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Aaron Lu , Josh Don , K Prateek Nayak , Subject: [PATCH 0/5] sched/fair: Allow account_cfs_rq_runtime() to throttle current hierarchy Date: Thu, 28 May 2026 09:48:25 +0000 Message-ID: <20260528094830.13291-1-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF00004FBD:EE_|CY5PR12MB6624:EE_ X-MS-Office365-Filtering-Correlation-Id: 51932cbc-dc42-44b2-2be9-08debc9e50c4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700016|376014|7416014|18002099003|13003099007|11063799006|5023799004|56012099006; X-Microsoft-Antispam-Message-Info: XC7buthYjmEUE4fwPqti2ImV8a+HyM1srs7NY9mPvZDkSPym1AUGMMbdzMTyA6JWPuRqB9Bd5UqcrRS6nlvy6znjxzkWOaPuBpJJdzblj2KPR0UUZ0si6aej9eJaqPBfDXjU+NB0Ady3XxL8qbDY2phcB30EQhatUwCCjNNbJoFbDyq0jaAr9V0+7RJEbV1jZdBSOu4MOLHiQXivzxlXqsCe2Bu4H8FXs4t2Uq6dv8ioy6Gl2PLojhHrLLc97hUOZTHgpYFhXMWRegW3PV/2/s8ZMGYoIBp8lNyF4uDyttwuyB4eQdIRvOwHsReErKP3Q+Ol+ucLZnwcOZJ9ReHJRvcMj0DmODj5dmvn6r0igOp6KoA7mcOsObLlLfPqcQU4aDXbyl6xcmA2kMlFK/rwqArXZd0wdQEFljtdDA+Mn7N+qchuXesvxb6Ll0W/Sv+O8RgyJNrOspKWpTdURraEAR+zKC154mBknWeRXKRoMOJQuTo00DDCA0iLBX907K6L1UjzN+K+RTk7IIeLUZbp1suiT6GvOv6E+IpXfXRi4RwcCtWRHfGg3uFZmL4FWJMEyDS55E8mG9/CpCz0b1+SKvUa4Gg0AS+Z8pYd4hx9Y8LQzmMXRSgfmlTCAZcyJVP0a0JyZFJAskIUpHp+QGD7uWwrMZwcrRa9KWbfxbz7TkCo8md5Nk6k9fRnyONL24B1 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700016)(376014)(7416014)(18002099003)(13003099007)(11063799006)(5023799004)(56012099006);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: eZ1ej6MiSG25LTiVSRM9rCMODbJkC5mTNZwnHlnDbY+Ft7SZHOMzxjDO03Ua7En9gzJpiWpuHd1wQEIDsWZBxHIDaU9xYLZo7nHSxeOdvbO7gVrhz2Kfo/N8mnx72G7bYlFGBonv+eGExRu3dKA9RskQnwQ538DIuWWoXjg+SWICuhf9xRHTU8J7zDSS8bPJJc7l7jo0saom3Uj2zqwUslhK7/8rmt4O/0Fz4jkBLIeW9G9SpJPU03AJFL54ATI0OlKKuNwYa4lPQDCCDfYDjYvc+LCuYEBZl0xGebPbZ4MfyfQABzeLZGt73jOfMGJL3+NHFUMU/Tc1Hb2EKL9p8gByVCFc5PXOa2C1aoGGJz/05N4AvrlB7VmmUiFke1Gc0bU5a+0WV92GqECubv4n5tV2Jbjx63Sw/dQV9LbyJD4xjLu0fqxzYpU8aAFk1tbZ X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 May 2026 09:48:47.8657 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 51932cbc-dc42-44b2-2be9-08debc9e50c4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF00004FBD.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6624 The current hierarchy is always throttled in __schedule() during the pick when update_curr() detects a cfs_rq running out of the bandwidth and issues a resched. This was necessary prior to per-task throttling where the entire throttled hierarchy was dequeued at the point of first throttle during the pick but with per-task throttling, tasks continue to run as usual until they exit to userspace and dequeue themselves one-by-one until the hierarchy is deemed fully throttled and the PELT is frozen. throttle_cfs_rq() is now simply a propagator of throttle indicators and nothing more. Unify the throttling for current hierarchy under account_cfs_rq_runtime() which is responsible for the time accounting. If the bandwidth runs out, account_cfs_rq_runtime() will request for sched_cfs_bandwidth_slice() and mark the hierarchy as throttled if it fails to grab bandwidth. throttle_cfs_rq() will do a task_throttle_setup_work() if it finds the current task to be on a throttled hierarchy and the task will naturally dequeue itself when it exits to the userspace without needing an explicit resched. First four patches are cleanups and preparation for the final bit that switches over to using account_cfs_rq_runtime() for throttling which was provided by Peter in [1]. Following are the results of running hackbench running 3 levels deep with the setup from "Testing" section on [2] when compared to tip:sched/core: kernel : tip tip + series Min : 207.33 202.20 Max : 210.20 222.47 Median : 207.83 218.33 AMean : 208.29 215.36 GMean : 208.29 215.25 HMean : 208.29 215.13 AMean Stddev : 1.02 7.37 AMean CoefVar : 0.49 pct 3.42 pct All numbers are in seconds. There is a slight boot to boot variation for this benchmark but the utilization numbers in top is more or less similar between the two. Additional testing and feedback is always appreciated as usual :-) Patches are based on tip:sched/core at commit 9e005ed21152 ("sched/topology: Allow multiple domains to claim sched_domain_shared") All testing was done on a dual socket 4th Generation EPYC system (2 x 128C/256T). CONFIG_CFS_BANDWIDTH=n was only build tested. Patches also cleanly apply on top of Zecheng's optimization from [3] when applied on top of the same base. Peter, there is only one trivial conflict with sched/flat, and Zecheng's optimization is generally beneficial for deep hierarchies even with flattened pick. References ========== [1] https://lore.kernel.org/lkml/20260512110932.GB1889694@noisy.programming.kicks-ass.net/ [2] https://lore.kernel.org/lkml/20250220093257.9380-1-kprateek.nayak@amd.com/ [3] https://lore.kernel.org/lkml/20260522141623.600235-4-zli94@ncsu.edu/ --- K Prateek Nayak (4): sched/fair: Convert cfs bandwidth throttling to use guards sched/fair: Use throttled_csd_list for local unthrottle sched/fair: Call update_curr() before unthrottling the hierarchy sched/fair: Move the throttled tasks to a local list in tg_unthrottle_up() Peter Zijlstra (1): sched/fair: Unify cfs_rq throttling via account_cfs_rq_runtime() kernel/sched/fair.c | 342 +++++++++++++++++++++++--------------------- 1 file changed, 181 insertions(+), 161 deletions(-) base-commit: 9e005ed21152d4a4bb0ceea71045ff8a642a6feb -- 2.34.1