From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from DB3PR0202CU003.outbound.protection.outlook.com (mail-northeuropeazon11010026.outbound.protection.outlook.com [52.101.84.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 725C343D515 for ; Tue, 10 Mar 2026 10:29:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.84.26 ARC-Seal:i=3; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773138555; cv=fail; b=oxNwYsgNXB/LMb9owKkIOvbyUjP2qq0BE+fkNAmk/OPS8rGbmW839Vb8L3fm71vehrnsEr4WqmJlKttbFpTMpQ30nHKjRDTzJyjh0dLBw7UqFgVKrTQrZDEULYL4/B94YNfZMzUs5bUvlxMCtbo/BKyM4bxDx8O7YKfoywvgL+Q= ARC-Message-Signature:i=3; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773138555; c=relaxed/simple; bh=utseYI4jUvvoUT+2E60gxoZtYmhek3SK9Ldp7VLAGWk=; h=Message-ID:Date:Subject:To:Cc:References:From:In-Reply-To: Content-Type:MIME-Version; b=EL+mbuQ+ZzVVGdS6kxr47GDHvIO+7qu154FKB9TChoTXQhmyPtfCRBSa7Fp0C+We/AkLYo+iLUWzvUnkpcpOvsTVD0d+oz+7uZ9dhOxdkRErM/N+wyEoHVJrgyAmtVSgC57fFiAbRxy3aDsHsjzwJYLH8IC3JqJFsCpCGWzMs40= ARC-Authentication-Results:i=3; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=S5ceQSxl; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=S5ceQSxl; arc=fail smtp.client-ip=52.101.84.26 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="S5ceQSxl"; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="S5ceQSxl" ARC-Seal: i=2; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=pass; b=Yyx35U5gAt5peRYv9SC79FREvGUHuz8wRKyVZ6iBDe+Q7kyL4v5qXYae89l25BmiBwYjfHKJv45lpSH7TP4tzwEO2fL/WKhc9SyQTWAG9zxTrmWfd8yi+2R6EKOssR6IZ+TJWSJGE9d2fRJcJbvgfE15dPDvKL61nGL/Uzlug3yNqVcGLfhsDXhQT2LvTlDp97Mf+oPtM8md6yb9YIsEK/eF9bsU6jbI9u5o+PNhgfiESSeLKoK/9iCYA6Xfvl87fbahpMZ+nRJNLuPCu3gCQJR5XyEWaT3bU7bsUpZUMOV97N1kcIspeDUuvPoPe1i9Jf3M5KHlsbVZ2mvdR+U+Fg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=utseYI4jUvvoUT+2E60gxoZtYmhek3SK9Ldp7VLAGWk=; b=oZnccuTBByWN7TgDNbZj6QLjmpjKB99Sqv9pr024r7Ki5plDJBl0l3KODUM1DGPBhksgjXz8FHj1llfjeZ/aKxdJKh38sW6RI6peBUK7oAME3EhUSwhxszOeE3OAXXSKCcU4U4kLmJjhwGjUcra5FRWF7Qwwk1UZAoK8157OBtL3A7s77FRjiz8H3Rcc+ayugY7/MqqGxYaLP6aPabkMBYoiFVKZ/SJUMouGwA02gy1h/RF9uM0oNHBFTlzDzqSXYpeTMO5ZrsdAggPp5sMEpvNMHqhtiA5Qwi+mEQLtOtg3LptYaeKfmISQwPrcwN+bo/5OZiPiD/lsIo5d0y79lQ== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 4.158.2.129) smtp.rcpttodomain=layalina.io smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=arm.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=utseYI4jUvvoUT+2E60gxoZtYmhek3SK9Ldp7VLAGWk=; b=S5ceQSxlwtLNKIT9vICbIu4rQHhEhXRcPQ86poQDQBH0DJw8vEMSxERillMNLl/4SEZSxnbh6YUWErC7U9rOs/a5p86GVb48T2Zrbdza392FpPqAaGh2QFLT5j+mXEuq8Z1WSgne+4kV36gURExqaL6iDtWqmb1uzGd1YySPaHM= Received: from AS4P189CA0046.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:659::11) by PAWPR08MB10305.eurprd08.prod.outlook.com (2603:10a6:102:367::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.25; Tue, 10 Mar 2026 10:29:05 +0000 Received: from AM2PEPF0001C714.eurprd05.prod.outlook.com (2603:10a6:20b:659:cafe::be) by AS4P189CA0046.outlook.office365.com (2603:10a6:20b:659::11) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.25 via Frontend Transport; Tue, 10 Mar 2026 10:29:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 4.158.2.129) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=arm.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 4.158.2.129 as permitted sender) receiver=protection.outlook.com; client-ip=4.158.2.129; helo=outbound-uk1.az.dlp.m.darktrace.com; pr=C Received: from outbound-uk1.az.dlp.m.darktrace.com (4.158.2.129) by AM2PEPF0001C714.mail.protection.outlook.com (10.167.16.184) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9678.18 via Frontend Transport; Tue, 10 Mar 2026 10:29:05 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rZQl0rCmDpVRZ1uasj1aTO2knaQwvQQUJ4Xn3/BFSwa4eNKmbfZkyqsYQqV8CShJ/NaoVkaTrkY3dsrc34TwTFtDLFGd1uNYu+efvAxHhez6aP8KJ1McPoBQ/v1gx2hkI/6L+ghzrlDdVcfUKqv7+k2y5ldxMt5/Og4xCRC2n4fE/PXFEIKJqEDuXVhqoCI4pUgIfOdcw28BZfD6HCmiq/sJP8rqZkWglWOpa9yvGA3QwTNSBvAi7QexBoTCB2o8ttPtNPnX0oKQ8ofLIG5e3gft/nzeeyzGUfKOrvI2oGWVW45/q/dtypnZJNnKI9mrSb9D9My/8vstgmXuGk/APw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=utseYI4jUvvoUT+2E60gxoZtYmhek3SK9Ldp7VLAGWk=; b=onHcA1JI47cPHnADtWNrSxQWr2bJveRA/oMv154p4QPlNQr9oi1+roBrTCOYt/n4rWHwcDwz746VSd2n0yHlzMvBx7FCjvJAUFLJ0X0c1o0zxJquQ3t6KbELZ0w0KDFyRFiNOfGz56jAIt6OgmeHkl4uhrNNKLa+G12iWqYnXSUj8L0BmsPK0iQVLEwqTSws5qmryZSK+9Sp7aGfetLQ2Bzf0agiedWgchsDk5qzVsFDO31qbVjbgLVncvNQcOGp+fz+RwOTCb8mKx4cn1qfs6D0WQWAaAHW2Lcob5bZLsSCvWAiB4Kr2P0yyb/UTOaFTiwvMwjW9RKFuAna2LVjjg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arm.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=utseYI4jUvvoUT+2E60gxoZtYmhek3SK9Ldp7VLAGWk=; b=S5ceQSxlwtLNKIT9vICbIu4rQHhEhXRcPQ86poQDQBH0DJw8vEMSxERillMNLl/4SEZSxnbh6YUWErC7U9rOs/a5p86GVb48T2Zrbdza392FpPqAaGh2QFLT5j+mXEuq8Z1WSgne+4kV36gURExqaL6iDtWqmb1uzGd1YySPaHM= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from VI0PR08MB10391.eurprd08.prod.outlook.com (2603:10a6:800:20c::6) by DU0PR08MB7690.eurprd08.prod.outlook.com (2603:10a6:10:3a6::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9678.24; Tue, 10 Mar 2026 10:28:01 +0000 Received: from VI0PR08MB10391.eurprd08.prod.outlook.com ([fe80::fa6b:9ba8:5c2f:ac91]) by VI0PR08MB10391.eurprd08.prod.outlook.com ([fe80::fa6b:9ba8:5c2f:ac91%4]) with mapi id 15.20.9678.020; Tue, 10 Mar 2026 10:27:58 +0000 Message-ID: <84da9a4c-8637-4303-91c4-cfd3b558ab8c@arm.com> Date: Tue, 10 Mar 2026 11:27:56 +0100 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 0/6 v8] sched/fair: Add push task mechanism and handle more EAS cases To: Qais Yousef Cc: Vincent Guittot , mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org, kprateek.nayak@amd.com, christian.loehle@arm.com References: <20251202181242.1536213-1-vincent.guittot@linaro.org> <520385b2-545a-4ac7-ae05-06d24845f512@arm.com> <20260310041638.imaqwuaau2hunxby@airbuntu> Content-Language: en-US From: Pierre Gondois X-Mozilla-Draft-Info: internal/draft; vcard=0; receipt=0; DSN=0; uuencode=0; attachmentreminder=0; deliveryformat=1 X-Identity-Key: id1 Fcc: imap://pierre.gondois%40arm.com@outlook.office365.com/Sent In-Reply-To: <20260310041638.imaqwuaau2hunxby@airbuntu> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: PA7P264CA0260.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:375::15) To VI0PR08MB10391.eurprd08.prod.outlook.com (2603:10a6:800:20c::6) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-TrafficTypeDiagnostic: VI0PR08MB10391:EE_|DU0PR08MB7690:EE_|AM2PEPF0001C714:EE_|PAWPR08MB10305:EE_ X-MS-Office365-Filtering-Correlation-Id: efec03a5-6656-47bc-592f-08de7e8fdaf3 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0;ARA:13230040|7416014|376014|1800799024|366016|10070799003; X-Microsoft-Antispam-Message-Info-Original: j/gWMUtlFNNLXZdZ7Ip8ofsB4qCObnRAlKuC6Rhc2pKIm3JaVra8cFpLdj9m5X3pJqNCCim5zHEwOOaY9UBzz8l/sUZqZaWme/CGFFfg9XFEso7K6g+ZuxeqoqMYdBL2XblpC7t/SGYwefUzOI5r+V43fFL+tQcuDYz8wt6snUY3zK2L+lZiRUEP7EIcpIXUCklGOvb/7fp9Skhx/NgfvcKdOd6zl7SCppDsaGMDPABb6FKrdgsMXM4GPrL+2K2BWma/uKbBuYm7WkHGcOJPAbc56IbJZk38Th1RKmvKl2CKj5NP3Nl0OGExgfZ/4HVVfj4AdrlqX6TyseLn0+P59/0mvx89sBFASCsb9McQdQrJSJkx01YD7e4erubS6QXfo9SmPp5Ww46soe6iWGPGalq6JwUIm/3hmMnFXIeT/WodjfjOXxETO03qX7jzxPCzFqKlWBzX2NmgP1btnRW5/+qvKvaZcMqpm/7AQMwcliGaj5vwjAEPdNOyYt+uTQp6A7ugSdczFt2l+kM2od/Hfpz5bNA6h1RfBvLbmwkEw+RVcXI/tXoI9r0hlDNMcwBIttCpfw5ODFCJONSxYoFblnLAKISqdXIngxa2g5ZPAgvgHHZKhVUuzDWm5q+SVcLR9kkVss32+FFCfhoFrFz+ySLsLViPp2SPVdv6a03jkMCbj9nYhL4kDh6iK4bbYENC4CwxPuYP0h35ltXrTnuW8IuMFA9iLPtN6TBGzvd2uruvS9CXx60o3rXPdRJ0HSQm X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI0PR08MB10391.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016)(10070799003);DIR:OUT;SFP:1101; X-Exchange-RoutingPolicyChecked: VqWoMH15kF+CIgAlwRLpe1NK3WMbWXbAppRpDF23/Vr2j/96pRb9RROc7JtP3xrLFkWzToDuTWRjpLhbN07GqUeAi3p/E+yq6D5nC0YnnoVQZa/7oFywTiNZWr6QE9NrAUJ5R8EfVrOibjJyB7Bdipb15mkkKLqT+ZWXqT3We0ABxFp+2kYUH6WzzauYfTStb0XeIZ8Rjr5pLHs2iD94077ROF+KXkoNhP6GXbMEbkCwyYI5pv+cUyMrr8jCJO5pNnJiof+BhXH7jmBVa156QlF0k21JvLVOh2SkXa593RKXdmiehar2R7x/ip70AIbxw6VmfPI2KtluqqmVRrVN8g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB7690 X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM2PEPF0001C714.eurprd05.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: f9464c1d-a721-4052-9c62-08de7e8fb339 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|14060799003|1800799024|376014|7416014|35042699022|36860700016|13003099007; X-Microsoft-Antispam-Message-Info: PeFGsCkHn6j/2g0ZkHRmZxeUQCzr4x9H4Oodp4JlqyLEnnjbxUxVUd/4DdVkijgUdPfrEbx/Qzkf6qk7z+kb9bi7nVnhrayL/ZGgh5msAr33KbiQWsQOks+raQrYvNbPBaQ9FAAZvJFAombVlvV8KEYCkNZEyodRDFizv1qMT4YC2aHmC0EEGPI3X/EtkLn/7K3GxmGl3wfsvUyiJHqUxza88oJ6Y6M8K1uwuydJLa6VXMfTUqBDlK21qan1xq6TERKfiQDoxBsG7OvMctxD9DfahdMofr6e1ikeGWW56OwMFeWvKx5ljmnMedI1DIQ49KDrkykf4FOcTmdGXUKI1gRXuU6vho3uFWRrxOjNaYzOr3DjM4pfcF9H1g6umHxOp2BI+MJUXDhRgwO5pS9SGZbbpepauehlyg1MLiBfSGmu9pqdAc5CjXRzgo9ze2fbYV8xqkDEqu2gBrrSwNmrzodDMtAfotm+pKXbkH4LA4c8hgpYTlaWQbWLKLwgW6X/+wyuRmMB7Uazi7lvyIPBLzJ8NatojwfL8I/atY4tWEHvAFrAQYDVX8KKmWs4DIe0cHawm2WMpAi3ht8+olc3cxblMepfXPRl8NaG1HP5OJZi5Jucq9hGHsCwjTOA4jIfHhec1vXHHNfbP5sOo3/0WF1mokOikBRQdGQIwJgr0zyTvYDrPrYEUIiDuwxUMP+PRrss4djfoSr0KYvKVvpdwx8n0RghCmpLiJ0F3MXwJDD5myS8uWvdKW+EqRDX5xCRPCcdqt93UcP9v5utug9r1Q== X-Forefront-Antispam-Report: CIP:4.158.2.129;CTRY:GB;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:outbound-uk1.az.dlp.m.darktrace.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(14060799003)(1800799024)(376014)(7416014)(35042699022)(36860700016)(13003099007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: fCtpRoq0PPDGvBaNIY0/BZsRhbzZeYHPBn1fshFgGsD0CHfHITX7Wtc3RKGe5ybwUrrlox61b1DJiTPFByl+zAo/PWYvDwvZsi5SN0nk1gYB7mCfslwWH4OqpWV2ZHOPao4zlee2LZ28okvB0gZ7wVU4VWywJAtHhICGFJqCD26BGhVhCAUaheMWd29HSEeH+EPyaa5cHxUlcc7lk+gbxoo75mon+QGyPYflDgaozxUied6RQicyHv0cFBkjBcAK6bB18nOUyQACM2chh5gtrB0MggL0bsUCTpl7pD8TPLWmWa6BFFedZ8neHNMAtRqS//XZr4BGKK8xA1fl+8TwthL6wrIgDWIeSeYUS6Wclc9H71zBis6jIL/Udx9fRVGLAiQyTfG1BG9+5tQ9nwqf5UPfEqC4aPjzIZAUDtXMK8y/7FN6pn0eBW3jFrZr1/fX X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Mar 2026 10:29:05.0872 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: efec03a5-6656-47bc-592f-08de7e8fdaf3 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[4.158.2.129];Helo=[outbound-uk1.az.dlp.m.darktrace.com] X-MS-Exchange-CrossTenant-AuthSource: AM2PEPF0001C714.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAWPR08MB10305 On 3/10/26 05:16, Qais Yousef wrote: > On 02/26/26 18:34, Pierre Gondois wrote: >> On 12/2/25 19:12, Vincent Guittot wrote: >>> This is a subset of [1] (sched/fair: Rework EAS to handle more cases) >>> >>> [1] https://lore.kernel.org/all/20250314163614.1356125-1-vincent.guittot@linaro.org/ >>> >>> The current Energy Aware Scheduler has some known limitations which have >>> became more and more visible with features like uclamp as an example. This >>> serie tries to fix some of those issues: >>> - tasks stacked on the same CPU of a PD >>> - tasks stuck on the wrong CPU. >> Following some other comments I think, I'm not sure I understand the use >> case >> the patchset tries to solve. >> - If this is for UCLAMP_MAX tasks: >> As Christian said (somwhere) the utilization of a long running task doesn't >> represent anything, so using EAS to do task placement cannot give a good >> placement. The push mechanism effectively allows to down-migrate UCLAMP_MAX >> tasks, but the repartition of these tasks is then subject to randomness. > Why randomness? We should distribute within the same perf domain, no? Yes right, but cf. the example below, UCLAMP_MAX tasks will be distributed regardless of the load. > >> On a Radxa Orion: >> - 12 CPUs >> - CPU[1-4] are little CPUs with capa=290 >> - using an artificial EM >> >> Running 8 CPU-bound tasks with UCLAMP_MAX=100, the task placement can be: >> - CPU1: 6 tasks >> - CPU2: 1 task >> - CPU3: 1 task >> - CPU4: idle >> The push mechanism triggers feec() and down-migrate tasks to little CPUs. >> However doesn't balance the ratio of (load / capacity) between CPUs as the >> load balancer could do. So the above placement is correct in that regard. > Hmm. Energy should tell us which perf domain is cheaper. But within the same > perf domain we pick the CPU with the most spare capacity. > > Do all the CPUs appear loaded with max_spare_cap = 0? Yes, as they all have no spare cycle. This results in prev_cpu being picked. In a way feec() does its job: this is a correct placement energy-wise. However feec() wasn't made to handle cases where utilization is not reliable. > > Worth noting as part of looking at enabling overloaded support, it is important > to look at nr_running which I think something we should look at as we evolve > this handling. But for now, I think max_spare_cap checks should distribute > within a perf domain. nr_running will handle this more gracefully which is > trivial to add later for feec(). But ideally we want all wake up code to look > at nr_running and I think better defer it to after initial merge. If we have 2 little CPUs (CPU0/CPU1) with 4 tasks: - TaskA: Nice=10 (i.e. weight=110) - Task[B,C,D]: Nice=15 (i.e. weight=36) Then using nr_running would yield a placement as with 2 tasks on each CPU: - CPU0: TaskA + TaskB   Total weight = 110 + 36 = 146 - CPU1: TaskC + TaskD   Total weight = 36 + 36 = 52 With such placement: - TaskA and TaskB are receiving less throughput - TaskC and TaskD are receiving more throughput than what they would if the placement was balanced. This is not compliant with the scheduler Nice interface. Also the documentation of UCLAMP states that it should only be treated as hints. A more balanced placement is: - CPU0: TaskA   Total weight = 110 - CPU1: TaskB + TaskC + TaskD   Total weight = 36 + 36 + 36 = 88 The previous versions of Vincent's patchset was already using nr_running to help balancing UCLAMP_MAX tasks in feec() IIRC. However this will likely lead to the creation of a second load balancer in feec(), as the example above shows. ------------ The push mechanism allows to down-migrate UCLAMP_MAX tasks, which is indeed a better handling of UCLAMP_MAX. However it is likely the first step toward more complicated issues. IMO the best way to handle UCLAMP_MAX tasks would be to make them second-class tasks, as the documentation describes them: """ Like explained for Android case in the introduction. Any app can lower UCLAMP_MAX for some background tasks that don't care about performance but could end up being busy and consume unnecessary system resources on the system. """ But this would require having QoS classes for fair tasks and this is also a large and complex problem. Another solution would be to force the policy of every UCLAMP_MAX task to SCHED_IDLE. This would also allow to just balance the number of h_nr_idle on each CPU, as you and Vincent want to do IIUC. Indeed if tasks have the same weights, the example above doesn't hold anymore. Using SCHED_IDLE for UCLAMP_MAX tasks can be viewed as a cheap implementation of a lower QoS task class. Their priority is lower than 'normal' CFS classes (i.e. without UCLAMP_MAX set) and they cannot steal time from 'normal' tasks. But: - the higher the Nice value of a task, the less true it becomes. - as UCLAMP_MAX tasks and normal CFS tasks are still part of the same   'class', they are competing for CPU time on the same level.   Thus UCLAMP_MAX tasks cannot be made 'background tasks' and actually   run on spare CPU cycles (and avoid going in the over-utilized state). ------------ So IMO placement of UCLAMP_MAX tasks can only be achieved once QoS classes are implemented. The push mechanism is still a good idea for misfit/overutilized handling (IMO).