From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2052.outbound.protection.outlook.com [40.107.92.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2ED763C for ; Mon, 23 Dec 2024 04:34:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.52 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734928480; cv=fail; b=mMfm0/fo56K2tpAGbsfbk6ht2uTyzGhdqx5Z6VUXysfPefhrc8ahP4BnbQJXhhgpMVyd/v40ClhTXttvfmtgAZw0wlNbzEkEI8FJrpW1vuZXJkW+Zvgu/IVQfBcRWphkRvCjnJ8DFRG2Bk9GKI/gahBfWJz+bHaqbYgrOEWh0eU= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734928480; c=relaxed/simple; bh=XsNbyYwxNIDWIC9wb7guIAm9TyPDPekvySfATtHBR9w=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=gRbOgKc6pZqkFA8rBax9lLyz7Sys1wCnJASIfsuJFUaUNKUcwVLaJvZA9TLFs6t8FIkmacbeNcF26HUdKEJrUG2KaLwcEeiFg7GHDn+s1fWJslijdAGQiNnaHOzttN+Ad9EhHMgxK2Bm5OfWzUcFhxxcndYyUE50v2hQ5N778Ag= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=hx8FWeUo; arc=fail smtp.client-ip=40.107.92.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="hx8FWeUo" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=zU2Iu0shQ6AgGrnT3BfAqZkpJDCYNo17yZnyJiHPPKFfmlvHYOwnbijUUB5fiEIXIS6RI+2Um0igVQhOfAlpTuu6eoEthGGFlQkbVspuIYkiPUPul/+q/9BQvRPwsmCjTIyB368HFisLunD/pSulBXApM1HM2Qb01lFE6qBU3YaqBHsCJGEU4IQtQCuEYvxUCRAxRIgPs69F3JvZjLA16jVFoYCfK0tQj7ERFWrT9hT3NcHcC/iyk8EqyIYEP99StY31v9iabsLdVkB1djf6Q3hBLXtU8LhvDOWLa9iBugVs1AGeS2xdqAZkt0exGNxdcqDE3ieJo/G9mu2/J13lvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TUBVQBaduRJzA3rXmhe6cMcrlCxL2+4aVXNmCf7WRzI=; b=ubyV5OhsusqaY2f1XkmPgxq34RoGk/ITrZHeVQLUT1LUX3xGcGBMgQO8lqaxyVnMfYWr8X3VMwUnfyWG8luAi3+hXvTTwWGwvegY/xy0Bc31CpEKeOq0Pj1g2F5CuFAUJu3qUQMpm7hJt78xUYA/RMmIvCVq59m3DNDuDkSRTLSDi0ESOkfhHlKNs7D+5MVrt0sJ6DPbJlj3C0wFxyr4HsxzFQ7KlF/lQL9NxM74r7Jdrnqt4103Az2g0CbHSkk3mCQ85chrOdokyBe1UPZteLZW84Ib8+VpjmSOGPZ0p+ItbCsM8Poc9pLZLPDleY0t1pDdPkvSWbex98PhGpsBpQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linutronix.de smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TUBVQBaduRJzA3rXmhe6cMcrlCxL2+4aVXNmCf7WRzI=; b=hx8FWeUo93TrIZU+klWmy+yZY2Gqo5ypSkv/EYrGJqi3h8wceXRdfoKIijcQxuIkkuAaY9G8JNmKjWtTxR3G2TD62Kzf5eTllHknrvnnDnj87k7Sf9uVenz0RPDrH9Y6OGyemkVRwbvO1VdT8H6c9O+UdrXQqcG27OOVHXVfVqE= Received: from DM6PR14CA0055.namprd14.prod.outlook.com (2603:10b6:5:18f::32) by CYXPR12MB9279.namprd12.prod.outlook.com (2603:10b6:930:d5::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8272.18; Mon, 23 Dec 2024 04:34:31 +0000 Received: from CY4PEPF0000FCC4.namprd03.prod.outlook.com (2603:10b6:5:18f:cafe::cf) by DM6PR14CA0055.outlook.office365.com (2603:10b6:5:18f::32) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8272.19 via Frontend Transport; Mon, 23 Dec 2024 04:34:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000FCC4.mail.protection.outlook.com (10.167.242.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8293.12 via Frontend Transport; Mon, 23 Dec 2024 04:34:30 +0000 Received: from BLRKPRNAYAK.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sun, 22 Dec 2024 22:34:22 -0600 From: K Prateek Nayak To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Peter Zijlstra , Juri Lelli , Vincent Guittot , , CC: "H. Peter Anvin" , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , "Rafael J. Wysocki" , Ricardo Neri , Tim Chen , Shrikanth Hegde , "Mario Limonciello" , Meng Li , Huang Rui , "Gautham R. Shenoy" , "K Prateek Nayak" Subject: [PATCH v2 0/8] x86, sched: Dynamic ITMT core ranking support and some yak shaving Date: Mon, 23 Dec 2024 04:33:59 +0000 Message-ID: <20241223043407.1611-1-kprateek.nayak@amd.com> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000FCC4:EE_|CYXPR12MB9279:EE_ X-MS-Office365-Filtering-Correlation-Id: 7c015ff5-0c42-487e-c81a-08dd230b17b8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|1800799024|7416014|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?873Fw/pkpBXx1oS8KrZBb0CJ4n/vKBgSrsp7I05LwH7o/GGIN0vvXIgJPl9b?= =?us-ascii?Q?CeH6xcdJXQlTCU878R35kehHOlzIjT9OMLSXoL3l1+2P0qAF5UAXJ5ol0Gf2?= =?us-ascii?Q?zS43heNYknfAXNX38We+J/QBhNsGSie+iTpHRnb7Hv6BnmvHbnoJJGI4VFwO?= =?us-ascii?Q?molPMzeYZRjyrXf73bNUssnMNgyUuKZJ5jIlf+Qljhpt6kr9bHTtZgV661Ln?= =?us-ascii?Q?hyk9rPLoL5n+MrItNbOZHoBKDNiUyVIs75bLBVaD5nenKVMA9UPUt5oTq7ai?= =?us-ascii?Q?hYY0vDexovTouuf2wQQB+MKuyZmv2miWGZAsgnqOXVZsgg97dhvUOWRI/YPo?= =?us-ascii?Q?Q9BSDfs8MdHE3TAVWbw7iYN/GUKces4YD8aO+NH1M+q7DpxZnb1Ex0Y4YuIE?= =?us-ascii?Q?5XMJB5r8HcA01i5O748lmO1H0ULfyyazXqQQfRBw89pLn7dYwou6Eo+vhaQY?= =?us-ascii?Q?dPRaCnbziJXZlAFeSVvkAH6olP7ODq0839lNGm6IndWZxSBj8jCes4UyRMDh?= =?us-ascii?Q?FWFMm9cbWxDWQfuyrnY/VawTbcbp2eqHwPcu59ETNue7gVKEYZkrfCtBTO7i?= =?us-ascii?Q?8bYPS06DrJYt2bCSv6SWZgr5sgepTWnGSZTC0uB+fscWwA7QucHhjPXVU++1?= =?us-ascii?Q?qD+nQJrmZbqeifbZE2QgJxSe6fqyBMrJa8Pyx2xDP0WY+bnJwxwOUyntHtdq?= =?us-ascii?Q?ZAcXGp8SQCc9ZZOLLvz1jsVfEhkX45NaDyqtawrbTwdp+mEYGVT8mxyAjm7s?= =?us-ascii?Q?vCSP2ulhOg2V87ZMZKJiyY6bsTQzT0ibkH2cSmXhuCsNe1usu5ZoPJU90mxX?= =?us-ascii?Q?ByUYgYJD0miazIOeMobxSBRbxxdTMWyht9awQMz4xqnW8hPpJRW6m+KZ5FIB?= =?us-ascii?Q?9RMh3pk+lDsHyldjHkTPLoJ0CBW4t0ax36TEpt4/frUygnFrLpz8pb244yDY?= =?us-ascii?Q?a10YvHCZrEi/7e3tLyzaSkSw6s7fSFzgFuuXOij0rDQaazJCKFcMN0lVlUst?= =?us-ascii?Q?MDVU++bIpuxZ+CooUN/1ogz0v9Lom1YvvkGh5IrbQN7y1qiFAMnUkfTX3sXE?= =?us-ascii?Q?/75Brc/JnZi472zO+ehBd5V2PITY0t8YOLubRa26xqHVnuCBJ/zLyMgeMpml?= =?us-ascii?Q?YeolVqn72qOFvHLPYLW6ylioBFN1Ad9hyQqbiq24ojmpfVqR1/NiGN1fcX5B?= =?us-ascii?Q?HO9mNE8ZI/S00NNODPpJ/rZZpEkQ5DIsdp2zBv8GhEFtHldWtKr1xqS2Ch+w?= =?us-ascii?Q?WGrZOYieK7fQVaF0ApG+yl3o6C9UV9az4Vrov4QPC/DESJFcZgRenVEIoKTu?= =?us-ascii?Q?OFSTUiVC/T96X6naoDlJn5Mpm9930NkvrMjYAL0Mg4gfFXy7DjcHDDDshpt8?= =?us-ascii?Q?zyiG4Remupdc6DyX/9HxAbJJKlnYtqR9E3vqSMAnVrd6GLorJMwynMpvad4x?= =?us-ascii?Q?I0bgloZnBzI9kKiuWmBYHO5zrc8gRdN/?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(376014)(1800799024)(7416014)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Dec 2024 04:34:30.4514 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7c015ff5-0c42-487e-c81a-08dd230b17b8 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000FCC4.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR12MB9279 The ITMT infrastructure currently assumes ITMT rankings to be static and is set correctly prior to enabling ITMT support which allows the CPU with the highest core ranking to be cached as the "asym_prefer_cpu" in the sched_group struct. However, with the introduction of Preferred Core support in amd-pstate, these rankings can change at runtime. v1: https://lore.kernel.org/lkml/20241211185552.4553-1-kprateek.nayak@amd.com/ Tim confirmed that that ITMT changes will not alter the behavior of Intel Systems that contains multiple MC groups in a PKG domain and support ITMT - both current and future ones. Patch 8 uncaches the asym_prefer_cpu from the sched_group struct and finds it during load balancing in update_sg_lb_stats() before it is used to make any scheduling decisions. This is the simplest approach; an alternate approach would be to move the asym_prefer_cpu to sched_domain_shared and allow the first load balancing instance post a priority change to update the cached asym_prefer_cpu. On systems with static priorities, this would allow benefits of caching while on systems with dynamic priorities, it'll reduce the overhead of finding "asym_prefer_cpu" each time update_sg_lb_stats() is called however the benefits come with added code complexity which is why Patch 8 is marked as an RFC. Srikanth confirmed it works as expected on a PowerPC VM however, there are no comments yet on the performance impact which is expected to be minimal if any since update_sglb_stats() is in load balancing slow-path. One notable comment that has not been addressed since v1 is moving of overutilized status to below idle CPU check. On an idle CPU, since there are no UCLAMP constraints, the cpu_overutilized() boils down to: !fits_capacity(cpu_util_cfs(cpu), capacity_of(cpu)) But the averages can capture blocked averages and capacity_of(cpu) can depends on arch_scale_cpu_capacity(). I couldn't say for sure with 100% confident that an idle CPU cannot appear overutilized as a result of blocked averages. find_energy_efficient_cpu() does not look at idle_cpu() and only performs search purely based on utilization and capacity - an idle CPu that may appear overutilized will be still skipped in this search path. If there are no concerns, that update can be moved to below idle_cpu() check too in Patch 6 as suggested by Srikanth. This series is based on git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git sched/core at commit af98d8a36a96 ("sched/fair: Fix CPU bandwidth limit bypass during CPU hotplug") and is a spiritual successor to a previous attempt at fixing the x86_die_flags() on Preferred Core enabled system by Mario that can be found at https://lore.kernel.org/lkml/20241203201129.31957-1-mario.limonciello@amd.com/ --- v1..v2: - Collected tags from Tim, Vincent, and Srikanth. - Modified the layout of struct sg_lb_stats to keep all fields concerning SD_ASYM_PACKING together (Srikanth) - Modified commit message of debugfs move to highlight "N"/"0" can be used to disable the feature and "Y"/"1" can be used to enable it back (Tim, Peter) --- K Prateek Nayak (8): x86/itmt: Convert "sysctl_sched_itmt_enabled" to boolean x86/itmt: Use guard() for itmt_update_mutex x86/itmt: Move the "sched_itmt_enabled" sysctl to debugfs x86/topology: Remove x86_smt_flags and use cpu_smt_flags directly x86/topology: Use x86_sched_itmt_flags for PKG domain unconditionally sched/fair: Do not compute NUMA Balancing stats unnecessarily during lb sched/fair: Do not compute overloaded status unnecessarily during lb sched/fair: Uncache asym_prefer_cpu and find it during update_sd_lb_stats() arch/x86/include/asm/topology.h | 4 +- arch/x86/kernel/itmt.c | 81 ++++++++++++++------------------- arch/x86/kernel/smpboot.c | 19 +------- kernel/sched/fair.c | 42 +++++++++++++---- kernel/sched/sched.h | 1 - kernel/sched/topology.c | 15 +----- 6 files changed, 70 insertions(+), 92 deletions(-) base-commit: af98d8a36a963e758e84266d152b92c7b51d4ecb -- 2.43.0