From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011048.outbound.protection.outlook.com [40.93.194.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90C5C33D51E for ; Tue, 3 Feb 2026 12:19:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.48 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770121174; cv=fail; b=har6/xakJcnSGFjwm90Sv1lTlHlDj+TxbJCYo/+o6g+JTkQ2QGrc7DbXnIOCk61881gmHz7psje5U0B9Iypm7FyliErXBi+OfMafKkktaqvSydRSr0R6G5Fx0EdwZSfhO+VLsVyzHQ6xQwfu6hmTvmNL2AJAsJu4lPGl1qOv6tc= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770121174; c=relaxed/simple; bh=thJ/LXF88r6nkq6PHeHOhqHn1LWhlpZuY6xVAfgynRI=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=M9yW7uTvWyEJc6GPX0xdFMZDeHaFp9eR5M7fEnmf5Y639rFEydC1B8pk1HZVBVbQJuR0Tc7P1PRg7zpwqOnMuytN1/gV8mXjmnSCgN6OmYswbNbB9ibJf+bcbJMjSxw2j3oe8iDbqdGzktZ3WGx9rOpQZRchUERADgmbhYVYmbE= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=hUFoF2IZ; arc=fail smtp.client-ip=40.93.194.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="hUFoF2IZ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=xM+RDNkmeCGFJlOxyuap2I3Ms3dg3ZhCCHrm64hu8/tnAa40GsTeagAZNgu2Xt75nTSblCU9irmqiXsgtn/dNK3eLQuD4a48QXKcHMqoHZo+prdoCvgWh4xOvWJzc9ZKGdENuJ+oTGLgDFRmApiJ64SoQ94lzTO9RsGzuJfncTWvlWMfQkLD51O/9nbgIOdeAOWMpM2pJrnN0y2oI4rwl7qN99zonWdplv8hYrPtoLeS3xVamtFsp133k+tWckpgx/m6+q8Zc6Q3pAVhlezgp+xe38sAKy+2VmBedUwxeYlYk7opzWzmFwyaBuWgkYHpJxjSKNG4LwOGPvz9hWlziw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hUVw/7XRVkF0T8F/YY/qANEehfAQIAUMpLXo1/C7gC0=; b=Epb2v+bim8H4J73wr+vhURrT6dNPJD9gm8216zleG+ZVdrIhUgs49+WlDfZkNwAF/VT0ZJMBWqCfbZfZVmicwGzRtCoODAxuSM3wHHti/PKpKoV/fNZwxpL/cUyuRZY15q8nWF2RARYVVWJkFMzkq0s6bIYk6dzbBDXNtirZaCFuCe3DBooYG2VOKy1YV33gRlgl7EDU9t9c2miYQIyFPGF1pRqe4zDn02ueON7JrYZRTnb4W8tOddYHhtBRqRB0UYHv/say52IHvly3nqi7c4OCU7fU7232L43nDRfs09OmSYQW9uMzZj3v7tJxHF5iKsKQr6kZUNCh60KNduO/Sw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=infradead.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hUVw/7XRVkF0T8F/YY/qANEehfAQIAUMpLXo1/C7gC0=; b=hUFoF2IZY4KHgfVffZcQjxA6HNNwZrmTCkdo4PaUaxZIpiMWXGf4VXjS1Be4haWurtILUrjx4QXDvJ4nsHxoSCLE4n3ux7bWZ80N2k+LisEABq8v7Se91ycXJMdqJp8SxFBdWGRc3SO3tRrRMRB4rxr73sby5eVaUhz0jNFtppQ= Received: from IA4P220CA0004.NAMP220.PROD.OUTLOOK.COM (2603:10b6:208:558::8) by DS0PR12MB9275.namprd12.prod.outlook.com (2603:10b6:8:1be::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9587.12; Tue, 3 Feb 2026 12:19:27 +0000 Received: from BL6PEPF0001AB4D.namprd04.prod.outlook.com (2603:10b6:208:558:cafe::b3) by IA4P220CA0004.outlook.office365.com (2603:10b6:208:558::8) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9564.12 via Frontend Transport; Tue, 3 Feb 2026 12:19:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by BL6PEPF0001AB4D.mail.protection.outlook.com (10.167.242.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9587.10 via Frontend Transport; Tue, 3 Feb 2026 12:19:27 +0000 Received: from Satlexmb09.amd.com (10.181.42.218) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Tue, 3 Feb 2026 06:19:27 -0600 Received: from satlexmb07.amd.com (10.181.42.216) by satlexmb09.amd.com (10.181.42.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Tue, 3 Feb 2026 04:19:26 -0800 Received: from [172.31.184.125] (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Tue, 3 Feb 2026 04:19:22 -0800 Message-ID: <38ef3462-4c4e-4f40-8d63-84dd71cbd043@amd.com> Date: Tue, 3 Feb 2026 17:49:16 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 0/4] sched: Various reweight_entity() fixes To: Peter Zijlstra CC: , , , , , , , , , , , , References: <20260130093439.803225718@infradead.org> <4c48fe59-8ff3-41fb-83cb-869409f6fbc6@amd.com> <20260203111134.GL1282955@noisy.programming.kicks-ass.net> Content-Language: en-US From: K Prateek Nayak In-Reply-To: <20260203111134.GL1282955@noisy.programming.kicks-ass.net> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB4D:EE_|DS0PR12MB9275:EE_ X-MS-Office365-Filtering-Correlation-Id: 9a3238ef-72d7-44a4-f924-08de631e79b1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|7416014|1800799024|30052699003|36860700013|13003099007; X-Microsoft-Antispam-Message-Info: =?utf-8?B?WG9JeXFHSFBRbjBZUmdwUld4MGNGMno3ZktFb0R6T2t4eGV0UFJScjNaRWhO?= =?utf-8?B?Ulg0UmM3eDZsQ1lrTUltZmp2WHhHcXhyOFFkOXZ2Z3l3ZThKOEF4SDNBYnY1?= =?utf-8?B?a3BML0pzanRLU0REUCtrcDdjT1JERGhuQkN4Zkh0SnlVWUhIR1YybVc0TU4x?= =?utf-8?B?VXpVamNpMkxrUHFlOFBERjNHNFRPMVVvMGRIdU9ZM1BSL2pDZkR3ZTFFRy9H?= =?utf-8?B?bzdvMXNPZEoxdjc2OHNhaktGWmlWbkhvejBmaGpyZ0tWQkswMi9TaHIrUGN0?= =?utf-8?B?TDFHNCt1YXB4a2lBcThSQXZUT3MwcnZzdmt2RDJiZVh1MDBjOUFoRVdkY2tr?= =?utf-8?B?RVJZRG1uYUxLcUNDcHl2RHJMR0dXemlSL043VjM4OVE4UDdiMjJKb04wTzk2?= =?utf-8?B?MHJzMFpnT25sNmxOeHNtd0F6THRBbU16cGxCOGRlTzJNOGg0bTJRNzZEbGYr?= =?utf-8?B?bXpDclkyZkVqVmZ2RXVUcEc4cWZpdHRrOEI5RDY2cVN2czAwQktHU29aRmtn?= =?utf-8?B?Z3FRUGNzcUxQVkJzVVhpSVlpd1RWNHZFOHZlN2h3U2JtTlFTcWl2MUM0QlFy?= =?utf-8?B?aGt2UmpaU1NKQnpmTkxheU1qU3lHa3dxNVlZc0M5YmFsMVQwNVZWNHhudkJs?= =?utf-8?B?YkpvcjZYOVRLUDJEZWxoVVA5dHdHL1o1c3hGZ0V0V0ZZRXFUMzlZY2NzK1pQ?= =?utf-8?B?dS9mb2dBejBFdGpLSnpwLy9Zbit6YnlNZDdjOS9ZclNKcDdNNmZIRS81Y3Ns?= =?utf-8?B?Tld3VDdzUW5aVW5YMUxZRHNIMDYrVG10THRTR21uenQzNVpSSXg5eUsyb2Rp?= =?utf-8?B?Vklkd2lCdWlBZ2ZTc1ozT29LZnpFbUFxOEg5SzN1dTZvekRXUTVqUDNXajAy?= =?utf-8?B?V2prMmx0MGNWTFo5Umk0OGlCbmxnTHorZGhpTm9Idkk1ZFYzMHpSYVpORVpo?= =?utf-8?B?NHkrL2gzbGw0VlFPeFQ4cGUzSGliUDJWdnVWalVNOFRpc2JXV3Bid2FMVXZh?= =?utf-8?B?WldIR21Xa0swWXpMZnBsUEx5VTREU291WFlrUTFQVmo4dWYwcTUySy9HTnZo?= =?utf-8?B?R3orQmRJN1dkbURmNUhSMDFzM2dBcjQ5K0tRVnpYWDB3VllLTDB3SlA2eEdk?= =?utf-8?B?amJXZjNCajd3QkpiTlNTZEhlc1A0a25saHppMkc4YWp5dlpVSWJ0VjZtakk4?= =?utf-8?B?akhLZG1yeVp6aGdITkNON0tJdk5Gek9uMXBDbVJSVC9EMldKdFBqZHF1RW9L?= =?utf-8?B?Qld6NVRRTlI4VDRvTWVlbUVodlhBalczQTBOUHphajNKbm9kbnp2NzFqejdI?= =?utf-8?B?emhmYnFRRDJFaWFESElPZmJQUVY3c0ltT1B6M0dwL0p4NlBFM2VSbldwcXdz?= =?utf-8?B?OE5pb0p1UUxCM2RqaW1LemJOOVRIZk9MNXdwaTdHdjY0alU2RFJ2aFU2Y25q?= =?utf-8?B?Y05jY2ZiSUNEbGw5N2ZxRzd5ell3MjVUN3pVcUFva2YvZjVtUkE3RmNBZ3NX?= =?utf-8?B?ZStFekR5SENUaEtweDV1S3VmQ2RUREtJTDN1eUFiZ255Z3hTSXhCTDhaVTI1?= =?utf-8?B?ano5VzdxdVFBWXJNUDJ0cDd5MllQUjFZalRiSzlxcDZ4N0VkWG02cEpFbVhh?= =?utf-8?B?THdhdC9ST08zbTFqODVwNnROeTg0Z09HdC9tZlNTVDVObWl4NnVSQ1hoQS9O?= =?utf-8?B?N3NIZXFFcysvRVBtbWdkT2lZd2NHR01WRWZEYkhyMWJGYnFlWjRYWVdJb0tz?= =?utf-8?B?dFAxc1ZxRlZ4bllvem0zQkNlbmlmV2prNURlMzl6NzNkUHltSEdzTUhBeGlz?= =?utf-8?B?UEVsQTV1UFBLMlI5WExCdldKek1JNzJUeFVvK0tnaTVmeGRMcTU0TUpwL3M5?= =?utf-8?B?SXRzdUdzRmZxb0pyc0o4ZkQ5L0JVNHovdHk4c1RLYXlEeTVObWV4VUUwblcr?= =?utf-8?B?Q3BwSVdEYzgvVkdNZ003S2pwRFUyNmN4NU9odEsyV1JhVDJ6dmw3QjUzOHN1?= =?utf-8?B?cXY4NEkrWjI1UVNvOUp5OTVjSHlNSS8yZjVEV1Y0cmpvMENZSnRxclhDMkNz?= =?utf-8?B?UldlUys4ZkJIeGJFUTFaNUxBeSs1M2paOGJuYUh4ZDNVV2ErcXVPOG0yZUZJ?= =?utf-8?B?TUpXN1lldEQ1d1RLbzN1bGYza3lFMGhKcDEzL1ppelJlbkUzelF3eGJyYm1z?= =?utf-8?Q?aS067V+NJDS9ARgtys5BZtU=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(376014)(7416014)(1800799024)(30052699003)(36860700013)(13003099007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: XD5uv4tXAdUgBnA+3JbiEPcoMzyQ1xX6HyIzHHBF8x8fX4nBgzoia/vm4h2W4EaurnYsxvP0QPph7yUeD3Ej2zdnDXsQzw8qbzRw4C/2XVs5/nRGPR52LYKAGUJiGuGsw374a11s2zWrJ/xNB3496jClwnZIeM5PjFEZT9YZR8W7KG1174Ia5xmsDN4iw7pYlrwk1In2oWU/hY4Ik4rev/QixeJFTlBx2Ir4FIPf9WiqIQyH2I/FiWQ4Tl9joeBMzuh3XdGVmA1P3mQWSpan9847OtGTEYpyDbwam7pmQ5Hu8+7OmtWCfqx6FKCdSIEsEUJ2ESDl5DAJev/uN473feLLet4Ql6KSrWRuFiWcsr/53XtH1oITj76qYuJF38Wpv4MxEQGatbmasstWdASCtk5pQ8Xnz0C7fkycp8oebeAS3JemM4khq4i5C4SpKtE+ X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2026 12:19:27.4624 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9a3238ef-72d7-44a4-f924-08de631e79b1 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB4D.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB9275 Hello Peter, On 2/3/2026 4:41 PM, Peter Zijlstra wrote: > On Tue, Feb 03, 2026 at 12:15:56PM +0530, K Prateek Nayak wrote: >> Hello Peter, >> >> On 1/30/2026 3:04 PM, Peter Zijlstra wrote: >>> Two issues related to reweight_entity() were raised; poking at all that got me >>> these patches. >>> >>> They're in queue.git/sched/core and I spend most of yesterday staring at traces >>> trying to find anything wrong. So far, so good. >>> >>> Please test. >> >> I put this on top of tip:sched/urgent + tip:sched/core which contains Ingo's >> cleanup of removing the union and at some point in the benchmark run I hit: >> >> BUG: kernel NULL pointer dereference, address: 0000000000000051 > > :-( > >> >> so something went sideways with the avg_vruntime calculation I presume. >> I'm rerunning with the PARANOID_AVG feat now. >> >> Just re-running the particular schbench variant hasn't crashed the kernel >> in the half hour it has been running so I've re-triggered the same set of >> benchmarks to see if flipping PARANOID_AVG makes any difference. > > If you run with PARANOID_AVG, the condition ends up visible as: > > grep shift /debug/sched/debug > > If any of the fields are !0, you tripped an overflow. Yup I see a few !0 values. Some inching closer to the BUG_ON() grep "shift.*: [^0]$" /sys/kernel/debug/sched/debug .sum_shift : 4 .sum_shift : 3 .sum_shift : 5 .sum_shift : 1 .sum_shift : 2 .sum_shift : 3 > > Once its !0, you can't get it back to 0 (except perhaps if its cgroup > things, in which case you can destroy and re-create the cgroups I > suppose) other than reboot. > > Anyway, if you can reproduce without PARANOID_AVG (or indeed have > tripped overflow) could you share the specific schbench invocation you > used? This trips when I'm running a (very) old version of schbench at commit e4aa540 ("Make sure rps isn't zero in auto_rps mode.") I'm running the following on a 512 CPU server: #!/bin/bash DIR=$1 MESSENGERS=1 MAX_ITERS=2 SCHBENCH=./schbench for i in 1 2 4 8 16 32 64 128 256 512 768 1024; do THISDIR=$DIR/$i-workers if [ ! -d $THISDIR ] then mkdir -p $THISDIR fi for j in `seq 0 $MAX_ITERS` do echo "===== Worker $i : Iter $j ======"; $SCHBENCH -m $MESSENGERS -t $i |& tee $THISDIR/iter-$j.log; sleep 2 done done Fails when it is running with 768 workers. Standalone runs didn't fail - have to run a cumulative runner that runs sched-messaging, stream, tbench, netperf, first before running schbench :-( > > I'm not sure I have valuable tracing patches, I just stick random > trace_printk()s in. I'll plop those in and update once the I get a log for sum_shift++. -- Thanks and Regards, Prateek