From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from SN4PR2101CU001.outbound.protection.outlook.com (mail-southcentralusazon11012061.outbound.protection.outlook.com [40.93.195.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B91A61367 for ; Tue, 6 Jan 2026 03:03:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.195.61 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767668607; cv=fail; b=VCEaN3xIt+BZYxk0bdMGaPH0j9CIZN6yaMffZ5CLxMZN+dc0kTyGNpvKrI2F7294doT3z4Ujuc7zgJbPBg0fD41wDAiCWtMWZl6IIXFOcMXqlvV3gWHnP4bIB4ASXN+tMablGHgo1s3WaB8ttR6YWTvoK5eRMBEW6WZHC3D+G+M= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767668607; c=relaxed/simple; bh=C1Tu7ubwQ3opbBSpIgXl3hlbfX5Dz2ey/Rx3m1a3bJc=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=Kj1a7AC1JZ7KB3XqJtFvcUm7UiIFe0SVJq2BEGFlssBwg8UEB+m7KEx6tFBgDYeWjeP4h6FFVGyPKfm3uqe5cJp8lIvigrxeFC3w9cAjCXlfRXUvmQOI3jEArMgDbTBBxXPazx8hL2UrEL5lad58pzG3Q0d2Rqp6COxOakSa940= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=jBN7CDvZ; arc=fail smtp.client-ip=40.93.195.61 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="jBN7CDvZ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mYGXhkP35yMmWEO1pnHZ1ml0rThFo1sg9VUuXDZ+uRjqPEUv2EVXkyKnILaaZ7laTkz+nm5EgzFwCI25Bh0gl0cX4cNJSxhtC2CRUHLvmnAhlua5oD9GsZuIz/a9L1TwBN5IpkFZ/RUh6vuXfnDOEtQzrRsWudhBbD2JNH1GbvoN4uYYEKcNrVSWZm2FGM9eNSrIgHzlVhD02y95UWLKBMH8dSwupsj963TnGtknbiRvKlm94eGoMaLhXPwAUA14x9Bk3aZbNVeMDCSZRHn2d2PPkIjINhD8RBwC863AHcMyfLFB+WFVX7EGRahMNpzag25uxUyw8v48mTdkqvVLxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ChWe6eq3CZ/ZGTv5K6dvUncrQeiW7i3q0KOaG8LTx8U=; b=lxjTs0hvvkTR64BW9nZJXdlwxHxw6GXXK2AAtFYtMxh+FkmPWxkqO96uzPzHUahBkedmLmrjieBzYsffKYpHcm9d13WEvvMmTol5ZtFdOhK2nkZxsFCu2+3zWKA9PkSxpYNBEp1jxjefRTw0aUyZiUpQM2gZyBSCYrHejgEeB7Nc+FJDxxeWr/7E3upzvRWgfqGqMifz+8E5fWq24zEt0T8nr/Sx9Zh6Kk0OW9Cw0Q+cr3N4ofiJvLw2R+uz5Ndmg7qzIbCxQupOBt7SMo/0Kap0roqtToC2rJTQbd/VcSbgnJLwL0txdpKkBIHCjK9CVZ8nbpWCugjW7TcxhF4jlg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ChWe6eq3CZ/ZGTv5K6dvUncrQeiW7i3q0KOaG8LTx8U=; b=jBN7CDvZDAlPm7m8MGSj/lAVhi34hc64+O2L68u20iu+L9iQQJAJCZiCuzd8FEz4+YRl/7R2LlIZq++2IYDHQAefyRTC95HTBjQ0voZJloKogP5eNS57WV/5xcWqDmdedA/VU7i0WarKU0O4w+baVbKo4NjqCWB4vTh4jBs5nVU= Received: from BN9P221CA0029.NAMP221.PROD.OUTLOOK.COM (2603:10b6:408:10a::27) by DM4PR12MB6254.namprd12.prod.outlook.com (2603:10b6:8:a5::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9478.4; Tue, 6 Jan 2026 03:03:21 +0000 Received: from BN3PEPF0000B06A.namprd21.prod.outlook.com (2603:10b6:408:10a:cafe::bd) by BN9P221CA0029.outlook.office365.com (2603:10b6:408:10a::27) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9478.5 via Frontend Transport; Tue, 6 Jan 2026 03:03:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by BN3PEPF0000B06A.mail.protection.outlook.com (10.167.243.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9520.0 via Frontend Transport; Tue, 6 Jan 2026 03:03:20 +0000 Received: from satlexmb07.amd.com (10.181.42.216) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Mon, 5 Jan 2026 21:03:20 -0600 Received: from [10.136.46.19] (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Mon, 5 Jan 2026 19:03:17 -0800 Message-ID: <7050ce6c-64d6-4359-9a4c-9fd4362e5290@amd.com> Date: Tue, 6 Jan 2026 08:33:16 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 2/3] sched/fair: Change likelyhood of nohz.nr_cpus and do stats update if its due To: Shrikanth Hegde CC: , , , , , , , , , , References: <20260102124744.360872-1-sshegde@linux.ibm.com> <20260102124744.360872-3-sshegde@linux.ibm.com> <880e3610-dbdb-42a8-9ddb-ab2e7d3cdc1f@linux.ibm.com> <71485a68-c0b0-446a-8326-7c10c583e076@linux.ibm.com> Content-Language: en-US From: K Prateek Nayak In-Reply-To: <71485a68-c0b0-446a-8326-7c10c583e076@linux.ibm.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN3PEPF0000B06A:EE_|DM4PR12MB6254:EE_ X-MS-Office365-Filtering-Correlation-Id: fd15b387-47c7-40b1-32db-08de4cd025f4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|36860700013|82310400026|7416014; X-Microsoft-Antispam-Message-Info: =?utf-8?B?L1g4L2hDR3lOZnpsZ0FxUkNpS0lVQ3JkQlprS3dSdWg2WGszUTBqM28zVGhD?= =?utf-8?B?S2ZRY1dUWFVvR3FXL0orMWZzcGdCK2ZQQ2N2Wnl5RDREUDBJTTdjUWU5aktG?= =?utf-8?B?eVE5UWlnWFRTVTl1VzROMklNMTJtamxOQzhKQkhKck5weGlIMUpuaCtVRHhY?= =?utf-8?B?RFVQTWlXSzBzMW0zYnR6R1B3dXRzSjBjWnZvMjh2OGZFRmd5ZWNnZkN1Mytk?= =?utf-8?B?bktKaUh2RDRuT0pBWDMva3ZTaFdra2k2a2JjdndiVUwvZGE2WHFjVkFPbE5y?= =?utf-8?B?S1hGY1RxNWVCUWRuNUtJSkdPTnk1ZTdabUxScGpEN1pDMDErWS9Ed0JkVVNK?= =?utf-8?B?dEFNak1ZeVdldE9zMGdlaitvWUVDbE0zczNEdzJpTlpUMllFUExGZWJRUkJN?= =?utf-8?B?c1F4dWFvR212alYzOVYwTUVxbFpaenRmQUFpU0RMQ0UxcTNnWmR0UmNUYnEx?= =?utf-8?B?d1dGLzVXTWp3T01Tak9GdC9sbFphWmhYRnBma3krM2N3ZWdrSkV2M0FKWjJ4?= =?utf-8?B?MzhiWENWWS8xRWFFQng1dDdhUzlnNDlMVjdtMS9XMzA3S0c0d3RwRnBCRDNh?= =?utf-8?B?UVNMa2YvQ3kwcVI3YTA4bll2ODZuckR3U0p5SzVqUzZaTndNOFhGNkRxUHB4?= =?utf-8?B?UlJWZVFGZmY5ZmZ2TXNzaGdTMkpIR1B3YVV1c1ltRitQRTlwdkNjZFA4bTJ1?= =?utf-8?B?dnVOTUdqTDA3TU5lK2QzVmFrZlo3VG8xSzJJTkVUWXd3cG9YSHoxZHUxckdW?= =?utf-8?B?ZnNsbHZhV3Q5RDBmNkpiUEtZRWRCZitHSVFrS2l3S3ZacVljZGxxeWpxSFVF?= =?utf-8?B?VXE2ZExkSHlZbThmaElQaUUrK1NWUUxjSGtPVUN6Z1pxT1ZnazJHeWI2eXZP?= =?utf-8?B?enFiNGk2SjR6VXlPVnJ0N1NqaFdsOUhUSXprYmo4QitSNHpUMW51L2pkQVFR?= =?utf-8?B?ajdkc2R5UUtSZ3BNcmk2YlhOWVhGK2s0TENBS2JlaFlwQmt2OXVsbmkzUWlo?= =?utf-8?B?cndQRTNScXFsTjZObGxvdjh6UGFMOXZsOE5GT1k4UXdHajlRdCtvNlphbTJH?= =?utf-8?B?UXl4WlpiYkluUDdVYWUxbmt4VGJrbU4yMk9QdGswc3JqaU5YaVZjd1lSMWRY?= =?utf-8?B?Q2tUMnIzWnhnSkY4RXpHZXorU0FwNXdoK2xLakxwcTErbytQUTQzR0dtcUor?= =?utf-8?B?TUsrK2MraVVkZlI5cGlYamlKWXQ0KzJ2SmE3UXdaQ2pNc2JmQ0ZvZnF5M0V4?= =?utf-8?B?WEN2Zld3THIzUTJrcFlwWTRaYmxoMTBVWFlnNCt3RC9BWTJFUW9GZGtqV2hW?= =?utf-8?B?cW5qbXkxL0p2dHVRVEpmWmp4d0ZqOGhYNGhHNlJ6dlVtcWtRd3hyK1JxTU9w?= =?utf-8?B?Tnk2MXhCb0V0aW1WcmxFdTcrNFNJSHA0eVRDWFlFN1hZdlM3QXIzSzdreVpk?= =?utf-8?B?M0UxdkgzdCt3SW9ISlRxaklEZy8rc0xyNzg1UUFiTUM2dmJjVHZtK3ZWN1JM?= =?utf-8?B?SEduQlJBZTF6Q2VZTEUxdEZLVHRJWUlXMUxkeVhzTkg5cFo2dEVBaGwvQlVP?= =?utf-8?B?N2VZeGxjbE9TU0loaUJWR0xRSDRjbFRZMmV6K1Jsd2c5OVdJT0ovYXNkZmov?= =?utf-8?B?R2M3SHlUeEFNNHhNY2I5Z0xaV0tyU3AyQUVlNGIyNnNBSzY4aGR4OEVtTnVN?= =?utf-8?B?TEp3Uk4raEdjMkUxMmhaRmdCaHlZbVBnOUhlVmtLUEFZOXBzUGJkWkxQalJx?= =?utf-8?B?SCtHQ2dEY2N0bmtvN2JkZnFMQXVVb3I3WHdjYi9JNkg0czJEV21TVnhLVTU5?= =?utf-8?B?bk9FSDNwMGJYeUFqZ25kU1hrUnA2MFpsNjAvN2MxRkk0bUI2ckJ0Y2hEeW9T?= =?utf-8?B?MkxLV2VlYm8xS0RhZE82Mkc2VlUyZFpNOVRibm9PZytFOGNsVUxnWDROUldv?= =?utf-8?B?MEZVUTJCcmRHZHJhUjNLM01UZFVpd1lwdk91MVl3R29ZT0QvdytyMk1EY0c0?= =?utf-8?B?VkVuMTl5SC9Zdjd4U1IxQXBOQURDdlowRm9BQnBFdTFSUVlKT1N2ZU9VS3NW?= =?utf-8?B?MzIrcjVXNzlqbERMR0xZazBjdHRoaU9tU1ZUdXk3Sll1RUsvYTZXZ1Jpdm02?= =?utf-8?Q?ET+E=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026)(7416014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jan 2026 03:03:20.6428 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fd15b387-47c7-40b1-32db-08de4cd025f4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN3PEPF0000B06A.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6254 Hello Shrikanth, On 1/5/2026 4:09 PM, Shrikanth Hegde wrote: >> So cumulatively, including Patch 3, we do: >> >>      flags = 0; >> >>      if (READ_ONCE(nohz.has_blocked_load) && ...) >>          flags = NOHZ_STATS_KICK; >> >>      if (time_before(now, nohz.next_balance)) >>          goto out; /* Checks nohz.idle_cpus_mask in find_new_ilb() ... (1) */ >> >>      if (unlikely(cpumask_empty(nohz.idle_cpus_mask))) >>          goto out; /* Still goes to kick_ilb()                     ... (2) */ >> >>      ... >> >> out: >>      if (READ_ONCE(nohz.needs_update)) >>          flags |= NOHZ_NEXT_KICK; >> >>      /* assume either NOHZ_STATS_KICK or NOHZ_NEXT_KICK is set */ >>      kick_ilb() >>      { >>           if (flags & NOHZ_BALANCE_KICK) /* Not possible */ >>               ... >> >>           ilb_cpu = find_new_ilb(); /* Find CPU in nohz.idle_cpus_mask */ >> >> >> If we arrive here from (2), we know "nohz.idle_cpus_mask" was empty a >> while back and we've not updated any global "nohz" state. If we don't >> find an ilb_cpu, we just do: >> >>          if (ilb_cpu < 0) >>              return; >> >> So why not simply return from (2)? >> > > I see, kick_ilb though called will not do a balance since ilb_cpu was not found. > > I don't want to have that return in between the two out's. > > How about we do below? When there are no idle CPUs left, both has_blocked_load > and needs_update should be reset. no? > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 805b53d9709e..fa0e6065bc9c 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -12377,6 +12377,15 @@ static inline int find_new_ilb(void) >                         return ilb_cpu; >         } >   > +       /* There is no idle CPU left. > +        * reset has_blocked_load and needs_update, such that unless > +        * some CPU enters idle state, it will not trigger kick_ilb > +        */ > +       if (READ_ONCE(nohz.has_blocked_load)) > +               WRITE_ONCE(nohz.has_blocked_load, 0); > +       if (READ_ONCE(nohz.needs_update)) > +               WRITE_ONCE(nohz.needs_update, 0); > + I'm slightly skeptical - find_new_ilb() will also fail to find any CPU if idle_cpu() returns false momentarily. Those CPUs can again go back to idle without updating "nohz.has_blocked_load" - tick_nohz_idle_stop_tick() will skip nohz_balance_enter_idle() if the CPU already had TS_FLAG_STOPPED set. For those cases, we'll need to retain the "nohz" state as is until a suitable ILB CPU can be found - this is the reason why we delay clearing the "nohz" state until _nohz_idle_balance() and reconstruct it once the CPU is done with idle balancing. There are also nuances like the smp_mb__after_atomic() in nohz_balance_enter_idle() which requires us to check the "nohz.idle_cpus_mask" after we are done clearing "nohz.needs_update" and "nohz.has_blocked_load". >         return -1; >  } > > -- Thanks and Regards, Prateek