From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from BL2PR02CU003.outbound.protection.outlook.com (mail-eastusazon11011014.outbound.protection.outlook.com [52.101.52.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8BD530FF36 for ; Wed, 1 Apr 2026 14:54:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.52.14 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775055276; cv=fail; b=iCcTkDZ4SwQ5lFJniezYiVrBcW4Lq7BFfAU5eitInObWp5Vo8nOHynvqCBF9EF1KSwDFtp08wpqheksi7jetKnWHdDAmvOkIqOb9ivuUsWusKlaAGXdVVuZJF6kgH/qctB7kvxNurUynSu3iUZ4Lj+JExhdw4G/NRvFqiJ+0iuw= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775055276; c=relaxed/simple; bh=yjXZixKTZPE3WI4HQvpReDy2ijyQY1wjWPF5gXos/8s=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=Krj+73SmbUWaAxpJ3emPFnX9fSyjnG6I6ODL+qSduIxncAn6tPceJP7ioHCj7eKVXU/8ysSW8uJduXOwZhuXyOYFX2bT4n+ONwmO5yMKcY/0TfgKFS9GeT4OvJy7xvp6DKo7EsWIZQy54FLaiXYzs5jmmYosWynQiDvc4IME+1k= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=jHAZ+vZC; arc=fail smtp.client-ip=52.101.52.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="jHAZ+vZC" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cjV0nJ6G2hHOay2aFCwoKdc4Fa8sg1jHWyqMXRq1WDrAMQVQ8/5TYiKcAnQyHcj0HA/g9NpOmPwSG4NwHC0lrjodoDZ8uWU9qiEhGMLZMXglIVXIBMcL4CQO40568+eqx2W/ihCjspySBjx/2+4BUKM/AgpABmP3xKtjatDg75zOnAaOohfzISkMjw3cCIBke8bYEoc7XYnsMZSUeI9JN0roqqXcq/kjvXhafmXXh7EoFaCQCAWUFWzYtI0rVsKsRyXOs9UrMzY8z3Dk2S1ZZQ6JO58tHXINLXEbpdkoT4QHEsW6aUd83PM46rLJllzhTGt6JQOgX5BzL5nW8WS5lQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Xh4IsIufQI101eG7V/+fmA6S0vpbRBEHcMZJvlaKhow=; b=mTHHL8okFBygP1JPCoSY2fLoX7UkZRBmR04fOgKDyfesc2H8PfS5Hda1RDtTwLTwtMWBOBDnnThkakUHHCcpAT1dnHXdV6Taf/veuJGTCZo6OPvlErGNBVvuldRGteaiZH55z7AfCDFwYjbumADQ9MvmFuhKriOPsPaWlRtF4CDU7iC1s67DD7ZkpKRw9Z+ERgyS7nP65JsImC4kUQwLCNs7Jm2WlSNZs4CMtWheUdrf/sTj16fKwtex7L/LoMNwhXW/khhnjA1x+AQt2jZ9yWKdGxQ6KSHPcmElHcBGx0srityZ6SEt1g13ZNeK5EBXCFEtn3/ojCXLXO9V3LrTPA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Xh4IsIufQI101eG7V/+fmA6S0vpbRBEHcMZJvlaKhow=; b=jHAZ+vZCNP7lent9Er52zloMEia4EWk+2BtA60+YG8C26X0mbx9lYxFG5snXxj2oIWbzG4FwhvEQ6ipnCLsXmyZ7VFRD9hm/4xMuZEVsxTj9lLu5rffMlpz06MHvRGiFPo/Ux0n/F9H2kcfucqJsk8yVpfWV9fpH5HM8utsHVZWQpURVQyr+dSo0jPu/G+kZ8jDxkPYNMB7128/nus7ejzmp4xHB2SC99/JF+oYXMhsFws+5xFAFhggVu89XJxGZ0K3kRKDqRr5sefTBum4rR4Ik6vUXfCgBGhfZt6vP5kQBwaFi3YAHyCOz1XAMm7MNH9r1hKWcnrXsA1bIObGQ3Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CY8PR12MB7171.namprd12.prod.outlook.com (2603:10b6:930:5c::20) by SA1PR12MB9469.namprd12.prod.outlook.com (2603:10b6:806:45a::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.12; Wed, 1 Apr 2026 14:54:25 +0000 Received: from CY8PR12MB7171.namprd12.prod.outlook.com ([fe80::4487:395f:3abf:ad9]) by CY8PR12MB7171.namprd12.prod.outlook.com ([fe80::4487:395f:3abf:ad9%4]) with mapi id 15.20.9769.011; Wed, 1 Apr 2026 14:54:24 +0000 Date: Wed, 1 Apr 2026 10:54:22 -0400 From: Yury Norov To: Valentin Schneider Cc: Peter Zijlstra , Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , linux-kernel@vger.kernel.org, Yury Norov Subject: Re: [PATCH] sched/topology: optimize sched_numa_find_nth_cpu() Message-ID: References: <20260319172607.926280-1-ynorov@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BN0PR03CA0055.namprd03.prod.outlook.com (2603:10b6:408:e7::30) To CY8PR12MB7171.namprd12.prod.outlook.com (2603:10b6:930:5c::20) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY8PR12MB7171:EE_|SA1PR12MB9469:EE_ X-MS-Office365-Filtering-Correlation-Id: 986dc570-0112-4e39-6cb0-08de8ffe90af X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|10070799003|7416014|1800799024|376014|366016|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: BTm/YDPe/U+bZBKlGr+5QmBNRxScpYpuhFJ4aB31KwF+v2RQU5yFD64aH/xF1mDFeBiR62Do9oBtpWSp87DG9BKU4jjh5BaPJBfu6JwcFBbxg/7Uc3Xdcwqc0xfWDmoh1tZvu1Av3k6twJY2KI7sNPj4WLYQ0RNL5BQ1jBn3AnO3P2geqYLGV+h3EjgSA6/Dj3v9itT4Er0MYnlrZVOVKHiS5BPUkxZehQ11irNfJngiX4+7+mGEPy91NSbIYTjBwQ5bfyMExseCJJY2KK3Hd+wd8BnNAKX7imARUeO1MhMzYmxnVu6EB9KVMvV9DB5h85bcRbyCifZ1j5CXCOLTKdGXxauto1K2y3rSdx1qFDEPjbg5v/G40oAjUHZG/45BYD9h44UrevtMyqpSSlyddCtaMaV1IcScZERzWFgXWQvAucWTKXRUJWf1nBIUmQrDH5LMtMktcTN8UKLXL83RLwXWMhyvDAddkq4kbLICBCG6Af4suAQrhMI5rbHbVJVNSmbnl9n26Dk1p87ebLJmLjpDEQX3EU5MsEvMY7Es4ePHKFD5oKBVMEyC0PiPZ+80ahbDOtPFfZqBRVPNwpfeMvOoIad2VasIc8C+uHi6jxD8lSP+bTd3KnEOm/7e2fPK6mKbdqfrKJ0Y0jEn1YwTSCzdaCb1x5BeOaL6HWrWKP+WuO4/iLqYAStWxnao9z/1rW6gSEDCPYN9pYmIGRUx27gaf9TQtvxmkEe307kU5Z4= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY8PR12MB7171.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(10070799003)(7416014)(1800799024)(376014)(366016)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?fQ7+/pvSQIUf7qUrtpUHNRm7XedjJ2o/L8Qf6mrhlo0QQSjQN/IEP+XtvIyh?= =?us-ascii?Q?ylrkxAsbdHny8ihlff4zW0AGGwiH5CmLWlvjMPmWeSO4sJwIKO8aYPbB7I+v?= =?us-ascii?Q?LJB8AQIIh70ezmsIaY7SB970Nbcse0QjL26m3iEzSwQ/WtOZMTVJ/4tei6P3?= =?us-ascii?Q?gN9mWRMrxzdRVdMGOAZVaPY4F98T6aA5oKjs7GR6HdmVJlNb+BTCJfs3/v4O?= =?us-ascii?Q?WHOY9nluNDqPU2h+dPnQ+o3V1onzCnw70U7Mgzhiq9BcfOC6vHIyINPf3UER?= =?us-ascii?Q?eCDcHzojtRKoOaqeuK6dx8XSuRK4A3eS3QJX25KQJOFFa4XA75HGWQNtouD0?= =?us-ascii?Q?OR8Ccf9tmh25Hu04yPw/12rwGbzNNa/6ZPIFL52K/J/eyzxsxjydL3C13E7r?= =?us-ascii?Q?nXa2hwHLivxPutT18+q2VvB/TMBNL2Y7s74RqHcgDT18rR/7RDk/xnmGgFjN?= =?us-ascii?Q?hLyl4e6En/wL+uupnkW02vfV7Sw2Le86L6SyxesxcinHAnZ8LbEGBLHZIz//?= =?us-ascii?Q?lQhPArawVpFDw+Cg932VvPQAFVOlfj33E7dH54T20wHWAHn5JMoQPHLANXfX?= =?us-ascii?Q?vWOlNclOVEMKhLlUHeujBy4Z62gU6+ds/aAiLB9hhHxYv7oXriBRwwZvDX2D?= =?us-ascii?Q?HM9QnctT9IcPqlQAHzebD4hgYSoIasVdpS0Wr1h0H1bYlIpjBIhT28Pyr6Ib?= =?us-ascii?Q?mLkvP5wRMT943shpPUEDeJ26181Eu+gcTEjpwSdxI2dt6aZX/ZzlZY1Iwnza?= =?us-ascii?Q?svzy4vrLPVUZ6ECHzutGScTSRUkVfxpcOrdv+4bu/lpeghI9U5CZUfvxoq5o?= =?us-ascii?Q?kKULWqna4ajRlCmcnD7XqSdgWZylEBHzVEOdUklm20DpNnhtRWJ9ho7j4v78?= =?us-ascii?Q?EsiIUsXJcf01NOnTLMbZKZu+wjatSMYmrbuxHsYUDnDUNV0P20DqnSoIN7lA?= =?us-ascii?Q?LQxHZrt1Kiqqxq4ewRAE9KZetn4IykL6vpy/SYKXl5zoryILOo0uQKhlaf2Z?= =?us-ascii?Q?8sx3nM8zveNefqWq+52X0K7lBfjcccSrm7YvS0j/a680mhITOi5xJs3Zm/EZ?= =?us-ascii?Q?n2QWPR+6J+rzdT28rev6T12aMgib5R2f3YzDtqA4eovQQByNusR5fsOXFBq7?= =?us-ascii?Q?6K+A1ch0Czd8o2kTJ4L6HutabGVEzwdet7qJ9nhHqsurO8mYywGefDHL8JmM?= =?us-ascii?Q?8QJMdztBxsV38jvESgJfFmyxO8zJgfEarmYT8MIZ7k0ZZ1KrM9Oatvh3h/1o?= =?us-ascii?Q?skRNRPucrA2z79fHjZEjjiglOdiZ1kS+qzVxtApcR5BymFASBMBu32MNxQGX?= =?us-ascii?Q?OphZA4u7viDnzSdxWZZTGJsSW/hG6KtJnG3aE5hy9UaGcPizAjRnU3bTaWo7?= =?us-ascii?Q?hZgmUqJHRLKn3IrP8SaiquQJLmNJdRBsu9G3GmjnrkEJVb9VXIHxxqcCuTgR?= =?us-ascii?Q?JZeK6dGIq3QuD2SuYD4npUR84qGdROKAAOxekoix/ql2ib1/gPbgVs+Prjh/?= =?us-ascii?Q?J9/6l/YtZcXs0veXmKNhhQPvaB3bUa5RGIuKvFyW+WPNlrciYCij/DOS7LWx?= =?us-ascii?Q?9QhlaWjvQYyoyFKPNIkyYUy335vDNU+XXB7Ok7BoCLMqqlpd15wpXnhh9iVc?= =?us-ascii?Q?SdAehp4oP5I3uJ87xhWzMqZ3vz/2zVaubUyncESgqBIXRG+bJElPL+BP8P7C?= =?us-ascii?Q?P0Z0wbOsLM2BtuH6QI1LsFHO3dqnxeZTYcORrBjzbVGh0JrXmacFxxj0ROpH?= =?us-ascii?Q?xsRSgZMgBAEgKPfbsPyGwe6ybHA5fJgunGTRWqXAm5QxSqev3b7X?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 986dc570-0112-4e39-6cb0-08de8ffe90af X-MS-Exchange-CrossTenant-AuthSource: CY8PR12MB7171.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Apr 2026 14:54:24.7355 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1sYi7OCoEJrLolHxOCVIGd9GuyBNl/N0DYMJ2YVwbpFLA6WgWgl8yQSlmeYxZLQKVkZL5Dvaop+e4nCxH+nR2Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB9469 On Wed, Apr 01, 2026 at 12:09:52PM +0200, Valentin Schneider wrote: > On 28/03/26 17:42, Yury Norov wrote: > > Please no kernel doc syntax out of kernel doc blocks. > > > > Why not? It's straight to the point and not uncommon in sched code. Because it's weird to see machine-readable marks in a text for human. In that particular file, I see it in: - NUMA topology huge comment block for 'level' of hops, which is not even a variable, so misleading; - in annotation to sched_numa_find_closest() - should be converted to kernel doc and fixed; and - in an inline comment in build_sched_domain: /* Fixup, ensure @sd has at least @child CPUs. */ So, a single more or less matching case in a 3k LOCs file. I'll send a patch. > >> hop = hop_masks - k.masks; > >> > >> + /* > >> + * @hop is constructed by hop_cmp() such that sched_domains_numa_masks[hop][node] > > > > hop_cmp() doesn't construct, it's a comparator for bsearch, which searches for > > the nearest hop containing at least N CPUs. > > > >> + * spans enough CPUs to return an @nth_cpu, and sched_domains_numa_masks[hop-1][node] > >> + * doesn't. > >> + * > >> + * Get a cpumask without the CPUs from sched_domains_numa_masks[hop-1][node] > >> + * subtract how many CPUs that contains (@k.w), and fetch our @nth_cpu from > >> + * the resulting mask. > > > > It explains only hop != NULL case. This sentence confuses more than > > explain, to me. The below code is quite self-explaining. > > > > My goal was to allow the reader to have a gist of how @hop_masks and @k are > arranged upon returning from bsearch() without having to dig down to how > bsearch()+hop_cmp() will interact. > > Right now nothing tells you what @k.w is unless you go dig for it. hop_masks is the bsearch output, so pretty straightforward to me. If you think it's vague, maybe just pick a better name? The k.w should be explained in struct __cmp_key description. And just in case, it's an output field containing the number of CPUs in a given hop, matching the user-provided 'cpus' mask. > >> + */ > >> ret = hop ? > >> cpumask_nth_and_andnot(cpu - k.w, cpus, k.masks[hop][node], k.masks[hop-1][node]) : > >> cpumask_nth_and(cpu, cpus, k.masks[0][node]);