From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CH1PR05CU001.outbound.protection.outlook.com (mail-northcentralusazon11010029.outbound.protection.outlook.com [52.101.193.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 658D93BCD17 for ; Tue, 4 Aug 2026 08:43:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.193.29 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785832986; cv=fail; b=lAlPCTMmikARoTXfzio5XFBf+ERtWRWeJnq5tptJGrBiy+90IFUOXTArdIIApOXfq33HSHSc7LTY1KurpiJdCvUqAsGdwpLTZUo4EGNs+rrnbi9apNNKQcQjHVjsY/3PcGjCKWgd+zMY2UdgmWerJexfCYv+9iSh8+wVGiNsPkk= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785832986; c=relaxed/simple; bh=PYxeoWs9UC14RzKZuONkl4hRLwyoyyefSYx3WH0SNBU=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=g8tw8cmfTZCfPa4ZmADi8nGZbGhM5A2zvFaPsiPYa/stQu2HQh4NpA3ePQTSPdUSouNod6z3b21pw5m/DJ+Ec9Z8tmYUqbdlex/UQlr2BnsFTIZnMCg2CxnMuzKD6g7JIXkjPeInkIB8OF0zH8eJLXfGhG1MyYax4MvMvnZhktg= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=qxpEb8qx; arc=fail smtp.client-ip=52.101.193.29 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="qxpEb8qx" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=i4m4SAhZNp6VKbvfiQUwKHoPmVoifOXooMgrcUzUDpT40qLRBztgNxOPIsp96uhv7KsqVN9Uf6oqojRe6U32KGfqPG1IUJULQj1dTABex/7vg4m7/H8fvmHZ4l/AH+ki271W4vY0FJfzt/bA0VlTjk8KUy51M9TF5J0TIqLwzVI7r7S81l4JPQo1Q0baGOjkyUskfB6cW2VsZgQYrVGW9giGvwlKbAxjtQZPfKv//qXyCnwuKNgZbVgmTEyuiVhejeNyPGlYWFDN6r2lnbc6AjxOP9vS56mBSMUMyOYCcE4BmTFqi/yb6Px55PzbBHn/OgbaxWovAsA5CODxqHIXVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=s633TB3/Pw0GvXx13524dw8KtscXfWF7NkJDDu4tjEw=; b=JPXHn1YcOTxQogGcYxDr6T9KycsXXx2UVf9BsueU7e+fu0kvgmLp/E+JOtCxzqn+1U0yvgSlE67IQRE/gYyysMIEeQCbnFdhE2JUBhvztnw+oS1J0SYqryq6aY1BtSDTckq+yGXefC+hk7mgz4SAsrGjwL2vGlsnBWY+0E6yPsb5GEx2yP9FUy4GoqwGC1XJch8JzMglLwcTfzGDvPnqmL9+YagGvTBJgyMioC/MC7AqbHNsdEErzWXwtBceiiOgDxDedz++RYuWt+gkCorKebEUjp9s4Eu05FxoBz+kl5cgwDgH/jt59hKzlQFaV6fPVx3Qx3/02P7rOmnvXtbluw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=s633TB3/Pw0GvXx13524dw8KtscXfWF7NkJDDu4tjEw=; b=qxpEb8qxNyNoe4kM9zfT0eXhs06pssrKv1TeEHrZR5wmLrNbXDsHG2BKNZcuS8Ub3yE7xFkBnqOx9PbL+bLQDG7lRz7UZAUidEQqmJqstMpkgM8cXrLIAqMDsRgD/v8fF/iaMpCsI+HTfTPdUdOl6B/mIw75w0vusdOvAu6rzqI= Received: from MN0P221CA0010.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:52a::23) by BY5PR12MB4212.namprd12.prod.outlook.com (2603:10b6:a03:202::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.270.18; Tue, 4 Aug 2026 08:42:57 +0000 Received: from BL6PEPF00020E60.namprd04.prod.outlook.com (2603:10b6:208:52a:cafe::15) by MN0P221CA0010.outlook.office365.com (2603:10b6:208:52a::23) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.292.16 via Frontend Transport; Tue, 4 Aug 2026 08:42:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by BL6PEPF00020E60.mail.protection.outlook.com (10.167.249.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.292.8 via Frontend Transport; Tue, 4 Aug 2026 08:42:56 +0000 Received: from satlexmb08.amd.com (10.181.42.217) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Tue, 4 Aug 2026 03:42:55 -0500 Received: from [10.136.47.95] (10.180.168.240) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server id 15.2.2562.41 via Frontend Transport; Tue, 4 Aug 2026 03:42:51 -0500 Message-ID: <97bc48b8-f00d-4c75-97ac-6feadf7d3372@amd.com> Date: Tue, 4 Aug 2026 14:12:51 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4] sched/fair: Preserve wake-affine CPU for non-SMT reciprocal sync wakeups To: Shrikanth Hegde , "Shubhang Kaushik (Ampere)" , Peter Zijlstra , Vincent Guittot , Ingo Molnar , Mel Gorman CC: "Christoph Lameter (Ampere)" , Shubhang Kaushik , , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Valentin Schneider , Christian Loehle , Madadi Vineeth Reddy References: <20260803-b4-sched-sync-wakeup-v4-1-52333b0cfb79@gentwo.org> <98bbe2d7-2401-4713-b49e-12f5dfc36471@linux.ibm.com> Content-Language: en-US From: K Prateek Nayak In-Reply-To: <98bbe2d7-2401-4713-b49e-12f5dfc36471@linux.ibm.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E60:EE_|BY5PR12MB4212:EE_ X-MS-Office365-Filtering-Correlation-Id: d5097726-9b37-46a0-c760-08def204618a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|23010399003|376014|1800799024|7416014|82310400026|13003099007|56012099006|4143699003|11063799006|3023799007|10067099003|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: YIjWN7iAnFgDHtnOc/SJ2ptzOgWrcdzVd95Odiz9skCK+p3tSGI7MJyIUVpAgic8QzgCUGkR//WXdtfMt4rGIMNhEmYJCgXrevmROgN47Xnus7YflCZhZPMjLZHh2+RZvANe8dk1Iecu4H0Ttfbxig0i3fQ5FBa4giBRBqZ3B13XgVTLHRoxWzCbuvO7dB2J/Tpi7jMY/0DeLqq9KeFC5N9nzWorLX5+uYeBmY84U4bWouX5/OOpHCj2iS6yw12BYfk+pjZu503W2WUKhabekdVXY4qpGY8yipR7n3Wd2mv+vHVxcgGhysWCHhUU1fvVZiXxH9aXcT92tXAr3DdGPbhHCKfNpU8lNseogyAqMUMdWRjxMOza1qExE4izo6DqBQZZnBa8Hh2M/7jDBuiUbnqULMUe62H8zxILM+XdfAFe5HEB0Xai+KeviQgpTME/IJV2L3vlUIRPy8zZx7aKANl+pX9xuf4N3iO+Tay2NOYuXwvFsjHGAdS5pmFgUvpFWQLnxfgSomA1UQgi+UmIDEBrTwuMBiyo3UqmRCK9K1rvAmoQyP2RWAPf0gFm/owovN6Z37JyIqzQO5N9XTa9EH6zk8PGIOhcBxU8xEriQlQ4yYfGlmP2OxYoXsYpdqzdjIkChv2fX7I18HI6A7bcF4eFGWiYNnGTUMWeRYJG7jLA+jz205D8crwNZLTPBzeO6+f2oPpsW83BakbqxsG1FQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700016)(23010399003)(376014)(1800799024)(7416014)(82310400026)(13003099007)(56012099006)(4143699003)(11063799006)(3023799007)(10067099003)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: fh9+MR6tY4jwiTxQsE/UWbLODmdHzGBvPCzcvnYFjbFpOSAd07ZJlfb8uOciTP9BRXohvwy8QD4a1hlCt+fADhbPAK6u97KzSkJClc/ggnEw+oQCXNLJP7b/nf6s6mKdxriJY/l2uBcteeijvaAbyB7WO1BwlUuM5zHwAGl0+1ZJTWxbemgesDCd8m/lx1Ea+XEBMqiJ8wT0amt7rmvzcMjUiXKLeLYx4Av0+5UtQoN+rHTt+DLmBz1/0x1ZzDEFpdD6IoUCTZs+r7Dl7iOqpe7Y+vBWWLMcQLGYnu+pgjJHaimMWsUwQAXamNZlBOfPe1RU81sWu6FyTxWhNFV98Gq40l8HprZ/mhEhEb4F7vVvMTO41nlgZ29xC3ST0H6Qa9XusLDAnSxnLvO4+1/Rg/CbA18pmjSflitEL7tx5afMUTVNi/FGuUe+QmJ0wHaj X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Aug 2026 08:42:56.3029 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d5097726-9b37-46a0-c760-08def204618a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E60.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4212 Hello Shrikanth, On 8/4/2026 10:10 AM, Shrikanth Hegde wrote: > As I said in v3, before we add bells/whistles to sync path, i want > to know what is expected of sync behavior today. > And that should be documented in Documentation/scheduler/ > > Be it, > - current way of hint only and scheduler can still choose an idle core/idle cpu etc. > - Should it be enforcing it to waker cpu if waker cpu has only one task. > - Whatever the policy maybe. > > Current api usage is tricky to use and effect is visible in real life workloads. > The case I mentioned in v3 of networking code using sync api leads to strange > results due to sync mechanism. > - It depends whether waker/wakee are running on same node. > - Result of wake_wide. > In other end, user sees inconsistent latency/throughput. > > We can keep on adding minor changes to sync api path, > but one benchmark will benefit and one will suffer. > Having the behavior documented is a good start. > > Peter, Ingo, Vincent, Mel, Prateek, > What do you guys think? Currently it is very arbitrary and WF_SYNC may, or may not, indicate a true voluntary blocking behavior. For example, anon_pipe_read() uses a wake_up_interruptible_sync_poll() to wake up writers once reader has drained the pipe but if you think about it, why would the reader block soon after just having the data it needed? Here are the results on my Zen4 system from running perf bench sched messaging (threads + pipes) at varying worker counts with all wake_up_interruptible_sync_poll converted to wake_up_interruptible_poll: Test: tip no_sync 1-groups: 3.79 (0.00 pct) 3.35 (11.60 pct) 2-groups: 3.85 (0.00 pct) 3.41 (11.42 pct) 4-groups: 4.02 (0.00 pct) 3.32 (17.41 pct) 8-groups: 4.33 (0.00 pct) 4.38 (-1.15 pct) 16-groups: 6.09 (0.00 pct) 6.12 (-0.49 pct) --- So seems like WF_SYNC hint on this machine with perf bench sched messaging (thread + pipes) pattern is actually holding it back. Lemme check processes ... Test: tip no_sync 1-groups: 3.48 (0.00 pct) 3.08 (11.49 pct) 2-groups: 3.80 (0.00 pct) 3.07 (19.21 pct) 4-groups: 3.91 (0.00 pct) 3.09 (20.97 pct) 8-groups: 4.13 (0.00 pct) 4.10 (0.72 pct) 16-groups: 5.81 (0.00 pct) 5.74 (1.20 pct) Similar stuff. At some point it was pretty bad for Zen3 but situation might have changed since ¯\_(ツ)_/¯ I'll let you know once I have a machine. But ... If I have true 1:1 waiting on pipe as in the case of "perf bench sched pipe -l 1000000" I go from ~2.5usecs/op on average to ~4.2usecs/op which is close to a 50% increase in the benchmark time so that WF_SYNC hint can also help if all we have is looping over a read waiting for one page worth of write. The way I look at WF_SYNC nowadays is that it indicates a local LLC wakeup is beneficial. wake_wide() doesn't even consider WF_SYNc and simply uses wake-wakee flips and then only at want_affine() do we actually check the sync hint. Most benefit come from wake_affine_idle() for the 1 task case where target is set to current and select_idle_sibling() uses that as the target from there on. It could purely be a coincidence that it benefits at all - most of these microbenchmark we have always hit the same two syscall (mostly read() and write() in turns) and a most of benefit for those comes from kernel instructions being primed in cache. I remember a while back, removing the effect of WF_SYNC on the networking side hampered a lot of performance - especially for localhost communications. This one specifically https://lore.kernel.org/lkml/20220711224704.1672831-1-libo.chen@oracle.com/ Let me see if things have miraculously changed there too but for TCP sockets in real world, with blocking for ACKs, I think WF_SYNC still makes sense there but I feel most of the benefits from sync are a second order effect. -- Thanks and Regards, Prateek