From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8A8D63191D6 for ; Thu, 23 Jul 2026 06:11:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784787093; cv=none; b=lHyA1vssUdfW/s4zDQ7XvEVTZG0aqOyNiVi8LnX/ucLnLLVBeo5qEvOeVxYdzdDtIHP0eqahtKnYmyPsBtdf+ewtesBQWhBHtnP1J4GQqJamSIvyii/UasrI9+6AqcOUfi8oaQlb7rWIVWA2rHdmqvlwLvmyl7rOYV/aWTjdQJ4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784787093; c=relaxed/simple; bh=P1jIUv/SDmBgWngwvm5d+F5H2CTzZ0BI6vXhsXAO8sg=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=AHxQkQlHuhLlsu6jvh1cqsLQr6g6hhd7Ci+KK6lKryW2H3yn/oVkCUOwp29i/lKkr2hDzOFpirOKfnFJ8hA14cTaE4saJxjOIjEe0xrsqJ2CRoo7EKiXHmXTWyezUTvX5DUJVfDraor1RKysIoFD6lsHONfoStZOyBCBYKdfj9w= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=d+T2jFMo; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="d+T2jFMo" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 506AD1595; Wed, 22 Jul 2026 23:11:26 -0700 (PDT) Received: from [10.1.39.91] (e127648.arm.com [10.1.39.91]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8522C3F86F; Wed, 22 Jul 2026 23:11:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1784787090; bh=P1jIUv/SDmBgWngwvm5d+F5H2CTzZ0BI6vXhsXAO8sg=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=d+T2jFModkY60t0SML2zLn71bV/LQ18bSjFhfqFdhu4Rrt7OtdpNmQmS+9hyrCtxb uCD4oPL4fP/8R5qXg+nL+5SxX0XawcfU+zuV3JDTqhgWfy/ZxMN+/S+bF7MGmP5UNY CbObz3h1itMaGMvJ6KuRt/Jafl37ub/bCtFJPKx8= Message-ID: <8641b701-c9a2-4b8e-a248-5f7ff7ac8f0c@arm.com> Date: Thu, 23 Jul 2026 07:11:25 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2] sched/fair: Prefer waker CPU for reciprocal sync wakeups To: "Shubhang Kaushik (Ampere)" , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak Cc: "Christoph Lameter (Ampere)" , Shubhang Kaushik , linux-kernel@vger.kernel.org References: <20260722-b4-sched-sync-wakeup-v2-1-f1164560b24b@gentwo.org> Content-Language: en-US From: Christian Loehle In-Reply-To: <20260722-b4-sched-sync-wakeup-v2-1-f1164560b24b@gentwo.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 7/22/26 23:50, Shubhang Kaushik (Ampere) wrote: > Pipe-style ping-pong workloads can be dominated by handoff cost. In > such cases, placing the wakee on an idle CPU can be slower than keeping > the pair on the same runqueue. > > Use the existing last_wakee and wake_wide() state to identify narrow > reciprocal WF_SYNC wakeups: > > A wakes B > B wakes A > A wakes B > ... > > When the wake-affine domain allows SD_WAKE_AFFINE, prefer the waker CPU > for these narrow reciprocal handoffs. Do so only when the waker CPU has no > other runnable fair task, the wakee is allowed on that CPU, and the wakee > fits there on asymmetric capacity systems. > > Wakeups that do not match this pattern continue through the existing > wake_affine() and select_idle_sibling() path. > > Signed-off-by: Shubhang Kaushik (Ampere) > --- > Tested on 80-core Ampere Altra: perf bench sched pipe -l 1000000 improved > by about 30%, averaged over 20 runs. Hackbench, schbench and SPECjBB > showed no material regression. > > Baseline ~ v7.2-rc4 (mainline origin/master at 248951ddc14d) > --- > Changes in v2: > - Move the reciprocal handoff preference under the existing > SD_WAKE_AFFINE domain check. > - Drop futex from the changelog motivation. > - Refresh perf bench sched pipe results after rebasing. > > Link to v1: https://lore.kernel.org/r/20260721-b4-sched-sync-wakeup-v1-1-dc94f184e27f@gentwo.org > --- > kernel/sched/fair.c | 28 ++++++++++++++++++++++++++++ > 1 file changed, 28 insertions(+) > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index d78467ec6ee1343050fcc2794dafb38ade3599e5..d188e91b85d74dd3ed3c8b99f171bd142e72da5e 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -8794,6 +8794,29 @@ static inline bool asym_fits_cpu(unsigned long util, > return true; > } > > +/* > + * For reciprocal WF_SYNC handoffs, prefer the waker CPU when it has no > + * other runnable fair task. > + */ > +static bool prefer_sync_pair_cpu(struct task_struct *p, int cpu) > +{ > + struct rq *rq = cpu_rq(cpu); > + > + if ((rq->nr_running - cfs_h_nr_delayed(rq)) != 1) > + return false; > + > + if (!cpumask_test_cpu(cpu, p->cpus_ptr)) > + return false; > + > + if (sched_asym_cpucap_active()) { > + sync_entity_load_avg(&p->se); > + if (!task_fits_cpu(p, cpu)) > + return false; > + } > + > + return true; > +} Maybe @Prateek: What about SMT, would we want to require the sibling to be idle too? I would appreciate a bench sched pipe on SMT result, if Shubhang doesn't have one I can dust one off, too. > + > /* > * Try and locate an idle core/thread in the LLC cache domain. > */ > @@ -9579,6 +9602,11 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags) > */ > if (want_affine && (tmp->flags & SD_WAKE_AFFINE) && > cpumask_test_cpu(prev_cpu, sched_domain_span(tmp))) { > + if (sync && > + READ_ONCE(p->last_wakee) == current && > + prefer_sync_pair_cpu(p, cpu)) > + return cpu; > + > if (cpu != prev_cpu) > new_cpu = wake_affine(tmp, p, cpu, prev_cpu, sync); > > > --- > base-commit: 248951ddc14de84de3910f9b13f51491a8cd91df > change-id: 20260721-b4-sched-sync-wakeup-04d40cbeb1da > > Best regards,