From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 53F6F2C08C8 for ; Wed, 22 Jul 2026 12:37:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784723838; cv=none; b=g1ec1EazHpupFXn7YEvMh89Fa0Uo5vl0atNwzqTxd6AI91K1QbjyF9QUKhSieW9I2GuIagDUiPbFGc+puB3I8GWR7dCbUQYvUCUoRfWnsPS+N92/GqetEgXSpCN6H7ucXsmwUOJRB9vGTpzU5QTIuLL777O17aTYPQO92zz584k= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784723838; c=relaxed/simple; bh=lV7dPblTcXtOXgWNLJbxCoz0q7zcmwRC3W0WSe6jipw=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=mm9SQGjEcOrI+wH37dFRz2PEMetM2fNs2wjnia0nU5Mf3kAIST9i+3bysyrrDRQMz6v45Pt4FxVvEELo7lliwSVmFbe7gHst6OrNqDgzrbGxiJuMNAm3I7E6WyaqAh9ie4vMBcNE+RzjbgnmzrfEJ+ckiJE7p0h2CTQXXpkuRL4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=hlf/rZVg; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="hlf/rZVg" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 158401595; Wed, 22 Jul 2026 05:37:11 -0700 (PDT) Received: from [10.1.39.91] (e127648.arm.com [10.1.39.91]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 457FF3F66F; Wed, 22 Jul 2026 05:37:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1784723835; bh=lV7dPblTcXtOXgWNLJbxCoz0q7zcmwRC3W0WSe6jipw=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=hlf/rZVgWTVsJ2tX9krS2UPlDYVXBpIFcuZ50hgZdk32T3VkiiVhQRQ12tU6a0n5j k31l0P3VIguCo1xTpzk7X0lMAaHVZ6mj3Gm8ED6vrsAdwgF8WzbKxbsCjweOEOTPWO SE0i+KdXwQy5HN8RkbWe3hd/Q2zf8Cidq09igjEw= Message-ID: <2aadc9bb-43d8-403d-a5ec-321c0f428026@arm.com> Date: Wed, 22 Jul 2026 13:37:10 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] sched/fair: Prefer waker CPU for reciprocal sync wakeups To: "Shubhang Kaushik (Ampere)" , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak Cc: "Christoph Lameter (Ampere)" , Shubhang Kaushik , linux-kernel@vger.kernel.org References: <20260721-b4-sched-sync-wakeup-v1-1-dc94f184e27f@gentwo.org> Content-Language: en-US From: Christian Loehle In-Reply-To: <20260721-b4-sched-sync-wakeup-v1-1-dc94f184e27f@gentwo.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 7/22/26 01:04, Shubhang Kaushik (Ampere) wrote: > Pipe and futex ping-pong workloads can be dominated by handoff cost. In > such cases, placing the wakee on an idle CPU can be slower than keeping > the pair on the same runqueue. > > Use the existing last_wakee and wake_wide() state to identify narrow > reciprocal WF_SYNC wakeups: > > A wakes B > B wakes A > A wakes B > ... But the futex ping-pong doesn't set WF_SYNC in the first place or am I missing something? > > For these wakeups, keep the wakee on the waker CPU only when this looks > like a narrow ping-pong: the wakee recently woke the current task, > wake_wide() does not classify the relationship as wide, and the waker CPU > has no other runnable CFS task. The wakee must also be allowed to run on > that CPU, and on asymmetric-capacity systems it must fit there. > > Wakeups that do not match this pattern continue through the existing > wake_affine() and select_idle_sibling() path. > > Signed-off-by: Shubhang Kaushik (Ampere) > --- > Tested on 80-core Ampere Altra: perf bench sched pipe -l 1000000 improved > about 21%, averaged over 20 runs. Hackbench, schbench and SPECjBB showed > no material regression. > > Baseline ~ v7.2-rc4 (mainline origin/master at b95f03f04d47) > --- > kernel/sched/fair.c | 31 ++++++++++++++++++++++++++++++- > 1 file changed, 30 insertions(+), 1 deletion(-) > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index d78467ec6ee1343050fcc2794dafb38ade3599e5..d9afc89bc7a45b0b05ebad536f9a31d5757a2753 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -8794,6 +8794,28 @@ static inline bool asym_fits_cpu(unsigned long util, > return true; > } > > +/* For reciprocal WF_SYNC handoffs, prefer the waker CPU when it has no > + * other runnable fair task. > + */ > +static bool prefer_sync_pair_cpu(struct task_struct *p, int cpu) > +{ > + struct rq *rq = cpu_rq(cpu); > + > + if ((rq->nr_running - cfs_h_nr_delayed(rq)) != 1) > + return false; > + > + if (!cpumask_test_cpu(cpu, p->cpus_ptr)) > + return false; > + > + if (sched_asym_cpucap_active()) { > + sync_entity_load_avg(&p->se); > + if (!task_fits_cpu(p, cpu)) > + return false; > + } > + > + return true; > +} > + > /* > * Try and locate an idle core/thread in the LLC cache domain. > */ > @@ -9548,6 +9570,7 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags) > int cpu = smp_processor_id(); > int new_cpu = prev_cpu; > int want_affine = 0; > + int wide = 0; > /* SD_flags and WF_flags share the first nibble */ > int sd_flag = wake_flags & 0xF; > > @@ -9557,6 +9580,7 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags) > lockdep_assert_held(&p->pi_lock); > if (wake_flags & WF_TTWU) { > record_wakee(p); > + wide = wake_wide(p); > > if ((wake_flags & WF_CURRENT_CPU) && > cpumask_test_cpu(cpu, p->cpus_ptr)) > @@ -9569,7 +9593,12 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags) > new_cpu = prev_cpu; > } > > - want_affine = !wake_wide(p) && cpumask_test_cpu(cpu, p->cpus_ptr); > + if (sync && !wide && > + READ_ONCE(p->last_wakee) == current && > + prefer_sync_pair_cpu(p, cpu)) > + return cpu; This must move further below (to check for SD_WAKE_AFFINE) otherwise it ignores isolcpus. > + > + want_affine = !wide && cpumask_test_cpu(cpu, p->cpus_ptr); > } > > for_each_domain(cpu, tmp) { > > --- > base-commit: b95f03f04d475aa6719d15a636ddf32222d55657 > change-id: 20260721-b4-sched-sync-wakeup-04d40cbeb1da > > Best regards,