From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from gentwo.org (gentwo.org [62.72.0.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 38507399019 for ; Fri, 31 Jul 2026 07:32:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=62.72.0.81 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785483143; cv=none; b=dLSyKwF0sOxi1o/4aBmt/yNCy0GDSSDeDTs/5mWwl3M/DroxHIJxEKppIgng2tju7QkoeZRjTW4ib/N+fskJmwAv23T+XzYgKSLIGcVbiiA+Yo9Wt8kTij23vH68QAUq3I/IAMEzbG2wQZku9udYyMy6ooX0rC/bqx5KFVq6MJs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785483143; c=relaxed/simple; bh=7KdacRQ/nFEvnozZcPUODNznH5HgAbC3MhomtCOFrJU=; h=Date:From:To:cc:Subject:In-Reply-To:Message-ID:References: MIME-Version:Content-Type; b=sYrFX7KFKWpB/o/rhm3H32GQd1cQKHJYH8JN2ZkdUlzhx2TrBhE/2D952WHdwQik4G63gDzRI/Xw5vuzF/ivy+MBlOPTwc/VUGSYpdkiINi+SOqDL19fI4XYfWxEAOBpvQD5ucbxBux/FB3pH93IFia/9AHxphyWJvM6BqQaA7s= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=gentwo.org; spf=pass smtp.mailfrom=gentwo.org; dkim=pass (1024-bit key) header.d=gentwo.org header.i=@gentwo.org header.b=sf5mJdIa; arc=none smtp.client-ip=62.72.0.81 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=gentwo.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gentwo.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=gentwo.org header.i=@gentwo.org header.b="sf5mJdIa" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gentwo.org; s=default; t=1785482538; bh=7KdacRQ/nFEvnozZcPUODNznH5HgAbC3MhomtCOFrJU=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=sf5mJdIayNsCjMVaYyTyfO6P4p/ABf3JA/fh6Q7cEH6O1y5rS6aEQRZULh6dwJmVl so09ulgqpXIbP74A27Xv1fR0bbEvCmDmPV1ndMX6i4IwtI1tHhh6N1CIvQNqIFB5OY cVSpCKPXElCWl9C9LCVyA1LoFHJxNZPOg32/QXo0= Received: by gentwo.org (Postfix, from userid 1007) id A2F44406D6; Fri, 31 Jul 2026 00:22:18 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by gentwo.org (Postfix) with ESMTP id 9FD14401C8; Fri, 31 Jul 2026 00:22:18 -0700 (PDT) Date: Fri, 31 Jul 2026 00:22:18 -0700 (PDT) From: Shubhang To: K Prateek Nayak cc: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Madadi Vineeth Reddy , "Christoph Lameter (Ampere)" , Shubhang Kaushik , linux-kernel@vger.kernel.org Subject: Re: [PATCH v3] sched/fair: Prefer waker CPU for non-SMT reciprocal sync wakeups In-Reply-To: Message-ID: <19f0f0ed-cc8f-1be3-ef8f-9e874c9b4407@gentwo.org> References: <20260727-b4-sched-sync-wakeup-v3-1-90cf481dbd85@gentwo.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Hello Prateek, On Thu, 30 Jul 2026, K Prateek Nayak wrote: > Hello Shubhang, > > Building on top of Chris' suggestion on v2 for systems with SMT, we can > push that check further down into select_idle_sibling() and can take a > call at the point where we know what test_idle_core() returns. > > This is what I tried out on top of tip:sched/core: > > (Lightly tested on a SMT-2 system) I retested both approaches on top of tip:sched/core on the 80-core non-SMT Ampere Altra. Baseline tip:sched/core, perf bench sched pipe -l 1000000, 20 runs: default: 4.063 usec/op mean, 4.100 median same-cpu-79: 3.192 usec/op mean, 3.192 median two-cpu-78-79: 4.175 usec/op mean, 4.262 median With your SIS-based change: default: 3.392 mean, 3.332 median same-cpu-79: 2.755 mean, 2.754 median two-cpu-78-79: 3.311 mean, 3.340 median With the narrow non-SMT direct wake-affine return: default: 2.886 mean, 2.859 median same-cpu-79: 2.737 mean, 2.736 median two-cpu-78-79: 2.799 mean, 2.800 median > + return select_idle_sibling(p, prev_cpu, new_cpu, sync); > > return new_cpu; > } > --- > > I'm currently seeing a ~10% improvement for the workload you mentioned > (perf bench sched pipe -l 1000000) on average. I haven't tried anything > else yet but would love to know your thoughts. > > I'm using rq->rd->span to know the CPUs covered by the cpuset instead of > sched_domain_span(sd_llc) in select_idle_smt() to make it work for > sched_asym_cpucap_active() + sched_smt_active() where some cores may > have more than one CPUs and the LLC is defined at core boundary. > > Basically I wanted to avoid this ugly: > > sd = rcu_dereference_all(per_cpu((sched_asym_cpucap_active()) ? sd_asym : sd_llc, target)); > > if (!sd) > goto skip; > > pattern and rq->rd->span seemed just fine since it doesn't need a null > check and gives the desired boundary. > > Could you please check if the improvements still persist on your system > with the check pushed down into select_idle_sibling(). Thank you. SIS direction does improve the baseline here: default perf bench sched pipe improves by about 16.5% mean / 18.7% median over 20 runs. But on non-SMT Altra system the direct wake-affine return is still about 15% faster than the SIS version for the default and two-CPU cases. I agree that for SMT systems this decision is better handled closer to SIS, since SIS can prefer an idle sibling on the waker core instead of stacking both tasks on one hardware thread. On non-SMT systems however, there is no sibling placement decision to make. Once the wakeup matches the narrow reciprocal WF_SYNC pattern, preserving the wake-affine waker CPU directly thus performs better than pushing the decision into SIS in my testing. Regards, Shubhang Kaushik