From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E4B1A3E7BC3; Tue, 30 Jun 2026 09:03:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782810226; cv=none; b=KNv3HuXksRCsU3gHy1ca66Yf0dGqbQA5C493ePYWtIXwxcweBWuCk631CF0g5Fy+pjoWEkezTH7FipNw/VVmuIASSUk7pBAgwJ6BGKc1RnZX56Eny+ju3vYQIrTIxRUrw7Ztr5YnTIk/57bRvh4Tk+oXVKFPv9/1ZmjizABfPIM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782810226; c=relaxed/simple; bh=uAhP2gsYXHgM4KoJPUx/AUNaIwhB4m6iywoN4ynO8Ns=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=ICl09b6+gYWmhWSsWGDYPJNE1g7163qbx/UC58L4ebyPvG2EYmmVR62u7k0erp9T0WQDKbSGSjKV8TUorURLeNGRmz+v8QMp46sG14ql13RhOlEqF5TPEWSkETeqQzwbLWSkXA4TpsC8XycSJbU1fNiGgc+bihljgZCEhX/YaOk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=0VQrwRv2; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=XrKaBBW3; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="0VQrwRv2"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="XrKaBBW3" Date: Tue, 30 Jun 2026 09:03:37 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1782810219; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hINqmI+L8d7OaWuEk5tqKNNL2NKnDi4TcFyGw8XCpxQ=; b=0VQrwRv2BZIgJSZK8I7uE57p9WJXtGyzy44nEnK9EF3l/lnRrOGJe52KW6bDBzNiqC7Ezh vkNn7rm6BkwIN6cqXXBdl+yoQGP+yEyQ+mqjZZoRyuu3zDQGr1rkO8snIqFkFas2Lxx0OJ GplantoGhTYiYyahXKdCCNaoplWOiDRfCY3NiXZDft3FNdXAUIjkIRzaDVy1/3LhLS+DJb 085RT16/MTvy/ib7uHPTO7lgAP3TPGJopZNybiBO896Zf3OazMuTbGXGJF3ccsdJYPfPi1 2aIgozAZy3awy6mDmrvsYB3YBr5eegrX8U5utLZ0IwIMEHIb5vXHvc4utlY+Nw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1782810219; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hINqmI+L8d7OaWuEk5tqKNNL2NKnDi4TcFyGw8XCpxQ=; b=XrKaBBW3l9W2PoTMUxw8Gy8KgkKfyG8IAur56h9Vmc+y/GcKHw2uIauKOKBLtyFEPkEPK4 FxPf5kzb7udtNtDw== From: "tip-bot2 for Xin Zhao" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: sched/core] sched/fair: Don't trigger active lb if src_rq->curr is not on_rq Cc: Xin Zhao , "Peter Zijlstra (Intel)" , Valentin Schneider , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260617072151.1173416-2-jackzxcui1989@163.com> References: <20260617072151.1173416-2-jackzxcui1989@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <178281021790.3843924.11008528899173889980.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the sched/core branch of tip: Commit-ID: cdd9e37ed46ac1a80c1c9d4ec430096a0f1c419c Gitweb: https://git.kernel.org/tip/cdd9e37ed46ac1a80c1c9d4ec430096a0f1= c419c Author: Xin Zhao AuthorDate: Wed, 17 Jun 2026 15:21:50 +08:00 Committer: Peter Zijlstra CommitterDate: Tue, 30 Jun 2026 10:56:56 +02:00 sched/fair: Don't trigger active lb if src_rq->curr is not on_rq Active load balancing relies on migration threads, which temporarily preempt tasks on the source runqueue (src_rq). This preemption can negatively impact overall system performance. The active balancing logic includes a check to verify whether the current task (curr) on src_rq can actually run on the destination runqueue (dst_rq). We have observed that when curr is a CFS task and its on_rq flag is 0, the active balancing failure rate is exceptionally high. The following table summarizes test data collected over 300 seconds on = an 18-CPU platform under a specific fillback task scenario: fair: busiest->curr->sched_class =3D=3D &fair_sched_class on_rq: busiest->curr->on_rq total: active balance count triggered of correspondent type fail: fail to migrate one task in active_load_balance_cpu_stop() fair && !on_rq !fair && !on_rq domain total fail total fail cpu0 0x00003 0 0 0 0 cpu0 0x3ffff 33 33 1 1 cpu1 0x00003 0 0 0 0 cpu1 0x3ffff 42 42 0 0 cpu2 0x0003c 4 4 0 0 cpu2 0x3ffff 12 12 0 0 cpu3 0x0003c 3 3 0 0 cpu3 0x3ffff 8 7 0 0 cpu4 0x0003c 2 2 0 0 cpu4 0x3ffff 5 4 0 0 cpu5 0x0003c 4 4 0 0 cpu5 0x3ffff 8 8 0 0 cpu6 0x003c0 60 60 0 0 cpu6 0x3ffff 28 27 0 0 cpu7 0x003c0 194 184 0 0 cpu7 0x3ffff 35 35 1 1 cpu8 0x003c0 240 228 0 0 cpu8 0x3ffff 28 28 0 0 cpu9 0x003c0 0 0 0 0 cpu9 0x3ffff 10 10 0 0 cpu10 0x03c00 52 50 0 0 cpu10 0x3ffff 0 0 0 0 cpu11 0x03c00 70 68 0 0 cpu11 0x3ffff 1 1 0 0 cpu12 0x03c00 73 72 0 0 cpu12 0x3ffff 0 0 0 0 cpu13 0x03c00 79 76 0 0 cpu13 0x3ffff 0 0 0 0 cpu14 0x3c000 0 0 0 0 cpu14 0x3ffff 57 55 1 0 cpu15 0x3c000 53 52 1 0 cpu15 0x3ffff 30 29 0 0 cpu16 0x3c000 344 341 10 6 cpu16 0x3ffff 103 100 2 1 cpu17 0x3c000 183 179 2 2 cpu17 0x3ffff 78 77 0 0 sum 1839 1791 18 11 In __schedule(), before curr is updated to next, pick_next_task() invokes sched_balance_rq(). This function temporarily unlocks and relocks the runqueu= e, creating a window where other CPUs may observe rq->curr->on_rq as 0. We can safely skip active balancing when src_rq->curr->on_rq =3D=3D 0, as oth= er eligible tasks have likely already been evaluated. We retain the affinity check on dst_rq to trigger active balancing, since such tasks are often woken by (or wake up) tasks on src_rq that share similar affinity constraints. Furthermore, detach_tasks() releases the runqueue lock; any tasks awakened during this window may preempt the previous CFS task. My testing (data not shown) indicates that active balancing succeeds in 98.4% of cases where !fair && on_rq. This scenario does not require a stop-work callback, but would necessitate an additional detach/attach path. As Valentin and Vincent have already discussed, this addition does not appear justified at this time (see [1]). Since can_migrate_task() already checks on_cpu during the cfs_tasks traversal, adding an on_rq check will have negligible performance overhead due to cache locality. There are two reasons for not combining the on_rq check with the cpumask_test_cpu() check: - Avoiding new scenarios that would skip the logic for resetting balance_interval to min_interval. - The existing check for whether the busiest CPU recently triggered active load balancing already filters more cases than the on_rq check. [1]: https://lore.kernel.org/lkml/20190815145107.5318-5-valentin.schneider@ar= m.com/ Signed-off-by: Xin Zhao Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Valentin Schneider Link: https://patch.msgid.link/20260617072151.1173416-2-jackzxcui1989@163.com --- kernel/sched/fair.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index c575910..7d9ad78 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -13584,12 +13584,21 @@ more_balance: * ->active_balance_work. Once set, it's cleared * only after active load balance is finished. */ - if (!busiest->active_balance) { - busiest->active_balance =3D 1; - busiest->push_cpu =3D this_cpu; - active_balance =3D 1; - } + if (busiest->active_balance) + goto no_active_balance; + + /* + * @busiest dropped its rq_lock in the middle of + * scheduling out its ->curr task (->on_rq :=3D 0), no + * need to forcefully punt it away with active balance. + */ + if (!busiest->curr->on_rq) + goto no_active_balance; =20 + busiest->active_balance =3D 1; + busiest->push_cpu =3D this_cpu; + active_balance =3D 1; +no_active_balance: preempt_disable(); raw_spin_rq_unlock_irqrestore(busiest, flags); if (active_balance) {