From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from canpmsgout03.his.huawei.com (canpmsgout03.his.huawei.com [113.46.200.218]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 987BB2D5436 for ; Thu, 18 Jun 2026 06:16:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=113.46.200.218 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781763410; cv=none; b=E1QnUBwxNvHWxlFme0AXFauyuFfV5H8PbhVA63HEfcQQmu9VFvI1NMI1ERai7kHEcBuTKPAp5NFKtx7s+7QhKIu/AJpe9tsZ1M8izeJKtLID0Lj3W7767XlLy8J6JHBplwQ/IDIHDofYp36W8xwX5L2kG7eXPSXicamjYx485Og= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781763410; c=relaxed/simple; bh=lWYb1xcT/uJZwNoJ6W1QZNQjLxmlFYJ3QhhoCiSyfPM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ZKtT1AdBidhlDK1WRyrijC+RXgZuvYafTgYBN6S30NQOd92qbsvf9XnPJ8mKtRA8AEtu86OF9I0ZaJGLIrEBlDjEy4CMQkTapYNbBqEvW/Lu73lUHxuSiohCalidITOszSKKwNFvvmDOuo3sBHaZqj1zkyxEk+Be28gor1qD/+Q= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b=Jt0WXBHE; arc=none smtp.client-ip=113.46.200.218 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=huawei.com header.i=@huawei.com header.b="Jt0WXBHE" dkim-signature: v=1; a=rsa-sha256; d=huawei.com; s=dkim; c=relaxed/relaxed; q=dns/txt; h=From; bh=siVoD7EudZ8s3ovZjKRPCfwAaecSvdTroP/RDDsZSWE=; b=Jt0WXBHEDA/+t2K52XRzCfaSbheWEAi83Slcxk4CQ8djEtn8BMEZ9xIYxHHfPagc0dUregq6G mSU8+Ecs7/AK9qh9rfv1ulVfPJ5Rx76wMkOWyHqPEEwi/RPuQtt0vTNaREToMipfzXBxXdF1Xcx KALceZycWfru4Z6HwOo29UQ= Received: from mail.maildlp.com (unknown [172.19.162.144]) by canpmsgout03.his.huawei.com (SkyGuard) with ESMTPS id 4ggqzY4tCVzpTTG; Thu, 18 Jun 2026 14:08:41 +0800 (CST) Received: from kwepemj100017.china.huawei.com (unknown [7.202.194.11]) by mail.maildlp.com (Postfix) with ESMTPS id 5085540538; Thu, 18 Jun 2026 14:16:46 +0800 (CST) Received: from huawei.com (10.67.174.193) by kwepemj100017.china.huawei.com (7.202.194.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.36; Thu, 18 Jun 2026 14:16:45 +0800 From: Luo Gengkun To: , , , , , CC: , , , , , , Subject: [PATCH v4 2/2] -- DO NOT APPLY!!! -- sched/cache/debug: Add trace event and sched feature to track scan cost Date: Thu, 18 Jun 2026 06:43:00 +0000 Message-ID: <20260618064300.80072-3-luogengkun2@huawei.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260618064300.80072-1-luogengkun2@huawei.com> References: <20260618064300.80072-1-luogengkun2@huawei.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: kwepems100001.china.huawei.com (7.221.188.238) To kwepemj100017.china.huawei.com (7.202.194.11) To evaluate the effectiveness of the previous patch, this debug patch re-adds get_scan_cpumasks() and introduces two sched features as well as trace events to facilitate testing. Signed-off-by: Luo Gengkun --- include/trace/events/sched.h | 21 +++++++++++++ kernel/sched/fair.c | 61 ++++++++++++++++++++++++++++++++++-- kernel/sched/features.h | 2 ++ 3 files changed, 81 insertions(+), 3 deletions(-) diff --git a/include/trace/events/sched.h b/include/trace/events/sched.h index 535860581f15..aced624f198d 100644 --- a/include/trace/events/sched.h +++ b/include/trace/events/sched.h @@ -10,6 +10,27 @@ #include #include +TRACE_EVENT(sched_cache_scan, + + TP_PROTO(struct task_struct *t, int scan), + + TP_ARGS(t, scan), + + TP_STRUCT__entry( + __string( comm, t->comm ) + __field( pid_t, pid ) + __field( int, scan ) + ), + + TP_fast_assign( + __assign_str(comm); + __entry->pid = t->pid; + __entry->scan = scan; + ), + + TP_printk("comm=%s pid=%d scan=%d", __get_str(comm), __entry->pid, + __entry->scan) +); /* * Tracepoint for calling kthread_stop, performed to end a kthread: */ diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 61f71857e9b1..4016b3291db4 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1647,7 +1647,8 @@ static unsigned long fraction_mm_sched(int cpu, guard(raw_spinlock_irqsave)(&rq->cpu_epoch_lock); /* Skip the rq that has not been hit for a long time */ - if ((rq->cpu_epoch - pcpu_sched->epoch_timeout) > llc_epoch_affinity_timeout) { + if (sched_feat(SC_VISIT) && + (rq->cpu_epoch - pcpu_sched->epoch_timeout) > llc_epoch_affinity_timeout) { cpumask_clear_cpu(cpu, &mm->sc_stat.visited_cpus); return 0; } @@ -1724,7 +1725,8 @@ void account_mm_sched(struct rq *rq, struct task_struct *p, s64 delta_exec) rq->cpu_runtime += delta_exec; epoch = rq->cpu_epoch; pcpu_sched->epoch_timeout = epoch; - if (!cpumask_test_cpu(cpu_of(rq), &mm->sc_stat.visited_cpus)) + if (sched_feat(SC_VISIT) && + !cpumask_test_cpu(cpu_of(rq), &mm->sc_stat.visited_cpus)) cpumask_set_cpu(cpu_of(rq), &mm->sc_stat.visited_cpus); } @@ -1776,6 +1778,51 @@ static void task_tick_cache(struct rq *rq, struct task_struct *p) } } +static void get_scan_cpumasks(cpumask_var_t cpus, struct task_struct *p) +{ +#ifdef CONFIG_NUMA_BALANCING + int cpu, curr_cpu, nid, pref_nid; + + if (!static_branch_likely(&sched_numa_balancing)) + goto out; + + cpu = READ_ONCE(p->mm->sc_stat.cpu); + if (cpu != -1) + nid = cpu_to_node(cpu); + curr_cpu = task_cpu(p); + + /* + * Scanning in the preferred NUMA node is ideal. However, the NUMA + * preferred node is per-task rather than per-process. It is possible + * for different threads of the process to have distinct preferred + * nodes; consequently, the process-wide preferred LLC may bounce + * between different nodes. As a workaround, maintain the scan + * CPU mask to also cover the process's current preferred LLC and the + * current running node to mitigate the bouncing risk. + * TBD: numa_group should be considered during task aggregation. + */ + pref_nid = p->numa_preferred_nid; + /* honor the task's preferred node */ + if (pref_nid == NUMA_NO_NODE) + goto out; + + cpumask_or(cpus, cpus, cpumask_of_node(pref_nid)); + + /* honor the task's preferred LLC CPU */ + if (cpu != -1 && !cpumask_test_cpu(cpu, cpus) && nid != NUMA_NO_NODE) + cpumask_or(cpus, cpus, cpumask_of_node(nid)); + + /* make sure the task's current running node is included */ + if (!cpumask_test_cpu(curr_cpu, cpus)) + cpumask_or(cpus, cpus, cpumask_of_node(cpu_to_node(curr_cpu))); + + return; + +out: +#endif + cpumask_copy(cpus, cpu_online_mask); +} + static inline void update_avg_scale(u64 *avg, u64 sample) { int factor = per_cpu(sd_llc_size, raw_smp_processor_id()); @@ -1803,6 +1850,7 @@ static void task_cache_work(struct callback_head *work) struct mm_struct *mm = p->mm; unsigned long m_a_occ = 0; cpumask_var_t cpus; + int scanned = 0; WARN_ON_ONCE(work != &p->cache_work); @@ -1836,7 +1884,12 @@ static void task_cache_work(struct callback_head *work) scoped_guard (cpus_read_lock) { guard(rcu)(); - cpumask_and(cpus, cpu_online_mask, &mm->sc_stat.visited_cpus); + if (sched_feat(SC_NODE)) + get_scan_cpumasks(cpus, p); + if (sched_feat(SC_VISIT)) + cpumask_and(cpus, cpu_online_mask, &mm->sc_stat.visited_cpus); + else + cpumask_copy(cpus, cpu_online_mask); for_each_cpu(cpu, cpus) { /* XXX sched_cluster_active */ @@ -1853,6 +1906,7 @@ static void task_cache_work(struct callback_head *work) cur->mm == mm) nr_running++; + scanned++; occ = fraction_mm_sched(i, mm); if (occ == 0) continue; @@ -1908,6 +1962,7 @@ static void task_cache_work(struct callback_head *work) update_avg_scale(&mm->sc_stat.nr_running_avg, nr_running); free_cpumask_var(cpus); + trace_sched_cache_scan(p, scanned); } void init_sched_mm(struct task_struct *p) diff --git a/kernel/sched/features.h b/kernel/sched/features.h index 8f0dee8fc475..219173454320 100644 --- a/kernel/sched/features.h +++ b/kernel/sched/features.h @@ -142,3 +142,5 @@ SCHED_FEAT(LATENCY_WARN, false) */ SCHED_FEAT(NI_RANDOM, true) SCHED_FEAT(NI_RATE, true) +SCHED_FEAT(SC_NODE, true) +SCHED_FEAT(SC_VISIT, true) -- 2.34.1