From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C11E346E5D for ; Wed, 17 Dec 2025 10:04:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765965860; cv=none; b=JriHoImSEak+1+cklnRApyiBjq64oj0D7tfWiWEq9q/g6FdqLpdGW1/n1aR8Ydb2+m0Tsq66LBBxMe+QP5MW+EINLKv655yOe66J+vn/qFZMXyLZbatk+/KlobEYEdPe4X93oegYcSc27GiLPpQ/nukiC8yHxRLlQD7gs7JU0BU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765965860; c=relaxed/simple; bh=mkif37fC+AGZ22chVvjFjMig3MN8ntjXIfpJV62Vp/o=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=Nd3DMQHgqlcx0/6DRc0yki9zcczC8FMaXdZNpfr6TEPJrXWCEizDQDSfN9OaOcQKLT5rCSgfmaPiejpoW70lZqjozZCwZ/ewR+pUgueuJyJASn4uv5k+5c4o+0u4Xs1gmou+AU/n01RY24CXcfTvNSXcjekMVeDVlz0NJ4QgdNw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=LjqiBnFs; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LjqiBnFs" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-7b852bb31d9so6442265b3a.0 for ; Wed, 17 Dec 2025 02:04:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1765965856; x=1766570656; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=73Lh5JnZRWXjvPysz1Pftlyfn8b6pPBf4lfY2agrNfA=; b=LjqiBnFsLjctrioOmx6tZSpUjF0svTaSqG2XK0e+1Jab/O3+R1orkp/7IyND2/WQ9b br5F2RXQbigvrrz/n61A32kLImwkXbEYVKrA5sOzpbP79hKXF/8ijoUWDI5oYjmxisqD 7gILREeHw6AtagLlbHd6JHclAXGvRJEbiD9Wrjzwuc8jCfICNIHukFjv2YCDR0hJcJFg srqM6KsfB23KAjdlDJYN9AgFkSIY50teuc+7S9C+xnNsZ2YZZ1vAn57m82d/w9VhIil3 xzhpKXBK/Mwx0PgLGRVCC4k/edu7xabM6lMq4VPtktDqFezcyAOSd3GTUj4xNpx7jRmg yq0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765965856; x=1766570656; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:x-gm-gg:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=73Lh5JnZRWXjvPysz1Pftlyfn8b6pPBf4lfY2agrNfA=; b=C0oUSCZgvRgqioWKmXzkGqU4F3/CyWLa3Wiyw9lD8dRSMtQyCagtZNeTkJeNgmYN6a 5LIf2Ei4wgZW6H5Fg5RbxNgstJ2UMDzAXvt9dK3Fg7kyLgKjypxVAvHt5z+z3M50AoOU 5wcXQ2wUMa6VgAXpwKJn3ofrSrsQlD/YG2/J7BSYP6xHMiV0Xw7YiJbZ2mTnZI5rPJlB JRygJ7k+E/B6VUHbyLT6wdPyw7lqsFIYFM/FohoMuT/QmPIFFtkl7/dj2HC6RWLpN9QJ o4k1HYzJXjtCeBjqUNmihchDRs4Ce+ocRZ5y+swQLJTRlh/vHdcdoo3G1oN1u/DUpHD7 WnoQ== X-Forwarded-Encrypted: i=1; AJvYcCVOwb/Sp/3XIAdoPwsyK6cul9YZaWrX6RiriCt3S0/4Ej/aZFiNL5BVPfzuK1PvJMH8VTUTXlW3cylxUxU=@vger.kernel.org X-Gm-Message-State: AOJu0YwkeLfnoDnU5cIpuAxhxhL+DcEtdRv91XbMP/Wv4VPYCuwsqW0K N5I8tOtz2pbv1XdkA8I0ywnfndEuVR9s7uC9CBX6xYWzjbSyTdIE+mjY X-Gm-Gg: AY/fxX6jFphaSphzaBj7UlYJ35jzNbdPuyqw9Jfr4yS8rFBK58KUURLeJwR7YKVrWZW 2XT60D0gnX1mF1yHFC0HFIBCsCzTNTuZJmkzvQrnFOvKnEAlckmeCBEVaPYWxl7K0clPXHb3jzX vFh4qCna32h8XTbwWWNkvPncrJRZYOeljky0YoHFOOk9h7pPcu219MLQshlRjscr3OX62aixvz1 dFxB/1j8cBhwttcKxv/KBZg8qvCC7b7o5D/eW4ZjzxekH1XtKD4GQ/r5jZgy2l22ZgOCcOE5Xjx 9OQop2BCQ5UER+VBQHXGW0UO+eLbf4ksPlquxCH+7uTBpKyK68YnOUpB4VdgnM+klM0hUyqBw0N ieF0Ecc0ECOp3g4GhmOceCv8DtYNqePzboV0uufevk4d1B92Ac1yrejids/qmZtGi49ZPrIuevt NyHMuI9uNg2pdGJ56+yf4Ar4R4D+exAjfhQb0//WJibttpDwbR X-Google-Smtp-Source: AGHT+IEhM2aXBTptwlpS8QbVpR0nuyxs48lpS802LUvSO9vswrPPrfh7yt1mHqkqbUiRLqqKGjjOWg== X-Received: by 2002:a05:6a00:420d:b0:7e8:43f5:bd2b with SMTP id d2e1a72fcca58-7f669c89073mr15978633b3a.64.1765965855927; Wed, 17 Dec 2025 02:04:15 -0800 (PST) Received: from [192.168.255.10] ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7fcb8744b95sm2243852b3a.9.2025.12.17.02.04.09 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 17 Dec 2025 02:04:15 -0800 (PST) Message-ID: Date: Wed, 17 Dec 2025 18:04:06 +0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 06/23] sched/cache: Track LLC-preferred tasks per runqueue To: Tim Chen , Peter Zijlstra , Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot Cc: Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Madadi Vineeth Reddy , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , linux-kernel@vger.kernel.org References: From: Vern Hao In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 2025/12/4 07:07, Tim Chen wrote: > For each runqueue, track the number of tasks with an LLC preference > and how many of them are running on their preferred LLC. This mirrors > nr_numa_running and nr_preferred_running for NUMA balancing, and will > be used by cache-aware load balancing in later patches. > > Signed-off-by: Tim Chen > --- > > Notes: > v1->v2: Invoke task_of() once and reuse its result afterwards. > (Peter Zijlstra) > Remove hacky reset_llc_stats() and introduce sched_llc_active flag > to properly pair enqueue/dequeue statistics update (Peter Zijlstra, K Prateek Nayak) > > include/linux/sched.h | 2 ++ > init/init_task.c | 1 + > kernel/sched/core.c | 5 ++++ > kernel/sched/fair.c | 60 ++++++++++++++++++++++++++++++++++++++++--- > kernel/sched/sched.h | 6 +++++ > 5 files changed, 71 insertions(+), 3 deletions(-) > > diff --git a/include/linux/sched.h b/include/linux/sched.h > index 1ad46220cd04..466ba8b7398c 100644 > --- a/include/linux/sched.h > +++ b/include/linux/sched.h > @@ -1408,6 +1408,8 @@ struct task_struct { > > #ifdef CONFIG_SCHED_CACHE > struct callback_head cache_work; > + /*the p is currently refcounted in a rq's preferred llc stats*/ > + bool sched_llc_active; > int preferred_llc; > #endif > > diff --git a/init/init_task.c b/init/init_task.c > index 44bae72b5b7d..ee78837b0aa2 100644 > --- a/init/init_task.c > +++ b/init/init_task.c > @@ -192,6 +192,7 @@ struct task_struct init_task __aligned(L1_CACHE_BYTES) = { > .numa_faults = NULL, > #endif > #ifdef CONFIG_SCHED_CACHE > + .sched_llc_active = false, > .preferred_llc = -1, > #endif > #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) > diff --git a/kernel/sched/core.c b/kernel/sched/core.c > index e8bdf03a4b7f..48626c81ba8e 100644 > --- a/kernel/sched/core.c > +++ b/kernel/sched/core.c > @@ -531,6 +531,11 @@ void __trace_set_current_state(int state_value) > } > EXPORT_SYMBOL(__trace_set_current_state); > > +int task_llc(const struct task_struct *p) > +{ > + return per_cpu(sd_llc_id, task_cpu(p)); > +} > + > /* > * Serialization rules: > * > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 10cec83f65d5..d46a70a9d9fb 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -1223,6 +1223,43 @@ static int llc_id(int cpu) > return llc; > } > > +static void account_llc_enqueue(struct rq *rq, struct task_struct *p) > +{ > + int pref_llc; > + > + if (!sched_cache_enabled()) > + return; > + > + pref_llc = p->preferred_llc; > + if (pref_llc < 0) > + return; > + > + rq->nr_llc_running++; > + rq->nr_pref_llc_running += (pref_llc == task_llc(p)); > + p->sched_llc_active = true; > +} > + > +static void account_llc_dequeue(struct rq *rq, struct task_struct *p) > +{ > + int pref_llc; > + > + /* > + * Borrow the uc_se->active from uclamp_rq_inc_id(), > + * uclamp_rq_dec_id() to avoid the unbalanced calculation > + * of rq statistics. > + */ > + if (unlikely(!p->sched_llc_active)) > + return; > + > + pref_llc = p->preferred_llc; > + if (pref_llc < 0) > + return; > + > + rq->nr_llc_running--; > + rq->nr_pref_llc_running -= (pref_llc == task_llc(p)); > + p->sched_llc_active = false; > +} > + > void mm_init_sched(struct mm_struct *mm, struct mm_sched __percpu *_pcpu_sched) > { > unsigned long epoch; > @@ -1294,6 +1331,8 @@ static unsigned long __no_profile fraction_mm_sched(struct rq *rq, struct mm_sch > return div64_u64(NICE_0_LOAD * pcpu_sched->runtime, rq->cpu_runtime + 1); > } > > +static unsigned int task_running_on_cpu(int cpu, struct task_struct *p); > + > static inline > void account_mm_sched(struct rq *rq, struct task_struct *p, s64 delta_exec) > { > @@ -1346,8 +1385,13 @@ void account_mm_sched(struct rq *rq, struct task_struct *p, s64 delta_exec) > #endif > } > > - if (p->preferred_llc != mm_sched_llc) > + /* task not on rq accounted later in account_entity_enqueue() */ > + if (task_running_on_cpu(rq->cpu, p) && > + p->preferred_llc != mm_sched_llc) { #ifdef CONFIG_NUMA_BALANCING                 /*                  * Don't assign preferred LLC if it                  * conflicts with NUMA balancing.                  */                 if (p->numa_preferred_nid >= 0 &&                     cpu_to_node(mm->mm_sched_cpu) != p->numa_preferred_nid)                         mm_sched_llc = -1; #endif         }         /* task not on rq accounted later in account_entity_enqueue() */         if (task_running_on_cpu(rq->cpu, p) &&             p->preferred_llc != mm_sched_llc) {                 account_llc_dequeue(rq, p);                 p->preferred_llc = mm_sched_llc;                 account_llc_enqueue(rq, p);         } I am a little concerned that there might be cases where both |p->preferred_llc| and |mm_sched_llc| are equal to -1 at this point.", Is it necessary to add a check here? > + account_llc_dequeue(rq, p); > p->preferred_llc = mm_sched_llc; > + account_llc_enqueue(rq, p); > + } > } > > static void task_tick_cache(struct rq *rq, struct task_struct *p) > @@ -1475,6 +1519,10 @@ void init_sched_mm(struct task_struct *p) { } > > static void task_tick_cache(struct rq *rq, struct task_struct *p) { } > > +static void account_llc_enqueue(struct rq *rq, struct task_struct *p) {} > + > +static void account_llc_dequeue(struct rq *rq, struct task_struct *p) {} > + > #endif > > /* > @@ -3965,9 +4013,11 @@ account_entity_enqueue(struct cfs_rq *cfs_rq, struct sched_entity *se) > { > update_load_add(&cfs_rq->load, se->load.weight); > if (entity_is_task(se)) { > + struct task_struct *p = task_of(se); > struct rq *rq = rq_of(cfs_rq); > > - account_numa_enqueue(rq, task_of(se)); > + account_numa_enqueue(rq, p); > + account_llc_enqueue(rq, p); > list_add(&se->group_node, &rq->cfs_tasks); > } > cfs_rq->nr_queued++; > @@ -3978,7 +4028,11 @@ account_entity_dequeue(struct cfs_rq *cfs_rq, struct sched_entity *se) > { > update_load_sub(&cfs_rq->load, se->load.weight); > if (entity_is_task(se)) { > - account_numa_dequeue(rq_of(cfs_rq), task_of(se)); > + struct task_struct *p = task_of(se); > + struct rq *rq = rq_of(cfs_rq); > + > + account_numa_dequeue(rq, p); > + account_llc_dequeue(rq, p); > list_del_init(&se->group_node); > } > cfs_rq->nr_queued--; > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h > index 728737641847..ee8b70647835 100644 > --- a/kernel/sched/sched.h > +++ b/kernel/sched/sched.h > @@ -1126,6 +1126,10 @@ struct rq { > unsigned int nr_preferred_running; > unsigned int numa_migrate_on; > #endif > +#ifdef CONFIG_SCHED_CACHE > + unsigned int nr_pref_llc_running; > + unsigned int nr_llc_running; > +#endif > #ifdef CONFIG_NO_HZ_COMMON > unsigned long last_blocked_load_update_tick; > unsigned int has_blocked_load; > @@ -1980,6 +1984,8 @@ init_numa_balancing(u64 clone_flags, struct task_struct *p) > > #endif /* !CONFIG_NUMA_BALANCING */ > > +int task_llc(const struct task_struct *p); > + > static inline void > queue_balance_callback(struct rq *rq, > struct balance_callback *head,