From: Luo Gengkun <luogengkun@huaweicloud.com>
To: K Prateek Nayak <kprateek.nayak@amd.com>,
peterz@infradead.org, mingo@redhat.com, juri.lelli@redhat.com,
vincent.guittot@linaro.org, tim.c.chen@linux.intel.com,
yu.c.chen@intel.com
Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org,
bsegall@google.com, mgorman@suse.de, vschneid@redhat.com,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v10 1/2] sched/cache: Reduce the overhead of task_cache_work by only scan the visisted cpus
Date: Mon, 9 Nov 2026 15:04:29 +0800 [thread overview]
Message-ID: <0cb3d965-1eb0-41fe-8abd-67292bc2cfeb@huaweicloud.com> (raw)
In-Reply-To: <1e9259b5-0758-4387-a572-37e2d32c47c0@amd.com>
On 2026/10/21 12:30, K Prateek Nayak wrote:
> Hello Luo,
>
> On 10/9/2026 8:59 AM, Luo Gengkun wrote:
>> The overhead of task_cache_work() is high, especially in multi-NUMA systems.
>> Currently, task_cache_work() tries to find the pref_llc by scanning all CPUs
>> in the system. However, most of these scans are meaningless, such as those
>> for CPUs that have never been visited or were accessed a long time ago.
>>
>> To address this problem, introduce visited_cpus to track the visited CPUs
>> and evict them once they have not been accessed for a duration exceeding
>> llc_epoch_affinity_timeout.
>>
>> Now that we know exactly which CPUs to scan from visited_cpus, we can remove
>> get_scan_cpumasks().
>
> nit. Could you include the performance numbers from the cover letter
> in the commit message here to keep it around for the record.
>
>>
>> Signed-off-by: Luo Gengkun <luogengkun@huaweicloud.com>
>
> I gave it a spin on my 4th Generation EPYC system (2 x 128C/256T) and
> I could see small improvements and no regressions greater than the
> expected run to run variance. Feel free to include:
>
> Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Hi Prateek,
Apologies for the delayed response! I have been away on a business trip
for the past few weeks and am just getting back to this now.
Thank you so much for reviewing and taking the time to test this patch
on your system.
I will include the performance numbers in the commit message and add
your Tested-by tag in the next version.
Best regards,
Luo Gengkun
>
> ---
>
> o Full benchmark results
>
> ==================================================================
> Test : hackbench
> Units : Normalized time in seconds
> Interpretation: Lower is better
> Statistic : AMean
> ==================================================================
> Case: tip[pct imp](CV) cpu-track[pct imp](CV)
> 1-groups 1.00 [ -0.00](14.00) 1.02 [ -1.98](11.49)
> 2-groups 1.00 [ -0.00]( 4.92) 0.91 [ 9.49](11.67)
> 4-groups 1.00 [ -0.00]( 2.01) 0.99 [ 0.51]( 2.98)
> 8-groups 1.00 [ -0.00]( 3.13) 1.02 [ -2.16]( 3.25)
> 16-groups 1.00 [ -0.00]( 1.52) 0.99 [ 1.03]( 0.83)
>
>
> ==================================================================
> Test : tbench
> Units : Normalized throughput
> Interpretation: Higher is better
> Statistic : AMean
> ==================================================================
> Clients: tip[pct imp](CV) cpu-track[pct imp](CV)
> 1 1.00 [ 0.00]( 0.27) 1.01 [ 0.81]( 0.62)
> 2 1.00 [ 0.00]( 0.22) 1.01 [ 1.01]( 0.70)
> 4 1.00 [ 0.00]( 0.36) 1.01 [ 0.65]( 0.13)
> 8 1.00 [ 0.00]( 0.23) 1.01 [ 1.15]( 0.41)
> 16 1.00 [ 0.00]( 0.17) 1.01 [ 1.04]( 0.17)
> 32 1.00 [ 0.00]( 0.16) 1.01 [ 0.95]( 0.12)
> 64 1.00 [ 0.00]( 0.16) 1.01 [ 0.64]( 0.21)
> 128 1.00 [ 0.00]( 0.04) 1.01 [ 0.83]( 0.40)
> 256 1.00 [ 0.00]( 0.14) 1.00 [ -0.05]( 0.25)
> 512 1.00 [ 0.00]( 0.25) 1.00 [ -0.12]( 0.63)
> 1024 1.00 [ 0.00]( 0.31) 1.00 [ 0.46]( 0.29)
> 2048 1.00 [ 0.00]( 0.41) 1.00 [ -0.20]( 0.49)
>
>
> ==================================================================
> Test : stream-10
> Units : Normalized Bandwidth, MB/s
> Interpretation: Higher is better
> Statistic : HMean
> ==================================================================
> Test: tip[pct imp](CV) cpu-track[pct imp](CV)
> Copy 1.00 [ 0.00]( 0.33) 0.98 [ -2.41]( 3.81)
> Scale 1.00 [ 0.00]( 0.63) 0.98 [ -1.64]( 3.10)
> Add 1.00 [ 0.00]( 0.49) 0.98 [ -1.56]( 2.58)
> Triad 1.00 [ 0.00]( 0.46) 0.98 [ -1.87]( 2.66)
>
>
> ==================================================================
> Test : stream-100
> Units : Normalized Bandwidth, MB/s
> Interpretation: Higher is better
> Statistic : HMean
> ==================================================================
> Test: tip[pct imp](CV) cpu-track[pct imp](CV)
> Copy 1.00 [ 0.00]( 0.42) 1.00 [ -0.50]( 1.43)
> Scale 1.00 [ 0.00]( 0.53) 0.99 [ -0.53]( 1.52)
> Add 1.00 [ 0.00]( 0.44) 0.99 [ -0.56]( 1.19)
> Triad 1.00 [ 0.00]( 0.44) 0.99 [ -0.54]( 1.23)
>
>
> ==================================================================
> Test : netperf
> Units : Normalized Througput
> Interpretation: Higher is better
> Statistic : AMean
> ==================================================================
> Clients: tip[pct imp](CV) cpu-track[pct imp](CV)
> 1-clients 1.00 [ 0.00]( 0.03) 1.01 [ 0.86]( 0.17)
> 2-clients 1.00 [ 0.00]( 0.16) 1.00 [ 0.47]( 0.31)
> 4-clients 1.00 [ 0.00]( 0.32) 1.00 [ 0.46]( 0.27)
> 8-clients 1.00 [ 0.00]( 0.45) 1.01 [ 0.55]( 0.26)
> 16-clients 1.00 [ 0.00]( 0.24) 1.01 [ 0.63]( 0.27)
> 32-clients 1.00 [ 0.00]( 0.76) 1.01 [ 0.76]( 0.69)
> 64-clients 1.00 [ 0.00]( 0.73) 1.01 [ 0.79]( 0.89)
> 128-clients 1.00 [ 0.00]( 0.97) 1.01 [ 0.53]( 1.28)
> 256-clients 1.00 [ 0.00]( 1.13) 1.01 [ 0.64]( 1.21)
> 512-clients 1.00 [ 0.00]( 2.23) 1.01 [ 0.57]( 1.74)
> 768-clients 1.00 [ 0.00](11.06) 1.00 [ 0.04](11.12)
> 1024-clients 1.00 [ 0.00](11.42) 1.00 [ 0.03](10.77)
>
>
> ==================================================================
> Test : schbench
> Units : Normalized 99th percentile latency in us
> Interpretation: Lower is better
> Statistic : Median
> ==================================================================
> #workers: tip[pct imp](CV) cpu-track[pct imp](CV)
> 1 1.00 [ -0.00](25.85) 0.76 [ 24.00]( 5.26)
> 2 1.00 [ -0.00](22.30) 1.17 [-16.67]( 4.41)
> 4 1.00 [ -0.00](16.86) 1.10 [-10.00]( 4.58)
> 8 1.00 [ -0.00](16.43) 0.88 [ 12.50](16.90)
> 16 1.00 [ -0.00]( 0.97) 1.03 [ -3.33]( 2.48)
> 32 1.00 [ -0.00]( 2.40) 1.01 [ -1.16]( 2.90)
> 64 1.00 [ -0.00]( 0.40) 1.00 [ -0.00]( 0.81)
> 128 1.00 [ -0.00]( 4.87) 1.01 [ -0.80]( 0.60)
> 256 1.00 [ -0.00]( 0.89) 1.01 [ -0.77]( 1.01)
> 512 1.00 [ -0.00]( 1.41) 1.00 [ -0.00]( 1.64)
> 768 1.00 [ -0.00]( 1.84) 0.98 [ 1.65]( 1.62)
> 1024 1.00 [ -0.00](13.59) 0.92 [ 7.57]( 4.64)
>
>
> ==================================================================
> Test : new-schbench-requests-per-second
> Units : Normalized Requests per second
> Interpretation: Higher is better
> Statistic : Median
> ==================================================================
> #workers: tip[pct imp](CV) cpu-track[pct imp](CV)
> 1 1.00 [ 0.00]( 0.00) 1.01 [ 0.59]( 0.15)
> 2 1.00 [ 0.00]( 0.15) 1.00 [ 0.30]( 0.00)
> 4 1.00 [ 0.00]( 0.00) 1.00 [ -0.29]( 0.00)
> 8 1.00 [ 0.00]( 0.00) 1.00 [ -0.29]( 0.00)
> 16 1.00 [ 0.00]( 0.15) 1.00 [ -0.29]( 0.00)
> 32 1.00 [ 0.00]( 0.15) 0.99 [ -0.59]( 0.00)
> 64 1.00 [ 0.00]( 0.00) 1.00 [ -0.29]( 0.00)
> 128 1.00 [ 0.00](14.82) 1.00 [ 0.00](17.39)
> 256 1.00 [ 0.00]( 0.15) 1.00 [ -0.28]( 0.64)
> 512 1.00 [ 0.00]( 1.40) 1.00 [ -0.39]( 1.73)
> 768 1.00 [ 0.00]( 1.81) 1.00 [ -0.25]( 1.56)
> 1024 1.00 [ 0.00]( 0.17) 1.00 [ -0.32]( 0.76)
>
>
> ==================================================================
> Test : new-schbench-wakeup-latency
> Units : Normalized 99th percentile latency in us
> Interpretation: Lower is better
> Statistic : Median
> ==================================================================
> #workers: tip[pct imp](CV) cpu-track[pct imp](CV)
> 1 1.00 [ -0.00](33.20) 1.21 [-21.43](25.04)
> 2 1.00 [ -0.00]( 5.00) 1.00 [ -0.00]( 8.94)
> 4 1.00 [ -0.00]( 5.53) 1.00 [ -0.00]( 5.53)
> 8 1.00 [ -0.00]( 0.00) 1.00 [ -0.00]( 0.00)
> 16 1.00 [ -0.00]( 4.43) 0.92 [ 8.33]( 8.13)
> 32 1.00 [ -0.00]( 4.84) 1.00 [ -0.00]( 0.00)
> 64 1.00 [ -0.00]( 0.00) 1.10 [-10.00]( 8.13)
> 128 1.00 [ -0.00]( 8.15) 0.90 [ 10.00](10.68)
> 256 1.00 [ -0.00]( 3.36) 0.95 [ 4.91]( 1.68)
> 512 1.00 [ -0.00](36.88) 1.01 [ -0.85](35.08)
> 768 1.00 [ -0.00]( 4.02) 1.00 [ -0.00]( 4.99)
> 1024 1.00 [ -0.00]( 1.61) 1.01 [ -0.78]( 0.87)
>
>
> ==================================================================
> Test : new-schbench-request-latency
> Units : Normalized 99th percentile latency in us
> Interpretation: Lower is better
> Statistic : Median
> ==================================================================
> #workers: tip[pct imp](CV) cpu-track[pct imp](CV)
> 1 1.00 [ -0.00]( 0.14) 1.00 [ 0.26]( 0.00)
> 2 1.00 [ -0.00]( 0.00) 1.00 [ 0.26]( 0.00)
> 4 1.00 [ -0.00]( 0.14) 1.00 [ -0.00]( 0.00)
> 8 1.00 [ -0.00]( 0.00) 1.01 [ -0.53]( 0.00)
> 16 1.00 [ -0.00]( 0.14) 1.01 [ -0.53]( 0.00)
> 32 1.00 [ -0.00]( 0.14) 1.01 [ -0.53]( 0.00)
> 64 1.00 [ -0.00]( 0.00) 1.01 [ -0.79]( 0.14)
> 128 1.00 [ -0.00]( 4.56) 0.99 [ 1.48](27.76)
> 256 1.00 [ -0.00]( 0.13) 1.00 [ -0.25]( 0.13)
> 512 1.00 [ -0.00](28.11) 1.00 [ -0.23](25.97)
> 768 1.00 [ -0.00](13.36) 0.99 [ 0.60](14.72)
> 1024 1.00 [ -0.00]( 1.46) 1.00 [ 0.32]( 6.83)
>
next prev parent reply other threads:[~2026-11-09 7:04 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-09 3:29 [PATCH v10 0/2] Cache aware scheduling: Reduce the overhead of task_cache_work Luo Gengkun
2026-10-09 3:29 ` [PATCH v10 1/2] sched/cache: Reduce the overhead of task_cache_work by only scan the visisted cpus Luo Gengkun
2026-10-21 4:30 ` K Prateek Nayak
2026-11-09 7:04 ` Luo Gengkun [this message]
2026-10-21 16:26 ` Chen Yu
2026-11-09 7:11 ` Luo Gengkun
2026-10-21 21:46 ` Tim Chen
2026-11-09 7:20 ` Luo Gengkun
2026-10-09 3:29 ` [PATCH v10 2/2] -- DO NOT APPLY!!! -- sched/cache/debug: Add trace event and sched feature to track scan cost Luo Gengkun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0cb3d965-1eb0-41fe-8abd-67292bc2cfeb@huaweicloud.com \
--to=luogengkun@huaweicloud.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tim.c.chen@linux.intel.com \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=yu.c.chen@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®