mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Aaron Lu" <ziqianlu@bytedance.com>
To: "Tim Chen" <tim.c.chen@linux.intel.com>
Cc: "Peter Zijlstra" <peterz@infradead.org>,
	 "Ingo Molnar" <mingo@redhat.com>,
	 "K Prateek Nayak" <kprateek.nayak@amd.com>,
	 "Gautham R . Shenoy" <gautham.shenoy@amd.com>,
	 "Vincent Guittot" <vincent.guittot@linaro.org>,
	 "Juri Lelli" <juri.lelli@redhat.com>,
	 "Dietmar Eggemann" <dietmar.eggemann@arm.com>,
	 "Steven Rostedt" <rostedt@goodmis.org>,
	 "Ben Segall" <bsegall@google.com>,
	"Mel Gorman" <mgorman@suse.de>,
	 "Valentin Schneider" <vschneid@redhat.com>,
	 "Madadi Vineeth Reddy" <vineethr@linux.ibm.com>,
	 "Hillf Danton" <hdanton@sina.com>,
	 "Shrikanth Hegde" <sshegde@linux.ibm.com>,
	 "Jianyong Wu" <jianyong.wu@outlook.com>,
	 "Yangyu Chen" <cyy@cyyself.name>,
	 "Tingyin Duan" <tingyin.duan@gmail.com>,
	 "Vern Hao" <vernhao@tencent.com>,
	"Vern Hao" <haoxing990@gmail.com>,
	 "Len Brown" <len.brown@intel.com>,
	"Aubrey Li" <aubrey.li@intel.com>,
	 "Zhao Liu" <zhao1.liu@intel.com>,
	"Chen Yu" <yu.chen.surf@gmail.com>,
	 "Chen Yu" <yu.c.chen@intel.com>,
	 "Adam Li" <adamli@os.amperecomputing.com>,
	 "Tim Chen" <tim.c.chen@intel.com>,
	<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2 00/23] Cache aware scheduling
Date: Fri, 19 Dec 2025 11:19:52 +0800	[thread overview]
Message-ID: <20251219031952.GA2598944@bytedance.com> (raw)
In-Reply-To: <cover.1764801860.git.tim.c.chen@linux.intel.com>

On Wed, Dec 03, 2025 at 03:07:19PM -0800, Tim Chen wrote:
... ...
> Test results:
> 
> The patch series was applied and tested on v6.18-rc7.
> See: https://github.com/timcchen1298/linux/commits/cache_aware_v2
> 
> The first test platform is a 2 socket Intel Sapphire Rapids with 30
> cores per socket. The DRAM interleaving is enabled in the BIOS so it
> essential has one NUMA node with two last level caches. There are 60
> CPUs associated with each last level cache.
> 
> The second test platform is a AMD Genoa. There are 4 Nodes and 32 CPUs
> per node. Each node has 2 CCXs and each CCX has 16 CPUs.
> 
> hackbench/schbench/netperf/stream/stress-ng/chacha20 were launched on
> these two platforms.
> 
> [TL;DR]
> Sappire Rapids:
> hackbench shows significant improvement when the number of
> different active threads is below the capacity of a LLC.
> schbench shows overall wakeup latency improvement.
> ChaCha20-xiangshan shows good throughput improvement.
> 
> Genoa:
> ChaCha20-xiangshan shows huge throughput improvement.
> No obvious difference is observed in hackbench/schbench

I think for small task number hackbench run, there should be some
improvement.

I tried thread/pipe/2fds/1group, i.e. 4 tasks on Genoa:
./hackbench -T -f 2 -g 1 -p -l 2000000
And I noticed performance improved a lot:
(Result in seconds, less is better)

       llc_off       llc_on          diff
time   4.755±1.6%    2.684±6.25%    +43.6%

llc_off means /sys/kernel/debug/sched/llc_enabled set to 0 while
llc_on means /sys/kernel/debug/sched/llc_enabled set to 1, other
tunnables are left unchanged.
Turbo is disabled and cpufreq set to performance.

I also tried redis and noticed when I set io-threads to 4 in redis.conf,
there is also some improvement on AMD Genoa:

                 llc_off        manual      diff     llc_on      diff
throughput      1536727±0%     1737619±0%  +13.1%   1737720±0%  +13.1%

Client cmdline:
numactl -N 1 redis-benchmark --threads 4 -t set -r 100000 -P 16 -n 10000000
server cmdline: numactl -N 0 redis-server ./redis.conf
I also tried to manually bind all tasks of redis server to a single LLC
to see if this workload benefits from aggregation and that's what manual
means: taskset -c 8-15,200-207 redis-server ./redis.conf

According to the result, I think this 'cache aware scheduling' works
as expected in that its performance is the same as manual binding; and
they all beat llc_off.

  parent reply	other threads:[~2025-12-19  3:20 UTC|newest]

Thread overview: 120+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-12-03 23:07 Tim Chen
2025-12-03 23:07 ` [PATCH v2 01/23] sched/cache: Introduce infrastructure for cache-aware load balancing Tim Chen
2025-12-09 11:12   ` Peter Zijlstra
2025-12-09 21:39     ` Tim Chen
2025-12-10  9:37   ` Peter Zijlstra
2025-12-10 13:57     ` Chen, Yu C
2025-12-10 15:11       ` Peter Zijlstra
2025-12-11  9:03   ` Vern Hao
2025-12-16  6:12     ` Chen, Yu C
2025-12-17  1:17       ` Vern Hao
2026-01-15 21:47         ` Tim Chen
     [not found]   ` <fbf52d91-0605-4608-b9cc-e8cc56115fd5@gmail.com>
2025-12-16 22:30     ` Tim Chen
2025-12-03 23:07 ` [PATCH v2 02/23] sched/cache: Record per-LLC utilization to guide cache-aware scheduling decisions Tim Chen
2025-12-09 11:21   ` Peter Zijlstra
2025-12-10 14:02     ` Chen, Yu C
2025-12-10 15:13       ` Peter Zijlstra
2025-12-10 23:58         ` Chen, Yu C
2025-12-03 23:07 ` [PATCH v2 03/23] sched/cache: Introduce helper functions to enforce LLC migration policy Tim Chen
2026-01-22 18:13   ` Yangyu Chen
2026-01-22 20:43     ` Tim Chen
2025-12-03 23:07 ` [PATCH v2 04/23] sched/cache: Make LLC id continuous Tim Chen
2025-12-09 11:58   ` Peter Zijlstra
2025-12-15 20:49     ` Tim Chen
2025-12-16  5:31       ` Chen, Yu C
2025-12-16 19:53         ` Tim Chen
2025-12-17  5:25           ` Chen, Yu C
2025-12-23  5:31   ` K Prateek Nayak
2025-12-24  7:08     ` Chen, Yu C
2025-12-24  8:19       ` K Prateek Nayak
2025-12-24  9:46         ` Chen, Yu C
2025-12-26  3:17           ` K Prateek Nayak
2025-12-03 23:07 ` [PATCH v2 05/23] sched/cache: Assign preferred LLC ID to processes Tim Chen
2025-12-09 12:11   ` Peter Zijlstra
2025-12-09 22:34     ` Tim Chen
2025-12-12  3:34   ` Vern Hao
2025-12-15 19:32     ` Tim Chen
2025-12-19  4:01       ` Vern Hao
2025-12-24 10:20         ` Chen, Yu C
2026-01-07  4:49   ` Jianyong Wu
2026-01-07  8:38     ` Chen, Yu C
2025-12-03 23:07 ` [PATCH v2 06/23] sched/cache: Track LLC-preferred tasks per runqueue Tim Chen
2025-12-09 12:16   ` Peter Zijlstra
2025-12-09 22:55     ` Tim Chen
2025-12-10  9:42       ` Peter Zijlstra
2025-12-16  0:20         ` Chen, Yu C
2025-12-17 10:04   ` Vern Hao
2025-12-17 12:37     ` Chen, Yu C
2025-12-03 23:07 ` [PATCH v2 07/23] sched/cache: Introduce per runqueue task LLC preference counter Tim Chen
2025-12-09 13:06   ` Peter Zijlstra
2025-12-09 23:17     ` Tim Chen
2025-12-10 12:43   ` Peter Zijlstra
2025-12-10 18:36     ` Tim Chen
2025-12-10 12:51   ` Peter Zijlstra
2025-12-10 18:49     ` Tim Chen
2025-12-11 10:31       ` Peter Zijlstra
2025-12-15 19:21         ` Tim Chen
2025-12-16 22:45         ` Tim Chen
2025-12-03 23:07 ` [PATCH v2 08/23] sched/cache: Calculate the per runqueue task LLC preference Tim Chen
2025-12-03 23:07 ` [PATCH v2 09/23] sched/cache: Count tasks prefering destination LLC in a sched group Tim Chen
2025-12-10 12:52   ` Peter Zijlstra
2025-12-10 14:05     ` Chen, Yu C
2025-12-10 15:16       ` Peter Zijlstra
2025-12-10 19:00         ` Tim Chen
2025-12-10 23:50         ` Chen, Yu C
2025-12-03 23:07 ` [PATCH v2 10/23] sched/cache: Check local_group only once in update_sg_lb_stats() Tim Chen
2025-12-03 23:07 ` [PATCH v2 11/23] sched/cache: Prioritize tasks preferring destination LLC during balancing Tim Chen
2025-12-03 23:07 ` [PATCH v2 12/23] sched/cache: Add migrate_llc_task migration type for cache-aware balancing Tim Chen
2025-12-10 13:32   ` Peter Zijlstra
2025-12-16  0:52     ` Chen, Yu C
2025-12-03 23:07 ` [PATCH v2 13/23] sched/cache: Handle moving single tasks to/from their preferred LLC Tim Chen
2025-12-03 23:07 ` [PATCH v2 14/23] sched/cache: Consider LLC preference when selecting tasks for load balancing Tim Chen
2025-12-10 15:58   ` Peter Zijlstra
2025-12-03 23:07 ` [PATCH v2 15/23] sched/cache: Respect LLC preference in task migration and detach Tim Chen
2025-12-10 16:30   ` Peter Zijlstra
2025-12-16  7:30     ` Chen, Yu C
2025-12-03 23:07 ` [PATCH v2 16/23] sched/cache: Introduce sched_cache_present to enable cache aware scheduling for multi LLCs NUMA node Tim Chen
2025-12-10 16:32   ` Peter Zijlstra
2025-12-10 16:52     ` Peter Zijlstra
2025-12-16  7:36       ` Chen, Yu C
2025-12-16  7:31     ` Chen, Yu C
2025-12-03 23:07 ` [PATCH v2 17/23] sched/cache: Record the number of active threads per process for cache-aware scheduling Tim Chen
2025-12-10 16:51   ` Peter Zijlstra
2025-12-16  7:40     ` Chen, Yu C
2025-12-17  9:40   ` Aaron Lu
2025-12-17 12:51     ` Chen, Yu C
2025-12-19  3:32       ` Aaron Lu
2025-12-03 23:07 ` [PATCH v2 18/23] sched/cache: Disable cache aware scheduling for processes with high thread counts Tim Chen
2025-12-03 23:07 ` [PATCH v2 19/23] sched/cache: Avoid cache-aware scheduling for memory-heavy processes Tim Chen
2025-12-18  3:59   ` Vern Hao
2025-12-18  8:32     ` Chen, Yu C
2025-12-18  9:42       ` Vern Hao
2025-12-19  3:14         ` K Prateek Nayak
2025-12-19 12:55           ` Chen, Yu C
2025-12-22  2:49             ` Vern Hao
2025-12-22  2:19           ` Vern Hao
2025-12-03 23:07 ` [PATCH v2 20/23] sched/cache: Add user control to adjust the parameters of cache-aware scheduling Tim Chen
2025-12-10 17:02   ` Peter Zijlstra
2025-12-16  7:42     ` Chen, Yu C
2025-12-19  4:14   ` Vern Hao
2025-12-19 13:21     ` Chen, Yu C
2025-12-19 13:39     ` Chen, Yu C
2025-12-23 12:12   ` Yangyu Chen
2025-12-23 16:44     ` Yangyu Chen
2025-12-24  3:28       ` Yangyu Chen
2025-12-24  7:51         ` Chen, Yu C
2025-12-24 12:15           ` Yangyu Chen
2026-01-15 10:03   ` Jianyong Wu
2026-01-15 12:13     ` Chen, Yu C
2026-01-21 15:21   ` Yangyu Chen
2026-01-21 15:38     ` Chen, Yu C
2025-12-03 23:07 ` [PATCH v2 21/23] -- DO NOT APPLY!!! -- sched/cache/stats: Add schedstat for cache aware load balancing Tim Chen
2025-12-19  5:03   ` Yangyu Chen
2025-12-19 14:41     ` Chen, Yu C
2025-12-19 14:48       ` Yangyu Chen
2025-12-03 23:07 ` [PATCH v2 22/23] -- DO NOT APPLY!!! -- sched/cache/debug: Add ftrace to track the load balance statistics Tim Chen
2025-12-03 23:07 ` [PATCH v2 23/23] -- DO NOT APPLY!!! -- sched/cache/debug: Display the per LLC occupancy for each process via proc fs Tim Chen
2025-12-17  9:59   ` Aaron Lu
2025-12-17 13:01     ` Chen, Yu C
2025-12-19  3:19 ` Aaron Lu [this message]
2025-12-19 13:04   ` [PATCH v2 00/23] Cache aware scheduling Chen, Yu C

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20251219031952.GA2598944@bytedance.com \
    --to=ziqianlu@bytedance.com \
    --cc=adamli@os.amperecomputing.com \
    --cc=aubrey.li@intel.com \
    --cc=bsegall@google.com \
    --cc=cyy@cyyself.name \
    --cc=dietmar.eggemann@arm.com \
    --cc=gautham.shenoy@amd.com \
    --cc=haoxing990@gmail.com \
    --cc=hdanton@sina.com \
    --cc=jianyong.wu@outlook.com \
    --cc=juri.lelli@redhat.com \
    --cc=kprateek.nayak@amd.com \
    --cc=len.brown@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@suse.de \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=sshegde@linux.ibm.com \
    --cc=tim.c.chen@intel.com \
    --cc=tim.c.chen@linux.intel.com \
    --cc=tingyin.duan@gmail.com \
    --cc=vernhao@tencent.com \
    --cc=vincent.guittot@linaro.org \
    --cc=vineethr@linux.ibm.com \
    --cc=vschneid@redhat.com \
    --cc=yu.c.chen@intel.com \
    --cc=yu.chen.surf@gmail.com \
    --cc=zhao1.liu@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

Powered by JetHome