Greeting, FYI, we noticed a -11.1% regression of fsmark.files_per_sec due to commit: commit: fe23914dddb126a29ac4929415eb318f60d97cac ("mm, vmscan: consider eligible zones in get_scan_count") https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master in testcase: fsmark on test machine: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 64G memory with following parameters: iterations: 1 nr_threads: 64 disk: 3HDD md: RAID5 fs: btrfs filesize: 4M test_size: 130G sync_method: NoSync cpufreq_governor: performance test-description: The fsmark is a file system benchmark to test synchronous write workloads, for example, mail servers workload. test-url: https://sourceforge.net/projects/fsmark/ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml testcase/path_params/tbox_group/run: fsmark/1-64-3HDD-RAID5-btrfs-4M-130G-NoSync-performance/ivb44 13b4e0592df44c15 fe23914dddb126a29ac4929415 ---------------- -------------------------- 57.60 -11% 51.20 fsmark.files_per_sec 608 10% 670 fsmark.time.elapsed_time 608 10% 670 fsmark.time.elapsed_time.max 141 5% 149 fsmark.time.system_time 613586 628565 fsmark.time.voluntary_context_switches 14682 ± 7% -25% 11031 ± 5% fsmark.time.involuntary_context_switches 301042 29% 389835 ± 5% interrupts.CAL:Function_call_interrupts 207732 -9% 188203 vmstat.io.bo 6654 6541 vmstat.system.cs 5.08 -4% 4.88 turbostat.RAMWatt 92 -5% 88 turbostat.Avg_MHz 3.23 ± 4% -8% 2.97 turbostat.%Busy 155081 ± 12% -1e+05 19550 ± 76% latency_stats.sum.btrfs_tree_lock.[btrfs].btrfs_search_slot.[btrfs].btrfs_insert_empty_items.[btrfs].btrfs_new_inode.[btrfs].btrfs_create.[btrfs].path_openat.do_filp_open.do_sys_open.SyS_open.entry_SYSCALL_64_fastpath 1454653 ± 3% -1e+06 209533 ± 47% latency_stats.sum.btrfs_tree_lock.[btrfs].btrfs_lock_root_node.[btrfs].btrfs_search_slot.[btrfs].btrfs_insert_empty_items.[btrfs].insert_with_overflow.[btrfs].btrfs_insert_dir_item.[btrfs].btrfs_add_link.[btrfs].btrfs_create.[btrfs].path_openat.do_filp_open.do_sys_open.SyS_open 3.351e+08 ± 6% -3e+08 71080683 ± 24% latency_stats.sum.wait_current_trans.[btrfs].start_transaction.[btrfs].btrfs_start_transaction.[btrfs].btrfs_create.[btrfs].path_openat.do_filp_open.do_sys_open.SyS_open.entry_SYSCALL_64_fastpath 17.26 ± 12% 31% 22.53 ± 15% perf-stat.node-store-miss-rate% 2.856e+11 ± 5% 14% 3.267e+11 ± 4% perf-stat.dTLB-loads 1.815e+09 ± 11% 14% 2.071e+09 ± 6% perf-stat.node-load-misses 1156825 10% 1268375 perf-stat.minor-faults 1156846 10% 1268396 perf-stat.page-faults 4058202 8% 4395013 perf-stat.context-switches 2.071e+08 ± 6% 8% 2.228e+08 perf-stat.iTLB-loads 32.23 ± 3% 7% 34.55 perf-stat.node-load-miss-rate% 2.322e+08 5% 2.441e+08 perf-stat.iTLB-load-misses 0.44 0.43 perf-stat.ipc 1.925e+10 ± 3% -7% 1.783e+10 ± 5% perf-stat.cache-references 1.702e+09 ± 9% -9% 1.542e+09 ± 6% perf-stat.branch-misses 49.17 -11% 43.60 ± 3% perf-stat.cache-miss-rate% 4.582e+09 -15% 3.904e+09 perf-stat.node-stores 9.459e+09 -18% 7.764e+09 ± 3% perf-stat.cache-misses 207580 -9% 188098 iostat.md0.wkB/s 12.24 ± 5% 42% 17.38 ± 10% iostat.sda.rrqm/s 103790 -9% 94071 iostat.sda.wkB/s 25722 -10% 23255 iostat.sda.wrqm/s 931 -16% 783 iostat.sda.avgrq-sz 120.88 -42% 69.92 ± 3% iostat.sda.r_await 26.14 -55% 11.88 ± 3% iostat.sda.avgqu-sz 117.72 -56% 52.03 ± 3% iostat.sda.await 117.47 -56% 51.76 ± 3% iostat.sda.w_await 15.66 ± 5% 37% 21.39 ± 8% iostat.sdb.rrqm/s 103763 -9% 94048 iostat.sdb.wkB/s 25717 -10% 23249 iostat.sdb.wrqm/s 930 -16% 783 iostat.sdb.avgrq-sz 120.42 -39% 73.21 ± 4% iostat.sdb.r_await 27.19 -54% 12.38 ± 3% iostat.sdb.avgqu-sz 122.32 -56% 54.23 ± 3% iostat.sdb.await 122.10 -56% 53.98 ± 3% iostat.sdb.w_await 11.84 ± 4% 41% 16.68 ± 7% iostat.sdc.rrqm/s 228 18% 270 iostat.sdc.w/s 103791 -9% 94074 iostat.sdc.wkB/s 25721 -10% 23246 iostat.sdc.wrqm/s 904 -21% 710 iostat.sdc.avgrq-sz 117.37 -40% 70.17 ± 3% iostat.sdc.r_await 26.85 -53% 12.71 iostat.sdc.avgqu-sz 118.03 -56% 52.24 ± 3% iostat.sdc.await 117.83 -56% 52.02 ± 3% iostat.sdc.w_await perf-stat.page-faults 1.3e+06 ++---------------------------------------------------------------+ | | 1.28e+06 O+O O OO O O O O | 1.26e+06 ++ O O OO O O | | O O O O | 1.24e+06 ++ | | | 1.22e+06 ++ | | | 1.2e+06 ++ | 1.18e+06 ++ | | | 1.16e+06 ++ .*. *. *.*. .*.* *. .*. .*. *.| *.**.* **. .* *.*.**.*.* *.** *.*.* * **.*.**.* * * 1.14e+06 ++----------*----------------------------------------------------+ perf-stat.minor-faults 1.3e+06 ++---------------------------------------------------------------+ | | 1.28e+06 O+O O OO O O O O | 1.26e+06 ++ O O OO O O | | O O O O | 1.24e+06 ++ | | | 1.22e+06 ++ | | | 1.2e+06 ++ | 1.18e+06 ++ | | | 1.16e+06 ++ .*. *. *.*. .*.* *. .*. .*. *.| *.**.* **. .* *.*.**.*.* *.** *.*.* * **.*.**.* * * 1.14e+06 ++----------*----------------------------------------------------+ fsmark.time.elapsed_time 680 ++--------------------------------------------------------------------+ | O O O O O | 670 O+ O O O O O OO O O O O O | 660 ++ O | | | 650 ++ | | | 640 ++ | | | 630 ++ | 620 ++ | | | 610 ++ .*. *. .*. *.*. .* .**.*. *. .*.| *.* * *.*.*.*.**.*.*.* * *.*.*.* *.*.*.*.* *.*.*.* * * 600 ++--------------------------------------------------------------------+ fsmark.time.elapsed_time.max 680 ++--------------------------------------------------------------------+ | O O O O O | 670 O+ O O O O O OO O O O O O | 660 ++ O | | | 650 ++ | | | 640 ++ | | | 630 ++ | 620 ++ | | | 610 ++ .*. *. .*. *.*. .* .**.*. *. .*.| *.* * *.*.*.*.**.*.*.* * *.*.*.* *.*.*.*.* *.*.*.* * * 600 ++--------------------------------------------------------------------+ fsmark.files_per_sec 58 ++---------------------------------------------------------------------+ *.*.*.**.*.*.*.*.*.*.**.*.*.*.*. .*.**.*. .*.*.*.**.*.*.*.*.*.*.**.*.*.* 57 ++ * * | | | 56 ++ | | | 55 ++ | | | 54 ++ | | | 53 ++ | | | 52 ++ | | | 51 O+O-O-OO-O-O-O-O-O-O-OO-O-O-O-O-O-O------------------------------------+ vmstat.io.bo 210000 ++------------------------------*--------*-------------------------+ *.*.**. .*.**.*.*.**.*.*. *. + **.*. + *.*.*.**.*.*.**.*.*. *.*.* | * * *.* * * | 205000 ++ | | | | | 200000 ++ | | | 195000 ++ | | | | | 190000 ++ O O | O OO O O O O O OO O O O O | | O O O | 185000 ++-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Xiaolong