FYI, we noticed the below changes on git://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git x86/entry commit f04e05b81e4d9ae88bee379f12176f551a24394a ("x86_64,entry: Use sysret to return to userspace when possible") f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 testbox/testcase/testparams ---------------- -------------------------- %stddev %change %stddev \ | \ 24.09 ± 4% -62.7% 8.99 ± 3% lkp-wsx02/aim9/performance-300s-creat-clo 24.09 -62.7% 8.99 GEO-MEAN time.user_time f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 1571 ± 1% -13.5% 1359 ± 0% lkp-wsx02/aim9/performance-300s-creat-clo 1571 -13.5% 1359 GEO-MEAN vmstat.system.cs f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 276 ± 0% +5.5% 291 ± 0% lkp-wsx02/aim9/performance-300s-creat-clo 276 +5.5% 291 GEO-MEAN time.system_time f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- --------------------------- 430 ± 36% -36.0% 275 ± 39% lkp-wsx02/aim9/performance-300s-creat-clo 430 -36.0% 275 GEO-MEAN sched_debug.cpu#38.ttwu_count f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 52761811 ± 29% -54.7% 23885493 ± 11% lkp-wsx02/aim9/performance-300s-creat-clo 52761811 -54.7% 23885493 GEO-MEAN cpuidle.C1E-NHM.time f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 21939 ± 8% +42.6% 31283 ± 23% lkp-wsx02/aim9/performance-300s-creat-clo 21939 +42.6% 31283 GEO-MEAN numa-meminfo.node1.SUnreclaim f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 5484 ± 8% +40.3% 7694 ± 22% lkp-wsx02/aim9/performance-300s-creat-clo 5484 +40.3% 7694 GEO-MEAN numa-vmstat.node1.nr_slab_unreclaimable f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 34004 ± 2% +69.9% 57781 ± 4% lkp-wsx02/aim9/performance-300s-creat-clo 34004 +69.9% 57781 GEO-MEAN slabinfo.kmalloc-256.active_objs f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 1075 ± 2% +68.8% 1815 ± 4% lkp-wsx02/aim9/performance-300s-creat-clo 1074 +68.8% 1814 GEO-MEAN slabinfo.kmalloc-256.active_slabs f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 1075 ± 2% +68.8% 1815 ± 4% lkp-wsx02/aim9/performance-300s-creat-clo 1074 +68.8% 1814 GEO-MEAN slabinfo.kmalloc-256.num_slabs f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 34415 ± 2% +68.8% 58099 ± 4% lkp-wsx02/aim9/performance-300s-creat-clo 34415 +68.8% 58099 GEO-MEAN slabinfo.kmalloc-256.num_objs f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 32500 ± 7% +31.0% 42586 ± 17% lkp-wsx02/aim9/performance-300s-creat-clo 32500 +31.0% 42586 GEO-MEAN numa-meminfo.node1.Slab f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 73657 ± 1% -23.4% 56388 ± 3% lkp-wsx02/aim9/performance-300s-creat-clo 73657 -23.4% 56388 GEO-MEAN softirqs.RCU f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 10427 ± 14% -16.8% 8675 ± 6% lkp-wsx02/aim9/performance-300s-creat-clo 10427 -16.8% 8675 GEO-MEAN sched_debug.cfs_rq[0]:/.exec_clock f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 24242 ± 8% +18.4% 28696 ± 12% lkp-wsx02/aim9/performance-300s-creat-clo 24242 +18.4% 28696 GEO-MEAN numa-meminfo.node3.Active f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 2076 ± 7% +12.1% 2328 ± 2% lkp-wsx02/aim9/performance-300s-creat-clo 2076 +12.1% 2327 GEO-MEAN numa-meminfo.node2.KernelStack f2ee9bebb99dd4a4 f04e05b81e4d9ae88bee379f12 ---------------- -------------------------- 1139198 ± 5% -6.6% 1063912 ± 0% lkp-wsx02/aim9/performance-300s-creat-clo 1139198 -6.6% 1063912 GEO-MEAN proc-vmstat.pgfault lkp-wsx02: Westmere-EX Memory: 128G time.user_time 26 ++-----------------*----------------*--------*-------------------------+ | .*..*... .*..*. *..*..*.. .. .*. .*..*.. .*..*..| 24 *+ *. * *. *...*. .. * 22 ++ * | | | 20 ++ | 18 ++ | | | 16 ++ | 14 ++ | | | 12 ++ | 10 ++ | O O O O O O O O O O | 8 ++-O------O--O---------O--------O---O--O--O--O------------O------------+ time.system_time 294 ++--------------------------------------------------------------------+ 292 ++ O O O O O O O | O O O O O O O O O O O O O | 290 ++ | 288 ++ | | | 286 ++ | 284 ++ | 282 ++ | | | 280 ++ | 278 ++ *.. | *.. ..*.. .*.. *.. .*..*.. .. . .* 276 ++ *..*. *..*..*..*..*...*. .. *...*. *..* *..*. | 274 ++---------------------------------*----------------------------------+ softirqs.RCU 80000 ++------------------------------------------------------------------+ | | 75000 ++ ..*..*..*.. .*.. .*.. *.. | *..*..*..*.. .*. *..*.. .*..*. *.. .. .. *..*..* | *. *. * * | 70000 ++ | | | 65000 ++ | | | 60000 ++ O | | | | O O O O O O O | 55000 O+ O O O O O O O O O O O | | | 50000 ++------------------------------------------------------------------+ slabinfo.kmalloc-256.active_objs 70000 ++------------------------------------------------------------------+ | | 65000 ++ O O | 60000 ++ O O O O O O O O | O O O O O O O O | 55000 ++ | | O O | 50000 ++ | | | 45000 ++ | 40000 ++ | | .*.. | 35000 ++.*.. .*. *.. *..*.. .*.. .*..*... .*.. .*..* *. *. . .. *..*..*. *..*. *. *..*. | 30000 ++-----------------*------------------------------------------------+ slabinfo.kmalloc-256.num_objs 70000 ++------------------------------------------------------------------+ | O | 65000 ++ O | 60000 ++ O O O O O O O O O | O O O O O O O | 55000 ++ | | O O | 50000 ++ | | | 45000 ++ | 40000 ++ | | .*.. | 35000 *+.*.. .*. *... .*..*.. .*.. .*..*...*..*..*.. .*..* | *. *. *..*..*. *..*. *. | 30000 ++------------------------------------------------------------------+ slabinfo.kmalloc-256.active_slabs 2200 ++-------------------------------------------------------------------+ | O | 2000 ++ O O O | | O O O O O O O O O | 1800 O+ O O O O | | O | 1600 ++ O | | | 1400 ++ | | | 1200 ++ *...*.. | *..*.. .. *.. .*..*.. .*...*..*.. .*..*..*..*...*.. .*..* 1000 ++ * *. *. *..*. *. | | | 800 ++-------------------------------------------------------------------+ slabinfo.kmalloc-256.num_slabs 2200 ++-------------------------------------------------------------------+ | O | 2000 ++ O O O | | O O O O O O O O O | 1800 O+ O O O O | | O | 1600 ++ O | | | 1400 ++ | | | 1200 ++ *...*.. | *..*.. .. *.. .*..*.. .*...*..*.. .*..*..*..*...*.. .*..* 1000 ++ * *. *. *..*. *. | | | 800 ++-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample To reproduce: apt-get install ruby ruby-oj git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git cd lkp-tests bin/setup-local job.yaml # the job file attached in this email bin/run-local job.yaml Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance.