From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757598AbcLSOTw (ORCPT ); Mon, 19 Dec 2016 09:19:52 -0500 Received: from mail-wj0-f174.google.com ([209.85.210.174]:32948 "EHLO mail-wj0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754930AbcLSOTS (ORCPT ); Mon, 19 Dec 2016 09:19:18 -0500 To: Arnaldo Carvalho de Melo , Wang Nan , namhyung@kernel.org Cc: "linux-kernel@vger.kernel.org" From: Gioh Kim Subject: [RFC] Too low cache-misses ratio Message-ID: <4ff1c39e-2825-ff3d-35a0-299e18feb021@profitbricks.com> Date: Mon, 19 Dec 2016 15:19:14 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, Some of my AMD servers often reported "L3 ECC error and L2 cache error". I thought I could generated the errors if I set the machines on the heavy load. So I ran stress-ng tool with various options but system did not generate any error. I wanted to check the stress-ng tool really generate heavy load on physical memory, not only CPU-internal cache. So I checked the "cache-misses" event like following and found "cache-misses" ratio is very low (0.02%). I ran the same test many times with other vm-method options. But the result was the same. The total memory size I tested was almost 100G. And the "Page Faults Minor" showed that stress-ng loaded 100G memory (25,000,000 faults = 4K X 25,000,000 = 100G) So I'm sure the stress-ng tool really allocated 100G memory and did RW on the memory. I don't understand why the "cache-misses" ratio is so low. I thanks you for any comment. ------------------------------------- 8< ----------------------------------------------------- $ ./stress-ng --version stress-ng, version 0.07.04 $ free -m total used free shared buffers cached Mem: 128405 2455 125950 919 0 919 -/+ buffers/cache: 1535 126870 Swap: 0 0 0 $ cat /proc/cpuinfo | head processor : 0 vendor_id : AuthenticAMD cpu family : 21 model : 2 model name : AMD Opteron(tm) Processor 6386 SE stepping : 0 microcode : 0x600084f cpu MHz : 1400.000 cache size : 2048 KB physical id : 0 $ sudo ./stress-ng --perf --vm 64 --vm-bytes $(awk '/MemFree/{printf "%d\n", $2 * 0.80 / 64 ;}' < /proc/meminfo)k --vm-keep --vm-method all --timeout 300 stress-ng: info: [20968] dispatching hogs: 64 vm stress-ng: info: [20968] cache allocate: default cache size: 6144K stress-ng: info: [20968] successful run completed in 302.50s (5 mins, 2.50 secs) stress-ng: info: [20968] vm: stress-ng: info: [20968] 59.625.548.465.728 CPU Cycles 0,20 T/sec stress-ng: info: [20968] 30.106.848.579.392 Instructions 99,53 B/sec (0,505 instr. per cycle) stress-ng: info: [20968] 4.800.220.379.264 Cache References 15,87 B/sec stress-ng: info: [20968] 982.117.952 Cache Misses 3,25 M/sec ( 0,02%) stress-ng: info: [20968] 2.755.103.880.192 Stalled Cycles Frontend 9,11 B/sec stress-ng: info: [20968] 17.182.333.136.704 Stalled Cycles Backend 56,80 B/sec stress-ng: info: [20968] 4.817.817.144.960 Branch Instructions 15,93 B/sec stress-ng: info: [20968] 5.185.048.384 Branch Misses 17,14 M/sec ( 0,11%) stress-ng: info: [20968] 25.898.048 Page Faults Minor 85,61 K/sec stress-ng: info: [20968] 0 Page Faults Major 0,00 /sec stress-ng: info: [20968] 100.224 Context Switches 331,32 /sec stress-ng: info: [20968] 448 CPU Migrations 1,48 /sec stress-ng: info: [20968] 0 Alignment Faults 0,00 /sec stress-ng: info: [20968] 25.897.920 Page Faults User 85,61 K/sec stress-ng: info: [20968] 128 Page Faults Kernel 0,42 /sec stress-ng: info: [20968] 5.696 System Call Enter 18,83 /sec stress-ng: info: [20968] 5.696 System Call Exit 18,83 /sec stress-ng: info: [20968] 5.632 TLB Flushes 18,62 /sec stress-ng: info: [20968] 3.200 Kmalloc 10,58 /sec stress-ng: info: [20968] 0 Kmalloc Node 0,00 /sec stress-ng: info: [20968] 1.024 Kfree 3,39 /sec stress-ng: info: [20968] 420.800 Kmem Cache Alloc 1,39 K/sec stress-ng: info: [20968] 64 Kmem Cache Alloc Node 0,21 /sec stress-ng: info: [20968] 24.448 Kmem Cache Free 80,82 /sec stress-ng: info: [20968] 25.901.504 MM Page Alloc 85,63 K/sec stress-ng: info: [20968] 25.895.168 MM Page Free 85,60 K/sec stress-ng: info: [20968] 5.839.360 RCU Utilization 19,30 K/sec stress-ng: info: [20968] 1.280 Sched Migrate Task 4,23 /sec stress-ng: info: [20968] 0 Sched Move NUMA 0,00 /sec stress-ng: info: [20968] 100.480 Sched Wakeup 332,17 /sec stress-ng: info: [20968] 320 Signal Generate 1,06 /sec stress-ng: info: [20968] 128 Signal Deliver 0,42 /sec stress-ng: info: [20968] 0 IRQ Entry 0,00 /sec stress-ng: info: [20968] 0 IRQ Exit 0,00 /sec stress-ng: info: [20968] 2.970.624 Soft IRQ Entry 9,82 K/sec stress-ng: info: [20968] 2.970.624 Soft IRQ Exit 9,82 K/sec stress-ng: info: [20968] 20.928 Writeback Dirty Inode 69,18 /sec stress-ng: info: [20968] 0 Writeback Dirty Page 0,00 /sec $ sudo perf stat -e cache-misses,cache-references -a ./stress-ng --vm 64 --vm-bytes $(awk '/MemFree/{printf "%d\n", $2 * 0.80 / 64 ;}' < /proc/meminfo)k --vm-keep --vm-method all --timeout 300 stress-ng: info: [22150] dispatching hogs: 64 vm stress-ng: info: [22150] cache allocate: default cache size: 6144K stress-ng: info: [22150] successful run completed in 300.59s (5 mins, 0.59 secs) Performance counter stats for 'system wide': 1.044.067.793 cache-misses # 0,016 % of all cache refs (100,00%) 6.532.742.696.151 cache-references 300,648153378 seconds time elapsed