From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753982AbcHQIWH (ORCPT ); Wed, 17 Aug 2016 04:22:07 -0400 Received: from mail-wm0-f65.google.com ([74.125.82.65]:33709 "EHLO mail-wm0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753620AbcHQIWE (ORCPT ); Wed, 17 Aug 2016 04:22:04 -0400 Date: Wed, 17 Aug 2016 10:22:00 +0200 From: Michal Hocko To: Robert Foss Cc: sonnyrao@chromium.org, corbet@lwn.net, akpm@linux-foundation.org, vbabka@suse.cz, koct9i@gmail.com, hughd@google.com, n-horiguchi@ah.jp.nec.com, minchan@kernel.org, john.stultz@linaro.org, ross.zwisler@linux.intel.com, jmarchan@redhat.com, hannes@cmpxchg.org, keescook@chromium.org, viro@zeniv.linux.org.uk, gorcunov@openvz.org, plaguedbypenguins@gmail.com, rientjes@google.com, eric.engestrom@imgtec.com, jdanis@google.com, calvinowens@fb.com, adobriyan@gmail.com, jann@thejh.net, kirill.shutemov@linux.intel.com, ldufour@linux.vnet.ibm.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Ben Zhang , Bryan Freed , Filipe Brandenburger , Mateusz Guzik Subject: Re: [PACTH v2 0/3] Implement /proc//totmaps Message-ID: <20160817082200.GA10547@dhcp22.suse.cz> References: <1471039462-16771-1-git-send-email-robert.foss@collabora.com> <20160814090430.GB9248@dhcp22.suse.cz> <20160815134227.GE3360@dhcp22.suse.cz> <20160816071243.GB5001@dhcp22.suse.cz> <336532d0-57f2-a430-d195-13c13f70e25a@collabora.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <336532d0-57f2-a430-d195-13c13f70e25a@collabora.com> User-Agent: Mutt/1.6.0 (2016-04-01) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue 16-08-16 12:46:51, Robert Foss wrote: [...] > $ /usr/bin/time -v -p zsh -c "repeat 25 { awk '/^Rss/{rss+=\$2} > /^Pss/{pss+=\$2} END {printf \"rss:%d pss:%d\n\", rss, pss}\' > /proc/5025/smaps }" > [...] > Command being timed: "zsh -c repeat 25 { awk '/^Rss/{rss+=$2} > /^Pss/{pss+=$2} END {printf "rss:%d pss:%d\n", rss, pss}\' /proc/5025/smaps > }" > User time (seconds): 0.37 > System time (seconds): 0.45 > Percent of CPU this job got: 92% > Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.89 This is really unexpected. Where is the user time spent? Anyway, rather than measuring some random processes I've tried to measure something resembling the worst case. So I've created a simple program to mmap as much as possible: #include #include #include #include int main() { while (mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_ANON|MAP_SHARED|MAP_POPULATE, -1, 0) != MAP_FAILED) ; printf("pid:%d\n", getpid()); pause(); return 0; } so depending on /proc/sys/vm/max_map_count you will get the maximum possible mmaps. I am using a default so 65k mappings. Then I have retried your 25x file parsing: $ cat s.sh #!/bin/sh pid=$1 for i in $(seq 25) do awk '/^Rss/{rss+=$2} /^Pss/{pss+=$2} END {printf "rss:%d pss:%d\n", rss, pss}' /proc/$pid/smaps done But I am getting different results from you: $ awk '/^[0-9a-f]/{print}' /proc/14808/smaps | wc -l 65532 [...] Command being timed: "sh s.sh 14808" User time (seconds): 0.00 System time (seconds): 20.10 Percent of CPU this job got: 99% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:20.20 The results are stable when I try multiple times, in fact there shouldn't be any reason for them not to be. Then I went on to increase max_map_count to 250k and that behaves consistently: $ awk '/^[0-9a-f]/{print}' /proc/16093/smaps | wc -l 250002 [...] Command being timed: "sh s.sh 16093" User time (seconds): 0.00 System time (seconds): 77.93 Percent of CPU this job got: 98% Elapsed (wall clock) time (h:mm:ss or m:ss): 1:19.09 So with a reasonable user space the parsing is really not all that time consuming wrt. smaps handling. That being said I am still very skeptical about a dedicated proc file which accomplishes what userspace can done in a trivial way. -- Michal Hocko SUSE Labs