From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752227AbcHPQrH (ORCPT ); Tue, 16 Aug 2016 12:47:07 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:52734 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751222AbcHPQrF (ORCPT ); Tue, 16 Aug 2016 12:47:05 -0400 Subject: Re: [PACTH v2 0/3] Implement /proc//totmaps To: Michal Hocko , sonnyrao@chromium.org References: <1471039462-16771-1-git-send-email-robert.foss@collabora.com> <20160814090430.GB9248@dhcp22.suse.cz> <20160815134227.GE3360@dhcp22.suse.cz> <20160816071243.GB5001@dhcp22.suse.cz> Cc: corbet@lwn.net, akpm@linux-foundation.org, vbabka@suse.cz, koct9i@gmail.com, hughd@google.com, n-horiguchi@ah.jp.nec.com, minchan@kernel.org, john.stultz@linaro.org, ross.zwisler@linux.intel.com, jmarchan@redhat.com, hannes@cmpxchg.org, keescook@chromium.org, viro@zeniv.linux.org.uk, gorcunov@openvz.org, plaguedbypenguins@gmail.com, rientjes@google.com, eric.engestrom@imgtec.com, jdanis@google.com, calvinowens@fb.com, adobriyan@gmail.com, jann@thejh.net, kirill.shutemov@linux.intel.com, ldufour@linux.vnet.ibm.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Ben Zhang , Bryan Freed , Filipe Brandenburger , Mateusz Guzik From: Robert Foss Message-ID: <336532d0-57f2-a430-d195-13c13f70e25a@collabora.com> Date: Tue, 16 Aug 2016 12:46:51 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <20160816071243.GB5001@dhcp22.suse.cz> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2016-08-16 03:12 AM, Michal Hocko wrote: > On Mon 15-08-16 12:25:10, Robert Foss wrote: >> >> >> On 2016-08-15 09:42 AM, Michal Hocko wrote: > [...] >>> The use case is to speed up monitoring of >>> memory consumption in environments where RSS isn't precise. >>> >>> For example Chrome tends to many processes which have hundreds of VMAs >>> with a substantial amount of shared memory, and the error of using >>> RSS rather than PSS tends to be very large when looking at overall >>> memory consumption. PSS isn't kept as a single number that's exported >>> like RSS, so to calculate PSS means having to parse a very large smaps >>> file. >>> >>> This process is slow and has to be repeated for many processes, and we >>> found that the just act of doing the parsing was taking up a >>> significant amount of CPU time, so this patch is an attempt to make >>> that process cheaper. > > Well, this is slow because it requires the pte walk otherwise you cannot > know how many ptes map the particular shared page. Your patch > (totmaps_proc_show) does the very same page table walk because in fact > it is unavoidable. So what exactly is the difference except for the > userspace parsing which is quite trivial e.g. my currently running Firefox > has > $ awk '/^[0-9a-f]/{print}' /proc/4950/smaps | wc -l > 984 > > quite some VMAs, yet parsing it spends basically all the time in the kernel... > > $ /usr/bin/time -v awk '/^Rss/{rss+=$2} /^Pss/{pss+=$2} END {printf "rss:%d pss:%d\n", rss, pss}' /proc/4950/smaps > rss:1112288 pss:1096435 > Command being timed: "awk /^Rss/{rss+=$2} /^Pss/{pss+=$2} END {printf "rss:%d pss:%d\n", rss, pss} /proc/4950/smaps" > User time (seconds): 0.00 > System time (seconds): 0.02 > Percent of CPU this job got: 91% > Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.02 > > So I am not really sure I see the performance benefit. > I did some performance measurements of my own, and it would seem like there is about a 2x performance gain to be had. To me that is substantial, and a larger gain than commonly seen. There naturally also the benefit that this is a lot easier to interact with programmatically. $ ps aux | grep firefox robertfoss 5025 24.3 13.7 3562820 2219616 ? Rl Aug15 277:44 /usr/lib/firefox/firefox https://allg.one/xpb $ awk '/^[0-9a-f]/{print}' /proc/5025/smaps | wc -l 1503 $ /usr/bin/time -v -p zsh -c "(repeat 25 {cat /proc/5025/totmaps})" [...] Command being timed: "zsh -c (repeat 25 {cat /proc/5025/totmaps})" User time (seconds): 0.00 System time (seconds): 0.40 Percent of CPU this job got: 90% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.45 $ /usr/bin/time -v -p zsh -c "repeat 25 { awk '/^Rss/{rss+=\$2} /^Pss/{pss+=\$2} END {printf \"rss:%d pss:%d\n\", rss, pss}\' /proc/5025/smaps }" [...] Command being timed: "zsh -c repeat 25 { awk '/^Rss/{rss+=$2} /^Pss/{pss+=$2} END {printf "rss:%d pss:%d\n", rss, pss}\' /proc/5025/smaps }" User time (seconds): 0.37 System time (seconds): 0.45 Percent of CPU this job got: 92% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.89