From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754315AbcDSRTw (ORCPT ); Tue, 19 Apr 2016 13:19:52 -0400 Received: from smtprelay0020.hostedemail.com ([216.40.44.20]:54982 "EHLO smtprelay.hostedemail.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752725AbcDSRTu (ORCPT ); Tue, 19 Apr 2016 13:19:50 -0400 X-Greylist: delayed 383 seconds by postgrey-1.27 at vger.kernel.org; Tue, 19 Apr 2016 13:19:50 EDT X-Session-Marker: 726F737465647440676F6F646D69732E6F7267 X-Spam-Summary: 2,0,0,,d41d8cd98f00b204,rostedt@goodmis.org,:::::::::::::::::::,RULES_HIT:41:355:379:541:599:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1437:1515:1516:1518:1534:1542:1593:1594:1711:1730:1747:1777:1792:2198:2199:2393:2553:2559:2562:2693:2731:3138:3139:3140:3141:3142:3355:3622:3865:3866:3867:3868:3870:3871:3872:3873:3874:4250:5007:6119:6261:7576:7875:7903:8603:9010:10004:10400:10848:10967:11026:11232:11658:11914:12043:12296:12517:12519:12555:12740:13141:13146:13161:13229:13230:13255:13439:14093:14096:14097:14181:14659:14721:21063:21080:30012:30034:30054:30090:30091,0,RBL:none,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: pin37_3881eea573d3d X-Filterd-Recvd-Size: 3635 Date: Tue, 19 Apr 2016 13:19:47 -0400 From: Steven Rostedt To: Mathieu Desnoyers Cc: linux-kernel@vger.kernel.org, Ingo Molnar , Andrew Morton , "H. Peter Anvin" , Thomas Gleixner , Jiri Olsa , Masami Hiramatsu , Namhyung Kim , linux-trace-users@vger.kernel.org Subject: Re: [RFC][PATCH 2/4] tracing: Use pid bitmap instead of a pid array for set_event_pid Message-ID: <20160419131947.3c5208b4@gandalf.local.home> In-Reply-To: <1694657549.62933.1461084928341.JavaMail.zimbra@efficios.com> References: <20160419143421.829909157@goodmis.org> <20160419143725.295928551@goodmis.org> <1694657549.62933.1461084928341.JavaMail.zimbra@efficios.com> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 19 Apr 2016 16:55:28 +0000 (UTC) Mathieu Desnoyers wrote: > ----- On Apr 19, 2016, at 10:34 AM, rostedt rostedt@goodmis.org wrote: > > > From: Steven Rostedt > > > > In order to add the ability to let tasks that are filtered by the events > > have their children also be traced on fork (and then not traced on exit), > > convert the array into a pid bitmask. Most of the time the number of pids is > > only 32768 pids or a 4k bitmask, which is the same size as the default list > > currently is, and that list could grow if more pids are listed. > > > > This also greatly simplifies the code. > > The maximum PID number can be increased with sysctl. > > See "pid_max" in Documentation/sysctl/kernel.txt > > What happens when you have a very large pid_max set ? I discussed this with HPA, and it appears that the pid_max max would require a bitmap of about 1/2 meg (the current default is 8k). This is also why I chose to keep the bitmap as vmalloc and not a continuous page allocation. > > You say "most of the time" as if this was a fast-path vs a slow-path, > but it is not the case here. I meant "most of the time" as "default". Yes, you can make the pid_max really big, but in that case you better have enough memory in your system to handle that many threads. Thus a 1/2 meg used for tracking pids shouldn't be an issue. > > This is a configuration option that can significantly hurt memory usage > in configurations using a large pid_max. No, it is created dynamically. If you never write anything into the set_event_pid file, then you have nothing to worry about, as nothing is allocated. It creates the array when a pid is added to the file, and only then. If it fails to allocate, the write will return -ENOMEM as the errno. Again, if you have a large pid_max your box had better have a lot of memory to begin with, because this array will be negligible compared to the memory required to handle large number of tasks. > > FWIW, I implement a similar feature with a hash table in lttng-modules. > I don't have the child process tracking though, which is a neat improvement. I originally had a complex hash algorithm because I too was worried about the size of pid_max and using a bitmap, but HPA convinced me it was the way to go. -- Steve