From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755947AbbDNRU6 (ORCPT ); Tue, 14 Apr 2015 13:20:58 -0400 Received: from mail-wg0-f46.google.com ([74.125.82.46]:35862 "EHLO mail-wg0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755756AbbDNRUv (ORCPT ); Tue, 14 Apr 2015 13:20:51 -0400 Date: Tue, 14 Apr 2015 20:20:47 +0300 From: Alexey Dobriyan To: mmarek@suse.cz, akpm@linux-foundation.org Cc: linux-kernel@vger.kernel.org Subject: [PATCH] tags: much faster, parallel "make tags" Message-ID: <20150414172047.GA5641@p183.telecom.by> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org ctags is single-threaded program. Split list of files to be tagged into equal parts, 1 part for each CPU and then merge the results. Speedup on one 2-way box I have is ~143 s => ~99 s (-31%). On another 4-way box: ~120 s => ~65 s (-46%!). Resulting "tags" files aren't byte-for-byte identical because ctags program numbers anon struct and enum declarations with "__anonNNN" symbols. If those lines are removed, "tags" file becomes byte-for-byte identical with those generated with current code. Signed-off-by: Alexey Dobriyan --- scripts/tags.sh | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) --- a/scripts/tags.sh +++ b/scripts/tags.sh @@ -152,7 +152,24 @@ dogtags() exuberant() { - all_target_sources | xargs $1 -a \ + NR_CPUS=1 + if [ -e /proc/cpuinfo ]; then + NR_CPUS=$(grep -e '^processor : ' /proc/cpuinfo | wc -l) + fi + + rm -f .make-tags.src.* .make-tags.* + + all_target_sources >.make-tags.src + # seems like Useless Use of cat(1) but not really + NR_LINES=$(cat .make-tags.src | wc -l) + NR_LINES=$((($NR_LINES + $NR_CPUS - 1) / $NR_CPUS)) + + split -a 6 -d -l $NR_LINES .make-tags.src .make-tags.src. + + for i in .make-tags.src.*; do + N=$(echo $i | sed -e 's/.*\.//') + # -u: don't sort now, sort later + cat $i | xargs $1 -a -f .make-tags.$N -u \ -I __initdata,__exitdata,__initconst, \ -I __cpuinitdata,__initdata_memblock \ -I __refdata,__attribute,__maybe_unused,__always_unused \ @@ -211,7 +228,20 @@ exuberant() --regex-c='/DEFINE_PCI_DEVICE_TABLE\((\w*)/\1/v/' \ --regex-c='/(^\s)OFFSET\((\w*)/\2/v/' \ --regex-c='/(^\s)DEFINE\((\w*)/\2/v/' \ - --regex-c='/DEFINE_HASHTABLE\((\w*)/\1/v/' + --regex-c='/DEFINE_HASHTABLE\((\w*)/\1/v/' \ + & + done + wait + rm -f .make-tags.src .make-tags.src.* + + # write header + $1 -f tags /dev/null + # remove header + for i in .make-tags.*; do + sed -i -e '/^!/d' $i + done + sort .make-tags.* >>tags + rm -f .make-tags.* all_kconfigs | xargs $1 -a \ --langdef=kconfig --language-force=kconfig \