From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01DA4C43441 for ; Sun, 25 Nov 2018 20:48:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B6FB22082E for ; Sun, 25 Nov 2018 20:48:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B6FB22082E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=gmx.us Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726584AbeKZHkY (ORCPT ); Mon, 26 Nov 2018 02:40:24 -0500 Received: from mout.gmx.net ([212.227.15.15]:48859 "EHLO mout.gmx.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725863AbeKZHkY (ORCPT ); Mon, 26 Nov 2018 02:40:24 -0500 Received: from ovpn-120-100.rdu2.redhat.com ([98.118.28.103]) by mail.gmx.com (mrgmx001 [212.227.17.184]) with ESMTPSA (Nemesis) id 0M5Lqx-1fU7JQ1Kpz-00zYgR; Sun, 25 Nov 2018 21:48:15 +0100 Subject: Re: [PATCH v4] debugobjects: scale the static pool size To: Thomas Gleixner Cc: akpm@linux-foundation.org, longman@redhat.com, yang.shi@linux.alibaba.com, arnd@arndb.de, linux-kernel@vger.kernel.org References: <20181120232810.2503-1-cai@gmx.us> <20181121021157.3061-1-cai@gmx.us> From: Qian Cai Message-ID: <245eeb16-e953-9375-5c54-4b9396959340@gmx.us> Date: Sun, 25 Nov 2018 15:48:13 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.3.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K1:SEL+UXnHw2VDWwDEgB0D7BcMlGccy9qfMkVjMzI6IVVdvOd28XC uP8ST2Wq1FCBKgA9M+1qwY1CriHXjl4rk8LszTrF6JOuPN+QmnyW6vyG8dkQD/vw8rNhSMV +aVw87R+fc0Z6cA4c3BZLmgR2TJ76R+8wLh5ZSupSeKDGk0WaVPyUfqTxwryqnyTTCeOmzA 4RJEXs0eHsFbEESUHUvOQ== X-UI-Out-Filterresults: notjunk:1;V03:K0:Lwth3Ax1+1s=:XzQHit2dqoV3pl0cAtjrCN J2aMqSRjrg2I9BrkLTqm1v0lMLHVK71QtsZT3s1zSonORj3ZoE9PiN2I8r0voCl0L7upsdM1C lxm4hpsr4zeNlVrz7RP5+CLV9nHb4Me0OMnhnqcnEbCYxYQ1coua2T4aPuHcuoEoEkv0ogqwv oLsEXYSyBGChO/dIJYSS5a2biNNT9HZWoSdpRVWitaCke2lUxL3m3G0KsO0ZGWpxH5tfw6mOF kyNcltZuukGUqb9CAlQwKSt5uLZ4562mi2xGeW3jtfzdF0YCSh3QbXtHbAWzaet6HnjUh0pJk 6G40ksUV2WQpBOMOYBb8rVTkW/Kjer3Qf1dzrpfB99d2PWM226/F2tEfCFnyjnjUMUAsFQq3x +4ZuizVsIw8YBpgzlTKVAJOFZTABeyrjwNkc0ANY54DtsnZ9g1PbNaRKCkId6Wq+VEHRw56kN 08kYG3Jy09998M8YYL7+KcHjdZb8V6FR+mnUHECuqSyPjQ5NH1L9RpZG3LjvW2Bfqg7G+R0iP zkBbd9U3CAYtCYO8w2d6kfaCpSTKSar/RaFxS6JNz50bZ/deZUlKMxwl+wPvM1V4iQ7Ialtif 1BOyH2xN3DTQcF+RDCqhc3W2HnxcchKx1fys6ZkJY1mi6mfSLjO6upTp3NwCiO6fhxOcbg8ZF nY2ljbcXz29E1qcwWE5dF++XITxT+bwNHR8ehccslJuID6m34qH0u7OMoT09Asoy++6tWW4uG 9BcEY4iqy+LRaRT0ENB64g53eBGV2b7GkmEUdLYTSjcVgYxo8y+0KQ09Z+A8JjBSLDii+yuj+ K5ZvNHysfow2pREjn6SW4J5bSMULztKASNw7N6/Bo1YcG4XlXpGnAXUGnDmpxyIZtnfK/M906 0DjAUrLSzLt2rMSl7nj3V09kZSxFhQQecT0ZC5OlY= Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 11/22/18 4:56 PM, Thomas Gleixner wrote: > On Tue, 20 Nov 2018, Qian Cai wrote: > > Looking deeper at that. > >> diff --git a/lib/debugobjects.c b/lib/debugobjects.c >> index 70935ed91125..140571aa483c 100644 >> --- a/lib/debugobjects.c >> +++ b/lib/debugobjects.c >> @@ -23,9 +23,81 @@ >> #define ODEBUG_HASH_BITS 14 >> #define ODEBUG_HASH_SIZE (1 << ODEBUG_HASH_BITS) >> >> -#define ODEBUG_POOL_SIZE 1024 >> +#define ODEBUG_DEFAULT_POOL 512 >> #define ODEBUG_POOL_MIN_LEVEL 256 >> >> +/* >> + * Some debug objects are allocated during the early boot. Enabling some options >> + * like timers or workqueue objects may increase the size required significantly >> + * with large number of CPUs. For example (as today, 20 Nov. 2018), >> + * >> + * No. CPUs x 2 (worker pool) objects: >> + * >> + * start_kernel >> + * workqueue_init_early >> + * init_worker_pool >> + * init_timer_key >> + * debug_object_init >> + * >> + * No. CPUs objects (CONFIG_HIGH_RES_TIMERS): >> + * >> + * sched_init >> + * hrtick_rq_init >> + * hrtimer_init >> + * >> + * CONFIG_DEBUG_OBJECTS_WORK: >> + * No. CPUs x 6 (workqueue) objects: >> + * >> + * workqueue_init_early >> + * alloc_workqueue >> + * __alloc_workqueue_key >> + * alloc_and_link_pwqs >> + * init_pwq >> + * >> + * Also, plus No. CPUs objects: >> + * >> + * perf_event_init >> + * __init_srcu_struct >> + * init_srcu_struct_fields >> + * init_srcu_struct_nodes >> + * __init_work > > None of the things are actually used or required _BEFORE_ > debug_objects_mem_init() is invoked. > > The reason why the call is at this place in start_kernel() is > historical. It's because back in the days when debugobjects were added the > memory allocator was enabled way later than today. So we can just move the > debug_objects_mem_init() call right before sched_init() I think. > >> + * Increase the number a bit more in case the implmentatins are changed in the >> + * future, as it is better to avoid OOM than spending a bit more kernel memory >> + * that will/can be freed. >> + * >> + * With all debug objects config options selected except the workqueue and the >> + * timers, kernel reports, >> + * 64-CPU: ODEBUG: 4 of 4 active objects replaced >> + * 256-CPU: ODEBUG: 4 of 4 active objects replaced >> + * >> + * all the options except the workqueue: >> + * 64-CPU: ODEBUG: 466 of 466 active objects replaced >> + * 256-CPU: ODEBUG: 1810 of 1810 active objects replaced >> + * >> + * all the options except the timers: >> + * 64-CPU: ODEBUG: 652 of 652 active objects replaced >> + * 256-CPU: ODEBUG: 2572 of 2572 active objects replaced >> + * >> + * all the options: >> + * 64-CPU: ODEBUG: 1114 of 1114 active objects replaced >> + * 256-CPU: ODEBUG: 4378 of 4378 active objects replaced >> + */ >> +#ifdef CONFIG_DEBUG_OBJECTS_WORK >> +#define ODEBUG_WORK_PCPU_CNT 10 >> +#else >> +#define ODEBUG_WORK_PCPU_CNT 0 >> +#endif /* CONFIG_DEBUG_OBJECTS_WORK */ >> + >> +#ifdef CONFIG_DEBUG_OBJECTS_TIMERS >> +#define ODEBUG_TIMERS_PCPU_CNT 10 >> +#else >> +#define ODEBUG_TIMERS_PCPU_CNT 0 >> +#endif /* CONFIG_DEBUG_OBJECTS_TIMERS */ > > Btw, the scaling here is way off. > >> +#define ODEBUG_POOL_SIZE (ODEBUG_DEFAULT_POOL + CONFIG_NR_CPUS * \ >> + (ODEBUG_TIMERS_PCPU_CNT + ODEBUG_WORK_PCPU_CNT)) > > CONFIG_NR_CPUS Pool size Storage size > > 256 256512 4M > > According to your list above it uses 4378 object for 256 CPUs. That's off > by an factor of ~58. > Hmm, it is not clear to me why this is off and where did the pool size 256512 come from. CONFIG_NR_CPUS: 256 Pool size: 1024 + 256 * (10 + 10) = 6144 Am I missing anything?