From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 407822BAF9 for ; Thu, 5 Sep 2024 03:45:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.188 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725507913; cv=none; b=m4nDCp/2LGqS4zwNZcOJuBhQxkKfkYJ3RPnFjPp6EzkPV13Axr3pyeWcjuTEyJGEpN6yW9XDomu/mE8Seu64xoHlkuXSt8YkurvkJ20epHul3yP1+xSZ5mtqIDy/zzcRZ5DG6g5TUyUGdCcsUiGh7PMMiWaCESvkP69BoI5ELYA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725507913; c=relaxed/simple; bh=C/qlmZ6+W5XHlRHlajuOu1MxPgi1tcEmYOGBIRSBdkY=; h=Subject:From:To:References:Message-ID:Date:MIME-Version: In-Reply-To:Content-Type; b=gZoqB6yjR/TezCYY0NBcf8B/UvkH0Y888/QqzAR5bWiDHS0LkWDLYxwDfHXth/jDijF8v30beYOz2h8SN7pTw3l9nQPRUT4UN4Dd8yVgVUHJjnK+Je7pxRvyLblqA3bLd/JdZN78zTzjA5bxuvPbYX2ovesjGteKwFTczUfFqD8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; arc=none smtp.client-ip=45.249.212.188 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Received: from mail.maildlp.com (unknown [172.19.162.254]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4WzlZ82zZkzpSvv; Thu, 5 Sep 2024 11:43:12 +0800 (CST) Received: from dggpemf100006.china.huawei.com (unknown [7.185.36.228]) by mail.maildlp.com (Postfix) with ESMTPS id 279DD180100; Thu, 5 Sep 2024 11:45:06 +0800 (CST) Received: from [10.174.178.55] (10.174.178.55) by dggpemf100006.china.huawei.com (7.185.36.228) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Thu, 5 Sep 2024 11:45:05 +0800 Subject: Re: [PATCH v2 4/6] debugobjects: Don't start fill if there are remaining nodes locally From: "Leizhen (ThunderTown)" To: Andrew Morton , Thomas Gleixner , References: <20240904133944.2124-1-thunder.leizhen@huawei.com> <20240904133944.2124-5-thunder.leizhen@huawei.com> <336109d9-2eea-4d67-ee22-ed218b9504c3@huawei.com> Message-ID: <7613ce35-0c65-341f-c6ed-412de79890e6@huawei.com> Date: Thu, 5 Sep 2024 11:45:05 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In-Reply-To: <336109d9-2eea-4d67-ee22-ed218b9504c3@huawei.com> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemf100006.china.huawei.com (7.185.36.228) On 2024/9/5 11:11, Leizhen (ThunderTown) wrote: > > > On 2024/9/4 21:39, Zhen Lei wrote: >> If the conditions for starting fill are met, it means that all cores that >> call fill() later are blocked until the first core completes the fill >> operation. But obviously, for a core that has free nodes locally, it does >> not need to be blocked(see below for why). This is good in stress >> situations. >> >> 1. In the case of no nesting, a core uses only one node at a time. As long >> as there is a local node, there is no need to use the free node in >> obj_pool. >> 2. In the case of nesting depth is one, nodes in obj_pool need to be used >> only when there is only one local node. >> #define ODEBUG_POOL_PERCPU_SIZE 64 >> #define ODEBUG_BATCH_SIZE 16 >> Assume that when nested, the probability of percpu_obj_pool having each >> number of nodes is the same. The probability of only one node is less >> than 1/17=6%. Assuming the probability of nesting is 5%, that's a >> pretty high estimate. Then the probability of using obj_pool is >> 6% * 5% = 0.3%. In other words, a 333-core environment produces only >> one core to compete for obj_pool. >> #define ODEBUG_POOL_MIN_LEVEL 256 >> #define ODEBUG_BATCH_SIZE 16 >> But we can tolerate "256 / (16 + 1)" = 15 cores competing at the same >> time. > > One detail is omitted. In function debug_objects_mem_init(), an extra batch > is reserved for each core. > extras = num_possible_cpus() * ODEBUG_BATCH_SIZE; > debug_objects_pool_min_level += extras; > > In addition, above method of calculating probabilities is wrong. The correct > calculation method is as follows: > When the number of local nodes is 0, fill is performed. When the number of > local nodes is 1 and nested, 16 nodes are moved from obj_pool to obj_pool. > As a result, the obj_pool resource pool keeps decreasing. When this happens > continuously(The number of local nodes equal 0 is not met), the resource > pool will eventually be exhausted. The error probability is: > (1/2)^((256+16^ncpus)/17) * (5% + 5%^2 + ... + 5%^N) * 2/17 < 1e-7 (ncpus=1). Should be: ==> (1/2)^((256+16^ncpus)/17) * 5% * 2/17 < 9e-8 (ncpus=1). > 1/2 ==> denominator sequence: 0,1; numerator sequence: 1 > (5% + 5%^2 + ... + 5%^N) < 5% + (5%^2) * 2 = 0.055 > 17 = ODEBUG_BATCH_SIZ + 1, amount moved from obj_pool when the number of local nodes is 0. > 2/17 ==> denominator sequence: 0-16; numerator sequence: 0,1 > The more cores, the lower the probability of exhaustion. > > If obj_pool is not filled only when there are more than two local nodes, > the probability of exhaustion is: > (1/3)^((256+16^ncpus)/17) * (5% + 5%^2 + ... + 5%^N) * 3/17 < < 2.3e-10 Should be: ==> (1/3)^((256+16^ncpus)/17) * (5%^2) * 3/17 < 1.03e-11 (ncpus=1). > 1/3 ==> denominator sequence: 0,1,2; numerator sequence: 2 > 3/17 ==> denominator sequence: 0-16; numerator sequence: 0,1,2 Hi, Thomas Gleixner: Seems to need to add an additional patch as follows to be foolproof. I'll prepare it. diff --git a/lib/debugobjects.c b/lib/debugobjects.c index e175cc74f7b7899..d3f8cc7dc1c9291 100644 --- a/lib/debugobjects.c +++ b/lib/debugobjects.c @@ -245,6 +245,21 @@ alloc_object(void *addr, struct debug_bucket *b, const struct debug_obj_descr *d raw_spin_lock(&pool_lock); obj = __alloc_object(&obj_pool); + if (!obj) { + raw_spin_unlock(&pool_lock); + obj = kmem_cache_zalloc(obj_cache, __GFP_HIGH | GFP_NOWAIT); + if (!obj) + return NULL; + + raw_spin_lock(&pool_lock); + debug_objects_allocated++; + + /* + * It can be understood that obj is allocated immediately after + * being added to obj_pool. + */ + obj_pool_used++; + } if (obj) { int cnt = 0; > >> 3. In the case of nesting depth more than one, the probability is lower >> and negligible. >> Nesting Depth=2: "2/17 * 5% * 5%" = 0.03% >> Nesting Depth=3: "3/17 * 5% * 5% * 5%" = 0.002% >> >> However, to ensure sufficient reliability, obj_pool is not filled only >> when there are more than two local nodes, reduce the probability of >> problems to the impossible. >> >> Signed-off-by: Zhen Lei >> --- >> lib/debugobjects.c | 10 ++++++++++ >> 1 file changed, 10 insertions(+) >> >> diff --git a/lib/debugobjects.c b/lib/debugobjects.c >> index 7a8ccc94cb037ba..4f64b5d4329c27d 100644 >> --- a/lib/debugobjects.c >> +++ b/lib/debugobjects.c >> @@ -131,6 +131,16 @@ static void fill_pool(void) >> struct debug_obj *obj; >> unsigned long flags; >> >> + /* >> + * The upper-layer function uses only one node at a time. If there are >> + * more than two local nodes, it means that even if nesting occurs, it >> + * doesn't matter. The probability of nesting depth >= 2 is extremely >> + * low, and the number of global free nodes guarded by >> + * debug_objects_pool_min_level is adequate. >> + */ >> + if (likely(obj_cache) && this_cpu_read(percpu_obj_pool.obj_free) >= 2) >> + return; >> + >> if (likely(READ_ONCE(obj_pool_free) >= debug_objects_pool_min_level)) >> return; >> >> > -- Regards, Zhen Lei