From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1757231AbbJIKFk (ORCPT <rfc822;w@1wt.eu>);
	Fri, 9 Oct 2015 06:05:40 -0400
Received: from out4133-146.mail.aliyun.com ([42.120.133.146]:40690 "EHLO
	out4133-146.mail.aliyun.com" rhost-flags-OK-OK-OK-OK)
	by vger.kernel.org with ESMTP id S1753062AbbJIKFj (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Fri, 9 Oct 2015 06:05:39 -0400
X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e02c03273;MF=hillf.zj@alibaba-inc.com;NM=1;PH=DS;RN=3;SR=0;
Reply-To: "Hillf Danton" <hillf.zj@alibaba-inc.com>
From: "Hillf Danton" <hillf.zj@alibaba-inc.com>
To: "'Xunlei Pang'" <pang.xunlei@linaro.org>
Cc: "linux-kernel" <linux-kernel@vger.kernel.org>, "Tejun Heo" <tj@kernel.org>
References: <065801d10279$2b3fe0c0$81bfa240$@alibaba-inc.com>
In-Reply-To: <065801d10279$2b3fe0c0$81bfa240$@alibaba-inc.com>
Subject: Re: [PATCH] workqueue: Allocate the unbound pool using local node memory
Date: Fri, 09 Oct 2015 18:05:20 +0800
Message-ID: <065901d1027a$01d3f9f0$057bedd0$@alibaba-inc.com>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Mailer: Microsoft Outlook 14.0
Thread-Index: AQKbpYjPzgCp3gBuW//i1la9oQdPtJzNp0pQ
Content-Language: zh-cn
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Xunlei Pang <pang.xunlei@linaro.org>
> 
> Currently, get_unbound_pool() uses kzalloc() to allocate the
> worker pool. Actually, we can use the right node to do the
> allocation, achieving local memory access.
> 
> This patch selects target node first, and uses kzalloc_node()
> instead.
> 
> Signed-off-by: Xunlei Pang <pang.xunlei@linaro.org>
> ---
>  kernel/workqueue.c | 26 ++++++++++++++------------
>  1 file changed, 14 insertions(+), 12 deletions(-)
> 
> diff --git a/kernel/workqueue.c b/kernel/workqueue.c
> index ca71582..96d3747 100644
> --- a/kernel/workqueue.c
> +++ b/kernel/workqueue.c
> @@ -3199,6 +3199,7 @@ static struct worker_pool *get_unbound_pool(const struct workqueue_attrs *attrs)
>  	u32 hash = wqattrs_hash(attrs);
>  	struct worker_pool *pool;
>  	int node;
> +	int target_node = NUMA_NO_NODE;
> 
>  	lockdep_assert_held(&wq_pool_mutex);
> 
> @@ -3210,13 +3211,25 @@ static struct worker_pool *get_unbound_pool(const struct workqueue_attrs *attrs)
>  		}
>  	}
> 
> +	/* if cpumask is contained inside a NUMA node, we belong to that node */
> +	if (wq_numa_enabled) {
> +		for_each_node(node) {
> +			if (cpumask_subset(attrs->cpumask,
> +					   wq_numa_possible_cpumask[node])) {
> +				target_node = node;
> +				break;
> +			}
> +		}
> +	}
> +
>  	/* nope, create a new one */
> -	pool = kzalloc(sizeof(*pool), GFP_KERNEL);
> +	pool = kzalloc_node(sizeof(*pool), GFP_KERNEL, target_node);

What if target_node is short of pages at the moment?

>  	if (!pool || init_worker_pool(pool) < 0)
>  		goto fail;
> 
>  	lockdep_set_subclass(&pool->lock, 1);	/* see put_pwq() */
>  	copy_workqueue_attrs(pool->attrs, attrs);
> +	pool->node = target_node;
> 
>  	/*
>  	 * no_numa isn't a worker_pool attribute, always clear it.  See
> @@ -3224,17 +3237,6 @@ static struct worker_pool *get_unbound_pool(const struct workqueue_attrs *attrs)
>  	 */
>  	pool->attrs->no_numa = false;
> 
> -	/* if cpumask is contained inside a NUMA node, we belong to that node */
> -	if (wq_numa_enabled) {
> -		for_each_node(node) {
> -			if (cpumask_subset(pool->attrs->cpumask,
> -					   wq_numa_possible_cpumask[node])) {
> -				pool->node = node;
> -				break;
> -			}
> -		}
> -	}
> -
>  	if (worker_pool_assign_id(pool) < 0)
>  		goto fail;
> 
> --
> 1.9.1