From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Cyrus-Session-Id: sloti22d1t05-288557-1516221206-2-2342097695124033363 X-Sieve: CMU Sieve 3.0 X-Spam-known-sender: no X-Spam-score: 0.0 X-Spam-hits: BAYES_00 -1.9, HEADER_FROM_DIFFERENT_DOMAINS 0.25, RCVD_IN_DNSWL_HI -5, T_RP_MATCHES_RCVD -0.01, LANGUAGES unknown, BAYES_USED global, SA_VERSION 3.4.0 X-Spam-source: IP='209.132.180.67', Host='vger.kernel.org', Country='US', FromHeader='org', MailFrom='org' X-Spam-charsets: X-Resolved-to: greg@kroah.com X-Delivered-to: greg@kroah.com X-Mail-from: linux-usb-owner@vger.kernel.org ARC-Seal: i=1; a=rsa-sha256; cv=none; d=messagingengine.com; s=arctest; t=1516221205; b=VSfFG5FuesGmaWEbj80uAZMhjAAtRQusgNdrSvUWVSQujfP fpil0wNzzZhbLRcopsh49HtjYPAm+BeuQqWqA/78W0Ug074OPyW/d7aVB6xqmYe7 6O9fH+fs+kdPgcjocZ0PttyCRekhhyqbpO1wQBnnUwD8PRG+lIg3A3E2y9lvhh/W zTE6+WJQv7J/YqhJRLQ9Mi+DkrOVSHnIZSascTOe/vmV/SENVM2tInMP1VzfX0b2 maua2jq8WReb+zFW0YSWJJPepUH214E7LPm1aNlT5iDon1vkK0LqXgn3YHoH32Iy daQU3ZMJ7+0cKbaPKprveCFOu3njsGXwMkD2RDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=from:to:cc:subject:date:message-id :in-reply-to:references:sender:list-id; s=arctest; t=1516221205; bh=ljkYQIjazERIyFIAypNUNvZpA5b0al+FZmaYP3DhFqs=; b=ft3Mq2rew3ce JESvOpAHoU2yVyp7G+1DGfPDvmz0wM+EzhyUSHHv+HncuTEh9AnsMq/JnZQJnAaM 8nnwc9m6BRi2YIyhZWvGIIpgyVEx9w+PRrig8pX6HecTdX7MEy2YiNrx8BXg6IVf v92vTJyTw/2Ko06rVHhkyqBYvd7B3E+gVo7hM5ox8wyV1j+VvwPkpgkPbOt6YW6i 401Jrzar4DmTfWsAviAFnxZHzGifOXevTKQtSoxclw16B5jZ0q6Onu+JHUOHLyuq 8hEZQVPBTnSkV6tYyTZ0YxvD+ZXDl5DsLsgXyDb+7w2W1kCRCQJDY73i7InR4Gz0 se5vDvGL8A== ARC-Authentication-Results: i=1; mx4.messagingengine.com; arc=none (no signatures found); dkim=fail (message has been altered; 2048-bit rsa key sha256) header.d=infradead.org header.i=@infradead.org header.b=SuKKTqmN x-bits=2048 x-keytype=rsa x-algorithm=sha256 x-selector=bombadil.20170209; dmarc=none (p=none,has-list-id=yes,d=none) header.from=infradead.org; iprev=pass policy.iprev=209.132.180.67 (vger.kernel.org); spf=none smtp.mailfrom=linux-usb-owner@vger.kernel.org smtp.helo=vger.kernel.org; x-aligned-from=fail; x-ptr=pass x-ptr-helo=vger.kernel.org x-ptr-lookup=vger.kernel.org; x-return-mx=pass smtp.domain=vger.kernel.org smtp.result=pass smtp_org.domain=kernel.org smtp_org.result=pass smtp_is_org_domain=no header.domain=infradead.org header.result=pass header_is_org_domain=yes Authentication-Results: mx4.messagingengine.com; arc=none (no signatures found); dkim=fail (message has been altered; 2048-bit rsa key sha256) header.d=infradead.org header.i=@infradead.org header.b=SuKKTqmN x-bits=2048 x-keytype=rsa x-algorithm=sha256 x-selector=bombadil.20170209; dmarc=none (p=none,has-list-id=yes,d=none) header.from=infradead.org; iprev=pass policy.iprev=209.132.180.67 (vger.kernel.org); spf=none smtp.mailfrom=linux-usb-owner@vger.kernel.org smtp.helo=vger.kernel.org; x-aligned-from=fail; x-ptr=pass x-ptr-helo=vger.kernel.org x-ptr-lookup=vger.kernel.org; x-return-mx=pass smtp.domain=vger.kernel.org smtp.result=pass smtp_org.domain=kernel.org smtp_org.result=pass smtp_is_org_domain=no header.domain=infradead.org header.result=pass header_is_org_domain=yes Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754563AbeAQUdE (ORCPT ); Wed, 17 Jan 2018 15:33:04 -0500 Received: from bombadil.infradead.org ([65.50.211.133]:43825 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753943AbeAQUW7 (ORCPT ); Wed, 17 Jan 2018 15:22:59 -0500 From: Matthew Wilcox To: linux-kernel@vger.kernel.org Cc: Matthew Wilcox , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, linux-nilfs@vger.kernel.org, linux-btrfs@vger.kernel.org, linux-xfs@vger.kernel.org, linux-usb@vger.kernel.org, Bjorn Andersson , Stefano Stabellini , iommu@lists.linux-foundation.org, linux-remoteproc@vger.kernel.org, linux-s390@vger.kernel.org, intel-gfx@lists.freedesktop.org, cgroups@vger.kernel.org, linux-sh@vger.kernel.org, David Howells Subject: [PATCH v6 79/99] blk-cgroup: Convert to XArray Date: Wed, 17 Jan 2018 12:21:43 -0800 Message-Id: <20180117202203.19756-80-willy@infradead.org> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180117202203.19756-1-willy@infradead.org> References: <20180117202203.19756-1-willy@infradead.org> Sender: linux-usb-owner@vger.kernel.org X-Mailing-List: linux-usb@vger.kernel.org X-getmail-retrieved-from-mailbox: INBOX X-Mailing-List: linux-kernel@vger.kernel.org List-ID: From: Matthew Wilcox This call to radix_tree_preload is awkward. At the point of allocation, we're under not only a local lock, but also under the queue lock. So we can't back out, drop the lock and retry the allocation. Replace this preload call with a call to xa_reserve() which will ensure the memory is allocated. Signed-off-by: Matthew Wilcox --- block/bfq-cgroup.c | 4 ++-- block/blk-cgroup.c | 52 ++++++++++++++++++++++------------------------ block/cfq-iosched.c | 4 ++-- include/linux/blk-cgroup.h | 5 ++--- 4 files changed, 31 insertions(+), 34 deletions(-) diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index da1525ec4c87..0648aaa6498b 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -860,7 +860,7 @@ static int bfq_io_set_weight_legacy(struct cgroup_subsys_state *css, return ret; ret = 0; - spin_lock_irq(&blkcg->lock); + xa_lock_irq(&blkcg->blkg_array); bfqgd->weight = (unsigned short)val; hlist_for_each_entry(blkg, &blkcg->blkg_list, blkcg_node) { struct bfq_group *bfqg = blkg_to_bfqg(blkg); @@ -894,7 +894,7 @@ static int bfq_io_set_weight_legacy(struct cgroup_subsys_state *css, bfqg->entity.prio_changed = 1; } } - spin_unlock_irq(&blkcg->lock); + xa_unlock_irq(&blkcg->blkg_array); return ret; } diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 4117524ca45b..37962d52f1a8 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -146,12 +146,12 @@ struct blkcg_gq *blkg_lookup_slowpath(struct blkcg *blkcg, struct blkcg_gq *blkg; /* - * Hint didn't match. Look up from the radix tree. Note that the + * Hint didn't match. Fetch from the xarray. Note that the * hint can only be updated under queue_lock as otherwise @blkg - * could have already been removed from blkg_tree. The caller is + * could have already been removed from blkg_array. The caller is * responsible for grabbing queue_lock if @update_hint. */ - blkg = radix_tree_lookup(&blkcg->blkg_tree, q->id); + blkg = xa_load(&blkcg->blkg_array, q->id); if (blkg && blkg->q == q) { if (update_hint) { lockdep_assert_held(q->queue_lock); @@ -223,8 +223,8 @@ static struct blkcg_gq *blkg_create(struct blkcg *blkcg, } /* insert */ - spin_lock(&blkcg->lock); - ret = radix_tree_insert(&blkcg->blkg_tree, q->id, blkg); + xa_lock(&blkcg->blkg_array); + ret = xa_err(__xa_store(&blkcg->blkg_array, q->id, blkg, GFP_NOWAIT)); if (likely(!ret)) { hlist_add_head_rcu(&blkg->blkcg_node, &blkcg->blkg_list); list_add(&blkg->q_node, &q->blkg_list); @@ -237,7 +237,7 @@ static struct blkcg_gq *blkg_create(struct blkcg *blkcg, } } blkg->online = true; - spin_unlock(&blkcg->lock); + xa_unlock(&blkcg->blkg_array); if (!ret) return blkg; @@ -314,7 +314,7 @@ static void blkg_destroy(struct blkcg_gq *blkg) int i; lockdep_assert_held(blkg->q->queue_lock); - lockdep_assert_held(&blkcg->lock); + lockdep_assert_held(&blkcg->blkg_array.xa_lock); /* Something wrong if we are trying to remove same group twice */ WARN_ON_ONCE(list_empty(&blkg->q_node)); @@ -334,7 +334,7 @@ static void blkg_destroy(struct blkcg_gq *blkg) blkg->online = false; - radix_tree_delete(&blkcg->blkg_tree, blkg->q->id); + xa_erase(&blkcg->blkg_array, blkg->q->id); list_del_init(&blkg->q_node); hlist_del_init_rcu(&blkg->blkcg_node); @@ -368,9 +368,9 @@ static void blkg_destroy_all(struct request_queue *q) list_for_each_entry_safe(blkg, n, &q->blkg_list, q_node) { struct blkcg *blkcg = blkg->blkcg; - spin_lock(&blkcg->lock); + xa_lock(&blkcg->blkg_array); blkg_destroy(blkg); - spin_unlock(&blkcg->lock); + xa_unlock(&blkcg->blkg_array); } q->root_blkg = NULL; @@ -443,7 +443,7 @@ static int blkcg_reset_stats(struct cgroup_subsys_state *css, int i; mutex_lock(&blkcg_pol_mutex); - spin_lock_irq(&blkcg->lock); + xa_lock_irq(&blkcg->blkg_array); /* * Note that stat reset is racy - it doesn't synchronize against @@ -462,7 +462,7 @@ static int blkcg_reset_stats(struct cgroup_subsys_state *css, } } - spin_unlock_irq(&blkcg->lock); + xa_unlock_irq(&blkcg->blkg_array); mutex_unlock(&blkcg_pol_mutex); return 0; } @@ -1012,7 +1012,7 @@ static void blkcg_css_offline(struct cgroup_subsys_state *css) { struct blkcg *blkcg = css_to_blkcg(css); - spin_lock_irq(&blkcg->lock); + xa_lock_irq(&blkcg->blkg_array); while (!hlist_empty(&blkcg->blkg_list)) { struct blkcg_gq *blkg = hlist_entry(blkcg->blkg_list.first, @@ -1023,13 +1023,13 @@ static void blkcg_css_offline(struct cgroup_subsys_state *css) blkg_destroy(blkg); spin_unlock(q->queue_lock); } else { - spin_unlock_irq(&blkcg->lock); + xa_unlock_irq(&blkcg->blkg_array); cpu_relax(); - spin_lock_irq(&blkcg->lock); + xa_lock_irq(&blkcg->blkg_array); } } - spin_unlock_irq(&blkcg->lock); + xa_unlock_irq(&blkcg->blkg_array); wb_blkcg_offline(blkcg); } @@ -1096,8 +1096,7 @@ blkcg_css_alloc(struct cgroup_subsys_state *parent_css) pol->cpd_init_fn(cpd); } - spin_lock_init(&blkcg->lock); - INIT_RADIX_TREE(&blkcg->blkg_tree, GFP_NOWAIT | __GFP_NOWARN); + xa_init_flags(&blkcg->blkg_array, XA_FLAGS_LOCK_IRQ); INIT_HLIST_HEAD(&blkcg->blkg_list); #ifdef CONFIG_CGROUP_WRITEBACK INIT_LIST_HEAD(&blkcg->cgwb_list); @@ -1132,14 +1131,14 @@ blkcg_css_alloc(struct cgroup_subsys_state *parent_css) int blkcg_init_queue(struct request_queue *q) { struct blkcg_gq *new_blkg, *blkg; - bool preloaded; int ret; new_blkg = blkg_alloc(&blkcg_root, q, GFP_KERNEL); if (!new_blkg) return -ENOMEM; - preloaded = !radix_tree_preload(GFP_KERNEL); + if (xa_reserve(&blkcg_root.blkg_array, q->id, GFP_KERNEL)) + return -ENOMEM; /* * Make sure the root blkg exists and count the existing blkgs. As @@ -1152,11 +1151,10 @@ int blkcg_init_queue(struct request_queue *q) spin_unlock_irq(q->queue_lock); rcu_read_unlock(); - if (preloaded) - radix_tree_preload_end(); - - if (IS_ERR(blkg)) + if (IS_ERR(blkg)) { + xa_erase(&blkcg_root.blkg_array, q->id); return PTR_ERR(blkg); + } q->root_blkg = blkg; q->root_rl.blkg = blkg; @@ -1374,8 +1372,8 @@ void blkcg_deactivate_policy(struct request_queue *q, __clear_bit(pol->plid, q->blkcg_pols); list_for_each_entry(blkg, &q->blkg_list, q_node) { - /* grab blkcg lock too while removing @pd from @blkg */ - spin_lock(&blkg->blkcg->lock); + /* grab xa_lock too while removing @pd from @blkg */ + xa_lock(&blkg->blkcg->blkg_array); if (blkg->pd[pol->plid]) { if (pol->pd_offline_fn) @@ -1384,7 +1382,7 @@ void blkcg_deactivate_policy(struct request_queue *q, blkg->pd[pol->plid] = NULL; } - spin_unlock(&blkg->blkcg->lock); + xa_unlock(&blkg->blkcg->blkg_array); } spin_unlock_irq(q->queue_lock); diff --git a/block/cfq-iosched.c b/block/cfq-iosched.c index 9f342ef1ad42..a51bef7af8df 100644 --- a/block/cfq-iosched.c +++ b/block/cfq-iosched.c @@ -1827,7 +1827,7 @@ static int __cfq_set_weight(struct cgroup_subsys_state *css, u64 val, if (val < min || val > max) return -ERANGE; - spin_lock_irq(&blkcg->lock); + xa_lock_irq(&blkcg->blkg_array); cfqgd = blkcg_to_cfqgd(blkcg); if (!cfqgd) { ret = -EINVAL; @@ -1859,7 +1859,7 @@ static int __cfq_set_weight(struct cgroup_subsys_state *css, u64 val, } out: - spin_unlock_irq(&blkcg->lock); + xa_unlock_irq(&blkcg->blkg_array); return ret; } diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h index e9825ff57b15..6278c49d3997 100644 --- a/include/linux/blk-cgroup.h +++ b/include/linux/blk-cgroup.h @@ -17,7 +17,7 @@ #include #include #include -#include +#include #include #include #include @@ -44,9 +44,8 @@ struct blkcg_gq; struct blkcg { struct cgroup_subsys_state css; - spinlock_t lock; - struct radix_tree_root blkg_tree; + struct xarray blkg_array; struct blkcg_gq __rcu *blkg_hint; struct hlist_head blkg_list; -- 2.15.1