From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752060AbbJWO5r (ORCPT ); Fri, 23 Oct 2015 10:57:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:46613 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751097AbbJWO5q (ORCPT ); Fri, 23 Oct 2015 10:57:46 -0400 From: Jeff Moyer To: axboe@kernel.dk, Jason Luo Cc: linux-kernel@vger.kernel.org, Guru Anbalagane , Feng Jin , tj@kernel.org, Ming Lei Subject: [patch] blk-mq: avoid excessive boot delays with large lun counts X-PGP-KeyID: 1F78E1B4 X-PGP-CertKey: F6FE 280D 8293 F72C 65FD 5A58 1FF8 A7CA 1F78 E1B4 X-PCLoadLetter: What the f**k does that mean? Date: Fri, 23 Oct 2015 10:57:44 -0400 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, Zhangqing Luo reported long boot times on a system with thousands of LUNs when scsi-mq was enabled. He narrowed the problem down to blk_mq_add_queue_tag_set, where every queue is frozen in order to set the BLK_MQ_F_TAG_SHARED flag. Each added device will freeze all queues added before it in sequence, which involves waiting for an RCU grace period for each one. We don't need to do this. After the second queue is added, only new queues need to be initialized with the shared tag. We can do that by percolating the flag up to the blk_mq_tag_set, since new hctxs already inherit the tag_set's flags. That's what the below patch does. The re-check in blk_mq_add_queue_tag_set is done because we could have two queues racing to be the second queue added for the tag set. In such a case, it's possible that not all of the hctxs within the loser will be initialized. Because this initialization is done sequentially, we can simply check for the BLK_MQ_F_TAG_SHARED flag in the first hctx. This problem was introduced by commit 0d2602ca30e41 (blk-mq: improve support for shared tags maps). Reported-and-tested-by: Jason Luo Signed-off-by: Jeff Moyer diff --git a/block/blk-mq.c b/block/blk-mq.c index 7785ae9..8b4c484 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1860,27 +1860,26 @@ static void blk_mq_map_swqueue(struct request_queue *q, } } -static void blk_mq_update_tag_set_depth(struct blk_mq_tag_set *set) +static void queue_set_hctx_shared(struct request_queue *q, bool shared) { struct blk_mq_hw_ctx *hctx; - struct request_queue *q; - bool shared; int i; - if (set->tag_list.next == set->tag_list.prev) - shared = false; - else - shared = true; + queue_for_each_hw_ctx(q, hctx, i) { + if (shared) + hctx->flags |= BLK_MQ_F_TAG_SHARED; + else + hctx->flags &= ~BLK_MQ_F_TAG_SHARED; + } +} + +static void blk_mq_update_tag_set_depth(struct blk_mq_tag_set *set, bool shared) +{ + struct request_queue *q; list_for_each_entry(q, &set->tag_list, tag_set_list) { blk_mq_freeze_queue(q); - - queue_for_each_hw_ctx(q, hctx, i) { - if (shared) - hctx->flags |= BLK_MQ_F_TAG_SHARED; - else - hctx->flags &= ~BLK_MQ_F_TAG_SHARED; - } + queue_set_hctx_shared(q, shared); blk_mq_unfreeze_queue(q); } } @@ -1891,7 +1890,13 @@ static void blk_mq_del_queue_tag_set(struct request_queue *q) mutex_lock(&set->tag_list_lock); list_del_init(&q->tag_set_list); - blk_mq_update_tag_set_depth(set); + + if (set->tag_list.next == set->tag_list.prev) { + /* just transitioned to unshared */ + set->flags &= ~BLK_MQ_F_TAG_SHARED; + /* update existing queue */ + blk_mq_update_tag_set_depth(set, false); + } mutex_unlock(&set->tag_list_lock); } @@ -1902,7 +1907,24 @@ static void blk_mq_add_queue_tag_set(struct blk_mq_tag_set *set, mutex_lock(&set->tag_list_lock); list_add_tail(&q->tag_set_list, &set->tag_list); - blk_mq_update_tag_set_depth(set); + + if (set->tag_list.next != set->tag_list.prev) { + /* + * Only update the tag set state if the state has + * actually changed. + */ + if (!(set->flags & BLK_MQ_F_TAG_SHARED)) { + /* just transitioned to shared tags */ + set->flags |= BLK_MQ_F_TAG_SHARED; + blk_mq_update_tag_set_depth(set, true); + } else { + /* ensure we didn't race with another addition */ + struct blk_mq_hw_ctx *hctx = queue_first_hw_ctx(q); + if ((hctx->flags & BLK_MQ_F_TAG_SHARED) != + BLK_MQ_F_TAG_SHARED) + queue_set_hctx_shared(q, true); + } + } mutex_unlock(&set->tag_list_lock); } diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 5e7d43a..12ffc40 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -254,6 +254,9 @@ static inline void *blk_mq_rq_to_pdu(struct request *rq) for ((i) = 0; (i) < (hctx)->nr_ctx && \ ({ ctx = (hctx)->ctxs[(i)]; 1; }); (i)++) +#define queue_first_hw_ctx(q) \ + (q)->queue_hw_ctx[0] + #define blk_ctx_sum(q, sum) \ ({ \ struct blk_mq_ctx *__x; \