From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751871AbeA2WhD (ORCPT ); Mon, 29 Jan 2018 17:37:03 -0500 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:30872 "EHLO esa4.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751457AbeA2WhB (ORCPT ); Mon, 29 Jan 2018 17:37:01 -0500 X-IronPort-AV: E=Sophos;i="5.46,432,1511798400"; d="scan'208";a="70023620" Subject: Re: [RFC PATCH] blk-mq: fixup RESTART when queue becomes idle To: Jens Axboe , Ming Lei Cc: "snitzer@redhat.com" , "dm-devel@redhat.com" , "hch@infradead.org" , "linux-kernel@vger.kernel.org" , "linux-block@vger.kernel.org" , "osandov@fb.com" References: <20180118024124.8079-1-ming.lei@redhat.com> <20180118170353.GB19734@redhat.com> <1516296056.2676.23.camel@wdc.com> <20180118183039.GA20121@redhat.com> <1516301278.2676.35.camel@wdc.com> <20180119023212.GA25413@ming.t460p> <20180119072623.GB25369@ming.t460p> <047f68ec-f51b-190f-2f89-f413325c2540@kernel.dk> From: Bart Van Assche Message-ID: Date: Mon, 29 Jan 2018 14:37:00 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <047f68ec-f51b-190f-2f89-f413325c2540@kernel.dk> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 01/19/18 07:24, Jens Axboe wrote: > That's what I thought. So for a low queue depth underlying queue, it's > quite possible that this situation can happen. Two potential solutions > I see: > > 1) As described earlier in this thread, having a mechanism for being > notified when the scarce resource becomes available. It would not > be hard to tap into the existing sbitmap wait queue for that. > > 2) Have dm set BLK_MQ_F_BLOCKING and just sleep on the resource > allocation. I haven't read the dm code to know if this is a > possibility or not. > > I'd probably prefer #1. It's a classic case of trying to get the > request, and if it fails, add ourselves to the sbitmap tag wait > queue head, retry, and bail if that also fails. Connecting the > scarce resource and the consumer is the only way to really fix > this, without bogus arbitrary delays. (replying to an e-mail from ten days ago) Implementing a notification mechanism for all cases in which blk_insert_cloned_request() returns BLK_STS_RESOURCE today would require a lot of work. If e.g. a SCSI LLD returns one of the SCSI_MLQUEUE_*_BUSY return codes from its .queuecommand() implementation then the SCSI core will translate that return code into BLK_STS_RESOURCE. From scsi_queue_rq(): reason = scsi_dispatch_cmd(cmd); if (reason) { scsi_set_blocked(cmd, reason); ret = BLK_STS_RESOURCE; goto out_dec_host_busy; } In other words, implementing a notification mechanism for all cases in which blk_insert_cloned_request() can return BLK_STS_RESOURCE would require to modify all SCSI LLDs. Bart.