From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751585AbdJMQR6 (ORCPT ); Fri, 13 Oct 2017 12:17:58 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40334 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750716AbdJMQR4 (ORCPT ); Fri, 13 Oct 2017 12:17:56 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 6FF824DAFB Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=ming.lei@redhat.com Date: Sat, 14 Oct 2017 00:17:40 +0800 From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, Christoph Hellwig , Bart Van Assche , Laurence Oberman , Paolo Valente , Oleksandr Natalenko , Tom Nguyen , linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, Omar Sandoval , John Garry Subject: Re: [PATCH V7 4/6] blk-mq: introduce .get_budget and .put_budget in blk_mq_ops Message-ID: <20171013161739.GB30899@ming.t460p> References: <20171012183704.22326-1-ming.lei@redhat.com> <20171012183704.22326-5-ming.lei@redhat.com> <9a741c03-90a3-e583-ddde-0ed71c8570a2@kernel.dk> <20171013001919.GA24715@ming.t460p> <6efdb459-8746-562d-06dc-5b3e172076e1@kernel.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6efdb459-8746-562d-06dc-5b3e172076e1@kernel.dk> User-Agent: Mutt/1.8.3 (2017-05-23) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 13 Oct 2017 16:17:56 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Oct 13, 2017 at 08:44:23AM -0600, Jens Axboe wrote: > On 10/12/2017 06:19 PM, Ming Lei wrote: > > On Thu, Oct 12, 2017 at 12:46:24PM -0600, Jens Axboe wrote: > >> On 10/12/2017 12:37 PM, Ming Lei wrote: > >>> For SCSI devices, there is often per-request-queue depth, which need > >>> to be respected before queuing one request. > >>> > >>> The current blk-mq always dequeues one request first, then calls .queue_rq() > >>> to dispatch the request to lld. One obvious issue of this way is that I/O > >>> merge may not be good, because when the per-request-queue depth can't be > >>> respected, .queue_rq() has to return BLK_STS_RESOURCE, then this request > >>> has to staty in hctx->dispatch list, and never got chance to participate > >>> into I/O merge. > >>> > >>> This patch introduces .get_budget and .put_budget callback in blk_mq_ops, > >>> then we can try to get reserved budget first before dequeuing request. > >>> Once we can't get budget for queueing I/O, we don't need to dequeue request > >>> at all, then I/O merge can get improved a lot. > >> > >> I can't help but think that it would be cleaner to just be able to > >> reinsert the request into the scheduler properly, if we fail to > >> dispatch it. Bart hinted at that earlier as well. > > > > Actually when I start to investigate the issue, the 1st thing I tried > > is to reinsert, but that way is even worse on qla2xxx. > > > > Once request is dequeued, the IO merge chance is decreased a lot. > > With none scheduler, it becomes not possible to merge because > > we only try to merge over the last 8 requests. With mq-deadline, > > when one request is reinserted, another request may be dequeued > > at the same time. > > I don't care too much about 'none'. If perfect merging is crucial for > getting to the performance level you want on the hardware you are using, > you should not be using 'none'. 'none' will work perfectly fine for NVMe > etc style devices, where we are not dependent on merging to the same > extent that we are on other devices. We still have some SCSI device, such as qla2xxx, which is 1:1 multi-queue device, like NVMe, in my test, the big lock of mq-deadline has been an issue for this kind of device, and none actually is better than mq-deadline, even though its merge isn't good. Thanks, Ming