From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754152AbdJSRrf (ORCPT ); Thu, 19 Oct 2017 13:47:35 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:41962 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751087AbdJSRrc (ORCPT ); Thu, 19 Oct 2017 13:47:32 -0400 X-IronPort-AV: E=Sophos;i="5.43,402,1503331200"; d="scan'208";a="154867185" From: Bart Van Assche To: "roman.penyaev@profitbricks.com" CC: Bart Van Assche , "linux-kernel@vger.kernel.org" , "linux-block@vger.kernel.org" , "hare@suse.com" , "axboe@fb.com" , "hch@lst.de" Subject: Re: [PATCH 1/1] [RFC] blk-mq: fix queue stalling on shared hctx restart Thread-Topic: [PATCH 1/1] [RFC] blk-mq: fix queue stalling on shared hctx restart Thread-Index: AQHTR/sENmxK85jYyU6rqB1jXfMKp6LrdKyA Date: Thu, 19 Oct 2017 17:47:24 +0000 Message-ID: <1508435243.2429.42.camel@wdc.com> References: <20171018102206.26020-1-roman.penyaev@profitbricks.com> In-Reply-To: <20171018102206.26020-1-roman.penyaev@profitbricks.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Bart.VanAssche@wdc.com; x-originating-ip: [63.163.107.100] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;CY1PR0401MB1535;20:BvgBJOoxv6R1uTJgPXRefIuMCqcTv4HrJ5hvHtAeFM1u+S3+wE/eY0RhUqoSt5uVHdmHPmsoKiD7DN7cTS4BS5LuMQaBe8QAdEhIS8pVjU3941N2eZGX6/7eMnux8VxbRqMdpo+Ec0PvfzUIb73diFEo9rei5JwxCvp1aWlE1Ck= x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-ms-office365-filtering-correlation-id: 5bf02c64-099e-41a0-5b9d-08d5171974fc x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(2017030254172)(48565401081)(4534020)(4602075)(4627075)(201703031133081)(201702281549075)(2017052603199)(201703131423095);SRVR:CY1PR0401MB1535; x-ms-traffictypediagnostic: CY1PR0401MB1535: wdcipoutbound: EOP-TRUE x-exchange-antispam-report-test: UriScan:; x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(10201501046)(100000703101)(100105400095)(3002001)(93006095)(93001095)(6055026)(6041248)(20161123564025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123555025)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:CY1PR0401MB1535;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:CY1PR0401MB1535; x-forefront-prvs: 0465429B7F x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6009001)(346002)(376002)(39860400002)(189002)(377424004)(199003)(24454002)(189998001)(77096006)(97736004)(54906003)(6486002)(14454004)(229853002)(72206003)(2950100002)(6916009)(2900100001)(6436002)(36756003)(25786009)(2501003)(5640700003)(6506006)(54356999)(86362001)(4326008)(3660700001)(50986999)(6246003)(99286003)(6512007)(101416001)(53936002)(478600001)(76176999)(81156014)(8676002)(81166006)(7736002)(2351001)(3846002)(3280700002)(102836003)(6116002)(8936002)(68736007)(2906002)(103116003)(305945005)(5660300001)(316002)(4001150100001)(106356001)(105586002)(33646002)(66066001);DIR:OUT;SFP:1102;SCL:1;SRVR:CY1PR0401MB1535;H:CY1PR0401MB1536.namprd04.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: MIME-Version: 1.0 X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Oct 2017 17:47:24.6016 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0401MB1535 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by nfs id v9JHldrY002392 On Wed, 2017-10-18 at 12:22 +0200, Roman Pen wrote: > the patch below fixes queue stalling when shared hctx marked for restart > (BLK_MQ_S_SCHED_RESTART bit) but q->shared_hctx_restart stays zero. The > root cause is that hctxs are shared between queues, but 'shared_hctx_restart' > belongs to the particular queue, which in fact may not need to be restarted, > thus we return from blk_mq_sched_restart() and leave shared hctx of another > queue never restarted. > > The fix is to make shared_hctx_restart counter belong not to the queue, but > to tags, thereby counter will reflect real number of shared hctx needed to > be restarted. Hello Roman, The patch you posted looks fine to me but seeing this patch and the patch description makes me wonder why this had not been noticed before. Are you perhaps using a block driver that returns BLK_STS_RESOURCE more often than other block drivers? Did you perhaps run into this with the Infiniband network block device (IBNBD) driver? No matter what driver triggered this, I think this bug should be fixed. Bart.