From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS, T_DKIMWL_WL_HIGH,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7573BC4321D for ; Thu, 16 Aug 2018 08:29:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 242512148C for ; Thu, 16 Aug 2018 08:29:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="zxiQQ1y2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 242512148C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390002AbeHPL0V (ORCPT ); Thu, 16 Aug 2018 07:26:21 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:43118 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729008AbeHPL0V (ORCPT ); Thu, 16 Aug 2018 07:26:21 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w7G8TP27160092; Thu, 16 Aug 2018 08:29:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=ZwUiwvKPW2ixFoZONtmJIMLGO/EZl7+1pWJ4SyCO2tA=; b=zxiQQ1y2O2ygV2L4bvALhBndp+KFynXlMsPRlH+bs9Dy4rnj5nUKI6sno9oyOqP6MZVL E0TPKWH87lNC7e7M/Eqm+4bJw1V72lO1WE0JOa0RAa/KX9lOCYoxcR/Q04MVjequ3tut qFojbx4xGfXYH0CHzZbyFGPssjUcFY4E5+XDnOL1+qqn5wl6zsONkKmzt4Sjym4QmQwf S4OQ85oiIcQNl0A1kPCgzE8Y9D6z2HW5WpZ5ew1gM4jInbEJsmq4pJM0Ww6akGms7hjN T+pAgi1Jy8MPsVXsBIF4m47RrfuyI45srrzQwNpWdvj7jTo5iZbYXPomQdHkh0nbXhHv TA== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2ksreq8yaf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 16 Aug 2018 08:29:25 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w7G8TOQM005397 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 16 Aug 2018 08:29:24 GMT Received: from abhmp0003.oracle.com (abhmp0003.oracle.com [141.146.116.9]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w7G8TNZA029161; Thu, 16 Aug 2018 08:29:23 GMT Received: from [10.182.69.179] (/10.182.69.179) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 16 Aug 2018 01:29:23 -0700 Subject: Re: [PATCH 2/2] blk-mq: sync the update nr_hw_queues with part_in_flight To: Ming Lei Cc: Jens Axboe , Bart Van Assche , Keith Busch , linux-block , Linux Kernel Mailing List References: <1534317915-5041-1-git-send-email-jianchao.w.wang@oracle.com> <1534317915-5041-3-git-send-email-jianchao.w.wang@oracle.com> From: "jianchao.wang" Message-ID: <8cd103e6-0986-2c45-00f2-8a2a540a2eb5@oracle.com> Date: Thu, 16 Aug 2018 16:29:33 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8986 signatures=668707 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=897 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1808160091 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 08/16/2018 12:50 PM, Ming Lei wrote: > On Wed, Aug 15, 2018 at 3:25 PM, Jianchao Wang > wrote: >> For blk-mq, part_in_flight/rw will invoke blk_mq_in_flight/rw to >> account the inflight requests. It will access the queue_hw_ctx and >> nr_hw_queues w/o any protection. When updating nr_hw_queues and >> blk_mq_in_flight/rw occur concurrently, panic comes up. > > When updating nr_hw_queues, all queues are frozen, and there shouldn't > be any inflight requests, so there shouldn't be such issue. > I get following crash when do updating nr_hw_queues test. [ 112.643189] BUG: unable to handle kernel NULL pointer dereference at 0000000000000174 [ 112.643275] PGD 40baf8067 P4D 40baf8067 PUD 40bb38067 PMD 0 [ 112.643334] Oops: 0000 [#1] PREEMPT SMP [ 112.643372] CPU: 7 PID: 1526 Comm: fio Kdump: loaded Not tainted 4.18.0-rc6+ #250 [ 112.643434] Hardware name: LENOVO 10MLS0E339/3106, BIOS M1AKT22A 06/27/2017 [ 112.643499] RIP: 0010:blk_mq_queue_tag_busy_iter+0x4d/0x250 [ 112.643548] Code: 48 89 54 24 20 c7 44 24 0c 00 00 00 00 85 c9 0f 84 25 01 00 00 48 8b 7c 24 10 48 63 44 24 0c 48 8b 97 88 01 00 00 4c 8b 34 c2 <41> 8b 96 74 01 00 00 4d 8b a6 e8 01 00 00 85 d2 0f 84 e0 00 00 00 [ 112.643791] RSP: 0018:ffff95708284fc70 EFLAGS: 00010202 [ 112.643840] RAX: 0000000000000002 RBX: ffff895b49570e18 RCX: 00000000000000ff [ 112.643899] RDX: ffff895b4fc1d6c0 RSI: ffffffff8b6a94b0 RDI: ffff895b49570e18 [ 112.643961] RBP: 000000000000001f R08: 0000000000000000 R09: 0000000000000000 [ 112.644021] R10: ffff95708284fcd8 R11: ffffffff8b6b547d R12: ffff895b4fc30b40 [ 112.644081] R13: 0000000000000000 R14: 0000000000000000 R15: ffff895b48c96c40 [ 112.644144] FS: 00007fa79fd4c700(0000) GS:ffff895b62dc0000(0000) knlGS:0000000000000000 [ 112.644212] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 112.644270] CR2: 0000000000000174 CR3: 000000040b9ae001 CR4: 00000000003606e0 [ 112.644357] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 112.644459] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 112.644552] Call Trace: [ 112.644603] ? blk_mq_stop_hw_queues+0x50/0x50 [ 112.644675] ? blk_mq_stop_hw_queues+0x50/0x50 [ 112.644748] blk_mq_in_flight+0x2e/0x40 [ 112.644792] part_round_stats+0x158/0x160 [ 112.644835] part_stat_show+0x9c/0x530 [ 112.644873] ? lock_acquire+0xab/0x200 [ 112.644913] ? kernfs_seq_start+0x32/0x90 [ 112.644959] dev_attr_show+0x19/0x50 [ 112.644996] sysfs_kf_seq_show+0xad/0x100 [ 112.645039] seq_read+0xa5/0x410 [ 112.645075] ? __mutex_lock+0x20e/0x990 [ 112.645117] __vfs_read+0x23/0x160 [ 112.645158] vfs_read+0xa0/0x140 [ 112.645193] ksys_read+0x45/0xa0 [ 112.645230] do_syscall_64+0x5a/0x1a0 [ 112.645267] entry_SYSCALL_64_after_hwframe+0x49/0xbe The blk_mq_in_flight will access the queue_hw_ctx and nr_hw_queues when updating nr_hw_queues is ongoing. Thanks Jianchao > > Thanks, > Ming Lei >