From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755647AbeARKLg (ORCPT ); Thu, 18 Jan 2018 05:11:36 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:54466 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755629AbeARKLe (ORCPT ); Thu, 18 Jan 2018 05:11:34 -0500 From: Jianchao Wang To: keith.busch@intel.com, axboe@fb.com, hch@lst.de, sagi@grimberg.me, maxg@mellanox.com, james.smart@broadcom.com Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH V5 0/2] nvme-pci: fix the timeout case when reset is ongoing Date: Thu, 18 Jan 2018 18:10:00 +0800 Message-Id: <1516270202-8051-1-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8777 signatures=668653 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1801180144 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello Please consider the following scenario. nvme_reset_ctrl -> set state to RESETTING -> queue reset_work (scheduling) nvme_reset_work -> nvme_dev_disable -> quiesce queues -> nvme_cancel_request on outstanding requests -------------------------------_boundary_ -> nvme initializing (issue request on adminq) Before the _boundary_, not only quiesce the queues, but only cancel all the outstanding requests. A request could expire when the ctrl state is RESETTING. - If the timeout occur before the _boundary_, the expired requests are from the previous work. - Otherwise, the expired requests are from the controller initializing procedure, such as sending cq/sq create commands to adminq to setup io queues. In current implementation, nvme_timeout cannot identify the _boundary_ so only handles second case above. In fact, after Sagi's commit (nvme-rdma: fix concurrent reset and reconnect), both nvme-fc/rdma have following pattern: RESETTING - quiesce blk-mq queues, teardown and delete queues/ connections, clear out outstanding IO requests... RECONNECTING - establish new queues/connections and some other initializing things. Introduce RECONNECTING to nvme-pci transport to do the same mark Then we get a coherent state definition among nvme pci/rdma/fc transports and nvme_timeout could identify the _boundary_. V5: - discard RESET_PREPARE and introduce RESETTING into nvme-pci - change the 1st patch's name and comment - other misc changes V4: - rebase patches on Jens' for-next - let RESETTING equal to RECONNECTING in terms of work procedure - change the 1st patch's name and comment - other misc changes V3: - fix wrong reference in loop.c - other misc changes V2: - split NVME_CTRL_RESETTING into NVME_CTRL_RESET_PREPARE and NVME_CTRL_RESETTING. Introduce new patch based on this. - distinguish the requests based on the new state in nvme_timeout - change comments of patch drivers/nvme/host/core.c | 2 +- drivers/nvme/host/pci.c | 43 ++++++++++++++++++++++++++++++++----------- 2 files changed, 33 insertions(+), 12 deletions(-) Thanks Jianchao