From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C05203A1E65 for ; Thu, 25 Dec 2025 17:17:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.181 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766683034; cv=none; b=XGZ3h/lQ/0Pm0tOH3ge5CABuBO/tXLsYXBvAmNuPIZmq65xXG5U/s/mfcJV3XtnHvmNXKGW7lhj4hpKdfu/damuJ62BcMxeuogx7vU4iSEOD3uxAYwQNTM88sKHWU+Icc3FsCZGCfZF/0zoz984Q9rgr6Ldt9Pa9j+ALH5irAyA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766683034; c=relaxed/simple; bh=1wtaVclmgQSimPKBhlUreE7cRBkqzLzKmDOkXysg2zc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=STi7C99NU24T0Dv1a6xzBfjxmRGOmrqPgRuDrorEWZvDxfJpFP6oeR99dXwZeBwVJqVW/1DsQek93MBX7bsHBavgBvSuS42reDMoalMF3B1sjWarmU3WJaUb/X7QV5JAanzpxGVSOwMGxMAo20UzzIq/ZcGI5GzF2x31y+jUNeE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=I3jxvHv0; arc=none smtp.client-ip=209.85.215.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="I3jxvHv0" Received: by mail-pg1-f181.google.com with SMTP id 41be03b00d2f7-bf1b402fa3cso6669944a12.3 for ; Thu, 25 Dec 2025 09:17:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1766683032; x=1767287832; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=xyZ6hMkW3DX+5TPW1p+QwoHRS86qiGeWn6NDq2N5Ct0=; b=I3jxvHv0hrq+VsE1pwj7n9WzBYLF9cNOmfBrgUN8+P4u0yjxfpIs9eFDVnBDT0mjhn 9/CSEnyU5IRVl4Iycw7OWMsu6GXwRc/m4NBjmq/Al/XmLRzBLydd8qE4OnI1L665q+9B kzT+sr3+bpzy/gnwLB36/UPxm0ynOhA+4k+csIyQ5ADqUGPTt+WWeDv/1UNrAG6B4p3z 1Lp62ikTjYlZTzXlxd8jkSijzimgsIkYKoxBhm/YVpCdwxZLjRaR3Npp5RKj8VfX1lj5 11yT7mOfaH5eEzBp00CnKh+/wmSXm6dMMA7rguBeQh4VJUQRb+uNmxJDcpkLpHzkCwqC lyrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766683032; x=1767287832; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xyZ6hMkW3DX+5TPW1p+QwoHRS86qiGeWn6NDq2N5Ct0=; b=cvBS2rHF5i2D3Z7tb+ZkO6k3uHC1i/RfFYgnHZITpx1Kh2J3881yQ+MHMMuMGfMNY0 5RSxFqIc+T9e4YaYz6ayvHU9+L+vH+ckRbxvHDQIgC5VPJPmFQk7f65iY2eJXd1eZZLV 0FxbRBuyr6o3QKGOOEtJU62FD68MM97kmdqr4UNGO3g7oeVmRhC3la6r41ddh10MObH5 aHzbvgJGzLqhQXXcn3qz/f7gTeDQVnuhcpeq/1tknK7hDpqQpqJExQ0IyoR/uN6KTj8C 6si3jIpcTSuSU5ln3clOpA0Z4XAfgqAuGBBzQMBFu50Vj66cOWSr+9WPVfP8lpW/2Zgy KOlw== X-Forwarded-Encrypted: i=1; AJvYcCVLoKmTj4kKksScRP2USfSsmzWs2bOMV7ex5AVDQ53r7Zko0z2vuOiczTzWF+OZDiFNa1oust1aTlaiSPs=@vger.kernel.org X-Gm-Message-State: AOJu0YzHHmik+2ez22eFFfUdtbLiorZnFgXfT0B7rLrtXhhli+zsXvM7 wHGKtCggS6Qznc0RBjn+jNowCNZMRPrR4sQ814/BqB9/W8fcTJbIrjJEaAQSdyrp7Qw= X-Gm-Gg: AY/fxX5jJxzy416H4bmpNN1oujBSxCMTAB+Pol0gNmbdjXMkg1zjsVteoB6ZRxcWwL4 dV3diF1LBUZEWtxXPiaHPmMxfujmmKjHUIjeZvS49JWVpfCrT3D202+uIo8QMXu8tFUKO0OHfSI wcR+OrZVDEY8ZXkWyGs0wUi4rGXCs5eN1vdKy/iW7sgfeMTg0flCvmJXRwUkj8CVBu8l/tsu83u +5wpOmOzhZj5O5FPTM3ILnVZXop6u8VUWO+EK1CwKxfKZbza9n5Wj50OBOUcIDJje4hcIElkk// HLGHv9dQgsGphCCzOhljGYBol7+PPb4TedaCN6eiyBot/g8mA82MVzw/IV7PT0kNsQ5mDxRZzFN noy/A9Tzm/Ez8sEw6HgdA0bmjiZLSDJQlFpgHlrv0xF6X44Bwn6qQHPu6WJshvDYUI2kIny1v56 M1kef5gdqCfZXRTXc= X-Google-Smtp-Source: AGHT+IEujUxnJ3Yf56OShfmRl6zR4FEe2J4/AA7VRwnwhDo1gdETfH6iNO+yk0TJK0yI38t5mq8J9g== X-Received: by 2002:a05:7022:7e84:b0:119:e56c:189c with SMTP id a92af1059eb24-121722aa627mr22924662c88.4.1766683031606; Thu, 25 Dec 2025 09:17:11 -0800 (PST) Received: from medusa.lab.kspace.sh ([2601:640:8202:6fb0::1305]) by smtp.googlemail.com with UTF8SMTPSA id a92af1059eb24-1217253c23csm75969311c88.9.2025.12.25.09.17.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Dec 2025 09:17:11 -0800 (PST) Date: Thu, 25 Dec 2025 09:17:09 -0800 From: Mohamed Khalfella To: Sagi Grimberg Cc: Chaitanya Kulkarni , Christoph Hellwig , Jens Axboe , Keith Busch , Aaron Dailey , Randy Jennings , John Meneghini , Hannes Reinecke , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [RFC PATCH 07/14] nvme: Add RECOVERING nvme controller state Message-ID: <20251225171709.GA8129-mkhalfella@purestorage.com> References: <20251126021250.2583630-1-mkhalfella@purestorage.com> <20251126021250.2583630-8-mkhalfella@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Thu 2025-12-25 15:29:52 +0200, Sagi Grimberg wrote: > > > On 26/11/2025 4:11, Mohamed Khalfella wrote: > > Add NVME_CTRL_RECOVERING as a new controller state to be used when > > impacted controller is being recovered. A LIVE controller enters > > RECOVERING state when an IO error is encountered. While recovering > > inflight IOs will not be canceled if they timeout. These IOs will be > > canceled after recovery finishes. Also, while recovering a controller > > can not be reset or deleted. This is intentional because reset or delete > > will result in canceling inflight IOs. When recovery finishes, the > > impacted controller transitions from RECOVERING state to RESETTING state. > > Reset codepath takes care of queues teardown and inflight requests > > cancellation. > > Is RECOVERING really capturing the nature of this state? Maybe RESETTLING? > or QUIESCING? Naming is hard. QUIESCING sounds better, I will renaming it to QUIESCING. > > > > > Note, there is no transition from RECOVERING to RESETTING added to > > nvme_change_ctrl_state(). The reason is that user should not be allowed > > to reset or delete a controller that is being recovered. > > > > Add NVME_CTRL_RECOVERED controller flag. This flag is set on a controller > > about to schedule delayed work for time based recovery. > > > > Signed-off-by: Mohamed Khalfella > > --- > > drivers/nvme/host/core.c | 10 ++++++++++ > > drivers/nvme/host/nvme.h | 2 ++ > > drivers/nvme/host/sysfs.c | 1 + > > 3 files changed, 13 insertions(+) > > > > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c > > index aa007a7b9606..f5b84bc327d3 100644 > > --- a/drivers/nvme/host/core.c > > +++ b/drivers/nvme/host/core.c > > @@ -574,6 +574,15 @@ bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl, > > break; > > } > > break; > > + case NVME_CTRL_RECOVERING: > > + switch (old_state) { > > + case NVME_CTRL_LIVE: > > + changed = true; > > + fallthrough; > > + default: > > + break; > > + } > > + break; > > That is a strange transition... Why is it strange? We transition to RECOVERING state only if controller is LIVE. This is when we expect to have inflight user IOs to be quiesced by CCR. We do not care about inflight requests in other states.