From: Hannes Reinecke <hare@suse.de>
To: Mohamed Khalfella <mkhalfella@purestorage.com>,
Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@kernel.dk>,
Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>
Cc: Justin Tee <justin.tee@broadcom.com>,
Naresh Gottumukkala <nareshgottumukkala83@gmail.com>,
Paul Ely <paul.ely@broadcom.com>,
Hannes Reinecke <hare@kernel.org>,
Chaitanya Kulkarni <kch@nvidia.com>,
James Smart <jsmart833426@gmail.com>,
Randy Jennings <randyj@purestorage.com>,
Daniel Wagner <dwagner@suse.de>,
linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v7 07/17] nvme: Introduce FENCING and FENCED controller states
Date: Mon, 16 Nov 2026 08:16:48 +0100 [thread overview]
Message-ID: <fdebda8c-6669-423f-b20c-562b8d996d07@suse.de> (raw)
In-Reply-To: <20261114030454.2836543-8-mkhalfella@purestorage.com>
On 11/14/26 4:04 AM, Mohamed Khalfella wrote:
> Introduce two new controller states, FENCING and FENCED, and the state
> machine transitions needed to support them. Transports will move a LIVE
> controller into these states when an error is encountered; this patch
> only adds the states themselves.
>
> FENCING is entered from LIVE. While in FENCING the queues remain alive
> but new requests are not allowed to be sent, and the controller cannot
> be reset, since a reset cancels inflight IOs that must be held until
> either CCR succeeds or time-based recovery completes. A new request
> issued to a controller in FENCING or FENCED is failed with a host path
> error, so a multipath request is retried on another path.
>
> FENCED is a short-lived state entered from FENCING once fencing is
> over. It is the only state from which RESETTING is reachable, so a
> manual reset cannot take effect while the controller is in FENCING.
>
> Deletion is allowed from both states. Callers of nvme_delete_ctrl()
> assume deleting a live controller cannot fail, and they cannot wait for
> the fence either. RDMA device removal and the TCP/RDMA module exit
> paths hold the transport's controller list mutex, which ->free_ctrl
> needs when the fence drops the last reference on another controller.
> Instead the transport's ->stop_ctrl flushes the fencing works, so the
> delete path waits the fence out without holding any lock.
>
> A controller deleted while fencing does not keep the fence's hold on
> its inflight requests. Once the state leaves FENCING they complete as
> in any other delete and may be retried on another path before the
> fence would have ended. This is a known limitation, to be addressed
> separately.
>
> Update nvme_available_path() to treat a controller in FENCING/FENCED
> as an available path, both states are non-terminal, add the state
> names to the sysfs state attribute and document them in
> Documentation/ABI/stable/sysfs-nvme.
>
> Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
> ---
> Documentation/ABI/stable/sysfs-nvme | 5 +++--
> drivers/nvme/host/core.c | 29 +++++++++++++++++++++++++++--
> drivers/nvme/host/multipath.c | 2 ++
> drivers/nvme/host/nvme.h | 4 ++++
> drivers/nvme/host/sysfs.c | 2 ++
> 5 files changed, 38 insertions(+), 4 deletions(-)
>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Cheers,
Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
hare@suse.de +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
next prev parent reply other threads:[~2026-11-16 7:16 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-14 3:04 [PATCH v7 00/17] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
2026-11-14 3:04 ` [PATCH v7 01/17] nvmet: Rapid Path Failure Recovery set controller identify fields Mohamed Khalfella
2026-11-14 3:04 ` [PATCH v7 02/17] nvmet/debugfs: Export controller CIU and CIRN via debugfs Mohamed Khalfella
2026-11-14 3:04 ` [PATCH v7 03/17] nvmet: Implement CCR nvme command Mohamed Khalfella
2026-11-16 8:45 ` Dmitry Bogdanov
2026-11-14 3:04 ` [PATCH v7 04/17] nvmet: Implement CCR logpage Mohamed Khalfella
2026-11-14 3:04 ` [PATCH v7 05/17] nvmet: Send an AEN on CCR completion Mohamed Khalfella
2026-11-14 3:04 ` [PATCH v7 06/17] nvme: Rapid Path Failure Recovery read controller identify fields Mohamed Khalfella
2026-11-14 3:04 ` [PATCH v7 07/17] nvme: Introduce FENCING and FENCED controller states Mohamed Khalfella
2026-11-16 7:16 ` Hannes Reinecke [this message]
2026-11-14 3:04 ` [PATCH v7 08/17] nvme: Implement cross-controller reset recovery Mohamed Khalfella
2026-11-14 3:04 ` [PATCH v7 09/17] nvme: Implement cross-controller reset completion Mohamed Khalfella
2026-11-16 7:17 ` Hannes Reinecke
2026-11-14 3:04 ` [PATCH v7 10/17] nvme-tcp: Use CCR to recover controller that hits an error Mohamed Khalfella
2026-11-14 3:04 ` [PATCH v7 11/17] nvme-rdma: " Mohamed Khalfella
2026-11-16 7:19 ` Hannes Reinecke
2026-11-14 3:04 ` [PATCH v7 12/17] nvme-fc: start error recovery instead of aborting timed out IOs Mohamed Khalfella
2026-11-16 7:31 ` Hannes Reinecke
2026-11-14 3:04 ` [PATCH v7 13/17] nvme-fc: perform error recovery directly from ioerr_work Mohamed Khalfella
2026-11-16 7:37 ` Hannes Reinecke
2026-11-14 3:04 ` [PATCH v7 14/17] nvme-fc: Use CCR to recover controller that hits an error Mohamed Khalfella
2026-11-16 7:38 ` Hannes Reinecke
2026-11-14 3:04 ` [PATCH v7 15/17] nvme-fc: Hold inflight requests while in FENCING state Mohamed Khalfella
2026-11-16 7:52 ` Hannes Reinecke
2026-11-14 3:04 ` [PATCH v7 16/17] nvmet: Add support for CQT to nvme target Mohamed Khalfella
2026-11-14 3:04 ` [PATCH v7 17/17] nvme: Add support for CQT to nvme host Mohamed Khalfella
2026-11-16 7:55 ` Hannes Reinecke
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=fdebda8c-6669-423f-b20c-562b8d996d07@suse.de \
--to=hare@suse.de \
--cc=axboe@kernel.dk \
--cc=dwagner@suse.de \
--cc=hare@kernel.org \
--cc=hch@lst.de \
--cc=jsmart833426@gmail.com \
--cc=justin.tee@broadcom.com \
--cc=kbusch@kernel.org \
--cc=kch@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=mkhalfella@purestorage.com \
--cc=nareshgottumukkala83@gmail.com \
--cc=paul.ely@broadcom.com \
--cc=randyj@purestorage.com \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®