From: Mohamed Khalfella <mkhalfella@purestorage.com>
To: Daniel Wagner <dwagner@suse.de>
Cc: Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@kernel.dk>,
Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>,
Justin Tee <justin.tee@broadcom.com>,
Naresh Gottumukkala <nareshgottumukkala83@gmail.com>,
Paul Ely <paul.ely@broadcom.com>,
Hannes Reinecke <hare@kernel.org>,
Chaitanya Kulkarni <kch@nvidia.com>,
James Smart <jsmart833426@gmail.com>,
Randy Jennings <randyj@purestorage.com>,
linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org,
Hannes Reinecke <hare@suse.de>
Subject: Re: [PATCH v6 03/18] nvmet: Implement CCR nvme command
Date: Mon, 19 Oct 2026 13:21:08 -0700 [thread overview]
Message-ID: <20261019202108.GL2931-mkhalfella@purestorage.com> (raw)
In-Reply-To: <179242560183.3619480.12432335570480465347.b4-review@b4>
On Mon 2026-10-19 18:00:01 +0200, Daniel Wagner wrote:
> > +void nvmet_execute_cross_ctrl_reset(struct nvmet_req *req)
> > +{
> > + struct nvmet_ctrl *ictrl, *sctrl = req->sq->ctrl;
> > + struct nvme_command *cmd = req->cmd;
> > + struct nvmet_ccr *ccr, *new_ccr;
> > + int ccr_active, ccr_total;
> > + u16 cntlid, status = NVME_SC_SUCCESS;
> > +
> > + if (!nvmet_check_transfer_len(req, 0))
> > + return;
> > +
> > + cntlid = le16_to_cpu(cmd->ccr.icid);
> > + if (sctrl->cntlid == cntlid) {
> > + req->error_loc =
> > + offsetof(struct nvme_cross_ctrl_reset_cmd, icid);
> > + status = NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
> > + goto out;
> > + }
> > +
> > + /* Find and get impacted controller */
> > + ictrl = nvmet_ctrl_find_get_ccr(sctrl->subsys, sctrl->hostnqn,
> > + cmd->ccr.ciu, cntlid,
> > + le64_to_cpu(cmd->ccr.cirn));
> > + if (!ictrl) {
> > + /* Immediate Reset Successful */
> > + nvmet_set_result(req, 1);
> > + status = NVME_SC_SUCCESS;
> > + goto out;
> > + }
> > +
> > + ccr_total = ccr_active = 0;
> > + mutex_lock(&sctrl->lock);
> > + list_for_each_entry(ccr, &sctrl->ccr_list, entry) {
> > + if (ccr->ctrl == ictrl) {
> > + status = NVME_SC_CCR_IN_PROGRESS | NVME_STATUS_DNR;
> > + goto out_unlock;
> > + }
> > +
> > + ccr_total++;
> > + if (ccr->ctrl)
> > + ccr_active++;
> > + }
> > +
> > + if (ccr_active >= NVMF_CCR_LIMIT) {
> > + status = NVME_SC_CCR_LIMIT_EXCEEDED;
> > + goto out_unlock;
> > + }
> > + if (ccr_total >= NVMF_CCR_PER_PAGE) {
> > + status = NVME_SC_CCR_LOGPAGE_FULL;
> > + goto out_unlock;
> > + }
> > +
> > + new_ccr = kmalloc_obj(*new_ccr, GFP_KERNEL);
> > + if (!new_ccr) {
> > + status = NVME_SC_INTERNAL;
> > + goto out_unlock;
> > + }
> > +
> > + new_ccr->ciu = cmd->ccr.ciu;
> > + new_ccr->icid = cntlid;
> > + new_ccr->ctrl = ictrl;
> > + list_add_tail(&new_ccr->entry, &sctrl->ccr_list);
> > +
> > +out_unlock:
> > + mutex_unlock(&sctrl->lock);
> > + if (status == NVME_SC_SUCCESS)
> > + nvmet_ctrl_fatal_error(ictrl);
>
> nvmet_ctrl_fatal_error will log an error, is this desired?
I think it is good to have the error logged. The fatal error might not
have been observed by the impacted controller we are failing yet, but it
is still okay to consider it as fatal error IMO.
>
> > +struct nvmet_ctrl *nvmet_ctrl_find_get_ccr(struct nvmet_subsys *subsys,
> > + const char *hostnqn, u8 ciu,
> > + u16 cntlid, u64 cirn)
> > +{
> > + struct nvmet_ctrl *ctrl, *ictrl = NULL;
> > + bool found = false;
> > +
> > + mutex_lock(&subsys->lock);
> > + list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) {
> > + if (ctrl->cntlid != cntlid)
> > + continue;
> > + if (strncmp(ctrl->hostnqn, hostnqn, NVMF_NQN_SIZE))
> > + continue;
> > +
> > + /* Avoid racing with a controller that is becoming ready */
>
> I think this comment could a bit more eleborate, it took me a while
> to figure out the impact of it. What about something like:
>
> /*
> * Enabling the controller (nvmet_start_ctrl()) regenerates
> * ciu and cirn under ctrl->lock. Hold the lock so that a
> * controller becoming ready concurrently is matched against
> * either the old or the new instance, never a mix of both.
> *
> * CSTS.RDY is not checked on purpose: a disabled controller
> * can still have commands in flight, so a matching instance
> * must go through the CCR path rather than being reported as
> * already reset.
> */
Okay, I will take the first paragraph and remove the mention to
nvmet_start_ctrl() because comments with function names do not
usually age well. The second paragraph is not directly related
to the lock IMO.
>
> --
> Daniel Wagner <wagi@kernel.org>
next prev parent reply other threads:[~2026-10-19 20:21 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-20 18:27 [PATCH v6 00/18] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
2026-09-20 18:27 ` [PATCH v6 01/18] nvmet: Rapid Path Failure Recovery set controller identify fields Mohamed Khalfella
2026-10-15 12:58 ` Daniel Wagner
2026-10-19 18:06 ` Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 02/18] nvmet/debugfs: Export controller CIU and CIRN via debugfs Mohamed Khalfella
2026-10-19 12:01 ` Daniel Wagner
2026-10-19 19:01 ` Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 03/18] nvmet: Implement CCR nvme command Mohamed Khalfella
2026-10-19 16:00 ` Daniel Wagner
2026-10-19 20:21 ` Mohamed Khalfella [this message]
2026-09-20 18:28 ` [PATCH v6 04/18] nvmet: Implement CCR logpage Mohamed Khalfella
2026-10-19 16:18 ` Daniel Wagner
2026-09-20 18:28 ` [PATCH v6 05/18] nvmet: Send an AEN on CCR completion Mohamed Khalfella
2026-10-19 16:31 ` Daniel Wagner
2026-09-20 18:28 ` [PATCH v6 06/18] nvme: Rapid Path Failure Recovery read controller identify fields Mohamed Khalfella
2026-10-19 16:56 ` Daniel Wagner
2026-10-19 21:18 ` Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 07/18] nvme: Introduce FENCING and FENCED controller states Mohamed Khalfella
2026-10-19 17:01 ` Daniel Wagner
2026-10-19 17:18 ` Daniel Wagner
2026-10-19 21:40 ` Mohamed Khalfella
2026-10-19 21:26 ` Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 08/18] nvme: Implement cross-controller reset recovery Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 09/18] nvme: Implement cross-controller reset completion Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 10/18] nvme-tcp: Use CCR to recover controller that hits an error Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 11/18] nvme-rdma: " Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 12/18] nvme-fc: start error recovery instead of aborting timed out IOs Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 13/18] nvme-fc: perform error recovery directly from ioerr_work Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 14/18] nvme-fc: Use CCR to recover controller that hits an error Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 15/18] nvme-fc: Hold inflight requests while in FENCING state Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 16/18] nvmet: Add support for CQT to nvme target Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 17/18] nvme: Add support for CQT to nvme host Mohamed Khalfella
2026-09-20 18:28 ` [PATCH v6 18/18] nvme: let controller deletion wait out a fencing window Mohamed Khalfella
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261019202108.GL2931-mkhalfella@purestorage.com \
--to=mkhalfella@purestorage.com \
--cc=axboe@kernel.dk \
--cc=dwagner@suse.de \
--cc=hare@kernel.org \
--cc=hare@suse.de \
--cc=hch@lst.de \
--cc=jsmart833426@gmail.com \
--cc=justin.tee@broadcom.com \
--cc=kbusch@kernel.org \
--cc=kch@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=nareshgottumukkala83@gmail.com \
--cc=paul.ely@broadcom.com \
--cc=randyj@purestorage.com \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®