From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A7DD4E50C3; Wed, 7 Oct 2026 19:25:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791401148; cv=none; b=fR2ZocqxiFh7xJlI95XnTpXFkKBxrV106ewDAydO/aczLUYF1Fr8O+iU29hw526zCRbNpXL1cCRrxuHfJQ/azhCjuYFseUEAzmtNRWicFLM/mgkPuuSKUguizx4PmF70bAl4zBiaF/8HyIU5YL/xOG6e/kfWkBuL2yMC8skTO+4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791401148; c=relaxed/simple; bh=uZNSQW5km3pGBDLJixQznEYC2jPcAQwv5Tuqp9DfZbE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=SUxlReelie4fEChfGod34/xRNGYnq8WyoZjZt+8TVWgokAnX63dkjyZP0FjA5xhlJqUA2vr+z8Bkg1xWI8JW6U81ugaUeRDlYw+BT0egYym8nPoZRzKCiSSl4RFUpgzQpd4Qodl9XXq3z49ybgG6Rk1Qt/n31pJW2hxfbdQ0Ba0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ExRcOm7R; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ExRcOm7R" Received: by smtp.kernel.org (Postfix) with ESMTPS id F34F6C2BCC7; Wed, 7 Oct 2026 19:25:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1791401148; bh=uZNSQW5km3pGBDLJixQznEYC2jPcAQwv5Tuqp9DfZbE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=ExRcOm7RwmrUBLjs9JJ6LdirFZxXdb8frO8/Q92I/5EM0HicOM871mFmsWeTyQR6+ 9xc+f6X/OThepLn03usTMCNItL94atNeNUNmTKUQpt7BppDKFQclVqHkyFHX336UdM Xvv5JOg7lFt9zjP1m1lKQok11Dne/eiSJuOMBlZxKR1IM8fQNiBBwbsYtcIyqmfp7w rVWAFmuAqHxeMjbXptaGcc8C9YRn71NhCn7MDV0tgeJ/oc1AJlW/rPLYDqJfJGDRRU YlIQiicDakWArWijTBRMGdDe+zzi+7AtvN5qQUaTeNyloSzr9cHV6KxOYAufNvrjRN E7tQzYjKN5BMg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1F03CA600B; Wed, 7 Oct 2026 19:25:47 +0000 (UTC) From: Dave Marquardt via B4 Relay Date: Wed, 07 Oct 2026 14:25:54 -0500 Subject: [PATCH v13 10/11] scsi: ibmvfc: register and use asynchronous sub CRQ for events Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20261007-ibmvfc-fpin-support-v13-10-b816abfa4fcc@linux.ibm.com> References: <20261007-ibmvfc-fpin-support-v13-0-b816abfa4fcc@linux.ibm.com> In-Reply-To: <20261007-ibmvfc-fpin-support-v13-0-b816abfa4fcc@linux.ibm.com> To: Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , "Christophe Leroy (CS GROUP)" , "James E.J. Bottomley" , "Martin K. Petersen" , Tyrel Datwyler Cc: linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, Dave Marquardt X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1791401146; l=9309; i=davemarq@linux.ibm.com; s=20260216; h=from:subject:message-id; bh=kiaSmVDkVPy6s9ZSr2SRkPqi/WeZj5R01MBu9VzmzZg=; b=ikTlhq8Gq0ALPRB0TKLB2l2JJqHypk7sPMyILTJMV557DjpBH140ogVWLMQj/e7cprPcY7IHm 9qISguW8fnOCc8zWOZpTZPvdwhmXMLPfU3L/jztMC2+/t5RugaRQIap X-Developer-Key: i=davemarq@linux.ibm.com; a=ed25519; pk=vy0/nfobrje6EqZxuyw6a3ZstytG8WK2vf5Y3xtGrEg= X-Endpoint-Received: by B4 Relay for davemarq@linux.ibm.com/20260216 with auth_id=689 X-Original-From: Dave Marquardt Reply-To: davemarq@linux.ibm.com From: Dave Marquardt Wire the async sub-CRQ into the sub-CRQ lifecycle so it is allocated, registered, and freed alongside the SCSI channel queues. In ibmvfc_probe(), initialize vhost->async_scrqs with desired_queues=1, max_queues=1, and protocol=IBMVFC_PROTO_ASYNC. Add ibmvfc_release_channels() as a helper to free a channels array and its queue buffers. In ibmvfc_init_sub_crqs(), allocate the async sub-CRQ via ibmvfc_alloc_channels() before allocating the SCSI channels. Register it with the VIOS via ibmvfc_reg_sub_crqs(). Either failure disables multi-queue and aborts init. In ibmvfc_alloc_channels(), select IBMVFC_ASYNC_SUB_CRQ_FMT for async protocol channels instead of IBMVFC_SUB_CRQ_FMT, so the queue is sized for ibmvfc_async_sub_crq entries. In ibmvfc_release_sub_crqs(), ibmvfc_reenable_crq_queue(), and ibmvfc_reset_crq(), deregister and re-register the async sub-CRQ alongside the SCSI channel queues. In ibmvfc_channel_setup_done(), capture the async sub-CRQ handle returned by the VIOS in the channel setup response and store it in vhost->async_scrqs.scrqs[0].vios_cookie. Pass the async sub-CRQ cookie to the VIOS in ibmvfc_channel_setup() via async_sub_crq_handle. In ibmvfc_set_login_info(), advertise IBMVFC_USE_ASYNC_SUBQ, IBMVFC_CAN_HANDLE_FPIN, and IBMVFC_YES_SCSI capabilities whenever multi-queue channels are enabled. IBMVFC_YES_SCSI was previously only set for NVMe-enabled configurations; move it to the common multi-queue path so it is always advertised when channels are in use. Signed-off-by: Dave Marquardt Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc-core.c | 82 ++++++++++++++++++++++++++----------- 1 file changed, 58 insertions(+), 24 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c index 76246002ac72..acc11f3a237e 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c @@ -964,6 +964,7 @@ static int ibmvfc_reenable_crq_queue(struct ibmvfc_host *vhost) struct vio_dev *vdev = to_vio_dev(vhost->dev); unsigned long flags; + ibmvfc_dereg_sub_crqs(vhost, &vhost->async_scrqs); ibmvfc_dereg_sub_crqs(vhost, &vhost->scsi_scrqs); ibmvfc_dereg_sub_crqs(vhost, &vhost->nvme_scrqs); @@ -986,6 +987,7 @@ static int ibmvfc_reenable_crq_queue(struct ibmvfc_host *vhost) spin_unlock(vhost->crq.q_lock); spin_unlock_irqrestore(&vhost->host->host_lock, flags); + ibmvfc_reg_sub_crqs(vhost, &vhost->async_scrqs); ibmvfc_reg_sub_crqs(vhost, &vhost->scsi_scrqs); ibmvfc_reg_sub_crqs(vhost, &vhost->nvme_scrqs); @@ -1006,6 +1008,7 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) struct vio_dev *vdev = to_vio_dev(vhost->dev); struct ibmvfc_queue *crq = &vhost->crq; + ibmvfc_dereg_sub_crqs(vhost, &vhost->async_scrqs); ibmvfc_dereg_sub_crqs(vhost, &vhost->scsi_scrqs); ibmvfc_dereg_sub_crqs(vhost, &vhost->nvme_scrqs); @@ -1042,6 +1045,7 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) spin_unlock(vhost->crq.q_lock); spin_unlock_irqrestore(&vhost->host->host_lock, flags); + ibmvfc_reg_sub_crqs(vhost, &vhost->async_scrqs); ibmvfc_reg_sub_crqs(vhost, &vhost->scsi_scrqs); ibmvfc_reg_sub_crqs(vhost, &vhost->nvme_scrqs); @@ -1584,9 +1588,11 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost) if (vhost->mq_enabled || vhost->using_channels) { login_info->capabilities |= cpu_to_be64(IBMVFC_CAN_USE_CHANNELS); + login_info->capabilities |= cpu_to_be64(IBMVFC_USE_ASYNC_SUBQ); + login_info->capabilities |= cpu_to_be64(IBMVFC_CAN_HANDLE_FPIN); + login_info->capabilities |= cpu_to_be64(IBMVFC_YES_SCSI); if (vhost->nvme_enabled) { login_info->capabilities |= cpu_to_be64(IBMVFC_YES_NVMEOF); - login_info->capabilities |= cpu_to_be64(IBMVFC_YES_SCSI); login_info->capabilities |= cpu_to_be64(IBMVFC_CAN_USE_WWPN_ALL); } } @@ -5791,6 +5797,7 @@ static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt) { struct ibmvfc_host *vhost = evt->vhost; struct ibmvfc_channel_setup *setup = vhost->channel_setup_buf; + struct ibmvfc_channels *async = &vhost->async_scrqs; struct ibmvfc_channels *scsi = &vhost->scsi_scrqs; struct ibmvfc_channels *nvme = &vhost->nvme_scrqs; u32 mad_status = be16_to_cpu(evt->xfer_iu->channel_setup.common.status); @@ -5821,6 +5828,7 @@ static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt) for (i = 0; i < nvme->active_queues; i++) nvme->scrqs[i].vios_cookie = be64_to_cpu(setup->channel_handles[scsi->active_queues + i]); + async->scrqs[0].vios_cookie = be64_to_cpu(setup->async_sub_crq_handle); ibmvfc_dbg(vhost, "Using %u SCSI channels\n", scsi->active_queues); @@ -5855,6 +5863,7 @@ static void ibmvfc_channel_setup(struct ibmvfc_host *vhost) struct ibmvfc_event *evt = ibmvfc_get_reserved_event(&vhost->crq); struct ibmvfc_channels *scsi = &vhost->scsi_scrqs; struct ibmvfc_channels *nvme = &vhost->nvme_scrqs; + struct ibmvfc_channels *async = &vhost->async_scrqs; unsigned int scsi_channels = min(scsi->desired_queues, vhost->max_vios_scsi_channels); unsigned int nvme_channels = @@ -5880,6 +5889,7 @@ static void ibmvfc_channel_setup(struct ibmvfc_host *vhost) for (i = 0; i < nvme_channels; i++) setup_buf->channel_handles[scsi_channels + i] = cpu_to_be64(nvme->scrqs[i].cookie); + setup_buf->async_sub_crq_handle = cpu_to_be64(async->scrqs[0].cookie); } ibmvfc_init_event(evt, ibmvfc_channel_setup_done, IBMVFC_MAD_FORMAT); @@ -6727,6 +6737,9 @@ static int ibmvfc_alloc_queue(struct ibmvfc_host *vhost, queue->evt_depth = scsi_qdepth; queue->reserved_depth = IBMVFC_NUM_INTERNAL_SUBQ_REQ; break; + case IBMVFC_ASYNC_SUB_CRQ_FMT: + fmt_size = sizeof(*queue->msgs.async_scrq); + break; default: dev_warn(dev, "Unknown command/response queue message format: %d\n", fmt); return -EINVAL; @@ -6998,6 +7011,7 @@ static int ibmvfc_alloc_channels(struct ibmvfc_host *vhost, struct ibmvfc_channels *channels) { struct ibmvfc_queue *scrq; + enum ibmvfc_msg_fmt fmt; int i, j; int rc = 0; @@ -7005,9 +7019,11 @@ static int ibmvfc_alloc_channels(struct ibmvfc_host *vhost, if (!channels->scrqs) return -ENOMEM; + fmt = (channels->protocol != IBMVFC_PROTO_ASYNC) ? IBMVFC_SUB_CRQ_FMT : + IBMVFC_ASYNC_SUB_CRQ_FMT; for (i = 0; i < channels->max_queues; i++) { scrq = &channels->scrqs[i]; - rc = ibmvfc_alloc_queue(vhost, scrq, IBMVFC_SUB_CRQ_FMT); + rc = ibmvfc_alloc_queue(vhost, scrq, fmt); if (rc) { for (j = i; j > 0; j--) { scrq = &channels->scrqs[j - 1]; @@ -7023,17 +7039,38 @@ static int ibmvfc_alloc_channels(struct ibmvfc_host *vhost, return rc; } +static void ibmvfc_release_channels(struct ibmvfc_host *vhost, + struct ibmvfc_channels *channels) +{ + struct ibmvfc_queue *scrq; + int i; + + if (channels->scrqs) { + for (i = 0; i < channels->max_queues; i++) { + scrq = &channels->scrqs[i]; + ibmvfc_free_queue(vhost, scrq); + } + + kfree(channels->scrqs); + channels->scrqs = NULL; + channels->active_queues = 0; + } +} + static void ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) { ENTER; if (!vhost->mq_enabled) return; - if (ibmvfc_alloc_channels(vhost, &vhost->scsi_scrqs)) { - vhost->do_enquiry = 0; - vhost->mq_enabled = 0; - return; - } + /* register async_sub_crq channel */ + if (ibmvfc_alloc_channels(vhost, &vhost->async_scrqs)) + goto out; + + ibmvfc_reg_sub_crqs(vhost, &vhost->async_scrqs); + + if (ibmvfc_alloc_channels(vhost, &vhost->scsi_scrqs)) + goto deregister_async_sub_crq; ibmvfc_reg_sub_crqs(vhost, &vhost->scsi_scrqs); @@ -7045,29 +7082,23 @@ static void ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) } LEAVE; -} - -static void ibmvfc_release_channels(struct ibmvfc_host *vhost, - struct ibmvfc_channels *channels) -{ - struct ibmvfc_queue *scrq; - int i; - - if (channels->scrqs) { - for (i = 0; i < channels->max_queues; i++) { - scrq = &channels->scrqs[i]; - ibmvfc_free_queue(vhost, scrq); - } + return; - kfree(channels->scrqs); - channels->scrqs = NULL; - channels->active_queues = 0; - } +deregister_async_sub_crq: + ibmvfc_dereg_sub_crqs(vhost, &vhost->async_scrqs); + ibmvfc_release_channels(vhost, &vhost->async_scrqs); +out: + vhost->do_enquiry = 0; + vhost->mq_enabled = 0; + return; } static void ibmvfc_release_sub_crqs(struct ibmvfc_host *vhost) { ENTER; + ibmvfc_dereg_sub_crqs(vhost, &vhost->async_scrqs); + ibmvfc_release_channels(vhost, &vhost->async_scrqs); + if (!vhost->scsi_scrqs.scrqs) return; @@ -7333,6 +7364,9 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) vhost->task_set = 1; vhost->mq_enabled = mq_enabled; + vhost->async_scrqs.desired_queues = 1; + vhost->async_scrqs.max_queues = 1; + vhost->async_scrqs.protocol = IBMVFC_PROTO_ASYNC; vhost->scsi_scrqs.desired_queues = min(shost->nr_hw_queues, nr_scsi_channels); vhost->scsi_scrqs.max_queues = shost->nr_hw_queues; vhost->scsi_scrqs.protocol = IBMVFC_PROTO_SCSI; -- 2.55.0