From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9D363C552B; Fri, 25 Sep 2026 04:48:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790311724; cv=none; b=lEFAHB8ziKHxu+jwDYsXaBrk0U6/73dCofnaUjqzNqrnV2nBkZMCx8Hg01nAk0LOKIJXxJ4jG42ZrW+eAE3N4LjxUwKB+ZSblvuJ4m+9oaH/vLIu1U5KP//xg2W9iUXls+HDx23IMAK4R7eAUNjKTgYnNHMGOjQTGPTbzzT25y8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790311724; c=relaxed/simple; bh=t6r4pf/YsMaDxUAdIRNhB+IIMqmvjY3Pdre48Vtpqog=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EStiB+P+j75jucvyi8J2F7uEC9CvwT7GR0pMe1U8Ta/qu5s+84T+JqpY30QepQpW7C5uO+wYoleYksZ2CMS2D+XTXz4n7HTwjaLOvGGtEb6Feo5RukdzWT3f3Qowsp+yxYpD0105PYb1C7T5OSdZ1bmNWFYGh8zkFsBuwzHahA8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Y/39kOCI; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Y/39kOCI" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 68P4ahgE4105127; Fri, 25 Sep 2026 04:48:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=+j7nTqNEwZtgxCCRu 7SnVrHZbKR8QZSzP8ucB+kUa5k=; b=Y/39kOCId0nA3Cevp6OYXBQep11aqLtoj Ze1e9UxogCfTtOPBeh3ijZMFPTPvvua/6qBo+cOZ2deOzhWXPfvdU2O3vVCi97nW Fm+BR7hDrYhthtNSQBseLyoKm6z711dHff6gHtb0BoCZBhqrpS5mh7YANHAOvkMY OBq69goJ+xw+nOZleMWzBdeTf1KIZmfqGvkPqpbTxbY9OnLmcbjq2mmZM2pP1/8m D/khRK5mzgMBh2XFw0/bHJL/66sEddPKNbQOIjqCSk1hbd6GdpsYJxQQG/D7WvBw GQDi0t8W48F1nwz66XzsF+PxPgvEMM2y9HIgSQ5EXWcnSKzFYnyTQ== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4gske1vcwy-1 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Fri, 25 Sep 2026 04:48:32 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.11/8.18.1.11) with ESMTP id 68P4lgvk3288275; Fri, 25 Sep 2026 04:48:32 GMT Received: from smtprelay04.wdc07v.mail.ibm.com ([172.16.1.71]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4gvbe21260-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 25 Sep 2026 04:48:32 +0000 (GMT) Received: from smtpav02.dal12v.mail.ibm.com (smtpav02.dal12v.mail.ibm.com [10.241.53.101]) by smtprelay04.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 68P4mVu054985034 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Sep 2026 04:48:31 GMT Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D560A5805F; Fri, 25 Sep 2026 04:48:30 +0000 (GMT) Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F3E8258051; Fri, 25 Sep 2026 04:48:29 +0000 (GMT) Received: from li-4c4c4544-0054-3910-8039-c3c04f423534.ibm.com.com (unknown [9.61.142.241]) by smtpav02.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 25 Sep 2026 04:48:29 +0000 (GMT) From: Tyrel Datwyler To: james.bottomley@hansenpartnership.com, martin.petersen@oracle.com Cc: linux-scsi@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, brking@linux.ibm.com, davemarq@linux.ibm.com, Tyrel Datwyler Subject: [PATCH v11 10/11] scsi: ibmvfc: register and use asynchronous sub CRQ for events Date: Thu, 24 Sep 2026 21:48:59 -0700 Message-ID: <20260925044900.414340-11-tyreld@linux.ibm.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260925044900.414340-1-tyreld@linux.ibm.com> References: <20260925044900.414340-1-tyreld@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: gDGEQOqyz8ctLBAQtzMRCwwbvdd5OyB1 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwOTI1MDAxOCBTYWx0ZWRfX+ra64T9Pa0q+ ynu3L84TfF8jpUwfyps+fxccbrCvIuctDQOYE5rrhy4XlUoBuAaAtjfTLDKrdwtxy+vLMlC6sdP TmWDGFqPmMI+dKEwFHVKNvQ8oK6L/JSMXv/6+mOKnWOcdgEUcUnF9V1xkEj3kFqc4vg+GrP24ax 0FgG6Mf8PvZty7o7deCxUVeghVc+22UqvR8xkBL+lwedVZiP5uCTRZhst4WwxgKxoAzOU0aHHO1 yBHZ8fIRLG17Q0h+NOEnt4WdfDYGnX2TkLOHahjN+WbzrAI9tJuoLkCAQHob5uxVRJpLXjV4xNq C0U26rLAP1V/r642S7beQL0UrdUKYDFXQJxg4GFZdni+fWTkBf2QKMHfl0hcZb1tojCEI4gY53h ehgzaFjhFQgATMOUKNURx+/rZR/o6iCU9GiqE1s1XOfVi63iorwq6AWnJN3++LS8kheCmsegjHc 5SMAJDVJ7PqoXR4qGcA== X-Authority-Analysis: v=2.4 cv=O/KsLx9W c=1 sm=1 tr=0 ts=6ab5fd21 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=VdqzKS8jKosA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=VnNF1IyMAAAA:8 a=QSdlH_gVqe-hz-qPZfMA:9 X-Proofpoint-Spam-Info: AW1haW4tMjYwOTI1MDAxOCBTYWx0ZWRfXyvgY+O7SKKji HuewNrqIu7SdG4eFNY8o4FBuUnkPvfnnQXW9BRgDe2BvFDsTnnkMTE+5qevolMD5i6Zu18OD/Wk H+UY/JtFUr+Oa+6KlfRYD2DZfjcBZgg= X-Proofpoint-GUID: gDGEQOqyz8ctLBAQtzMRCwwbvdd5OyB1 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-09-25_02,2026-09-21_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 malwarescore=0 phishscore=0 impostorscore=0 suspectscore=0 bulkscore=0 priorityscore=1501 lowpriorityscore=0 adultscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2609040000 definitions=main-2609250018 From: Dave Marquardt Wire the async sub-CRQ into the sub-CRQ lifecycle so it is allocated, registered, and freed alongside the SCSI channel queues. In ibmvfc_probe(), initialize vhost->async_scrqs with desired_queues=1, max_queues=1, and protocol=IBMVFC_PROTO_ASYNC. Add ibmvfc_release_channels() as a helper to free a channels array and its queue buffers. In ibmvfc_init_sub_crqs(), allocate the async sub-CRQ via ibmvfc_alloc_channels() before allocating the SCSI channels. Register it with the VIOS via ibmvfc_reg_sub_crqs(). Either failure disables multi-queue and aborts init. In ibmvfc_alloc_channels(), select IBMVFC_ASYNC_SUB_CRQ_FMT for async protocol channels instead of IBMVFC_SUB_CRQ_FMT, so the queue is sized for ibmvfc_async_sub_crq entries. In ibmvfc_release_sub_crqs(), ibmvfc_reenable_crq_queue(), and ibmvfc_reset_crq(), deregister and re-register the async sub-CRQ alongside the SCSI channel queues. In ibmvfc_channel_setup_done(), capture the async sub-CRQ handle returned by the VIOS in the channel setup response and store it in vhost->async_scrqs.scrqs[0].vios_cookie. Pass the async sub-CRQ cookie to the VIOS in ibmvfc_channel_setup() via async_sub_crq_handle. In ibmvfc_set_login_info(), advertise IBMVFC_USE_ASYNC_SUBQ, IBMVFC_CAN_HANDLE_FPIN, and IBMVFC_YES_SCSI capabilities whenever multi-queue channels are enabled. IBMVFC_YES_SCSI was previously only set for NVMe-enabled configurations; move it to the common multi-queue path so it is always advertised when channels are in use. Signed-off-by: Dave Marquardt Signed-off-by: Tyrel Datwyler --- drivers/scsi/ibmvscsi/ibmvfc-core.c | 82 ++++++++++++++++++++--------- 1 file changed, 58 insertions(+), 24 deletions(-) diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c index b967c82e9943..a7bdae2c8571 100644 --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c @@ -964,6 +964,7 @@ static int ibmvfc_reenable_crq_queue(struct ibmvfc_host *vhost) struct vio_dev *vdev = to_vio_dev(vhost->dev); unsigned long flags; + ibmvfc_dereg_sub_crqs(vhost, &vhost->async_scrqs); ibmvfc_dereg_sub_crqs(vhost, &vhost->scsi_scrqs); ibmvfc_dereg_sub_crqs(vhost, &vhost->nvme_scrqs); @@ -986,6 +987,7 @@ static int ibmvfc_reenable_crq_queue(struct ibmvfc_host *vhost) spin_unlock(vhost->crq.q_lock); spin_unlock_irqrestore(vhost->host->host_lock, flags); + ibmvfc_reg_sub_crqs(vhost, &vhost->scsi_scrqs); ibmvfc_reg_sub_crqs(vhost, &vhost->scsi_scrqs); ibmvfc_reg_sub_crqs(vhost, &vhost->nvme_scrqs); @@ -1006,6 +1008,7 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) struct vio_dev *vdev = to_vio_dev(vhost->dev); struct ibmvfc_queue *crq = &vhost->crq; + ibmvfc_dereg_sub_crqs(vhost, &vhost->async_scrqs); ibmvfc_dereg_sub_crqs(vhost, &vhost->scsi_scrqs); ibmvfc_dereg_sub_crqs(vhost, &vhost->nvme_scrqs); @@ -1042,6 +1045,7 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost) spin_unlock(vhost->crq.q_lock); spin_unlock_irqrestore(vhost->host->host_lock, flags); + ibmvfc_reg_sub_crqs(vhost, &vhost->async_scrqs); ibmvfc_reg_sub_crqs(vhost, &vhost->scsi_scrqs); ibmvfc_reg_sub_crqs(vhost, &vhost->nvme_scrqs); @@ -1584,9 +1588,11 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost) if (vhost->mq_enabled || vhost->using_channels) { login_info->capabilities |= cpu_to_be64(IBMVFC_CAN_USE_CHANNELS); + login_info->capabilities |= cpu_to_be64(IBMVFC_USE_ASYNC_SUBQ); + login_info->capabilities |= cpu_to_be64(IBMVFC_CAN_HANDLE_FPIN); + login_info->capabilities |= cpu_to_be64(IBMVFC_YES_SCSI); if (vhost->nvme_enabled) { login_info->capabilities |= cpu_to_be64(IBMVFC_YES_NVMEOF); - login_info->capabilities |= cpu_to_be64(IBMVFC_YES_SCSI); login_info->capabilities |= cpu_to_be64(IBMVFC_CAN_USE_WWPN_ALL); } } @@ -5783,6 +5789,7 @@ static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt) { struct ibmvfc_host *vhost = evt->vhost; struct ibmvfc_channel_setup *setup = vhost->channel_setup_buf; + struct ibmvfc_channels *async = &vhost->async_scrqs; struct ibmvfc_channels *scsi = &vhost->scsi_scrqs; struct ibmvfc_channels *nvme = &vhost->nvme_scrqs; u32 mad_status = be16_to_cpu(evt->xfer_iu->channel_setup.common.status); @@ -5813,6 +5820,7 @@ static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt) for (i = 0; i < nvme->active_queues; i++) nvme->scrqs[i].vios_cookie = be64_to_cpu(setup->channel_handles[scsi->active_queues + i]); + async->scrqs[0].vios_cookie = be64_to_cpu(setup->async_sub_crq_handle); ibmvfc_dbg(vhost, "Using %u SCSI channels\n", scsi->active_queues); @@ -5847,6 +5855,7 @@ static void ibmvfc_channel_setup(struct ibmvfc_host *vhost) struct ibmvfc_event *evt = ibmvfc_get_reserved_event(&vhost->crq); struct ibmvfc_channels *scsi = &vhost->scsi_scrqs; struct ibmvfc_channels *nvme = &vhost->nvme_scrqs; + struct ibmvfc_channels *async = &vhost->async_scrqs; unsigned int scsi_channels = min(scsi->desired_queues, vhost->max_vios_scsi_channels); unsigned int nvme_channels = @@ -5872,6 +5881,7 @@ static void ibmvfc_channel_setup(struct ibmvfc_host *vhost) for (i = 0; i < nvme_channels; i++) setup_buf->channel_handles[scsi_channels + i] = cpu_to_be64(nvme->scrqs[i].cookie); + setup_buf->async_sub_crq_handle = cpu_to_be64(async->scrqs[0].cookie); } ibmvfc_init_event(evt, ibmvfc_channel_setup_done, IBMVFC_MAD_FORMAT); @@ -6719,6 +6729,9 @@ static int ibmvfc_alloc_queue(struct ibmvfc_host *vhost, queue->evt_depth = scsi_qdepth; queue->reserved_depth = IBMVFC_NUM_INTERNAL_SUBQ_REQ; break; + case IBMVFC_ASYNC_SUB_CRQ_FMT: + fmt_size = sizeof(*queue->msgs.async_scrq); + break; default: dev_warn(dev, "Unknown command/response queue message format: %d\n", fmt); return -EINVAL; @@ -6993,6 +7006,7 @@ static int ibmvfc_alloc_channels(struct ibmvfc_host *vhost, struct ibmvfc_channels *channels) { struct ibmvfc_queue *scrq; + enum ibmvfc_msg_fmt fmt; int i, j; int rc = 0; @@ -7000,9 +7014,10 @@ static int ibmvfc_alloc_channels(struct ibmvfc_host *vhost, if (!channels->scrqs) return -ENOMEM; + fmt = (channels->protocol != IBMVFC_PROTO_ASYNC) ? IBMVFC_SUB_CRQ_FMT : IBMVFC_ASYNC_SUB_CRQ_FMT; for (i = 0; i < channels->max_queues; i++) { scrq = &channels->scrqs[i]; - rc = ibmvfc_alloc_queue(vhost, scrq, IBMVFC_SUB_CRQ_FMT); + rc = ibmvfc_alloc_queue(vhost, scrq, fmt); if (rc) { for (j = i; j > 0; j--) { scrq = &channels->scrqs[j - 1]; @@ -7018,17 +7033,39 @@ static int ibmvfc_alloc_channels(struct ibmvfc_host *vhost, return rc; } +static void ibmvfc_release_channels(struct ibmvfc_host *vhost, + struct ibmvfc_channels *channels) +{ + struct ibmvfc_queue *scrq; + int i; + + if (channels->scrqs) { + for (i = 0; i < channels->max_queues; i++) { + scrq = &channels->scrqs[i]; + ibmvfc_free_queue(vhost, scrq); + } + + kfree(channels->scrqs); + channels->scrqs = NULL; + channels->active_queues = 0; + } +} + static void ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) { + ENTER; if (!vhost->mq_enabled) return; - if (ibmvfc_alloc_channels(vhost, &vhost->scsi_scrqs)) { - vhost->do_enquiry = 0; - vhost->mq_enabled = 0; - return; - } + /* register async_sub_crq channel */ + if (ibmvfc_alloc_channels(vhost, &vhost->async_scrqs)) + goto out; + + ibmvfc_reg_sub_crqs(vhost, &vhost->async_scrqs); + + if (ibmvfc_alloc_channels(vhost, &vhost->scsi_scrqs)) + goto deregister_async_sub_crq; ibmvfc_reg_sub_crqs(vhost, &vhost->scsi_scrqs); @@ -7040,29 +7077,23 @@ static void ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost) } LEAVE; -} - -static void ibmvfc_release_channels(struct ibmvfc_host *vhost, - struct ibmvfc_channels *channels) -{ - struct ibmvfc_queue *scrq; - int i; - - if (channels->scrqs) { - for (i = 0; i < channels->max_queues; i++) { - scrq = &channels->scrqs[i]; - ibmvfc_free_queue(vhost, scrq); - } + return; - kfree(channels->scrqs); - channels->scrqs = NULL; - channels->active_queues = 0; - } +deregister_async_sub_crq: + ibmvfc_dereg_sub_crqs(vhost, &vhost->async_scrqs); + ibmvfc_release_channels(vhost, &vhost->async_scrqs); +out: + vhost->do_enquiry = 0; + vhost->mq_enabled = 0; + return; } static void ibmvfc_release_sub_crqs(struct ibmvfc_host *vhost) { ENTER; + ibmvfc_dereg_sub_crqs(vhost, &vhost->async_scrqs); + ibmvfc_release_channels(vhost, &vhost->async_scrqs); + if (!vhost->scsi_scrqs.scrqs) return; @@ -7328,6 +7359,9 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id) vhost->task_set = 1; vhost->mq_enabled = mq_enabled; + vhost->async_scrqs.desired_queues = 1; + vhost->async_scrqs.max_queues = 1; + vhost->async_scrqs.protocol = IBMVFC_PROTO_ASYNC; vhost->scsi_scrqs.desired_queues = min(shost->nr_hw_queues, nr_scsi_channels); vhost->scsi_scrqs.max_queues = shost->nr_hw_queues; vhost->scsi_scrqs.protocol = IBMVFC_PROTO_SCSI; -- 2.55.0