From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from verein.lst.de (verein.lst.de [213.95.11.211])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by smtp.subspace.kernel.org (Postfix) with ESMTPS id 312DF175A92
	for <linux-kernel@vger.kernel.org>; Fri, 26 Jun 2026 07:40:08 +0000 (UTC)
Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.95.11.211
ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;
	t=1782459609; cv=none; b=AgCcreC+9hylvOiOydqgmsdl+tFogVa2u7GvBqZRVgnyb+yO2qGyNOYsktowDkOGqOsoZWB5yfx1dve2F6l9jfo4cHQ2pmU0EEqzC+dOsnFZYpHc+85OVC6G6fCHxhSkBsVm4nk4RNq8GJuOfoX2Abl/qECcZvze9BK6t9GEh9I=
ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org;
	s=arc-20240116; t=1782459609; c=relaxed/simple;
	bh=/VumXm8jTUEJWC4WeZfWqVC4h6jiwTpTELoTcnmXGvc=;
	h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version:
	 Content-Type:Content-Disposition:In-Reply-To; b=nw+YKnRK/7trOnKJi0qwo16gwlD5YOI92/glcyBWDPpAc5LtNX+teXc0Tc+VeQx6RYScVjT6kFGkYU2p5onMydpTh8T3b75wCigBjEpGm9uj+EW4Q1iP07NNP4NqAvNIaBT5VZejf51gDh3aVvzOSk+HNiWjf5/xO6443Bc28ME=
ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lst.de; spf=pass smtp.mailfrom=lst.de; arc=none smtp.client-ip=213.95.11.211
Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=lst.de
Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=lst.de
Received: by verein.lst.de (Postfix, from userid 2407)
	id 85E5B68AFE; Fri, 26 Jun 2026 09:40:04 +0200 (CEST)
Date: Fri, 26 Jun 2026 09:40:04 +0200
From: Christoph Hellwig <hch@lst.de>
To: Surabhi Gogte <sgogte@purestorage.com>
Cc: Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@kernel.dk>,
	Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>,
	linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org,
	mkhalfella@purestorage.com, randyj@purestorage.com,
	adailey@purestorage.com
Subject: Re: [PATCH v3 2/2] nvme-rdma: parallelize I/O queue allocation and
 startup
Message-ID: <20260626074004.GB12072@lst.de>
References: <20260625212722.1302344-1-sgogte@purestorage.com> <20260625212722.1302344-3-sgogte@purestorage.com>
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20260625212722.1302344-3-sgogte@purestorage.com>
User-Agent: Mutt/1.5.17 (2007-11-01)

On Thu, Jun 25, 2026 at 03:27:22PM -0600, Surabhi Gogte wrote:
> -static int nvme_rdma_alloc_io_queues(struct nvme_rdma_ctrl *ctrl)
> +static int nvme_rdma_setup_io_queues(struct nvme_rdma_ctrl *ctrl, unsigned int first,

Overly long line.

> +		unsigned int last, size_t queue_size)
>  {
> +	ASYNC_DOMAIN_EXCLUSIVE(queue_domain);
> +	struct nvme_rdma_setup_ctx *ctxs;
> +	int nr_queues = last - first;
> +	int err = 0, i, ret;
>  
> +	ctxs = kmalloc_array(nr_queues, sizeof(*ctxs), GFP_KERNEL);

This should use kmalloc_objs in the brave new world.

> +	if (!ctxs)
>  		return -ENOMEM;
>  
> +	for (i = 0; i < nr_queues; i++) {
> +		struct nvme_rdma_queue *queue = &ctrl->queues[first + i];
> +
> +		queue->ctrl = ctrl;
> +		queue->queue_size = queue_size;
> +
> +		ctxs[i].queue = queue;
> +		ctxs[i].err = &err;
> +		async_schedule_domain(nvme_rdma_setup_queue_async, &ctxs[i],
> +				&queue_domain);
>  	}
>  
> +	async_synchronize_full_domain(&queue_domain);
> +	kfree(ctxs);

It would nice if the async domain had a way to do the error propagation.
Well, that would be a nice follow on if you're interested.

Talking about follow ons:  do you plan to do a similar change
to nvme-tcp?  It would be great to keep the setup path for
both in sync.