From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BCC836680C for ; Tue, 9 Jun 2026 20:07:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781035656; cv=none; b=o9vrffD1oWfCEBZQWofkCJoN3nck0+S87772gP9jqTjouc0L1lER+Ui2VBt3tOkUiBpedC1oMMnorZ1/08hvKVZOLH+CDw79rmf0jhECNY0swr8SbZPQ2UkjmDCbL16sxhKuwUMIuZ2uN6hxn5d69clDbVlAOm7RWX+bHBlOMwA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781035656; c=relaxed/simple; bh=MXFwQ8DiiM/SjxmqYpcfBnRgrt3i63ILwoSaxs7gk8s=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=uKz6c/iqurhYNNgTsWf6HamgH6IJVtWTpzg16XY9g19gNi59KHaYXaNKAhEkPnWOPLbmHUgLiN+LIIj/i/Z5601m9dmgVLRjr2G3I0oCDxgS6GJ5fQUS29Mn8PVBsTJc/RJOpkALzk76/8WfbnJ9oHhp1u9kd+c10K5DiwVfKqs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KbThtSC6; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KbThtSC6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F17DD1F00893; Tue, 9 Jun 2026 20:07:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1781035655; bh=V9w5gs711UCSXeS2EY2vTrWuTnDtwk1iyZbBn9SnfCo=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=KbThtSC6yGNFVLNdkjIErdeOxddOR4UABWFDKcsD281GD8LVZLWopWKr5L9EC49nZ rD356upeKjIBf+a6BlqoTSPmDRn3sst4h+fnqWvWpL3rEy9/Ld88UKSv/SIo6XJj7H 3JnTQQz/DS7HfT05I2QOTgWdt79zQEAjeiQJuKbxosZzM1Fl0eB0EihGoByu762G0r LeIbs8tvAHrXYV6wjm5S9jqUywj1nTVwocMBydozOrG4ckcztBhZeFyrRpxFGRHh4f U8bWzickN2Vrd3kZqiF6/RkPEcg7uqEmSKsVO9yHE+FshqAuqBmN45yG2kHRcJVBhn Ksmm1mBAldiqQ== Date: Tue, 9 Jun 2026 14:07:33 -0600 From: Keith Busch To: Surabhi Gogte Cc: axboe@kernel.dk, hch@lst.de, sagi@grimberg.me, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, mkhalfella@purestorage.com, randyj@purestorage.com Subject: Re: [PATCH v2] nvme-rdma: parallelize I/O queue allocation and startup Message-ID: References: <20260604195321.2232838-1-sgogte@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260604195321.2232838-1-sgogte@purestorage.com> On Thu, Jun 04, 2026 at 01:53:21PM -0600, Surabhi Gogte wrote: > Refactor nvme rdma I/O queue setup to use async API, combining > allocation and startup into a single parallel operation per queue. This > reduces connection and reconnection setup time when there are delays in > establishing connections, which is especially important for > high-core-count hosts. Mostly looks fine. > @@ -16,6 +16,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -125,6 +126,7 @@ struct nvme_rdma_ctrl { > struct nvme_ctrl ctrl; > bool use_inline_data; > u32 io_queues[HCTX_MAX_TYPES]; > + atomic_t qsetup_err; > }; This new field serves only to propogate an error from a local context, so I don't want to introduce a new field for it at this scope. I prefer you declare a special context struct for it to use with the async usage: struct nvme_rdma_setup_ctx { struct nvme_rdma_queue *queue; int *err; }; And then make that the cookie passed to the async setup. I don't think it needs to be atomic here either: we really don't care if we see the first or last error, so forcing a cmpxchg for the first one is a bit overkill; you can just do READ/WRITE_ONCE instead and accept the race.