[PATCH] RDMA/rtrs-clt: Fix CQ pool leak when connect is interrupted

[PATCH] RDMA/rtrs-clt: Fix CQ pool leak when connect is interrupted

[Quanye Yang via B4 Relay]

From: Quanye Yang <quanyeyang@proton.me>

The client borrows shared CQ credits in the ADDR_RESOLVED handler via
ib_cq_pool_get(), before the peer is connected. create_cm() can return
-ERESTARTSYS from wait_event_interruptible_timeout() without destroying
the CM ID. The init_conns() and stop-and-destroy paths then call
destroy_con_cq_qp() while cq is still NULL (no PUT) and only afterwards
rdma_destroy_id().

CMA serializes the handler against rdma_destroy_id() with handler_mutex,
but that does not order the GET against destroy_con_cq_qp(). If
ADDR_RESOLVED has already passed the DESTROYING check, it can take
con_mutex, GET credits, and then lose the con to kfree. Device
unregister later hits WARN_ON(cq->cqe_used) in ib_cq_pool_cleanup().

Set a per-connection flag under con_mutex before CQ/QP teardown so a
racing ADDR_RESOLVED cannot borrow credits after teardown has begun.

Reported-by: syzbot+d396918a29afb8543e1c@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=d396918a29afb8543e1c
Fixes: 3b89e92c2a95 ("RDMA/rtrs: Use new shared CQ mechanism")
Signed-off-by: Quanye Yang <quanyeyang@proton.me>
---
Fix a syzbot WARNING in ib_cq_pool_cleanup(): an ADDR_RESOLVED handler
can ib_cq_pool_get() after connection teardown has already skipped the
matching PUT.

Reproduced on rxe with rnbd-client only (no rtrs server): write
map_device with path=ip:127.0.0.1, interrupt create_cm() with a signal,
then rdma link delete. Device unregister no longer reports leftover
cqe_used after this change.
---
 drivers/infiniband/ulp/rtrs/rtrs-clt.c | 8 ++++++++
 drivers/infiniband/ulp/rtrs/rtrs-clt.h | 2 ++
 2 files changed, 10 insertions(+)

diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
index 7b2c51ae614f..eac38b57b00d 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
@@ -1732,6 +1732,8 @@ static void destroy_con_cq_qp(struct rtrs_clt_con *con)
 	/*
 	 * Be careful here: destroy_con_cq_qp() can be called even
 	 * create_con_cq_qp() failed, see comments there.
+	 * Caller must set con->destroyed under this lock first so a
+	 * racing ADDR_RESOLVED cannot ib_cq_pool_get() after we PUT/SKIP.
 	 */
 	lockdep_assert_held(&con->con_mutex);
 	rtrs_cq_qp_destroy(&con->c);
@@ -1766,6 +1768,10 @@ static int rtrs_rdma_addr_resolved(struct rtrs_clt_con *con)
 	int err;
 
 	mutex_lock(&con->con_mutex);
+	if (con->destroyed) {
+		mutex_unlock(&con->con_mutex);
+		return -ECONNABORTED;
+	}
 	err = create_con_cq_qp(con);
 	mutex_unlock(&con->con_mutex);
 	if (err) {
@@ -2221,6 +2227,7 @@ static void rtrs_clt_stop_and_destroy_conns(struct rtrs_clt_path *clt_path)
 			break;
 		con = to_clt_con(clt_path->s.con[cid]);
 		mutex_lock(&con->con_mutex);
+		con->destroyed = true;
 		destroy_con_cq_qp(con);
 		mutex_unlock(&con->con_mutex);
 		destroy_cm(con);
@@ -2387,6 +2394,7 @@ static int init_conns(struct rtrs_clt_path *clt_path)
 		if (con->c.cm_id) {
 			stop_cm(con);
 			mutex_lock(&con->con_mutex);
+			con->destroyed = true;
 			destroy_con_cq_qp(con);
 			mutex_unlock(&con->con_mutex);
 			destroy_cm(con);
diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.h b/drivers/infiniband/ulp/rtrs/rtrs-clt.h
index 1305601a6251..ad64f4517c4b 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt.h
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.h
@@ -75,6 +75,8 @@ struct rtrs_clt_con {
 	unsigned int		cpu;
 	struct mutex		con_mutex;
 	int			cm_err;
+	/* Set under con_mutex before CQ/QP teardown. */
+	bool			destroyed;
 };
 
 /**

---
base-commit: 08dbfad3f5040f5bdb6c529da20d6d4e81fefd72
change-id: 20260830-rdma-rtrs-clt-cq-pool-leak-78f1b077e341

Best regards,
--  
Quanye Yang <quanyeyang@proton.me>

Re: [PATCH] RDMA/rtrs-clt: Fix CQ pool leak when connect is interrupted

[Leon Romanovsky]

On Sun, Aug 30, 2026 at 03:09:55PM +0800, Quanye Yang via B4 Relay wrote:
> From: Quanye Yang <quanyeyang@proton.me>
> 
> The client borrows shared CQ credits in the ADDR_RESOLVED handler via
> ib_cq_pool_get(), before the peer is connected. create_cm() can return
> -ERESTARTSYS from wait_event_interruptible_timeout() without destroying
> the CM ID. The init_conns() and stop-and-destroy paths then call
> destroy_con_cq_qp() while cq is still NULL (no PUT) and only afterwards
> rdma_destroy_id().
> 
> CMA serializes the handler against rdma_destroy_id() with handler_mutex,
> but that does not order the GET against destroy_con_cq_qp(). If
> ADDR_RESOLVED has already passed the DESTROYING check, it can take
> con_mutex, GET credits, and then lose the con to kfree. Device
> unregister later hits WARN_ON(cq->cqe_used) in ib_cq_pool_cleanup().
> 
> Set a per-connection flag under con_mutex before CQ/QP teardown so a
> racing ADDR_RESOLVED cannot borrow credits after teardown has begun.
> 
> Reported-by: syzbot+d396918a29afb8543e1c@syzkaller.appspotmail.com
> Closes: https://syzkaller.appspot.com/bug?extid=d396918a29afb8543e1c
> Fixes: 3b89e92c2a95 ("RDMA/rtrs: Use new shared CQ mechanism")
> Signed-off-by: Quanye Yang <quanyeyang@proton.me>
> ---
> Fix a syzbot WARNING in ib_cq_pool_cleanup(): an ADDR_RESOLVED handler
> can ib_cq_pool_get() after connection teardown has already skipped the
> matching PUT.
> 
> Reproduced on rxe with rnbd-client only (no rtrs server): write
> map_device with path=ip:127.0.0.1, interrupt create_cm() with a signal,
> then rdma link delete. Device unregister no longer reports leftover
> cqe_used after this change.
> ---
>  drivers/infiniband/ulp/rtrs/rtrs-clt.c | 8 ++++++++
>  drivers/infiniband/ulp/rtrs/rtrs-clt.h | 2 ++
>  2 files changed, 10 insertions(+)

Haris, Jack?

Thanks

Re: [PATCH] RDMA/rtrs-clt: Fix CQ pool leak when connect is interrupted

[Jinpu Wang]

Hi Leon, hi Quanye,

I had a look at the report. The syzbot bug seems to be triggered with
smbdirect (ib_dev[syz2] removed), so I'm not sure how this change in
RTRS client would fix that issue.

Thanks,
Jinpu

On Wed, Sep 2, 2026 at 6:25 PM Leon Romanovsky <leon@kernel.org> wrote:
>
> On Sun, Aug 30, 2026 at 03:09:55PM +0800, Quanye Yang via B4 Relay wrote:
> > From: Quanye Yang <quanyeyang@proton.me>
> >
> > The client borrows shared CQ credits in the ADDR_RESOLVED handler via
> > ib_cq_pool_get(), before the peer is connected. create_cm() can return
> > -ERESTARTSYS from wait_event_interruptible_timeout() without destroying
> > the CM ID. The init_conns() and stop-and-destroy paths then call
> > destroy_con_cq_qp() while cq is still NULL (no PUT) and only afterwards
> > rdma_destroy_id().
> >
> > CMA serializes the handler against rdma_destroy_id() with handler_mutex,
> > but that does not order the GET against destroy_con_cq_qp(). If
> > ADDR_RESOLVED has already passed the DESTROYING check, it can take
> > con_mutex, GET credits, and then lose the con to kfree. Device
> > unregister later hits WARN_ON(cq->cqe_used) in ib_cq_pool_cleanup().
> >
> > Set a per-connection flag under con_mutex before CQ/QP teardown so a
> > racing ADDR_RESOLVED cannot borrow credits after teardown has begun.
> >
> > Reported-by: syzbot+d396918a29afb8543e1c@syzkaller.appspotmail.com
> > Closes: https://syzkaller.appspot.com/bug?extid=d396918a29afb8543e1c
> > Fixes: 3b89e92c2a95 ("RDMA/rtrs: Use new shared CQ mechanism")
> > Signed-off-by: Quanye Yang <quanyeyang@proton.me>
> > ---
> > Fix a syzbot WARNING in ib_cq_pool_cleanup(): an ADDR_RESOLVED handler
> > can ib_cq_pool_get() after connection teardown has already skipped the
> > matching PUT.
> >
> > Reproduced on rxe with rnbd-client only (no rtrs server): write
> > map_device with path=ip:127.0.0.1, interrupt create_cm() with a signal,
> > then rdma link delete. Device unregister no longer reports leftover
> > cqe_used after this change.
> > ---
> >  drivers/infiniband/ulp/rtrs/rtrs-clt.c | 8 ++++++++
> >  drivers/infiniband/ulp/rtrs/rtrs-clt.h | 2 ++
> >  2 files changed, 10 insertions(+)
>
> Haris, Jack?
>
> Thanks

Re: [PATCH] RDMA/rtrs-clt: Fix CQ pool leak when connect is interrupted

[quanyeyang]

On Thursday, September 3rd, 2026 at AM 12:48, Jinpu Wang <jinpu.wang@cloud.ionos.com> wrote:

> Hi Leon, hi Quanye,
> 
> I had a look at the report. The syzbot bug seems to be triggered with
> smbdirect (ib_dev[syz2] removed), so I'm not sure how this change in
> RTRS client would fix that issue.

Hi Jinpu,

The smbdirect line is from ib_client->remove() during disable_device(),
which always runs before ib_cq_pool_cleanup(). smbdirect just logs that
the IB device is going away; it does not use ib_cq_pool_get().

rtrs is not an ib_client, so it does not appear in that remove loop.
The leftover cqe_used comes from rtrs-clt borrowing pool credits in
ADDR_RESOLVED and then dropping the con without PUT, which we hit
locally with rnbd-client only (no smbdirect traffic): map_device with
path=ip:127.0.0.1, interrupt create_cm(), then rdma link delete.

The same "ib_dev[...] removed" line shows up in that repro right before
the WARN, for the same reason it shows up on syzbot.

Thanks,
Quanye

> 
> Thanks,
> Jinpu

Re: [PATCH] RDMA/rtrs-clt: Fix CQ pool leak when connect is interrupted

[Jinpu Wang]

On Thu, Sep 3, 2026 at 6:06 AM quanyeyang <quanyeyang@proton.me> wrote:
>
> On Thursday, September 3rd, 2026 at AM 12:48, Jinpu Wang <jinpu.wang@cloud.ionos.com> wrote:
>
> > Hi Leon, hi Quanye,
> >
> > I had a look at the report. The syzbot bug seems to be triggered with
> > smbdirect (ib_dev[syz2] removed), so I'm not sure how this change in
> > RTRS client would fix that issue.
>
> Hi Jinpu,
>
> The smbdirect line is from ib_client->remove() during disable_device(),
> which always runs before ib_cq_pool_cleanup(). smbdirect just logs that
> the IB device is going away; it does not use ib_cq_pool_get().
>
> rtrs is not an ib_client, so it does not appear in that remove loop.
> The leftover cqe_used comes from rtrs-clt borrowing pool credits in
> ADDR_RESOLVED and then dropping the con without PUT, which we hit
> locally with rnbd-client only (no smbdirect traffic): map_device with
> path=ip:127.0.0.1, interrupt create_cm(), then rdma link delete.
>
> The same "ib_dev[...] removed" line shows up in that repro right before
> the WARN, for the same reason it shows up on syzbot.
>
> Thanks,
> Quanye
>
> >
> > Thanks,
> > Jinpu
>
>
Hi Quanye,

Thanks for the detailed explanation. That makes sense now.

The patch looks good to me:

Reviewed-by: Jack Wang <jinpu.wang@cloud.ionos.com>

Thanks,

Re: [PATCH] RDMA/rtrs-clt: Fix CQ pool leak when connect is interrupted

[Leon Romanovsky]

On Sun, 30 Aug 2026 15:09:55 +0800, Quanye Yang wrote:
> The client borrows shared CQ credits in the ADDR_RESOLVED handler via
> ib_cq_pool_get(), before the peer is connected. create_cm() can return
> -ERESTARTSYS from wait_event_interruptible_timeout() without destroying
> the CM ID. The init_conns() and stop-and-destroy paths then call
> destroy_con_cq_qp() while cq is still NULL (no PUT) and only afterwards
> rdma_destroy_id().
> 
> [...]

Applied, thanks!

[1/1] RDMA/rtrs-clt: Fix CQ pool leak when connect is interrupted
      https://git.kernel.org/rdma/rdma/c/2ae16aaa78b5ed

Best regards,
-- 
Leon Romanovsky <leon@kernel.org>