[PATCH] io_uring/zcrx: requeue multishot receives stopped by a local resource

[Junyuan Feng <a@lollipopkit.com>]

A multishot RECV_ZC can go idle with unread TCP data when a receiver-local
resource runs out. An empty copy-fallback niov freelist returns -ENOMEM;
a full CQ returns -ENOSPC. After partial progress the stop reason is
lost: io_zcrx_copy_chunk() and io_zcrx_recv_skb() report the copied
bytes, and tcp_read_sock() drops an error from a later call once earlier
data was consumed. The edge-triggered request is then not requeued, so
it cannot consume the remaining data until another socket event arrives.

Record in io_zcrx_recv_skb() when a walk stops before consuming the
length it was offered, and requeue if the pass still returned data. The
failed chunk is first on the next pass. The resource is usually still
exhausted by then, so that pass fails at once and ends the request with
an error CQE, which the application must handle by re-arming. The
request does not spin, and the stall becomes a visible error. Keep the
existing skb limit and SOCK_DONE handling.

Fixes: 11ed914bbf94 ("io_uring/zcrx: add io_recvzc request")
Fixes: bc57c7d36c4c ("io_uring/zcrx: add copy fallback")
Cc: stable@vger.kernel.org
Assisted-by: Codex:gpt-6-sol
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Junyuan Feng <a@lollipopkit.com>
---
Reproduction and validation (measured on this revision):
- Loopback CQE-face A/B (mid-skb path), v7.2.6 + this fix behind a test-only
  runtime switch (one binary, both arms), 64 connections, large area so the
  pool stays healthy, 64-entry CQ, 10 runs per arm:
    fix off: 10/10 runs stalled (allocation_errors 0 -- CQ ring, not pool)
    fix on:  0/10 runs stalled
- On header-split hardware an skb usually carries a single frag, so resource
  exhaustion is expected to fall on an skb boundary -- the case a multi-frag
  loopback run rarely samples. Validated on GCP c4a (gve, TCP data split),
  v7.2.6 + this fix behind a test-only runtime switch so one binary runs both
  arms; pool face, 64 connections, 4.03 MiB area, copy-fallback path, 3 runs
  per arm (numbers in parentheses are stalled connections):
    fix off: 3/3 runs stalled (1, 10, 9 connections)
    fix on:  0/3 runs stalled, 64/64 delivered
  Allocation errors were 7141-14995 per run with the fix off and 4796-5213
  with it on: the pool was exhausted in both arms, but with the fix
  exhausted requests end with an error CQE that the receiver re-arms and no
  connection stalls.
  The off arm behaves as upstream (same binary, fix disabled), so this shows
  the fix clears the stall; it does not separate the boundary case from the
  mid-skb case.
- iou-zcrx selftest (test_zcrx data path, zero-copy on the bound queue) on the
  same kernel passes with the fix both on and off.
- Recovery needs an explicit cancel: closing a stalled receiver fd leaves the
  socket established while the ring is alive; a cancel + rearm recovers it.

 io_uring/zcrx.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/io_uring/zcrx.c b/io_uring/zcrx.c
index 1b3b11405..beb35077f 100644
--- a/io_uring/zcrx.c
+++ b/io_uring/zcrx.c
@@ -374,6 +374,7 @@ struct io_zcrx_args {
 	struct io_kiocb		*req;
 	struct io_zcrx_ifq	*ifq;
 	unsigned		nr_skbs;
+	bool			stopped_early;
 };
 
 static const struct memory_provider_ops io_uring_pp_zc_ops;
@@ -1899,6 +1900,9 @@ io_zcrx_recv_skb(read_descriptor_t *desc, struct sk_buff *skb,
 	}
 
 out:
+	/* Bytes left in len mean an error stopped the walk early. */
+	if (len)
+		args->stopped_early = true;
 	if (offset == start_off)
 		return ret;
 	desc->count -= (offset - start_off);
@@ -1935,8 +1939,9 @@ static int io_zcrx_tcp_recvmsg(struct io_kiocb *req, struct io_zcrx_ifq *ifq,
 			ret = -ENOTCONN;
 		else
 			ret = -EAGAIN;
-	} else if (unlikely(args.nr_skbs > IO_SKBS_PER_CALL_LIMIT) &&
-		   (issue_flags & IO_URING_F_MULTISHOT)) {
+	} else if ((issue_flags & IO_URING_F_MULTISHOT) &&
+		   (unlikely(args.nr_skbs > IO_SKBS_PER_CALL_LIMIT) ||
+		    args.stopped_early)) {
 		ret = IOU_REQUEUE;
 	} else if (sock_flag(sk, SOCK_DONE)) {
 		/* Make it to retry until it finally gets 0. */

base-commit: ab394388d05977f369e8e8d1beceae47fc3c5e72
-- 
2.54.0 (Apple Git-157)