mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH net v2 0/2] ionic: fix Tx stalls on TSO skbs larger than the Tx ring
@ 2026-11-17 19:18 Nikhil P. Rao
  2026-11-17 19:18 ` [PATCH net v2 1/2] ionic: reject Tx ring sizes below the minimum Nikhil P. Rao
  2026-11-17 19:18 ` [PATCH net v2 2/2] ionic: limit TSO segments to the minimum Tx ring size Nikhil P. Rao
  0 siblings, 2 replies; 4+ messages in thread
From: Nikhil P. Rao @ 2026-11-17 19:18 UTC (permalink / raw)
  To: Brett Creeley, Andrew Lunn, David S. Miller, Eric Dumazet,
	Jakub Kicinski, Paolo Abeni
  Cc: netdev, linux-kernel, Breno Leitao, Eric Joyner,
	Abhijit Gangurde, stable, Nikhil P. Rao

ionic uses one Tx descriptor per TSO segment but never sets
tso_max_segs, so the stack can build TSO skbs with more segments than
the Tx ring holds. Such an skb can never be posted: the queue stops,
nothing wakes it, and the netdev watchdog fires. Guest TCP traffic with
an MSS below 64 hit this on the default 1024-entry ring.

Patch 1 rejects Tx ring sizes below 64. The completion path wakes a
stopped Tx queue only once 44 descriptors are free, so a smaller ring
never wakes once stopped.

Patch 2 raises the minimum Tx ring size to 128 and sets tso_max_segs
to 127, so every Tx ring can hold the largest TSO skb and larger GSO
skbs are segmented in software.

The two patches fix separate bugs; patch 2 raises the Tx ring minimum
that patch 1 adds from 64 to 128.

Changes in v2:
- Patch 2: use a fixed limit of 127 segments and a 128-entry minimum
  Tx ring instead of following the Tx ring size, as suggested by Jakub.
  This also covers the hwstamp and kdump Tx rings.
- Add testing notes to both patches, and Cc stable.
- Link to v1: https://lore.kernel.org/all/20261107-b4-ionic-tso-max-segs-v1-0-67c248a5b681@amd.com/

This series was developed with the help of an LLM-based coding
assistant (Claude).

---
Nikhil P. Rao (2):
      ionic: reject Tx ring sizes below the minimum
      ionic: limit TSO segments to the minimum Tx ring size

 drivers/net/ethernet/pensando/ionic/ionic_dev.h     |  3 ++-
 drivers/net/ethernet/pensando/ionic/ionic_ethtool.c |  7 +++++++
 drivers/net/ethernet/pensando/ionic/ionic_lif.c     | 10 ++++++----
 3 files changed, 15 insertions(+), 5 deletions(-)
---
base-commit: a2fd526d644233d6a892c28173f07c61e21e4c93
change-id: 20261107-b4-ionic-tso-max-segs-057bde9cfe49

Best regards,
--  
Nikhil P. Rao <nikhil.rao@amd.com>


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH net v2 1/2] ionic: reject Tx ring sizes below the minimum
  2026-11-17 19:18 [PATCH net v2 0/2] ionic: fix Tx stalls on TSO skbs larger than the Tx ring Nikhil P. Rao
@ 2026-11-17 19:18 ` Nikhil P. Rao
  2026-11-17 19:18 ` [PATCH net v2 2/2] ionic: limit TSO segments to the minimum Tx ring size Nikhil P. Rao
  1 sibling, 0 replies; 4+ messages in thread
From: Nikhil P. Rao @ 2026-11-17 19:18 UTC (permalink / raw)
  To: Brett Creeley, Andrew Lunn, David S. Miller, Eric Dumazet,
	Jakub Kicinski, Paolo Abeni
  Cc: netdev, linux-kernel, Breno Leitao, Eric Joyner,
	Abhijit Gangurde, stable, Nikhil P. Rao

ionic_tx_cq_service() wakes a stopped Tx queue only once
IONIC_TSO_DESCS_NEEDED (44) descriptors are free. The threshold is
sized for a 64 KB TSO skb at a 1500 byte MSS and assumes the ring can
have that many free; a Tx ring of 32 or fewer descriptors cannot, so a
stopped queue is never woken and the netdev watchdog fires.
ionic_set_ringparam() only checks that the size is a power of 2, and
the ethtool core only checks the maximum, so such sizes are accepted.

Reject Tx ring sizes below IONIC_MIN_TXRX_DESC.

The stall was confirmed from the wake and stop thresholds, not
reproduced. Tested on an AMD Pensando NIC: "ethtool -G <dev> tx 32" is
rejected and leaves the ring unchanged, and "ethtool -G <dev> tx 64" is
accepted.

Fixes: 061b9bedbef1 ("ionic: Rework Tx start/stop flow")
Cc: stable@vger.kernel.org
Reported-by: sashiko-bot@kernel.org # Running on a local machine
Signed-off-by: Nikhil P. Rao <nikhil.rao@amd.com>
Reviewed-by: Brett Creeley <brett.creeley@amd.com>
---
 drivers/net/ethernet/pensando/ionic/ionic_ethtool.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
index 119ac720c078..3518fbf5882d 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
@@ -708,6 +708,13 @@ static int ionic_set_ringparam(struct net_device *netdev,
 		return -EINVAL;
 	}
 
+	if (ring->tx_pending < IONIC_MIN_TXRX_DESC) {
+		NL_SET_ERR_MSG_FMT_MOD(extack,
+				       "Tx descriptor count must be at least %d",
+				       IONIC_MIN_TXRX_DESC);
+		return -EINVAL;
+	}
+
 	/* if nothing to do return success */
 	if (ring->tx_pending == lif->ntxq_descs &&
 	    ring->rx_pending == lif->nrxq_descs &&

-- 
2.43.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* [PATCH net v2 2/2] ionic: limit TSO segments to the minimum Tx ring size
  2026-11-17 19:18 [PATCH net v2 0/2] ionic: fix Tx stalls on TSO skbs larger than the Tx ring Nikhil P. Rao
  2026-11-17 19:18 ` [PATCH net v2 1/2] ionic: reject Tx ring sizes below the minimum Nikhil P. Rao
@ 2026-11-17 19:18 ` Nikhil P. Rao
  2026-11-20 19:58   ` Rao, Nikhil
  1 sibling, 1 reply; 4+ messages in thread
From: Nikhil P. Rao @ 2026-11-17 19:18 UTC (permalink / raw)
  To: Brett Creeley, Andrew Lunn, David S. Miller, Eric Dumazet,
	Jakub Kicinski, Paolo Abeni
  Cc: netdev, linux-kernel, Breno Leitao, Eric Joyner,
	Abhijit Gangurde, stable, Nikhil P. Rao

ionic uses one Tx descriptor per TSO segment, and a Tx ring holds at
most num_descs - 1 of them. The driver does not set tso_max_segs, so
the stack can hand it GSO skbs with more segments than the Tx ring
can hold. ionic_tx() then stops the queue waiting for descriptors
that can never become free, and the netdev watchdog fires repeatedly.

This was hit in production: guest TCP traffic with an MSS below 64
produced 64 KB GSO skbs on the default 1024-entry ring.

Raise the minimum Tx ring size to IONIC_MIN_TX_DESC (128), for the
hwstamp and kdump Tx rings as well, and set tso_max_segs to
IONIC_MIN_TX_DESC - 1 so that every Tx ring can hold the largest TSO
skb. Larger GSO skbs are segmented in software. 127 segments still
covers a 64 KB skb at the 536 byte default MSS.

IONIC_MIN_TXRX_DESC now only applies to Rx, so rename it to
IONIC_MIN_RX_DESC.

Tested on an AMD Pensando NIC with GSO frames of about 1500 segments,
injected through a tap with a virtio-net header and redirected to the
ionic netdev with tc mirred: without this patch the Tx queue stalls and
the watchdog fires, and with it they are sent.

Fixes: 0f3154e6bcb3 ("ionic: Add Tx and Rx handling")
Cc: stable@vger.kernel.org
Suggested-by: Jakub Kicinski <kuba@kernel.org>
Link: https://lore.kernel.org/all/20261112082753.6ee90fc8@kernel.org/
Signed-off-by: Nikhil P. Rao <nikhil.rao@amd.com>
Reviewed-by: Brett Creeley <brett.creeley@amd.com>
---
 drivers/net/ethernet/pensando/ionic/ionic_dev.h     |  3 ++-
 drivers/net/ethernet/pensando/ionic/ionic_ethtool.c |  4 ++--
 drivers/net/ethernet/pensando/ionic/ionic_lif.c     | 10 ++++++----
 3 files changed, 10 insertions(+), 7 deletions(-)

diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.h b/drivers/net/ethernet/pensando/ionic/ionic_dev.h
index db90e39a1442..4b3e244e6bbb 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_dev.h
+++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.h
@@ -16,7 +16,8 @@
 
 #define IONIC_MAX_TX_DESC		8192
 #define IONIC_MAX_RX_DESC		16384
-#define IONIC_MIN_TXRX_DESC		64
+#define IONIC_MIN_RX_DESC		64
+#define IONIC_MIN_TX_DESC		128
 #define IONIC_DEF_TXRX_DESC		1024
 #define IONIC_RX_FILL_THRESHOLD		16
 #define IONIC_RX_FILL_DIV		8
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
index 3518fbf5882d..24fd34b4cfcc 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
@@ -708,10 +708,10 @@ static int ionic_set_ringparam(struct net_device *netdev,
 		return -EINVAL;
 	}
 
-	if (ring->tx_pending < IONIC_MIN_TXRX_DESC) {
+	if (ring->tx_pending < IONIC_MIN_TX_DESC) {
 		NL_SET_ERR_MSG_FMT_MOD(extack,
 				       "Tx descriptor count must be at least %d",
-				       IONIC_MIN_TXRX_DESC);
+				       IONIC_MIN_TX_DESC);
 		return -EINVAL;
 	}
 
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
index 7f0da2b0defd..1f24c00334bd 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
@@ -997,7 +997,7 @@ int ionic_lif_create_hwstamp_txq(struct ionic_lif *lif)
 
 	features = IONIC_Q_F_2X_CQ_DESC | IONIC_TXQ_F_HWSTAMP;
 
-	num_desc = IONIC_MIN_TXRX_DESC;
+	num_desc = IONIC_MIN_TX_DESC;
 	desc_sz = sizeof(struct ionic_txq_desc);
 	comp_sz = 2 * sizeof(struct ionic_txq_comp);
 
@@ -1062,7 +1062,7 @@ int ionic_lif_create_hwstamp_rxq(struct ionic_lif *lif)
 
 	features = IONIC_Q_F_2X_CQ_DESC | IONIC_RXQ_F_HWSTAMP;
 
-	num_desc = IONIC_MIN_TXRX_DESC;
+	num_desc = IONIC_MIN_RX_DESC;
 	desc_sz = sizeof(struct ionic_rxq_desc);
 	comp_sz = 2 * sizeof(struct ionic_rxq_comp);
 	sg_desc_sz = sizeof(struct ionic_rxq_sg_desc);
@@ -1642,6 +1642,8 @@ static int ionic_init_nic_features(struct ionic_lif *lif)
 	netdev->features |= netdev->hw_features;
 	netdev->vlan_features |= netdev->features & ~NETIF_F_VLAN_FEATURES;
 
+	netif_set_tso_max_segs(netdev, IONIC_MIN_TX_DESC - 1);
+
 	netdev->priv_flags |= IFF_UNICAST_FLT |
 			      IFF_LIVE_ADDR_CHANGE;
 
@@ -3284,8 +3286,8 @@ int ionic_lif_alloc(struct ionic *ionic)
 	lif->index = 0;
 
 	if (is_kdump_kernel()) {
-		lif->ntxq_descs = IONIC_MIN_TXRX_DESC;
-		lif->nrxq_descs = IONIC_MIN_TXRX_DESC;
+		lif->ntxq_descs = IONIC_MIN_TX_DESC;
+		lif->nrxq_descs = IONIC_MIN_RX_DESC;
 	} else {
 		lif->ntxq_descs = IONIC_DEF_TXRX_DESC;
 		lif->nrxq_descs = IONIC_DEF_TXRX_DESC;

-- 
2.43.0


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH net v2 2/2] ionic: limit TSO segments to the minimum Tx ring size
  2026-11-17 19:18 ` [PATCH net v2 2/2] ionic: limit TSO segments to the minimum Tx ring size Nikhil P. Rao
@ 2026-11-20 19:58   ` Rao, Nikhil
  0 siblings, 0 replies; 4+ messages in thread
From: Rao, Nikhil @ 2026-11-20 19:58 UTC (permalink / raw)
  To: Nikhil P. Rao, Brett Creeley, Andrew Lunn, David S. Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni
  Cc: netdev, linux-kernel, Breno Leitao, Eric Joyner,
	Abhijit Gangurde, stable

On 11/17/2026 11:18 AM, Nikhil P. Rao wrote:
> ionic uses one Tx descriptor per TSO segment, and a Tx ring holds at
> most num_descs - 1 of them. The driver does not set tso_max_segs, so
> the stack can hand it GSO skbs with more segments than the Tx ring
> can hold. ionic_tx() then stops the queue waiting for descriptors
> that can never become free, and the netdev watchdog fires repeatedly.
> 
> This was hit in production: guest TCP traffic with an MSS below 64
> produced 64 KB GSO skbs on the default 1024-entry ring.
> 
> Raise the minimum Tx ring size to IONIC_MIN_TX_DESC (128), for the
> hwstamp and kdump Tx rings as well, and set tso_max_segs to
> IONIC_MIN_TX_DESC - 1 so that every Tx ring can hold the largest TSO
> skb. Larger GSO skbs are segmented in software. 127 segments still
> covers a 64 KB skb at the 536 byte default MSS.
> 
> IONIC_MIN_TXRX_DESC now only applies to Rx, so rename it to
> IONIC_MIN_RX_DESC.
> 
> Tested on an AMD Pensando NIC with GSO frames of about 1500 segments,
> injected through a tap with a virtio-net header and redirected to the
> ionic netdev with tc mirred: without this patch the Tx queue stalls and
> the watchdog fires, and with it they are sent.
> 
> Fixes: 0f3154e6bcb3 ("ionic: Add Tx and Rx handling")
> Cc: stable@vger.kernel.org
> Suggested-by: Jakub Kicinski <kuba@kernel.org>
> Link: https://lore.kernel.org/all/20261112082753.6ee90fc8@kernel.org/
> Signed-off-by: Nikhil P. Rao <nikhil.rao@amd.com>
> Reviewed-by: Brett Creeley <brett.creeley@amd.com>
> ---
>   drivers/net/ethernet/pensando/ionic/ionic_dev.h     |  3 ++-
>   drivers/net/ethernet/pensando/ionic/ionic_ethtool.c |  4 ++--
>   drivers/net/ethernet/pensando/ionic/ionic_lif.c     | 10 ++++++----
>   3 files changed, 10 insertions(+), 7 deletions(-)
> 
[...]
A note for the stable backport: on kernels before v6.19 this patch alone
does not stop the stall for guest traffic. A guest's GSO skb arrives as
SKB_GSO_DODGY with gso_segs still 0, so gso_features_check() never sees
it exceed gso_max_segs. That is the production case here: guest TCP
forwarded through the host to an ionic VF. be1b70ab21cb ("net: init
shinfo->gso_segs from qdisc_pkt_len_init()") fills in gso_segs before
that check.

Tested on v6.8 with an AMD Pensando NIC, using the same tap + tc mirred
injection: with this series alone the Tx queue still stalls; with
be1b70ab21cb added (a one-line backport there) the frames are sent.

So the stable tag on this patch should have been:

Cc: stable@vger.kernel.org # 6.18.x and older: be1b70ab21cb: net: init 
shinfo->gso_segs from qdisc_pkt_len_init()


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2026-11-20 19:58 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-17 19:18 [PATCH net v2 0/2] ionic: fix Tx stalls on TSO skbs larger than the Tx ring Nikhil P. Rao
2026-11-17 19:18 ` [PATCH net v2 1/2] ionic: reject Tx ring sizes below the minimum Nikhil P. Rao
2026-11-17 19:18 ` [PATCH net v2 2/2] ionic: limit TSO segments to the minimum Tx ring size Nikhil P. Rao
2026-11-20 19:58   ` Rao, Nikhil

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®