* [PATCH net v2 0/2] ionic: fix Tx stalls on TSO skbs larger than the Tx ring
@ 2026-11-17 19:18 Nikhil P. Rao
2026-11-17 19:18 ` [PATCH net v2 1/2] ionic: reject Tx ring sizes below the minimum Nikhil P. Rao
2026-11-17 19:18 ` [PATCH net v2 2/2] ionic: limit TSO segments to the minimum Tx ring size Nikhil P. Rao
0 siblings, 2 replies; 4+ messages in thread
From: Nikhil P. Rao @ 2026-11-17 19:18 UTC (permalink / raw)
To: Brett Creeley, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni
Cc: netdev, linux-kernel, Breno Leitao, Eric Joyner,
Abhijit Gangurde, stable, Nikhil P. Rao
ionic uses one Tx descriptor per TSO segment but never sets
tso_max_segs, so the stack can build TSO skbs with more segments than
the Tx ring holds. Such an skb can never be posted: the queue stops,
nothing wakes it, and the netdev watchdog fires. Guest TCP traffic with
an MSS below 64 hit this on the default 1024-entry ring.
Patch 1 rejects Tx ring sizes below 64. The completion path wakes a
stopped Tx queue only once 44 descriptors are free, so a smaller ring
never wakes once stopped.
Patch 2 raises the minimum Tx ring size to 128 and sets tso_max_segs
to 127, so every Tx ring can hold the largest TSO skb and larger GSO
skbs are segmented in software.
The two patches fix separate bugs; patch 2 raises the Tx ring minimum
that patch 1 adds from 64 to 128.
Changes in v2:
- Patch 2: use a fixed limit of 127 segments and a 128-entry minimum
Tx ring instead of following the Tx ring size, as suggested by Jakub.
This also covers the hwstamp and kdump Tx rings.
- Add testing notes to both patches, and Cc stable.
- Link to v1: https://lore.kernel.org/all/20261107-b4-ionic-tso-max-segs-v1-0-67c248a5b681@amd.com/
This series was developed with the help of an LLM-based coding
assistant (Claude).
---
Nikhil P. Rao (2):
ionic: reject Tx ring sizes below the minimum
ionic: limit TSO segments to the minimum Tx ring size
drivers/net/ethernet/pensando/ionic/ionic_dev.h | 3 ++-
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c | 7 +++++++
drivers/net/ethernet/pensando/ionic/ionic_lif.c | 10 ++++++----
3 files changed, 15 insertions(+), 5 deletions(-)
---
base-commit: a2fd526d644233d6a892c28173f07c61e21e4c93
change-id: 20261107-b4-ionic-tso-max-segs-057bde9cfe49
Best regards,
--
Nikhil P. Rao <nikhil.rao@amd.com>
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH net v2 1/2] ionic: reject Tx ring sizes below the minimum
2026-11-17 19:18 [PATCH net v2 0/2] ionic: fix Tx stalls on TSO skbs larger than the Tx ring Nikhil P. Rao
@ 2026-11-17 19:18 ` Nikhil P. Rao
2026-11-17 19:18 ` [PATCH net v2 2/2] ionic: limit TSO segments to the minimum Tx ring size Nikhil P. Rao
1 sibling, 0 replies; 4+ messages in thread
From: Nikhil P. Rao @ 2026-11-17 19:18 UTC (permalink / raw)
To: Brett Creeley, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni
Cc: netdev, linux-kernel, Breno Leitao, Eric Joyner,
Abhijit Gangurde, stable, Nikhil P. Rao
ionic_tx_cq_service() wakes a stopped Tx queue only once
IONIC_TSO_DESCS_NEEDED (44) descriptors are free. The threshold is
sized for a 64 KB TSO skb at a 1500 byte MSS and assumes the ring can
have that many free; a Tx ring of 32 or fewer descriptors cannot, so a
stopped queue is never woken and the netdev watchdog fires.
ionic_set_ringparam() only checks that the size is a power of 2, and
the ethtool core only checks the maximum, so such sizes are accepted.
Reject Tx ring sizes below IONIC_MIN_TXRX_DESC.
The stall was confirmed from the wake and stop thresholds, not
reproduced. Tested on an AMD Pensando NIC: "ethtool -G <dev> tx 32" is
rejected and leaves the ring unchanged, and "ethtool -G <dev> tx 64" is
accepted.
Fixes: 061b9bedbef1 ("ionic: Rework Tx start/stop flow")
Cc: stable@vger.kernel.org
Reported-by: sashiko-bot@kernel.org # Running on a local machine
Signed-off-by: Nikhil P. Rao <nikhil.rao@amd.com>
Reviewed-by: Brett Creeley <brett.creeley@amd.com>
---
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
index 119ac720c078..3518fbf5882d 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
@@ -708,6 +708,13 @@ static int ionic_set_ringparam(struct net_device *netdev,
return -EINVAL;
}
+ if (ring->tx_pending < IONIC_MIN_TXRX_DESC) {
+ NL_SET_ERR_MSG_FMT_MOD(extack,
+ "Tx descriptor count must be at least %d",
+ IONIC_MIN_TXRX_DESC);
+ return -EINVAL;
+ }
+
/* if nothing to do return success */
if (ring->tx_pending == lif->ntxq_descs &&
ring->rx_pending == lif->nrxq_descs &&
--
2.43.0
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH net v2 2/2] ionic: limit TSO segments to the minimum Tx ring size
2026-11-17 19:18 [PATCH net v2 0/2] ionic: fix Tx stalls on TSO skbs larger than the Tx ring Nikhil P. Rao
2026-11-17 19:18 ` [PATCH net v2 1/2] ionic: reject Tx ring sizes below the minimum Nikhil P. Rao
@ 2026-11-17 19:18 ` Nikhil P. Rao
2026-11-20 19:58 ` Rao, Nikhil
1 sibling, 1 reply; 4+ messages in thread
From: Nikhil P. Rao @ 2026-11-17 19:18 UTC (permalink / raw)
To: Brett Creeley, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni
Cc: netdev, linux-kernel, Breno Leitao, Eric Joyner,
Abhijit Gangurde, stable, Nikhil P. Rao
ionic uses one Tx descriptor per TSO segment, and a Tx ring holds at
most num_descs - 1 of them. The driver does not set tso_max_segs, so
the stack can hand it GSO skbs with more segments than the Tx ring
can hold. ionic_tx() then stops the queue waiting for descriptors
that can never become free, and the netdev watchdog fires repeatedly.
This was hit in production: guest TCP traffic with an MSS below 64
produced 64 KB GSO skbs on the default 1024-entry ring.
Raise the minimum Tx ring size to IONIC_MIN_TX_DESC (128), for the
hwstamp and kdump Tx rings as well, and set tso_max_segs to
IONIC_MIN_TX_DESC - 1 so that every Tx ring can hold the largest TSO
skb. Larger GSO skbs are segmented in software. 127 segments still
covers a 64 KB skb at the 536 byte default MSS.
IONIC_MIN_TXRX_DESC now only applies to Rx, so rename it to
IONIC_MIN_RX_DESC.
Tested on an AMD Pensando NIC with GSO frames of about 1500 segments,
injected through a tap with a virtio-net header and redirected to the
ionic netdev with tc mirred: without this patch the Tx queue stalls and
the watchdog fires, and with it they are sent.
Fixes: 0f3154e6bcb3 ("ionic: Add Tx and Rx handling")
Cc: stable@vger.kernel.org
Suggested-by: Jakub Kicinski <kuba@kernel.org>
Link: https://lore.kernel.org/all/20261112082753.6ee90fc8@kernel.org/
Signed-off-by: Nikhil P. Rao <nikhil.rao@amd.com>
Reviewed-by: Brett Creeley <brett.creeley@amd.com>
---
drivers/net/ethernet/pensando/ionic/ionic_dev.h | 3 ++-
drivers/net/ethernet/pensando/ionic/ionic_ethtool.c | 4 ++--
drivers/net/ethernet/pensando/ionic/ionic_lif.c | 10 ++++++----
3 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.h b/drivers/net/ethernet/pensando/ionic/ionic_dev.h
index db90e39a1442..4b3e244e6bbb 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_dev.h
+++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.h
@@ -16,7 +16,8 @@
#define IONIC_MAX_TX_DESC 8192
#define IONIC_MAX_RX_DESC 16384
-#define IONIC_MIN_TXRX_DESC 64
+#define IONIC_MIN_RX_DESC 64
+#define IONIC_MIN_TX_DESC 128
#define IONIC_DEF_TXRX_DESC 1024
#define IONIC_RX_FILL_THRESHOLD 16
#define IONIC_RX_FILL_DIV 8
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
index 3518fbf5882d..24fd34b4cfcc 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c
@@ -708,10 +708,10 @@ static int ionic_set_ringparam(struct net_device *netdev,
return -EINVAL;
}
- if (ring->tx_pending < IONIC_MIN_TXRX_DESC) {
+ if (ring->tx_pending < IONIC_MIN_TX_DESC) {
NL_SET_ERR_MSG_FMT_MOD(extack,
"Tx descriptor count must be at least %d",
- IONIC_MIN_TXRX_DESC);
+ IONIC_MIN_TX_DESC);
return -EINVAL;
}
diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
index 7f0da2b0defd..1f24c00334bd 100644
--- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c
+++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c
@@ -997,7 +997,7 @@ int ionic_lif_create_hwstamp_txq(struct ionic_lif *lif)
features = IONIC_Q_F_2X_CQ_DESC | IONIC_TXQ_F_HWSTAMP;
- num_desc = IONIC_MIN_TXRX_DESC;
+ num_desc = IONIC_MIN_TX_DESC;
desc_sz = sizeof(struct ionic_txq_desc);
comp_sz = 2 * sizeof(struct ionic_txq_comp);
@@ -1062,7 +1062,7 @@ int ionic_lif_create_hwstamp_rxq(struct ionic_lif *lif)
features = IONIC_Q_F_2X_CQ_DESC | IONIC_RXQ_F_HWSTAMP;
- num_desc = IONIC_MIN_TXRX_DESC;
+ num_desc = IONIC_MIN_RX_DESC;
desc_sz = sizeof(struct ionic_rxq_desc);
comp_sz = 2 * sizeof(struct ionic_rxq_comp);
sg_desc_sz = sizeof(struct ionic_rxq_sg_desc);
@@ -1642,6 +1642,8 @@ static int ionic_init_nic_features(struct ionic_lif *lif)
netdev->features |= netdev->hw_features;
netdev->vlan_features |= netdev->features & ~NETIF_F_VLAN_FEATURES;
+ netif_set_tso_max_segs(netdev, IONIC_MIN_TX_DESC - 1);
+
netdev->priv_flags |= IFF_UNICAST_FLT |
IFF_LIVE_ADDR_CHANGE;
@@ -3284,8 +3286,8 @@ int ionic_lif_alloc(struct ionic *ionic)
lif->index = 0;
if (is_kdump_kernel()) {
- lif->ntxq_descs = IONIC_MIN_TXRX_DESC;
- lif->nrxq_descs = IONIC_MIN_TXRX_DESC;
+ lif->ntxq_descs = IONIC_MIN_TX_DESC;
+ lif->nrxq_descs = IONIC_MIN_RX_DESC;
} else {
lif->ntxq_descs = IONIC_DEF_TXRX_DESC;
lif->nrxq_descs = IONIC_DEF_TXRX_DESC;
--
2.43.0
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH net v2 2/2] ionic: limit TSO segments to the minimum Tx ring size
2026-11-17 19:18 ` [PATCH net v2 2/2] ionic: limit TSO segments to the minimum Tx ring size Nikhil P. Rao
@ 2026-11-20 19:58 ` Rao, Nikhil
0 siblings, 0 replies; 4+ messages in thread
From: Rao, Nikhil @ 2026-11-20 19:58 UTC (permalink / raw)
To: Nikhil P. Rao, Brett Creeley, Andrew Lunn, David S. Miller,
Eric Dumazet, Jakub Kicinski, Paolo Abeni
Cc: netdev, linux-kernel, Breno Leitao, Eric Joyner,
Abhijit Gangurde, stable
On 11/17/2026 11:18 AM, Nikhil P. Rao wrote:
> ionic uses one Tx descriptor per TSO segment, and a Tx ring holds at
> most num_descs - 1 of them. The driver does not set tso_max_segs, so
> the stack can hand it GSO skbs with more segments than the Tx ring
> can hold. ionic_tx() then stops the queue waiting for descriptors
> that can never become free, and the netdev watchdog fires repeatedly.
>
> This was hit in production: guest TCP traffic with an MSS below 64
> produced 64 KB GSO skbs on the default 1024-entry ring.
>
> Raise the minimum Tx ring size to IONIC_MIN_TX_DESC (128), for the
> hwstamp and kdump Tx rings as well, and set tso_max_segs to
> IONIC_MIN_TX_DESC - 1 so that every Tx ring can hold the largest TSO
> skb. Larger GSO skbs are segmented in software. 127 segments still
> covers a 64 KB skb at the 536 byte default MSS.
>
> IONIC_MIN_TXRX_DESC now only applies to Rx, so rename it to
> IONIC_MIN_RX_DESC.
>
> Tested on an AMD Pensando NIC with GSO frames of about 1500 segments,
> injected through a tap with a virtio-net header and redirected to the
> ionic netdev with tc mirred: without this patch the Tx queue stalls and
> the watchdog fires, and with it they are sent.
>
> Fixes: 0f3154e6bcb3 ("ionic: Add Tx and Rx handling")
> Cc: stable@vger.kernel.org
> Suggested-by: Jakub Kicinski <kuba@kernel.org>
> Link: https://lore.kernel.org/all/20261112082753.6ee90fc8@kernel.org/
> Signed-off-by: Nikhil P. Rao <nikhil.rao@amd.com>
> Reviewed-by: Brett Creeley <brett.creeley@amd.com>
> ---
> drivers/net/ethernet/pensando/ionic/ionic_dev.h | 3 ++-
> drivers/net/ethernet/pensando/ionic/ionic_ethtool.c | 4 ++--
> drivers/net/ethernet/pensando/ionic/ionic_lif.c | 10 ++++++----
> 3 files changed, 10 insertions(+), 7 deletions(-)
>
[...]
A note for the stable backport: on kernels before v6.19 this patch alone
does not stop the stall for guest traffic. A guest's GSO skb arrives as
SKB_GSO_DODGY with gso_segs still 0, so gso_features_check() never sees
it exceed gso_max_segs. That is the production case here: guest TCP
forwarded through the host to an ionic VF. be1b70ab21cb ("net: init
shinfo->gso_segs from qdisc_pkt_len_init()") fills in gso_segs before
that check.
Tested on v6.8 with an AMD Pensando NIC, using the same tap + tc mirred
injection: with this series alone the Tx queue still stalls; with
be1b70ab21cb added (a one-line backport there) the frames are sent.
So the stable tag on this patch should have been:
Cc: stable@vger.kernel.org # 6.18.x and older: be1b70ab21cb: net: init
shinfo->gso_segs from qdisc_pkt_len_init()
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-11-20 19:58 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-17 19:18 [PATCH net v2 0/2] ionic: fix Tx stalls on TSO skbs larger than the Tx ring Nikhil P. Rao
2026-11-17 19:18 ` [PATCH net v2 1/2] ionic: reject Tx ring sizes below the minimum Nikhil P. Rao
2026-11-17 19:18 ` [PATCH net v2 2/2] ionic: limit TSO segments to the minimum Tx ring size Nikhil P. Rao
2026-11-20 19:58 ` Rao, Nikhil
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®