* [PATCH net-next v1 1/2] mlxbf_gige: protect against MAC disable during TX
2026-11-20 20:24 [PATCH net-next v1 0/2] mlxbf_gige: disable MAC after panic Chris Babroski
@ 2026-11-20 20:24 ` Chris Babroski
2026-11-20 20:24 ` [PATCH net-next v1 2/2] mlxbf_gige: disable MAC after kernel panic Chris Babroski
1 sibling, 0 replies; 3+ messages in thread
From: Chris Babroski @ 2026-11-20 20:24 UTC (permalink / raw)
To: netdev, linux-kernel
Cc: andrew+netdev, davem, edumazet, kuba, pabeni, davthompson, cbabroski
On BlueField, firmware can disable and reset the OOB MAC when it handles
a fatal error or panic. If Linux is still actively transmitting packets
while the MAC is disabled, then it's possible for the HW consumer index
to be reset while the SW completion index in the driver remains the
same. This can cause the two indices to suddenly become far apart, with
a difference that is potentially larger than the TX queue length itself.
Previously, mlxbf_gige_handle_tx_complete() would continue to loop
around the TX queue, counting packets and freeing their memory, until
the two indices were equal. If entries were already freed the first time
through the queue, then a second attempt could cause a NULL pointer
dereference.
Bound the loop over the length of the TX queue so entries are not
handled more than once, and check that the skb at tx_wqe_index is not
NULL before handling.
After the loop, set the SW completion index to the new HW consumer index
to prevent future polls from walking the queue again. In normal
operation the loop already exits with the two indices equal, so there is
no change in behavior.
TX is not recovered after a sudden MAC disable. If firmware has disabled
the MAC then the system is in a state where it will be rebooting
automatically or needs manual intervention to recover.
Signed-off-by: Chris Babroski <cbabroski@nvidia.com>
---
.../mellanox/mlxbf_gige/mlxbf_gige_tx.c | 29 ++++++++++++-------
1 file changed, 19 insertions(+), 10 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_tx.c b/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_tx.c
index 04982e888c63..73c3033591d5 100644
--- a/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_tx.c
+++ b/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_tx.c
@@ -130,6 +130,7 @@ bool mlxbf_gige_handle_tx_complete(struct mlxbf_gige *priv)
u64 *tx_wqe_addr;
u64 tx_status;
u16 tx_ci;
+ u16 entry;
tx_status = readq(priv->base + MLXBF_GIGE_TX_STATUS);
if (tx_status & MLXBF_GIGE_TX_STATUS_DATA_FIFO_FULL)
@@ -143,7 +144,8 @@ bool mlxbf_gige_handle_tx_complete(struct mlxbf_gige *priv)
* to be supported, that is TX consumer index wrapped from 0xFFFF
* to 0 while TX completion index is still < 0xFFFF.
*/
- for (; priv->prev_tx_ci != tx_ci; priv->prev_tx_ci++) {
+ for (entry = 0; priv->prev_tx_ci != tx_ci && entry < priv->tx_q_entries;
+ priv->prev_tx_ci++, entry++) {
tx_wqe_index = priv->prev_tx_ci % priv->tx_q_entries;
/* Each TX WQE is 16 bytes. The 8 MSB store the 2KB TX
* buffer address and the 8 LSB contain information
@@ -152,18 +154,25 @@ bool mlxbf_gige_handle_tx_complete(struct mlxbf_gige *priv)
tx_wqe_addr = priv->tx_wqe_base +
(tx_wqe_index * MLXBF_GIGE_TX_WQE_SZ_QWORDS);
- stats->tx_packets++;
- stats->tx_bytes += MLXBF_GIGE_TX_WQE_PKT_LEN(tx_wqe_addr);
-
- dma_unmap_single(priv->dev, *tx_wqe_addr,
- priv->tx_skb[tx_wqe_index]->len, DMA_TO_DEVICE);
- dev_consume_skb_any(priv->tx_skb[tx_wqe_index]);
- priv->tx_skb[tx_wqe_index] = NULL;
+ if (priv->tx_skb[tx_wqe_index]) {
+ stats->tx_packets++;
+ stats->tx_bytes += MLXBF_GIGE_TX_WQE_PKT_LEN(tx_wqe_addr);
+ dma_unmap_single(priv->dev, *tx_wqe_addr,
+ priv->tx_skb[tx_wqe_index]->len, DMA_TO_DEVICE);
+ dev_consume_skb_any(priv->tx_skb[tx_wqe_index]);
+ priv->tx_skb[tx_wqe_index] = NULL;
- /* Ensure completion of updates across all cores */
- mb();
+ /* Ensure completion of updates across all cores */
+ mb();
+ }
}
+ /* At this point the completion index should be equal to the
+ * consumer index. If they aren't then the driver is out of sync
+ * from the HW's state and needs to be adjusted.
+ */
+ priv->prev_tx_ci = tx_ci;
+
/* Since the TX ring was likely just drained, check if TX queue
* had previously been stopped and now that there are TX buffers
* available the TX queue can be awakened.
--
2.34.1
^ permalink raw reply [flat|nested] 3+ messages in thread* [PATCH net-next v1 2/2] mlxbf_gige: disable MAC after kernel panic
2026-11-20 20:24 [PATCH net-next v1 0/2] mlxbf_gige: disable MAC after panic Chris Babroski
2026-11-20 20:24 ` [PATCH net-next v1 1/2] mlxbf_gige: protect against MAC disable during TX Chris Babroski
@ 2026-11-20 20:24 ` Chris Babroski
1 sibling, 0 replies; 3+ messages in thread
From: Chris Babroski @ 2026-11-20 20:24 UTC (permalink / raw)
To: netdev, linux-kernel
Cc: andrew+netdev, davem, edumazet, kuba, pabeni, davthompson, cbabroski
After a kernel panic the MAC continues to receive packets into the RX
FIFO even though the driver can no longer process them. By default, the
MAC HW is designed to repeatedly send pause frames to its link partner
when the RX FIFO is full. The internal Ethernet switch on BlueField that
is connected to this port is also designed to stop traffic on all ports
when it receives a high number of continuous pause frames from one of
the ports.
What this means is that when the kernel panics on BlueField, it is
possible to reproduce a scenario under steady and high network load
where the network becomes inaccessible for other devices connected to
the 3-port switch, namely the OOB host and DPU-BMC.
Register a panic notifier at highest priority that disables MAC RX and
clears the RX FIFO to ensure that the MAC can no longer receive packets
or send pause frames.
Signed-off-by: Chris Babroski <cbabroski@nvidia.com>
---
.../ethernet/mellanox/mlxbf_gige/mlxbf_gige.h | 2 +
.../mellanox/mlxbf_gige/mlxbf_gige_main.c | 39 +++++++++++++++++++
2 files changed, 41 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h b/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h
index e7777700ee18..c63bdfae4344 100644
--- a/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h
+++ b/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige.h
@@ -13,6 +13,7 @@
#include <linux/io-64-nonatomic-lo-hi.h>
#include <linux/irqreturn.h>
#include <linux/netdevice.h>
+#include <linux/notifier.h>
#include <linux/irq.h>
#include <linux/phy.h>
@@ -130,6 +131,7 @@ struct mlxbf_gige {
u8 hw_version;
struct mlxbf_gige_mdio_gw *mdio_gw;
int prev_speed;
+ struct notifier_block panic_nb;
};
/* Rx Work Queue Element definitions */
diff --git a/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c b/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c
index d1f8a72cae53..76da71108345 100644
--- a/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c
+++ b/drivers/net/ethernet/mellanox/mlxbf_gige/mlxbf_gige_main.c
@@ -12,6 +12,7 @@
#include <linux/interrupt.h>
#include <linux/iopoll.h>
#include <linux/module.h>
+#include <linux/panic_notifier.h>
#include <linux/phy.h>
#include <linux/platform_device.h>
#include <linux/rtnetlink.h>
@@ -127,6 +128,27 @@ static int mlxbf_gige_clean_port(struct mlxbf_gige *priv)
return err;
}
+static void mlxbf_gige_disable_port(struct mlxbf_gige *priv)
+{
+ u64 control;
+
+ /* Disable RX DMA */
+ control = readq(priv->base + MLXBF_GIGE_RX_DMA);
+ control &= ~MLXBF_GIGE_RX_DMA_EN;
+ writeq(control, priv->base + MLXBF_GIGE_RX_DMA);
+
+ /* Disable port */
+ control = readq(priv->base + MLXBF_GIGE_CONTROL);
+ control &= ~MLXBF_GIGE_CONTROL_PORT_EN;
+ writeq(control, priv->base + MLXBF_GIGE_CONTROL);
+
+ /* Disable interrupts */
+ writeq(0, priv->base + MLXBF_GIGE_INT_EN);
+
+ /* Ensure completion of disable writes before cleaning the port */
+ mb();
+}
+
static int mlxbf_gige_open(struct net_device *netdev)
{
struct mlxbf_gige *priv = netdev_priv(netdev);
@@ -382,6 +404,17 @@ static struct mlxbf_gige_link_cfg mlxbf_gige_link_cfgs[] = {
}
};
+static int mlxbf_gige_on_panic(struct notifier_block *nb,
+ unsigned long action, void *data)
+{
+ struct mlxbf_gige *priv;
+
+ priv = container_of(nb, struct mlxbf_gige, panic_nb);
+ mlxbf_gige_disable_port(priv);
+ mlxbf_gige_clean_port(priv);
+ return NOTIFY_DONE;
+}
+
static int mlxbf_gige_probe(struct platform_device *pdev)
{
struct phy_device *phydev;
@@ -446,6 +479,10 @@ static int mlxbf_gige_probe(struct platform_device *pdev)
/* Write initial MAC address to hardware */
mlxbf_gige_initial_mac(priv);
+ priv->panic_nb.priority = INT_MAX;
+ priv->panic_nb.notifier_call = mlxbf_gige_on_panic;
+ atomic_notifier_chain_register(&panic_notifier_list, &priv->panic_nb);
+
err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
if (err) {
dev_err(&pdev->dev, "DMA configuration failed: %pe\n", ERR_PTR(err));
@@ -497,6 +534,7 @@ static int mlxbf_gige_probe(struct platform_device *pdev)
return 0;
out:
+ atomic_notifier_chain_unregister(&panic_notifier_list, &priv->panic_nb);
mlxbf_gige_mdio_remove(priv);
return err;
}
@@ -507,6 +545,7 @@ static void mlxbf_gige_remove(struct platform_device *pdev)
unregister_netdev(priv->netdev);
phy_disconnect(priv->netdev->phydev);
+ atomic_notifier_chain_unregister(&panic_notifier_list, &priv->panic_nb);
mlxbf_gige_mdio_remove(priv);
platform_set_drvdata(pdev, NULL);
}
--
2.34.1
^ permalink raw reply [flat|nested] 3+ messages in thread