[PATCH RFC 2/2] net/mlx5e: Rate limit the HW stats refresh from ndo_get_stats64

[Breno Leitao <leitao@debian.org>]

mlx5e_get_stats() and mlx5e_rep_get_stats() return the counters cached
by the previous refresh and queue update_stats_work to refresh them in
the background. queue_work() only merges requests while the work is
still pending, so with several readers each netdev keeps refreshing,
and every refresh sends one to three firmware commands. A single read
of /proc/net/dev requests a refresh on every netdev, representors
included.

Until commit ed56c5193ad8 ("net/mlx5e: Update NIC HW stats on demand
only") the driver refreshed these counters every 200 msecs. Bring back
that interval as the minimum time between two on-demand refreshes, and
skip the requests that arrive sooner.

Readers already get the values of the previous refresh, so at worst
they are now up to 200 msecs older. The monitor counter event path
queues the work directly and is not rate limited.

In switchdev mode with 8 VFs, one /proc/net/dev reader at about 75 Hz
caused 13.4k firmware commands in 13 seconds, and 1.8k with this
change. With two readers the count drops from 27.3k to 1.6k.

Signed-off-by: Breno Leitao <leitao@debian.org>
---
 drivers/net/ethernet/mellanox/mlx5/core/en.h      | 2 ++
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 9 +++++++++
 2 files changed, 11 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index a3945195f454e3..b66d40cdfdda53 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -151,6 +151,7 @@ struct page_pool;
 #define MLX5E_MAX_NUM_CHANNELS         256
 #define MLX5E_TX_CQ_POLL_BUDGET        128
 #define MLX5E_TX_XSK_POLL_BUDGET       64
+#define MLX5E_UPDATE_STATS_INTERVAL    200 /* msecs */
 #define MLX5E_SQ_RECOVER_MIN_INTERVAL  500 /* msecs */
 
 #define mlx5e_state_dereference(priv, p) \
@@ -928,6 +929,7 @@ struct mlx5e_priv {
 	struct work_struct         set_rx_mode_work;
 	struct work_struct         tx_timeout_work;
 	struct work_struct         update_stats_work;
+	unsigned long              update_stats_jiffies;
 	struct work_struct         monitor_counters_work;
 	struct mlx5_nb             monitor_counters_nb;
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index c3c2c5720c6f32..307505e0a827c6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -169,12 +169,21 @@ static void mlx5e_update_stats_work(struct work_struct *work)
 
 void mlx5e_queue_update_stats(struct mlx5e_priv *priv)
 {
+	unsigned long last;
+
 	if (!priv->profile->update_stats)
 		return;
 
 	if (unlikely(test_bit(MLX5E_STATE_DESTROYING, &priv->state)))
 		return;
 
+	/* Stats readers may poll often, limit the rate of FW queries */
+	last = READ_ONCE(priv->update_stats_jiffies);
+	if (time_in_range(jiffies, last,
+			  last + msecs_to_jiffies(MLX5E_UPDATE_STATS_INTERVAL)))
+		return;
+
+	WRITE_ONCE(priv->update_stats_jiffies, jiffies);
 	queue_work(priv->wq, &priv->update_stats_work);
 }
 

-- 
2.53.0-Meta