Re: [PATCH net-next v2] net/mlx5e: Prefetch the packet a few strides ahead in the linear MPWQE path

[Dragos Tatulea <dtatulea@nvidia.com>]

Hi,

On 04.11.26 20:52, Antonio Quartulli wrote:
> Without cache injection (DDIO/SDCI), the first accesses to a received
> packet's headers and skb_shared_info are DRAM misses, and the
> net_prefetch() in mlx5e_skb_from_cqe_mpwrq_linear() is too late to hide
> them. This caps the packet rate of a queue.
> 
> In the linear striding RQ path a packet takes one stride and the device
> fills the strides in order, so the packet MLX5E_RX_PREFETCH_DIST (4)
> strides ahead is known without its CQE: prefetch its headers and page,
> and the line its skb_shared_info will use if it has the length of the
> current packet, as in bulk flows. That line matters when every packet
> becomes its own skb: TCP inside an ovpn UDP tunnel gains 34% with it and
> 4% without.
> 
> AMD Ryzen 9 9950X, ConnectX-5 Ex, one RX queue on a dedicated CPU,
> MTU 1500, mean of 3 runs:
> 
>                                    before       after
>   UDP 512B flood, pktgen         2.52 Mpps    3.44 Mpps   +36%
>   TCP, 1 stream                  44.6 Gb/s    48.0 Gb/s    +8%
>     NAPI CPU cycles/packet            1334          715   -46%
>   XDP_DROP reading the packet,
>     NAPI CPU cycles/packet             409          280   -32%
>   XDP_TX, NAPI CPU cycles/packet       829          554   -33%
> 
> With the change, the XDP packet rates were limited by the sender, so
> cycles per packet are compared.
> 
Thanks for your patch! These look indeed like enticing numbers. I'd
like to take some time to try this at scale and on other NICs (CX8)
and CPUs (Grace) and also check latency.

Thanks,
Dragos