Re: [RFC v2 00/24] Per queue configs and large rx buffer support for zcrx

[Pavel Begunkov <asml.silence@gmail.com>]

On 8/13/25 16:39, Dragos Tatulea wrote:
> Hi Pavel,
> 
> On Fri, Aug 08, 2025 at 03:54:23PM +0100, Pavel Begunkov wrote:
>> [...]
>> For example, for 200Gbit broadcom NIC, 4K vs 32K buffers, and napi and
>> userspace pinned to the same CPU:
>>
>> packets=23987040 (MB=2745098), rps=199559 (MB/s=22837)
>> CPU    %usr   %nice    %sys %iowait    %irq   %soft   %idle
>>    0    1.53    0.00   27.78    2.72    1.31   66.45    0.22
>> packets=24078368 (MB=2755550), rps=200319 (MB/s=22924)
>> CPU    %usr   %nice    %sys %iowait    %irq   %soft   %idle
>>    0    0.69    0.00    8.26   31.65    1.83   57.00    0.57
>>
>> And for napi and userspace on different CPUs:
>>
>> packets=10725082 (MB=1227388), rps=198285 (MB/s=22692)
>> CPU    %usr   %nice    %sys %iowait    %irq   %soft   %idle
>>    0    0.10    0.00    0.50    0.00    0.50   74.50    24.40
>>    1    4.51    0.00   44.33   47.22    2.08    1.85    0.00
>> packets=14026235 (MB=1605175), rps=198388 (MB/s=22703)
>> CPU    %usr   %nice    %sys %iowait    %irq   %soft   %idle
>>    0    0.10    0.00    0.70    0.00    1.00   43.78   54.42
>>    1    1.09    0.00   31.95   62.91    1.42    2.63    0.00
>>
> What did you use for this benchmark, send-zerocopy? Could you share a
> branch and how you ran it please?
> 
> I have added some initial support to mlx5 for rx-buf-len and would like
> to benchmark it and compare it to what you posted.

You can use this branch:
https://github.com/isilence/liburing.git zcrx/rx-buf-len

# server
examples/zcrx -p <port> -q <queue_idx> -i <interface_name> -A1 \
              -B <rx_buf_len> -S <area size / memory provided>

"-A1" here is for using huge pages, so don't forget to configure
/proc/sys/vm/nr_hugepages.

# client
examples/send-zerocopy -6 tcp -D <ip addr> -p <port>
                        -t <runtime secs>
                        -l -b1 -n1 -z1 -d -s<send size>

I had to play with the client a bit for it to keep up with
the server. "-l" enables huge pages, and had to bump up the
send size. You can also add -v to both for a basic payload
verification.

-- 
Pavel Begunkov