mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Paolo Abeni <pabeni@redhat.com>
To: Stanislav Fomichev <sdf.kernel@gmail.com>
Cc: netdev@vger.kernel.org, davem@davemloft.net, edumazet@google.com,
	kuba@kernel.org, michael.chan@broadcom.com,
	pavan.chebbi@broadcom.com, andrew+netdev@lunn.ch,
	edumazet@kernel.org, horms@kernel.org, shuah@kernel.org,
	almasrymina@google.com, sdf@fomichev.me, leitao@debian.org,
	dtatulea@nvidia.com, asml.silence@gmail.com,
	bobbyeshleman@meta.com, daniel@iogearbox.net,
	razor@blackwall.org, linux-kernel@vger.kernel.org,
	linux-kselftest@vger.kernel.org
Subject: Re: [PATCH net-next v4 0/5] net: devmem: size TX net_iovs from the dmabuf layout
Date: Tue, 17 Nov 2026 11:23:25 +0100	[thread overview]
Message-ID: <09156786-c9c6-47e3-8674-816b80300a3f@redhat.com> (raw)
In-Reply-To: <avZZ-6GGciPC0Lti@devvm16445.scu0.facebook.com>

On 11/13/26 01:41, Stanislav Fomichev wrote:
> On 11/12, Stanislav Fomichev wrote:
>> On 11/12, Paolo Abeni wrote:
>>> I'm wondering if making a comparison vs MAX_SKB_FRAGS==45 would be
>>> complex and if you could do that, too.
>>
>> Originally I did not want to include bigtcp to keep it apples to apples:
>> same tso size, just 16 frags in skb vs 1 (in ideal case).
>>
>> I just tried running mlx5 with 45 frags configured and it fails to start with
>> the following error:
>> mlx5e_attach_netdev() warns "MLX5E: Max SQ WQEBBs firmware capability: 16, needed 23" and returns -EIO.
>>
>> (fwiw quick review with llm claims that mlx5 only supports up to 20 skb frags)
>>
>> What I also tried just now is `gso_max_size 524280` (on top of this series)
>> and it ups single flow throughput from 200Gbps to 240Gbps (presumably
>> by making the receiver go faster with 512K TSOs vs 64K TSOs) and halves
>> the sender softirq..
>>
>> I hope this gives you more data, not sure running with 20 frags is worth
>> it (vs 16)? LMK if you disagree or want me to run something else.

Thanks, the above is quite exhaustive :)

> Fun fact, also setting gro_max_size on the receiver gets line rate over
> single flow:
> 
> I   client.c: Connected 1:cpu 39 | 1:cpu 118
> client: == Source 2401:db00:255a:4a:bace::b
> client:   Tx389.448 Gbps (2921248849920 bytes in 60008026 usec)
> client:   Rx  0.000 Gbps (0 bytes in 60008026 usec)
> client:   TCP retrans reord rtt rttvar d_ce snd_wnd cwnd
> client:             2   125  86      2    0 15300608 10142
> client:   net CPU 39: usr: 0.03% sys: 0.06% idle:67.39% iow: 0.00% irq: 5.69% sirq:26.81%
> client:   app CPU 40: usr: 0.05% sys: 2.91% idle:95.67% iow: 0.00% irq: 0.21% sirq: 1.14%
> client: == Target 2401:db00:255a:44:bace::b
> client:   Tx  0.000 Gbps (0 bytes in 60009306 usec)
> client:   Rx389.438 Gbps (2921238435008 bytes in 60009306 usec)
> client:   TCP retrans reord rtt rttvar d_ce snd_wnd cwnd
> client:             0     0 210    122    0  999424   10
> client:   net CPU118: usr: 0.00% sys: 0.00% idle: 1.60% iow: 0.00% irq: 0.33% sirq:98.05%
> client:   app CPU119: usr: 2.75% sys:77.46% idle:18.95% iow: 0.00% irq: 0.47% sirq: 0.34%
> 
> 4200MTU, 32K rx-buf-len, this patch series, gso/gro_mas_size of 524280,
> still the default number of frags (enough on rx to fill 512K GRO with 32K
> chunks).

That looks just great!

FTR, my understanding is that Jakub wants to say his last word here, so:

Acked-by: Paolo Abeni <pabeni@redhat.com>


  parent reply	other threads:[~2026-11-17 10:23 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-11 22:47 Stanislav Fomichev
2026-11-11 22:47 ` [PATCH net-next v4 1/5] bnxt_en: let the stack segment GSO skbs with too many frags Stanislav Fomichev
2026-11-12 22:48   ` netdev-bot+sashiko
2026-11-11 22:47 ` [PATCH net-next v4 2/5] net: devmem: charge the frag length to TX skb truesize Stanislav Fomichev
2026-11-12 22:48   ` netdev-bot+sashiko
2026-11-11 22:47 ` [PATCH net-next v4 3/5] net: devmem: use niov_shift in net_devmem_get_niov_at() Stanislav Fomichev
2026-11-13 12:05   ` Breno Leitao
2026-11-11 22:47 ` [PATCH net-next v4 4/5] net: devmem: size TX net_iovs from the dmabuf sg table Stanislav Fomichev
2026-11-12 22:49   ` netdev-bot+sashiko
2026-11-13 12:10   ` Breno Leitao
2026-11-13 12:20   ` Breno Leitao
2026-11-11 22:47 ` [PATCH net-next v4 5/5] selftests: drv-net: devmem: TX from a hugepage-backed dmabuf Stanislav Fomichev
2026-11-12 22:49   ` netdev-bot+sashiko
2026-11-13 12:21   ` Breno Leitao
2026-11-12  8:25 ` [PATCH net-next v4 0/5] net: devmem: size TX net_iovs from the dmabuf layout Paolo Abeni
2026-11-12 23:00   ` Stanislav Fomichev
2026-11-13  0:41     ` Stanislav Fomichev
2026-11-13  2:41       ` Mina Almasry
2026-11-13 17:02         ` Stanislav Fomichev
2026-11-17 10:23       ` Paolo Abeni [this message]
2026-11-18  1:18         ` Jakub Kicinski
2026-11-13 14:20 ` Nikolay Aleksandrov
2026-11-18  1:20 ` patchwork-bot+netdevbpf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=09156786-c9c6-47e3-8674-816b80300a3f@redhat.com \
    --to=pabeni@redhat.com \
    --cc=almasrymina@google.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=asml.silence@gmail.com \
    --cc=bobbyeshleman@meta.com \
    --cc=daniel@iogearbox.net \
    --cc=davem@davemloft.net \
    --cc=dtatulea@nvidia.com \
    --cc=edumazet@google.com \
    --cc=edumazet@kernel.org \
    --cc=horms@kernel.org \
    --cc=kuba@kernel.org \
    --cc=leitao@debian.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=michael.chan@broadcom.com \
    --cc=netdev@vger.kernel.org \
    --cc=pavan.chebbi@broadcom.com \
    --cc=razor@blackwall.org \
    --cc=sdf.kernel@gmail.com \
    --cc=sdf@fomichev.me \
    --cc=shuah@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®