mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Simon Schippers <simon.schippers@tu-dortmund.de>
To: Eric Dumazet <edumazet@google.com>
Cc: oneukum@suse.com, andrew+netdev@lunn.ch, davem@davemloft.net,
	kuba@kernel.org, pabeni@redhat.com, netdev@vger.kernel.org,
	linux-usb@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH net-next v1 0/1] usbnet: Add support for Byte Queue Limits (BQL)
Date: Wed, 5 Nov 2025 11:40:14 +0100	[thread overview]
Message-ID: <f2a363d3-40d7-4a5f-a884-ec147a167ef5@tu-dortmund.de> (raw)
In-Reply-To: <CANn89iLLwWvbnCKKRrV2c7eo+4UduLVgZUWR=ZoZ+SPHRGf=wg@mail.gmail.com>

On 11/4/25 18:02, Eric Dumazet wrote:
> On Tue, Nov 4, 2025 at 8:14 AM Simon Schippers
> <simon.schippers@tu-dortmund.de> wrote:
>>
>> During recent testing, I observed significant latency spikes when using
>> Quectel 5G modems under load. Investigation revealed that the issue was
>> caused by bufferbloat in the usbnet driver.
>>
>> In the current implementation, usbnet uses a fixed tx_qlen of:
>>
>> USB2: 60 * 1518 bytes = 91.08 KB
>> USB3: 60 * 5 * 1518 bytes = 454.80 KB
>>
>> Such large transmit queues can be problematic, especially for cellular
>> modems. For example, with a typical celluar link speed of 10 Mbit/s, a
>> fully occupied USB3 transmit queue results in:
>>
>> 454.80 KB / (10 Mbit/s / 8 bit/byte) = 363.84 ms
>>
>> of additional latency.
> 
> Doesn't 5G need to push more packets to the driver to get good aggregation ?
> 

Yes, but not 455 KB for low speeds. 5G requires a queue of a few ms to
aggregate enough packets for a frame but not of several hundred ms as
calculated in my example. And yes, there are situations where 5G,
especially FR2 mmWave, reaches Gbit/s speeds where a big queue is
required. But the dynamic queue limit approach of BQL should be well
suited for these varying speeds.

>>
>> To address this issue, this patch introduces support for
>> Byte Queue Limits (BQL) [1][2] in the usbnet driver. BQL dynamically
>> limits the amount of data queued in the driver, effectively reducing
>> latency without impacting throughput.
>> This implementation was successfully tested on several devices as
>> described in the commit.
>>
>>
>>
>> Future work
>>
>> Due to offloading, TCP often produces SKBs up to 64 KB in size.
> 
> Only for rates > 500 Mbit. After BQL, we had many more improvements in
> the stack.
> https://lwn.net/Articles/564978/
> 
> 

I also saw these large SKBs, for example, for my USB2 Android tethering,
which advertises a network speed of < 500 Mbit/s.
I saw these large SKBs by looking at the file:

cat /sys/class/net/INTERFACE/queues/tx-0/byte_queue_limits/inflight

For UDP-only traffic, inflight always maxed out at MTU size.

Thank you for your replies!

>> To
>> further decrease buffer bloat, I tried to disable TSO, GSO and LRO but it
>> did not have the intended effect in my tests. The only dirty workaround I
>> found so far was to call netif_stop_queue() whenever BQL sets
>> __QUEUE_STATE_STACK_XOFF. However, a proper solution to this issue would
>> be desirable.
>>
>> I also plan to publish a scientific paper on this topic in the near
>> future.
>>
>> Thanks,
>> Simon
>>
>> [1] https://medium.com/@tom_84912/byte-queue-limits-the-unauthorized-biography-61adc5730b83
>> [2] https://lwn.net/Articles/469652/
>>
>> Simon Schippers (1):
>>   usbnet: Add support for Byte Queue Limits (BQL)
>>
>>  drivers/net/usb/usbnet.c | 8 ++++++++
>>  1 file changed, 8 insertions(+)
>>
>> --
>> 2.43.0
>>

  reply	other threads:[~2025-11-05 10:40 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-11-04 16:13 Simon Schippers
2025-11-04 16:13 ` [PATCH net-next v1 1/1] " Simon Schippers
2025-11-04 17:00   ` Eric Dumazet
2025-11-05 10:35     ` Simon Schippers
2025-11-05 11:28       ` Eric Dumazet
2025-11-05 12:20         ` Simon Schippers
2025-11-05 12:34           ` Eric Dumazet
2025-11-05 12:58             ` Simon Schippers
2025-11-05 13:05               ` Eric Dumazet
2025-11-05 13:29                 ` Simon Schippers
2025-11-05 15:47                   ` Simon Schippers
2025-11-06 10:40                     ` Eric Dumazet
2025-11-06 11:07                       ` Eric Dumazet
2025-11-04 17:02 ` [PATCH net-next v1 0/1] " Eric Dumazet
2025-11-05 10:40   ` Simon Schippers [this message]
2025-11-05 10:35     ` Daniele Palmas
2025-11-05 11:04       ` Simon Schippers
2025-11-06  8:38         ` Daniele Palmas
2025-11-06 10:00           ` Simon Schippers
2025-11-07 13:34             ` Daniele Palmas
2025-11-05 11:34     ` Eric Dumazet

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f2a363d3-40d7-4a5f-a884-ec147a167ef5@tu-dortmund.de \
    --to=simon.schippers@tu-dortmund.de \
    --cc=andrew+netdev@lunn.ch \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-usb@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=oneukum@suse.com \
    --cc=pabeni@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®