From: Simon Schippers <simon.schippers@tu-dortmund.de>
To: Daniele Palmas <dnlplm@gmail.com>
Cc: Eric Dumazet <edumazet@google.com>,
oneukum@suse.com, andrew+netdev@lunn.ch, davem@davemloft.net,
kuba@kernel.org, pabeni@redhat.com, netdev@vger.kernel.org,
linux-usb@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH net-next v1 0/1] usbnet: Add support for Byte Queue Limits (BQL)
Date: Thu, 6 Nov 2025 11:00:43 +0100 [thread overview]
Message-ID: <676869a2-2e0d-4527-8494-db910b3a0018@tu-dortmund.de> (raw)
In-Reply-To: <CAGRyCJE1_xQQDfu1Tk3miZX-5T-+6rarzgPGo3=K-1zsFKpr+g@mail.gmail.com>
On 11/6/25 09:38, Daniele Palmas wrote:
> Hi Simon,
>
> Il giorno mer 5 nov 2025 alle ore 12:05 Simon Schippers
> <simon.schippers@tu-dortmund.de> ha scritto:
>>
>> On 11/5/25 11:35, Daniele Palmas wrote:
>>> Hello Simon,
>>>
>>> Il giorno mer 5 nov 2025 alle ore 11:40 Simon Schippers
>>> <simon.schippers@tu-dortmund.de> ha scritto:
>>>>
>>>> On 11/4/25 18:02, Eric Dumazet wrote:
>>>>> On Tue, Nov 4, 2025 at 8:14 AM Simon Schippers
>>>>> <simon.schippers@tu-dortmund.de> wrote:
>>>>>>
>>>>>> During recent testing, I observed significant latency spikes when using
>>>>>> Quectel 5G modems under load. Investigation revealed that the issue was
>>>>>> caused by bufferbloat in the usbnet driver.
>>>>>>
>>>>>> In the current implementation, usbnet uses a fixed tx_qlen of:
>>>>>>
>>>>>> USB2: 60 * 1518 bytes = 91.08 KB
>>>>>> USB3: 60 * 5 * 1518 bytes = 454.80 KB
>>>>>>
>>>>>> Such large transmit queues can be problematic, especially for cellular
>>>>>> modems. For example, with a typical celluar link speed of 10 Mbit/s, a
>>>>>> fully occupied USB3 transmit queue results in:
>>>>>>
>>>>>> 454.80 KB / (10 Mbit/s / 8 bit/byte) = 363.84 ms
>>>>>>
>>>>>> of additional latency.
>>>>>
>>>>> Doesn't 5G need to push more packets to the driver to get good aggregation ?
>>>>>
>>>>
>>>> Yes, but not 455 KB for low speeds. 5G requires a queue of a few ms to
>>>> aggregate enough packets for a frame but not of several hundred ms as
>>>> calculated in my example. And yes, there are situations where 5G,
>>>> especially FR2 mmWave, reaches Gbit/s speeds where a big queue is
>>>> required. But the dynamic queue limit approach of BQL should be well
>>>> suited for these varying speeds.
>>>>
>>>
>>> out of curiosity, related to the test with 5G Quectel, did you test
>>> enabling aggregation through QMAP (kernel module rmnet) or simply
>>> qmi_wwan raw_ip ?
>>>
>>> Regards,
>>> Daniele
>>>
>>
>> Hi Daniele,
>>
>> I simply used qmi_wwan. I actually never touched rmnet before.
>> Is the aggregation through QMAP what you and Eric mean with aggregation?
>> Because then I misunderstood it, because I was thinking about aggregating
>> enough (and not too many) packets in the usbnet queue.
>>
>
> I can't speak for Eric, but, yes, that is what I meant for
> aggregation, this is the common way those high-cat modems are used:
Hi Daniele,
I think I *really* have to take a look at rmnet and aggregation through
QMAP for future projects :)
> it's not clear to me if the change you are proposing could have any
> impact when rmnet is used, that's why I was asking the test
> conditions.
>
> Thanks,
> Daniele
>
This patch has an impact on the underlying USB physical transport of
rmnet. From my understanding, the call stack is as follows:
rmnet_map_tx_aggregate or rmnet_send_skb
|
| Calling dev_queue_xmit(skb)
V
qmi_wwan used for USB modem
|
| ndo_start_xmit(skb, net) is called
V
usbnet_start_xmit is executed where the size of the internal queue is
dynamically changed using the Byte Queue Limits algorithm by this patch.
Correct me if I am wrong, but I think in the end usbnet is used.
Thanks,
Simon
>> Thanks
>>
>>>>>>
>>>>>> To address this issue, this patch introduces support for
>>>>>> Byte Queue Limits (BQL) [1][2] in the usbnet driver. BQL dynamically
>>>>>> limits the amount of data queued in the driver, effectively reducing
>>>>>> latency without impacting throughput.
>>>>>> This implementation was successfully tested on several devices as
>>>>>> described in the commit.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Future work
>>>>>>
>>>>>> Due to offloading, TCP often produces SKBs up to 64 KB in size.
>>>>>
>>>>> Only for rates > 500 Mbit. After BQL, we had many more improvements in
>>>>> the stack.
>>>>> https://lwn.net/Articles/564978/
>>>>>
>>>>>
>>>>
>>>> I also saw these large SKBs, for example, for my USB2 Android tethering,
>>>> which advertises a network speed of < 500 Mbit/s.
>>>> I saw these large SKBs by looking at the file:
>>>>
>>>> cat /sys/class/net/INTERFACE/queues/tx-0/byte_queue_limits/inflight
>>>>
>>>> For UDP-only traffic, inflight always maxed out at MTU size.
>>>>
>>>> Thank you for your replies!
>>>>
>>>>>> To
>>>>>> further decrease buffer bloat, I tried to disable TSO, GSO and LRO but it
>>>>>> did not have the intended effect in my tests. The only dirty workaround I
>>>>>> found so far was to call netif_stop_queue() whenever BQL sets
>>>>>> __QUEUE_STATE_STACK_XOFF. However, a proper solution to this issue would
>>>>>> be desirable.
>>>>>>
>>>>>> I also plan to publish a scientific paper on this topic in the near
>>>>>> future.
>>>>>>
>>>>>> Thanks,
>>>>>> Simon
>>>>>>
>>>>>> [1] https://medium.com/@tom_84912/byte-queue-limits-the-unauthorized-biography-61adc5730b83
>>>>>> [2] https://lwn.net/Articles/469652/
>>>>>>
>>>>>> Simon Schippers (1):
>>>>>> usbnet: Add support for Byte Queue Limits (BQL)
>>>>>>
>>>>>> drivers/net/usb/usbnet.c | 8 ++++++++
>>>>>> 1 file changed, 8 insertions(+)
>>>>>>
>>>>>> --
>>>>>> 2.43.0
>>>>>>
>>>>
next prev parent reply other threads:[~2025-11-06 10:00 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-11-04 16:13 Simon Schippers
2025-11-04 16:13 ` [PATCH net-next v1 1/1] " Simon Schippers
2025-11-04 17:00 ` Eric Dumazet
2025-11-05 10:35 ` Simon Schippers
2025-11-05 11:28 ` Eric Dumazet
2025-11-05 12:20 ` Simon Schippers
2025-11-05 12:34 ` Eric Dumazet
2025-11-05 12:58 ` Simon Schippers
2025-11-05 13:05 ` Eric Dumazet
2025-11-05 13:29 ` Simon Schippers
2025-11-05 15:47 ` Simon Schippers
2025-11-06 10:40 ` Eric Dumazet
2025-11-06 11:07 ` Eric Dumazet
2025-11-04 17:02 ` [PATCH net-next v1 0/1] " Eric Dumazet
2025-11-05 10:40 ` Simon Schippers
2025-11-05 10:35 ` Daniele Palmas
2025-11-05 11:04 ` Simon Schippers
2025-11-06 8:38 ` Daniele Palmas
2025-11-06 10:00 ` Simon Schippers [this message]
2025-11-07 13:34 ` Daniele Palmas
2025-11-05 11:34 ` Eric Dumazet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=676869a2-2e0d-4527-8494-db910b3a0018@tu-dortmund.de \
--to=simon.schippers@tu-dortmund.de \
--cc=andrew+netdev@lunn.ch \
--cc=davem@davemloft.net \
--cc=dnlplm@gmail.com \
--cc=edumazet@google.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-usb@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=oneukum@suse.com \
--cc=pabeni@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®