Re: [PATCH net-next] vhost/net: check peek_head_len after signal to guest to avoid delays

[Jon Kohler <jon@nutanix.com>]

> On Nov 25, 2025, at 6:50 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> 
> !-------------------------------------------------------------------|
>  CAUTION: External Email
> 
> |-------------------------------------------------------------------!
> 
> On Tue, Nov 25, 2025 at 11:00:33AM -0700, Jon Kohler wrote:
>> In non-busypoll handle_rx paths, if peek_head_len returns 0, the RX
>> loop breaks, the RX wait queue is re-enabled, and vhost_net_signal_used
>> is called to flush done_idx and notify the guest if needed.
>> 
>> However, signaling the guest can take non-trivial time. During this
>> window, additional RX payloads may arrive on rx_ring without further
>> kicks. These new payloads will sit unprocessed until another kick
>> arrives, increasing latency. In high-rate UDP RX workloads, this was
>> observed to occur over 20k times per second.
>> 
>> To minimize this window and improve opportunities to process packets
>> promptly, immediately call peek_head_len after signaling. If new packets
>> are found, treat it as a busy poll interrupt and requeue handle_rx,
>> improving fairness to TX handlers and other pending CPU work. This also
>> helps suppress unnecessary thread wakeups, reducing waker CPU demand.
>> 
>> Signed-off-by: Jon Kohler <jon@nutanix.com>
> 
> Given this is supposed to be a performance improvement,
> pls include info on the effect this has on performance. Thanks!

I had already mentioned we’re avoiding ~20k schedulers/IPIs in that
example, but I can add more detail. Let’s resolve the other parts of
the thread first and go from there?

> 
>> ---
>> drivers/vhost/net.c | 21 +++++++++++++++++++++
>> 1 file changed, 21 insertions(+)
>> 
>> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
>> index 35ded4330431..04cb5f1dc6e4 100644
>> --- a/drivers/vhost/net.c
>> +++ b/drivers/vhost/net.c
>> @@ -1015,6 +1015,27 @@ static int vhost_net_rx_peek_head_len(struct vhost_net *net, struct sock *sk,
>> struct vhost_virtqueue *tvq = &tnvq->vq;
>> int len = peek_head_len(rnvq, sk);
>> 
>> + if (!len && rnvq->done_idx) {
>> + /* When idle, flush signal first, which can take some
>> + * time for ring management and guest notification.
>> + * Afterwards, check one last time for work, as the ring
>> + * may have received new work during the notification
>> + * window.
>> + */
>> + vhost_net_signal_used(rnvq, *count);
>> + *count = 0;
>> + if (peek_head_len(rnvq, sk)) {
>> + /* More work came in during the notification
>> + * window. To be fair to the TX handler and other
>> + * potentially pending work items, pretend like
>> + * this was a busy poll interruption so that
>> + * the RX handler will be rescheduled and try
>> + * again.
>> + */
>> + *busyloop_intr = true;
>> + }
>> + }
>> +
>> if (!len && rvq->busyloop_timeout) {
>> /* Flush batched heads first */
>> vhost_net_signal_used(rnvq, *count);
>> -- 
>> 2.43.0
>