[PATCH net-next] veth: set the real number of queues before registering the devices

[PATCH net-next] veth: set the real number of queues before registering the devices

[Qiliang Yuan]

Since commit 9d3684c24a52 ("veth: create by default nr_possible_cpus
queues"), each veth device is allocated with one rx and one tx queue
per possible CPU, and veth_newlink() reduces the real number of queues
to one unless the user asked for a number.

Both devices are already registered at that point, so
netif_set_real_num_tx_queues() and netif_set_real_num_rx_queues()
remove the sysfs objects of every extra queue that register_netdevice()
has just created, each one emitting an add and a remove uevent, and the
tx side also waits for an expedited RCU grace period. This happens with
RTNL held, twice per pair, and grows with the number of possible CPUs:
with 240 of them a veth pair emits about 1900 uevents, and udev
processing them slows down later sysfs work such as deleting devices.

Set the number of queues before registering each device. For a device
that is not registered yet this only updates the counters, so the extra
queues never appear in sysfs, and the error paths that undid a
registration are no longer needed.

Creating and then deleting 500 veth pairs, one ip -batch each, in a 16
vCPU guest, median of 3 runs, per pair:

                                   before      after
  16 possible CPUs    create      1644 us     520 us
                      delete     11822 us    9750 us
  240 possible CPUs   create     17253 us    1339 us
                      delete     17082 us    9941 us
                      uevents        1918          6

Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
---
 drivers/net/veth.c | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 38f263d4f06b4..92ad064567327 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -1875,6 +1875,10 @@ static int veth_newlink(struct net_device *dev,
 
 	netif_inherit_tso_max(peer, dev);
 
+	err = veth_init_queues(peer, tb);
+	if (err)
+		goto err_register_peer;
+
 	err = register_netdevice(peer);
 	if (err < 0)
 		goto err_register_peer;
@@ -1904,6 +1908,10 @@ static int veth_newlink(struct net_device *dev,
 	else
 		snprintf(dev->name, IFNAMSIZ, DRV_NAME "%%d");
 
+	err = veth_init_queues(dev, tb);
+	if (err)
+		goto err_register_dev;
+
 	err = register_netdevice(dev);
 	if (err < 0)
 		goto err_register_dev;
@@ -1917,16 +1925,10 @@ static int veth_newlink(struct net_device *dev,
 	priv = netdev_priv(dev);
 	rcu_assign_pointer(priv->peer, peer);
 	netdev_hold(peer, &priv->peer_tracker, GFP_KERNEL);
-	err = veth_init_queues(dev, tb);
-	if (err)
-		goto err_queues;
 
 	priv = netdev_priv(peer);
 	rcu_assign_pointer(priv->peer, dev);
 	netdev_hold(dev, &priv->peer_tracker, GFP_KERNEL);
-	err = veth_init_queues(peer, tb);
-	if (err)
-		goto err_peer_queues;
 
 	veth_disable_gro(dev);
 	/* update XDP supported features */
@@ -1935,12 +1937,6 @@ static int veth_newlink(struct net_device *dev,
 
 	return 0;
 
-err_peer_queues:
-	netdev_put(dev, &priv->peer_tracker);
-	priv = netdev_priv(dev);
-err_queues:
-	netdev_put(peer, &priv->peer_tracker);
-	unregister_netdevice(dev);
 err_register_dev:
 	/* nothing to do */
 err_configure_peer:

---
base-commit: 6b3e73774815910a39d8ca75b9ad7df963f61d50
change-id: 20261016-bug-veth-init-queues-before-register-eebe763a01a7

Best regards,
-- 
Qiliang Yuan <odys.yuan@gmail.com>

Re: [PATCH net-next] veth: set the real number of queues before registering the devices

[Wang Zhan]

On Fri, 16 Oct 2026 14:51:05 +0800 Qiliang Yuan wrote:
> Since commit 9d3684c24a52 ("veth: create by default nr_possible_cpus
> queues"), each veth device is allocated with one rx and one tx queue
> per possible CPU, and veth_newlink() reduces the real number of queues
> to one unless the user asked for a number.
>
> Both devices are already registered at that point, so
> netif_set_real_num_tx_queues() and netif_set_real_num_rx_queues()
> remove the sysfs objects of every extra queue that register_netdevice()
> has just created, each one emitting an add and a remove uevent, and the
> tx side also waits for an expedited RCU grace period. This happens with
> RTNL held, twice per pair, and grows with the number of possible CPUs:
> with 240 of them a veth pair emits about 1900 uevents, and udev
> processing them slows down later sysfs work such as deleting devices.
>
> Set the number of queues before registering each device. For a device
> that is not registered yet this only updates the counters, so the extra
> queues never appear in sysfs, and the error paths that undid a
> registration are no longer needed.
>
> Creating and then deleting 500 veth pairs, one ip -batch each, in a 16
> vCPU guest, median of 3 runs, per pair:
>
>                                    before      after
>   16 possible CPUs    create      1644 us     520 us
>                       delete     11822 us    9750 us
>   240 possible CPUs   create     17253 us    1339 us
>                       delete     17082 us    9941 us
>                       uevents        1918          6
>
> Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>

Looks fine to me. virtio-net uses this ordering too. I checked that
veth's ndo_init() leaves the real queue counts alone, and I don't see
any missing cleanup after the move.

I ran some basic veth tests under QEMU TCG; no regressions found.

Reviewed-by: Wang Zhan <wang.zhan@smartx.com>
Tested-by: Wang Zhan <wang.zhan@smartx.com>

BTW, Li RongQing also posted a net fix changing veth_init_queues(peer, tb)
to use tbp, so there may be a conflict.

https://lore.kernel.org/netdev/20261014013620.2497-1-lirongqing@baidu.com/

Re: [PATCH net-next] veth: set the real number of queues before registering the devices

[Eric Dumazet]

Le ven. 16 oct. 2026 à 08:51, Qiliang Yuan <odys.yuan@gmail.com> a écrit :
>
> Since commit 9d3684c24a52 ("veth: create by default nr_possible_cpus
> queues"), each veth device is allocated with one rx and one tx queue
> per possible CPU, and veth_newlink() reduces the real number of queues
> to one unless the user asked for a number.
>
> Both devices are already registered at that point, so
> netif_set_real_num_tx_queues() and netif_set_real_num_rx_queues()
> remove the sysfs objects of every extra queue that register_netdevice()
> has just created, each one emitting an add and a remove uevent, and the
> tx side also waits for an expedited RCU grace period. This happens with
> RTNL held, twice per pair, and grows with the number of possible CPUs:
> with 240 of them a veth pair emits about 1900 uevents, and udev
> processing them slows down later sysfs work such as deleting devices.
>
> Set the number of queues before registering each device. For a device
> that is not registered yet this only updates the counters, so the extra
> queues never appear in sysfs, and the error paths that undid a
> registration are no longer needed.
>
> Creating and then deleting 500 veth pairs, one ip -batch each, in a 16
> vCPU guest, median of 3 runs, per pair:
>
>                                    before      after
>   16 possible CPUs    create      1644 us     520 us
>                       delete     11822 us    9750 us
>   240 possible CPUs   create     17253 us    1339 us
>                       delete     17082 us    9941 us
>                       uevents        1918          6
>
> Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
> ---
>  drivers/net/veth.c | 20 ++++++++------------
>  1 file changed, 8 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/net/veth.c b/drivers/net/veth.c
> index 38f263d4f06b4..92ad064567327 100644
> --- a/drivers/net/veth.c
> +++ b/drivers/net/veth.c
> @@ -1875,6 +1875,10 @@ static int veth_newlink(struct net_device *dev,
>
>         netif_inherit_tso_max(peer, dev);
>
> +       err = veth_init_queues(peer, tb);
> +       if (err)
> +               goto err_register_peer;
> +
>         err = register_netdevice(peer);
>         if (err < 0)
>                 goto err_register_peer;
> @@ -1904,6 +1908,10 @@ static int veth_newlink(struct net_device *dev,
>         else
>                 snprintf(dev->name, IFNAMSIZ, DRV_NAME "%%d");
>
> +       err = veth_init_queues(dev, tb);
> +       if (err)
> +               goto err_register_dev;
> +
>         err = register_netdevice(dev);
>         if (err < 0)
>                 goto err_register_dev;
> @@ -1917,16 +1925,10 @@ static int veth_newlink(struct net_device *dev,
>         priv = netdev_priv(dev);
>         rcu_assign_pointer(priv->peer, peer);
>         netdev_hold(peer, &priv->peer_tracker, GFP_KERNEL);
> -       err = veth_init_queues(dev, tb);
> -       if (err)
> -               goto err_queues;
>
>         priv = netdev_priv(peer);
>         rcu_assign_pointer(priv->peer, dev);
>         netdev_hold(dev, &priv->peer_tracker, GFP_KERNEL);
> -       err = veth_init_queues(peer, tb);

Orthogonal to your patch, but here we should have used @tbp instead of @tb.

# ip link add va type veth peer vb numtxqueues 4

-> vb has 4 tx queues, but only 1 real tx queue, which is quite unexpected.

# ip -d link sh vb
13: vb@va: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN
mode DEFAULT group default qlen 1000
    link/ether e2:53:93:f3:f4:f8 brd ff:ff:ff:ff:ff:ff promiscuity 0
allmulti 0 minmtu 68 maxmtu 65535
    veth addrgenmode eui64 numtxqueues 4 numrxqueues 8 gso_max_size
65536 gso_max_segs 65535 tso_max_size 524280 tso_max_segs 65535
gro_max_size 65536 gso_ipv4_max_size 65536 gro_ipv4_max_size 65536

# ls /sys/class/net/vb/queues/
rx-0  tx-0