* [PATCH net-next 0/3] net: make moving a device to another netns cheaper
@ 2026-10-16 6:09 Qiliang Yuan
2026-10-16 6:09 ` [PATCH net-next 1/3] ipv4: draw rt_genid from a sequence shared by all netns Qiliang Yuan
` (2 more replies)
0 siblings, 3 replies; 11+ messages in thread
From: Qiliang Yuan @ 2026-10-16 6:09 UTC (permalink / raw)
To: David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, David Ahern, Ido Schimmel, Nikolay Aleksandrov
Cc: Kuniyuki Iwashima, netdev, linux-kernel, Qiliang Yuan
Moving a device to another netns waits in rcu_barrier() with RTNL held.
Container and sandbox managers move one end of a veth pair for every
new network namespace, so this barrier limits how fast such namespaces
can be created on the whole system, to about 75 per second in a 16
vCPU guest no matter how many threads create them.
The barrier was added for NETDEV_UNREGISTER_FINAL, which is long gone,
but it still keeps cached IPv4 routes through the moved device from
being accepted in the old netns. Routes through nexthop objects are not
covered by it: their cached dst entries are released by an RCU callback
queued from another RCU callback, so a socket in the old netns can still
send through a device that now belongs to another netns whenever the
route generation numbers of the two netns happen to match.
Patches 1 and 2 make the IPv4 rt_genid and the IPv6 fib6_sernum unique
across netns, which fixes that leak for nexthop objects and leaves
nothing that relies on the barrier. Patch 3 removes it. Patches 1 and 2
fix bugs that exist today and can go to net on their own if preferred.
Creating 400 netns, each with a veth pair whose host end is moved back
to the initial netns, a tap device, addresses and routes, median of 5
runs:
before after
1 thread 67 /s 440 /s
4 threads 75 /s 409 /s
16 threads 76 /s 283 /s
Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
---
Qiliang Yuan (3):
ipv4: draw rt_genid from a sequence shared by all netns
ipv6: take fib6_sernum from a sequence shared by all netns
net: remove rcu_barrier() from __dev_change_net_namespace()
include/net/net_namespace.h | 5 +----
net/core/dev.c | 1 -
net/ipv4/route.c | 16 +++++++++++++++-
net/ipv6/af_inet6.c | 1 -
net/ipv6/ip6_fib.c | 14 ++++++++++++--
5 files changed, 28 insertions(+), 9 deletions(-)
---
base-commit: 6b3e73774815910a39d8ca75b9ad7df963f61d50
change-id: 20261016-bug-net-netns-move-rcu-barrier-d998c6395830
Best regards,
--
Qiliang Yuan <odys.yuan@gmail.com>
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH net-next 1/3] ipv4: draw rt_genid from a sequence shared by all netns
2026-10-16 6:09 [PATCH net-next 0/3] net: make moving a device to another netns cheaper Qiliang Yuan
@ 2026-10-16 6:09 ` Qiliang Yuan
2026-10-17 4:35 ` kuniyu
2026-10-20 6:56 ` netdev-bot+sashiko
2026-10-16 6:09 ` [PATCH net-next 2/3] ipv6: take fib6_sernum " Qiliang Yuan
2026-10-16 6:09 ` [PATCH net-next 3/3] net: remove rcu_barrier() from __dev_change_net_namespace() Qiliang Yuan
2 siblings, 2 replies; 11+ messages in thread
From: Qiliang Yuan @ 2026-10-16 6:09 UTC (permalink / raw)
To: David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, David Ahern, Ido Schimmel, Nikolay Aleksandrov
Cc: Kuniyuki Iwashima, netdev, linux-kernel, Qiliang Yuan
An IPv4 route records the rt_genid of the netns it was created in, and
rt_is_expired() compares it with the rt_genid of the netns its device
belongs to when the route is checked. Every netns starts its rt_genid
at 0 and increments it on each cache flush, so different netns often
hold the same value.
When a device moves to another netns, routes cached through it remain
usable until dst_dev_put() points them to blackhole_netdev. For a route
using a nexthop object this happens in nexthop_free_rcu(), which
free_fib_info_rcu() queues from inside an RCU callback, so the
rcu_barrier() in __dev_change_net_namespace() does not wait for it. If
the rt_genid of the new netns matches the cached one in that window,
ipv4_dst_check() accepts the route and a socket in the old netns keeps
sending through a device that now belongs to another netns.
Take every rt_genid, including the initial one, from one global
sequence, so that a cached route never matches the rt_genid of another
netns.
A connected UDP socket in netns A caches a route through a nexthop
object on a veth. The veth is moved to netns B and brought up with one
RTM_NEWLINK, then the socket sends 200000 datagrams. B's rt_genid is
bumped 0 to 20 times before the move, 63 trials:
before after
trials sending after move 2 0
datagrams sent after move 2353 0
Fixes: 4c7e8084fd46 ("ipv4: Plumb support for nexthop object in a fib_info")
Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
---
include/net/net_namespace.h | 5 +----
net/ipv4/route.c | 16 +++++++++++++++-
2 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h
index 46b4c67e2966f..9c78f1d90fb62 100644
--- a/include/net/net_namespace.h
+++ b/include/net/net_namespace.h
@@ -556,10 +556,7 @@ static inline int rt_genid_ipv6(const struct net *net)
}
#endif
-static inline void rt_genid_bump_ipv4(struct net *net)
-{
- atomic_inc(&net->ipv4.rt_genid);
-}
+void rt_genid_bump_ipv4(struct net *net);
extern void (*__fib6_flush_trees)(struct net *net);
static inline void rt_genid_bump_ipv6(struct net *net)
diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index 595a3b7f17830..bd52d14476f6f 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -393,6 +393,20 @@ static inline int ip_rt_proc_init(void)
}
#endif /* CONFIG_PROC_FS */
+/*
+ * Draw every rt_genid from one sequence shared by all netns. A route keeps
+ * the genid of the netns it was created in, and is checked against the netns
+ * its device is in now. Distinct values make that check fail once the device
+ * has moved to another netns.
+ */
+static atomic_t rt_genid_seq;
+
+void rt_genid_bump_ipv4(struct net *net)
+{
+ atomic_set(&net->ipv4.rt_genid, atomic_inc_return(&rt_genid_seq));
+}
+EXPORT_SYMBOL_GPL(rt_genid_bump_ipv4);
+
static inline bool rt_is_expired(const struct rtable *rth)
{
bool res;
@@ -3716,7 +3730,7 @@ static struct pernet_operations __net_initdata ip_rt_ops = {
static __net_init int rt_genid_init(struct net *net)
{
- atomic_set(&net->ipv4.rt_genid, 0);
+ rt_genid_bump_ipv4(net);
atomic_set(&net->fnhe_genid, 0);
atomic_set(&net->ipv4.dev_addr_genid, get_random_u32());
return 0;
--
2.43.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH net-next 2/3] ipv6: take fib6_sernum from a sequence shared by all netns
2026-10-16 6:09 [PATCH net-next 0/3] net: make moving a device to another netns cheaper Qiliang Yuan
2026-10-16 6:09 ` [PATCH net-next 1/3] ipv4: draw rt_genid from a sequence shared by all netns Qiliang Yuan
@ 2026-10-16 6:09 ` Qiliang Yuan
2026-10-17 4:40 ` kuniyu
` (2 more replies)
2026-10-16 6:09 ` [PATCH net-next 3/3] net: remove rcu_barrier() from __dev_change_net_namespace() Qiliang Yuan
2 siblings, 3 replies; 11+ messages in thread
From: Qiliang Yuan @ 2026-10-16 6:09 UTC (permalink / raw)
To: David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, David Ahern, Ido Schimmel, Nikolay Aleksandrov
Cc: Kuniyuki Iwashima, netdev, linux-kernel, Qiliang Yuan
A per-cpu IPv6 route of a nexthop object records fib6_sernum of the
netns it was created in, and rt6_is_valid() compares it with fib6_sernum
of the netns its device belongs to when the route is checked. Every netns
starts its fib6_sernum at 1 and increments it on each tree change, so
different netns often hold the same value.
When a device moves to another netns, such a route remains usable until
nexthop_free_rcu() releases it, and fib6_info_destroy_rcu() queues that
callback from inside an RCU callback, after the device has moved. If the
fib6_sernum of the new netns matches the cached one in that window,
ip6_dst_check() accepts the route and a socket in the old netns keeps
sending through a device that now belongs to another netns.
Take every fib6_sernum, including the initial one, from one global
sequence, so that a cached route never matches the fib6_sernum of
another netns.
A connected UDP socket in netns A caches a route through an IPv6 nexthop
object on a veth. The veth is moved to netns B and brought up with one
RTM_NEWLINK, then the socket sends 200000 datagrams. A route is added
to and removed from B 0 to 20 times before the move, 63 trials:
before after
trials sending after move 3 0
datagrams sent after move 8187 0
Fixes: 8f34e53b60b3 ("ipv6: Use global sernum for dst validation with nexthop objects")
Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
---
net/ipv6/af_inet6.c | 1 -
net/ipv6/ip6_fib.c | 14 ++++++++++++--
2 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c
index a9b8b4b42a1af..db4cccfdc565c 100644
--- a/net/ipv6/af_inet6.c
+++ b/net/ipv6/af_inet6.c
@@ -945,7 +945,6 @@ static int __net_init inet6_net_init(struct net *net)
net->ipv6.sysctl.max_dst_opts_len = IP6_DEFAULT_MAX_DST_OPTS_LEN;
net->ipv6.sysctl.max_hbh_opts_len = IP6_DEFAULT_MAX_HBH_OPTS_LEN;
net->ipv6.sysctl.fib_notify_on_flag_change = 0;
- atomic_set(&net->ipv6.fib6_sernum, 1);
net->ipv6.sysctl.ioam6_id = IOAM6_DEFAULT_ID;
net->ipv6.sysctl.ioam6_id_wide = IOAM6_DEFAULT_ID_WIDE;
diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
index 0dc6ebee6e568..c66466be64451 100644
--- a/net/ipv6/ip6_fib.c
+++ b/net/ipv6/ip6_fib.c
@@ -89,13 +89,21 @@ static void fib6_walker_unlink(struct net *net, struct fib6_walker *w)
write_unlock_bh(&net->ipv6.fib6_walker_lock);
}
+/*
+ * Draw every fib6_sernum from one sequence shared by all netns, so that a
+ * route cached against one netns never matches the sernum of another one,
+ * e.g. after its device moved.
+ */
+static atomic_t fib6_sernum_seq;
+
static int fib6_new_sernum(struct net *net)
{
- int new, old = atomic_read(&net->ipv6.fib6_sernum);
+ int new, old = atomic_read(&fib6_sernum_seq);
do {
new = old < INT_MAX ? old + 1 : 1;
- } while (!atomic_try_cmpxchg(&net->ipv6.fib6_sernum, &old, new));
+ } while (!atomic_try_cmpxchg(&fib6_sernum_seq, &old, new));
+ atomic_set(&net->ipv6.fib6_sernum, new);
return new;
}
@@ -2466,6 +2474,8 @@ static int __net_init fib6_net_init(struct net *net)
size_t size = sizeof(struct hlist_head) * FIB6_TABLE_HASHSZ;
int err;
+ fib6_new_sernum(net);
+
err = fib6_notifier_init(net);
if (err)
return err;
--
2.43.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH net-next 3/3] net: remove rcu_barrier() from __dev_change_net_namespace()
2026-10-16 6:09 [PATCH net-next 0/3] net: make moving a device to another netns cheaper Qiliang Yuan
2026-10-16 6:09 ` [PATCH net-next 1/3] ipv4: draw rt_genid from a sequence shared by all netns Qiliang Yuan
2026-10-16 6:09 ` [PATCH net-next 2/3] ipv6: take fib6_sernum " Qiliang Yuan
@ 2026-10-16 6:09 ` Qiliang Yuan
2026-10-16 9:19 ` Eric Dumazet
2026-10-17 4:50 ` kuniyu
2 siblings, 2 replies; 11+ messages in thread
From: Qiliang Yuan @ 2026-10-16 6:09 UTC (permalink / raw)
To: David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, David Ahern, Ido Schimmel, Nikolay Aleksandrov
Cc: Kuniyuki Iwashima, netdev, linux-kernel, Qiliang Yuan
Commit 6549dd43c043 ("net: dev: fix the incorrect hold of net
namespace's lo device") added rcu_barrier() before
NETDEV_UNREGISTER_FINAL in dev_change_net_namespace(), so that dst
entries released by RCU callbacks were on the dst garbage list before
dst_ifdown() moved them to the loopback device of the old netns.
Commit 070f2d7e264a ("net: Drop NETDEV_UNREGISTER_FINAL") removed the
event but kept the barrier, and dead dst entries now point to
blackhole_netdev instead of a loopback device.
The barrier waits for the RCU callbacks pending on every CPU while RTNL
is held, which takes a full grace period on each netns move. Container
and sandbox managers move one end of a veth pair for every new network
namespace, so this limits how fast such namespaces can be created on the
whole system, regardless of how many threads create them.
Remove the barrier. The RCU callbacks queued by NETDEV_UNREGISTER
handlers free memory and drop device references, which the move does not
wait for. Cached IPv4 and IPv6 routes through the moved device are
rejected once its netns changes, since rt_genid and fib6_sernum are now
unique across netns.
Creating 400 netns, each with a veth pair whose host end is moved back
to the initial netns, a tap device, addresses and routes, in a 16 vCPU
guest, median of 5 runs:
before after
1 thread 68 /s 440 /s
4 threads 74 /s 409 /s
16 threads 77 /s 283 /s
Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
---
net/core/dev.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/net/core/dev.c b/net/core/dev.c
index 8eb9bf1ff9eb6..ef172905109ee 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -12994,7 +12994,6 @@ int __dev_change_net_namespace(struct net_device *dev, struct net *net,
* the device is just moving and can keep their slaves up.
*/
call_netdevice_notifiers(NETDEV_UNREGISTER, dev);
- rcu_barrier();
new_nsid = peernet2id_alloc(dev_net(dev), net, GFP_KERNEL);
--
2.43.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH net-next 3/3] net: remove rcu_barrier() from __dev_change_net_namespace()
2026-10-16 6:09 ` [PATCH net-next 3/3] net: remove rcu_barrier() from __dev_change_net_namespace() Qiliang Yuan
@ 2026-10-16 9:19 ` Eric Dumazet
2026-10-17 4:50 ` kuniyu
1 sibling, 0 replies; 11+ messages in thread
From: Eric Dumazet @ 2026-10-16 9:19 UTC (permalink / raw)
To: Qiliang Yuan
Cc: David S. Miller, Jakub Kicinski, Paolo Abeni, Simon Horman,
David Ahern, Ido Schimmel, Nikolay Aleksandrov,
Kuniyuki Iwashima, netdev, linux-kernel
Le ven. 16 oct. 2026 à 08:10, Qiliang Yuan <odys.yuan@gmail.com> a écrit :
>
> Commit 6549dd43c043 ("net: dev: fix the incorrect hold of net
> namespace's lo device") added rcu_barrier() before
> NETDEV_UNREGISTER_FINAL in dev_change_net_namespace(), so that dst
> entries released by RCU callbacks were on the dst garbage list before
> dst_ifdown() moved them to the loopback device of the old netns.
> Commit 070f2d7e264a ("net: Drop NETDEV_UNREGISTER_FINAL") removed the
> event but kept the barrier, and dead dst entries now point to
> blackhole_netdev instead of a loopback device.
>
> The barrier waits for the RCU callbacks pending on every CPU while RTNL
> is held, which takes a full grace period on each netns move. Container
> and sandbox managers move one end of a veth pair for every new network
> namespace, so this limits how fast such namespaces can be created on the
> whole system, regardless of how many threads create them.
>
> Remove the barrier. The RCU callbacks queued by NETDEV_UNREGISTER
> handlers free memory and drop device references, which the move does not
> wait for. Cached IPv4 and IPv6 routes through the moved device are
> rejected once its netns changes, since rt_genid and fib6_sernum are now
> unique across netns.
>
> Creating 400 netns, each with a veth pair whose host end is moved back
> to the initial netns, a tap device, addresses and routes, in a 16 vCPU
> guest, median of 5 runs:
>
> before after
> 1 thread 68 /s 440 /s
> 4 threads 74 /s 409 /s
> 16 threads 77 /s 283 /s
>
> Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
> ---
I think this is mostly fine. I was working on this months ago but got
distracted.
I sent two patches fixing pre-existing issues that would be widened by
your patch.
https://patchwork.kernel.org/project/netdevbpf/patch/20261016084802.2725187-1-edumazet@kernel.org/
https://patchwork.kernel.org/project/netdevbpf/patch/20261016091556.2750902-1-edumazet@kernel.org/
Reviewed-by: Eric Dumazet <edumazet@kernel.org>
Thanks.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH net-next 1/3] ipv4: draw rt_genid from a sequence shared by all netns
2026-10-16 6:09 ` [PATCH net-next 1/3] ipv4: draw rt_genid from a sequence shared by all netns Qiliang Yuan
@ 2026-10-17 4:35 ` kuniyu
2026-10-20 6:56 ` netdev-bot+sashiko
1 sibling, 0 replies; 11+ messages in thread
From: kuniyu @ 2026-10-17 4:35 UTC (permalink / raw)
To: Qiliang Yuan
Cc: David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, David Ahern, Ido Schimmel, Nikolay Aleksandrov,
netdev, linux-kernel
On Fri, 16 Oct 2026 14:09:53 +0800, Qiliang Yuan wrote:
> An IPv4 route records the rt_genid of the netns it was created in, and
> rt_is_expired() compares it with the rt_genid of the netns its device
> belongs to when the route is checked. Every netns starts its rt_genid
> at 0 and increments it on each cache flush, so different netns often
> hold the same value.
>
> When a device moves to another netns, routes cached through it remain
> usable until dst_dev_put() points them to blackhole_netdev. For a route
> using a nexthop object this happens in nexthop_free_rcu(), which
> free_fib_info_rcu() queues from inside an RCU callback, so the
> rcu_barrier() in __dev_change_net_namespace() does not wait for it. If
> the rt_genid of the new netns matches the cached one in that window,
> ipv4_dst_check() accepts the route and a socket in the old netns keeps
> sending through a device that now belongs to another netns.
>
> Take every rt_genid, including the initial one, from one global
> sequence, so that a cached route never matches the rt_genid of another
> netns.
>
> A connected UDP socket in netns A caches a route through a nexthop
> object on a veth. The veth is moved to netns B and brought up with one
> RTM_NEWLINK, then the socket sends 200000 datagrams. B's rt_genid is
> bumped 0 to 20 times before the move, 63 trials:
>
> before after
> trials sending after move 2 0
> datagrams sent after move 2353 0
>
> Fixes: 4c7e8084fd46 ("ipv4: Plumb support for nexthop object in a fib_info")
> Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH net-next 2/3] ipv6: take fib6_sernum from a sequence shared by all netns
2026-10-16 6:09 ` [PATCH net-next 2/3] ipv6: take fib6_sernum " Qiliang Yuan
@ 2026-10-17 4:40 ` kuniyu
2026-10-17 12:32 ` Eric Dumazet
2026-10-20 6:56 ` netdev-bot+sashiko
2 siblings, 0 replies; 11+ messages in thread
From: kuniyu @ 2026-10-17 4:40 UTC (permalink / raw)
To: Qiliang Yuan
Cc: David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, David Ahern, Ido Schimmel, Nikolay Aleksandrov,
netdev, linux-kernel
On Fri, 16 Oct 2026 14:09:54 +0800, Qiliang Yuan wrote:
> A per-cpu IPv6 route of a nexthop object records fib6_sernum of the
> netns it was created in, and rt6_is_valid() compares it with fib6_sernum
> of the netns its device belongs to when the route is checked. Every netns
> starts its fib6_sernum at 1 and increments it on each tree change, so
> different netns often hold the same value.
>
> When a device moves to another netns, such a route remains usable until
> nexthop_free_rcu() releases it, and fib6_info_destroy_rcu() queues that
> callback from inside an RCU callback, after the device has moved. If the
> fib6_sernum of the new netns matches the cached one in that window,
> ip6_dst_check() accepts the route and a socket in the old netns keeps
> sending through a device that now belongs to another netns.
>
> Take every fib6_sernum, including the initial one, from one global
> sequence, so that a cached route never matches the fib6_sernum of
> another netns.
>
> A connected UDP socket in netns A caches a route through an IPv6 nexthop
> object on a veth. The veth is moved to netns B and brought up with one
> RTM_NEWLINK, then the socket sends 200000 datagrams. A route is added
> to and removed from B 0 to 20 times before the move, 63 trials:
>
> before after
> trials sending after move 3 0
> datagrams sent after move 8187 0
>
> Fixes: 8f34e53b60b3 ("ipv6: Use global sernum for dst validation with nexthop objects")
> Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH net-next 3/3] net: remove rcu_barrier() from __dev_change_net_namespace()
2026-10-16 6:09 ` [PATCH net-next 3/3] net: remove rcu_barrier() from __dev_change_net_namespace() Qiliang Yuan
2026-10-16 9:19 ` Eric Dumazet
@ 2026-10-17 4:50 ` kuniyu
1 sibling, 0 replies; 11+ messages in thread
From: kuniyu @ 2026-10-17 4:50 UTC (permalink / raw)
To: Qiliang Yuan
Cc: David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, David Ahern, Ido Schimmel, Nikolay Aleksandrov,
netdev, linux-kernel
On Fri, 16 Oct 2026 14:09:55 +0800, Qiliang Yuan wrote:
> Commit 6549dd43c043 ("net: dev: fix the incorrect hold of net
> namespace's lo device") added rcu_barrier() before
> NETDEV_UNREGISTER_FINAL in dev_change_net_namespace(), so that dst
> entries released by RCU callbacks were on the dst garbage list before
> dst_ifdown() moved them to the loopback device of the old netns.
> Commit 070f2d7e264a ("net: Drop NETDEV_UNREGISTER_FINAL") removed the
> event but kept the barrier, and dead dst entries now point to
> blackhole_netdev instead of a loopback device.
>
> The barrier waits for the RCU callbacks pending on every CPU while RTNL
> is held, which takes a full grace period on each netns move. Container
> and sandbox managers move one end of a veth pair for every new network
> namespace, so this limits how fast such namespaces can be created on the
> whole system, regardless of how many threads create them.
>
> Remove the barrier. The RCU callbacks queued by NETDEV_UNREGISTER
> handlers free memory and drop device references, which the move does not
> wait for. Cached IPv4 and IPv6 routes through the moved device are
> rejected once its netns changes, since rt_genid and fib6_sernum are now
> unique across netns.
>
> Creating 400 netns, each with a veth pair whose host end is moved back
> to the initial netns, a tap device, addresses and routes, in a 16 vCPU
> guest, median of 5 runs:
>
> before after
> 1 thread 68 /s 440 /s
> 4 threads 74 /s 409 /s
> 16 threads 77 /s 283 /s
>
> Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Thanks !
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH net-next 2/3] ipv6: take fib6_sernum from a sequence shared by all netns
2026-10-16 6:09 ` [PATCH net-next 2/3] ipv6: take fib6_sernum " Qiliang Yuan
2026-10-17 4:40 ` kuniyu
@ 2026-10-17 12:32 ` Eric Dumazet
2026-10-20 6:56 ` netdev-bot+sashiko
2 siblings, 0 replies; 11+ messages in thread
From: Eric Dumazet @ 2026-10-17 12:32 UTC (permalink / raw)
To: Qiliang Yuan
Cc: David S. Miller, Jakub Kicinski, Paolo Abeni, Simon Horman,
David Ahern, Ido Schimmel, Nikolay Aleksandrov,
Kuniyuki Iwashima, netdev, linux-kernel
Le ven. 16 oct. 2026 à 08:10, Qiliang Yuan <odys.yuan@gmail.com> a écrit :
>
> A per-cpu IPv6 route of a nexthop object records fib6_sernum of the
> netns it was created in, and rt6_is_valid() compares it with fib6_sernum
> of the netns its device belongs to when the route is checked. Every netns
> starts its fib6_sernum at 1 and increments it on each tree change, so
> different netns often hold the same value.
>
> When a device moves to another netns, such a route remains usable until
> nexthop_free_rcu() releases it, and fib6_info_destroy_rcu() queues that
> callback from inside an RCU callback, after the device has moved. If the
> fib6_sernum of the new netns matches the cached one in that window,
> ip6_dst_check() accepts the route and a socket in the old netns keeps
> sending through a device that now belongs to another netns.
>
> Take every fib6_sernum, including the initial one, from one global
> sequence, so that a cached route never matches the fib6_sernum of
> another netns.
>
> A connected UDP socket in netns A caches a route through an IPv6 nexthop
> object on a veth. The veth is moved to netns B and brought up with one
> RTM_NEWLINK, then the socket sends 200000 datagrams. A route is added
> to and removed from B 0 to 20 times before the move, 63 trials:
>
> before after
> trials sending after move 3 0
> datagrams sent after move 8187 0
>
> Fixes: 8f34e53b60b3 ("ipv6: Use global sernum for dst validation with nexthop objects")
> Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
> ---
> net/ipv6/af_inet6.c | 1 -
> net/ipv6/ip6_fib.c | 14 ++++++++++++--
> 2 files changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c
> index a9b8b4b42a1af..db4cccfdc565c 100644
> --- a/net/ipv6/af_inet6.c
> +++ b/net/ipv6/af_inet6.c
> @@ -945,7 +945,6 @@ static int __net_init inet6_net_init(struct net *net)
> net->ipv6.sysctl.max_dst_opts_len = IP6_DEFAULT_MAX_DST_OPTS_LEN;
> net->ipv6.sysctl.max_hbh_opts_len = IP6_DEFAULT_MAX_HBH_OPTS_LEN;
> net->ipv6.sysctl.fib_notify_on_flag_change = 0;
> - atomic_set(&net->ipv6.fib6_sernum, 1);
>
> net->ipv6.sysctl.ioam6_id = IOAM6_DEFAULT_ID;
> net->ipv6.sysctl.ioam6_id_wide = IOAM6_DEFAULT_ID_WIDE;
> diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
> index 0dc6ebee6e568..c66466be64451 100644
> --- a/net/ipv6/ip6_fib.c
> +++ b/net/ipv6/ip6_fib.c
> @@ -89,13 +89,21 @@ static void fib6_walker_unlink(struct net *net, struct fib6_walker *w)
> write_unlock_bh(&net->ipv6.fib6_walker_lock);
> }
>
> +/*
> + * Draw every fib6_sernum from one sequence shared by all netns, so that a
> + * route cached against one netns never matches the sernum of another one,
> + * e.g. after its device moved.
> + */
> +static atomic_t fib6_sernum_seq;
> +
> static int fib6_new_sernum(struct net *net)
> {
> - int new, old = atomic_read(&net->ipv6.fib6_sernum);
> + int new, old = atomic_read(&fib6_sernum_seq);
>
> do {
> new = old < INT_MAX ? old + 1 : 1;
> - } while (!atomic_try_cmpxchg(&net->ipv6.fib6_sernum, &old, new));
> + } while (!atomic_try_cmpxchg(&fib6_sernum_seq, &old, new));
> + atomic_set(&net->ipv6.fib6_sernum, new);
>
The atomic_set() of net->ipv6.fib6_sernum after the cmpxchg()
is not ordered with other writers. fib6_new_sernum() callers only
hold the per-table tb6_lock, so two tables of the same netns can
race and leave the netns with the older of two values. Since values
are never reused, I think this only causes extra invalidations, but
the changelog should say so: fib6_sernum is no longer monotonic per
netns.
> return new;
> }
> @@ -2466,6 +2474,8 @@ static int __net_init fib6_net_init(struct net *net)
> size_t size = sizeof(struct hlist_head) * FIB6_TABLE_HASHSZ;
> int err;
>
> + fib6_new_sernum(net);
There is no need to draw a value from the shared sequence at netns
creation. Netns creation is exactly what you are trying to make
cheaper, and the shared cacheline does not need to be written there.
You can instead keep per-netns initial values that the shared
sequence never hands out:
keep atomic_set(&net->ipv6.fib6_sernum, 1) in
inet6_net_init() (0 is not usable, rt->sernum == 0 and
FIB6_NO_SERNUM_CHANGE have their own meaning), and make the
sequence never return 1 (restart at 2)
A netns that never bumped can then not match a value from the
sequence. Also, the moved device is down after the move, and
NETDEV_UP in the new netns bumps it (rt_cache_flush() for IPv4,
the route inserted by addrconf_add_dev() for IPv6) before the
device can transmit.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH net-next 1/3] ipv4: draw rt_genid from a sequence shared by all netns
2026-10-16 6:09 ` [PATCH net-next 1/3] ipv4: draw rt_genid from a sequence shared by all netns Qiliang Yuan
2026-10-17 4:35 ` kuniyu
@ 2026-10-20 6:56 ` netdev-bot+sashiko
1 sibling, 0 replies; 11+ messages in thread
From: netdev-bot+sashiko @ 2026-10-20 6:56 UTC (permalink / raw)
To: odys.yuan
Cc: davem, edumazet, kuba, pabeni, horms, dsahern, idosch, razor,
kuniyu, netdev, linux-kernel
Thank you for your contribution! Sashiko AI review found 1 potential
issue(s) to consider.
Critical: 0 · High: 0 · Medium: 1 · Low: 0
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] The patch says it stops cached routes from matching another
netns's genid, but that guarantee breaks once the 32-bit counter wraps.
--
ipv4: draw rt_genid from a sequence shared by all netns
This patch takes every IPv4 rt_genid from one global atomic sequence
instead of a per-netns counter starting at 0. That includes the initial
value set in rt_genid_init(). The aim is that a cached route never
matches the rt_genid of another netns after its device moves, which
closes the window before the deferred dst_dev_put() for nexthop-backed
routes.
> diff --git a/net/ipv4/route.c b/net/ipv4/route.c
> index 595a3b7f17830..bd52d14476f6f 100644
> --- a/net/ipv4/route.c
> +++ b/net/ipv4/route.c
> @@ -393,6 +393,20 @@ static inline int ip_rt_proc_init(void)
> }
> #endif /* CONFIG_PROC_FS */
>
> +/*
> + * Draw every rt_genid from one sequence shared by all netns. A route keeps
> + * the genid of the netns it was created in, and is checked against the netns
> + * its device is in now. Distinct values make that check fail once the device
> + * has moved to another netns.
> + */
> +static atomic_t rt_genid_seq;
> +
> +void rt_genid_bump_ipv4(struct net *net)
> +{
> + atomic_set(&net->ipv4.rt_genid, atomic_inc_return(&rt_genid_seq));
> +}
[Severity: Medium]
Does the uniqueness guarantee still hold once rt_genid_seq wraps?
The commit message says:
"so that a cached route never matches the rt_genid of another netns."
rt_genid_seq is a 32-bit atomic_t shared by every netns. Its values are
only distinct modulo 2^32, and nothing stops it from handing out a value
that a live netns or a cached rtable still holds.
Every rt_cache_flush() in any netns advances the counter. One source is
ipv4_sysctl_rtcache_flush(), which calls rt_cache_flush(net) on every
write to /proc/sys/net/ipv4/route/flush with no rate limit.
sysctl_route_net_init() exposes that sysctl in netns owned by
non-initial user namespaces:
net/ipv4/route.c:sysctl_route_net_init() {
...
/* Don't export non-whitelisted sysctls to unprivileged users */
if (net->user_ns != &init_user_ns) {
if (tbl[0].procname != ipv4_route_flush_procname)
table_size = 0;
}
...
}
After a wrap, can this sequence still happen?
rt_dst_alloc() stamps a route through dev D in netns A with genid g
D moves to netns B
B's latest bump (e.g. a fib_netdev_event() flush on link up) is g
ipv4_dst_check()
rt_is_expired()
rth->rt_genid != rt_genid_ipv4(dev_net_rcu(rth->dst.dev)) /* false */
route is accepted
If so, a socket in A keeps sending through D until the deferred RCU
teardown reaches dst_dev_put(). The chance per move is small (about
2^-32, and userspace can't see the counter), but the "never" in the
commit message is not strictly true.
The later patch in this series, "net: remove rcu_barrier() from
__dev_change_net_namespace()", removes the barrier because "rt_genid and
fib6_sernum are now unique across netns". After that change, genid
equality modulo 2^32 becomes the only protection for all cached IPv4
routes through a moved device, not just the nexthop-backed ones.
Would it make sense to state the modular limit in the commit message,
widen the genid, or have rt_is_expired() compare the originating netns
directly instead of relying only on genid equality?
> +EXPORT_SYMBOL_GPL(rt_genid_bump_ipv4);
> +
> static inline bool rt_is_expired(const struct rtable *rth)
> {
> bool res;
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261016-bug-net-netns-move-rcu-barrier-v1-0-4157d798658f%40gmail.com
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH net-next 2/3] ipv6: take fib6_sernum from a sequence shared by all netns
2026-10-16 6:09 ` [PATCH net-next 2/3] ipv6: take fib6_sernum " Qiliang Yuan
2026-10-17 4:40 ` kuniyu
2026-10-17 12:32 ` Eric Dumazet
@ 2026-10-20 6:56 ` netdev-bot+sashiko
2 siblings, 0 replies; 11+ messages in thread
From: netdev-bot+sashiko @ 2026-10-20 6:56 UTC (permalink / raw)
To: odys.yuan
Cc: davem, edumazet, kuba, pabeni, horms, dsahern, idosch, razor,
kuniyu, netdev, linux-kernel
Thank you for your contribution! Sashiko AI review found 1 potential
issue(s) to consider.
Critical: 0 · High: 0 · Medium: 1 · Low: 0
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] The patch makes every netns take fib6_sernum (and fn_sernum)
from one static counter, fib6_sernum_seq.
--
ipv6: take fib6_sernum from a sequence shared by all netns
This patch replaces the per-netns fib6_sernum counter with one static
sequence, fib6_sernum_seq, that all netns draw from, including for the
initial value. The goal is that a nexthop pcpu route cached against one
netns can no longer match the fib6_sernum of another netns after its
device moves.
> diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
> index 0dc6ebee6e568..c66466be64451 100644
> --- a/net/ipv6/ip6_fib.c
> +++ b/net/ipv6/ip6_fib.c
> @@ -89,13 +89,21 @@ static void fib6_walker_unlink(struct net *net, struct fib6_walker *w)
> write_unlock_bh(&net->ipv6.fib6_walker_lock);
> }
>
> +/*
> + * Draw every fib6_sernum from one sequence shared by all netns, so that a
> + * route cached against one netns never matches the sernum of another one,
> + * e.g. after its device moved.
> + */
> +static atomic_t fib6_sernum_seq;
> +
> static int fib6_new_sernum(struct net *net)
> {
> - int new, old = atomic_read(&net->ipv6.fib6_sernum);
> + int new, old = atomic_read(&fib6_sernum_seq);
>
> do {
> new = old < INT_MAX ? old + 1 : 1;
> - } while (!atomic_try_cmpxchg(&net->ipv6.fib6_sernum, &old, new));
> + } while (!atomic_try_cmpxchg(&fib6_sernum_seq, &old, new));
> + atomic_set(&net->ipv6.fib6_sernum, new);
>
> return new;
> }
[Severity: Medium]
The comment says a cached route "never matches the sernum of another one".
Does the code guarantee that once fib6_sernum_seq wraps?
The shared counter still wraps to 1 at INT_MAX:
new = old < INT_MAX ? old + 1 : 1;
Nothing checks whether the reissued value is still held by a live netns, a
fib6_node, or a cached dst.
Validation has not changed. ip6_rt_pcpu_alloc() records only the value:
if (f6i->nh)
pcpu_rt->sernum = rt_genid_ipv6(dev_net(dev));
rt6_is_valid() then compares it for plain equality against the device's
current netns:
return rt6->sernum == rt_genid_ipv6(dev_net(rt6->dst.dev));
and ip6_dst_check() returns the dst on a match:
if (rt->sernum)
return rt6_is_valid(rt) ? dst : NULL;
After a wrap, couldn't netns B be given the same value S that a socket in
netns A cached? If A's device then moves into B while B holds S, the
original problem comes back in the window before nexthop_free_rcu() runs.
This patch also links netns that used to be independent. Each successful
fib6_add(), fib6_flush_trees(), the link up/down sync paths, and
rt6_insert_exception() (reachable from a received ICMPv6 Packet Too Big)
now advance the shared counter. Any netns can therefore push it through a
full cycle, including one owned by an unprivileged user namespace.
Before this patch, netns N could only reissue an old value after N itself
made about 2^31 draws. Now, once the global counter has wrapped, can a
route add in N draw exactly N's current fib6_sernum, or a value still held
by a stale cookie? In that case the cached dsts that fib6_add() should
invalidate would still pass ip6_dst_check() and keep using the old route.
A hit needs about 2^31 draws plus an exact match, so it is far rarer than
the per-netns collisions this patch removes. The commit message makes the
same absolute claim, though:
so that a cached route never matches the fib6_sernum of
another netns.
The later patch in this series, "net: remove rcu_barrier() from
__dev_change_net_namespace()", relies on this as well. It cites that
"rt_genid and fib6_sernum are now unique across netns" as its reason for
dropping the barrier. The end of the series doesn't seem to add a wrap
guard, a wider generation, or a check on the originating netns.
Could the comment and the commit message be reworded so they don't claim
uniqueness? Or, if the rcu_barrier() removal depends on uniqueness, could
the series handle the wrap explicitly?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261016-bug-net-netns-move-rcu-barrier-v1-0-4157d798658f%40gmail.com
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2026-10-20 6:57 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-10-16 6:09 [PATCH net-next 0/3] net: make moving a device to another netns cheaper Qiliang Yuan
2026-10-16 6:09 ` [PATCH net-next 1/3] ipv4: draw rt_genid from a sequence shared by all netns Qiliang Yuan
2026-10-17 4:35 ` kuniyu
2026-10-20 6:56 ` netdev-bot+sashiko
2026-10-16 6:09 ` [PATCH net-next 2/3] ipv6: take fib6_sernum " Qiliang Yuan
2026-10-17 4:40 ` kuniyu
2026-10-17 12:32 ` Eric Dumazet
2026-10-20 6:56 ` netdev-bot+sashiko
2026-10-16 6:09 ` [PATCH net-next 3/3] net: remove rcu_barrier() from __dev_change_net_namespace() Qiliang Yuan
2026-10-16 9:19 ` Eric Dumazet
2026-10-17 4:50 ` kuniyu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®