From: Qiliang Yuan <qiliang.yuan@linux.dev>
To: "David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
Jakub Kicinski <kuba@kernel.org>,
Paolo Abeni <pabeni@redhat.com>, Simon Horman <horms@kernel.org>,
David Ahern <dsahern@kernel.org>,
Ido Schimmel <idosch@nvidia.com>,
Nikolay Aleksandrov <razor@blackwall.org>
Cc: Kuniyuki Iwashima <kuniyu@google.com>,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
Qiliang Yuan <qiliang.yuan@linux.dev>
Subject: [PATCH net-next v4 3/4] nexthop: release cached routes when a nexthop leaves a group
Date: Tue, 17 Nov 2026 21:28:32 -0500 [thread overview]
Message-ID: <20261117-bug-net-netns-move-rcu-barrier-v4-3-43e1ca0aac7f@linux.dev> (raw)
In-Reply-To: <20261117-bug-net-netns-move-rcu-barrier-v4-0-43e1ca0aac7f@linux.dev>
When a nexthop is removed while other members keep its group alive, only
the group array changes: the routes through the group stay in the FIB,
and the dsts they cached through the removed nexthop are released when
that nexthop is freed, from an RCU callback.
Until then those dsts stay valid. IPv6 route exceptions, such as the
ones created on Packet Too Big, are checked against the cookie of their
FIB node, which does not change, so a socket keeps sending through the
removed nexthop. When the nexthop goes away with its device, removing
the device's IPv6 addresses usually flushes the routing tree first, but
not when the device carries no IPv6 address.
Once all readers see the new group arrays, release the per-cpu routes and
exceptions of a removed IPv6 nexthop, as replacing a group already does
for its old members, and flush the IPv4 route cache once if any of the
groups carries IPv4 routes. The routes using the nexthop directly are
already gone at that point.
A connected UDP socket in netns A sends to a route through group {nh 1,
nh 2}, on an IPv6 exception of nh 1 created by Packet Too Big, while nh 1
is deleted. Datagrams still sent through nh 1 after the new group is
published and before nh 1 is freed, 10 trials, median and range, the
ones left after the change being sent before the dsts are released:
before (datagrams) after (datagrams) change
per trial 1140 (367-3788) 1 (1-3) -99.9%
Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
net/ipv4/nexthop.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/net/ipv4/nexthop.c b/net/ipv4/nexthop.c
index 42e55b5a755ee..3fccd4f2f916b 100644
--- a/net/ipv4/nexthop.c
+++ b/net/ipv4/nexthop.c
@@ -2116,6 +2116,7 @@ remove_nexthop_from_groups(struct net *net, struct nexthop *nh,
struct nh_grp_entry *nhge, *tmp;
LIST_HEAD(deferred_free);
bool need_flush = false;
+ struct nh_info *nhi;
/* If there is nothing to do, let's avoid the costly call to
* synchronize_net()
@@ -2130,6 +2131,20 @@ remove_nexthop_from_groups(struct net *net, struct nexthop *nh,
/* make sure all see the newly published array before releasing rtnl */
synchronize_net();
+ /* Routes through the groups can still hold dsts cached through @nh,
+ * which are only released when @nh is freed. Release them now that
+ * no reader can pick @nh from the old groups any more.
+ */
+ nhi = rtnl_dereference(nh->nh_info);
+ if (nhi->family == AF_INET6)
+ fib6_nh_release_dsts(&nhi->fib6_nh);
+ list_for_each_entry(nhge, &deferred_free, nh_list) {
+ if (!list_empty(&nhge->nh_parent->fi_list)) {
+ rt_cache_flush(net);
+ break;
+ }
+ }
+
/* Now safe to free percpu stats — all RCU readers have finished */
list_for_each_entry_safe(nhge, tmp, &deferred_free, nh_list) {
list_del(&nhge->nh_list);
--
2.43.0
next prev parent reply other threads:[~2026-11-18 2:29 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 2:28 [PATCH net-next v4 0/4] net: make moving a device to another netns cheaper Qiliang Yuan
2026-11-18 2:28 ` [PATCH net-next v4 1/4] ipv4: draw rt_genid from a sequence shared by all netns Qiliang Yuan
2026-11-18 2:28 ` [PATCH net-next v4 2/4] ipv6: take fib6_sernum " Qiliang Yuan
2026-11-18 2:28 ` Qiliang Yuan [this message]
2026-11-18 2:28 ` [PATCH net-next v4 4/4] net: remove rcu_barrier() from __dev_change_net_namespace() Qiliang Yuan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261117-bug-net-netns-move-rcu-barrier-v4-3-43e1ca0aac7f@linux.dev \
--to=qiliang.yuan@linux.dev \
--cc=davem@davemloft.net \
--cc=dsahern@kernel.org \
--cc=edumazet@kernel.org \
--cc=horms@kernel.org \
--cc=idosch@nvidia.com \
--cc=kuba@kernel.org \
--cc=kuniyu@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=razor@blackwall.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®