* [PATCH net] ipvs: wait for RCU callbacks before dst_ops teardown
@ 2026-11-14 19:54 pavankumaryalagada
2026-11-15 16:09 ` Julian Anastasov
0 siblings, 1 reply; 2+ messages in thread
From: pavankumaryalagada @ 2026-11-14 19:54 UTC (permalink / raw)
To: horms, ja
Cc: pablo, fw, phil, davem, edumazet, kuba, pabeni, netdev,
lvs-devel, netfilter-devel, coreteam, linux-kernel, skhan,
Yalagada Pavan Kumar, syzbot+84e27adb95fd2a1fe9bb, stable
From: Yalagada Pavan Kumar <pavankumaryalagada@gmail.com>
During network namespace cleanup, IPVS releases cached destinations
using RCU callbacks. However, ip6_route_net_exit() destroys the
per-CPU destination counters before these callbacks complete.
The RCU callbacks call dst_release(), which updates the destroyed
counters and causes a NULL pointer dereference in
percpu_counter_add_batch().
Call rcu_barrier() at the end of __ip_vs_cleanup_batch() to make
sure all pending IPVS RCU callbacks finish before the destination
counters are destroyed.
Reported-by: syzbot+84e27adb95fd2a1fe9bb@syzkaller.appspotmail.com
Link: https://lore.kernel.org/all/6af795e2.98e60839.1e023a.00d4.GAE@google.com/T/
Closes: https://syzkaller.appspot.com/bug?extid=84e27adb95fd2a1fe9bb
Fixes: 026ace060dfe ("ipvs: optimize dst usage for real server")
Cc: stable@vger.kernel.org
Signed-off-by: Yalagada Pavan Kumar <pavankumaryalagada@gmail.com>
---
net/netfilter/ipvs/ip_vs_core.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c
index 8a3c880fdb26..9c456efe13c7 100644
--- a/net/netfilter/ipvs/ip_vs_core.c
+++ b/net/netfilter/ipvs/ip_vs_core.c
@@ -2553,6 +2553,11 @@ static void __net_exit __ip_vs_cleanup_batch(struct list_head *net_list)
IP_VS_DBG(2, "ipvs netns %d released\n", ipvs->gen);
net->ipvs = NULL;
}
+
+ /*
+ * Wait for ip_vs_dest_dst_rcu_free() callbacks before dst_ops teardown
+ */
+ rcu_barrier();
}
static void __net_exit __ip_vs_dev_cleanup_batch(struct list_head *net_list)
--
2.43.0
^ permalink raw reply [flat|nested] 2+ messages in thread* Re: [PATCH net] ipvs: wait for RCU callbacks before dst_ops teardown
2026-11-14 19:54 [PATCH net] ipvs: wait for RCU callbacks before dst_ops teardown pavankumaryalagada
@ 2026-11-15 16:09 ` Julian Anastasov
0 siblings, 0 replies; 2+ messages in thread
From: Julian Anastasov @ 2026-11-15 16:09 UTC (permalink / raw)
To: Yalagada Pavan Kumar
Cc: Simon Horman, pablo, fw, phil, David S. Miller, edumazet, kuba,
pabeni, netdev, lvs-devel, netfilter-devel, coreteam,
linux-kernel, skhan, syzbot+84e27adb95fd2a1fe9bb, stable,
Wei Wang
Hello,
Adding Wei Wang to the CC list...
On Sun, 15 Nov 2026, pavankumaryalagada@gmail.com wrote:
> From: Yalagada Pavan Kumar <pavankumaryalagada@gmail.com>
>
> During network namespace cleanup, IPVS releases cached destinations
> using RCU callbacks. However, ip6_route_net_exit() destroys the
> per-CPU destination counters before these callbacks complete.
> The RCU callbacks call dst_release(), which updates the destroyed
> counters and causes a NULL pointer dereference in
> percpu_counter_add_batch().
>
> Call rcu_barrier() at the end of __ip_vs_cleanup_batch() to make
> sure all pending IPVS RCU callbacks finish before the destination
> counters are destroyed.
What you see is a result of more fundamental problem.
IPVS is supposed to catch all NETDEV_DOWN events coming from the
pernet device phase which come before the pernet subsys phase.
So, on cleanup_net IPVS is supposed to run its subsys cleanup
with no traffic, no devs, no cached dsts, no netfilter hooks, etc.
But in my tests free_fib_info_rcu -> fib_nh_release ->
fib_nh_common_release -> rt_fibinfo_free_cpus calls
dst_dev_put() where the dst device that IPVS listens for is replaced
with a blackhole_netdev. This happens before the NETDEV_DOWN
event and as result IPVS misses the event because dst->dev is
already replaced and does not match. When such dst_cache
remains for the pernet subsys cleanup you see these crashes
because IPVS has another problem that it does not call
dst_release immediately but from RCU callback.
I now created a change to call dst_release immediately
but the problem with the blackholing before NETDEV_DOWN remains.
It can hurt other event listeners.
My tests were with tools/testing/selftests/net/netfilter/ipvs.sh
where I modify it to run only the NAT test. What I see is veth12
to be blackholed before its NETDEV_DOWN event and ip_vs_forget_dev()
missed it. Trying to find more information...
>
> Reported-by: syzbot+84e27adb95fd2a1fe9bb@syzkaller.appspotmail.com
> Link: https://lore.kernel.org/all/6af795e2.98e60839.1e023a.00d4.GAE@google.com/T/
> Closes: https://syzkaller.appspot.com/bug?extid=84e27adb95fd2a1fe9bb
> Fixes: 026ace060dfe ("ipvs: optimize dst usage for real server")
> Cc: stable@vger.kernel.org
> Signed-off-by: Yalagada Pavan Kumar <pavankumaryalagada@gmail.com>
> ---
> net/netfilter/ipvs/ip_vs_core.c | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c
> index 8a3c880fdb26..9c456efe13c7 100644
> --- a/net/netfilter/ipvs/ip_vs_core.c
> +++ b/net/netfilter/ipvs/ip_vs_core.c
> @@ -2553,6 +2553,11 @@ static void __net_exit __ip_vs_cleanup_batch(struct list_head *net_list)
> IP_VS_DBG(2, "ipvs netns %d released\n", ipvs->gen);
> net->ipvs = NULL;
> }
> +
> + /*
> + * Wait for ip_vs_dest_dst_rcu_free() callbacks before dst_ops teardown
> + */
> + rcu_barrier();
> }
>
> static void __net_exit __ip_vs_dev_cleanup_batch(struct list_head *net_list)
> --
> 2.43.0
Regards
--
Julian Anastasov <ja@ssi.bg>
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2026-11-15 16:10 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-14 19:54 [PATCH net] ipvs: wait for RCU callbacks before dst_ops teardown pavankumaryalagada
2026-11-15 16:09 ` Julian Anastasov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®