Re: [PATCH] vxlan: serialize nexthop fdb_list against cross-device concurrent updates

[Ido Schimmel <idosch@nvidia.com>]

On Tue, Sep 01, 2026 at 02:02:53PM +0900, Seungwon Bae wrote:
> struct nexthop.fdb_list is a per-nexthop global list of the vxlan fdb
> entries that reference an fdb-nexthop.  vxlan serializes it only with the
> per-device vxlan->hash_lock.  When two vxlan devices reference the same
> fdb-nexthop, their entries share one nh->fdb_list but each device takes
> only its own hash_lock.  vxlan_cleanup() (the aging GC timer) runs in
> softirq with only its device's hash_lock and calls vxlan_fdb_destroy() ->
> list_del_rcu(&f->nh_list); a concurrent list_add_tail_rcu()/list_del_rcu()
> from the other device corrupts the shared list, leaving a freed
> kmalloc-128 struct vxlan_fdb linked on nh->fdb_list (use-after-free later
> consumed by vxlan_fdb_nh_flush()).
> 
> On CONFIG_DEBUG_LIST/KASAN this reproduces as:
> 
>   list_del corruption. next->prev should be ..., but was dead000000000122.
>   __list_del_entry_valid_or_report <- vxlan_fdb_destroy <- vxlan_cleanup
>   <- run_timer_softirq
>   BUG: KASAN: slab-use-after-free in vxlan_fdb_destroy
> 
> Serialize all nh->fdb_list mutations with the nexthop's own spinlock
> (nh->lock), which - unlike rtnl/hash_lock - is usable from the softirq
> aging path.  Lock order is hash_lock -> nh->lock at every mutation site;
> vxlan_fdb_nh_flush() keeps traversing under rcu_read_lock().
> 
> Verified with a KASAN + CONFIG_DEBUG_LIST kernel and an unprivileged
> (userns+netns) reproducer that runs two vxlan devices churning
> add + aging-delete on a shared fdb-nexthop: BEFORE = 490 list_del
> corruptions + slab-use-after-free; AFTER = 0 corruptions, 0 KASAN
> reports, no deadlock.

vxlan_cleanup() shouldn't age out FDB entries that point to a nexthop
ID. While their timestamp is updated by vxlan_snoop(), such entries
cannot roam, unlike normal entries.

I see two possible fixes:

1. Have vxlan_cleanup() skip such entries.

2. Reject FDB entries that point to a nexthop ID from being dynamic,
both when created and when an existing entry is updated.

The first is safer, but the second is the more correct fix and I doubt
anyone is relying on this behavior. Therefore, I suggest going with the
second option. Something like [1]. AFAICT, this means that
nexthop.fdb_list will only be manipulated under RTNL.

Also you need a Fixes tag:

Fixes: 1274e1cc4226 ("vxlan: ecmp support for mac fdb entries")

Please read [2] and run [3] and [4] on v2 and future submissions.

Thanks

[1]
diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c
index 459f19f7071e..be95af64a1f5 100644
--- a/drivers/net/vxlan/vxlan_core.c
+++ b/drivers/net/vxlan/vxlan_core.c
@@ -996,6 +996,12 @@ static int vxlan_fdb_update_existing(struct vxlan_dev *vxlan,
 		return -EOPNOTSUPP;
 	}
 
+	if (rcu_access_pointer(f->nh) &&
+	    !(state & (NUD_PERMANENT | NUD_NOARP))) {
+		NL_SET_ERR_MSG(extack, "Cannot make a nexthop fdb dynamic");
+		return -EOPNOTSUPP;
+	}
+
 	/* Do not allow an externally learned entry to take over an entry added
 	 * by the user.
 	 */
@@ -1257,6 +1263,11 @@ static int vxlan_fdb_add(struct ndmsg *ndm, struct nlattr *tb[],
 	if (err)
 		return err;
 
+	if (nhid && !(ndm->ndm_state & (NUD_PERMANENT | NUD_NOARP))) {
+		NL_SET_ERR_MSG(extack, "A nexthop fdb cannot be dynamic");
+		return -EINVAL;
+	}
+
 	if (vxlan->default_dst.remote_ip.sa.sa_family != ip.sa.sa_family)
 		return -EAFNOSUPPORT;
 
diff --git a/tools/testing/selftests/net/fib_nexthops.sh b/tools/testing/selftests/net/fib_nexthops.sh
index 3d347126730a..431d7bed7622 100755
--- a/tools/testing/selftests/net/fib_nexthops.sh
+++ b/tools/testing/selftests/net/fib_nexthops.sh
@@ -533,6 +533,20 @@ ipv6_fdb_grp_fcnal()
 	run_cmd "$BRIDGE fdb add 02:02:00:00:00:14 dev vx10 nhid 61 self"
 	log_test $? 255 "Fdb mac add with nexthop"
 
+	# fdb entries with a nexthop group cannot be aged out
+	run_cmd "$BRIDGE fdb add 02:02:00:00:00:15 dev vx10 nhid 102 self static"
+	log_test $? 0 "Fdb mac add with nexthop group and static state"
+
+	run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic"
+	log_test $? 255 "Fdb mac add with nexthop group and dynamic state"
+
+	run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self"
+	run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic"
+	log_test $? 255 "Fdb mac replace with nexthop group and dynamic state"
+
+	run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 2001:db8:91::11 self dynamic"
+	log_test $? 255 "Fdb mac append with nexthop group and dynamic state"
+
 	run_cmd "$IP -6 ro add 2001:db8:101::1/128 nhid 66"
 	log_test $? 2 "Route add with fdb nexthop"
 
@@ -669,6 +683,20 @@ ipv4_fdb_grp_fcnal()
 	run_cmd "$BRIDGE fdb add 02:02:00:00:00:14 dev vx10 nhid 12 self"
 	log_test $? 255 "Fdb mac add with nexthop"
 
+	# fdb entries with a nexthop group cannot be aged out
+	run_cmd "$BRIDGE fdb add 02:02:00:00:00:15 dev vx10 nhid 102 self static"
+	log_test $? 0 "Fdb mac add with nexthop group and static state"
+
+	run_cmd "$BRIDGE fdb add 02:02:00:00:00:16 dev vx10 nhid 102 self dynamic"
+	log_test $? 255 "Fdb mac add with nexthop group and dynamic state"
+
+	run_cmd "$BRIDGE fdb add 02:02:00:00:00:17 dev vx10 nhid 102 self"
+	run_cmd "$BRIDGE fdb replace 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic"
+	log_test $? 255 "Fdb mac replace with nexthop group and dynamic state"
+
+	run_cmd "$BRIDGE fdb append 02:02:00:00:00:17 dev vx10 dst 10.0.0.3 self dynamic"
+	log_test $? 255 "Fdb mac append with nexthop group and dynamic state"
+
 	run_cmd "$IP ro add 172.16.0.0/22 nhid 16"
 	log_test $? 2 "Route add with fdb nexthop"

[2] https://docs.kernel.org/next/process/maintainer-netdev.html
[3] https://github.com/linux-netdev/nipa#running-locally
[4] https://github.com/sashiko-dev/sashiko#usage