mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH net-next v3 0/3] ipv6: tie the dst cookie to the lookup that returned the dst
@ 2026-11-17  9:21 Ralf Lici
  2026-11-17  9:21 ` [PATCH net-next v3 1/3] ipv6: stamp the default route node with a fresh serial on link failure Ralf Lici
                   ` (2 more replies)
  0 siblings, 3 replies; 12+ messages in thread
From: Ralf Lici @ 2026-11-17  9:21 UTC (permalink / raw)
  To: netdev
  Cc: David Ahern, Ido Schimmel, David S. Miller, Eric Dumazet,
	Jakub Kicinski, Paolo Abeni, Simon Horman, Neal Cardwell,
	Kuniyuki Iwashima, Steffen Klassert, Herbert Xu,
	Florian Westphal, Sabrina Dubroca, Antonio Quartulli,
	linux-kernel

Hi,

This series implements a first part of the model proposed in [1] to
address the late cookie sampling race: a dst can be stored together with
a cookie read after a FIB change that its lookup did not see, and then
passes dst_check until the next change.

The case that made me post it as a series is xfrm. Adding a policy
restamps every fib6 node, so a socket whose lookup ran just before the
insertion keeps its plain dst and leaks plaintext although the policy
matches. I reproduced it for TCP connect and connected UDP, and the same
window lets a more specific blackhole route be bypassed for routes using
nexthop objects. The ovpn patch [2] hit the same pattern in
dst_cache_set_ip6.

The conversion is initially limited to tcp_v6_connect,
ip6_datagram_dst_update and ip6_sk_dst_lookup_flow. Other TCP TX cache
paths, including established-socket route refills, header rebuilds and
accepted-socket setup, still use late cookie sampling. This gives the
lookup-cookie API a small set of users and concrete fixes to review. If
the approach is accepted, I plan to follow up with a broader conversion
across the networking core and drivers.

Compared to [1], the fallback cookie is 0 instead of the pre-lookup
generation, so fib6_flush_trees is left alone, and nexthop-object dsts
are checked in ip6_dst_check instead of passing the generation down to
ip6_rt_pcpu_alloc. The other cache users (dst_cache, sctp, ipvs,
flowtable, sk_rx_dst, the xfrm bundle cookies) are not converted and
behave as before.

I tested it in a VM with lockdep, PROVE_RCU and DEBUG_ATOMIC_SLEEP,
using a debug-only delay after the lookup to hit the window: no
warnings, and the plaintext and blackhole bypass are gone in these test
cases.

I am targeting net-next to get feedback on the lookup-cookie API and its
incremental adoption before extending the conversion across the
remaining cache users. This series provides the writer-side
prerequisites and converts three socket-cache paths; it does not
complete the conversion. I am not requesting stable backports at this
stage. Backport scope and dependencies can be assessed separately once
the approach has been reviewed.

[1] https://lore.kernel.org/netdev/20260901122501.482920-1-ralf@mandelbit.com/
[2] https://lore.kernel.org/netdev/cover.1789639346.git.ralf@mandelbit.com/

Thanks,

Ralf Lici
Mandelbit Srl

---
Changes since v2 https://lore.kernel.org/all/cover.1794384146.git.ralf@mandelbit.com/
- Rebase onto current net-next, preserving the connected-only cache
  lookup introduced by f778e80b3daf ("ipv6: fix cached route reuse for
  control messages"). (Jakub)
- Move the serial update after rt6_multipath_rebalance in fib6_ifup and
  fib6_ifdown. (Sashiko)
- Add Fixes tags and improve wording. (Sashiko)
- Clarify that this is the groundwork for fixing the race and a partial
  conversion to the new API: other users will be covered in a follow-up
  series.

Changes since v1 https://lore.kernel.org/netdev/cover.1793806941.git.ralf@mandelbit.com/
- Fix kdoc return statements in patch 2/3. (NIPA)

Ralf Lici (3):
  ipv6: stamp the default route node with a fresh serial on link failure
  ipv6: add ip6_dst_lookup_flow_cookie
  ipv6: use lookup cookies in selected socket dst cache paths

 include/net/ip6_route.h | 18 +++++++++----
 include/net/ipv6.h      | 13 ++++++++++
 net/ipv6/datagram.c     |  6 +++--
 net/ipv6/ip6_output.c   | 57 ++++++++++++++++++++++++++++++++++++++---
 net/ipv6/route.c        | 30 +++++++++++-----------
 net/ipv6/tcp_ipv6.c     |  5 ++--
 6 files changed, 101 insertions(+), 28 deletions(-)


base-commit: 6ff4c5942a301675eb12e5b49d53a81e604624a1
-- 
2.56.0


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH net-next v3 1/3] ipv6: stamp the default route node with a fresh serial on link failure
  2026-11-17  9:21 [PATCH net-next v3 0/3] ipv6: tie the dst cookie to the lookup that returned the dst Ralf Lici
@ 2026-11-17  9:21 ` Ralf Lici
  2026-11-19  9:22   ` netdev-bot+sashiko
  2026-11-17  9:21 ` [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie Ralf Lici
  2026-11-17  9:21 ` [PATCH net-next v3 3/3] ipv6: use lookup cookies in selected socket dst cache paths Ralf Lici
  2 siblings, 1 reply; 12+ messages in thread
From: Ralf Lici @ 2026-11-17  9:21 UTC (permalink / raw)
  To: netdev
  Cc: David Ahern, Ido Schimmel, David S. Miller, Eric Dumazet,
	Jakub Kicinski, Paolo Abeni, Simon Horman, Neal Cardwell,
	Kuniyuki Iwashima, Steffen Klassert, Herbert Xu,
	Florian Westphal, Sabrina Dubroca, Antonio Quartulli,
	linux-kernel

ip6_link_failure invalidates the cached dsts of an RTF_DEFAULT route by
writing -1 into the node's fn_sernum. Once the node already holds -1,
another link failure leaves the serial unchanged, so dsts cached with
that cookie can survive subsequent failures.

The assignment also leaves net->ipv6.fib6_sernum unchanged, making the
invalidation invisible to lookups watching the namespace generation.

Use fib6_update_sernum under tb6_lock to allocate and stamp a fresh
serial. This makes repeated invalidations distinguishable and allows the
lookup-cookie generation checks introduced later in this series to
detect them.

Advancing the namespace generation also invalidates unrelated dsts using
nexthop objects, which are validated against that generation.

Found while reviewing the IPv6 routing code to address the dst-cache
race.

Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
Signed-off-by: Ralf Lici <ralf@mandelbit.com>
---
Changes since v2 https://lore.kernel.org/all/143461e2c94a20060ff4c7d6e5eb46c47ac46aa0.1794384146.git.ralf@mandelbit.com/
- Add a Fixes tag referencing the initial git import. (Sashiko)
- Clarify that the nexthop-object invalidation is a harmless side
  effect, rather than a deficiency being fixed. (Sashiko)

No changes since v1 https://lore.kernel.org/netdev/5b039a5a24808f2678eedd283c29f8dae514ef5e.1793806941.git.ralf@mandelbit.com/

 net/ipv6/route.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 4a4c5ace63c5..cc505817399b 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -2873,13 +2873,13 @@ static void ip6_link_failure(struct sk_buff *skb)
 			rt6_remove_exception_rt(rt);
 		} else {
 			struct fib6_info *from;
-			struct fib6_node *fn;
 
 			from = rcu_dereference(rt->from);
-			if (from) {
-				fn = rcu_dereference(from->fib6_node);
-				if (fn && (rt->rt6i_flags & RTF_DEFAULT))
-					WRITE_ONCE(fn->fn_sernum, -1);
+			if (from && from->fib6_table &&
+			    (rt->rt6i_flags & RTF_DEFAULT)) {
+				spin_lock_bh(&from->fib6_table->tb6_lock);
+				fib6_update_sernum(rt6_exception_net(rt), from);
+				spin_unlock_bh(&from->fib6_table->tb6_lock);
 			}
 		}
 		rcu_read_unlock();
-- 
2.56.0


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie
  2026-11-17  9:21 [PATCH net-next v3 0/3] ipv6: tie the dst cookie to the lookup that returned the dst Ralf Lici
  2026-11-17  9:21 ` [PATCH net-next v3 1/3] ipv6: stamp the default route node with a fresh serial on link failure Ralf Lici
@ 2026-11-17  9:21 ` Ralf Lici
  2026-11-18  8:48   ` Xia Yan
  2026-11-19  9:22   ` netdev-bot+sashiko
  2026-11-17  9:21 ` [PATCH net-next v3 3/3] ipv6: use lookup cookies in selected socket dst cache paths Ralf Lici
  2 siblings, 2 replies; 12+ messages in thread
From: Ralf Lici @ 2026-11-17  9:21 UTC (permalink / raw)
  To: netdev
  Cc: David Ahern, Ido Schimmel, David S. Miller, Eric Dumazet,
	Jakub Kicinski, Paolo Abeni, Simon Horman, Neal Cardwell,
	Kuniyuki Iwashima, Steffen Klassert, Herbert Xu,
	Florian Westphal, Sabrina Dubroca, Antonio Quartulli,
	linux-kernel

Persistent dst caches read the validation cookie with rt6_get_cookie
after route lookup has returned. For routes without their own sernum,
this reads the current fn_sernum of the associated node. A concurrent
FIB change or xfrm policy insertion can therefore pair an old dst with a
new serial, allowing subsequent dst_check calls to accept the stale
result until another relevant change.

Add ip6_dst_lookup_flow_cookie to return the dst together with its
validation cookie. Sample the cookie between two reads of the IPv6
generation and return cookie 0 if the generation changed. This lets the
route dst serve the current packet while preventing its subsequent reuse
through cache validation. xfrm bundles retain their existing validation
through their own route and path cookies.

For nexthop-object dsts, return the pre-lookup generation and require
the supplied cookie to match rt->sernum in ip6_dst_check. This also
rejects a dst allocated with a newer serial after route selection.
Existing callers passing rt6_get_cookie retain their validation
behavior.

The generation checks require writers to publish routing changes before
advancing the serial. Move the serial updates in fib6_ifup and
fib6_ifdown after rt6_multipath_rebalance, preserving their existing
invalidation scope. Otherwise, a lookup can observe the new serial while
selecting a nexthop using the old ECMP bounds, without detecting a
generation change.

This establishes the API and writer ordering needed to address late
cookie sampling. The following patch converts selected socket cache
users while other persistent caches are left for follow-up conversions.

Signed-off-by: Ralf Lici <ralf@mandelbit.com>
---
Changes since v2 https://lore.kernel.org/all/e48cac0cdf9f3adcedecbbfff4a314a98bc22d01.1794384146.git.ralf@mandelbit.com/
- Move the serial update after rt6_multipath_rebalance in fib6_ifup and
  fib6_ifdown. (Sashiko)
- Clarify that this introduces the new API while the following patch
  converts *some* users to it while leaving the rest as follow-up.

Changes since v1 https://lore.kernel.org/netdev/e40f48e925a0950eac99a3f084e485e82cabe88f.1793806941.git.ralf@mandelbit.com/
- Fix kdoc return statement in ip6_dst_lookup_flow and
  ip6_dst_lookup_flow_cookie. (NIPA)

 include/net/ipv6.h    | 13 +++++++++++
 net/ipv6/ip6_output.c | 51 +++++++++++++++++++++++++++++++++++++++++--
 net/ipv6/route.c      |  8 +++----
 3 files changed, 66 insertions(+), 6 deletions(-)

diff --git a/include/net/ipv6.h b/include/net/ipv6.h
index 579df1516d25..5a6c66504056 100644
--- a/include/net/ipv6.h
+++ b/include/net/ipv6.h
@@ -1062,6 +1062,11 @@ int ip6_dst_lookup(struct net *net, struct sock *sk, struct dst_entry **dst,
 #if IS_ENABLED(CONFIG_IPV6)
 struct dst_entry *ip6_dst_lookup_flow(struct net *net, const struct sock *sk, struct flowi6 *fl6,
 				      const struct in6_addr *final_dst);
+struct dst_entry *ip6_dst_lookup_flow_cookie(struct net *net,
+					     const struct sock *sk,
+					     struct flowi6 *fl6,
+					     const struct in6_addr *final_dst,
+					     u32 *cookie);
 #else
 static inline struct dst_entry *ip6_dst_lookup_flow(struct net *net, const struct sock *sk,
 						    struct flowi6 *fl6,
@@ -1069,6 +1074,14 @@ static inline struct dst_entry *ip6_dst_lookup_flow(struct net *net, const struc
 {
 	return ERR_PTR(-EAFNOSUPPORT);
 }
+
+static inline struct dst_entry *
+ip6_dst_lookup_flow_cookie(struct net *net, const struct sock *sk,
+			   struct flowi6 *fl6, const struct in6_addr *final_dst,
+			   u32 *cookie)
+{
+	return ERR_PTR(-EAFNOSUPPORT);
+}
 #endif
 
 struct dst_entry *ip6_sk_dst_lookup_flow(struct sock *sk, struct flowi6 *fl6,
diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
index 3cc911f6618e..b87ccb5586cf 100644
--- a/net/ipv6/ip6_output.c
+++ b/net/ipv6/ip6_output.c
@@ -1279,8 +1279,7 @@ EXPORT_SYMBOL_GPL(ip6_dst_lookup);
  *
  *	This function performs a route lookup on the given flow.
  *
- *	It returns a valid dst pointer on success, or a pointer encoded
- *	error code.
+ *	Return: a valid dst pointer on success or a pointer encoded error code.
  */
 struct dst_entry *ip6_dst_lookup_flow(struct net *net, const struct sock *sk, struct flowi6 *fl6,
 				      const struct in6_addr *final_dst)
@@ -1300,6 +1299,54 @@ struct dst_entry *ip6_dst_lookup_flow(struct net *net, const struct sock *sk, st
 }
 EXPORT_SYMBOL_GPL(ip6_dst_lookup_flow);
 
+/**
+ *	ip6_dst_lookup_flow_cookie - perform route lookup on flow with ipsec
+ *				     and return the matching dst cookie
+ *	@net: Network namespace to perform lookup in
+ *	@sk: socket which provides route info
+ *	@fl6: flow to lookup
+ *	@final_dst: final destination address for ipsec lookup
+ *	@cookie: where to store the cookie for the returned dst
+ *
+ *	Same as ip6_dst_lookup_flow, but also returns the cookie describing
+ *	the routing state the dst was obtained from. A caller caching the dst
+ *	should use this cookie when validating it with dst_check. If the
+ *	IPv6 FIB changed during the lookup, a route dst no longer validates
+ *	with it. An xfrm bundle is validated by xfrm and ignores the cookie.
+ *
+ *	Return: a valid dst pointer on success or a pointer encoded error code.
+ */
+struct dst_entry *ip6_dst_lookup_flow_cookie(struct net *net,
+					     const struct sock *sk,
+					     struct flowi6 *fl6,
+					     const struct in6_addr *final_dst,
+					     u32 *cookie)
+{
+	const struct rt6_info *rt;
+	struct dst_entry *dst;
+	int gen0, gen1;
+
+	gen0 = rt_genid_ipv6(net);
+	/* read the generation before looking at the FIB or xfrm policies */
+	smp_rmb();
+
+	dst = ip6_dst_lookup_flow(net, sk, fl6, final_dst);
+	if (IS_ERR(dst))
+		return dst;
+
+	rt = dst_rt6_info(dst);
+	*cookie = rt->sernum ? gen0 : rt6_get_cookie(rt);
+
+	/* keep the FIB and cookie reads before the final generation read */
+	smp_rmb();
+	gen1 = rt_genid_ipv6(net);
+	if (unlikely(gen0 != gen1))
+		*cookie = 0;
+
+	return dst;
+}
+EXPORT_SYMBOL_GPL(ip6_dst_lookup_flow_cookie);
+
 /**
  *	ip6_sk_dst_lookup_flow - perform socket cached route lookup on flow
  *	@sk: socket which provides the dst cache and route info
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index cc505817399b..c54844b83828 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -2817,7 +2817,7 @@ INDIRECT_CALLABLE_SCOPE struct dst_entry *ip6_dst_check(struct dst_entry *dst,
 	rt = dst_rt6_info(dst);
 
 	if (rt->sernum)
-		return rt6_is_valid(rt) ? dst : NULL;
+		return (rt6_is_valid(rt) && cookie == rt->sernum) ? dst : NULL;
 
 	rcu_read_lock();
 
@@ -4911,8 +4911,8 @@ static int fib6_ifup(struct fib6_info *rt, void *p_arg)
 	if (rt != net->ipv6.fib6_null_entry && !rt->nh &&
 	    rt->fib6_nh->fib_nh_dev == arg->dev) {
 		rt->fib6_nh->fib_nh_flags &= ~arg->nh_flags;
-		fib6_update_sernum_upto_root(net, rt);
 		rt6_multipath_rebalance(rt);
+		fib6_update_sernum_upto_root(net, rt);
 	}
 
 	return 0;
@@ -5015,8 +5015,8 @@ static int fib6_ifdown(struct fib6_info *rt, void *p_arg)
 			}
 			rt6_multipath_nh_flags_set(rt, dev, RTNH_F_DEAD |
 						   RTNH_F_LINKDOWN);
-			fib6_update_sernum(net, rt);
 			rt6_multipath_rebalance(rt);
+			fib6_update_sernum(net, rt);
 		}
 		return -2;
 	case NETDEV_CHANGE:
@@ -5024,8 +5024,8 @@ static int fib6_ifdown(struct fib6_info *rt, void *p_arg)
 		    rt->fib6_flags & (RTF_LOCAL | RTF_ANYCAST))
 			break;
 		rt->fib6_nh->fib_nh_flags |= RTNH_F_LINKDOWN;
-		fib6_update_sernum(net, rt);
 		rt6_multipath_rebalance(rt);
+		fib6_update_sernum(net, rt);
 		break;
 	}
 
-- 
2.56.0


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH net-next v3 3/3] ipv6: use lookup cookies in selected socket dst cache paths
  2026-11-17  9:21 [PATCH net-next v3 0/3] ipv6: tie the dst cookie to the lookup that returned the dst Ralf Lici
  2026-11-17  9:21 ` [PATCH net-next v3 1/3] ipv6: stamp the default route node with a fresh serial on link failure Ralf Lici
  2026-11-17  9:21 ` [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie Ralf Lici
@ 2026-11-17  9:21 ` Ralf Lici
  2026-11-19  9:22   ` netdev-bot+sashiko
  2 siblings, 1 reply; 12+ messages in thread
From: Ralf Lici @ 2026-11-17  9:21 UTC (permalink / raw)
  To: netdev
  Cc: David Ahern, Ido Schimmel, David S. Miller, Eric Dumazet,
	Jakub Kicinski, Paolo Abeni, Simon Horman, Neal Cardwell,
	Kuniyuki Iwashima, Willem de Bruijn, Christian Brauner, Xia Yan,
	Jeff Layton, Sebastian Andrzej Siewior, Qi Tang,
	Steffen Klassert, Herbert Xu, Florian Westphal, Sabrina Dubroca,
	Antonio Quartulli, linux-kernel

tcp_v6_connect, ip6_datagram_dst_update and ip6_sk_dst_lookup_flow
obtain a dst and sample its validation cookie afterwards. If a matching
xfrm policy is inserted between lookup and cookie sampling, the plain
route can be cached with a post-insertion serial. Subsequent packets
then pass dst validation and bypass the new policy until another
relevant FIB change.

Store the cookie returned by ip6_dst_lookup_flow_cookie in these paths
instead. Add ip6_dst_store_cookie to accept the supplied cookie,
retaining ip6_dst_store for unconverted callers.

This is a partial conversion. Established TCP route refills, header
rebuilds and accepted-socket setup still use late cookie sampling. Those
paths and other persistent caches, including SCTP and dst_cache users,
are left for follow-up conversions.

This race was found while investigating how mutable socket routing
properties affect the dst cache in ovpn's UDP TX path.

The socket-cache case was then reproduced in a VM by adding a debug-only
kernel delay after the route and policy lookup, then inserting a
matching xfrm policy before cookie sampling. TCP connect and connected
UDP sockets continued sending plaintext after the policy insertion. With
lookup cookies, the stale cached route was rejected on reuse.

Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
Signed-off-by: Ralf Lici <ralf@mandelbit.com>
---
Changes since v2 https://lore.kernel.org/all/43917b9119e13e54dcde0e2941d3f4a1fe255df7.1794384146.git.ralf@mandelbit.com/
- Add a Fixes tag referencing the initial git import. (Sashiko)
- Clearly state that this is only a partial conversion to the new API.

No changes since v1 https://lore.kernel.org/netdev/832892eaa7d2371f20459c3134c72914e42970b0.1793806941.git.ralf@mandelbit.com/

 include/net/ip6_route.h | 18 +++++++++++++-----
 net/ipv6/datagram.c     |  6 ++++--
 net/ipv6/ip6_output.c   |  6 ++++--
 net/ipv6/route.c        | 12 ++++++------
 net/ipv6/tcp_ipv6.c     |  5 +++--
 5 files changed, 30 insertions(+), 17 deletions(-)

diff --git a/include/net/ip6_route.h b/include/net/ip6_route.h
index 95f936be32c6..2684f6069a5e 100644
--- a/include/net/ip6_route.h
+++ b/include/net/ip6_route.h
@@ -252,13 +252,13 @@ static inline const struct rt6_info *skb_rt6_info(const struct sk_buff *skb)
 /*
  *	Store a destination cache entry in a socket
  */
-static inline void ip6_dst_store(struct sock *sk, struct dst_entry *dst,
-				 bool daddr_set,
-				 bool saddr_set)
+static inline void ip6_dst_store_cookie(struct sock *sk, struct dst_entry *dst,
+					u32 cookie, bool daddr_set,
+					bool saddr_set)
 {
 	struct ipv6_pinfo *np = inet6_sk(sk);
 
-	np->dst_cookie = rt6_get_cookie(dst_rt6_info(dst));
+	np->dst_cookie = cookie;
 	sk_setup_caps(sk, dst);
 	np->daddr_cache = daddr_set;
 #ifdef CONFIG_IPV6_SUBTREES
@@ -266,8 +266,16 @@ static inline void ip6_dst_store(struct sock *sk, struct dst_entry *dst,
 #endif
 }
 
+static inline void ip6_dst_store(struct sock *sk, struct dst_entry *dst,
+				 bool daddr_set,
+				 bool saddr_set)
+{
+	ip6_dst_store_cookie(sk, dst, rt6_get_cookie(dst_rt6_info(dst)),
+			     daddr_set, saddr_set);
+}
+
 void ip6_sk_dst_store_flow(struct sock *sk, struct dst_entry *dst,
-			   const struct flowi6 *fl6);
+			   const struct flowi6 *fl6, u32 cookie);
 
 static inline bool ipv6_unicast_destination(const struct sk_buff *skb)
 {
diff --git a/net/ipv6/datagram.c b/net/ipv6/datagram.c
index ff7a7ffbf983..c18609bab366 100644
--- a/net/ipv6/datagram.c
+++ b/net/ipv6/datagram.c
@@ -78,6 +78,7 @@ int ip6_datagram_dst_update(struct sock *sk, bool fix_sk_saddr)
 	struct in6_addr *final_p;
 	struct dst_entry *dst;
 	struct flowi6 *fl6;
+	u32 cookie;
 	int err = 0;
 
 	if (inet6_test_bit(SNDFLOW, sk) &&
@@ -94,7 +95,8 @@ int ip6_datagram_dst_update(struct sock *sk, bool fix_sk_saddr)
 	final_p = fl6_update_dst(fl6, opt, &np->final);
 	rcu_read_unlock();
 
-	dst = ip6_dst_lookup_flow(sock_net(sk), sk, fl6, final_p);
+	dst = ip6_dst_lookup_flow_cookie(sock_net(sk), sk, fl6, final_p,
+					 &cookie);
 	if (IS_ERR(dst)) {
 		err = PTR_ERR(dst);
 		goto out;
@@ -112,7 +114,7 @@ int ip6_datagram_dst_update(struct sock *sk, bool fix_sk_saddr)
 		}
 	}
 
-	ip6_sk_dst_store_flow(sk, dst, fl6);
+	ip6_sk_dst_store_flow(sk, dst, fl6, cookie);
 
 out:
 	fl6_sock_release(flowlabel);
diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
index b87ccb5586cf..d51986d5ea4b 100644
--- a/net/ipv6/ip6_output.c
+++ b/net/ipv6/ip6_output.c
@@ -1370,6 +1370,7 @@ struct dst_entry *ip6_sk_dst_lookup_flow(struct sock *sk, struct flowi6 *fl6,
 					 bool connected)
 {
 	struct dst_entry *dst;
+	u32 cookie;
 
 	if (connected) {
 		dst = sk_dst_check(sk, inet6_sk(sk)->dst_cookie);
@@ -1378,9 +1379,10 @@ struct dst_entry *ip6_sk_dst_lookup_flow(struct sock *sk, struct flowi6 *fl6,
 			return dst;
 	}
 
-	dst = ip6_dst_lookup_flow(sock_net(sk), sk, fl6, final_dst);
+	dst = ip6_dst_lookup_flow_cookie(sock_net(sk), sk, fl6, final_dst,
+					 &cookie);
 	if (connected && !IS_ERR(dst))
-		ip6_sk_dst_store_flow(sk, dst_clone(dst), fl6);
+		ip6_sk_dst_store_flow(sk, dst_clone(dst), fl6, cookie);
 
 	return dst;
 }
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index c54844b83828..b19bc0d0c78d 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -3048,19 +3048,19 @@ void ip6_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, __be32 mtu)
 }
 
 void ip6_sk_dst_store_flow(struct sock *sk, struct dst_entry *dst,
-			   const struct flowi6 *fl6)
+			   const struct flowi6 *fl6, u32 cookie)
 {
 #ifdef CONFIG_IPV6_SUBTREES
 	struct ipv6_pinfo *np = inet6_sk(sk);
 #endif
 
-	ip6_dst_store(sk, dst,
-		      ipv6_addr_equal(&fl6->daddr, &sk->sk_v6_daddr),
+	ip6_dst_store_cookie(sk, dst, cookie,
+			     ipv6_addr_equal(&fl6->daddr, &sk->sk_v6_daddr),
 #ifdef CONFIG_IPV6_SUBTREES
-		      ipv6_addr_equal(&fl6->saddr, &np->saddr) ?
-		      true :
+			     ipv6_addr_equal(&fl6->saddr, &np->saddr) ?
+			     true :
 #endif
-		      false);
+			     false);
 }
 
 static bool ip6_redirect_nh_match(const struct fib6_result *res,
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 9e22ed61a6de..9dc031437b42 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -145,6 +145,7 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr_unsized *uaddr,
 	struct dst_entry *dst;
 	struct flowi6 *fl6;
 	int addr_type;
+	u32 cookie;
 	int err;
 
 	if (addr_len < SIN6_LEN_RFC2133)
@@ -288,7 +289,7 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr_unsized *uaddr,
 	 */
 	ip6_ecmp_set_mp_hash(net, fl6, sk->sk_txhash);
 
-	dst = ip6_dst_lookup_flow(net, sk, fl6, final_p);
+	dst = ip6_dst_lookup_flow_cookie(net, sk, fl6, final_p, &cookie);
 	if (IS_ERR(dst)) {
 		err = PTR_ERR(dst);
 		goto failure;
@@ -312,7 +313,7 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr_unsized *uaddr,
 	inet->inet_rcv_saddr = LOOPBACK4_IPV6;
 
 	sk->sk_gso_type = SKB_GSO_TCPV6;
-	ip6_dst_store(sk, dst, false, false);
+	ip6_dst_store_cookie(sk, dst, cookie, false, false);
 
 	icsk->icsk_ext_hdr_len = 0;
 	if (opt)
-- 
2.56.0


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie
  2026-11-17  9:21 ` [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie Ralf Lici
@ 2026-11-18  8:48   ` Xia Yan
  2026-11-18 14:39     ` Ralf Lici
  2026-11-19  9:22   ` netdev-bot+sashiko
  1 sibling, 1 reply; 12+ messages in thread
From: Xia Yan @ 2026-11-18  8:48 UTC (permalink / raw)
  To: Ralf Lici
  Cc: netdev, David Ahern, Ido Schimmel, David S. Miller, Eric Dumazet,
	Jakub Kicinski, Paolo Abeni, Simon Horman, Neal Cardwell,
	Kuniyuki Iwashima, Steffen Klassert, Herbert Xu,
	Florian Westphal, Sabrina Dubroca, Antonio Quartulli,
	linux-kernel, Xia Yan, yana06070

I traced the race and the fix against current net-next.

On the writer side, xfrm_policy_insert() links the policy and then calls
rt_genid_bump_ipv6(), which resolves to fib6_flush_trees(). A fresh serial
is allocated from the namespace counter first, then every node is
restamped under tb6_lock. A late rt6_get_cookie() can pick up that
post-insertion serial for a dst whose lookup ran before the insertion,
and the pair keeps passing dst_check until the next flush. Sampling the
cookie between two generation reads closes the in-lookup window. A
policy inserted after the helper returns is still caught later, because
the restamp changes fn_sernum itself and the stored pre-insertion cookie
stops matching. fib6_new_sernum() never returns 0, so the fallback
cookie always fails fib6_check().

1/3 checks out as well. The old code kept writing -1 into the node, so a
second failure changed nothing and the namespace generation never moved.
The fresh allocation under tb6_lock matches the discipline the
fib6_ifup()/fib6_ifdown() paths already get from __fib6_clean_all().
The conversion boundary in 3/3 also matches the tree: rawv6 does not
cache the dst, and the accepted-socket and refill stores in tcp_ipv6.c
are listed as follow-up.

One thing I could not fully work out is the ordering behind the
generation checks. When the lookup misses a change, the reader relies on
the two generation reads to notice it. My understanding is that this
needs the change visible before the counter advance, and the counter
advance visible before the node restamps. I could not establish either
hop, though I may well be missing something.

In xfrm_policy_insert(), the policy is linked into the hash and
rt_genid_bump_ipv6() runs within the same critical section. As far as I
can tell, the linking stores and the cmpxchg on fib6_sernum are all
relaxed, with no barrier between them. Could a reader read the advanced
generation in gen0, miss the policy during the lookup, sample a
restamped fn_sernum, and still read the same generation in gen1?

In fib6_flush_trees(), the serial is allocated before __fib6_clean_all()
takes tb6_lock, and fib6_clean_node() stamps the nodes with a plain
WRITE_ONCE(). spin_lock() is an ACQUIRE, and my reading of
memory-barriers.txt is that operations issued before an ACQUIRE may
complete after it. So a reader could sample the restamped fn_sernum
between two reads that both still return the old generation. The only
pairing I found written down here is fib6_get_cookie_safe()'s smp_rmb()
against the smp_wmb() in __fib6_update_sernum_upto_root().

If either gap is real, the reader ends up caching the plain dst with a
post-flush serial that stays valid until the next flush. That is the
state the series sets out to prevent, so I suspect I am missing
something. Your September RFC [1] seems to anticipate this in item 1 of
its writer contract: "a routing-visible update is published before its
serial is allocated and stamped". Could you check whether the two hops
are covered somewhere I did not look?

[1] https://lore.kernel.org/netdev/20260901122501.482920-1-ralf@mandelbit.com/

If there is an answer to the above that I missed, please feel free to
add:

Reviewed-by: Xia Yan <yanxia@cmss.chinamobile.com>



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie
  2026-11-18  8:48   ` Xia Yan
@ 2026-11-18 14:39     ` Ralf Lici
  2026-11-18 15:07       ` Xia Yan
  0 siblings, 1 reply; 12+ messages in thread
From: Ralf Lici @ 2026-11-18 14:39 UTC (permalink / raw)
  To: Xia Yan
  Cc: netdev, David Ahern, Ido Schimmel, David S. Miller, Eric Dumazet,
	Jakub Kicinski, Paolo Abeni, Simon Horman, Neal Cardwell,
	Kuniyuki Iwashima, Steffen Klassert, Herbert Xu,
	Florian Westphal, Sabrina Dubroca, Antonio Quartulli,
	linux-kernel, yana06070

Hi Xia,

On Wed, 18 Nov 2026 16:48:34 +0800, Xia Yan <yanxia@cmss.chinamobile.com> wrote:
> I traced the race and the fix against current net-next.
>
> On the writer side, xfrm_policy_insert() links the policy and then calls
> rt_genid_bump_ipv6(), which resolves to fib6_flush_trees(). A fresh serial
> is allocated from the namespace counter first, then every node is
> restamped under tb6_lock. A late rt6_get_cookie() can pick up that
> post-insertion serial for a dst whose lookup ran before the insertion,
> and the pair keeps passing dst_check until the next flush. Sampling the
> cookie between two generation reads closes the in-lookup window. A
> policy inserted after the helper returns is still caught later, because
> the restamp changes fn_sernum itself and the stored pre-insertion cookie
> stops matching. fib6_new_sernum() never returns 0, so the fallback
> cookie always fails fib6_check().
>

Yes, that's a pretty good summary of the change with the small
correction that a newly linked node holds fn_sernum 0 until fib6_add
stamps it. So a cookie-0 dst can pass a check in that window, and fails
as soon as the stamp lands.

> 1/3 checks out as well. The old code kept writing -1 into the node, so a
> second failure changed nothing and the namespace generation never moved.
> The fresh allocation under tb6_lock matches the discipline the
> fib6_ifup()/fib6_ifdown() paths already get from __fib6_clean_all().
> The conversion boundary in 3/3 also matches the tree: rawv6 does not
> cache the dst, and the accepted-socket and refill stores in tcp_ipv6.c
> are listed as follow-up.
>

In fact there are many users of cached IPv6 dsts that will need
conversion and this series is mainly here to introduce the new API.

> One thing I could not fully work out is the ordering behind the
> generation checks. When the lookup misses a change, the reader relies on
> the two generation reads to notice it. My understanding is that this
> needs the change visible before the counter advance, and the counter
> advance visible before the node restamps. I could not establish either
> hop, though I may well be missing something.
>
> In xfrm_policy_insert(), the policy is linked into the hash and
> rt_genid_bump_ipv6() runs within the same critical section. As far as I
> can tell, the linking stores and the cmpxchg on fib6_sernum are all
> relaxed, with no barrier between them. Could a reader read the advanced
> generation in gen0, miss the policy during the lookup, sample a
> restamped fn_sernum, and still read the same generation in gen1?
>

Both hops are covered by fib6_new_sernum. It allocates the serial with
atomic_try_cmpxchg, which is fully ordered when it succeeds, so it acts
as an smp_mb before and after the update of fib6_sernum.

For the first hop, the policy (or route) is linked before
fib6_new_sernum runs, which gives message passing against the smp_rmb
after gen0: a reader that reads the advanced generation in gen0 also
finds the policy.

> In fib6_flush_trees(), the serial is allocated before __fib6_clean_all()
> takes tb6_lock, and fib6_clean_node() stamps the nodes with a plain
> WRITE_ONCE(). spin_lock() is an ACQUIRE, and my reading of
> memory-barriers.txt is that operations issued before an ACQUIRE may
> complete after it. So a reader could sample the restamped fn_sernum

For the second hop, every fn_sernum write follows a successful
fib6_new_sernum on the same CPU: fib6_update_sernum,
__fib6_update_sernum_upto_root, and fib6_clean_node in the flush case.
The full barrier orders the counter update before the tb6_lock
acquisition and the stamps, so the ACQUIRE semantics of spin_lock don't
matter here. A reader that samples a restamped fn_sernum reads the
advanced generation in gen1 after the smp_rmb, and the helper returns
cookie 0.

> between two reads that both still return the old generation. The only
> pairing I found written down here is fib6_get_cookie_safe()'s smp_rmb()
> against the smp_wmb() in __fib6_update_sernum_upto_root().
>

Right, this isn't documented. The barrier comments in
ip6_dst_lookup_flow_cookie should name fib6_new_sernum as the pairing
side, and fib6_new_sernum should state that its full ordering is relied
upon.

I also checked both hops with an LKMM litmus test: the outcome you
describe is forbidden with atomic_cmpxchg and becomes reachable with
atomic_cmpxchg_relaxed.

While going through the writers I also noticed that xfrm_policy_insert
unlinks a replaced policy only after rt_genid_bump_ipv6, so a lookup
after the bump can still match the old policy. That is a writer-side
ordering issue independent of this series, and it affects the current
late cookie sampling in the same way. I'll send a separate fix.

Does this explanation address your concern? If so, I'll carry your
Reviewed-by. I'm waiting for the Sashiko review of v3: if it reports
anything that needs a respin, I'll add the barrier comments in v4,
otherwise I'll send them as a follow-up.

Thanks for the careful review!

-- 
Ralf Lici
Mandelbit Srl

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie
  2026-11-18 14:39     ` Ralf Lici
@ 2026-11-18 15:07       ` Xia Yan
  0 siblings, 0 replies; 12+ messages in thread
From: Xia Yan @ 2026-11-18 15:07 UTC (permalink / raw)
  To: Ralf Lici
  Cc: netdev, David Ahern, Ido Schimmel, David S. Miller, Eric Dumazet,
	Jakub Kicinski, Paolo Abeni, Simon Horman, Neal Cardwell,
	Kuniyuki Iwashima, Steffen Klassert, Herbert Xu,
	Florian Westphal, Sabrina Dubroca, Antonio Quartulli,
	linux-kernel, yana06070

Hi Ralf,

Thanks, that clears it up. I missed the ordering from atomic_try_cmpxchg()
and the window before a new node is stamped. Your plan sounds good.

Reviewed-by: Xia Yan <yanxia@cmss.chinamobile.com>

Thanks,
Xia Yan



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH net-next v3 1/3] ipv6: stamp the default route node with a fresh serial on link failure
  2026-11-17  9:21 ` [PATCH net-next v3 1/3] ipv6: stamp the default route node with a fresh serial on link failure Ralf Lici
@ 2026-11-19  9:22   ` netdev-bot+sashiko
  0 siblings, 0 replies; 12+ messages in thread
From: netdev-bot+sashiko @ 2026-11-19  9:22 UTC (permalink / raw)
  To: ralf
  Cc: netdev, dsahern, idosch, davem, edumazet, kuba, pabeni, horms,
	ncardwell, kuniyu, steffen.klassert, herbert, fw, sd, antonio,
	linux-kernel

Thank you for your contribution! Sashiko AI review found more severe
issue(s) in other patches of this series. This mail is only an FYI, in case
you want to address the nit picks below when you respin.

Low priority / nit picks for your consideration:

- [Low] ip6_link_failure() (net/ipv6/route.c) used to do one idempotent
  store, WRITE_ONCE(fn->fn_sernum, -1).
- [Low] The patch reuses rt6_exception_net() in ip6_link_failure() for a
  dst that is not an exception (the !RTF_CACHE, RTF_DEFAULT branch).

--

ipv6: stamp the default route node with a fresh serial on link failure

For a non-cached RTF_DEFAULT dst, ip6_link_failure() used to write -1
into fn_sernum. With this patch it takes tb6_lock and calls
fib6_update_sernum() instead. That stamps the node with a fresh serial and
advances net->ipv6.fib6_sernum, so the lookup-cookie checks added later in
the series can tell repeated invalidations apart.

> diff --git a/net/ipv6/route.c b/net/ipv6/route.c
> index 4a4c5ace63c54..cc505817399ba 100644
> --- a/net/ipv6/route.c
> +++ b/net/ipv6/route.c
> @@ -2873,13 +2873,13 @@ static void ip6_link_failure(struct sk_buff *skb)
>  			rt6_remove_exception_rt(rt);
>  		} else {
>  			struct fib6_info *from;
> -			struct fib6_node *fn;
>  
>  			from = rcu_dereference(rt->from);
> -			if (from) {
> -				fn = rcu_dereference(from->fib6_node);
> -				if (fn && (rt->rt6i_flags & RTF_DEFAULT))
> -					WRITE_ONCE(fn->fn_sernum, -1);
> +			if (from && from->fib6_table &&
> +			    (rt->rt6i_flags & RTF_DEFAULT)) {
> +				spin_lock_bh(&from->fib6_table->tb6_lock);
> +				fib6_update_sernum(rt6_exception_net(rt), from);
> +				spin_unlock_bh(&from->fib6_table->tb6_lock);

[Severity: Low]
The old code did a single store that gave the same result every time.
The new code takes the table-wide tb6_lock and advances the netns-wide
net->ipv6.fib6_sernum on every call. ip6_link_failure() runs once per
skb, not once per event.

Is that per-skb cost intended on these paths?

For example, neigh_invalidate()->ndisc_error_report() calls
dst_link_failure() for every skb queued on the default router's
arp_queue:

net/ipv6/ndisc.c:ndisc_error_report() {
    ...
	dst_link_failure(skb);
    ...
}

The err_count paths in ip6_tnl_xmit() and ipip6_tunnel_xmit() also reach
this code, at one link failure per received ICMP error. These are 1:1, so
they don't amplify anything. The sit path can't be reached for ISATAP,
because ipip6_err() returns before incrementing err_count when
parms->iph.daddr == 0.

Each bump goes through fib6_new_sernum():

	} while (!atomic_try_cmpxchg(&net->ipv6.fib6_sernum, &old, new));

Nexthop-object pcpu dsts are seeded from that generation in
ip6_rt_pcpu_alloc() and checked against it:

net/ipv6/route.c:rt6_is_valid() {
	return rt6->sernum == rt_genid_ipv6(dev_net(rt6->dst.dev));
}

So each call drops and reallocates every nexthop-object pcpu dst in the
netns. ip6_dst_check() also rejects every socket-cached one, which forces
a re-lookup.

The commit message says:

    Advancing the namespace generation also invalidates unrelated dsts using
    nexthop objects, which are validated against that generation.

and the v3 changelog calls this harmless. Could the commit message say
that this happens once per skb? That includes bursts when the neighbor
entry for a default router fails.

For reference, rt6_insert_exception() already does the same tb6_lock plus
fib6_update_sernum() once per event. Per-event invalidation is also part
of the fix itself: coalescing would let dsts cached between two failures
survive. The later patches in the series don't change ip6_link_failure(),
and "ipv6: add ip6_dst_lookup_flow_cookie" relies on this generation
bump.

[Severity: Low]
This isn't a bug, but rt6_exception_net() is now used for a dst that is
not an exception (the !RTF_CACHE, RTF_DEFAULT branch). Here it picks the
netns whose fib6_sernum gets advanced. Its name and comment still only
describe exception accounting:

/* Exceptions are accounted to the netns their dst was allocated in.
 * Unlike dev_net(dst->dev), it does not change when the device moves to
 * another netns or when dst_dev_put() points the dst to blackhole_netdev.
 */
static struct net *rt6_exception_net(const struct rt6_info *rt)

The netns it returns looks correct here, and dev_net(rt->dst.dev) would
be wrong after dst_dev_put(). Other readers of the same counter, such as
rt6_is_valid() and ip6_rt_pcpu_alloc(), use dev_net(), so it isn't
obvious why this one is different.

Would it make sense to update the comment, or rename the helper, so that
it covers this use?

[ ... ]

-- 
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/cover.1794906743.git.ralf%40mandelbit.com

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie
  2026-11-17  9:21 ` [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie Ralf Lici
  2026-11-18  8:48   ` Xia Yan
@ 2026-11-19  9:22   ` netdev-bot+sashiko
  2026-11-19 12:31     ` Ralf Lici
  1 sibling, 1 reply; 12+ messages in thread
From: netdev-bot+sashiko @ 2026-11-19  9:22 UTC (permalink / raw)
  To: ralf
  Cc: netdev, dsahern, idosch, davem, edumazet, kuba, pabeni, horms,
	ncardwell, kuniyu, steffen.klassert, herbert, fw, sd, antonio,
	linux-kernel

Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider.

Critical: 0 · High: 1 · Medium: 0 · Low: 1

If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).

- [High] ip6_dst_lookup_flow_cookie() (net/ipv6/ip6_output.c) is meant to
  close the late-cookie-sampling race.
- [Low] The new kernel-doc for ip6_dst_lookup_flow_cookie() says 'If the
  IPv6 FIB changed during the lookup, a route dst no longer validates…

--

ipv6: add ip6_dst_lookup_flow_cookie

This adds ip6_dst_lookup_flow_cookie(). It returns a dst together with a
validation cookie that is sampled between two reads of the IPv6
generation, and the cookie is 0 if the generation changed. For
nexthop-object dsts, ip6_dst_check() now also requires the cookie to
match rt->sernum, and fib6_ifup()/fib6_ifdown() now advance the serial
after rt6_multipath_rebalance().

> diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
> index 3cc911f6618e7..b87ccb5586cf6 100644
> --- a/net/ipv6/ip6_output.c
> +++ b/net/ipv6/ip6_output.c

[ ... ]

> @@ -1300,6 +1299,54 @@ struct dst_entry *ip6_dst_lookup_flow(struct net *net, const struct sock *sk, st
>  }
>  EXPORT_SYMBOL_GPL(ip6_dst_lookup_flow);
>  
> +/**
> + *	ip6_dst_lookup_flow_cookie - perform route lookup on flow with ipsec
> + *				     and return the matching dst cookie

[ ... ]

> + *	Same as ip6_dst_lookup_flow, but also returns the cookie describing
> + *	the routing state the dst was obtained from. A caller caching the dst
> + *	should use this cookie when validating it with dst_check. If the
> + *	IPv6 FIB changed during the lookup, a route dst no longer validates
> + *	with it. An xfrm bundle is validated by xfrm and ignores the cookie.

[Severity: Low]
Does this guarantee hold for nexthop-object dsts (rt->sernum != 0) when
the FIB change is a route deletion?

For those dsts the helper returns gen0, and ip6_dst_check() only checks
this:

	if (rt->sernum)
		return (rt6_is_valid(rt) && cookie == rt->sernum) ? dst : NULL;

It never looks at rt->from.

Route deletion goes through
fib6_del()->fib6_del_route()->fib6_purge_rt()->__fib6_drop_pcpu_from().
None of these call fib6_new_sernum(), so net->ipv6.fib6_sernum does not
change. Deletion only clears pcpu_rt->from:

	from = unrcu_pointer(xchg(&pcpu_rt->from, NULL));
	fib6_info_release(from);

Say the selected nexthop-object route is deleted between the gen0 and
gen1 reads. Then gen0 == gen1, and the cookie still matches
pcpu_rt->sernum. The cached dst keeps validating until some unrelated
generation bump, such as any route insertion in the netns.

The stale-dst behavior itself predates this patch, since callers of
rt6_get_cookie() also get rt->sernum. What is new is this kdoc claim on
a new exported API.

Could the comment be narrowed so future callers don't assume deletions
are covered?

> + *
> + *	Return: a valid dst pointer on success or a pointer encoded error code.
> + */
> +struct dst_entry *ip6_dst_lookup_flow_cookie(struct net *net,
> +					     const struct sock *sk,
> +					     struct flowi6 *fl6,
> +					     const struct in6_addr *final_dst,
> +					     u32 *cookie)
> +{

[ ... ]

> +	gen0 = rt_genid_ipv6(net);
> +	/* read the generation before looking at the FIB or xfrm policies */
> +	smp_rmb();
> +
> +	dst = ip6_dst_lookup_flow(net, sk, fl6, final_dst);
> +	if (IS_ERR(dst))
> +		return dst;
> +
> +	rt = dst_rt6_info(dst);
> +	*cookie = rt->sernum ? gen0 : rt6_get_cookie(rt);
> +
> +	/* keep the FIB and cookie reads before the final generation read */
> +	smp_rmb();
> +	gen1 = rt_genid_ipv6(net);
> +	if (unlikely(gen0 != gen1))
> +		*cookie = 0;

[Severity: High]
Does setting the cookie to 0 here have any effect when
ip6_dst_lookup_flow() returned an xfrm bundle?

The commit message says:

    xfrm bundles retain their existing validation
    through their own route and path cookies.

However, those cookies are sampled late too, which is the pattern the
commit calls buggy. Both are set inside xfrm_bundle_create(). That runs
after ip6_dst_lookup_tail() picked the route, and after policy lookup and
state resolution:

net/xfrm/xfrm_policy.c:xfrm_init_path() {
    ...
		path->path_cookie = rt6_get_cookie(dst_rt6_info(dst));
    ...
}

net/ipv6/xfrm6_policy.c:xfrm6_fill_dst() {
    ...
	xdst->route_cookie = rt6_get_cookie(rt);
    ...
}

xfrm_bundle_ok() then validates only against those two cookies:

	if (!dst_check(xfrm_dst_path(dst), ((struct xfrm_dst *)dst)->path_cookie) ||
    ...
		if (!dst_check(xdst->route, xdst->route_cookie))
			return 0;

xfrm_dst_check() ignores the cookie the socket stored:

	if (READ_ONCE(dst->obsolete) < 0 && !stale_bundle(dst))
		return dst;

So when a change lands while the bundle is being built, the helper sees
gen0 != gen1, but nothing acts on that result.

Here is one sequence. A socket is covered by IPsec policy P1. During
tcp_v6_connect(), someone inserts a higher-priority policy P2 that does
not replace P1, for example a block policy or a different tunnel:

xfrm_policy_insert()
    rt_genid_bump_ipv6(net)   -> fib6_flush_trees(), new node sernums
    (xfrm_policy_kill() only runs for a replaced delpol, so the genid
     of P1 is untouched)

If path_cookie and route_cookie are read after the flush, they already
hold the new fn_sernum.

Would the cached P1 bundle then pass xfrm_bundle_ok() on every later
send, and bypass P2 until some other relevant FIB change?

The next patch in the series ("ipv6: use lookup cookies in selected
socket dst cache paths") fixes this policy bypass for plain routes. That
patch only routes tcp_v6_connect(), ip6_datagram_dst_update() and
ip6_sk_dst_lookup_flow() through this helper, so the bundle case is still
open at the end of the series.

Could the bundle route and path cookies be sampled differently, or could
the detected generation change be passed into the bundle?

[ ... ]

-- 
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/cover.1794906743.git.ralf%40mandelbit.com

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH net-next v3 3/3] ipv6: use lookup cookies in selected socket dst cache paths
  2026-11-17  9:21 ` [PATCH net-next v3 3/3] ipv6: use lookup cookies in selected socket dst cache paths Ralf Lici
@ 2026-11-19  9:22   ` netdev-bot+sashiko
  2026-11-19 12:34     ` Ralf Lici
  0 siblings, 1 reply; 12+ messages in thread
From: netdev-bot+sashiko @ 2026-11-19  9:22 UTC (permalink / raw)
  To: ralf
  Cc: netdev, dsahern, idosch, davem, edumazet, kuba, pabeni, horms,
	ncardwell, kuniyu, steffen.klassert, herbert, fw, sd, antonio,
	linux-kernel

Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider.

Critical: 0 · High: 1 · Medium: 0 · Low: 1

If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).

- [High] The patch fixes the IPsec-bypass race (late cookie sampling) in
  only three paths: tcp_v6_connect(), ip6_datagram_dst_update() and…
- [Low] This patch carries a Fixes: tag and is the one stable selection
  will pick.

--

ipv6: use lookup cookies in selected socket dst cache paths

This commit changes tcp_v6_connect(), ip6_datagram_dst_update() and
ip6_sk_dst_lookup_flow(). They now store the cookie returned by
ip6_dst_lookup_flow_cookie() instead of sampling it after the lookup.
The commit also adds ip6_dst_store_cookie() and keeps ip6_dst_store() as
a wrapper for callers that are not converted yet.

> Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")

[Severity: Low]
This patch depends on the earlier commit "ipv6: add
ip6_dst_lookup_flow_cookie". That commit does three things:

- adds ip6_dst_lookup_flow_cookie()
- changes ip6_dst_check() so that a dst with rt->sernum must match the
  supplied cookie
- moves the fib6_update_sernum() calls in fib6_ifup()/fib6_ifdown() to
  after rt6_multipath_rebalance()

The prerequisite has no Fixes: tag and no stable note. This commit does
not mention the dependency either.

If stable selection picks only this commit because of its Fixes: tag, it
will not build, since ip6_dst_lookup_flow_cookie() is undefined. Could the
prerequisite be tagged too, or the dependency be noted here, so both get
backported together?

> diff --git a/include/net/ip6_route.h b/include/net/ip6_route.h
> index 95f936be32c6c..2684f6069a5e1 100644
> --- a/include/net/ip6_route.h
> +++ b/include/net/ip6_route.h

[ ... ]

> @@ -266,8 +266,16 @@ static inline void ip6_dst_store(struct sock *sk, struct dst_entry *dst,
>  #endif
>  }
>  
> +static inline void ip6_dst_store(struct sock *sk, struct dst_entry *dst,
> +				 bool daddr_set,
> +				 bool saddr_set)
> +{
> +	ip6_dst_store_cookie(sk, dst, rt6_get_cookie(dst_rt6_info(dst)),
> +			     daddr_set, saddr_set);
> +}

[Severity: High]
This wrapper still samples the cookie after the lookup. Three other
writers of np->dst_cookie on TCPv6 sockets still go through it:

net/ipv6/inet6_connection_sock.c:inet6_csk_route_socket() {
    ...
	dst = ip6_dst_lookup_flow(sock_net(sk), sk, fl6, final_p);

	if (!IS_ERR(dst))
		ip6_dst_store(sk, dst, false, false);
    ...
}

net/ipv6/af_inet6.c:inet6_sk_rebuild_header() {
    ...
	dst = ip6_dst_lookup_flow(sock_net(sk), sk, fl6, final_p);
    ...
	ip6_dst_store(sk, dst, false, false);
    ...
}

net/ipv6/tcp_ipv6.c:tcp_v6_syn_recv_sock() {
    ...
	memcpy(newnp, np, sizeof(struct ipv6_pinfo));

	ip6_dst_store(newsk, dst, false, false);
    ...
}

Can the same xfrm policy bypass still happen through these paths?

xfrm_policy_insert() links the policy and then calls
rt_genid_bump_ipv6(). Suppose an insertion lands between
ip6_dst_lookup_flow() returning a plain route and the rt6_get_cookie()
call inside ip6_dst_store(). The plain dst is then cached with the new
serial, and ip6_dst_check() keeps accepting it.

These paths run much more often than tcp_v6_connect():

- After connect(), every refill for an established socket goes through
  inet6_csk_xmit()->inet6_csk_route_socket() whenever __sk_dst_check()
  fails. That includes the invalidation caused by the policy insertion
  itself.
- A remote ICMPv6 Packet Too Big can force refills via
  tcp_v6_mtu_reduced()->inet6_csk_update_pmtu()->inet6_csk_route_socket().
- For accepted sockets, tcp_v6_syn_recv_sock() reads the cookie only
  after inet6_csk_route_req(), tcp_create_openreq_child() and the
  ipv6_pinfo memcpy. That window is wider than the one in connect().

The commit message says this is a partial conversion. With the Fixes:
tag, though, stable trees would get a fix that leaves established active
sockets and all accepted sockets exposed. The new kernel-doc for
ip6_dst_lookup_flow_cookie() also says "A caller caching the dst should
use this cookie when validating it with dst_check". These callers don't
do that.

Would it make sense to convert inet6_csk_route_socket(),
inet6_sk_rebuild_header() and tcp_v6_syn_recv_sock() in this patch as
well?

[ ... ]

-- 
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/cover.1794906743.git.ralf%40mandelbit.com

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie
  2026-11-19  9:22   ` netdev-bot+sashiko
@ 2026-11-19 12:31     ` Ralf Lici
  0 siblings, 0 replies; 12+ messages in thread
From: Ralf Lici @ 2026-11-19 12:31 UTC (permalink / raw)
  To: netdev-bot+sashiko
  Cc: netdev, dsahern, idosch, davem, edumazet, kuba, pabeni, horms,
	ncardwell, kuniyu, steffen.klassert, herbert, fw, sd, antonio,
	linux-kernel

On Thu, 19 Nov 2026 09:22:54 +0000, netdev-bot+sashiko@kernel.org wrote:
> > @@ -1300,6 +1299,54 @@ struct dst_entry *ip6_dst_lookup_flow(struct net *net, const struct sock *sk, st
> >  }
> >  EXPORT_SYMBOL_GPL(ip6_dst_lookup_flow);
> >  
> > +/**
> > + *	ip6_dst_lookup_flow_cookie - perform route lookup on flow with ipsec
> > + *				     and return the matching dst cookie
>
> [ ... ]
>
> > + *	Same as ip6_dst_lookup_flow, but also returns the cookie describing
> > + *	the routing state the dst was obtained from. A caller caching the dst
> > + *	should use this cookie when validating it with dst_check. If the
> > + *	IPv6 FIB changed during the lookup, a route dst no longer validates
> > + *	with it. An xfrm bundle is validated by xfrm and ignores the cookie.
>
> [Severity: Low]
> Does this guarantee hold for nexthop-object dsts (rt->sernum != 0) when
> the FIB change is a route deletion?
>
> For those dsts the helper returns gen0, and ip6_dst_check() only checks
> this:
>
> 	if (rt->sernum)
> 		return (rt6_is_valid(rt) && cookie == rt->sernum) ? dst : NULL;
>
> It never looks at rt->from.
>
> Route deletion goes through
> fib6_del()->fib6_del_route()->fib6_purge_rt()->__fib6_drop_pcpu_from().
> None of these call fib6_new_sernum(), so net->ipv6.fib6_sernum does not
> change. Deletion only clears pcpu_rt->from:
>
> 	from = unrcu_pointer(xchg(&pcpu_rt->from, NULL));
> 	fib6_info_release(from);
>
> Say the selected nexthop-object route is deleted between the gen0 and
> gen1 reads. Then gen0 == gen1, and the cookie still matches
> pcpu_rt->sernum. The cached dst keeps validating until some unrelated
> generation bump, such as any route insertion in the netns.
>
> The stale-dst behavior itself predates this patch, since callers of
> rt6_get_cookie() also get rt->sernum. What is new is this kdoc claim on
> a new exported API.
>
> Could the comment be narrowed so future callers don't assume deletions
> are covered?
>

Right, deletion does not advance the generation, and the rt->sernum
branch of ip6_dst_check never looks at rt->from. This predates the
series, but the kdoc should not promise more than the helper does. I'll
narrow it to FIB changes that advance the IPv6 generation in the next
revision.

> > +	gen0 = rt_genid_ipv6(net);
> > +	/* read the generation before looking at the FIB or xfrm policies */
> > +	smp_rmb();
> > +
> > +	dst = ip6_dst_lookup_flow(net, sk, fl6, final_dst);
> > +	if (IS_ERR(dst))
> > +		return dst;
> > +
> > +	rt = dst_rt6_info(dst);
> > +	*cookie = rt->sernum ? gen0 : rt6_get_cookie(rt);
> > +
> > +	/* keep the FIB and cookie reads before the final generation read */
> > +	smp_rmb();
> > +	gen1 = rt_genid_ipv6(net);
> > +	if (unlikely(gen0 != gen1))
> > +		*cookie = 0;
>
> [Severity: High]
> Does setting the cookie to 0 here have any effect when
> ip6_dst_lookup_flow() returned an xfrm bundle?
>

No, it doesn't. Bundles are validated by xfrm_bundle_ok against their
own route and path cookies, which are still sampled late in
xfrm_bundle_create. The cover letter lists the xfrm bundle cookies among
the cache users this series does not convert.

> The commit message says:
>
>     xfrm bundles retain their existing validation
>     through their own route and path cookies.
>
> However, those cookies are sampled late too, which is the pattern the
> commit calls buggy. Both are set inside xfrm_bundle_create(). That runs
> after ip6_dst_lookup_tail() picked the route, and after policy lookup and
> state resolution:
>
> net/xfrm/xfrm_policy.c:xfrm_init_path() {
>     ...
> 		path->path_cookie = rt6_get_cookie(dst_rt6_info(dst));
>     ...
> }
>
> net/ipv6/xfrm6_policy.c:xfrm6_fill_dst() {
>     ...
> 	xdst->route_cookie = rt6_get_cookie(rt);
>     ...
> }
>
> xfrm_bundle_ok() then validates only against those two cookies:
>
> 	if (!dst_check(xfrm_dst_path(dst), ((struct xfrm_dst *)dst)->path_cookie) ||
>     ...
> 		if (!dst_check(xdst->route, xdst->route_cookie))
> 			return 0;
>
> xfrm_dst_check() ignores the cookie the socket stored:
>
> 	if (READ_ONCE(dst->obsolete) < 0 && !stale_bundle(dst))
> 		return dst;
>
> So when a change lands while the bundle is being built, the helper sees
> gen0 != gen1, but nothing acts on that result.
>
> Here is one sequence. A socket is covered by IPsec policy P1. During
> tcp_v6_connect(), someone inserts a higher-priority policy P2 that does
> not replace P1, for example a block policy or a different tunnel:
>
> xfrm_policy_insert()
>     rt_genid_bump_ipv6(net)   -> fib6_flush_trees(), new node sernums
>     (xfrm_policy_kill() only runs for a replaced delpol, so the genid
>      of P1 is untouched)
>
> If path_cookie and route_cookie are read after the flush, they already
> hold the new fn_sernum.
>
> Would the cached P1 bundle then pass xfrm_bundle_ok() on every later
> send, and bypass P2 until some other relevant FIB change?
>
> The next patch in the series ("ipv6: use lookup cookies in selected
> socket dst cache paths") fixes this policy bypass for plain routes. That
> patch only routes tcp_v6_connect(), ip6_datagram_dst_update() and
> ip6_sk_dst_lookup_flow() through this helper, so the bundle case is still
> open at the end of the series.
>
> Could the bundle route and path cookies be sampled differently, or could
> the detected generation change be passed into the bundle?
>

Passing the cookie from this helper into the bundle would only cover
dst_orig: in tunnel mode the outer routes are looked up inside bundle
creation. The fix I have in mind applies the same bracketing within
xfrm, reading the generation before the policy lookup and storing 0 as
route and path cookie if it changed by the time they are sampled. That
also covers bundles created through xfrm_lookup outside of this helper,
so I'd rather do it as a separate change on top of this API.

-- 
Ralf Lici
Mandelbit Srl

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH net-next v3 3/3] ipv6: use lookup cookies in selected socket dst cache paths
  2026-11-19  9:22   ` netdev-bot+sashiko
@ 2026-11-19 12:34     ` Ralf Lici
  0 siblings, 0 replies; 12+ messages in thread
From: Ralf Lici @ 2026-11-19 12:34 UTC (permalink / raw)
  To: netdev-bot+sashiko
  Cc: netdev, dsahern, idosch, davem, edumazet, kuba, pabeni, horms,
	ncardwell, kuniyu, steffen.klassert, herbert, fw, sd, antonio,
	linux-kernel

On Thu, 19 Nov 2026 09:22:55 +0000, netdev-bot+sashiko@kernel.org wrote:
> ipv6: use lookup cookies in selected socket dst cache paths
>
> This commit changes tcp_v6_connect(), ip6_datagram_dst_update() and
> ip6_sk_dst_lookup_flow(). They now store the cookie returned by
> ip6_dst_lookup_flow_cookie() instead of sampling it after the lookup.
> The commit also adds ip6_dst_store_cookie() and keeps ip6_dst_store() as
> a wrapper for callers that are not converted yet.
>
> > Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
>
> [Severity: Low]
> This patch depends on the earlier commit "ipv6: add
> ip6_dst_lookup_flow_cookie". That commit does three things:
>
> - adds ip6_dst_lookup_flow_cookie()
> - changes ip6_dst_check() so that a dst with rt->sernum must match the
>   supplied cookie
> - moves the fib6_update_sernum() calls in fib6_ifup()/fib6_ifdown() to
>   after rt6_multipath_rebalance()
>
> The prerequisite has no Fixes: tag and no stable note. This commit does
> not mention the dependency either.
>
> If stable selection picks only this commit because of its Fixes: tag, it
> will not build, since ip6_dst_lookup_flow_cookie() is undefined. Could the
> prerequisite be tagged too, or the dependency be noted here, so both get
> backported together?
>

Since this targets net-next and I'm not asking for a backport, I'll drop
the Fixes tag from this patch in the next revision.

> > diff --git a/include/net/ip6_route.h b/include/net/ip6_route.h
> > index 95f936be32c6c..2684f6069a5e1 100644
> > --- a/include/net/ip6_route.h
> > +++ b/include/net/ip6_route.h
>
> [ ... ]
>
> > @@ -266,8 +266,16 @@ static inline void ip6_dst_store(struct sock *sk, struct dst_entry *dst,
> >  #endif
> >  }
> >  
> > +static inline void ip6_dst_store(struct sock *sk, struct dst_entry *dst,
> > +				 bool daddr_set,
> > +				 bool saddr_set)
> > +{
> > +	ip6_dst_store_cookie(sk, dst, rt6_get_cookie(dst_rt6_info(dst)),
> > +			     daddr_set, saddr_set);
> > +}
>
> [Severity: High]
> This wrapper still samples the cookie after the lookup. Three other
> writers of np->dst_cookie on TCPv6 sockets still go through it:
>
> net/ipv6/inet6_connection_sock.c:inet6_csk_route_socket() {
>     ...
> 	dst = ip6_dst_lookup_flow(sock_net(sk), sk, fl6, final_p);
>
> 	if (!IS_ERR(dst))
> 		ip6_dst_store(sk, dst, false, false);
>     ...
> }
>
> net/ipv6/af_inet6.c:inet6_sk_rebuild_header() {
>     ...
> 	dst = ip6_dst_lookup_flow(sock_net(sk), sk, fl6, final_p);
>     ...
> 	ip6_dst_store(sk, dst, false, false);
>     ...
> }
>
> net/ipv6/tcp_ipv6.c:tcp_v6_syn_recv_sock() {
>     ...
> 	memcpy(newnp, np, sizeof(struct ipv6_pinfo));
>
> 	ip6_dst_store(newsk, dst, false, false);
>     ...
> }
>
> Can the same xfrm policy bypass still happen through these paths?
>
> xfrm_policy_insert() links the policy and then calls
> rt_genid_bump_ipv6(). Suppose an insertion lands between
> ip6_dst_lookup_flow() returning a plain route and the rt6_get_cookie()
> call inside ip6_dst_store(). The plain dst is then cached with the new
> serial, and ip6_dst_check() keeps accepting it.
>
> These paths run much more often than tcp_v6_connect():
>
> - After connect(), every refill for an established socket goes through
>   inet6_csk_xmit()->inet6_csk_route_socket() whenever __sk_dst_check()
>   fails. That includes the invalidation caused by the policy insertion
>   itself.
> - A remote ICMPv6 Packet Too Big can force refills via
>   tcp_v6_mtu_reduced()->inet6_csk_update_pmtu()->inet6_csk_route_socket().
> - For accepted sockets, tcp_v6_syn_recv_sock() reads the cookie only
>   after inet6_csk_route_req(), tcp_create_openreq_child() and the
>   ipv6_pinfo memcpy. That window is wider than the one in connect().
>
> The commit message says this is a partial conversion. With the Fixes:
> tag, though, stable trees would get a fix that leaves established active
> sockets and all accepted sockets exposed. The new kernel-doc for
> ip6_dst_lookup_flow_cookie() also says "A caller caching the dst should
> use this cookie when validating it with dst_check". These callers don't
> do that.
>
> Would it make sense to convert inet6_csk_route_socket(),
> inet6_sk_rebuild_header() and tcp_v6_syn_recv_sock() in this patch as
> well?
>

This is the same point as on v2: the partial conversion is deliberate
and stated in the cover letter and in this commit message.

That said, these are the only remaining ip6_dst_store users. Converting
them would let me drop the late-sampling wrapper, so every write to
np->dst_cookie would use a lookup cookie. inet6_csk_route_socket and
inet6_sk_rebuild_header are direct conversions. tcp_v6_syn_recv_sock
needs the cookie to follow the dst that Fast Open passes in from
tcp_conn_request.

David, Ido, would you prefer this series to cover all ip6_dst_store
users, or to keep the current three paths and convert the rest along
with the other cache users in a follow-up? I'll respin accordingly.

The next revision will also include the small changes from this round: a
note in 1/3 that the generation advances once per ip6_link_failure call,
a wider comment for rt6_exception_net, the narrowed kdoc and the barrier
pairing comments in 2/3.

-- 
Ralf Lici
Mandelbit Srl

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2026-11-19 12:35 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-17  9:21 [PATCH net-next v3 0/3] ipv6: tie the dst cookie to the lookup that returned the dst Ralf Lici
2026-11-17  9:21 ` [PATCH net-next v3 1/3] ipv6: stamp the default route node with a fresh serial on link failure Ralf Lici
2026-11-19  9:22   ` netdev-bot+sashiko
2026-11-17  9:21 ` [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie Ralf Lici
2026-11-18  8:48   ` Xia Yan
2026-11-18 14:39     ` Ralf Lici
2026-11-18 15:07       ` Xia Yan
2026-11-19  9:22   ` netdev-bot+sashiko
2026-11-19 12:31     ` Ralf Lici
2026-11-17  9:21 ` [PATCH net-next v3 3/3] ipv6: use lookup cookies in selected socket dst cache paths Ralf Lici
2026-11-19  9:22   ` netdev-bot+sashiko
2026-11-19 12:34     ` Ralf Lici

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®