mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Ralf Lici <ralf@mandelbit.com>
To: Xia Yan <yanxia@cmss.chinamobile.com>
Cc: netdev@vger.kernel.org, David Ahern <dsahern@kernel.org>,
	Ido Schimmel <idosch@nvidia.com>,
	"David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@kernel.org>,
	Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
	Simon Horman <horms@kernel.org>,
	Neal Cardwell <ncardwell@google.com>,
	Kuniyuki Iwashima <kuniyu@google.com>,
	Steffen Klassert <steffen.klassert@secunet.com>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	Florian Westphal <fw@strlen.de>,
	Sabrina Dubroca <sd@queasysnail.net>,
	Antonio Quartulli <antonio@mandelbit.com>,
	linux-kernel@vger.kernel.org, yana06070@163.com
Subject: Re: [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie
Date: Wed, 18 Nov 2026 15:39:08 +0100	[thread overview]
Message-ID: <20261118143909.115729-1-ralf@mandelbit.com> (raw)
In-Reply-To: <1794991714000.1115021.7710558447771708161@cmss.chinamobile.com>

Hi Xia,

On Wed, 18 Nov 2026 16:48:34 +0800, Xia Yan <yanxia@cmss.chinamobile.com> wrote:
> I traced the race and the fix against current net-next.
>
> On the writer side, xfrm_policy_insert() links the policy and then calls
> rt_genid_bump_ipv6(), which resolves to fib6_flush_trees(). A fresh serial
> is allocated from the namespace counter first, then every node is
> restamped under tb6_lock. A late rt6_get_cookie() can pick up that
> post-insertion serial for a dst whose lookup ran before the insertion,
> and the pair keeps passing dst_check until the next flush. Sampling the
> cookie between two generation reads closes the in-lookup window. A
> policy inserted after the helper returns is still caught later, because
> the restamp changes fn_sernum itself and the stored pre-insertion cookie
> stops matching. fib6_new_sernum() never returns 0, so the fallback
> cookie always fails fib6_check().
>

Yes, that's a pretty good summary of the change with the small
correction that a newly linked node holds fn_sernum 0 until fib6_add
stamps it. So a cookie-0 dst can pass a check in that window, and fails
as soon as the stamp lands.

> 1/3 checks out as well. The old code kept writing -1 into the node, so a
> second failure changed nothing and the namespace generation never moved.
> The fresh allocation under tb6_lock matches the discipline the
> fib6_ifup()/fib6_ifdown() paths already get from __fib6_clean_all().
> The conversion boundary in 3/3 also matches the tree: rawv6 does not
> cache the dst, and the accepted-socket and refill stores in tcp_ipv6.c
> are listed as follow-up.
>

In fact there are many users of cached IPv6 dsts that will need
conversion and this series is mainly here to introduce the new API.

> One thing I could not fully work out is the ordering behind the
> generation checks. When the lookup misses a change, the reader relies on
> the two generation reads to notice it. My understanding is that this
> needs the change visible before the counter advance, and the counter
> advance visible before the node restamps. I could not establish either
> hop, though I may well be missing something.
>
> In xfrm_policy_insert(), the policy is linked into the hash and
> rt_genid_bump_ipv6() runs within the same critical section. As far as I
> can tell, the linking stores and the cmpxchg on fib6_sernum are all
> relaxed, with no barrier between them. Could a reader read the advanced
> generation in gen0, miss the policy during the lookup, sample a
> restamped fn_sernum, and still read the same generation in gen1?
>

Both hops are covered by fib6_new_sernum. It allocates the serial with
atomic_try_cmpxchg, which is fully ordered when it succeeds, so it acts
as an smp_mb before and after the update of fib6_sernum.

For the first hop, the policy (or route) is linked before
fib6_new_sernum runs, which gives message passing against the smp_rmb
after gen0: a reader that reads the advanced generation in gen0 also
finds the policy.

> In fib6_flush_trees(), the serial is allocated before __fib6_clean_all()
> takes tb6_lock, and fib6_clean_node() stamps the nodes with a plain
> WRITE_ONCE(). spin_lock() is an ACQUIRE, and my reading of
> memory-barriers.txt is that operations issued before an ACQUIRE may
> complete after it. So a reader could sample the restamped fn_sernum

For the second hop, every fn_sernum write follows a successful
fib6_new_sernum on the same CPU: fib6_update_sernum,
__fib6_update_sernum_upto_root, and fib6_clean_node in the flush case.
The full barrier orders the counter update before the tb6_lock
acquisition and the stamps, so the ACQUIRE semantics of spin_lock don't
matter here. A reader that samples a restamped fn_sernum reads the
advanced generation in gen1 after the smp_rmb, and the helper returns
cookie 0.

> between two reads that both still return the old generation. The only
> pairing I found written down here is fib6_get_cookie_safe()'s smp_rmb()
> against the smp_wmb() in __fib6_update_sernum_upto_root().
>

Right, this isn't documented. The barrier comments in
ip6_dst_lookup_flow_cookie should name fib6_new_sernum as the pairing
side, and fib6_new_sernum should state that its full ordering is relied
upon.

I also checked both hops with an LKMM litmus test: the outcome you
describe is forbidden with atomic_cmpxchg and becomes reachable with
atomic_cmpxchg_relaxed.

While going through the writers I also noticed that xfrm_policy_insert
unlinks a replaced policy only after rt_genid_bump_ipv6, so a lookup
after the bump can still match the old policy. That is a writer-side
ordering issue independent of this series, and it affects the current
late cookie sampling in the same way. I'll send a separate fix.

Does this explanation address your concern? If so, I'll carry your
Reviewed-by. I'm waiting for the Sashiko review of v3: if it reports
anything that needs a respin, I'll add the barrier comments in v4,
otherwise I'll send them as a follow-up.

Thanks for the careful review!

-- 
Ralf Lici
Mandelbit Srl

  reply	other threads:[~2026-11-18 14:39 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-17  9:21 [PATCH net-next v3 0/3] ipv6: tie the dst cookie to the lookup that returned the dst Ralf Lici
2026-11-17  9:21 ` [PATCH net-next v3 1/3] ipv6: stamp the default route node with a fresh serial on link failure Ralf Lici
2026-11-17  9:21 ` [PATCH net-next v3 2/3] ipv6: add ip6_dst_lookup_flow_cookie Ralf Lici
2026-11-18  8:48   ` Xia Yan
2026-11-18 14:39     ` Ralf Lici [this message]
2026-11-18 15:07       ` Xia Yan
2026-11-17  9:21 ` [PATCH net-next v3 3/3] ipv6: use lookup cookies in selected socket dst cache paths Ralf Lici

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118143909.115729-1-ralf@mandelbit.com \
    --to=ralf@mandelbit.com \
    --cc=antonio@mandelbit.com \
    --cc=davem@davemloft.net \
    --cc=dsahern@kernel.org \
    --cc=edumazet@kernel.org \
    --cc=fw@strlen.de \
    --cc=herbert@gondor.apana.org.au \
    --cc=horms@kernel.org \
    --cc=idosch@nvidia.com \
    --cc=kuba@kernel.org \
    --cc=kuniyu@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=ncardwell@google.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=sd@queasysnail.net \
    --cc=steffen.klassert@secunet.com \
    --cc=yana06070@163.com \
    --cc=yanxia@cmss.chinamobile.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®