* Re: [PATCH net-next] net: lwtunnel: use GFP_KERNEL in build_state paths
2026-11-06 11:40 [PATCH net-next] net: lwtunnel: use GFP_KERNEL in build_state paths Max Boone
@ 2026-11-06 11:46 ` Boone, Max
2026-11-08 14:21 ` Ido Schimmel
1 sibling, 0 replies; 3+ messages in thread
From: Boone, Max @ 2026-11-06 11:46 UTC (permalink / raw)
To: David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, David Ahern, Ido Schimmel, Justin Iurman,
Andrea Mayer
Cc: Donald Sharp, netdev, linux-kernel, bpf, Banerjee, Debabrata,
Hudson, Nick
[-- Attachment #1: Type: text/plain, Size: 7884 bytes --]
Ran on a VM with 4 vCPUs, CONFIG_PROVE_LOCKING, CONFIG_PROVE_RCU
and CONFIG_DEBUG_ATOMIC_SLEEP and iproute2 5.15.0.
Lockdep before test:
lock-classes: 784 [max: 8192]
debug_locks: 1
Commands ran:
ip link add lwt0 type dummy
ip link set lwt0 up
ip -6 addr add fc00::1/64 dev lwt0 nodad
ip addr add 192.0.2.1/24 dev lwt0
# ip6_route_add -> fib6_nh_init -> seg6_build_state
ip -6 route add fc00:1::/64 encap seg6 mode encap \
segs fc00:ffff::2,fc00:ffff::3 dev lwt0
# fib_create_info -> fib_nh_init -> seg6_build_state
ip route add 198.51.100.0/24 encap seg6 mode encap \
segs fc00:ffff::2,fc00:ffff::3 dev lwt0
# ip6_route_multipath_add
ip -6 route add fc00:2::/64 \
nexthop via fc00::2 encap seg6 mode encap segs fc00:ffff::4 dev lwt0 \
nexthop via fc00::3 encap seg6 mode encap segs fc00:ffff::5 dev lwt0
# fib_table_delete -> fib_nh_match -> fib_encap_match
ip route del 198.51.100.0/24 encap seg6 mode encap \
segs fc00:ffff::2,fc00:ffff::3 dev lwt0
# rtm_new_nexthop -> nh_create_ipv4 -> fib_nh_init -> seg6_build_state
ip nexthop add id 1 encap seg6 mode encap \
segs fc00:ffff::2,fc00:ffff::3 dev lwt0
ip route add 203.0.113.0/24 nhid 1
# rtm_new_nexthop -> nh_create_ipv6 -> fib6_nh_init -> seg6_build_state
ip -6 nexthop add id 2 encap seg6 mode encap \
segs fc00:ffff::2,fc00:ffff::3 dev lwt0
ip -6 route add fc00:3::/64 nhid 2
All commands succeeded:
# ip nexthop show
id 1 encap seg6 mode encap segs 2 [ fc00:ffff::2 fc00:ffff::3 ] dev lwt0 scope host
id 2 encap seg6 mode encap segs 2 [ fc00:ffff::2 fc00:ffff::3 ] dev lwt0 scope link
# ip -6 route show
fc00::/64 dev lwt0 proto kernel metric 256 pref medium
fc00:1::/64 encap seg6 mode encap segs 2 [ fc00:ffff::2 fc00:ffff::3 ] dev lwt0 metric 1024 pref medium
fc00:2::/64 metric 1024 pref medium
nexthop encap seg6 mode encap segs 1 [ fc00:ffff::4 ] via fc00::2 dev lwt0 weight 1
nexthop encap seg6 mode encap segs 1 [ fc00:ffff::5 ] via fc00::3 dev lwt0 weight 1
fc00:3::/64 nhid 2 encap seg6 mode encap segs 2 [ fc00:ffff::2 fc00:ffff::3 ] dev lwt0 metric 1024 pref medium
fe80::/64 dev lwt0 proto kernel metric 256 pref medium
Burst test:
seq 3 20002 | sed "s/.*/nexthop add id & encap seg6 mode encap \
segs fc00:ffff::2 dev lwt0/" > /tmp/batch
ip -6 -force -batch /tmp/batch
nexthops present: 20002 (expected 20002), ENOMEM failures: 0
Kernel messages grep for RCU splats:
dmesg | grep -E 'BUG: sleeping function|suspicious RCU usage|Illegal
context switch|possible circular locking|inconsistent lock
state|WARNING:|BUG:|percpu: allocation failed'
(no matches)
Lockdep state after test:
debug_locks: 1
Reran the same test with rcu_read_lock() around ops->build_state() in
lwtunnel_build_state().
First "ip -6 route add" triggered "suspicious RCU usage” and a sleeping
function from invalid context BUG from seg6_build_state -> __kmalloc,
debug_locks dropped to 0.
> On Nov 6, 2026, at 12:40 PM, Max Boone <mboone@akamai.com> wrote:
>
> Since 3d25eabbbf67 ("ip_tunnel: Add dst_cache support in lwtunnel_state
> of ip tunnel"), build_state runs outside RCU and all its callers can
> sleep, but paths still use GFP_ATOMIC.
>
> For dst caches, that limits allocation to populated percpu pages. A
> large burst of SRv6 nexthops outruns the refill worker and will fail
> with -ENOMEM despite there being free memory:
>
> percpu: allocation failed, size=40 align=8 atomic=1, atomic alloc
> failed, no space left
>
> Batching 20000 RTM_NEWNEXTHOP under QEMU gave 10-109 failures per run
> before, none after.
>
> This commit converts the allocations reached through ops->build_state()
> and are called after the rcu_read_unlock():
>
> - ila_build_state(), ioam6_build_state(), rpl_build_state(),
> seg6_build_state(): build_state ops
> - bpf_parse_prog(): only called from bpf_build_state()
> - lwtunnel_state_alloc(): only called from build_state ops (also
> affects mpls, ip/ip6 tunnel, xfrmi and seg6_local)
>
> Signed-off-by: Max Boone <mboone@akamai.com>
> Reviewed-by: Debabrata Banerjee <dbanerje@akamai.com>
> Reviewed-by: Nick Hudson <nhudson@akamai.com>
> ---
> net/core/lwt_bpf.c | 2 +-
> net/core/lwtunnel.c | 2 +-
> net/ipv6/ila/ila_lwt.c | 2 +-
> net/ipv6/ioam6_iptunnel.c | 2 +-
> net/ipv6/rpl_iptunnel.c | 2 +-
> net/ipv6/seg6_iptunnel.c | 4 ++--
> 6 files changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/net/core/lwt_bpf.c b/net/core/lwt_bpf.c
> index da49364ec63d..6d1059ff0c49 100644
> --- a/net/core/lwt_bpf.c
> +++ b/net/core/lwt_bpf.c
> @@ -359,7 +359,7 @@ static int bpf_parse_prog(struct nlattr *attr, struct bpf_lwt_prog *prog,
> if (!tb[LWT_BPF_PROG_FD] || !tb[LWT_BPF_PROG_NAME])
> return -EINVAL;
>
> - prog->name = nla_memdup(tb[LWT_BPF_PROG_NAME], GFP_ATOMIC);
> + prog->name = nla_memdup(tb[LWT_BPF_PROG_NAME], GFP_KERNEL);
> if (!prog->name)
> return -ENOMEM;
>
> diff --git a/net/core/lwtunnel.c b/net/core/lwtunnel.c
> index 8223c44f10c8..f2f4d1ace0e9 100644
> --- a/net/core/lwtunnel.c
> +++ b/net/core/lwtunnel.c
> @@ -70,7 +70,7 @@ struct lwtunnel_state *lwtunnel_state_alloc(int encap_len)
> {
> struct lwtunnel_state *lws;
>
> - lws = kzalloc(sizeof(*lws) + encap_len, GFP_ATOMIC);
> + lws = kzalloc(sizeof(*lws) + encap_len, GFP_KERNEL);
>
> return lws;
> }
> diff --git a/net/ipv6/ila/ila_lwt.c b/net/ipv6/ila/ila_lwt.c
> index 7bb9edc5c28c..01fd78a232e7 100644
> --- a/net/ipv6/ila/ila_lwt.c
> +++ b/net/ipv6/ila/ila_lwt.c
> @@ -228,7 +228,7 @@ static int ila_build_state(struct net *net, struct nlattr *nla,
> return -ENOMEM;
>
> ilwt = ila_lwt_lwtunnel(newts);
> - ret = dst_cache_init(&ilwt->dst_cache, GFP_ATOMIC);
> + ret = dst_cache_init(&ilwt->dst_cache, GFP_KERNEL);
> if (ret) {
> kfree(newts);
> return ret;
> diff --git a/net/ipv6/ioam6_iptunnel.c b/net/ipv6/ioam6_iptunnel.c
> index cfb2c41634a0..3093446ad89a 100644
> --- a/net/ipv6/ioam6_iptunnel.c
> +++ b/net/ipv6/ioam6_iptunnel.c
> @@ -166,7 +166,7 @@ static int ioam6_build_state(struct net *net, struct nlattr *nla,
> return -ENOMEM;
>
> ilwt = ioam6_lwt_state(lwt);
> - err = dst_cache_init(&ilwt->cache, GFP_ATOMIC);
> + err = dst_cache_init(&ilwt->cache, GFP_KERNEL);
> if (err)
> goto free_lwt;
>
> diff --git a/net/ipv6/rpl_iptunnel.c b/net/ipv6/rpl_iptunnel.c
> index 4e10adcd70e8..838ede10a79d 100644
> --- a/net/ipv6/rpl_iptunnel.c
> +++ b/net/ipv6/rpl_iptunnel.c
> @@ -102,7 +102,7 @@ static int rpl_build_state(struct net *net, struct nlattr *nla,
>
> rlwt = rpl_lwt_lwtunnel(newts);
>
> - err = dst_cache_init(&rlwt->cache, GFP_ATOMIC);
> + err = dst_cache_init(&rlwt->cache, GFP_KERNEL);
> if (err) {
> kfree(newts);
> return err;
> diff --git a/net/ipv6/seg6_iptunnel.c b/net/ipv6/seg6_iptunnel.c
> index 0d39a38a5dd2..7c64680e4632 100644
> --- a/net/ipv6/seg6_iptunnel.c
> +++ b/net/ipv6/seg6_iptunnel.c
> @@ -833,11 +833,11 @@ static int seg6_build_state(struct net *net, struct nlattr *nla,
>
> slwt = seg6_lwt_lwtunnel(newts);
>
> - err = dst_cache_init(&slwt->cache_input, GFP_ATOMIC);
> + err = dst_cache_init(&slwt->cache_input, GFP_KERNEL);
> if (err)
> goto err_free_newts;
>
> - err = dst_cache_init(&slwt->cache_output, GFP_ATOMIC);
> + err = dst_cache_init(&slwt->cache_output, GFP_KERNEL);
> if (err)
> goto err_destroy_input;
>
>
> ---
> base-commit: 0822d2fe38262e8b55c5ea55c970022bca8dd2a9
> change-id: 20261105-build-state-use-gfp-kernel-c3598d190bfb
>
> Best regards,
> --
> Max Boone <mboone@akamai.com>
>
[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3061 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH net-next] net: lwtunnel: use GFP_KERNEL in build_state paths
2026-11-06 11:40 [PATCH net-next] net: lwtunnel: use GFP_KERNEL in build_state paths Max Boone
2026-11-06 11:46 ` Boone, Max
@ 2026-11-08 14:21 ` Ido Schimmel
1 sibling, 0 replies; 3+ messages in thread
From: Ido Schimmel @ 2026-11-08 14:21 UTC (permalink / raw)
To: Max Boone
Cc: David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, David Ahern, Justin Iurman, Andrea Mayer,
Donald Sharp, netdev, linux-kernel, bpf, Debabrata Banerjee,
Nick Hudson
On Fri, Nov 06, 2026 at 12:40:55PM +0100, Max Boone wrote:
> Since 3d25eabbbf67 ("ip_tunnel: Add dst_cache support in lwtunnel_state
> of ip tunnel"), build_state runs outside RCU and all its callers can
> sleep, but paths still use GFP_ATOMIC.
>
> For dst caches, that limits allocation to populated percpu pages. A
> large burst of SRv6 nexthops outruns the refill worker and will fail
> with -ENOMEM despite there being free memory:
>
> percpu: allocation failed, size=40 align=8 atomic=1, atomic alloc
> failed, no space left
>
> Batching 20000 RTM_NEWNEXTHOP under QEMU gave 10-109 failures per run
> before, none after.
>
> This commit converts the allocations reached through ops->build_state()
> and are called after the rcu_read_unlock():
>
> - ila_build_state(), ioam6_build_state(), rpl_build_state(),
> seg6_build_state(): build_state ops
> - bpf_parse_prog(): only called from bpf_build_state()
> - lwtunnel_state_alloc(): only called from build_state ops (also
> affects mpls, ip/ip6 tunnel, xfrmi and seg6_local)
>
> Signed-off-by: Max Boone <mboone@akamai.com>
> Reviewed-by: Debabrata Banerjee <dbanerje@akamai.com>
> Reviewed-by: Nick Hudson <nhudson@akamai.com>
Reviewed-by: Ido Schimmel <idosch@nvidia.com>
^ permalink raw reply [flat|nested] 3+ messages in thread