Re: [PATCH net 2/2] selftests: net: check TCP IP IDs across tunnels after GRO and GSO

Re: [PATCH net 2/2] selftests: net: check TCP IP IDs across tunnels after GRO and GSO

[netdev-bot+sashiko]

Thank you for your contribution! Sashiko AI review found 8 potential
issue(s) to consider.

Critical: 1 · High: 0 · Medium: 2 · Low: 5

If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).

- [Critical] ipv6: seg6: use-after-free of hdr in ipv6_srh_rcv() after
  iptunnel_pull_offloads()
- [Medium] The test says it checks that "the inner IDs must come out as
  they went in" (tcp_fixedid.sh header, lines 16-18, and the commit…
- [Medium] udp: gso: inner hw_enc_features re-masking clobbers remcsum
  NETIF_F_HW_CSUM
- [Low] In tools/testing/selftests/net/Makefile, the patch adds
  tcp_fixedid.sh to TEST_PROGS at line 109, between tcp_ecmp_failover.sh…
- [Low] The "GRO did not merge" skip in check() (tcp_fixedid.sh lines
  154-156) uses the change in tun0's statistics/tx_packets as its input.
- [Low] run_decap() in tcp_fixedid.sh ignores the return value of `tunnel
  "$NRTR" tun0 ipip 4 10.2.0.1 10.2.0.2` (line 212).
- [Low] tcp_fixedid.sh creates a temporary file with `RESULT=$(mktemp)`
  (line 276).
- [Low] capture() in tcp_fixedid.sh (lines 162-172) starts the receiver
  in the background and polls RESULT for "ready" up to 100 times at 0.1…

--

selftests: net: check TCP IP IDs across tunnels after GRO and GSO

This adds a selftest that sends TCP segments with fixed or incrementing
IPv4 IDs onto a veth with GRO enabled. The merged packets are forwarded
through IPIP, GRE and VXLAN tunnels with software segmentation, and the
inner and outer IP IDs are checked. It also covers VXLAN decapsulation into
IPIP, and a tunnel that is decapsulated without segmentation before
forwarding.

> diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile
> index 9180b9bec5155..dd402814bc748 100644
> --- a/tools/testing/selftests/net/Makefile
> +++ b/tools/testing/selftests/net/Makefile
> @@ -106,6 +106,7 @@ TEST_PROGS := \
>  	srv6_iptunnel_cache.sh \
>  	stress_reuseport_listen.sh \
>  	tcp_ecmp_failover.sh \
> +	tcp_fixedid.sh \
>  	tcp_fastopen_backup_key.sh \

[Severity: Low]
This isn't a bug, but tcp_fixedid.sh is out of alphabetical order here. It
should come after tcp_fastopen_backup_key.sh. The TEST_GEN_FILES and
.gitignore entries in this patch are in the right place.

[ ... ]

> diff --git a/tools/testing/selftests/net/tcp_fixedid.sh b/tools/testing/selftests/net/tcp_fixedid.sh
> new file mode 100755
> index 0000000000000..88f9298174077
> --- /dev/null
> +++ b/tools/testing/selftests/net/tcp_fixedid.sh
> @@ -0,0 +1,291 @@

[ ... ]

> +cleanup() {
> +	cleanup_all_ns
> +}
> +
> +trap cleanup EXIT

[Severity: Low]
Should cleanup() also remove "$RESULT"?

The file is created with RESULT=$(mktemp) near the end of the script. It is
only removed by the rm -f "$RESULT" on the normal exit path.

If the script is interrupted, for example by Ctrl-C or a kselftest runner
timeout, the EXIT trap only calls cleanup_all_ns. The temp file is then left
in $TMPDIR.

[ ... ]

> +# check <name> <result> <want: fixed|incr> <outer: 0|1> <merged-into-dev>
> +check() {
> +	local name=$1 res=$2 want=$3 outer=$4 sent=$5
> +	local frames outer_ids inner_ids
> +
> +	read -r _ frames _ _ _ outer_ids _ inner_ids <<< "$res"
> +	RET=0
> +	check_err $((frames != TOTAL)) "$frames of $TOTAL segments received"
> +	if [ "$want" = fixed ]; then
> +		check_err $((inner_ids != 1)) "$inner_ids inner IDs, want 1"
> +	else
> +		check_err $((inner_ids != TOTAL)) \
> +			"$inner_ids inner IDs, want $TOTAL"
> +	fi

[Severity: Medium]
The header comment and the commit message say "The inner IDs must come out
as they went in". Does anything compare the received IDs with the values
that were sent?

do_tx() in tcp_fixedid.c writes known values:

    uint16_t id = fixed_id ? FIXED_ID : 0x1000 + i;

do_rx(), though, only records each received ID in a bitmap through
count_id() and prints how many distinct values it saw. check() then only
compares that count with 1 or TOTAL.

A regression that changed every fixed 0x1234 inner ID to some other
constant, such as 0, would still give inner_ids == 1 and pass. The same is
true for a regression that replaced the 0x1000..0x1000+TOTAL-1 sequence with
any other set of TOTAL unique values.

Could the receiver check the actual values instead? For example, fixed IDs
could be required to equal FIXED_ID, and incrementing IDs to fall in the
range the sender used.

> +	if [ "$outer" = 1 ]; then
> +		check_err $((outer_ids != frames)) \
> +			"$outer_ids outer IDs for $frames segments"
> +	fi
> +	if [ "$RET" -eq 0 ] && [ "$sent" -ge "$TOTAL" ]; then
> +		ret_set_ksft_status "$ksft_skip" "GRO did not merge"
> +	fi

[Severity: Low]
The skip decision uses the change in tun0's statistics/tx_packets, which the
comment above calls <merged-into-dev>. Doesn't tx_packets count every skb
sent through tun0, not just the forwarded test stream?

gre, ip6gre and vxlan devices get an IPv6 link-local address from addrconf.
That means MLD reports, router solicitations and, for vxlan, DAD neighbour
solicitations can go out through tun0 between the t0 read and the final
tx_packets read.

If GRO merged only a few segments, this extra traffic could push sent to
TOTAL or above. A run that did merge would then be reported as SKIP. This
can only happen when GRO barely merged anything, and it cannot cause a false
PASS.

The header comment also says "Packets that GRO did not merge are not
tested". In practice the receiver counts every frame and check() looks at
all of them. The only related rule skips the whole run when sent >= TOTAL.
Could the comment be reworded to match what the code does?

> +	log_test "$name"
> +}
> +
> +# capture <ns> <dev>: start the receiver in the background, pid in CAP_PID,
> +# and wait until it is bound
> +capture() {
> +	local i
> +
> +	: > "$RESULT"
> +	ip netns exec "$1" "$TX" -R -i "$2" -c "$TOTAL" -w 3000 > "$RESULT" &
> +	CAP_PID=$!
> +	for i in $(seq 100); do
> +		grep -q ready "$RESULT" && return
> +		sleep 0.1
> +	done
> +}

[Severity: Low]
What happens if the receiver never prints "ready"? After 100 iterations the
loop falls through, and capture() returns success without checking whether
CAP_PID is still running.

The callers start the sender right away and ignore the exit status of:

    wait "$CAP_PID"

Suppose the receiver failed in main() at socket(), if_nametoindex() or
bind(), or took more than 10 seconds to start. Then RESULT has no result
line, and check() reports something like " of 400 segments received" with
empty counts instead of a setup error.

Could capture() fail, or the test skip, when "ready" never appears or the
receiver has already exited?

[ ... ]

> +# run_decap <fixed: inner|outer>: VXLAN from NSRC, decapsulated in NRTR
> +run_decap() {
> +	local fixed=$1 want=incr flags=-F
> +	local r0mac t0
> +
> +	setup || { log_test_skip "decap"; return; }
> +	ip -n "$NRTR" link add vx0 address "$VX_MAC" type vxlan id 42 \
> +		dstport 4789 local 10.1.0.2 nolearning ||
> +		{ log_test_skip "decap"; cleanup_all_ns; return; }
> +	ip -n "$NRTR" addr add 10.3.0.2/24 dev vx0
> +	ip -n "$NRTR" link set vx0 up
> +	tunnel "$NRTR" tun0 ipip 4 10.2.0.1 10.2.0.2
> +	route_into "$NRTR" tun0

[Severity: Low]
Should the return value of tunnel() be checked here, as run_encap() and
run_encap_decap() do?

If the ipip link can't be created, the route add in route_into() fails, but
the function still returns success:

    route_into() {
        ip -n "$1" route add "$INNER_DST/32" dev "$2"
        ...
        return 0
    }

Nothing then reaches m1. The receiver waits 10 * 3000 ms for a first frame
and prints frames 0, and check() reports a FAIL instead of a SKIP.

On such a kernel the ipip encap cases SKIP, while both decap cases FAIL
after about 30 seconds each. The standard selftests config sets
CONFIG_NET_IPIP=y, so this only shows up with other configs.

[ ... ]

> @@ -0,0 +1,291 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Check the IPv4 IDs of a TCP stream that GRO merged and software
> +# segmentation split again across a tunnel. A sender in NSRC writes
> +# segments with fixed or incrementing IDs onto a veth with GRO on in
> +# NRTR, which forwards the merged packets:
> +#
> +#  encap:       into an IPIP, GRE or VXLAN tunnel over IPv4 or IPv6,
> +#               segmented on the underlay towards NMID
> +#  decap:       out of VXLAN, received from NSRC with one header fixed and
> +#               the other incrementing, into an IPIP tunnel towards NMID
> +#  encap+decap: into a tunnel that NMID ends without segmenting, and on
> +#               to NDST, segmented there
> +#
> +# The inner IDs must come out as they went in, and the IDs of a new outer
> +# IPv4 header must be unique. Packets that GRO did not merge are not
> +# tested, so a run where nothing was merged is skipped.
> +
> +source lib.sh
> +
> +NSRC=""
> +NRTR=""
> +NMID=""
> +NDST=""
> +
> +SEGS=40
> +BURSTS=10
> +TOTAL=$((SEGS * BURSTS))
> +INNER_SRC=10.1.0.1
> +INNER_DST=10.9.0.2
> +VX_MAC=02:00:00:00:00:42
> +TUN_MAC=02:00:00:00:00:99
> +MID_MAC=02:00:00:00:00:98
> +TX=./tcp_fixedid
> +
> +NOSEG="tso gso tx-gre-segmentation tx-gre-csum-segmentation
> +       tx-ipxip4-segmentation tx-ipxip6-segmentation
> +       tx-udp_tnl-segmentation tx-udp_tnl-csum-segmentation"

[Severity: Medium]
Also outside this patch, but on the software UDP tunnel segmentation
path that NOSEG forces. With SKB_GSO_CB(skb)->encap_level = 0 removed
from __skb_udp_tunnel_segment(), the inner IP GSO handler now sees
encap_level > 0. That undoes the checksum setup for remote checksum
offload.

__skb_udp_tunnel_segment() masks features with
skb->dev->hw_enc_features. For SKB_GSO_TUNNEL_REMCSUM it then clears
NETIF_F_CSUM_MASK. It sets NETIF_F_HW_CSUM if the outer checksum is
not needed or can be offloaded. That way the inner segmentation does
not compute checksums that remcsum replaces anyway.

gso_inner_segment() then calls inet_gso_segment() or
ipv6_gso_segment(). Both of them do:

    encap = SKB_GSO_CB(skb)->encap_level > 0;
    if (encap)
        features &= skb->dev->hw_enc_features;

This used to be skipped for the inner header. Now features is masked a
second time, after the remcsum adjustment, so NETIF_F_HW_CSUM only
survives if hw_enc_features contains it.

Some devices have hw_enc_features of 0. Others only advertise
NETIF_F_IP_CSUM and NETIF_F_IPV6_CSUM, which remcsum already cleared.
On either kind of device no checksum bit is left, and skb_segment()
takes the !csum path. It then computes a software checksum for every
segment and linearizes segments with shared frags. It also can no
longer use GSO_PARTIAL or frag_list splitting.

As far as I can see the packets are still correct. But this is a
performance regression for remcsum users, and the series does not
mention it.

Was the second mask intended? If the goal is only to tell the inner
header that it is encapsulated, for the IP ID handling, could that be
signalled some other way? Or could the remcsum features be preserved
across the inner mask?

The vxlan cases in this test do not enable remcsumtx, so the test
would not notice this.

> +
> +cleanup() {
> +	cleanup_all_ns
> +}
> +
> +trap cleanup EXIT
> +
> +# NSRC s0 - r0 NRTR r1 - m1 NMID m2 - d2 NDST
> +setup() {
> +	setup_ns NSRC NRTR NMID NDST || return "$ksft_skip"
> +
> +	ip link add s0 netns "$NSRC" type veth peer name r0 netns "$NRTR"
> +	ip link add r1 netns "$NRTR" type veth peer name m1 netns "$NMID"
> +	ip link add m2 netns "$NMID" type veth peer name d2 netns "$NDST"
> +
> +	ip -n "$NRTR" addr add 10.1.0.2/24 dev r0
> +	ip -n "$NRTR" addr add 10.2.0.1/24 dev r1
> +	ip -n "$NRTR" addr add fd02::1/64 dev r1 nodad
> +	ip -n "$NMID" addr add 10.2.0.2/24 dev m1
> +	ip -n "$NMID" addr add fd02::2/64 dev m1 nodad
> +	ip -n "$NMID" addr add 10.4.0.1/24 dev m2
> +	ip -n "$NDST" addr add 10.4.0.2/24 dev d2
> +	ip -n "$NSRC" link set s0 up
> +	ip -n "$NRTR" link set r0 up
> +	ip -n "$NRTR" link set r1 up
> +	ip -n "$NMID" link set m1 up
> +	ip -n "$NMID" link set m2 up
> +	ip -n "$NDST" link set d2 up
> +
> +	ip netns exec "$NRTR" sysctl -qw net.ipv4.ip_forward=1
> +	ip netns exec "$NMID" sysctl -qw net.ipv4.ip_forward=1
> +	for ns in "$NRTR" "$NMID"; do
> +		ip netns exec "$ns" sysctl -qw net.ipv4.conf.all.rp_filter=0 \
> +			net.ipv4.conf.default.rp_filter=0
> +	done
> +
> +	# veth passes packets to GRO only if the sender does not do TSO
> +	ip netns exec "$NRTR" ethtool -K r0 gro on >/dev/null ||
> +		return "$ksft_skip"
> +	ip netns exec "$NSRC" ethtool -K s0 tso off >/dev/null ||
> +		return "$ksft_skip"
> +	# release the sender's segments in bursts for GRO to merge
> +	tc -n "$NSRC" qdisc add dev s0 root netem slot 20ms 20ms \
> +		packets "$SEGS" || return "$ksft_skip"
> +}
> +
> +# tunnel <ns> <name> <type> <family> <local> <remote>
> +tunnel() {
> +	local ns=$1 name=$2 type=$3 family=$4 local=$5 remote=$6
> +
> +	case "$type" in
> +	ipip)
> +		if [ "$family" = 4 ]; then
> +			ip -n "$ns" link add "$name" type ipip \
> +				local "$local" remote "$remote"
> +		else
> +			ip -n "$ns" link add "$name" type ip6tnl mode ipip6 \
> +				local "$local" remote "$remote" encaplimit none
> +		fi ;;
> +	gre)
> +		if [ "$family" = 4 ]; then
> +			ip -n "$ns" link add "$name" type gre \
> +				local "$local" remote "$remote"
> +		else
> +			ip -n "$ns" link add "$name" type ip6gre \
> +				local "$local" remote "$remote" encaplimit none
> +		fi ;;
> +	vxlan)
> +		ip -n "$ns" link add "$name" address "$TUN_MAC" type vxlan \
> +			id 7 dstport 4789 local "$local" remote "$remote" \
> +			nolearning ;;
> +	esac || return 1
> +	ip -n "$ns" link set "$name" up
> +}
> +
> +# route_into <ns> <dev>: send INNER_DST into <dev>
> +route_into() {
> +	ip -n "$1" route add "$INNER_DST/32" dev "$2"
> +	ip -n "$1" neigh add "$INNER_DST" lladdr "$TUN_MAC" dev "$2" \
> +		2>/dev/null
> +	return 0
> +}
> +
> +# noseg <ns> <dev>: segment everything in software before <dev>
> +noseg() {
> +	local f
> +
> +	for f in $NOSEG; do
> +		ip netns exec "$1" ethtool -K "$2" "$f" off >/dev/null 2>&1
> +	done
> +}
> +
> +tx_packets() {
> +	ip netns exec "$1" cat "/sys/class/net/$2/statistics/tx_packets"
> +}
> +
> +# check <name> <result> <want: fixed|incr> <outer: 0|1> <merged-into-dev>
> +check() {
> +	local name=$1 res=$2 want=$3 outer=$4 sent=$5
> +	local frames outer_ids inner_ids
> +
> +	read -r _ frames _ _ _ outer_ids _ inner_ids <<< "$res"
> +	RET=0
> +	check_err $((frames != TOTAL)) "$frames of $TOTAL segments received"
> +	if [ "$want" = fixed ]; then
> +		check_err $((inner_ids != 1)) "$inner_ids inner IDs, want 1"
> +	else
> +		check_err $((inner_ids != TOTAL)) \
> +			"$inner_ids inner IDs, want $TOTAL"
> +	fi
> +	if [ "$outer" = 1 ]; then
> +		check_err $((outer_ids != frames)) \
> +			"$outer_ids outer IDs for $frames segments"
> +	fi
> +	if [ "$RET" -eq 0 ] && [ "$sent" -ge "$TOTAL" ]; then
> +		ret_set_ksft_status "$ksft_skip" "GRO did not merge"
> +	fi
> +	log_test "$name"
> +}
> +
> +# capture <ns> <dev>: start the receiver in the background, pid in CAP_PID,
> +# and wait until it is bound
> +capture() {
> +	local i
> +
> +	: > "$RESULT"
> +	ip netns exec "$1" "$TX" -R -i "$2" -c "$TOTAL" -w 3000 > "$RESULT" &
> +	CAP_PID=$!
> +	for i in $(seq 100); do
> +		grep -q ready "$RESULT" && return
> +		sleep 0.1
> +	done
> +}
> +
> +run_encap() {
> +	local type=$1 family=$2 want=$3
> +	local r0mac t0 laddr=10.2.0.1 raddr=10.2.0.2
> +	local f=""
> +
> +	setup || { log_test_skip "encap $type IPv$family"; return; }
> +	if [ "$family" = 6 ]; then
> +		laddr=fd02::1
> +		raddr=fd02::2
> +	fi
> +	tunnel "$NRTR" tun0 "$type" "$family" "$laddr" "$raddr" ||
> +		{ log_test_skip "encap $type IPv$family"; cleanup_all_ns; return; }
> +	route_into "$NRTR" tun0
> +	noseg "$NRTR" r1
> +
> +	r0mac=$(ip -n "$NRTR" -br link show r0 | awk '{ print $3 }')
> +	capture "$NMID" m1
> +	t0=$(tx_packets "$NRTR" tun0)
> +	[ "$want" = fixed ] && f=-f
> +	ip netns exec "$NSRC" "$TX" -T -i s0 -M "$r0mac" -s "$INNER_SRC" \
> +		-d "$INNER_DST" -n "$SEGS" -b "$BURSTS" $f
> +	wait "$CAP_PID"
> +	check "encap $type IPv$family, $want inner IDs" "$(tail -n 1 "$RESULT")" \
> +		"$want" $((family == 4)) $(($(tx_packets "$NRTR" tun0) - t0))
> +	cleanup_all_ns
> +}
> +
> +# run_decap <fixed: inner|outer>: VXLAN from NSRC, decapsulated in NRTR
> +run_decap() {
> +	local fixed=$1 want=incr flags=-F
> +	local r0mac t0
> +
> +	setup || { log_test_skip "decap"; return; }
> +	ip -n "$NRTR" link add vx0 address "$VX_MAC" type vxlan id 42 \
> +		dstport 4789 local 10.1.0.2 nolearning ||
> +		{ log_test_skip "decap"; cleanup_all_ns; return; }
> +	ip -n "$NRTR" addr add 10.3.0.2/24 dev vx0
> +	ip -n "$NRTR" link set vx0 up
> +	tunnel "$NRTR" tun0 ipip 4 10.2.0.1 10.2.0.2
> +	route_into "$NRTR" tun0
> +	noseg "$NRTR" r1
> +
> +	if [ "$fixed" = inner ]; then
> +		want=fixed
> +		flags=-f
> +	fi
> +	r0mac=$(ip -n "$NRTR" -br link show r0 | awk '{ print $3 }')
> +	capture "$NMID" m1
> +	t0=$(tx_packets "$NRTR" tun0)
> +	ip netns exec "$NSRC" "$TX" -T -i s0 -M "$r0mac" -s 10.3.0.1 \
> +		-d "$INNER_DST" -n "$SEGS" -b "$BURSTS" \
> +		-V 10.1.0.1,10.1.0.2 -m "$VX_MAC" $flags
> +	wait "$CAP_PID"
> +	check "decap VXLAN with fixed $fixed IDs, into IPIP" \
> +		"$(tail -n 1 "$RESULT")" "$want" 1 \
> +		$(($(tx_packets "$NRTR" tun0) - t0))
> +	cleanup_all_ns
> +}
> +
> +# run_encap_decap <type>: NRTR encapsulates, NMID decapsulates the GSO
> +# packet without segmenting it and forwards it to NDST
> +run_encap_decap() {
> +	local type=$1
> +	local r0mac t0
> +
> +	setup || { log_test_skip "encap+decap $type"; return; }
> +	if ! tunnel "$NRTR" tun0 "$type" 4 10.2.0.1 10.2.0.2 ||
> +	   ! tunnel "$NMID" tun0 "$type" 4 10.2.0.2 10.2.0.1; then
> +		log_test_skip "encap+decap $type"
> +		cleanup_all_ns
> +		return
> +	fi
> +	route_into "$NRTR" tun0
> +	# VXLAN drops frames from its own address as loops
> +	if [ "$type" = vxlan ]; then
> +		ip -n "$NMID" link set tun0 address "$MID_MAC"
> +		ip -n "$NRTR" neigh replace "$INNER_DST" lladdr "$MID_MAC" \
> +			dev tun0
> +	fi
> +	ip -n "$NMID" route add "$INNER_DST/32" via 10.4.0.2
> +	ip -n "$NDST" addr add "$INNER_DST/32" dev d2
> +	noseg "$NMID" m2
> +
> +	r0mac=$(ip -n "$NRTR" -br link show r0 | awk '{ print $3 }')
> +	capture "$NDST" d2
> +	t0=$(tx_packets "$NRTR" tun0)
> +	ip netns exec "$NSRC" "$TX" -T -i s0 -M "$r0mac" -s "$INNER_SRC" \
> +		-d "$INNER_DST" -n "$SEGS" -b "$BURSTS" -f
> +	wait "$CAP_PID"
> +	check "encap+decap $type, fixed inner IDs" "$(tail -n 1 "$RESULT")" fixed 0 \
> +		$(($(tx_packets "$NRTR" tun0) - t0))
> +	cleanup_all_ns
> +}
> +
> +require_command ethtool
> +require_command tc
> +
> +if [ ! -x "$TX" ]; then
> +	echo "SKIP: $TX not built"
> +	exit "$ksft_skip"
> +fi
> +
> +RESULT=$(mktemp)
> +
> +for type in ipip gre vxlan; do
> +	for family in 4 6; do
> +		run_encap "$type" "$family" fixed
> +		run_encap "$type" "$family" incr
> +	done
> +done
> +run_decap inner
> +run_decap outer
> +for type in ipip gre vxlan; do
> +	run_encap_decap "$type"
> +done
> +
> +rm -f "$RESULT"
> +exit "$EXIT_STATUS"

[Severity: Critical]
This is outside this patch, but the GSO decap path these tests are
about has a use-after-free in ipv6_srh_rcv() in net/ipv6/exthdrs.c,
now that it calls iptunnel_pull_offloads().

hdr is set from skb_transport_header(skb) at the top of the function,
so it points into skb->head. The segments_left == 0 branch handles an
IPv6 or IPv4 next header. It pulls the outer IPv6 header and the SRH,
resets the network and transport headers, and then calls
iptunnel_pull_offloads(skb). After that call, it still reads
hdr->nexthdr to decide whether skb->protocol becomes ETH_P_IP.

For a GSO skb, iptunnel_pull_offloads() calls skb_unclone(). Suppose
the skb is cloned, for example because a packet tap holds another
reference. Then pskb_expand_head() moves the data to a new head and
drops its reference on the old one. If the other clone has already
been consumed, the old head is freed, and the hdr->nexthdr read is a
use-after-free. That can crash, show up as a KASAN report, or give the
decapsulated packet the wrong inner protocol.

The skb_cloned() check later in ipv6_srh_rcv() only runs when
segments_left is non-zero. Nothing on this path makes sure the skb is
unshared first.

Could hdr->nexthdr be saved in a local before the pull? Or could
skb->protocol be set before iptunnel_pull_offloads() is called?

None of the cases in this selftest use SRv6, so it would not catch
this. Would a seg6 decap case be worth adding?

-- 
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/cover.1792857378.git.julius%40bairaktaris.de

[PATCH net 0/2] net: keep the TCP fixed-ID flags on the header they describe

[Julius Bairaktaris]

GRO sets SKB_GSO_TCP_FIXEDID for the outer IPv4 header of a TCP packet
whose IDs do not change, and SKB_GSO_TCP_FIXEDID_INNER for the inner
header of an encapsulated one, since commit 3271f19bf7b9 ("net: gso:
restore ids of outer ip headers correctly"). Encapsulation,
decapsulation and the GRE and UDP tunnel segmentation paths do not keep
the flags on the header they describe, so a TCP stream that GRO merged
and that crosses a tunnel boundary is segmented with the fixed-ID state
of the wrong header. Sashiko found this in review of an ipmr series:
https://lore.kernel.org/netdev/cover.1792293419.git.julius@bairaktaris.de/

Patch 1 moves the flag on encapsulation and decapsulation and keeps the
encapsulation level below GRE and UDP tunnels. The three changes are one
patch because each alone loses fixed IDs in a case that works today.
Patch 2 adds a selftest.

Tested in QEMU (KASAN) with the selftest, 17 cases, 400 segments each:
- net (7076f684d513): 5 cases fail: fixed inner IDs incremented after
  IPIP over IPv4 and IPv6 (40 distinct inner IDs), outer IDs repeated
  after GRE and VXLAN over IPv4 (10 distinct outer IDs for 400
  segments), and a VXLAN stream with fixed outer IDs whose outer state
  survives decapsulation into IPIP. The same 5 failed in 3 runs of an
  earlier version of the test.
- this series: 17 of 17 pass, 3 of 3 runs.
- with only the decapsulation part, or only the encapsulation and
  segmentation parts: 8 and 5 cases fail, among them every
  encapsulation followed by decapsulation without segmentation, which
  passes on net.
- fou_gre_gro.sh and bareudp.sh pass on net and with the series;
  gre_gso.sh and udpgro_fwd.sh (3 paired runs) show the same failures
  on both, which come from the slow emulated machine.

I used Claude (Anthropic) to help write these patches.

Julius Bairaktaris (2):
  net: keep the TCP fixed-ID flags on the header they describe
  selftests: net: check TCP IP IDs across tunnels after GRO and GSO

 include/linux/skbuff.h                     |  30 ++
 include/net/ip_tunnels.h                   |   2 +
 net/core/filter.c                          |  11 +-
 net/core/lwt_bpf.c                         |   1 +
 net/ipv4/gre_offload.c                     |   1 -
 net/ipv4/ip_tunnel_core.c                  |   5 +-
 net/ipv4/udp_offload.c                     |   1 -
 net/ipv6/exthdrs.c                         |  11 +-
 tools/testing/selftests/net/.gitignore     |   1 +
 tools/testing/selftests/net/Makefile       |   2 +
 tools/testing/selftests/net/tcp_fixedid.c  | 396 +++++++++++++++++++++
 tools/testing/selftests/net/tcp_fixedid.sh | 291 +++++++++++++++
 12 files changed, 744 insertions(+), 8 deletions(-)
 create mode 100644 tools/testing/selftests/net/tcp_fixedid.c
 create mode 100755 tools/testing/selftests/net/tcp_fixedid.sh


base-commit: cb284914e749388eef402938bdb3115fe1f6d337
-- 
2.53.0

[PATCH net 2/2] selftests: net: check TCP IP IDs across tunnels after GRO and GSO

[Julius Bairaktaris]

Send TCP segments with fixed or incrementing IPv4 IDs from a packet
socket, let GRO on a veth merge them, forward the merged packets into
IPIP, GRE and VXLAN tunnels over IPv4 and IPv6 with segmentation done in
software, and check that the inner IDs come out as they went in and the
IDs of a new outer IPv4 header are unique. Also forward a VXLAN stream
with one header's IDs fixed after decapsulation into IPIP, and send the
merged packets through a tunnel that the far end decapsulates without
segmenting them before it forwards them.

A test where GRO merged nothing is skipped.

Signed-off-by: Julius Bairaktaris <julius@bairaktaris.de>
---
 tools/testing/selftests/net/.gitignore     |   1 +
 tools/testing/selftests/net/Makefile       |   2 +
 tools/testing/selftests/net/tcp_fixedid.c  | 396 +++++++++++++++++++++
 tools/testing/selftests/net/tcp_fixedid.sh | 291 +++++++++++++++
 4 files changed, 690 insertions(+)
 create mode 100644 tools/testing/selftests/net/tcp_fixedid.c
 create mode 100755 tools/testing/selftests/net/tcp_fixedid.sh

diff --git a/tools/testing/selftests/net/.gitignore b/tools/testing/selftests/net/.gitignore
index 21412510ac26b..96263d5978258 100644
--- a/tools/testing/selftests/net/.gitignore
+++ b/tools/testing/selftests/net/.gitignore
@@ -46,6 +46,7 @@ so_rcv_listener
 stress_reuseport_listen
 tap
 tcp_fastopen_backup_key
+tcp_fixedid
 tcp_inq
 tcp_mmap
 tcp_port_share
diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile
index 9180b9bec5155..dd402814bc748 100644
--- a/tools/testing/selftests/net/Makefile
+++ b/tools/testing/selftests/net/Makefile
@@ -106,6 +106,7 @@ TEST_PROGS := \
 	srv6_iptunnel_cache.sh \
 	stress_reuseport_listen.sh \
 	tcp_ecmp_failover.sh \
+	tcp_fixedid.sh \
 	tcp_fastopen_backup_key.sh \
 	test_bridge_backup_port.sh \
 	test_bridge_neigh_suppress.sh \
@@ -169,6 +170,7 @@ TEST_GEN_FILES := \
 	socket \
 	stress_reuseport_listen \
 	tcp_fastopen_backup_key \
+	tcp_fixedid \
 	tcp_inq \
 	tcp_mmap \
 	tfo \
diff --git a/tools/testing/selftests/net/tcp_fixedid.c b/tools/testing/selftests/net/tcp_fixedid.c
new file mode 100644
index 0000000000000..d507d12b46308
--- /dev/null
+++ b/tools/testing/selftests/net/tcp_fixedid.c
@@ -0,0 +1,396 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Send TCP segments with fixed or incrementing IPv4 IDs, optionally in
+ * VXLAN, and report the IP IDs of TCP segments received on a device.
+ *
+ * tx: tcp_fixedid -T -i dev -M dmac -s saddr -d daddr [-f] [-n segs]
+ *                 [-b bursts] [-l len] [-V osaddr,odaddr -m inner_dmac [-F]]
+ * rx: tcp_fixedid -R -i dev [-c segs] [-w ms]
+ *
+ * The receiver prints "ready" once it is bound, then the number of
+ * segments, of segments with an outer IPv4 header, and of distinct outer
+ * and inner IDs.
+ */
+
+#include <arpa/inet.h>
+#include <errno.h>
+#include <error.h>
+#include <linux/if_packet.h>
+#include <net/ethernet.h>
+#include <net/if.h>
+#include <netinet/ip.h>
+#include <netinet/ip6.h>
+#include <netinet/tcp.h>
+#include <netinet/udp.h>
+#include <poll.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/socket.h>
+#include <time.h>
+#include <unistd.h>
+
+#define TEST_PORT	9000
+#define VXLAN_PORT	4789
+#define FIXED_ID	0x1234
+
+static const char *ifname;
+static bool fixed_id, fixed_outer;
+static int segs = 40, bursts = 10, len = 1000, wait_ms = 2000, stop;
+static struct in_addr saddr, daddr, osaddr, odaddr;
+static bool vxlan;
+static unsigned char dmac[ETH_ALEN], inner_dmac[ETH_ALEN];
+
+static void parse_mac(const char *s, unsigned char *mac)
+{
+	if (sscanf(s, "%hhx:%hhx:%hhx:%hhx:%hhx:%hhx", &mac[0], &mac[1],
+		   &mac[2], &mac[3], &mac[4], &mac[5]) != 6)
+		error(1, 0, "bad mac %s", s);
+}
+
+static void parse_addr(const char *s, struct in_addr *a)
+{
+	if (inet_pton(AF_INET, s, a) != 1)
+		error(1, 0, "bad address %s", s);
+}
+
+static uint32_t csum_add(uint32_t sum, const void *data, size_t n)
+{
+	const uint8_t *p = data;
+
+	while (n > 1) {
+		sum += (p[0] << 8) | p[1];
+		p += 2;
+		n -= 2;
+	}
+	if (n)
+		sum += p[0] << 8;
+	return sum;
+}
+
+static uint16_t csum_fold(uint32_t sum)
+{
+	while (sum >> 16)
+		sum = (sum & 0xffff) + (sum >> 16);
+	return htons(~sum & 0xffff);
+}
+
+static void build_ip(struct iphdr *iph, int proto, int payload, uint16_t id,
+		     struct in_addr s, struct in_addr d)
+{
+	memset(iph, 0, sizeof(*iph));
+	iph->version = 4;
+	iph->ihl = 5;
+	iph->tot_len = htons(sizeof(*iph) + payload);
+	iph->id = htons(id);
+	iph->frag_off = htons(IP_DF);
+	iph->ttl = 64;
+	iph->protocol = proto;
+	iph->saddr = s.s_addr;
+	iph->daddr = d.s_addr;
+	iph->check = csum_fold(csum_add(0, iph, sizeof(*iph)));
+}
+
+/* Ethernet, IPv4 and TCP header plus payload for segment @i at @p */
+static int build_tcp(unsigned char *p, const unsigned char *mac, int i,
+		     uint16_t id)
+{
+	struct ethhdr *eth = (struct ethhdr *)p;
+	struct iphdr *iph = (struct iphdr *)(eth + 1);
+	struct tcphdr *th = (struct tcphdr *)(iph + 1);
+	struct {
+		uint32_t s, d;
+		uint8_t zero, proto;
+		uint16_t len;
+	} ph;	/* 12 bytes, no padding */
+	uint32_t sum;
+
+	memcpy(eth->h_dest, mac, ETH_ALEN);
+	memset(eth->h_source, 0, ETH_ALEN);
+	eth->h_source[0] = 0x02;
+	eth->h_source[5] = 0x01;
+	eth->h_proto = htons(ETH_P_IP);
+	build_ip(iph, IPPROTO_TCP, sizeof(*th) + len, id, saddr, daddr);
+
+	memset(th, 0, sizeof(*th));
+	th->source = htons(TEST_PORT + 1);
+	th->dest = htons(TEST_PORT);
+	th->seq = htonl(1 + i * len);
+	th->ack_seq = htonl(1);
+	th->doff = 5;
+	th->ack = 1;
+	th->window = htons(65535);
+	memset(th + 1, 'a' + i % 26, len);
+
+	ph.s = iph->saddr;
+	ph.d = iph->daddr;
+	ph.zero = 0;
+	ph.proto = IPPROTO_TCP;
+	ph.len = htons(sizeof(*th) + len);
+	sum = csum_add(0, &ph, sizeof(ph));
+	th->check = csum_fold(csum_add(sum, th, sizeof(*th) + len));
+
+	return sizeof(*eth) + sizeof(*iph) + sizeof(*th) + len;
+}
+
+static int do_tx(int fd, struct sockaddr_ll *sll)
+{
+	unsigned char buf[ETH_FRAME_LEN + 128];
+	int b, s, i = 0, n;
+
+	for (b = 0; b < bursts; b++) {
+		for (s = 0; s < segs; s++, i++) {
+			uint16_t id = fixed_id ? FIXED_ID : 0x1000 + i;
+			unsigned char *p = buf;
+
+			if (vxlan) {
+				struct ethhdr *eth = (struct ethhdr *)p;
+				struct iphdr *iph = (struct iphdr *)(eth + 1);
+				struct udphdr *uh = (struct udphdr *)(iph + 1);
+				uint32_t *vx = (uint32_t *)(uh + 1);
+				int inner;
+
+				inner = build_tcp((unsigned char *)(vx + 2),
+						  inner_dmac, i, id);
+				memcpy(eth->h_dest, dmac, ETH_ALEN);
+				memset(eth->h_source, 0, ETH_ALEN);
+				eth->h_source[0] = 0x02;
+				eth->h_source[5] = 0x01;
+				eth->h_proto = htons(ETH_P_IP);
+				memset(uh, 0, sizeof(*uh));
+				uh->source = htons(VXLAN_PORT + 1);
+				uh->dest = htons(VXLAN_PORT);
+				uh->len = htons(sizeof(*uh) + 8 + inner);
+				vx[0] = htonl(0x08000000);
+				vx[1] = htonl(42 << 8);
+				build_ip(iph, IPPROTO_UDP,
+					 sizeof(*uh) + 8 + inner,
+					 fixed_outer ? FIXED_ID : 0x2000 + i,
+					 osaddr, odaddr);
+				n = sizeof(*eth) + sizeof(*iph) + sizeof(*uh) +
+				    8 + inner;
+			} else {
+				n = build_tcp(p, dmac, i, id);
+			}
+
+			if (sendto(fd, buf, n, 0, (struct sockaddr *)sll,
+				   sizeof(*sll)) != n)
+				error(1, errno, "sendto");
+		}
+		usleep(50000);
+	}
+	return 0;
+}
+
+/* Return the inner IPv4 header of a TCP segment to TEST_PORT, or NULL */
+static const struct iphdr *find_inner(const unsigned char *p, int n,
+				      const struct iphdr **outer4)
+{
+	const struct ethhdr *eth = (const struct ethhdr *)p;
+	const unsigned char *end = p + n;
+	const unsigned char *l4;
+	int proto, depth;
+
+	*outer4 = NULL;
+	if (n < (int)sizeof(*eth))
+		return NULL;
+	p += sizeof(*eth);
+
+	for (depth = 0; depth < 2; depth++) {
+		uint16_t etype = depth ? htons(ETH_P_IP) : eth->h_proto;
+
+		if (etype == htons(ETH_P_IP)) {
+			const struct iphdr *iph = (const struct iphdr *)p;
+
+			if (p + sizeof(*iph) > end)
+				return NULL;
+			proto = iph->protocol;
+			l4 = p + iph->ihl * 4;
+			if (proto == IPPROTO_TCP) {
+				const struct tcphdr *th;
+
+				th = (const struct tcphdr *)l4;
+				if (l4 + sizeof(*th) > end ||
+				    th->dest != htons(TEST_PORT))
+					return NULL;
+				return iph;
+			}
+			if (depth)
+				return NULL;
+			*outer4 = iph;
+		} else if (etype == htons(ETH_P_IPV6)) {
+			const struct ip6_hdr *ip6h = (const struct ip6_hdr *)p;
+
+			if (p + sizeof(*ip6h) > end)
+				return NULL;
+			proto = ip6h->ip6_nxt;
+			l4 = p + sizeof(*ip6h);
+		} else {
+			return NULL;
+		}
+
+		/* step over the tunnel header to the inner IPv4 header */
+		if (proto == IPPROTO_IPIP) {
+			p = l4;
+		} else if (proto == IPPROTO_GRE) {
+			uint16_t flags;
+
+			if (l4 + 4 > end)
+				return NULL;
+			flags = (l4[0] << 8) | l4[1];
+			if (((l4[2] << 8) | l4[3]) != ETH_P_IP)
+				return NULL;
+			p = l4 + 4 + 4 * (!!(flags & 0x8000) +
+					  !!(flags & 0x2000) +
+					  !!(flags & 0x1000));
+		} else if (proto == IPPROTO_UDP) {
+			const struct udphdr *uh = (const struct udphdr *)l4;
+
+			if (l4 + sizeof(*uh) + 8 + sizeof(*eth) > end ||
+			    uh->dest != htons(VXLAN_PORT))
+				return NULL;
+			p = l4 + sizeof(*uh) + 8 + sizeof(*eth);
+		} else {
+			return NULL;
+		}
+	}
+	return NULL;
+}
+
+static void count_id(uint8_t *map, unsigned int *distinct, uint16_t id)
+{
+	if (!(map[id / 8] & (1 << (id % 8)))) {
+		map[id / 8] |= 1 << (id % 8);
+		(*distinct)++;
+	}
+}
+
+static int do_rx(int fd)
+{
+	static uint8_t outer_map[8192], inner_map[8192];
+	unsigned int frames = 0, outer4 = 0, outer_ids = 0, inner_ids = 0;
+	unsigned char buf[65536];
+	struct pollfd pfd = { .fd = fd, .events = POLLIN };
+
+	printf("ready\n");
+	fflush(stdout);
+
+	/* wait longer for the first segment than between segments */
+	while (poll(&pfd, 1, frames ? wait_ms : 10 * wait_ms) > 0) {
+		const struct iphdr *iph, *oiph;
+		struct sockaddr_ll from;
+		socklen_t fl = sizeof(from);
+		int n;
+
+		n = recvfrom(fd, buf, sizeof(buf), 0, (struct sockaddr *)&from,
+			     &fl);
+		if (n < 0)
+			error(1, errno, "recvfrom");
+		if (from.sll_pkttype == PACKET_OUTGOING)
+			continue;
+		iph = find_inner(buf, n, &oiph);
+		if (!iph)
+			continue;
+		frames++;
+		count_id(inner_map, &inner_ids, ntohs(iph->id));
+		if (oiph) {
+			outer4++;
+			count_id(outer_map, &outer_ids, ntohs(oiph->id));
+		}
+		if (stop && frames == (unsigned int)stop)
+			break;
+	}
+
+	printf("frames %u outer4 %u outer_ids %u inner_ids %u\n", frames,
+	       outer4, outer_ids, inner_ids);
+	return 0;
+}
+
+int main(int argc, char **argv)
+{
+	struct sockaddr_ll sll = {};
+	bool tx = false, rx = false;
+	char *comma;
+	int c, fd;
+
+	while ((c = getopt(argc, argv, "TRi:M:s:d:fn:b:l:V:m:Fw:c:")) != -1) {
+		switch (c) {
+		case 'T':
+			tx = true;
+			break;
+		case 'R':
+			rx = true;
+			break;
+		case 'i':
+			ifname = optarg;
+			break;
+		case 'M':
+			parse_mac(optarg, dmac);
+			break;
+		case 's':
+			parse_addr(optarg, &saddr);
+			break;
+		case 'd':
+			parse_addr(optarg, &daddr);
+			break;
+		case 'f':
+			fixed_id = true;
+			break;
+		case 'n':
+			segs = atoi(optarg);
+			break;
+		case 'b':
+			bursts = atoi(optarg);
+			break;
+		case 'l':
+			len = atoi(optarg);
+			break;
+		case 'V':
+			comma = strchr(optarg, ',');
+			if (!comma)
+				error(1, 0, "-V needs outer saddr,daddr");
+			*comma = '\0';
+			parse_addr(optarg, &osaddr);
+			parse_addr(comma + 1, &odaddr);
+			vxlan = true;
+			break;
+		case 'm':
+			parse_mac(optarg, inner_dmac);
+			break;
+		case 'F':
+			fixed_outer = true;
+			break;
+		case 'w':
+			wait_ms = atoi(optarg);
+			break;
+		case 'c':
+			stop = atoi(optarg);
+			break;
+		default:
+			error(1, 0, "bad option");
+		}
+	}
+	if (tx == rx || !ifname || len < 1 || len > 1400 || segs < 1 ||
+	    bursts < 1)
+		error(1, 0, "usage: see the comment at the top of %s",
+		      __FILE__);
+
+	fd = socket(AF_PACKET, SOCK_RAW, htons(tx ? 0 : ETH_P_ALL));
+	if (fd < 0)
+		error(1, errno, "socket");
+	sll.sll_family = AF_PACKET;
+	sll.sll_protocol = htons(ETH_P_ALL);
+	sll.sll_ifindex = if_nametoindex(ifname);
+	if (!sll.sll_ifindex)
+		error(1, errno, "%s", ifname);
+	if (bind(fd, (struct sockaddr *)&sll, sizeof(sll)))
+		error(1, errno, "bind");
+	sll.sll_halen = ETH_ALEN;
+	memcpy(sll.sll_addr, dmac, ETH_ALEN);
+
+	c = tx ? do_tx(fd, &sll) : do_rx(fd);
+	close(fd);
+	return c;
+}
diff --git a/tools/testing/selftests/net/tcp_fixedid.sh b/tools/testing/selftests/net/tcp_fixedid.sh
new file mode 100755
index 0000000000000..88f9298174077
--- /dev/null
+++ b/tools/testing/selftests/net/tcp_fixedid.sh
@@ -0,0 +1,291 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Check the IPv4 IDs of a TCP stream that GRO merged and software
+# segmentation split again across a tunnel. A sender in NSRC writes
+# segments with fixed or incrementing IDs onto a veth with GRO on in
+# NRTR, which forwards the merged packets:
+#
+#  encap:       into an IPIP, GRE or VXLAN tunnel over IPv4 or IPv6,
+#               segmented on the underlay towards NMID
+#  decap:       out of VXLAN, received from NSRC with one header fixed and
+#               the other incrementing, into an IPIP tunnel towards NMID
+#  encap+decap: into a tunnel that NMID ends without segmenting, and on
+#               to NDST, segmented there
+#
+# The inner IDs must come out as they went in, and the IDs of a new outer
+# IPv4 header must be unique. Packets that GRO did not merge are not
+# tested, so a run where nothing was merged is skipped.
+
+source lib.sh
+
+NSRC=""
+NRTR=""
+NMID=""
+NDST=""
+
+SEGS=40
+BURSTS=10
+TOTAL=$((SEGS * BURSTS))
+INNER_SRC=10.1.0.1
+INNER_DST=10.9.0.2
+VX_MAC=02:00:00:00:00:42
+TUN_MAC=02:00:00:00:00:99
+MID_MAC=02:00:00:00:00:98
+TX=./tcp_fixedid
+
+NOSEG="tso gso tx-gre-segmentation tx-gre-csum-segmentation
+       tx-ipxip4-segmentation tx-ipxip6-segmentation
+       tx-udp_tnl-segmentation tx-udp_tnl-csum-segmentation"
+
+cleanup() {
+	cleanup_all_ns
+}
+
+trap cleanup EXIT
+
+# NSRC s0 - r0 NRTR r1 - m1 NMID m2 - d2 NDST
+setup() {
+	setup_ns NSRC NRTR NMID NDST || return "$ksft_skip"
+
+	ip link add s0 netns "$NSRC" type veth peer name r0 netns "$NRTR"
+	ip link add r1 netns "$NRTR" type veth peer name m1 netns "$NMID"
+	ip link add m2 netns "$NMID" type veth peer name d2 netns "$NDST"
+
+	ip -n "$NRTR" addr add 10.1.0.2/24 dev r0
+	ip -n "$NRTR" addr add 10.2.0.1/24 dev r1
+	ip -n "$NRTR" addr add fd02::1/64 dev r1 nodad
+	ip -n "$NMID" addr add 10.2.0.2/24 dev m1
+	ip -n "$NMID" addr add fd02::2/64 dev m1 nodad
+	ip -n "$NMID" addr add 10.4.0.1/24 dev m2
+	ip -n "$NDST" addr add 10.4.0.2/24 dev d2
+	ip -n "$NSRC" link set s0 up
+	ip -n "$NRTR" link set r0 up
+	ip -n "$NRTR" link set r1 up
+	ip -n "$NMID" link set m1 up
+	ip -n "$NMID" link set m2 up
+	ip -n "$NDST" link set d2 up
+
+	ip netns exec "$NRTR" sysctl -qw net.ipv4.ip_forward=1
+	ip netns exec "$NMID" sysctl -qw net.ipv4.ip_forward=1
+	for ns in "$NRTR" "$NMID"; do
+		ip netns exec "$ns" sysctl -qw net.ipv4.conf.all.rp_filter=0 \
+			net.ipv4.conf.default.rp_filter=0
+	done
+
+	# veth passes packets to GRO only if the sender does not do TSO
+	ip netns exec "$NRTR" ethtool -K r0 gro on >/dev/null ||
+		return "$ksft_skip"
+	ip netns exec "$NSRC" ethtool -K s0 tso off >/dev/null ||
+		return "$ksft_skip"
+	# release the sender's segments in bursts for GRO to merge
+	tc -n "$NSRC" qdisc add dev s0 root netem slot 20ms 20ms \
+		packets "$SEGS" || return "$ksft_skip"
+}
+
+# tunnel <ns> <name> <type> <family> <local> <remote>
+tunnel() {
+	local ns=$1 name=$2 type=$3 family=$4 local=$5 remote=$6
+
+	case "$type" in
+	ipip)
+		if [ "$family" = 4 ]; then
+			ip -n "$ns" link add "$name" type ipip \
+				local "$local" remote "$remote"
+		else
+			ip -n "$ns" link add "$name" type ip6tnl mode ipip6 \
+				local "$local" remote "$remote" encaplimit none
+		fi ;;
+	gre)
+		if [ "$family" = 4 ]; then
+			ip -n "$ns" link add "$name" type gre \
+				local "$local" remote "$remote"
+		else
+			ip -n "$ns" link add "$name" type ip6gre \
+				local "$local" remote "$remote" encaplimit none
+		fi ;;
+	vxlan)
+		ip -n "$ns" link add "$name" address "$TUN_MAC" type vxlan \
+			id 7 dstport 4789 local "$local" remote "$remote" \
+			nolearning ;;
+	esac || return 1
+	ip -n "$ns" link set "$name" up
+}
+
+# route_into <ns> <dev>: send INNER_DST into <dev>
+route_into() {
+	ip -n "$1" route add "$INNER_DST/32" dev "$2"
+	ip -n "$1" neigh add "$INNER_DST" lladdr "$TUN_MAC" dev "$2" \
+		2>/dev/null
+	return 0
+}
+
+# noseg <ns> <dev>: segment everything in software before <dev>
+noseg() {
+	local f
+
+	for f in $NOSEG; do
+		ip netns exec "$1" ethtool -K "$2" "$f" off >/dev/null 2>&1
+	done
+}
+
+tx_packets() {
+	ip netns exec "$1" cat "/sys/class/net/$2/statistics/tx_packets"
+}
+
+# check <name> <result> <want: fixed|incr> <outer: 0|1> <merged-into-dev>
+check() {
+	local name=$1 res=$2 want=$3 outer=$4 sent=$5
+	local frames outer_ids inner_ids
+
+	read -r _ frames _ _ _ outer_ids _ inner_ids <<< "$res"
+	RET=0
+	check_err $((frames != TOTAL)) "$frames of $TOTAL segments received"
+	if [ "$want" = fixed ]; then
+		check_err $((inner_ids != 1)) "$inner_ids inner IDs, want 1"
+	else
+		check_err $((inner_ids != TOTAL)) \
+			"$inner_ids inner IDs, want $TOTAL"
+	fi
+	if [ "$outer" = 1 ]; then
+		check_err $((outer_ids != frames)) \
+			"$outer_ids outer IDs for $frames segments"
+	fi
+	if [ "$RET" -eq 0 ] && [ "$sent" -ge "$TOTAL" ]; then
+		ret_set_ksft_status "$ksft_skip" "GRO did not merge"
+	fi
+	log_test "$name"
+}
+
+# capture <ns> <dev>: start the receiver in the background, pid in CAP_PID,
+# and wait until it is bound
+capture() {
+	local i
+
+	: > "$RESULT"
+	ip netns exec "$1" "$TX" -R -i "$2" -c "$TOTAL" -w 3000 > "$RESULT" &
+	CAP_PID=$!
+	for i in $(seq 100); do
+		grep -q ready "$RESULT" && return
+		sleep 0.1
+	done
+}
+
+run_encap() {
+	local type=$1 family=$2 want=$3
+	local r0mac t0 laddr=10.2.0.1 raddr=10.2.0.2
+	local f=""
+
+	setup || { log_test_skip "encap $type IPv$family"; return; }
+	if [ "$family" = 6 ]; then
+		laddr=fd02::1
+		raddr=fd02::2
+	fi
+	tunnel "$NRTR" tun0 "$type" "$family" "$laddr" "$raddr" ||
+		{ log_test_skip "encap $type IPv$family"; cleanup_all_ns; return; }
+	route_into "$NRTR" tun0
+	noseg "$NRTR" r1
+
+	r0mac=$(ip -n "$NRTR" -br link show r0 | awk '{ print $3 }')
+	capture "$NMID" m1
+	t0=$(tx_packets "$NRTR" tun0)
+	[ "$want" = fixed ] && f=-f
+	ip netns exec "$NSRC" "$TX" -T -i s0 -M "$r0mac" -s "$INNER_SRC" \
+		-d "$INNER_DST" -n "$SEGS" -b "$BURSTS" $f
+	wait "$CAP_PID"
+	check "encap $type IPv$family, $want inner IDs" "$(tail -n 1 "$RESULT")" \
+		"$want" $((family == 4)) $(($(tx_packets "$NRTR" tun0) - t0))
+	cleanup_all_ns
+}
+
+# run_decap <fixed: inner|outer>: VXLAN from NSRC, decapsulated in NRTR
+run_decap() {
+	local fixed=$1 want=incr flags=-F
+	local r0mac t0
+
+	setup || { log_test_skip "decap"; return; }
+	ip -n "$NRTR" link add vx0 address "$VX_MAC" type vxlan id 42 \
+		dstport 4789 local 10.1.0.2 nolearning ||
+		{ log_test_skip "decap"; cleanup_all_ns; return; }
+	ip -n "$NRTR" addr add 10.3.0.2/24 dev vx0
+	ip -n "$NRTR" link set vx0 up
+	tunnel "$NRTR" tun0 ipip 4 10.2.0.1 10.2.0.2
+	route_into "$NRTR" tun0
+	noseg "$NRTR" r1
+
+	if [ "$fixed" = inner ]; then
+		want=fixed
+		flags=-f
+	fi
+	r0mac=$(ip -n "$NRTR" -br link show r0 | awk '{ print $3 }')
+	capture "$NMID" m1
+	t0=$(tx_packets "$NRTR" tun0)
+	ip netns exec "$NSRC" "$TX" -T -i s0 -M "$r0mac" -s 10.3.0.1 \
+		-d "$INNER_DST" -n "$SEGS" -b "$BURSTS" \
+		-V 10.1.0.1,10.1.0.2 -m "$VX_MAC" $flags
+	wait "$CAP_PID"
+	check "decap VXLAN with fixed $fixed IDs, into IPIP" \
+		"$(tail -n 1 "$RESULT")" "$want" 1 \
+		$(($(tx_packets "$NRTR" tun0) - t0))
+	cleanup_all_ns
+}
+
+# run_encap_decap <type>: NRTR encapsulates, NMID decapsulates the GSO
+# packet without segmenting it and forwards it to NDST
+run_encap_decap() {
+	local type=$1
+	local r0mac t0
+
+	setup || { log_test_skip "encap+decap $type"; return; }
+	if ! tunnel "$NRTR" tun0 "$type" 4 10.2.0.1 10.2.0.2 ||
+	   ! tunnel "$NMID" tun0 "$type" 4 10.2.0.2 10.2.0.1; then
+		log_test_skip "encap+decap $type"
+		cleanup_all_ns
+		return
+	fi
+	route_into "$NRTR" tun0
+	# VXLAN drops frames from its own address as loops
+	if [ "$type" = vxlan ]; then
+		ip -n "$NMID" link set tun0 address "$MID_MAC"
+		ip -n "$NRTR" neigh replace "$INNER_DST" lladdr "$MID_MAC" \
+			dev tun0
+	fi
+	ip -n "$NMID" route add "$INNER_DST/32" via 10.4.0.2
+	ip -n "$NDST" addr add "$INNER_DST/32" dev d2
+	noseg "$NMID" m2
+
+	r0mac=$(ip -n "$NRTR" -br link show r0 | awk '{ print $3 }')
+	capture "$NDST" d2
+	t0=$(tx_packets "$NRTR" tun0)
+	ip netns exec "$NSRC" "$TX" -T -i s0 -M "$r0mac" -s "$INNER_SRC" \
+		-d "$INNER_DST" -n "$SEGS" -b "$BURSTS" -f
+	wait "$CAP_PID"
+	check "encap+decap $type, fixed inner IDs" "$(tail -n 1 "$RESULT")" fixed 0 \
+		$(($(tx_packets "$NRTR" tun0) - t0))
+	cleanup_all_ns
+}
+
+require_command ethtool
+require_command tc
+
+if [ ! -x "$TX" ]; then
+	echo "SKIP: $TX not built"
+	exit "$ksft_skip"
+fi
+
+RESULT=$(mktemp)
+
+for type in ipip gre vxlan; do
+	for family in 4 6; do
+		run_encap "$type" "$family" fixed
+		run_encap "$type" "$family" incr
+	done
+done
+run_decap inner
+run_decap outer
+for type in ipip gre vxlan; do
+	run_encap_decap "$type"
+done
+
+rm -f "$RESULT"
+exit "$EXIT_STATUS"
-- 
2.53.0