[PATCH ipsec] xfrm: use software GSO for inner tunnels without crypto offload

[PATCH ipsec] xfrm: use software GSO for inner tunnels without crypto offload

[Usman Malik via B4 Relay]

From: Usman Malik <usmanmalik@engineer.com>

Commit 0aae2867aa60 ("xfrm_output: Force software GSO only in tunnel
mode") limits the software GSO of packets with skb->inner_protocol set
to tunnel mode. Its message says that packets with the crypto in
software still get software GSO in the else branch. That holds only
while no ESP GSO handler is registered. With esp4_offload or
esp6_offload loaded when the SA is created, x->type_offload is set and
xfrm_dev_offload_ok() returns true for an SA without an offload device.
A transport-mode SA with software crypto then sends such a packet down
the SKB_GSO_ESP path.

That path does not handle an inner IP tunnel. For an ipip or ip6tnl
packet, xfrm{4,6}_transport_gso_segment() pass the packet to
inet_gso_segment(), which leaves the transport header of each segment at
the inner TCP header. __xfrm_transport_prep() expects it right after the
ESP header and IV, so esp_xmit() and esp6_xmit() write the ESP header at
offset 4 of the inner IPv4 header and encrypt from the TCP header. The
first 4 bytes of the inner IPv4 header go out in clear, the receiver
drops the segment with XfrmInStateProtoError, and bulk TCP through the
tunnel falls to tens of Mbit/s. The v6.13 release does not show this,
v6.14 does, and so do the stable kernels that carry 0aae2867aa60
(measured on 5.4.293, 5.15.223 and 6.1.190).

Use software GSO for packets with skb->inner_protocol set when the SA
has no offload device, as before 0aae2867aa60. An SA with crypto offload
on a device keeps the SKB_GSO_ESP path in transport mode, which is what
that commit is for.

The transport GSO handlers themselves still do not handle an inner IP
tunnel. An SA with crypto offload that falls back to software
segmentation in validate_xmit_xfrm() takes the same path. That is a
separate change in xfrm{4,6}_transport_gso_segment() and is not part of
this patch, which restores the behaviour before 0aae2867aa60 for SAs
without an offload device.

Tested on v7.3 in a VM with two network namespaces on a veth, a
rfc4106(gcm(aes)) SA between the outer addresses, esp4_offload and
esp6_offload loaded, TSO on and 3 MB of bulk TCP per run, plus a 4 s
iperf3 run. Both test kernels also carried a local esp6_xmit() change
that builds the IV from the sequence number read before the increment,
as esp_xmit() does, so that the capture analysis could check nonces as
well. That change does not touch the path changed here. The sender
capture was decrypted with the SA keys. Without this patch, two ip6tnl
runs had 22 and 24 malformed packets at 36 Mbit/s, two ipip runs 18 and
36 at about 50 Mbit/s, and iperf3 over ip6tnl gave 195 Mbit/s with
XfrmInStateProtoError +358. With it, every ESP packet decrypts at offset
0, no inner header byte is in clear, XfrmInStateProtoError stays at +0,
the same runs reach 1.0 to 2.4 Gbit/s and iperf3 gives 2.62 Gbit/s. A
function profile shows these packets now go through xfrm_output_gso(),
while plain TCP over the same SA still goes through esp6_gso_segment().
vxlan and geneve over a transport-mode and a tunnel-mode SA were clean
before and after. In transport mode they now take the software GSO path
as well. Hardware ESP offload was not tested, as no such NIC was
available, and neither were BEET and IP-TFS mode, nor GRE over a
transport-mode SA, which this change also moves to software GSO.

The bug was found while measuring bulk TCP throughput through an ip6tnl
tunnel over a transport-mode ESP SA.

Fixes: 0aae2867aa60 ("xfrm_output: Force software GSO only in tunnel mode")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Usman Malik <usmanmalik@engineer.com>
---
The malformed layout is visible without keys. On the sender,

  tcpdump -ni va 'ip6 proto 50 and ip6[56:2] = 0x4500 and ip6[60:4] = 0x1001'

lists exactly the malformed packets (22 of 22 in one v7.3 run, 0 with
this patch). For ipip the offsets are ip[36:2] and ip[40:4].

Reproducer for the ip6tnl case. It prints the sender rate and the
receiver's XfrmInStateProtoError delta with TSO on and off. Run as root
with iproute2, ethtool and iperf3. Rerun after "ip netns del a" and "ip
netns del b" if it stops early. On a plain v7.3 it printed 107 Mbit/s
and +366 with TSO on, 2.72 Gbit/s and +0 with TSO off. With this patch
it printed 2.62 Gbit/s and +0 with TSO on.

  #!/bin/sh
  set -e
  modprobe -a esp4_offload esp6_offload
  A="ip netns exec a"; B="ip netns exec b"
  ip netns add a; ip netns add b
  ip link add va type veth peer name vb
  ip link set va netns a; ip link set vb netns b
  $A ip link set lo up; $B ip link set lo up
  $A ip link set dev va mtu 9000 up; $B ip link set dev vb mtu 9000 up
  $A ip -6 addr add fd00:a::1/64 dev va nodad
  $B ip -6 addr add fd00:a::2/64 dev vb nodad
  $A ip -6 tunnel add t mode any local fd00:a::1 remote fd00:a::2 \
     encaplimit none hoplimit 64
  $B ip -6 tunnel add t mode any local fd00:a::2 remote fd00:a::1 \
     encaplimit none hoplimit 64
  $A ip link set dev t mtu 8926 up; $B ip link set dev t mtu 8926 up
  $A ip addr add 10.9.0.1/30 dev t; $B ip addr add 10.9.0.2/30 dev t
  key() { head -c 36 /dev/urandom | od -An -tx1 | tr -d ' \n'; }
  K1=0x$(key); K2=0x$(key)
  for n in a b; do
    ip netns exec $n ip xfrm state add src fd00:a::1 dst fd00:a::2 \
      proto esp spi 0x1001 reqid 1 mode transport replay-window 128 \
      aead 'rfc4106(gcm(aes))' $K1 128
    ip netns exec $n ip xfrm state add src fd00:a::2 dst fd00:a::1 \
      proto esp spi 0x1002 reqid 2 mode transport replay-window 128 \
      aead 'rfc4106(gcm(aes))' $K2 128
  done
  $A ip xfrm policy add src fd00:a::1/128 dst fd00:a::2/128 dir out \
     tmpl proto esp reqid 1 mode transport
  $A ip xfrm policy add src fd00:a::2/128 dst fd00:a::1/128 dir in \
     tmpl proto esp reqid 2 mode transport
  $B ip xfrm policy add src fd00:a::2/128 dst fd00:a::1/128 dir out \
     tmpl proto esp reqid 2 mode transport
  $B ip xfrm policy add src fd00:a::1/128 dst fd00:a::2/128 dir in \
     tmpl proto esp reqid 1 mode transport
  $B iperf3 -s -D
  err() { $B awk '$1=="XfrmInStateProtoError"{print $2}' \
          /proc/net/xfrm_stat; }
  run() { e0=$(err)
    $A iperf3 -c 10.9.0.2 -t 4 | awk '/sender/{print $7,$8,"retr",$9}'
    echo "XfrmInStateProtoError +$(( $(err) - e0 ))"; }
  echo "tso on";  run
  $A ethtool -K t tso off
  echo "tso off"; run
  pkill iperf3; ip netns del a; ip netns del b

The esp6_xmit() change in the test kernels does what Jeremy Jean's
"[PATCH ipsec 1/7] xfrm: esp6: use the current sequence number for the
IV" does, except that esp.seqno is built after the seq.hi update, as in
esp_xmit(). The two differ only at a seq.low wrap. This patch was not
run on a v7.3 without that change.
https://lore.kernel.org/netdev/20260930144523.435271-3-Jeremy.Jean@oss.cyber.gouv.fr/

Jeremy Jean's "[PATCH ipsec 6/7] xfrm: segment untrusted GSO packets
before sequence allocation", not applied yet, changes the same condition
in xfrm_output() and moves the skb->encapsulation line above it, so
whichever lands second needs a rebase. 6/7 adds SKB_GSO_DODGY packets to
the software GSO case. The packets here come from a local TCP socket and
are not DODGY, so 6/7 does not cover this case.
https://lore.kernel.org/netdev/20260930144523.435271-8-Jeremy.Jean@oss.cyber.gouv.fr/
---
 net/xfrm/xfrm_output.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
index e305ba32e..c370d28a8 100644
--- a/net/xfrm/xfrm_output.c
+++ b/net/xfrm/xfrm_output.c
@@ -808,7 +808,8 @@ int xfrm_output(struct sock *sk, struct sk_buff *skb)
 		skb->encapsulation = 1;
 
 		if (skb_is_gso(skb)) {
-			if (skb->inner_protocol && x->props.mode == XFRM_MODE_TUNNEL)
+			if (skb->inner_protocol &&
+			    (x->props.mode == XFRM_MODE_TUNNEL || !x->xso.dev))
 				return xfrm_output_gso(net, sk, skb);
 
 			skb_shinfo(skb)->gso_type |= SKB_GSO_ESP;

---
base-commit: 835912998ca54a76b5c57f8c8459a86b1b45de40
change-id: 20261027-xfrm-inner-gso-f8b956ce89d5

Best regards,
--  
Usman Malik <usmanmalik@engineer.com>

Re: [PATCH ipsec] xfrm: use software GSO for inner tunnels without crypto offload

[netdev-bot+sashiko]

Thank you for your contribution! Sashiko AI review found 1 potential
issue(s) to consider.

Critical: 1 · High: 0 · Medium: 0 (1) · Low: 0 (1)

If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).

- [Critical] The patch sends more traffic into xfrm_output_gso(), and on
  that path AES-GCM IVs can repeat.

Pre-existing issues:
- [Medium] This problem wasn't introduced by this patch, but transport-
  mode SAs that have a crypto offload device (x->xso.dev != NULL) still…
- [Low] This problem wasn't introduced by this patch, but xfrm_output()
  reads x->xso.dev with plain, unmarked loads: in xfrm_dev_offload_ok()…

--

xfrm: use software GSO for inner tunnels without crypto offload

This commit changes xfrm_output() to use xfrm_output_gso() for GSO
packets with skb->inner_protocol set. Before, that only happened in
tunnel mode. Now it also happens when the SA has no offload device. The
goal is to stop the SKB_GSO_ESP path from producing malformed ESP packets
for ipip/ip6tnl traffic over transport-mode software SAs when
esp4_offload/esp6_offload are loaded.

> diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
> index e305ba32e356b..c370d28a83f50 100644
> --- a/net/xfrm/xfrm_output.c
> +++ b/net/xfrm/xfrm_output.c
> @@ -808,7 +808,8 @@ int xfrm_output(struct sock *sk, struct sk_buff *skb)
>  		skb->encapsulation = 1;
>  
>  		if (skb_is_gso(skb)) {
> -			if (skb->inner_protocol && x->props.mode == XFRM_MODE_TUNNEL)
> +			if (skb->inner_protocol &&
> +			    (x->props.mode == XFRM_MODE_TUNNEL || !x->xso.dev))
>  				return xfrm_output_gso(net, sk, skb);

[Severity: Critical]
Can this cause AES-GCM IV reuse for the transport-mode software SA
traffic that this change sends to xfrm_output_gso()?

Before this point, xfrm_output() has attached a sec_path with
sp->olen = 1, so the skb carries an xfrm_offload. xfrm_output_gso() then
calls skb_gso_segment(). In skb_segment()->__copy_skb_header(),
__skb_ext_copy() only takes a reference on the extension:

include/linux/skbuff.h:__skb_ext_copy() {
	...
		refcount_inc(&ext->refcnt);
		dst->extensions = ext;
	...
}

So all segments seem to share one struct xfrm_offload.

For each segment, xfrm_output2()->...->xfrm_replay_overflow_offload()
writes to that shared xo:

net/xfrm/xfrm_replay.c:xfrm_replay_overflow_offload() {
	...
		if (!skb_is_gso(skb)) {
			XFRM_SKB_CB(skb)->seq.output.low = ++oseq;
			xo->seq.low = oseq;
	...
}

The XFRM_SKB_CB() copy belongs to each skb, but xo->seq.low is in the
shared extension.

Encryption happens later, at qdisc dequeue time, in
validate_xmit_skb()->validate_xmit_xfrm()->esp_xmit(). That code reads
the shared value:

net/ipv4/esp4_offload.c:esp_xmit() {
	...
	seq = xo->seq.low;
	...
	esp.seqno = cpu_to_be64(seq + ((u64)xo->seq.hi << 32));
	...
}

XFRM_GSO_SEGMENT is not set on this path, so nothing advances
xo->seq.low per segment. esp_output_tail() then copies esp->seqno into
the AEAD IV. esp6_xmit() does the same.

Segments can wait in the qdisc while the xfrm_output_gso() loop keeps
running. For example, dequeue_skb() returns nothing while a
TCQ_F_ONETXQUEUE txq is stopped by BQL or a full ring, or another CPU
may already be running the qdisc. In that case, won't later segments
overwrite xo->seq.low before the earlier ones are encrypted?

Each packet would still have a distinct esph->seq_no on the wire, but
the explicit IVs would be the same. That is GCM nonce reuse under one
key. The receiver takes the IV from the packet, so decryption still
succeeds and nothing reports the problem.

veth is noqueue, so in the test setup validate_xmit_skb() runs before
the next segment's xfrm_output_one(). That would hide this.

Tunnel-mode software SAs already use this mechanism today. In the base
tree, however, these transport-mode packets, including the vxlan and
geneve traffic that worked, went down the SKB_GSO_ESP path. There,
xo->seq.low advances per segment within one validate_xmit_xfrm() call,
on an extension owned by a single GSO skb.

Could the sec_path be unshared per segment, or the IV be taken from the
per-skb XFRM_SKB_CB() sequence? Another option would be to fix
xfrm{4,6}_transport_gso_segment() instead of rerouting these packets.

[Severity: Low]
This is a pre-existing issue, but x->xso.dev is read here with a plain
load, and xfrm_dev_state_free() can clear it at the same time:

net/xfrm/xfrm_state.c:xfrm_dev_state_free() {
	...
		WRITE_ONCE(xso->dev, NULL);
		xso->type = XFRM_DEV_OFFLOAD_UNSPECIFIED;
		netdev_put(dev, &xso->dev_tracker);
	...
}

The new !x->xso.dev test is only used as a boolean, and either value
leads to a valid path. At worst it is a data race that KCSAN can see,
like the one already in xfrm_dev_offload_ok():

	struct net_device *dev = x->xso.dev;

Further down in xfrm_output(), the check-then-dereference also uses two
plain loads. This patch does not touch it:

	if (x->xso.dev && x->xso.dev->features & NETIF_F_HW_ESP_TX_CSUM)

Would it make sense to read x->xso.dev once with READ_ONCE() into a
local variable and use that for these checks?

>  
>  			skb_shinfo(skb)->gso_type |= SKB_GSO_ESP;

[Severity: Medium]
This isn't a bug introduced by this patch, but transport-mode SAs with
a crypto offload device (x->xso.dev != NULL) still set SKB_GSO_ESP here
for packets with skb->inner_protocol set.

The commit message says:

  "An SA with crypto offload that falls back to software
  segmentation in validate_xmit_xfrm() takes the same path."

Is the reroute/overflow fallback in validate_xmit_xfrm() the only way to
get there? validate_xmit_skb() also does this:

net/core/dev.c:validate_xmit_skb() {
	...
	if (netif_needs_gso(skb, features)) {
		struct sk_buff *segs;

		segs = skb_gso_segment(skb, features);
	...
}

That runs whenever the device's hw_enc_features don't cover the whole
gso_type, for example SKB_GSO_ESP | SKB_GSO_IPXIP4. The call chain would
then be:

skb_gso_segment()
  esp4_gso_segment()
    xfrm4_transport_gso_segment()
      inet_gso_segment()   /* inner IPIP, transport header at inner L4 */

followed by validate_xmit_xfrm()->__xfrm_transport_prep()->esp_xmit().
That writes the ESP header 4 bytes into the inner IP header, which is
the same malformed layout the commit message describes.
xfrm6_transport_gso_segment() has the same limitation.

Could the commit message mention this fallback as well?

-- 
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261028-xfrm-inner-gso-v1-1-1908f0eb7313%40engineer.com