* [PATCH net v8 0/4] net: hsr: fix super-packet forwarding
@ 2026-11-07 22:05 Xin Xie
2026-11-07 22:05 ` [PATCH net v8 1/4] net: hsr: keep GRO disabled on HSR/PRP ports Xin Xie
` (3 more replies)
0 siblings, 4 replies; 10+ messages in thread
From: Xin Xie @ 2026-11-07 22:05 UTC (permalink / raw)
To: netdev, linux-kselftest, linux-kernel
Cc: davem, edumazet, kuba, pabeni, horms, andrew+netdev, shuah, kees,
petr.wozniak, qingfang.deng, fmaurer, luka.gejak, bigeasy,
xiaoliang.yang_1, skhawaja, liuhangbin, stable, sdf.kernel,
xiexinet, lukma, fmancera, danishanwar
HSR/PRP requires per-wire-frame tags/RCTs and sequence numbers, and
duplicate discard is per frame. RX GRO and TX GSO can present multiple
frames as one skb and violate that assumption: a super-skb is either
rejected by a constrained lower device, or forwarded without valid
per-frame trailers and sequence numbers. An oversized PRP aggregate
can also truncate the RCT's 12-bit LSDU size field.
Patch 1 keeps software GRO off while devices are direct HSR/PRP
members, in the requested as well as in the active features. Fixed-on
or driver-required GRO_HW may remain enabled.
Patch 2 refuses bridge, bond, team, OVS, macvlan and MACsec devices,
and devices without an RX handler such as ipvlan and VRF, as HSR/PRP
ports. This closes the syzbot report of recursive seqnr_lock locking
through a bridge whose port is a second HSR master. seqnr_lock stays as
it is.
Patch 3 segments valid GSO before per-frame processing, so each wire
frame gets its own tag/RCT and sequence number; segmentation runs
outside seqnr_lock. Patch 4 tests member GRO policy, port refusal, GSO
forwarding and local submission order.
A typical GSO source is a container connected to a PRP RedBox
interlink through veth. An existing TCP/UDP GSO skb can reach the
interlink with GRO disabled:
container/netns host PRP RedBox
prp0-peer ---- veth ---- prp0-int (interlink RX)
|
split GSO into frames
|
sequence number + RCT
/ \
LAN A LAN B
Compatibility with older PRP senders:
Valid PRP frames from older senders remain supported. Some old Linux
PRP senders incorrectly append one RCT to an entire GSO skb. This
violates the per-frame PRP requirement. Patch 3 drops such intact
aggregates when trailing bytes extend past a known, nonzero IP length;
zero lengths and GSO_PARTIAL are not covered by that check.
On virtio-net/TAP paths, disabling GRO_HW in patch 1 can make the host
segment an old sender's malformed aggregate before it reaches the
receiving VM. The RCT then becomes transport payload, which the
receiver cannot reliably identify or remove. Upgrade the sender or
disable GSO/TSO on its HSR/PRP master.
Validation:
On a lockdep kernel with the series, the GRO selftest passed 15/15
and the ordered selftest 5/5. The syzbot reproducer reports recursive
locking on the base kernel and nothing on the series in five minutes.
It needed one adjusted ifindex, because the caif device it counts on
no longer exists. Down/up of an attached veth and detach of a veth
attached while down hang or hit BUG_ON with v7 and pass with this
series.
W=1 allmodconfig and allyesconfig builds show no new warnings against
the base.
Based on net fafbc4051123 (2026-11-05). The series applies without
conflicts to net 30c71476d842 and net-next fa5196392691, where net/hsr
is identical. The net-next conflicts reported for v7 came from the
sequence-number changes in v7's patch 2, which this version drops.
No stable backport is requested here. The remaining HSR dev->stats
races are left to a separate series, as Paolo suggested [1].
Pre-existing shared-skb mutations are also handled separately.
[1] https://lore.kernel.org/netdev/4fc3b9f1-4bef-4b34-ae7a-e89037cce829@redhat.com/
Changes since v7:
- Patch 1: also clear GRO and configurable GRO_HW from wanted_features.
veth adds and deletes its NAPI instances according to the request, so
the v7 split between requested and active GRO could hang
"ip link set down" or hit the BUG_ON() in napi_enable() (the critical
NIPA finding on patch 1). GRO is no longer restored when a port leaves,
as for LRO; the comments only claim GRO off for direct members.
- Patch 2: replace the ordered forwarding consumer with a check of the
port type, as Sebastian suggested [2]; seqnr_lock is unchanged. The
rule is by device type rather than "no lower devices" as I wrote in
the thread, because DSA user ports are uppers of their conduit.
Dropping the consumer also drops the review points on its admission
limit, supervision budget and PREEMPT_RT behavior. Narrowing
seqnr_lock is left to a separate net-next change, where Hangbin's
patch fits [3]. Ports whose RX handler is in use or not supported are
refused before any state change too, so a refused device keeps its
GRO request.
- Patch 3: segment before taking seqnr_lock and number each segment
under it; the lock moves from the callers into the per-frame helper.
A still-shared GSO input is unshared before segmentation. New gate
drops are counted once on the entry device.
- Patch 4: the GRO checks follow the new policy, including the
requested state. New cases cover down/up while attached, attach while
down then up and detach, a request while attached, and port refusal.
From the NIPA review: CONFIG_MACVLAN and CONFIG_BONDING in config,
captures scoped to their case, read-back instead of ethtool output
matching, PRP trailers decoded only on PRP LANs with the kernel's
fallback, capture sockets bound before they receive, producer errors
not hidden by the barrier, INT/TERM blocked while namespaces or
handlers switch, completion on the expected frames rather than a quiet
period, and only the common prefix compared where supervision runs
through the lane cutoff. Unused code is removed. The helper stays
standalone instead of using net/lib/py, which has no in-process,
namespace-bound AF_PACKET capture or stop-after-failure TAP handling.
The remaining low-severity remarks are addressed or went away with
the consumer.
[2] https://lore.kernel.org/netdev/20261014151838.6RyMIRVM@linutronix.de/
[3] https://lore.kernel.org/netdev/20260805-hsr_deadlock-v1-1-d8a6f06fc5a8@kylinos.cn/
Previous postings (earlier design, newest first):
v7: https://lore.kernel.org/netdev/20261009201324.17-1-xiexinet@gmail.com/
v6: https://lore.kernel.org/netdev/20260809121455.1745-1-xiexinet@gmail.com/
v5: https://lore.kernel.org/netdev/20260807140751.1351-1-xiexinet@gmail.com/
v4: https://lore.kernel.org/netdev/20260803222211.877-1-xiexinet@gmail.com/
v3: https://lore.kernel.org/netdev/20260731090224.18-1-xiexinet@gmail.com/
v2: https://lore.kernel.org/netdev/20260724161253.79-1-xiexinet@gmail.com/
v1: https://lore.kernel.org/netdev/20260722171836.196-1-xiexinet@gmail.com/
Xin Xie (4):
net: hsr: keep GRO disabled on HSR/PRP ports
net: hsr: refuse forwarding devices as HSR/PRP ports
net: hsr: segment GSO before per-frame forwarding
selftests: net: hsr: verify GRO policy and ordered forwarding
.../networking/net_cachelines/net_device.rst | 1 +
include/linux/netdevice.h | 7 +
net/core/dev.c | 15 +
net/hsr/hsr_device.c | 6 +-
net/hsr/hsr_forward.c | 154 ++
net/hsr/hsr_forward.h | 1 +
net/hsr/hsr_slave.c | 94 +-
tools/testing/selftests/net/hsr/Makefile | 7 +-
tools/testing/selftests/net/hsr/config | 4 +
.../selftests/net/hsr/hsr_gro_superpacket.py | 1376 +++++++++++++++++
.../selftests/net/hsr/hsr_gro_superpacket.sh | 9 +
.../net/hsr/hsr_ordered_forwarding.sh | 9 +
12 files changed, 1668 insertions(+), 15 deletions(-)
create mode 100755 tools/testing/selftests/net/hsr/hsr_gro_superpacket.py
create mode 100755 tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh
create mode 100755 tools/testing/selftests/net/hsr/hsr_ordered_forwarding.sh
base-commit: fafbc4051123d5459ea6950899cec08ed946e278
--
2.53.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH net v8 1/4] net: hsr: keep GRO disabled on HSR/PRP ports
2026-11-07 22:05 [PATCH net v8 0/4] net: hsr: fix super-packet forwarding Xin Xie
@ 2026-11-07 22:05 ` Xin Xie
2026-11-08 22:08 ` netdev-bot+sashiko
2026-11-07 22:05 ` [PATCH net v8 2/4] net: hsr: refuse forwarding devices as " Xin Xie
` (2 subsequent siblings)
3 siblings, 1 reply; 10+ messages in thread
From: Xin Xie @ 2026-11-07 22:05 UTC (permalink / raw)
To: netdev, linux-kselftest, linux-kernel
Cc: davem, edumazet, kuba, pabeni, horms, andrew+netdev, shuah, kees,
petr.wozniak, qingfang.deng, fmaurer, luka.gejak, bigeasy,
xiaoliang.yang_1, skhawaja, liuhangbin, stable, sdf.kernel,
xiexinet, lukma, fmancera, danishanwar, corbet, skhan, rdunlap,
linux-doc
HSR and PRP tag and de-duplicate each wire frame separately. GRO on a
member device merges several frames into one skb that cannot get valid
per-frame metadata, and userspace can turn GRO back on at any time.
Mark direct members (slave A/B, interlink) with a kernel role bit.
While it is set, __netdev_update_features() drops software GRO and
configurable GRO_HW from wanted_features as well as from the computed
features, before ndo_fix_features(). GRO_HW that the driver keeps on
is left alone. The request has to go too: veth adds and deletes its
NAPI instances according to wanted_features, so with GRO still
requested, closing an attached veth spins in napi_disable(), and
detaching a veth that was attached while down and then brought up hits
the BUG_ON() in napi_enable(). As for LRO disabled by an upper device,
GRO is not restored when the port leaves; ethtool can turn it back on.
The attach fails if software GRO stays active. A failed attach does
not restore the GRO request either; the next patch refuses the usual
causes before any state is changed. GSO skbs can still arrive, for
example from a stacked member whose lower device runs GRO; a later
patch segments them.
Fixes: 5055cccfc2d1 ("net: hsr: Provide RedBox support (HSR-SAN)")
Signed-off-by: Xin Xie <xiexinet@gmail.com>
---
.../networking/net_cachelines/net_device.rst | 1 +
include/linux/netdevice.h | 7 +++
net/core/dev.c | 15 ++++++
net/hsr/hsr_slave.c | 52 +++++++++++++++++++
4 files changed, 75 insertions(+)
diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Documentation/networking/net_cachelines/net_device.rst
index 8eceaa80b686..0d87e4d92a58 100644
--- a/Documentation/networking/net_cachelines/net_device.rst
+++ b/Documentation/networking/net_cachelines/net_device.rst
@@ -169,6 +169,7 @@ unsigned_long:1 see_all_hwtstamp_requests
unsigned_long:1 change_proto_down
unsigned_long:1 netns_immutable
unsigned_long:1 fcoe_mtu
+unsigned_long:1 hsr_port
struct list_head net_notifier_list
struct macsec_ops* macsec_ops
struct udp_tunnel_nic_info* udp_tunnel_nic_info
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 435be7880d7e..818d69623456 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -2110,6 +2110,7 @@ enum netdev_reg_state {
* @change_proto_down: device supports setting carrier via IFLA_PROTO_DOWN
* @netns_immutable: interface can't change network namespaces
* @fcoe_mtu: device supports maximum FCoE MTU, 2158 bytes
+ * @hsr_port: direct HSR/PRP member (slave A/B or interlink)
*
* @net_notifier_list: List of per-net netdev notifier block
* that follow this device when it is moved
@@ -2531,6 +2532,12 @@ struct net_device {
unsigned long change_proto_down:1;
unsigned long netns_immutable:1;
unsigned long fcoe_mtu:1;
+ /* Direct HSR/PRP member: slave A/B or interlink. While set,
+ * software GRO and configurable GRO_HW are dropped, requested and
+ * active; GRO_HW that the driver keeps on is left alone. Kernel
+ * role state only; no user ABI or offload capability.
+ */
+ unsigned long hsr_port:1;
struct list_head net_notifier_list;
diff --git a/net/core/dev.c b/net/core/dev.c
index df1a840b0334..2916b17f4169 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -11269,6 +11269,21 @@ int __netdev_update_features(struct net_device *dev)
features = netdev_get_wanted_features(dev);
+ /* A direct HSR/PRP port needs per-frame metadata. Drop software
+ * GRO and configurable GRO_HW from the request as well as from the
+ * result, as the core does for upper-disabled features: drivers
+ * such as veth derive state from wanted_features and need the
+ * requested and the active bit to agree. Fixed-on or
+ * driver-required GRO_HW is left to the driver.
+ */
+ if (dev->hsr_port) {
+ netdev_features_t gro = NETIF_F_GRO |
+ (dev->hw_features & NETIF_F_GRO_HW);
+
+ dev->wanted_features &= ~gro;
+ features &= ~gro;
+ }
+
if (dev->netdev_ops->ndo_fix_features)
features = dev->netdev_ops->ndo_fix_features(dev, features);
diff --git a/net/hsr/hsr_slave.c b/net/hsr/hsr_slave.c
index a546f70f9cc8..bc340ae0b52e 100644
--- a/net/hsr/hsr_slave.c
+++ b/net/hsr/hsr_slave.c
@@ -11,6 +11,7 @@
#include <linux/etherdevice.h>
#include <linux/if_arp.h>
#include <linux/if_vlan.h>
+#include <net/netdev_lock.h>
#include "hsr_main.h"
#include "hsr_device.h"
#include "hsr_forward.h"
@@ -137,6 +138,35 @@ static int hsr_check_dev_ok(struct net_device *dev,
return 0;
}
+/* Member role and feature policy.
+ * Setting the role makes __netdev_update_features() clear GRO
+ * and configurable GRO_HW from both the requested and the active
+ * features of this device. Only the role and feature update run
+ * under the device ops lock; unlink, promiscuous updates and the
+ * master recompute stay outside.
+ */
+static void hsr_portdev_role_set(struct net_device *dev)
+{
+ netdev_lock_ops(dev);
+ dev->hsr_port = true;
+ netdev_change_features(dev);
+ netdev_unlock_ops(dev);
+}
+
+static void hsr_portdev_role_clear(struct net_device *dev)
+{
+ netdev_lock_ops(dev);
+ dev->hsr_port = false;
+ /* Recompute features without the role. GRO was dropped from
+ * the request while the role was set and stays off; the user
+ * can turn it back on with ethtool. A dying device just drops
+ * the role, a netns move stays registered and is recomputed.
+ */
+ if (dev->reg_state == NETREG_REGISTERED)
+ netdev_change_features(dev);
+ netdev_unlock_ops(dev);
+}
+
/* Setup device to be added to the HSR bridge. */
static int hsr_portdev_setup(struct hsr_priv *hsr, struct net_device *dev,
struct hsr_port *port,
@@ -169,6 +199,22 @@ static int hsr_portdev_setup(struct hsr_priv *hsr, struct net_device *dev,
if (res)
goto fail_upper_dev_link;
+ /* Enable the member role and recompute before the RX handler
+ * is published, so that software GRO on this device is off
+ * before frames can arrive. This covers the direct member
+ * only: a stacked member receives from a lower device that
+ * may still run GRO and can deliver merged frames as GSO skbs.
+ * A still-active software GRO rejects the port, a remaining
+ * GRO_HW alone does not.
+ */
+ hsr_portdev_role_set(dev);
+ if (dev->features & NETIF_F_GRO) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "software GRO still on after feature update, cannot join");
+ res = -EBUSY;
+ goto fail_role_policy;
+ }
+
res = netdev_rx_handler_register(dev, hsr_handle_frame, port);
if (res)
goto fail_rx_handler;
@@ -177,7 +223,12 @@ static int hsr_portdev_setup(struct hsr_priv *hsr, struct net_device *dev,
return 0;
fail_rx_handler:
+fail_role_policy:
+ /* Rollback order: unlink the upper, clear the role and
+ * recompute, then undo this setup's promiscuous increment.
+ */
netdev_upper_dev_unlink(dev, hsr_dev);
+ hsr_portdev_role_clear(dev);
fail_upper_dev_link:
if (!port->hsr->fwd_offloaded || port->type == HSR_PT_INTERLINK)
dev_set_promiscuity(dev, -1);
@@ -248,6 +299,7 @@ void hsr_del_port(struct hsr_port *port)
if (port->type == HSR_PT_SLAVE_A || port->type == HSR_PT_SLAVE_B)
vlan_vids_del_by_dev(port->dev, master->dev);
netdev_upper_dev_unlink(port->dev, master->dev);
+ hsr_portdev_role_clear(port->dev);
if (hsr->prot_version == PRP_V1 &&
port->type == HSR_PT_SLAVE_B) {
eth_hw_addr_set(port->dev, port->original_macaddress);
--
2.53.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH net v8 2/4] net: hsr: refuse forwarding devices as HSR/PRP ports
2026-11-07 22:05 [PATCH net v8 0/4] net: hsr: fix super-packet forwarding Xin Xie
2026-11-07 22:05 ` [PATCH net v8 1/4] net: hsr: keep GRO disabled on HSR/PRP ports Xin Xie
@ 2026-11-07 22:05 ` Xin Xie
2026-11-08 22:08 ` netdev-bot+sashiko
2026-11-07 22:05 ` [PATCH net v8 3/4] net: hsr: segment GSO before per-frame forwarding Xin Xie
2026-11-07 22:05 ` [PATCH net v8 4/4] selftests: net: hsr: verify GRO policy and ordered forwarding Xin Xie
3 siblings, 1 reply; 10+ messages in thread
From: Xin Xie @ 2026-11-07 22:05 UTC (permalink / raw)
To: netdev, linux-kselftest, linux-kernel
Cc: davem, edumazet, kuba, pabeni, horms, andrew+netdev, shuah, kees,
petr.wozniak, qingfang.deng, fmaurer, luka.gejak, bigeasy,
xiaoliang.yang_1, skhawaja, liuhangbin, stable, sdf.kernel,
xiexinet, lukma, fmancera, danishanwar,
syzbot+fbf74291c3b7e753b481, sd
syzbot reported recursive locking with an HSR device whose slave is a
bridge, and that bridge has the master of a second HSR device as a port.
The first device transmits its supervision frames with its seqnr_lock
held; the bridge floods them to the second HSR master, whose
hsr_dev_xmit() takes the seqnr_lock of that device:
WARNING: possible recursive locking detected
...
hsr_dev_xmit+0x18a/0x210 net/hsr/hsr_device.c:234
...
br_dev_xmit+0x1202/0x1b40
...
hsr_forward_skb+0x171c/0x2ac0 net/hsr/hsr_forward.c:728
send_hsr_supervision_frame+0x63b/0xcc0 net/hsr/hsr_device.c:351
hsr_announce+0x1f8/0x3a0 net/hsr/hsr_device.c:408
Interlink RX forwards to the slaves with the same lock held. A chain of
two devices does not deadlock, but such a port makes no sense: an HSR
ring and the two PRP LANs are attached to the node directly.
Refuse bridge and OVS masters, bond and team masters (also without
ports, since ports can be added later), and macvlan and MACsec devices
as slave or interlink. The rule is by device type, not by the presence
of lower devices, because a DSA user port is an upper of its conduit
and can offload HSR. VLAN devices are already refused. A tunnel device
whose route leads back through the HSR master, or a software device
without an identifying flag such as batman-adv, is not covered.
Also refuse, before any state is changed, a device whose RX handler is
in use or not supported (IFF_NO_RX_HANDLER: ipvlan, VRF), with an
extack message and the same error codes. Some of them, such as a
macvlan lower, ipvlan or VRF, used to fail only in
netdev_rx_handler_register(), after the previous patch had dropped
their GRO request; a refused device now keeps its state.
A bridge can no longer be used directly as the interlink; connect it
through a veth pair.
Reported-by: syzbot+fbf74291c3b7e753b481@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=fbf74291c3b7e753b481
Fixes: 06afd2c31d33 ("hsr: Synchronize sending frames to have always incremented outgoing seq nr.")
Fixes: 430d67bdcb04 ("net: hsr: Use the seqnr lock for frames received via interlink port.")
Signed-off-by: Xin Xie <xiexinet@gmail.com>
---
net/hsr/hsr_slave.c | 30 ++++++++++++++++++++++++++++--
1 file changed, 28 insertions(+), 2 deletions(-)
diff --git a/net/hsr/hsr_slave.c b/net/hsr/hsr_slave.c
index bc340ae0b52e..26f16d98e479 100644
--- a/net/hsr/hsr_slave.c
+++ b/net/hsr/hsr_slave.c
@@ -120,6 +120,22 @@ static int hsr_check_dev_ok(struct net_device *dev,
return -EINVAL;
}
+ /* The RX handler is taken by devices such as bridge/bond/team
+ * ports, macvlan and ipvlan lowers and HSR ports, or unsupported
+ * (IFF_NO_RX_HANDLER). Refuse before any state change, so a
+ * failed attach leaves the device as it was.
+ */
+ if (dev->priv_flags & IFF_NO_RX_HANDLER) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "This device does not support an RX handler.");
+ return -EINVAL;
+ }
+ if (netdev_is_rx_handler_busy(dev)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "RX handler busy: device is already a lower of another device.");
+ return -EBUSY;
+ }
+
if (is_vlan_dev(dev)) {
NL_SET_ERR_MSG_MOD(extack, "HSR on top of VLAN is not yet supported in this driver.");
return -EINVAL;
@@ -131,9 +147,19 @@ static int hsr_check_dev_ok(struct net_device *dev,
return -EOPNOTSUPP;
}
- /* HSR over bonded devices has not been tested, but I'm not sure it
- * won't work...
+ /* Forwarding devices feed HSR frames into another device's
+ * transmit path. A bridge, bond, team or OVS master or a macvlan
+ * or MACsec device makes no sense as a ring or LAN port and can
+ * nest seqnr_lock through a second HSR device. Refuse them by
+ * type: devices with lower devices in general stay valid, DSA
+ * user ports are uppers of their conduit.
*/
+ if (netif_is_any_bridge_master(dev) || netif_is_lag_master(dev) ||
+ netif_is_macvlan(dev) || netif_is_macsec(dev)) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Cannot use a bridge, bond, team, OVS, macvlan or MACsec device as HSR port.");
+ return -EINVAL;
+ }
return 0;
}
--
2.53.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH net v8 3/4] net: hsr: segment GSO before per-frame forwarding
2026-11-07 22:05 [PATCH net v8 0/4] net: hsr: fix super-packet forwarding Xin Xie
2026-11-07 22:05 ` [PATCH net v8 1/4] net: hsr: keep GRO disabled on HSR/PRP ports Xin Xie
2026-11-07 22:05 ` [PATCH net v8 2/4] net: hsr: refuse forwarding devices as " Xin Xie
@ 2026-11-07 22:05 ` Xin Xie
2026-11-08 22:08 ` netdev-bot+sashiko
2026-11-07 22:05 ` [PATCH net v8 4/4] selftests: net: hsr: verify GRO policy and ordered forwarding Xin Xie
3 siblings, 1 reply; 10+ messages in thread
From: Xin Xie @ 2026-11-07 22:05 UTC (permalink / raw)
To: netdev, linux-kselftest, linux-kernel
Cc: davem, edumazet, kuba, pabeni, horms, andrew+netdev, shuah, kees,
petr.wozniak, qingfang.deng, fmaurer, luka.gejak, bigeasy,
xiaoliang.yang_1, skhawaja, liuhangbin, stable, sdf.kernel,
xiexinet, lukma, fmancera, danishanwar
HSR and PRP tag and de-duplicate each wire frame separately. A GSO
super-packet reaching the forwarding path hides several wire frames,
which cannot be tagged or numbered per frame. Such skbs reach a member
even with GRO off, for example from a container or VM attached through
veth, tap or virtio to a software RedBox, or from a member whose GRO_HW
cannot be turned off.
Add one forwarding entry, hsr_forward_input(), for master TX and for
frames received on any port. It segments GSO input outside seqnr_lock;
each segment is then numbered and submitted under the lock, one frame
at a time as before, and follows the normal forwarding and local
delivery paths. Master TX and interlink RX now take the lock there
instead of in their callers.
Classify by the effective inner protocol with the core's bounded VLAN
decoder (802.1Q, 802.1AD, nested or accelerated). Drop aggregates with
inner HSR/PRP, unreadable fragments or undecodable headers, and IPv4
or IPv6 aggregates whose nonzero IP length ends before the skb does,
since the trailing bytes would become segment payload. GSO_PARTIAL and
zero IP lengths are left to the segmenter. Drops are counted once on
the entry device in the core per-CPU drop counters. Segmentation writes
the control block and headers, so a still-shared input is unshared
first.
Older PRP senders can append one RCT to a whole GSO skb. Such
aggregates, including locally destined ones, are now dropped instead
of being delivered.
Remove the NETIF_F_GSO_MASK types from the master's hw_features so the
core segments local traffic before ndo_start_xmit().
Fixes: f421436a591d ("net/hsr: Add support for the High-availability Seamless Redundancy protocol (HSRv0)")
Signed-off-by: Xin Xie <xiexinet@gmail.com>
---
net/hsr/hsr_device.c | 6 +-
net/hsr/hsr_forward.c | 154 ++++++++++++++++++++++++++++++++++++++++++
net/hsr/hsr_forward.h | 1 +
net/hsr/hsr_slave.c | 12 ++--
4 files changed, 161 insertions(+), 12 deletions(-)
diff --git a/net/hsr/hsr_device.c b/net/hsr/hsr_device.c
index fc512204600c..2521a72280ac 100644
--- a/net/hsr/hsr_device.c
+++ b/net/hsr/hsr_device.c
@@ -232,9 +232,7 @@ static netdev_tx_t hsr_dev_xmit(struct sk_buff *skb, struct net_device *dev)
skb->dev = master->dev;
skb_reset_mac_header(skb);
skb_reset_mac_len(skb);
- spin_lock_bh(&hsr->seqnr_lock);
- hsr_forward_skb(skb, master);
- spin_unlock_bh(&hsr->seqnr_lock);
+ hsr_forward_input(skb, master);
} else {
dev_core_stats_tx_dropped_inc(dev);
dev_kfree_skb_any(skb);
@@ -686,7 +684,7 @@ void hsr_dev_setup(struct net_device *dev)
dev->needs_free_netdev = true;
dev->hw_features = NETIF_F_SG | NETIF_F_FRAGLIST | NETIF_F_HIGHDMA |
- NETIF_F_GSO_MASK | NETIF_F_HW_CSUM |
+ NETIF_F_HW_CSUM |
NETIF_F_HW_VLAN_CTAG_TX |
NETIF_F_HW_VLAN_CTAG_FILTER;
diff --git a/net/hsr/hsr_forward.c b/net/hsr/hsr_forward.c
index 44f6d9c83180..c9cd5d920879 100644
--- a/net/hsr/hsr_forward.c
+++ b/net/hsr/hsr_forward.c
@@ -12,6 +12,9 @@
#include <linux/skbuff.h>
#include <linux/etherdevice.h>
#include <linux/if_vlan.h>
+#include <linux/ip.h>
+#include <linux/ipv6.h>
+#include <net/gso.h>
#include "hsr_main.h"
#include "hsr_framereg.h"
@@ -794,3 +797,154 @@ void hsr_forward_skb(struct sk_buff *skb, struct hsr_port *port)
port->dev->stats.tx_dropped++;
kfree_skb(skb);
}
+
+/* Master TX and interlink RX are numbered locally: number and submit
+ * one frame at a time under seqnr_lock, as before. LAN A/B input
+ * keeps its received number and needs no lock.
+ */
+static void hsr_forward_frame(struct sk_buff *skb, struct hsr_port *port)
+{
+ struct hsr_priv *hsr = port->hsr;
+ bool numbered = port->type == HSR_PT_MASTER ||
+ port->type == HSR_PT_INTERLINK;
+
+ if (numbered)
+ spin_lock_bh(&hsr->seqnr_lock);
+ hsr_forward_skb(skb, port);
+ if (numbered)
+ spin_unlock_bh(&hsr->seqnr_lock);
+}
+
+/* One count per rejected GSO input on its entry device. */
+static void hsr_gso_drop_stat(struct hsr_port *port)
+{
+ if (port->type == HSR_PT_MASTER)
+ dev_core_stats_tx_dropped_inc(port->dev);
+ else
+ dev_core_stats_rx_dropped_inc(port->dev);
+}
+
+/* Return the inner EtherType using the core's bounded 802.1Q/802.1AD
+ * decoder. An accelerated outer tag is metadata, not frame bytes.
+ */
+static __be16 hsr_gso_effective_proto(const struct sk_buff *skb, int *l3off)
+{
+ struct ethhdr eh;
+ const struct ethhdr *eth;
+
+ if (skb_vlan_tag_present(skb) &&
+ !eth_type_vlan(skb->vlan_proto))
+ return 0;
+
+ eth = skb_header_pointer(skb, 0, sizeof(eh), &eh);
+ if (!eth)
+ return 0;
+
+ return __vlan_get_protocol(skb, eth->h_proto, l3off);
+}
+
+/* Bytes beyond the IP datagram must not become segment payload. Older
+ * PRP senders can append an RCT to a whole GSO skb. Zero IP lengths and
+ * GSO_PARTIAL headers do not provide the full aggregate length here.
+ */
+static bool hsr_gso_trailing_data(const struct sk_buff *skb, __be16 proto,
+ int l3off)
+{
+ unsigned int l3len;
+
+ if (skb_shinfo(skb)->gso_type & SKB_GSO_PARTIAL)
+ return false;
+
+ if (proto == htons(ETH_P_IP)) {
+ struct iphdr buffer;
+ const struct iphdr *iph;
+
+ iph = skb_header_pointer(skb, l3off, sizeof(buffer), &buffer);
+ if (!iph)
+ return true;
+ l3len = ntohs(iph->tot_len);
+ } else if (proto == htons(ETH_P_IPV6)) {
+ struct ipv6hdr buffer;
+ const struct ipv6hdr *ip6h;
+
+ ip6h = skb_header_pointer(skb, l3off, sizeof(buffer), &buffer);
+ if (!ip6h)
+ return true;
+ l3len = ntohs(ip6h->payload_len);
+ if (l3len)
+ l3len += sizeof(*ip6h);
+ } else {
+ return false;
+ }
+
+ return l3len && l3off + l3len < skb->len;
+}
+
+/* Unfold GSO super-packets before per-frame processing so that each
+ * wire frame gets its own HSR tag or PRP RCT and sequence number.
+ * Segmentation runs outside seqnr_lock; every segment is numbered and
+ * submitted under it like any other frame.
+ */
+void hsr_forward_input(struct sk_buff *skb, struct hsr_port *port)
+{
+ struct sk_buff *segs, *next;
+ __be16 proto;
+ int l3off;
+
+ if (likely(!skb_is_gso(skb))) {
+ hsr_forward_frame(skb, port);
+ return;
+ }
+
+ /* The gate is content-based, not port-based. An aggregate
+ * whose effective protocol is ETH_P_HSR or ETH_P_PRP cannot be
+ * segmented safely and is dropped, as is any skb whose headers
+ * cannot be read or that is marked unreadable (net_iov device
+ * memory): software segmentation cannot read its payload, so
+ * each segment would carry uninitialized data. Ordinary VLAN
+ * GSO (802.1Q/802.1AD at any reachable tag depth, accelerated
+ * or in-band) is classified to its inner protocol through the
+ * core VLAN decoder and segmented like plain input. With
+ * NETIF_F_HW_HSR_TAG_RM the lower has already stripped the tag,
+ * so such aggregates arrive plain.
+ */
+ if (!skb_frags_readable(skb))
+ goto drop;
+ proto = hsr_gso_effective_proto(skb, &l3off);
+ if (!proto)
+ goto drop;
+ if (proto == htons(ETH_P_HSR) || proto == htons(ETH_P_PRP))
+ goto drop;
+ if (hsr_gso_trailing_data(skb, proto, l3off))
+ goto drop;
+
+ /* Segmentation writes skb->cb and header metadata: work on a
+ * private skb if the input is still shared with a packet socket.
+ */
+ skb = skb_share_check(skb, GFP_ATOMIC);
+ if (!skb) {
+ hsr_gso_drop_stat(port);
+ return;
+ }
+
+ /* features = 0 requests full software segmentation into single
+ * wire frames; tx_path is true only for locally generated
+ * traffic on the master.
+ */
+ segs = __skb_gso_segment(skb, 0, port->type == HSR_PT_MASTER, 0);
+ if (IS_ERR_OR_NULL(segs))
+ goto drop;
+
+ consume_skb(skb);
+ while (segs) {
+ next = segs->next;
+ segs->next = NULL;
+ hsr_forward_frame(segs, port);
+ segs = next;
+ }
+ return;
+
+drop:
+ hsr_gso_drop_stat(port);
+ kfree_skb(skb);
+}
diff --git a/net/hsr/hsr_forward.h b/net/hsr/hsr_forward.h
index 206636750b30..ec08c4330259 100644
--- a/net/hsr/hsr_forward.h
+++ b/net/hsr/hsr_forward.h
@@ -14,6 +14,7 @@
#include "hsr_main.h"
void hsr_forward_skb(struct sk_buff *skb, struct hsr_port *port);
+void hsr_forward_input(struct sk_buff *skb, struct hsr_port *port);
struct sk_buff *prp_create_tagged_frame(struct hsr_frame_info *frame,
struct hsr_port *port);
struct sk_buff *hsr_create_tagged_frame(struct hsr_frame_info *frame,
diff --git a/net/hsr/hsr_slave.c b/net/hsr/hsr_slave.c
index 26f16d98e479..73f4e555e18e 100644
--- a/net/hsr/hsr_slave.c
+++ b/net/hsr/hsr_slave.c
@@ -75,15 +75,11 @@ static rx_handler_result_t hsr_handle_frame(struct sk_buff **pskb)
skb_reset_mac_len(skb);
/* Only the frames received over the interlink port will assign a
- * sequence number and require synchronisation vs other sender.
+ * sequence number and require synchronisation vs other sender;
+ * hsr_forward_input() takes seqnr_lock for them, one frame at a
+ * time.
*/
- if (port->type == HSR_PT_INTERLINK) {
- spin_lock_bh(&hsr->seqnr_lock);
- hsr_forward_skb(skb, port);
- spin_unlock_bh(&hsr->seqnr_lock);
- } else {
- hsr_forward_skb(skb, port);
- }
+ hsr_forward_input(skb, port);
finish_consume:
return RX_HANDLER_CONSUMED;
--
2.53.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH net v8 4/4] selftests: net: hsr: verify GRO policy and ordered forwarding
2026-11-07 22:05 [PATCH net v8 0/4] net: hsr: fix super-packet forwarding Xin Xie
` (2 preceding siblings ...)
2026-11-07 22:05 ` [PATCH net v8 3/4] net: hsr: segment GSO before per-frame forwarding Xin Xie
@ 2026-11-07 22:05 ` Xin Xie
2026-11-08 22:08 ` netdev-bot+sashiko
3 siblings, 1 reply; 10+ messages in thread
From: Xin Xie @ 2026-11-07 22:05 UTC (permalink / raw)
To: netdev, linux-kselftest, linux-kernel
Cc: davem, edumazet, kuba, pabeni, horms, andrew+netdev, shuah, kees,
petr.wozniak, qingfang.deng, fmaurer, luka.gejak, bigeasy,
xiaoliang.yang_1, skhawaja, liuhangbin, stable, sdf.kernel,
xiexinet, lukma, fmancera, danishanwar
Add two tests, run by shell entries that exec one installed Python
helper.
hsr_gro_superpacket.sh checks the member GRO policy: GRO is off,
requested and active, while a device is a member and after it leaves,
and can then be turned on again; a device refused at attach keeps its
request. It covers close and reopen of an attached veth, a veth
attached while down, the master's GSO type mask, a refused bridge port
with rollback of the port added before it, a busy macvlan lower, and
the refusal of bridge, bond, macvlan and ipvlan devices without leaving
an upper link or a promiscuity reference on any port. Natural TCP from
the interlink is compared byte for byte, retransmissions included, with
both LAN captures. TCP from a SAN on one PRP LAN, in plain and VLAN
forms, is compared the same way with the ingress LAN and the local
delivery, and its reply must appear once on each LAN. Tags/RCT,
sequence numbers and VLAN stacks are checked.
hsr_ordered_forwarding.sh checks local data around interlink data and
UDP GSO aggregates, local and proxy supervision, two concurrent
producers and an observed sequence-number wrap.
Work is bounded to 120 seconds and cleanup to 20; INT/TERM remove the
test's namespaces. Packets are judged only on complete captures without
socket loss or truncation. A missing prerequisite SKIPs the affected
case; query errors and missing packet evidence FAIL, and the cases
after a failure are reported as not ok.
Signed-off-by: Xin Xie <xiexinet@gmail.com>
---
tools/testing/selftests/net/hsr/Makefile | 7 +-
tools/testing/selftests/net/hsr/config | 4 +
.../selftests/net/hsr/hsr_gro_superpacket.py | 1376 +++++++++++++++++
.../selftests/net/hsr/hsr_gro_superpacket.sh | 9 +
.../net/hsr/hsr_ordered_forwarding.sh | 9 +
5 files changed, 1404 insertions(+), 1 deletion(-)
create mode 100755 tools/testing/selftests/net/hsr/hsr_gro_superpacket.py
create mode 100755 tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh
create mode 100755 tools/testing/selftests/net/hsr/hsr_ordered_forwarding.sh
diff --git a/tools/testing/selftests/net/hsr/Makefile b/tools/testing/selftests/net/hsr/Makefile
index 2150e487ac7d..70f8a804b44a 100644
--- a/tools/testing/selftests/net/hsr/Makefile
+++ b/tools/testing/selftests/net/hsr/Makefile
@@ -3,6 +3,8 @@
top_srcdir = ../../../../..
TEST_PROGS := \
+ hsr_gro_superpacket.sh \
+ hsr_ordered_forwarding.sh \
hsr_ping.sh \
hsr_prp_redbox.sh \
hsr_redbox.sh \
@@ -10,6 +12,9 @@ TEST_PROGS := \
prp_ping.sh \
# end of TEST_PROGS
-TEST_FILES += hsr_common.sh
+TEST_FILES += \
+ hsr_common.sh \
+ hsr_gro_superpacket.py \
+# end of TEST_FILES
include ../../lib.mk
diff --git a/tools/testing/selftests/net/hsr/config b/tools/testing/selftests/net/hsr/config
index 205cc4d3d64b..601f346eb489 100644
--- a/tools/testing/selftests/net/hsr/config
+++ b/tools/testing/selftests/net/hsr/config
@@ -1,6 +1,10 @@
+CONFIG_BONDING=y
CONFIG_BRIDGE=y
CONFIG_HSR=y
CONFIG_IPV6=y
+CONFIG_IPVLAN=y
+CONFIG_MACVLAN=y
CONFIG_NET_SCH_NETEM=m
+CONFIG_PACKET=y
CONFIG_VETH=y
CONFIG_VLAN_8021Q=m
diff --git a/tools/testing/selftests/net/hsr/hsr_gro_superpacket.py b/tools/testing/selftests/net/hsr/hsr_gro_superpacket.py
new file mode 100755
index 000000000000..23feb0ce4df4
--- /dev/null
+++ b/tools/testing/selftests/net/hsr/hsr_gro_superpacket.py
@@ -0,0 +1,1376 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""HSR/PRP GRO policy, segmentation and functional ordering regressions."""
+# pylint: disable=too-many-lines
+
+from array import array
+from collections import Counter
+from contextlib import contextmanager
+import ctypes
+import errno
+import json
+import os
+import re
+import selectors
+import shutil
+import signal
+import socket
+import struct
+import subprocess
+import sys
+import threading
+import time
+import uuid
+
+
+class Skip(Exception):
+ """A missing prerequisite for one case."""
+
+
+# INT/TERM only record the signal; Lab.remaining() raises at the next check.
+SIGNALLED = []
+
+# Errors a case reports as "not ok"; anything else is a bug in this script.
+CASE_ERRORS = (OSError, ValueError, RuntimeError, LookupError,
+ subprocess.SubprocessError, struct.error)
+
+
+def require(condition, message):
+ """Raise ValueError(message) unless condition holds."""
+ if not condition:
+ raise ValueError(message)
+
+
+def features(text):
+ """Parse ethtool -k: name -> (active, fixed, requested)."""
+ result = {}
+ for line in text.splitlines():
+ if ":" not in line or line.startswith("Features for"):
+ continue
+ name, state = line.strip().split(":", 1)
+ match = re.fullmatch(r"\s*(on|off)((?:\s+\[[^\]]*\])*)", state)
+ require(match is not None, "malformed feature line: " + line)
+ # ethtool marks a wanted bit that differs from the active one.
+ requested = re.search(r"\[requested (on|off)\]", match[2])
+ result[name] = (match[1], "[fixed]" in match[2],
+ requested[1] if requested else match[1])
+ require(result, "empty feature listing")
+ return result
+
+
+def decode(raw, ancillary=(), prp=False):
+ """Decode one captured frame; PRP RCTs are recognized only if prp."""
+ require(len(raw) >= 14, "short Ethernet capture")
+ proto = struct.unpack_from("!H", raw, 12)[0]
+ off, stack, aux, checksum_partial = 14, [], (), False
+ for level, kind, data in ancillary:
+ if level == 263 and kind == 8:
+ require(len(data) >= 20, "short PACKET_AUXDATA")
+ status, length, snap, _, _, tci, tpid = struct.unpack("IIIHHHH", data[:20])
+ require(length == snap, "truncated packet auxiliary length")
+ checksum_partial = bool(status & 8)
+ if status & 0x10:
+ aux = ((tpid if status & 0x40 else 0x8100, tci),)
+ while proto in (0x8100, 0x88a8):
+ require(off + 4 <= len(raw), "short VLAN header")
+ tci, inner = struct.unpack_from("!HH", raw, off)
+ stack.append((proto, tci))
+ proto, off = inner, off + 4
+ row = {"raw": raw, "size": len(raw) + 4 * len(aux), "vlans": aux + tuple(stack),
+ "aux": aux, "inband": tuple(stack), "src": raw[6:12], "dst": raw[:6],
+ "proto": proto, "seq": None, "path": None, "payload": b"", "ports": (),
+ "tcpseq": None, "supervision": None, "supseq": None, "tag": False,
+ "encoding": "plain", "checksum_partial": checksum_partial}
+ end = len(raw)
+ if proto == 0x892f:
+ require(off + 6 <= end, "short HSR tag")
+ field, seq, proto = struct.unpack_from("!HHH", raw, off)
+ row.update(seq=seq, path=field >> 12, tag=True, encoding="hsr")
+ expected = len(raw) - 14 - (4 if raw[12:14] == b"\x81\x00" else 0)
+ require(field & 0xfff == expected, "HSR LSDU size differs")
+ off += 6
+ elif prp and end >= off + 6 and raw[-2:] == b"\x88\xfb":
+ seq, field = struct.unpack_from("!HH", raw, end - 6)
+ expected = len(raw) - 14 - (4 if raw[12:14] == b"\x81\x00" else 0)
+ # Like prp_fill_frame_info(): a size mismatch means a plain frame.
+ if field & 0xfff == expected:
+ row.update(seq=seq, path=field >> 12, tag=True, encoding="prp")
+ end -= 6
+ row["proto"] = proto
+ row["network"] = raw[off:end]
+ if raw[:3] == b"\x01\x15\x4e" and proto == 0x88fb:
+ require(off + 12 <= end, "short supervision payload")
+ _, seq, kind, size = struct.unpack_from("!HHBB", raw, off)
+ require(kind in (20, 21, 22, 23) and size in (6, 12), "invalid supervision TLV")
+ row.update(supervision=raw[off + 6:off + 12], supseq=seq)
+ return row
+ if proto != 0x0800:
+ return row
+ require(off + 20 <= end and raw[off] >> 4 == 4, "invalid IPv4 header")
+ ihl, total = (raw[off] & 15) * 4, struct.unpack_from("!H", raw, off + 2)[0]
+ require(ihl >= 20 and total >= ihl and off + total <= end, "invalid IP extent")
+ protocol, l4 = raw[off + 9], off + ihl
+ ipend = off + total
+ row.update(ip_src=raw[off + 12:off + 16], ip_dst=raw[off + 16:off + 20])
+ if protocol == 6:
+ require(l4 + 20 <= ipend, "short TCP header")
+ size = (raw[l4 + 12] >> 4) * 4
+ require(size >= 20 and l4 + size <= ipend, "invalid TCP extent")
+ row.update(ports=struct.unpack_from("!HH", raw, l4),
+ tcpseq=struct.unpack_from("!I", raw, l4 + 4)[0],
+ tcpflags=raw[l4 + 13], payload=raw[l4 + size:ipend])
+ elif protocol == 17:
+ require(l4 + 8 <= ipend, "short UDP header")
+ size = struct.unpack_from("!H", raw, l4 + 4)[0]
+ require(size >= 8 and l4 + size == ipend, "invalid UDP extent")
+ row.update(ports=struct.unpack_from("!HH", raw, l4), payload=raw[l4 + 8:ipend])
+ return row
+
+
+class Capture:
+ """One AF_PACKET receive socket bound to a single device."""
+
+ def __init__(self, lab, ns, dev, pkttype=None, prp=False):
+ self.lab, self.rows, self.seen = lab, [], 0
+ self.ns, self.dev, self.packets = ns, dev, None
+ self.pkttype, self.name, self.prp = pkttype, ns + "/" + dev, prp
+ self.complete, self.drops, self.truncated = False, None, 0
+ self.last_seen = time.monotonic()
+ # Protocol 0 registers no receive hook until bind() names the device.
+ self.socket = lab.sock(ns, socket.AF_PACKET, socket.SOCK_RAW, 0)
+ self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 8 * 1024 * 1024)
+ self.socket.setsockopt(263, 8, 1)
+ self.socket.bind((dev, 3))
+ self.socket.setblocking(False)
+ lab.selector.register(self.socket, selectors.EVENT_READ, self)
+
+ def drain(self):
+ """Read queued frames; return (count, queue empty)."""
+ count = 0
+ while count < 256:
+ self.lab.remaining()
+ try:
+ raw, anc, flags, addr = self.socket.recvmsg(131072, 256)
+ except BlockingIOError:
+ return count, True
+ self.seen += 1
+ self.last_seen = time.monotonic()
+ count += 1
+ if flags & (socket.MSG_TRUNC | socket.MSG_CTRUNC):
+ self.truncated += 1
+ raise ValueError(self.name + ": capture truncated")
+ if addr[0] == self.dev and (self.pkttype is None or addr[2] == self.pkttype):
+ self.rows.append(decode(raw, anc, self.prp))
+ return count, False
+
+ def validate(self):
+ """Fail unless the capture completed without loss."""
+ require(self.complete and self.drops == 0 and self.truncated == 0,
+ self.name + ": incomplete/dropped/truncated capture")
+
+
+@contextmanager
+def signals_blocked():
+ """Hold INT/TERM back while namespace or handler state switches."""
+ previous = signal.pthread_sigmask(signal.SIG_BLOCK, {signal.SIGINT, signal.SIGTERM})
+ try:
+ yield
+ finally:
+ signal.pthread_sigmask(signal.SIG_SETMASK, previous)
+
+
+class Lab:
+ """Owned namespaces, sockets and captures of one test run."""
+
+ def __init__(self, deadline):
+ self.deadline = deadline
+ self.names, self.sockets, self.captures, self.threads = [], [], [], []
+ self.case_start = 0
+ self.selector = selectors.DefaultSelector()
+ self.original = os.open("/proc/self/ns/net", os.O_RDONLY)
+ self.token, self.restore_failed = uuid.uuid4().hex[:10], False
+ self.stop = threading.Event()
+ self.libc = ctypes.CDLL(None, use_errno=True)
+
+ def begin(self):
+ """Start a case: captures left open by an earlier case are closed."""
+ for cap in self.captures[self.case_start:]:
+ if not cap.complete:
+ self.selector.unregister(cap.socket)
+ cap.socket.close()
+ self.case_start = len(self.captures)
+
+ def remaining(self):
+ """Seconds left in the work deadline; raise once it or a signal hit."""
+ if SIGNALLED:
+ raise InterruptedError(f"signal {SIGNALLED[0]}")
+ value = self.deadline - time.monotonic()
+ require(value > 0, "120-second work deadline expired")
+ return value
+
+ def run(self, args, check=True):
+ """Run a command with a bounded timeout."""
+ require(not self.restore_failed, "network namespace restoration failed")
+ result = subprocess.run([str(arg) for arg in args], text=True, capture_output=True,
+ timeout=min(5, self.remaining()), check=False)
+ require(not check or result.returncode == 0,
+ f"{args}: rc{result.returncode} {result.stderr.strip()}")
+ return result
+
+ def ip(self, ns, *args, check=True):
+ """Run ip(8) in a namespace."""
+ return self.run(["ip", "-n", ns, *args], check)
+
+ def cmd(self, ns, *args, check=True):
+ """Run a command in a namespace."""
+ return self.run(["ip", "netns", "exec", ns, *args], check)
+
+ def features(self, ns, dev):
+ """Return the parsed ethtool -k listing of a device."""
+ return features(self.cmd(ns, "ethtool", "-k", dev).stdout)
+
+ def ns(self):
+ """Create an owned namespace with loopback up."""
+ name = f"hsrp4-{self.token}-{len(self.names)}"
+ require(not os.path.lexists("/var/run/netns/" + name), "namespace collision")
+ self.names.append(name)
+ self.run(["ip", "netns", "add", name])
+ self.ip(name, "link", "set", "dev", "lo", "up")
+ return name
+
+ def veth(self, ns, dev, peer_ns, peer):
+ """Create a veth pair across two namespaces."""
+ self.ip(ns, "link", "add", "name", dev, "type", "veth", "peer", "name", peer,
+ "netns", peer_ns)
+
+ def setns(self, descriptor):
+ """Switch this thread's network namespace."""
+ if self.libc.setns(descriptor, 0x40000000):
+ error = ctypes.get_errno()
+ raise OSError(error, os.strerror(error))
+
+ @contextmanager
+ def enter(self, ns):
+ """Run the body in a namespace and always switch back."""
+ descriptor, entered = None, False
+ try:
+ with signals_blocked():
+ descriptor = os.open("/var/run/netns/" + ns, os.O_RDONLY)
+ self.setns(descriptor)
+ entered = True
+ yield
+ finally:
+ with signals_blocked():
+ try:
+ if entered:
+ self.setns(self.original)
+ except BaseException:
+ self.restore_failed = True
+ raise
+ finally:
+ if descriptor is not None:
+ os.close(descriptor)
+
+ def sock(self, ns, family, kind, proto=0):
+ """Open an owned socket bound to a namespace."""
+ with self.enter(ns):
+ result = socket.socket(family, kind, proto)
+ self.sockets.append(result)
+ result.settimeout(min(3, self.remaining()))
+ return result
+
+ def capture(self, ns, dev, pkttype=None, prp=False):
+ """Start a capture owned by the current case."""
+ result = Capture(self, ns, dev, pkttype, prp)
+ self.captures.append(result)
+ return result
+
+ def observe(self, seconds, pump=None):
+ """Drain ready captures for some time, calling pump() each round."""
+ end = time.monotonic() + seconds
+ while True:
+ self.remaining()
+ if pump is not None:
+ pump()
+ for key, _ in self.selector.select(min(0.01, max(0, end - time.monotonic()))):
+ key.data.drain()
+ if time.monotonic() >= end:
+ return
+
+ def finish(self, done=None):
+ """Complete this case's captures.
+
+ done() states when every expected frame has been captured; the
+ deadline bounds the wait. Only without such a condition does a
+ 100 ms quiet period end the wait.
+ """
+ active = [cap for cap in self.captures[self.case_start:] if not cap.complete]
+ end = min(self.deadline, time.monotonic() + 2)
+ quiet = time.monotonic()
+ while time.monotonic() < end:
+ self.observe(0.01)
+ empty = True
+ for cap in active:
+ count, drained = cap.drain()
+ if count or not drained:
+ empty = False
+ quiet = max([quiet] + [cap.last_seen for cap in active])
+ if done is not None:
+ if done():
+ break
+ elif empty and time.monotonic() - quiet >= 0.1:
+ break
+ else:
+ # A missing expected frame is reported by the case's verifier.
+ require(done is not None, "capture failed to drain within fixed deadline")
+ end = max(end, min(self.deadline, time.monotonic() + 1))
+ for ns in sorted({cap.ns for cap in active}):
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ require(ns in self.names, "capture namespace is not owned")
+ self.ip(ns, "link", "set", "dev", "lo", "down")
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ state = link_state(self, ns, "lo")
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ require("UP" not in state["flags"],
+ "capture cutoff loopback is still up")
+ # Rebinding to a different down device synchronizes the old receive hook.
+ # Its ENETDOWN socket error must be cleared before draining queued frames.
+ for cap in active:
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ require(cap.dev != "lo", "capture already uses the cutoff device")
+ cap.socket.bind(("lo", 0))
+ require(cap.socket.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR) == errno.ENETDOWN,
+ cap.name + ": capture cutoff did not report ENETDOWN")
+ for cap in active:
+ while True:
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ if cap.drain()[1]:
+ break
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ packets, cap.drops = struct.unpack("II", cap.socket.getsockopt(263, 6, 8))
+ cap.packets = packets
+ require(time.monotonic() < end, "capture cutoff/drain deadline expired")
+ require(packets == cap.seen + cap.drops,
+ f"{cap.name}: queued capture remains packets={packets} "
+ f"seen={cap.seen} drops={cap.drops}")
+ cap.complete = True
+ cap.validate()
+ self.selector.unregister(cap.socket)
+ cap.socket.close()
+ print(f"# capture {cap.name} packets={cap.seen} drops=0 truncated=0 complete=1",
+ flush=True)
+
+ def close(self):
+ """Stop threads, close sockets and delete owned namespaces."""
+ self.stop.set()
+ end, errors = time.monotonic() + 20, []
+ for sock in self.sockets:
+ sock.close()
+ for thread in self.threads:
+ if thread.ident is not None:
+ thread.join(max(0, end - time.monotonic()))
+ if thread.is_alive():
+ errors.append("sender thread did not stop")
+ self.selector.close()
+ if self.restore_failed:
+ # ip netns del removes the /var/run/netns bind mount, which does
+ # not depend on the network namespace of this thread.
+ errors.append("network namespace restoration failed")
+ for name in reversed(self.names):
+ if time.monotonic() >= end:
+ errors.append("cleanup deadline expired with owned namespaces remaining")
+ break
+ try:
+ result = subprocess.run(["ip", "netns", "del", name],
+ capture_output=True, text=True, check=False,
+ timeout=min(3, end - time.monotonic()))
+ require(result.returncode == 0 and
+ not os.path.lexists("/var/run/netns/" + name),
+ "namespace cleanup failed: " + name + " " + result.stderr)
+ except (OSError, ValueError, subprocess.SubprocessError) as error:
+ errors.append(str(error))
+ os.close(self.original)
+ directory = os.getenv("HSR_CAPTURE_EVIDENCE")
+ if directory:
+ try:
+ for number, cap in enumerate(self.captures):
+ require(time.monotonic() < end, "cleanup/evidence deadline expired")
+ path = os.path.join(directory, f"{self.token}-{number}.json")
+ record = {"name": cap.name, "seen": cap.seen, "packets": cap.packets,
+ "drops": cap.drops, "truncated": cap.truncated,
+ "complete": cap.complete, "pkttype": cap.pkttype,
+ "rows": cap.rows}
+ with open(path, "x", encoding="utf-8") as stream:
+ json.dump(record, stream, default=lambda value: value.hex())
+ except (OSError, ValueError, TypeError) as error:
+ errors.append("capture evidence: " + str(error))
+ require(not errors, "; ".join(errors))
+
+
+def coverage(frames, expected):
+ """Count how often each stream byte occurs in (offset, payload) frames."""
+ counts = array("I", [0]) * len(expected)
+ for offset, payload in frames:
+ require(payload and offset >= 0 and offset + len(payload) <= len(expected),
+ "invalid stream extent")
+ require(payload == expected[offset:offset + len(payload)], "stream payload differs")
+ for position in range(offset, offset + len(payload)):
+ counts[position] += 1
+ return counts
+
+
+def protocol_order(rows, wrap=False, key="seq"):
+ """Require strictly increasing 16-bit protocol sequence numbers."""
+ numbers = [row[key] for row in rows]
+ require(numbers and all(value is not None for value in numbers), "missing protocol sequence")
+ # Successive pairs; itertools.pairwise() needs Python 3.10.
+ steps = [(numbers[index - 1], numbers[index]) for index in range(1, len(numbers))]
+ require(all(0 < ((right - left) & 0xffff) < 0x8000 for left, right in steps),
+ "protocol sequence out of order")
+ if wrap:
+ require(any(left > right for left, right in steps), "no actual sequence wrap")
+
+
+def contains(have, wanted):
+ """Multiset inclusion of Counters (their >= needs Python 3.10)."""
+ return all(have[key] >= count for key, count in wanted.items())
+
+
+def wire_rows(rows, port, stack, tagged=True):
+ """Return and check one lane's frames of a flow."""
+ data = [row for row in rows if row["ports"] and row["ports"][1] == port and row["payload"]]
+ require(data, "no captured flow payload")
+ for row in data:
+ require(row["tag"] == tagged, "flow tag/RCT missing or unexpected")
+ require(row["encoding"] == ("hsr" if tagged else "plain"), "wrong protocol tag kind")
+ require(row["vlans"] == stack, "flow VLAN stack differs")
+ require(row["size"] <= 1520 + 4 * len(stack), "unsegmented output frame")
+ require(len({row["src"] for row in data}) == 1 and
+ len({row["dst"] for row in data}) == 1, "flow MAC addresses differ within lane")
+ if tagged:
+ protocol_order(data)
+ return data
+
+
+def copy_identity(row):
+ """Fields the two LAN copies of a frame share."""
+ # Lane/path and source substitution differ legitimately between the two ports.
+ return (row["seq"], row["vlans"], row["dst"], row["proto"], row["network"])
+
+
+def verify_stream(captures, port, payload, stack=(), local=False, inband=False, ingress=None):
+ """Check every TCP byte occurrence on the sender against the outputs."""
+ for cap in captures:
+ cap.validate()
+ source = [row for row in captures[0].rows if row["ports"] and
+ row["ports"][1] == port and row["payload"]]
+ require(source, "sender TCP capture empty")
+ syn = [row for row in captures[0].rows if row["ports"] and
+ row["ports"][1] == port and row.get("tcpflags", 0) & 2]
+ require(syn, "missing captured TCP initial sequence")
+ isn = (syn[0]["tcpseq"] + 1) & 0xffffffff
+ source_counts = coverage([((row["tcpseq"] - isn) & 0xffffffff, row["payload"])
+ for row in source], payload)
+ require(0 not in source_counts, "sender did not send complete stream")
+ aggregate = [row for row in source if len(row["payload"]) > 1500]
+ if ingress is not None:
+ ingress.validate()
+ inputs = [row for row in ingress.rows if row["ports"] and
+ row["ports"][1] == port and row["payload"]]
+ require(all(row["vlans"] == stack and row["encoding"] == "plain" for row in inputs),
+ "HSR/PRP ingress classification/VLAN differs")
+ require(coverage([((row["tcpseq"] - isn) & 0xffffffff, row["payload"])
+ for row in inputs], payload) == source_counts,
+ "sender/actual ingress stream occurrence differs")
+ require(not aggregate or any(len(row["payload"]) > 1500 for row in inputs),
+ "no actual ingress aggregate")
+ require(all(row["vlans"] == stack for row in source), "sender VLAN stack differs")
+ if stack:
+ require(all(row["aux"] == (() if inband else stack[:1]) and
+ row["inband"] == (stack if inband else stack[1:]) for row in aggregate),
+ "sender aggregate VLAN encoding differs")
+ if local:
+ outputs = [wire_rows(captures[1].rows, port, stack, tagged=False)]
+ require(all(row["ports"][1] == port for row in outputs[0]),
+ "unexpected local flow direction")
+ else:
+ outputs = [wire_rows(cap.rows, port, stack) for cap in captures[1:]]
+ for index, rows in enumerate(outputs):
+ require(all(row["path"] == index for row in rows), "wrong HSR lane/path")
+ require([copy_identity(row) for row in outputs[0]] ==
+ [copy_identity(row) for row in outputs[1]], "LAN copies differ")
+ for rows in outputs:
+ observed = coverage([((row["tcpseq"] - isn) & 0xffffffff, row["payload"])
+ for row in rows], payload)
+ require(observed == source_counts,
+ "lost/duplicate stream occurrence (including retransmission)")
+ # Missing aggregation is a prerequisite gap only after transfer and all other checks.
+ if not aggregate:
+ raise Skip("no sender GSO aggregate observed")
+ print(f"# TCP bytes={len(payload)} sent-occurrences={sum(source_counts)} "
+ f"aggregates={len(aggregate)} outputs={[len(rows) for rows in outputs]}", flush=True)
+
+
+def verify_udp(case, captures, expected, ports, vlans=(), expect_wrap=False,
+ input_order=False, producer_order=False):
+ """Check both lanes against the expected UDP payloads."""
+ lanes = []
+ for index, cap in enumerate(captures):
+ cap.validate()
+ rows = udp_rows(cap, ports)
+ require(Counter(row["payload"] for row in rows) == Counter(expected),
+ case + ": UDP payload/ID set differs")
+ if input_order:
+ require([row["payload"] for row in rows] == expected, case + ": input order differs")
+ if producer_order:
+ for producer in (1, 2):
+ observed = [row["payload"] for row in rows if row["payload"][6] == producer]
+ require(observed == [body for body in expected if body[6] == producer],
+ case + ": producer order differs")
+ for row in rows:
+ require(row["tag"] and row["size"] <= 1520 + 4 * len(vlans),
+ case + ": missing tag or unsegmented frame")
+ require(row["vlans"] == vlans, case + ": VLAN stack differs")
+ require(row["path"] == index, case + ": wrong HSR lane/path")
+ require(row["encoding"] == "hsr", case + ": wrong protocol tag kind")
+ protocol_order(rows, expect_wrap)
+ lanes.append(rows)
+ require([copy_identity(row) for row in lanes[0]] == [copy_identity(row) for row in lanes[1]],
+ case + ": unequal LAN copies")
+ print(f"# {case} expected={len(expected)} lanes={[len(rows) for rows in lanes]}",
+ flush=True)
+ return lanes
+
+
+def local_pair(lab, ns, dev):
+ """Create an up veth pair dev/devp inside one namespace."""
+ lab.veth(ns, dev, ns, dev + "p")
+ for name in (dev, dev + "p"):
+ lab.ip(ns, "link", "set", "dev", name, "up")
+
+
+def link_state(lab, ns, dev):
+ """Return the ip -j -d link description of a device."""
+ result = lab.ip(ns, "-j", "-d", "link", "show", "dev", dev)
+ rows = json.loads(result.stdout)
+ require(len(rows) == 1, f"{dev}: missing link description")
+ state = rows[0]
+ require(isinstance(state.get("ifindex"), int), "missing ifindex")
+ require(isinstance(state.get("promiscuity"), int),
+ f"{dev}: missing promiscuity count")
+ return state
+
+
+def gro_state(lab, ns, dev, expected, label):
+ """Require software GRO to be expected, as requested and as active."""
+ listing = lab.features(ns, dev)
+ require("generic-receive-offload" in listing,
+ f"{dev}: missing software GRO feature")
+ active, _, requested = listing["generic-receive-offload"]
+ require(active == expected and requested == expected,
+ f"{label}: {dev}: GRO expected {expected}, "
+ f"observed active {active} requested {requested}")
+
+
+def set_gro(lab, ns, dev, wanted, active=None):
+ """Request GRO; the read-back, not ethtool's output, is the oracle."""
+ result = lab.cmd(ns, "ethtool", "-K", dev, "gro", wanted, check=False)
+ # ethtool exits 1 when the kernel keeps a feature from changing.
+ require(result.returncode == 0 or (active is not None and result.returncode == 1),
+ f"{dev}: GRO setter rc{result.returncode} {result.stderr.strip()}")
+ gro_state(lab, ns, dev, active if active is not None else wanted,
+ "requested GRO " + wanted)
+
+
+def interlink_unsupported(stderr):
+ """True if iproute2 does not know the interlink option."""
+ error = stderr.lower()
+ return "interlink" in error and "rtnetlink" not in error and any(
+ word in error for word in ("unknown command", "unknown option",
+ "unknown argument", 'what is "interlink"'))
+
+
+def hsr_attach(lab, ns, master, first, second, interlink=None):
+ """Create an HSRv1 device over the given ports and bring it up."""
+ args = ["link", "add", "name", master, "type", "hsr",
+ "slave1", first, "slave2", second, "version", "1"]
+ if interlink is not None:
+ args.extend(("interlink", interlink))
+ result = lab.ip(ns, *args, check=False)
+ if result.returncode and interlink is not None:
+ # Tool-option absence needs a successful ordinary-HSR control.
+ control = lab.ip(ns, "link", "add", "ctl", "type", "hsr",
+ "slave1", first, "slave2", second,
+ "version", "1", check=False)
+ if control.returncode == 0:
+ lab.ip(ns, "link", "del", "ctl")
+ require(control.returncode == 0,
+ "interlink failed and ordinary HSR control failed: "
+ f"{result.stderr.strip()} / {control.stderr.strip()}")
+ if interlink_unsupported(result.stderr):
+ raise Skip("iproute2 has no interlink option")
+ require(result.returncode == 0,
+ "HSR attach failed: " + result.stderr.strip())
+ lab.ip(ns, "link", "set", "dev", master, "up")
+
+
+def policy(lab):
+ """All roles, initial preferences and last requests, twice each."""
+ ns = lab.ns()
+ for dev in ("m", "p", "q"):
+ local_pair(lab, ns, dev)
+ for role in ("A", "B", "interlink"):
+ for initial in ("on", "off"):
+ for last in ("on", "off"):
+ label = f"{role} initial={initial} last={last}"
+ before = link_state(lab, ns, "m")
+ for _ in range(2):
+ set_gro(lab, ns, "m", initial)
+ first = "m" if role == "A" else "p"
+ second = "m" if role == "B" else "q"
+ hsr_attach(lab, ns, "hsr0", first, second,
+ "m" if role == "interlink" else None)
+ gro_state(lab, ns, "m", "off", label + " after attach")
+ for wanted in ("on", "off", last):
+ set_gro(lab, ns, "m", wanted, active="off")
+ lab.ip(ns, "link", "del", "hsr0")
+ after = link_state(lab, ns, "m")
+ require("master" not in after,
+ label + ": upper remains after detach")
+ require(after["promiscuity"] == before["promiscuity"],
+ label + ": promiscuity changed after detach")
+ # The request was dropped while attached and stays off.
+ gro_state(lab, ns, "m", "off", label + " after detach")
+ set_gro(lab, ns, "m", "on")
+ print(f"# policy {label} cycles=2", flush=True)
+
+
+def master_gso(lab):
+ """The HSR master advertises no GSO types."""
+ ns = lab.ns()
+ for dev in ("a", "b"):
+ local_pair(lab, ns, dev)
+ hsr_attach(lab, ns, "hsr0", "a", "b")
+ listing = lab.features(ns, "hsr0")
+ names = [name for name in listing if
+ ("segmentation" in name or name in
+ ("tx-gso-robust", "tx-gso-partial", "tx-gso-list"))
+ and name != "generic-segmentation-offload"]
+ require(names, "master GSO type mask is absent from feature report")
+ enabled = [name for name in names if listing[name][0] != "off"]
+ require(not enabled, "master retains GSO types: " + ", ".join(enabled))
+ print("# master GSO types off: " + ", ".join(sorted(names)), flush=True)
+
+
+def rollback_bridge(lab):
+ """B's existing bridge causes A's earlier attachment to unwind."""
+ ns = lab.ns()
+ for dev in ("a", "b", "c"):
+ local_pair(lab, ns, dev)
+ for dev in ("a", "b"):
+ set_gro(lab, ns, dev, "on")
+ lab.ip(ns, "link", "add", "br0", "type", "bridge")
+ lab.ip(ns, "link", "set", "dev", "b", "master", "br0")
+ lab.ip(ns, "link", "set", "dev", "br0", "up")
+ before = {dev: link_state(lab, ns, dev) for dev in ("a", "b")}
+ result = lab.ip(ns, "link", "add", "hsr0", "type", "hsr",
+ "slave1", "a", "slave2", "b", "version", "1",
+ check=False)
+ require(result.returncode != 0, "second-port attachment unexpectedly passed")
+ require(RX_BUSY in result.stderr,
+ "missing busy RX-handler refusal: " + result.stderr.strip())
+ # B is refused before any state change.
+ refused = link_state(lab, ns, "b")
+ require(refused.get("master") == "br0" and
+ refused["promiscuity"] == before["b"]["promiscuity"],
+ "refused second port changed its bridge membership or promiscuity")
+ gro_state(lab, ns, "b", "on", "refused second port")
+ # A was a member until B was refused and leaves like any member.
+ after = link_state(lab, ns, "a")
+ require("master" not in after, "rolled-back A retains an upper")
+ require(after["promiscuity"] == before["a"]["promiscuity"],
+ "rolled-back A retains a promiscuity reference")
+ gro_state(lab, ns, "a", "off", "rolled-back A")
+ set_gro(lab, ns, "a", "on")
+ # This positive control detects a residual role or RX handler on A.
+ hsr_attach(lab, ns, "hsr1", "a", "c")
+ lab.ip(ns, "link", "del", "hsr1")
+ gro_state(lab, ns, "a", "off", "rollback positive control")
+ set_gro(lab, ns, "a", "on")
+ print("# bridge second-port rollback released A; reattach passed", flush=True)
+
+
+def multicast_flow(lab, ns, phase):
+ """Exact multicast receipt on both lower and macvlan, with no workers."""
+ captures = [lab.capture(ns, dev) for dev in ("m", "mv")]
+ sender = lab.sock(ns, socket.AF_PACKET, socket.SOCK_RAW, 0)
+ sender.bind(("mp", 0))
+ sender.setblocking(False)
+ source = bytes.fromhex(link_state(lab, ns, "mp")["address"].replace(":", ""))
+ prefix = f"hsr-macvlan-{phase}:".encode()
+ frames = [bytes.fromhex("01005e000009") + source + b"\x88\xb5" +
+ prefix + struct.pack("!H", index) + b"m" * 96
+ for index in range(50)]
+ for frame in frames:
+ require(sender.send(frame) == len(frame), "short multicast send")
+ lab.observe(0.002)
+ expected = Counter(frames)
+
+ def copies(capture):
+ return Counter(row["raw"] for row in capture.rows
+ if row["raw"][12:14] == b"\x88\xb5" and
+ row["raw"][14:].startswith(prefix))
+
+ # finish() establishes real empty queues and valid loss/truncation stats.
+ lab.finish(lambda: all(contains(copies(capture), expected) for capture in captures))
+ for dev, capture in zip(("m", "mv"), captures):
+ observed = copies(capture)
+ require(observed == expected,
+ f"{phase} {dev} multicast copies differ: "
+ f"missing={sum((expected - observed).values())} "
+ f"excess={sum((observed - expected).values())}")
+ print(f"# macvlan {phase} multicast lower=50/50 upper=50/50", flush=True)
+
+
+def kind_unavailable(stderr):
+ """True if ip link add failed because the device kind is not available."""
+ error = stderr.lower()
+ return "unknown device type" in error or "operation not supported" in error
+
+
+def rollback_macvlan(lab):
+ """A non-master upper occupies the RX handler; delivery survives."""
+ ns = lab.ns()
+ for dev in ("m", "p"):
+ local_pair(lab, ns, dev)
+ result = lab.ip(ns, "link", "add", "link", "m", "name", "mv",
+ "type", "macvlan", "mode", "bridge", check=False)
+ if result.returncode:
+ if kind_unavailable(result.stderr):
+ raise Skip("macvlan device kind unavailable: " + result.stderr.strip())
+ require(False, "macvlan creation failed: " + result.stderr.strip())
+ lab.ip(ns, "link", "set", "dev", "mv", "up")
+ lab.ip(ns, "maddress", "add", "01:00:5e:00:00:09", "dev", "mv")
+ set_gro(lab, ns, "m", "on")
+ before = link_state(lab, ns, "m")
+ require("master" not in before, "macvlan lower already has a master")
+
+ def association():
+ index = lab.cmd(ns, "cat", "/sys/class/net/mv/iflink").stdout.strip()
+ require(index == str(link_state(lab, ns, "m")["ifindex"]),
+ "macvlan no longer belongs to the original lower")
+ membership = lab.ip(ns, "maddress", "show", "dev", "mv").stdout.lower()
+ require("01:00:5e:00:00:09" in membership,
+ "macvlan multicast membership missing")
+
+ association()
+ multicast_flow(lab, ns, "before")
+ result = lab.ip(ns, "link", "add", "hsr0", "type", "hsr",
+ "slave1", "m", "slave2", "p", "version", "1",
+ check=False)
+ require(result.returncode != 0 and RX_BUSY in result.stderr,
+ "expected busy RX-handler refusal: " + result.stderr.strip())
+ after = link_state(lab, ns, "m")
+ require("master" not in after, "failed attachment left an HSR upper")
+ require(after["promiscuity"] == before["promiscuity"],
+ "failed attachment changed promiscuity")
+ # The lower is refused before any state change and keeps its request.
+ gro_state(lab, ns, "m", "on", "refused macvlan lower")
+ association()
+ multicast_flow(lab, ns, "after")
+
+
+def member_down_up(lab):
+ """Close and reopen a member attached with GRO requested."""
+ ns = lab.ns()
+ for dev in ("m", "p"):
+ local_pair(lab, ns, dev)
+ set_gro(lab, ns, "m", "on")
+ hsr_attach(lab, ns, "hsr0", "m", "p")
+ gro_state(lab, ns, "m", "off", "attached member")
+ # Each command must return within the bound of Lab.run().
+ lab.ip(ns, "link", "set", "dev", "m", "down")
+ lab.ip(ns, "link", "set", "dev", "m", "up")
+ lab.ip(ns, "link", "del", "hsr0")
+ gro_state(lab, ns, "m", "off", "detached member")
+ set_gro(lab, ns, "m", "on")
+ print("# attached member down/up returned; GRO re-enabled after detach", flush=True)
+
+
+def attach_down_up_detach(lab):
+ """Attach a down member with GRO requested, open it, detach while up."""
+ ns = lab.ns()
+ for dev in ("m", "p"):
+ local_pair(lab, ns, dev)
+ lab.ip(ns, "link", "set", "dev", "m", "down")
+ set_gro(lab, ns, "m", "on")
+ hsr_attach(lab, ns, "hsr0", "m", "p")
+ gro_state(lab, ns, "m", "off", "attached down member")
+ lab.ip(ns, "link", "set", "dev", "m", "up")
+ lab.ip(ns, "link", "del", "hsr0")
+ gro_state(lab, ns, "m", "off", "member detached while up")
+ lab.ip(ns, "link", "set", "dev", "m", "down")
+ lab.ip(ns, "link", "set", "dev", "m", "up")
+ set_gro(lab, ns, "m", "on")
+ print("# down member attached, opened, detached up; GRO re-enabled", flush=True)
+
+
+def request_while_attached(lab):
+ """A GRO request on an attached member is dropped, also after detach."""
+ ns = lab.ns()
+ for dev in ("m", "p"):
+ local_pair(lab, ns, dev)
+ set_gro(lab, ns, "m", "off")
+ hsr_attach(lab, ns, "hsr0", "m", "p")
+ set_gro(lab, ns, "m", "on", active="off")
+ lab.ip(ns, "link", "del", "hsr0")
+ gro_state(lab, ns, "m", "off", "member after detach")
+ print("# GRO request while attached dropped; stays off after detach", flush=True)
+
+
+PORT_ERROR = "Cannot use a bridge, bond, team, OVS, macvlan or MACsec device as HSR port"
+RX_BUSY = "RX handler busy"
+NO_RX_HANDLER = "does not support an RX handler"
+
+
+def port_types(lab):
+ """Forwarding devices are refused as slave or interlink, by type."""
+ ns = lab.ns()
+ for dev in ("p", "q", "l", "s"):
+ local_pair(lab, ns, dev)
+ lab.ip(ns, "link", "add", "br0", "type", "bridge")
+ candidates, refused, skipped = [("bridge", "br0", PORT_ERROR)], [], []
+ for kind, dev, args, error in (
+ ("bond", "bond0", ("bond0", "type", "bond"), PORT_ERROR),
+ ("macvlan", "mv0", ("link", "l", "name", "mv0", "type", "macvlan",
+ "mode", "bridge"), PORT_ERROR),
+ ("ipvlan", "ipv0", ("link", "s", "name", "ipv0", "type", "ipvlan",
+ "mode", "l2"), NO_RX_HANDLER)):
+ result = lab.ip(ns, "link", "add", *args, check=False)
+ if not result.returncode:
+ candidates.append((kind, dev, error))
+ elif kind_unavailable(result.stderr):
+ skipped.append(f"{kind} device kind unavailable")
+ else:
+ require(False, f"{kind} creation failed: {result.stderr.strip()}")
+ before = {dev: link_state(lab, ns, dev)
+ for dev in ("p", "q", *(dev for _, dev, _ in candidates))}
+ for kind, dev, error in candidates:
+ for role, ports in (("slave1", ("slave1", dev, "slave2", "p")),
+ ("slave2", ("slave1", "p", "slave2", dev)),
+ ("interlink", ("slave1", "p", "slave2", "q", "interlink", dev))):
+ label = f"{kind} as {role}"
+ result = lab.ip(ns, "link", "add", "name", "hsr0", "type", "hsr", *ports,
+ "version", "1", check=False)
+ require(result.returncode != 0, label + ": accepted")
+ if role == "interlink" and interlink_unsupported(result.stderr):
+ skipped.append(label + ": iproute2 has no interlink option")
+ continue
+ require(error in result.stderr,
+ f"{label}: missing refusal message: {result.stderr.strip()}")
+ require(lab.ip(ns, "link", "show", "dev", "hsr0", check=False).returncode,
+ label + ": HSR device left behind")
+ for name, state in before.items():
+ after = link_state(lab, ns, name)
+ require("master" not in after and
+ after["promiscuity"] == state["promiscuity"],
+ f"{label}: {name} changed by the refused attach")
+ refused.append(label)
+ print("# refused: " + ", ".join(refused), flush=True)
+ if skipped:
+ raise Skip("; ".join(skipped) + "; refused: " + ", ".join(refused))
+
+
+GSO_ROWS = (
+ ("plain", (), False, "A"),
+ ("s-tag-inband", ((0x88a8, 100),), True, "A"),
+ ("c-tag-accelerated", ((0x8100, 50),), False, "A"),
+ ("s-tag-accelerated", ((0x88a8, 100),), False, "B"),
+ ("sc-nested-accelerated", ((0x88a8, 200), (0x8100, 30)), False, "B"),
+ ("cc-nested-inband", ((0x8100, 40), (0x8100, 20)), True, "A"),
+)
+
+
+def vlan_chain(lab, ns, parent, stack, prefix):
+ """Stack VLAN devices on parent; return the innermost one."""
+ for depth, (protocol, identifier) in enumerate(stack):
+ dev = f"{prefix}{depth}"
+ lab.ip(ns, "link", "add", "link", parent, "name", dev,
+ "type", "vlan", "id", str(identifier), "protocol",
+ "802.1AD" if protocol == 0x88a8 else "802.1Q")
+ lab.ip(ns, "link", "set", "dev", dev, "up")
+ parent = dev
+ return parent
+
+
+def vlan_tx_mode(lab, ns, lower, stack, inband):
+ """Select accelerated or in-band insertion of the outer VLAN tag."""
+ if not stack:
+ return
+ aliases = (("tx-vlan-stag-hw-insert", "tx-vlan-stag-offload")
+ if stack[0][0] == 0x88a8 else
+ ("tx-vlan-offload", "tx-vlan-hw-insert"))
+ listing = lab.features(ns, lower)
+ name = next((alias for alias in aliases if alias in listing), None)
+ if name is None:
+ raise Skip("no VLAN insertion feature: " + ", ".join(aliases))
+ wanted = "off" if inband else "on"
+ if listing[name][1] and listing[name][0] != wanted:
+ raise Skip("fixed VLAN insertion feature cannot be " + wanted)
+ lab.cmd(ns, "ethtool", "-K", lower, name, wanted)
+ require(lab.features(ns, lower)[name][0] == wanted,
+ "VLAN insertion mode did not match its requested state")
+
+
+def ordered_payload(producer, index, size=64):
+ """Payload naming its producer and index."""
+ prefix = b"HSRORD" + bytes([producer]) + struct.pack("!I", index)
+ return prefix + bytes([producer ^ 0xA5]) * (size - len(prefix))
+
+
+def ordered_socket(lab, ns, address):
+ """UDP multicast sender bound to address."""
+ sock = lab.sock(ns, socket.AF_INET, socket.SOCK_DGRAM)
+ sock.bind((address, 0))
+ sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 1)
+ sock.settimeout(1)
+ return sock
+
+
+def ordered_lanes(lab, topology):
+ """Capture the frames the DUT transmits on its two LAN ports."""
+ return [lab.capture(topology["dut"], lane, pkttype=4)
+ for lane in ("da", "db")]
+
+
+def ordered_send(lab, sock, producer, first, count, port, segment=0):
+ """Send count datagrams, or UDP GSO aggregates of 16; return the payloads."""
+ expected = []
+ if segment:
+ sock.setsockopt(socket.IPPROTO_UDP, 103, segment)
+ blocks = 16 if segment else 1
+ size = segment or 64
+ for number in range(count):
+ payloads = [ordered_payload(producer, first + number * blocks + n,
+ size) for n in range(blocks)]
+ body = b"".join(payloads)
+ require(sock.sendto(body, ("239.0.0.9", port)) == len(body),
+ "short UDP send")
+ expected.extend(payloads)
+ # Keep capture receive queues bounded throughout long wrap warmups.
+ if number % 8 == 0:
+ lab.observe(0)
+ lab.observe(0.1)
+ return expected
+
+
+def udp_rows(cap, ports):
+ """Captured UDP frames addressed to one of ports."""
+ return [row for row in cap.rows if row["ports"] and row["ports"][1] in ports]
+
+
+def lanes_hold(caps, expected, ports):
+ """Completion condition: every lane captured each expected payload."""
+ wanted = Counter(expected)
+ return lambda: all(contains(Counter(row["payload"] for row in udp_rows(cap, ports)), wanted)
+ for cap in caps)
+
+
+def ordered_data(lab, gso=False):
+ """Local data around interlink data or aggregates keeps its order."""
+ topology = topo(lab)
+ caps = ordered_lanes(lab, topology)
+ local = ordered_socket(lab, topology["dut"], "100.64.3.1")
+ san = ordered_socket(lab, topology["tx"], "100.64.2.1")
+ # Observe interlink aggregates before the two tagged LAN copies expand.
+ sent = lab.capture(topology["tx"], "t0", pkttype=4)
+ ingress = lab.capture(topology["dut"], "dil")
+ expected = ordered_send(lab, local, 1, 0, 32, 5100)
+ expected += ordered_send(lab, san, 2, 0, 16, 5101, segment=512 if gso else 0)
+ expected += ordered_send(lab, local, 1, 32, 32, 5100)
+ lanes_done = lanes_hold(caps, expected, {5100, 5101})
+ lab.finish(lambda: lanes_done() and
+ len(udp_rows(ingress, {5101})) >= len(udp_rows(sent, {5101})) > 0)
+ verify_udp("local and aggregate boundaries", caps, expected,
+ ports={5100, 5101}, input_order=True)
+ inputs = udp_rows(sent, {5101})
+ if gso:
+ expected_inputs = [b"".join(ordered_payload(2, n * 16 + part, 512)
+ for part in range(16)) for n in range(16)]
+ else:
+ expected_inputs = expected[32:48]
+ aggregated = any(len(row["payload"]) > 1500 for row in inputs)
+ if gso and not aggregated:
+ expected_inputs = expected[32:-32]
+ require([row["payload"] for row in inputs] == expected_inputs,
+ "UDP sender input set/order differs")
+ incoming = udp_rows(ingress, {5101})
+ require([row["payload"] for row in incoming] == expected_inputs,
+ "actual UDP ingress input set/order differs")
+ require(all(row["encoding"] == "plain" and not row["vlans"] for row in inputs + incoming),
+ "UDP input classification/VLAN differs")
+ if gso and not aggregated:
+ raise Skip("interlink UDP GSO aggregate not observed")
+
+
+def ordered_supervision(lab):
+ """Local and proxy supervision frames are numbered with the data."""
+ topology = topo(lab)
+ caps = ordered_lanes(lab, topology)
+ san = ordered_socket(lab, topology["tx"], "100.64.2.1")
+ expected = ordered_send(lab, san, 3, 0, 8, 5110)
+ # Both timers have time for multiple life checks; no fake worker state.
+ lab.observe(10)
+ data_done = lanes_hold(caps, expected, {5110})
+ macs = (topology["mac_master"], topology["mac_tx"])
+ lab.finish(lambda: data_done() and all(
+ sum(row["supervision"] == mac for row in cap.rows) >= 2
+ for cap in caps for mac in macs))
+ verify_udp("proxy-learning inputs", caps, expected, ports={5110},
+ input_order=True)
+ verify_supervision(caps, topology["mac_master"], topology["mac_tx"], {5110})
+
+
+def ordered_concurrent(lab):
+ """Master and interlink producers on two CPUs keep per-producer order."""
+ cpus = sorted(os.sched_getaffinity(0))[:2]
+ if len(cpus) != 2:
+ raise Skip("two functional producers need two allowed CPUs")
+ topology = topo(lab)
+ caps = ordered_lanes(lab, topology)
+ sockets = [ordered_socket(lab, topology["tx"], "100.64.2.1"),
+ ordered_socket(lab, topology["dut"], "100.64.3.1")]
+ barrier = threading.Barrier(3)
+ stop = threading.Event()
+ errors = []
+ windows = [(), ()]
+ until = min(lab.deadline, time.monotonic() + 15)
+
+ def producer(index):
+ try:
+ # Socket namespace binding was completed before threads existed.
+ os.sched_setaffinity(0, {cpus[index]})
+ barrier.wait(timeout=2)
+ start = time.monotonic()
+ for number in range(128):
+ require(not stop.is_set() and not lab.stop.is_set() and
+ time.monotonic() < until, "producer work deadline")
+ body = ordered_payload(index + 1, number)
+ require(sockets[index].sendto(body, ("239.0.0.9", 5120 + index)) == len(body),
+ "short producer send")
+ time.sleep(0.001)
+ windows[index] = (start, time.monotonic())
+ except (OSError, ValueError, RuntimeError) as error:
+ errors.append(error)
+ stop.set()
+ barrier.abort()
+
+ threads = [threading.Thread(target=producer, args=(index,))
+ for index in range(2)]
+ unfinished = []
+ try:
+ for thread in threads:
+ lab.threads.append(thread)
+ thread.start()
+ try:
+ barrier.wait(timeout=2)
+ except threading.BrokenBarrierError:
+ # A producer that failed broke the barrier: report its error.
+ if errors:
+ raise errors[0] from None
+ raise
+ while any(thread.is_alive() for thread in threads):
+ require(time.monotonic() < until, "producer join deadline")
+ lab.observe(0.01)
+ finally:
+ stop.set()
+ barrier.abort()
+ join_end = min(lab.deadline, time.monotonic() + 2)
+ for thread in threads:
+ if thread.ident is not None:
+ thread.join(max(0, join_end - time.monotonic()))
+ unfinished = [thread for thread in threads if thread.is_alive()]
+ if unfinished:
+ print("# cleanup failure: sender thread remained alive")
+ if errors:
+ raise errors[0]
+ require(not unfinished, "sender thread remained alive")
+ require(all(windows) and windows[0][0] < windows[1][1] and
+ windows[1][0] < windows[0][1], "producer send windows do not overlap")
+ print(f"# functional producers CPUs={cpus} windows={windows}")
+ expected = [ordered_payload(producer, number)
+ for producer in (1, 2) for number in range(128)]
+ lab.finish(lanes_hold(caps, expected, {5120, 5121}))
+ verify_udp("concurrent master/interlink inputs", caps, expected,
+ ports={5120, 5121}, producer_order=True)
+
+
+def ordered_wrap(lab):
+ """The interlink sequence number wraps with both lanes intact."""
+ topology = topo(lab)
+ caps = ordered_lanes(lab, topology)
+ sender = ordered_socket(lab, topology["tx"], "100.64.2.1")
+ probe = ordered_send(lab, sender, 4, 0, 1, 5130)
+ # This only selects the warmup count; final loss-free captures certify it.
+ rows = [udp_rows(cap, {5130}) for cap in caps]
+ require(all(len(lane) == 1 for lane in rows) and
+ rows[0][0]["seq"] == rows[1][0]["seq"], "wrap probe missing or unequal")
+ start = rows[0][0]["seq"]
+ warm_count = (65400 - ((start + 1) & 0xFFFF)) & 0xFFFF
+ warm = ordered_send(lab, sender, 5, 0, warm_count, 5131)
+ # Separate IDs/ports prevent sequence-space reuse from hiding a loss.
+ expected = ordered_send(lab, sender, 6, 0, 256, 5132)
+ lab.finish(lanes_hold(caps, probe + warm + expected, {5130, 5131, 5132}))
+ verify_udp("wrap start", caps, probe, ports={5130}, input_order=True)
+ if warm:
+ verify_udp("wrap warmup", caps, warm, ports={5131}, input_order=True)
+ verify_udp("actual protocol wrap", caps, expected, ports={5132},
+ expect_wrap=True, input_order=True)
+
+
+ORDERED_CASES = (("local-data-order", ordered_data),
+ ("local-data-aggregate-boundaries", lambda lab: ordered_data(lab, True)),
+ ("local-proxy-supervision", ordered_supervision),
+ ("two-functional-producers", ordered_concurrent),
+ ("actual-sequence-wrap", ordered_wrap))
+
+
+def topo(lab):
+ """SAN - interlink - HSR DUT - two LANs - receiver namespaces."""
+ tx, dut, rx = lab.ns(), lab.ns(), lab.ns()
+ for ns, dev, peer_ns, peer in ((tx, "t0", dut, "dil"),
+ (dut, "da", rx, "ra"), (dut, "db", rx, "rb")):
+ lab.veth(ns, dev, peer_ns, peer)
+ for owner, link in ((ns, dev), (peer_ns, peer)):
+ lab.ip(owner, "link", "set", "dev", link, "up")
+ hsr_attach(lab, dut, "hm", "da", "db", "dil")
+ attributes = link_state(lab, dut, "hm")["linkinfo"]["info_data"]
+ require(attributes.get("proto") == 0 and attributes.get("version") == 1 and
+ all(attributes.get(key) == value for key, value in
+ (("slave1", "da"), ("slave2", "db"), ("interlink", "dil"))),
+ "HSR topology has unexpected protocol/version")
+ lab.ip(tx, "addr", "add", "100.64.2.1/24", "dev", "t0")
+ lab.ip(dut, "addr", "add", "100.64.3.1/24", "dev", "hm")
+ for ns, dev in ((tx, "t0"), (dut, "hm")):
+ lab.ip(ns, "route", "add", "239.0.0.0/8", "dev", dev)
+ return {"tx": tx, "dut": dut, "rx": rx, "master": "hm",
+ "mac_master": bytes.fromhex(link_state(lab, dut, "hm")["address"].replace(":", "")),
+ "mac_tx": bytes.fromhex(link_state(lab, tx, "t0")["address"].replace(":", ""))}
+
+
+def verify_supervision(captures, master, proxy, ports):
+ """Check local and proxy supervision TLVs and their order on both lanes."""
+ tagged, observed = [], []
+ for lane, cap in enumerate(captures):
+ cap.validate()
+ frames = [row for row in cap.rows if row["encoding"] == "hsr"]
+ protocol_order(frames)
+ # Only this case's frames are compared between the lanes; the
+ # namespaces also send IPv6 neighbor discovery and MLD frames.
+ tagged.append([row for row in frames if row["supervision"] is not None or
+ (row["ports"] and row["ports"][1] in ports)])
+ rows = [row for row in cap.rows if row["supervision"] is not None]
+ require(all(row["encoding"] == "hsr" and row["path"] == lane for row in rows),
+ "wrong supervision tag kind/path")
+ require(all(row["supervision"] in (master, proxy) for row in rows),
+ "unexpected supervised MAC")
+ for mac in (master, proxy):
+ group = [row for row in rows if row["supervision"] == mac]
+ require(len(group) >= 2, "missing repeated local/proxy supervision TLV")
+ protocol_order(group, key="supseq")
+ protocol_order(rows)
+ observed.append(rows)
+ # The two lanes are cut off one after the other while supervision goes
+ # on: compare the common prefix; a lane may end in extra supervision.
+ for lanes, what in ((observed, "supervision"), (tagged, "data/supervision")):
+ common = min(len(rows) for rows in lanes)
+ require([copy_identity(row) for row in lanes[0][:common]] ==
+ [copy_identity(row) for row in lanes[1][:common]],
+ what + " LAN order differs")
+ # The lanes are cut off one synchronize_net() apart. In that window
+ # the two supervision sources (the local node and the proxied SAN),
+ # each sending every two seconds, add at most one frame each.
+ tail = [row for rows in lanes for row in rows[common:]]
+ require(len(tail) <= 2 and all(row["supervision"] is not None for row in tail),
+ what + " LAN copies differ beyond the capture cutoff")
+ print(f"# supervision exact local/proxy TLV sets lanes={[len(rows) for rows in observed]}",
+ flush=True)
+
+
+def tcp_transfer(lab, sender_ns, receiver_ns, dev, source, destination, port):
+ """Send a stream and a reply; the caller completes its captures."""
+ payload = bytes(range(256)) * 512
+ reply = b"HSR-TCP-REPLY"
+ listener = lab.sock(receiver_ns, socket.AF_INET, socket.SOCK_STREAM)
+ listener.bind((destination, port))
+ listener.listen(1)
+ client = lab.sock(sender_ns, socket.AF_INET, socket.SOCK_STREAM)
+ client.setsockopt(socket.SOL_SOCKET, socket.SO_BINDTODEVICE, dev.encode() + b"\0")
+ client.bind((source, 0))
+ client.connect((destination, port))
+ server, _ = listener.accept()
+ lab.sockets.append(server)
+ for sock in (client, server):
+ sock.setblocking(False)
+ sent, received, echoed = 0, bytearray(), bytearray()
+ state = {"reply": False, "write_closed": False}
+ end = min(lab.deadline, time.monotonic() + 12)
+
+ def pump():
+ nonlocal sent
+ if sent < len(payload):
+ try:
+ sent += client.send(payload[sent:sent + 65536])
+ except BlockingIOError:
+ pass
+ elif not state["write_closed"]:
+ client.shutdown(socket.SHUT_WR)
+ state["write_closed"] = True
+ try:
+ data = server.recv(65536)
+ received.extend(data)
+ except BlockingIOError:
+ pass
+ if len(received) == len(payload) and not state["reply"]:
+ require(server.send(reply) == len(reply), "short TCP reply")
+ state["reply"] = True
+ try:
+ echoed.extend(client.recv(65536))
+ except BlockingIOError:
+ pass
+
+ while len(echoed) < len(reply) and time.monotonic() < end:
+ lab.observe(0.01, pump)
+ require(bytes(received) == payload and bytes(echoed) == reply,
+ "TCP application did not receive complete expected bytes")
+ for sock in (server, client, listener):
+ sock.close()
+ return payload, reply
+
+
+def fin_seen(caps, port):
+ """Completion condition: each capture saw a FIN of the connection."""
+ return lambda: all(any(port in row["ports"] and row.get("tcpflags", 0) & 1
+ for row in cap.rows) for cap in caps)
+
+
+def tcp_interlink(lab):
+ """A natural TCP stream from the interlink reaches both LANs intact."""
+ topology = topo(lab)
+ tx, dut, rx = (topology[key] for key in ("tx", "dut", "rx"))
+ lab.ip(rx, "link", "add", "hr", "type", "hsr",
+ "slave1", "ra", "slave2", "rb", "version", "1")
+ lab.ip(rx, "link", "set", "dev", "hr", "up")
+ lab.ip(tx, "addr", "add", "10.67.0.2/24", "dev", "t0")
+ lab.ip(rx, "addr", "add", "10.67.0.3/24", "dev", "hr")
+ lab.cmd(tx, "ethtool", "-K", "t0", "tso", "on", "gso", "on")
+ caps = [lab.capture(tx, "t0", pkttype=4)] + ordered_lanes(lab, topology)
+ ingress = lab.capture(dut, "dil")
+ payload, _ = tcp_transfer(lab, tx, rx, "t0", "10.67.0.2", "10.67.0.3", 5200)
+ lab.finish(fin_seen(caps + [ingress], 5200))
+ verify_stream(caps, 5200, payload, ingress=ingress)
+
+
+def tcp_prp(lab, row):
+ """TCP GSO from a SAN on one PRP LAN is delivered locally, segmented."""
+ name, stack, inband, side = row
+ tx, dut = lab.ns(), lab.ns()
+ for lane in ("a", "b"):
+ lab.veth(tx, "t" + lane, dut, "d" + lane)
+ lab.ip(tx, "link", "set", "dev", "t" + lane, "up")
+ lab.ip(dut, "link", "set", "dev", "d" + lane, "up")
+ lab.ip(dut, "link", "add", "pm", "type", "hsr", "slave1", "da", "slave2", "db", "proto", "1")
+ lab.ip(dut, "link", "set", "dev", "pm", "up")
+ attributes = link_state(lab, dut, "pm")["linkinfo"]["info_data"]
+ require(attributes.get("proto") == 1 and
+ attributes.get("slave1") == "da" and attributes.get("slave2") == "db",
+ "PRP topology has unexpected protocol/ports")
+ lower = "t" + side.lower()
+ sender = vlan_chain(lab, tx, lower, stack, "txv")
+ receiver = vlan_chain(lab, dut, "pm", stack, "rxv")
+ lab.ip(tx, "addr", "add", "10.68.0.2/24", "dev", sender)
+ lab.ip(dut, "addr", "add", "10.68.0.1/24", "dev", receiver)
+ vlan_tx_mode(lab, tx, lower, stack, inband)
+ lab.cmd(tx, "ethtool", "-K", lower, "tso", "on", "gso", "on")
+ caps = [lab.capture(tx, lower, pkttype=4), lab.capture(dut, "pm", pkttype=0)]
+ ingress = lab.capture(dut, "d" + side.lower(), prp=True)
+ lanes = [lab.capture(dut, dev, pkttype=4, prp=True) for dev in ("da", "db")]
+ payload, reply = tcp_transfer(lab, tx, dut, sender, "10.68.0.2", "10.68.0.1", 5201)
+ lab.finish(fin_seen(caps + lanes + [ingress], 5201))
+ verify_stream(caps, 5201, payload, stack, local=True, inband=inband, ingress=ingress)
+ copies = []
+ for index, cap in enumerate(lanes):
+ cap.validate()
+ rows = [frame for frame in cap.rows if frame["ports"] and
+ frame["ports"][0] == 5201 and frame["payload"]]
+ require(len(rows) == 1 and rows[0]["payload"] == reply,
+ "PRP reply not exactly once per LAN")
+ require(rows[0]["tag"] and rows[0]["vlans"] == stack and
+ rows[0]["size"] <= 1520 + 4 * len(stack), "invalid PRP reply tag/size/VLAN")
+ require(rows[0]["path"] == 10 + index, "wrong PRP LAN identifier")
+ require(rows[0]["encoding"] == "prp", "wrong PRP reply tag kind")
+ copies.append(copy_identity(rows[0]))
+ require(copies[0] == copies[1], "PRP reply LAN copies differ")
+ print(f"# PRP LAN {name} local data segmented; reply each LAN=1/1", flush=True)
+
+
+def interrupt(signum, _frame):
+ """INT/TERM handler: record the signal for Lab.remaining()."""
+ SIGNALLED.append(signum)
+
+
+def main():
+ """Run the gro or ordered cases as TAP."""
+ mode = sys.argv[1] if len(sys.argv) == 2 else ""
+ require(mode in ("gro", "ordered"), "usage: hsr_gro_superpacket.py gro|ordered")
+ cases = [("GRO policy", policy), ("master GSO mask", master_gso),
+ ("bridge rollback", rollback_bridge), ("busy-handler rollback", rollback_macvlan),
+ ("member down/up while attached", member_down_up),
+ ("attach down, up, detach while up", attach_down_up_detach),
+ ("GRO request while attached", request_while_attached),
+ ("port type rejection", port_types),
+ ("natural TCP interlink", tcp_interlink)]
+ cases += [("PRP LAN " + row[0], lambda lab, row=row: tcp_prp(lab, row)) for row in GSO_ROWS]
+ if mode == "ordered":
+ cases = ORDERED_CASES
+ print(f"TAP version 13\n1..{len(cases)}", flush=True)
+ missing = [tool for tool in ("ip", "ethtool") if shutil.which(tool) is None]
+ if os.geteuid() or missing:
+ for number, (name, _) in enumerate(cases, 1):
+ print(f"ok {number} - {name} # SKIP need root/ip/ethtool {missing}", flush=True)
+ return 4
+ with signals_blocked():
+ for sig in (signal.SIGINT, signal.SIGTERM):
+ signal.signal(sig, interrupt)
+ lab = Lab(time.monotonic() + 120)
+ failure, passed = None, 0
+ try:
+ for number, (name, function) in enumerate(cases, 1):
+ try:
+ lab.begin()
+ function(lab)
+ except Skip as error:
+ print(f"ok {number} - {name} # SKIP {error}", flush=True)
+ continue
+ except CASE_ERRORS as error:
+ failure = error
+ print(f"not ok {number} - {name}: {error}", flush=True)
+ for later, (other, _) in enumerate(cases[number:], number + 1):
+ print(f"not ok {later} - {other}: stopped after prior failure", flush=True)
+ break
+ print(f"ok {number} - {name}", flush=True)
+ passed += 1
+ finally:
+ with signals_blocked():
+ for sig in (signal.SIGINT, signal.SIGTERM):
+ signal.signal(sig, signal.SIG_IGN)
+ try:
+ lab.close()
+ except CASE_ERRORS as error:
+ print("# cleanup failure: " + str(error), flush=True)
+ failure = failure or error
+ return 1 if failure is not None else (0 if passed else 4)
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh b/tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh
new file mode 100755
index 000000000000..9da711ad2ab2
--- /dev/null
+++ b/tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh
@@ -0,0 +1,9 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+if ! command -v python3 > /dev/null; then
+ echo "SKIP: missing python3"
+ exit 4
+fi
+
+exec python3 -B "$(dirname "$0")/hsr_gro_superpacket.py" gro "$@"
diff --git a/tools/testing/selftests/net/hsr/hsr_ordered_forwarding.sh b/tools/testing/selftests/net/hsr/hsr_ordered_forwarding.sh
new file mode 100755
index 000000000000..b43218709455
--- /dev/null
+++ b/tools/testing/selftests/net/hsr/hsr_ordered_forwarding.sh
@@ -0,0 +1,9 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+if ! command -v python3 > /dev/null; then
+ echo "SKIP: missing python3"
+ exit 4
+fi
+
+exec python3 -B "$(dirname "$0")/hsr_gro_superpacket.py" ordered "$@"
--
2.53.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net v8 1/4] net: hsr: keep GRO disabled on HSR/PRP ports
2026-11-07 22:05 ` [PATCH net v8 1/4] net: hsr: keep GRO disabled on HSR/PRP ports Xin Xie
@ 2026-11-08 22:08 ` netdev-bot+sashiko
0 siblings, 0 replies; 10+ messages in thread
From: netdev-bot+sashiko @ 2026-11-08 22:08 UTC (permalink / raw)
To: xiexinet
Cc: netdev, linux-kselftest, linux-kernel, davem, edumazet, kuba,
pabeni, horms, andrew+netdev, shuah, kees, petr.wozniak,
qingfang.deng, fmaurer, luka.gejak, bigeasy, xiaoliang.yang_1,
skhawaja, liuhangbin, stable, sdf.kernel, lukma, fmancera,
danishanwar
Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider.
Critical: 0 · High: 0 · Medium: 2 · Low: 0
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] hsr_portdev_setup() (net/hsr/hsr_slave.c:210) calls
hsr_portdev_role_set() on every non-master port without checking…
- [Medium] hsr_portdev_role_set() calls netdev_change_features() with
dev->hsr_port set.
--
net: hsr: keep GRO disabled on HSR/PRP ports
This patch adds an hsr_port role bit to struct net_device. The bit is set
on HSR/PRP slave A/B and interlink ports. While it is set,
__netdev_update_features() drops software GRO and configurable GRO_HW
from both wanted_features and the computed features, and the attach
fails if software GRO stays active.
> diff --git a/net/core/dev.c b/net/core/dev.c
> index df1a840b03343..2916b17f4169f 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -11269,6 +11269,21 @@ int __netdev_update_features(struct net_device *dev)
>
> features = netdev_get_wanted_features(dev);
>
> + /* A direct HSR/PRP port needs per-frame metadata. Drop software
> + * GRO and configurable GRO_HW from the request as well as from the
> + * result, as the core does for upper-disabled features: drivers
> + * such as veth derive state from wanted_features and need the
> + * requested and the active bit to agree. Fixed-on or
> + * driver-required GRO_HW is left to the driver.
> + */
> + if (dev->hsr_port) {
> + netdev_features_t gro = NETIF_F_GRO |
> + (dev->hw_features & NETIF_F_GRO_HW);
> +
> + dev->wanted_features &= ~gro;
> + features &= ~gro;
> + }
> +
> if (dev->netdev_ops->ndo_fix_features)
> features = dev->netdev_ops->ndo_fix_features(dev, features);
>
> diff --git a/net/hsr/hsr_slave.c b/net/hsr/hsr_slave.c
> index a546f70f9cc8b..bc340ae0b52e1 100644
> --- a/net/hsr/hsr_slave.c
> +++ b/net/hsr/hsr_slave.c
[ ... ]
> +static void hsr_portdev_role_clear(struct net_device *dev)
> +{
> + netdev_lock_ops(dev);
> + dev->hsr_port = false;
> + /* Recompute features without the role. GRO was dropped from
> + * the request while the role was set and stays off; the user
> + * can turn it back on with ethtool. A dying device just drops
> + * the role, a netns move stays registered and is recomputed.
> + */
> + if (dev->reg_state == NETREG_REGISTERED)
> + netdev_change_features(dev);
> + netdev_unlock_ops(dev);
> +}
[ ... ]
> @@ -169,6 +199,22 @@ static int hsr_portdev_setup(struct hsr_priv *hsr, struct net_device *dev,
> if (res)
> goto fail_upper_dev_link;
>
> + /* Enable the member role and recompute before the RX handler
> + * is published, so that software GRO on this device is off
> + * before frames can arrive. This covers the direct member
> + * only: a stacked member receives from a lower device that
> + * may still run GRO and can deliver merged frames as GSO skbs.
> + * A still-active software GRO rejects the port, a remaining
> + * GRO_HW alone does not.
> + */
> + hsr_portdev_role_set(dev);
> + if (dev->features & NETIF_F_GRO) {
[Severity: Medium]
The role is set here for every non-master port. Nothing checks
prot_version, port->type or hsr->fwd_offloaded first. Is GRO actually a
problem on all of these ports?
On PRP slave A/B, frames that carry an RCT trailer do not appear to be
merged. inet_gro_receive() flushes when the IP tot_len does not match
skb_gro_len():
net/ipv4/af_inet.c:inet_gro_receive() {
...
flush = (u16)((ntohl(*(__be32 *)iph) ^ skb_gro_len(skb)) | (ntohl(*(__be32 *)&iph->id) & ~IP_DF));
...
}
prp_is_lan_dup() also drops A/B forwarding. Untagged LAN traffic goes to
the master or interlink untagged, the same way a bridge forwards GRO
skbs.
Some slaves have HW forwarding offload (NETIF_F_HW_HSR_FWD, for example
icssg, ksz9477 and xrs700x). For those, hsr_drop_frame() does:
net/hsr/hsr_forward.c:hsr_drop_frame() {
if (port->dev->features & NETIF_F_HW_HSR_FWD)
return prp_is_lan_dup(frame->port_rcv->type, port);
...
}
so merged skbs are never re-tagged towards the other slave.
There is also no GRO offload for ETH_P_HSR or ETH_P_PRP. Outside
net/hsr, the only references are in flow_dissector, so tagged frames
cannot be merged. The per-frame metadata problem seems to exist only
where untagged frames get tagged: the interlink, and SAN frames
forwarded between non-offloaded HSR slaves.
Because wanted_features is cleared and not restored when the port
leaves, PRP ports and HW-offloaded ports lose software GRO for good.
Host-bound receive then costs more CPU and gets less throughput. The
commit message does not mention this cost. The Fixes: tag would also
bring this behaviour change into stable kernels.
Could the role be limited to the ports where GRO actually breaks
per-frame tagging?
> + NL_SET_ERR_MSG_MOD(extack,
> + "software GRO still on after feature update, cannot join");
> + res = -EBUSY;
> + goto fail_role_policy;
> + }
> +
> res = netdev_rx_handler_register(dev, hsr_handle_frame, port);
> if (res)
> goto fail_rx_handler;
> @@ -177,7 +223,12 @@ static int hsr_portdev_setup(struct hsr_priv *hsr, struct net_device *dev,
> return 0;
>
> fail_rx_handler:
> +fail_role_policy:
> + /* Rollback order: unlink the upper, clear the role and
> + * recompute, then undo this setup's promiscuous increment.
> + */
> netdev_upper_dev_unlink(dev, hsr_dev);
> + hsr_portdev_role_clear(dev);
[Severity: Medium]
Can a failed attach leave GRO disabled on a device that never became an
HSR member?
hsr_portdev_role_set() leads to __netdev_update_features(), which clears
NETIF_F_GRO (and configurable GRO_HW) from dev->wanted_features. This
happens before the attach is known to succeed. The previous request is
not saved anywhere. hsr_portdev_role_clear() only recomputes from the
reduced wanted_features.
This rollback undoes the upper link, the role bit and promiscuity, but
not the GRO request. The -EBUSY check above can fire, for example, when
ndo_set_features() fails. In that case __netdev_update_features()
returns early with dev->features unchanged, but wanted_features has
already been cleared.
A second path goes through hsr_del_port(). hsr_dev_finalize() attaches
the ports one at a time:
net/hsr/hsr_device.c:hsr_dev_finalize() {
...
res = hsr_add_port(hsr, slave[0], HSR_PT_SLAVE_A, extack);
if (res)
goto err_unregister;
res = hsr_add_port(hsr, slave[1], HSR_PT_SLAVE_B, extack);
if (res)
goto err_unregister;
...
}
Slave B or the interlink can fail, for example in hsr_check_dev_ok() or
in the new -EBUSY check. Then err_unregister runs
hsr_del_ports()->hsr_del_port()->hsr_portdev_role_clear() on slave A,
which had attached successfully, and slave A keeps GRO disabled. For
example:
ip link add hsr0 type hsr slave1 eth0 slave2 lo
fails, but GRO stays off on eth0.
The commit message says:
A failed attach does not restore the GRO request either; the next patch
refuses the usual causes before any state is changed.
The next patch, "net: hsr: refuse forwarding devices as HSR/PRP ports",
only adds checks to hsr_check_dev_ok() for the port being attached.
Wouldn't that still leave out ports attached earlier in
hsr_dev_finalize(), and ndo_set_features() failures?
A stable backport selected through the Fixes: tag could also take this
patch without the follow-up.
> fail_upper_dev_link:
> if (!port->hsr->fwd_offloaded || port->type == HSR_PT_INTERLINK)
> dev_set_promiscuity(dev, -1);
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261107220544.20-1-xiexinet%40gmail.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net v8 2/4] net: hsr: refuse forwarding devices as HSR/PRP ports
2026-11-07 22:05 ` [PATCH net v8 2/4] net: hsr: refuse forwarding devices as " Xin Xie
@ 2026-11-08 22:08 ` netdev-bot+sashiko
2026-11-08 22:28 ` Xin Xie
0 siblings, 1 reply; 10+ messages in thread
From: netdev-bot+sashiko @ 2026-11-08 22:08 UTC (permalink / raw)
To: xiexinet
Cc: netdev, linux-kselftest, linux-kernel, davem, edumazet, kuba,
pabeni, horms, andrew+netdev, shuah, kees, petr.wozniak,
qingfang.deng, fmaurer, luka.gejak, bigeasy, xiaoliang.yang_1,
skhawaja, liuhangbin, stable, sdf.kernel, lukma, fmancera,
danishanwar
Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider.
Critical: 0 · High: 1 · Medium: 1 · Low: 0
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [High] The fix does not close the bug class it targets.
- [Medium] hsr_check_dev_ok() now refuses every macvlan and MACsec device
as an HSR/PRP slave or interlink with -EINVAL, whatever its lower…
--
net: hsr: refuse forwarding devices as HSR/PRP ports
This patch makes hsr_check_dev_ok() refuse bridge, OVS, bond, team,
macvlan and MACsec devices as an HSR/PRP slave or interlink. It also
refuses devices with IFF_NO_RX_HANDLER or a busy RX handler before any
state is changed. The target is a syzbot recursive locking report: a
supervision frame sent with seqnr_lock held goes to a bridge slave, which
floods it into a second HSR master.
> diff --git a/net/hsr/hsr_slave.c b/net/hsr/hsr_slave.c
> index bc340ae0b52e1..26f16d98e4793 100644
> --- a/net/hsr/hsr_slave.c
> +++ b/net/hsr/hsr_slave.c
[ ... ]
> @@ -131,9 +147,19 @@ static int hsr_check_dev_ok(struct net_device *dev,
> return -EOPNOTSUPP;
> }
>
> - /* HSR over bonded devices has not been tested, but I'm not sure it
> - * won't work...
> + /* Forwarding devices feed HSR frames into another device's
> + * transmit path. A bridge, bond, team or OVS master or a macvlan
> + * or MACsec device makes no sense as a ring or LAN port and can
> + * nest seqnr_lock through a second HSR device. Refuse them by
> + * type: devices with lower devices in general stay valid, DSA
> + * user ports are uppers of their conduit.
> */
> + if (netif_is_any_bridge_master(dev) || netif_is_lag_master(dev) ||
> + netif_is_macvlan(dev) || netif_is_macsec(dev)) {
> + NL_SET_ERR_MSG_MOD(extack,
> + "Cannot use a bridge, bond, team, OVS, macvlan or MACsec device as HSR port.");
> + return -EINVAL;
> + }
[Severity: High]
Does this type check close the seqnr_lock nesting, or only the bridge
reproducer from syzbot?
The lock is still held across the whole transmit path of the port. For
example, in hsr_dev_xmit():
net/hsr/hsr_device.c:hsr_dev_xmit() {
...
spin_lock_bh(&hsr->seqnr_lock);
hsr_forward_skb(skb, master);
spin_unlock_bh(&hsr->seqnr_lock);
...
}
send_hsr_supervision_frame(), send_prp_supervision_frame() and the
interlink branch of hsr_handle_frame() follow the same pattern. The call
hsr_forward_skb()->hsr_xmit()->dev_queue_xmit(port) runs the port's tc,
BPF and ndo_start_xmit with the lock held.
spin_lock_init() is only called from hsr_dev_finalize(), so all HSR
instances share one lockdep class.
Any ARPHRD_ETHER port that passes these checks but whose egress reaches an
HSR master synchronously seems to hit the same splat:
- A plain veth or NIC slave with a clsact egress filter doing
"mirred egress redirect dev hsr1". In that case
tcf_mirred_act()->tcf_mirred_forward() calls dev_queue_xmit(hsr1). The
mirred loop check only compares against devices already in
sched_mirred_dev[], so it does not catch the first redirect.
bpf_redirect() from tcx or cls_bpf behaves the same way.
- A virt_wifi device created on top of an HSR master. virt_wifi_newlink()
accepts any lower device, and the virt_wifi device is set up with
ether_setup() and none of the refused priv_flags.
virt_wifi_start_xmit() then does:
skb->dev = priv->lowerdev;
return dev_queue_xmit(skb);
- Ethernet tunnels whose underlay routes through an HSR master. The
commit message already mentions these.
Suppose the redirect targets the same HSR master: veth0 is a slave of
hsr0, and veth0's egress filter redirects to hsr0. Can this deadlock the
CPU?
hsr_announce() timer
spin_lock_bh(&hsr0->seqnr_lock)
hsr_forward_skb()->hsr_xmit()->dev_queue_xmit(veth0)
sch_handle_egress()->tcf_mirred_act()->tcf_mirred_forward()
dev_queue_xmit(hsr0)
hsr_dev_xmit()
spin_lock_bh(&hsr0->seqnr_lock)
hsr0 is lltx and IFF_NO_QUEUE, so __dev_queue_xmit() calls ndo_start_xmit
directly. Neither the dev_xmit_recursion() limit nor MIRRED_NEST_LIMIT is
reached before the second spin_lock_bh(), so the CPU spins on a lock it
already holds, with BH disabled. Setting this up only needs CAP_NET_ADMIN,
which a user namespace provides.
The commit message says:
A chain of two devices does not deadlock, but such a port makes no
sense
It lists only tunnels and batman-adv as not covered. Should it also
mention tc/BPF redirects, virt_wifi and the single-device self-deadlock?
The later commit "net: hsr: segment GSO before per-frame forwarding" moves
the lock into hsr_forward_frame(). It still holds the lock around
hsr_forward_skb(), and so around dev_queue_xmit(), so the problem appears
to remain at the end of the series.
Would it be better to change the locking itself? For example, the
sequence number could be assigned under seqnr_lock and the frame sent
after dropping the lock. At minimum, each instance could get its own
lockdep key. Either would replace this denylist rather than extend it, so
the Closes: tag for the syzbot report looks premature.
[Severity: Medium]
Is it intended to refuse every macvlan and MACsec device here, whatever
its lower device is?
Before this patch both types were accepted as a slave or interlink. Both
are ARPHRD_ETHER with ETH_ALEN, neither sets IFF_DONT_BRIDGE or
IFF_NO_RX_HANDLER, and their RX handler is on the lower device.
The commit message justifies the type rule with "ports can be added
later", but that only applies to bridge, bond and team masters. A
macvlan's lower and a MACsec real_dev are fixed when the device is
created. After that, macvlan_port or macsec holds the lower's RX handler,
so the lower cannot join a bridge or bond later either.
Could a single walk of the lower chain at attach time close the nesting
path through these two types, refusing an HSR master or a bridge/LAG
master found there? Configurations that cannot nest seqnr_lock would then
keep working. Examples are PRP LANs or HSR links over MACsec-protected
physical links, and PRP ports on macvlan in containers.
Because of the Fixes: tags, stable kernels would start rejecting these
previously valid rtnetlink requests with -EINVAL, for example
"ip link add ... type hsr slave1 macsec0 ...". Should the commit message
say that these configurations are no longer allowed?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261107220544.20-1-xiexinet%40gmail.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net v8 3/4] net: hsr: segment GSO before per-frame forwarding
2026-11-07 22:05 ` [PATCH net v8 3/4] net: hsr: segment GSO before per-frame forwarding Xin Xie
@ 2026-11-08 22:08 ` netdev-bot+sashiko
0 siblings, 0 replies; 10+ messages in thread
From: netdev-bot+sashiko @ 2026-11-08 22:08 UTC (permalink / raw)
To: xiexinet
Cc: netdev, linux-kselftest, linux-kernel, davem, edumazet, kuba,
pabeni, horms, andrew+netdev, shuah, kees, petr.wozniak,
qingfang.deng, fmaurer, luka.gejak, bigeasy, xiaoliang.yang_1,
skhawaja, liuhangbin, stable, sdf.kernel, lukma, fmancera,
danishanwar
Thank you for your contribution! Sashiko AI review found 3 potential
issue(s) to consider.
Critical: 0 · High: 0 · Medium: 2 · Low: 1
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] Setups with HSR offload now lose GSO on TX, and the commit
message does not mention it.
- [Medium] hsr_forward_input() protects software segmentation only with
skb_share_check(), which handles skb_shared() (users > 1) by cloning.
- [Low] hsr_forward_input() now drops every GSO skb whose effective
protocol is ETH_P_HSR or ETH_P_PRP (net/hsr/hsr_forward.c:916-917),…
--
net: hsr: segment GSO before per-frame forwarding
This patch adds hsr_forward_input() as the single forwarding entry point
for master TX and slave RX. It segments GSO skbs in software before the
per-frame HSR/PRP tagging and sequence numbering, and drops aggregates
with inner HSR/PRP, unreadable frags or trailing data. It also removes
NETIF_F_GSO_MASK from the master's hw_features.
> diff --git a/net/hsr/hsr_device.c b/net/hsr/hsr_device.c
> index fc512204600cd..2521a72280aca 100644
> --- a/net/hsr/hsr_device.c
> +++ b/net/hsr/hsr_device.c
[ ... ]
> @@ -686,7 +684,7 @@ void hsr_dev_setup(struct net_device *dev)
> dev->needs_free_netdev = true;
>
> dev->hw_features = NETIF_F_SG | NETIF_F_FRAGLIST | NETIF_F_HIGHDMA |
> - NETIF_F_GSO_MASK | NETIF_F_HW_CSUM |
> + NETIF_F_HW_CSUM |
> NETIF_F_HW_VLAN_CTAG_TX |
> NETIF_F_HW_VLAN_CTAG_FILTER;
[Severity: Medium]
Does removing NETIF_F_GSO_MASK here cause a TX performance regression on
lowers with HSR tag insertion offload, such as xrs700x or icssg?
Before this patch the master kept TSO even when the lowers had none. The
master port is in hsr->ports, so hsr_features_recompute() folds in the
master's own features:
net/hsr/hsr_device.c:hsr_features_recompute() {
...
features &= ~NETIF_F_ONE_FOR_ALL;
hsr_for_each_port_rtnl(hsr, port)
features = netdev_increment_features(features,
port->dev->features,
mask);
...
}
On NETIF_F_HW_HSR_TAG_INS lowers, the aggregate was passed on unchanged:
net/hsr/hsr_forward.c:hsr_create_tagged_frame() {
...
} else if (port->dev->features & NETIF_F_HW_HSR_TAG_INS) {
return skb_clone(frame->skb_std, GFP_ATOMIC);
}
...
}
prp_create_tagged_frame() does the same. The lower's validate_xmit_skb()
then segmented the aggregate, and the hardware tagged and numbered each
wire frame. So GSO already worked correctly on these setups.
The mask is now removed whatever the lowers support. The core therefore
segments every local super-packet before hsr_dev_xmit(). Each MSS-sized
frame then goes through these steps on its own: seqnr_lock,
fill_frame_info(), the node list lookup, hsr_register_frame_in(), a
clone per port and dev_queue_xmit().
RX is affected too. Plain aggregates that used to be delivered correctly
are now segmented in hsr_forward_input() with features = 0. That means
linear copies and software checksums. One example is a PRP SAN aggregate
arriving from a veth on LAN A/B. Plain TAG_RM aggregates would be
affected as well. In-tree offload drivers should stop producing those
once the GRO patch in this series is applied.
This patch carries a Fixes: tag, so it will be picked up for stable.
Should the commit message mention the cost for offload setups?
Could the feature removal and the segmentation apply only when software
tagging is actually needed?
> diff --git a/net/hsr/hsr_forward.c b/net/hsr/hsr_forward.c
> index 44f6d9c83180a..c9cd5d9208795 100644
> --- a/net/hsr/hsr_forward.c
> +++ b/net/hsr/hsr_forward.c
[ ... ]
> @@ -794,3 +797,154 @@ void hsr_forward_skb(struct sk_buff *skb, struct hsr_port *port)
[ ... ]
> +void hsr_forward_input(struct sk_buff *skb, struct hsr_port *port)
> +{
[ ... ]
> + if (!skb_frags_readable(skb))
> + goto drop;
> + proto = hsr_gso_effective_proto(skb, &l3off);
> + if (!proto)
> + goto drop;
> + if (proto == htons(ETH_P_HSR) || proto == htons(ETH_P_PRP))
> + goto drop;
[Severity: Low]
Will this also drop locally destined traffic from unpatched HSR senders?
An unpatched HSR master advertises TSO, and hsr_create_tagged_frame()
pushes one HSR tag onto the whole aggregate. A virtio/tap lower with
HW_CSUM and TSO carries that skb to the peer unchanged, as a
VIRTIO_NET_HDR_GSO_TCPV4 packet.
Before this patch, a receiver with a tap/virtio HSR slave stripped the
tag in create_stripped_skb_hsr(). That uses __pskb_copy(), which keeps
the GSO metadata, and the result went to the master for local delivery.
So local TCP worked.
With this patch, hsr_gso_effective_proto() returns ETH_P_HSR and the skb
is dropped before local delivery. TCP from an unpatched HSR guest to a
patched host would then degrade to RTO-driven single-MSS
retransmissions.
The commit message covers the PRP case:
"Older PRP senders can append one RCT to a whole GSO skb. Such
aggregates, including locally destined ones, are now dropped instead
of being delivered."
Should it also mention the HSR case? It only happens with mixed kernel
versions over tap/virtio. veth cannot trigger it, because both ends run
the same kernel.
> + if (hsr_gso_trailing_data(skb, proto, l3off))
> + goto drop;
> +
> + /* Segmentation writes skb->cb and header metadata: work on a
> + * private skb if the input is still shared with a packet socket.
> + */
> + skb = skb_share_check(skb, GFP_ATOMIC);
> + if (!skb) {
> + hsr_gso_drop_stat(port);
> + return;
> + }
> +
> + /* features = 0 requests full software segmentation into single
> + * wire frames; tx_path is true only for locally generated
> + * traffic on the master.
> + */
> + segs = __skb_gso_segment(skb, 0, port->type == HSR_PT_MASTER, 0);
[Severity: Medium]
Is skb_share_check() enough to protect the packet data here? It only
handles skb_shared() (users > 1). A cloned skb (users == 1,
cloned == 1) still shares its data with another holder. So does the
clone that skb_share_check() creates itself.
On RX ports tx_path is false, so __skb_gso_segment() calls
skb_cow_head() only for CHECKSUM_NONE:
net/core/gso.c:skb_needs_check() {
if (tx_path)
return skb->ip_summed != CHECKSUM_PARTIAL &&
skb->ip_summed != CHECKSUM_UNNECESSARY;
return skb->ip_summed == CHECKSUM_NONE;
}
The GSO handlers then write checksum fields into the head's linear data.
udp4_ufo_fragment() does this for every SKB_GSO_UDP input:
net/ipv4/udp_offload.c:udp4_ufo_fragment() {
...
uh->check = 0;
csum = skb_checksum(skb, 0, skb->len, 0);
uh->check = udp_v4_check(skb->len, iph->saddr, iph->daddr, csum);
...
}
tcp4_gso_segment() also rewrites th->check whenever ip_summed is not
CHECKSUM_PARTIAL.
An RX skb reaching hsr_handle_frame() can be cloned. Two ways this
happens:
- packet_rcv() on a non-mmap AF_PACKET socket bound to the port
- a tc ingress mirred mirror
A guest or tap user can inject UFO frames through
virtio_net_hdr_to_skb(), either on the interlink or on a PRP LAN as a
SAN.
In that case, could the other clone holder see the rewritten UDP
checksum bytes? It could also race with them if it segments or
checksums the same data at the same time.
The comment above says segmentation writes skb->cb and header metadata,
but it writes packet bytes as well. Before this patch, software HSR
worked on __pskb_copy() copies with private heads, so this write appears
to be new.
Would calling skb_cow_head(skb, 0) or skb_unshare() before
__skb_gso_segment() on the RX path close this? For reference,
openvswitch's RX path uses the same skb_share_check() plus
tx_path = false pattern.
> + if (IS_ERR_OR_NULL(segs))
> + goto drop;
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261107220544.20-1-xiexinet%40gmail.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net v8 4/4] selftests: net: hsr: verify GRO policy and ordered forwarding
2026-11-07 22:05 ` [PATCH net v8 4/4] selftests: net: hsr: verify GRO policy and ordered forwarding Xin Xie
@ 2026-11-08 22:08 ` netdev-bot+sashiko
0 siblings, 0 replies; 10+ messages in thread
From: netdev-bot+sashiko @ 2026-11-08 22:08 UTC (permalink / raw)
To: xiexinet
Cc: netdev, linux-kselftest, linux-kernel, davem, edumazet, kuba,
pabeni, horms, andrew+netdev, shuah, kees, petr.wozniak,
qingfang.deng, fmaurer, luka.gejak, bigeasy, xiaoliang.yang_1,
skhawaja, liuhangbin, stable, sdf.kernel, lukma, fmancera,
danishanwar
Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider.
Critical: 0 · High: 0 · Medium: 1 · Low: 1
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] topo() in
tools/testing/selftests/net/hsr/hsr_gro_superpacket.py treats a…
- [Low] ordered_wrap() in
tools/testing/selftests/net/hsr/hsr_gro_superpacket.py checks the…
--
selftests: net: hsr: verify GRO policy and ordered forwarding
This commit adds two HSR/PRP selftests. Each is a shell entry point that
execs a shared Python helper. hsr_gro_superpacket.sh covers the GRO policy
on member ports, refusal of forwarding port types, and TCP over the
interlink and PRP LANs. hsr_ordered_forwarding.sh covers the ordering of
local and interlink data, supervision frames, concurrent producers and the
sequence number wrap.
> diff --git a/tools/testing/selftests/net/hsr/hsr_gro_superpacket.py b/tools/testing/selftests/net/hsr/hsr_gro_superpacket.py
> new file mode 100755
> index 0000000000000..23feb0ce4df4c
> --- /dev/null
> +++ b/tools/testing/selftests/net/hsr/hsr_gro_superpacket.py
> @@ -0,0 +1,1376 @@
[ ... ]
> +def ordered_wrap(lab):
> + """The interlink sequence number wraps with both lanes intact."""
> + topology = topo(lab)
> + caps = ordered_lanes(lab, topology)
> + sender = ordered_socket(lab, topology["tx"], "100.64.2.1")
> + probe = ordered_send(lab, sender, 4, 0, 1, 5130)
> + # This only selects the warmup count; final loss-free captures certify it.
> + rows = [udp_rows(cap, {5130}) for cap in caps]
> + require(all(len(lane) == 1 for lane in rows) and
> + rows[0][0]["seq"] == rows[1][0]["seq"], "wrap probe missing or unequal")
[Severity: Low]
Could this check run before the probe has reached the da/db captures?
The only wait between sending the probe and reading the captures is the
fixed drain at the end of ordered_send(), and it has no completion
condition:
ordered_send() {
...
lab.observe(0.1)
return expected
}
The probe goes from t0 over the veth to dil, through the HSR rx_handler in
NET_RX softirq, and then out on da/db. That softirq usually runs
synchronously inside sendto(). If net_rx_action() runs out of budget, the
remaining work is handed to ksoftirqd.
On a loaded CI VM, can ksoftirqd take longer than 100 ms to get scheduled?
If so, the case would fail with "wrap probe missing or unequal" even though
the kernel behaves correctly.
The other packet checks in the script wait through lab.finish() with a
completion predicate. Would polling lab.observe() until both lanes hold the
probe, bounded by lab.remaining(), close this window?
> + start = rows[0][0]["seq"]
[ ... ]
> +def topo(lab):
> + """SAN - interlink - HSR DUT - two LANs - receiver namespaces."""
> + tx, dut, rx = lab.ns(), lab.ns(), lab.ns()
> + for ns, dev, peer_ns, peer in ((tx, "t0", dut, "dil"),
> + (dut, "da", rx, "ra"), (dut, "db", rx, "rb")):
> + lab.veth(ns, dev, peer_ns, peer)
> + for owner, link in ((ns, dev), (peer_ns, peer)):
> + lab.ip(owner, "link", "set", "dev", link, "up")
> + hsr_attach(lab, dut, "hm", "da", "db", "dil")
> + attributes = link_state(lab, dut, "hm")["linkinfo"]["info_data"]
> + require(attributes.get("proto") == 0 and attributes.get("version") == 1 and
> + all(attributes.get(key) == value for key, value in
> + (("slave1", "da"), ("slave2", "db"), ("interlink", "dil"))),
> + "HSR topology has unexpected protocol/version")
[Severity: Medium]
What happens here with an iproute2 that does not print the HSR version
attribute?
The kernel has only sent IFLA_HSR_VERSION for non-PRP devices since commit
16a2206354d169 ("net/hsr: add protocol version to fill_info output"), in
hsr_fill_info():
else if (nla_put_u8(skb, IFLA_HSR_VERSION, hsr->prot_version))
Whether a "version" key shows up in the ip -j -d link show output then
depends on the iproute2 HSR print code. Older iproute2 releases print
slave1, slave2, interlink, seq_nr, supervision and proto, but not version.
In that case attributes.get("version") returns None and require() raises
ValueError.
ValueError is in CASE_ERRORS, so main() reports the case as not ok. It then
marks every later case as failed too:
for later, (other, _) in enumerate(cases[number:], number + 1):
print(f"not ok {later} - {other}: stopped after prior failure", flush=True)
In gro mode, "natural TCP interlink" is the first caller of topo(). The six
"PRP LAN" cases after it never call topo(), but they would still be
reported as not ok. In ordered mode all five cases call topo(), so the
whole test would fail.
A missing interlink option in iproute2 is already reported as SKIP through
interlink_unsupported(). Could a missing "version" key get the same
treatment, or simply not be required?
> + lab.ip(tx, "addr", "add", "100.64.2.1/24", "dev", "t0")
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261107220544.20-1-xiexinet%40gmail.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net v8 2/4] net: hsr: refuse forwarding devices as HSR/PRP ports
2026-11-08 22:08 ` netdev-bot+sashiko
@ 2026-11-08 22:28 ` Xin Xie
0 siblings, 0 replies; 10+ messages in thread
From: Xin Xie @ 2026-11-08 22:28 UTC (permalink / raw)
To: netdev-bot+sashiko
Cc: netdev, linux-kselftest, linux-kernel, davem, edumazet, kuba,
pabeni, horms, andrew+netdev, shuah, kees, petr.wozniak,
qingfang.deng, fmaurer, luka.gejak, bigeasy, xiaoliang.yang_1,
skhawaja, liuhangbin, stable, sdf.kernel, lukma, fmancera,
danishanwar
The point about cloned data is right: skb_share_check() only
handles a shared skb, but when packet_rcv() or a tc mirror holds a
clone, the TCP/UDP GSO handlers on the RX path still write the checksum
into the shared head. v9 will make the head private with
skb_cow_head(skb, 0) before __skb_gso_segment().
I will check the TX GSO cost on HW tag-insertion offloads and mention
aggregates from older HSR senders as well as older PRP senders in the
v9 message.
v8 also conflicts with Sebastian's pending "hsr: inline PTP header" v10
in net-next; v9 will describe how the two combine.
--
Xin
pw-bot: cr
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2026-11-08 22:28 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-07 22:05 [PATCH net v8 0/4] net: hsr: fix super-packet forwarding Xin Xie
2026-11-07 22:05 ` [PATCH net v8 1/4] net: hsr: keep GRO disabled on HSR/PRP ports Xin Xie
2026-11-08 22:08 ` netdev-bot+sashiko
2026-11-07 22:05 ` [PATCH net v8 2/4] net: hsr: refuse forwarding devices as " Xin Xie
2026-11-08 22:08 ` netdev-bot+sashiko
2026-11-08 22:28 ` Xin Xie
2026-11-07 22:05 ` [PATCH net v8 3/4] net: hsr: segment GSO before per-frame forwarding Xin Xie
2026-11-08 22:08 ` netdev-bot+sashiko
2026-11-07 22:05 ` [PATCH net v8 4/4] selftests: net: hsr: verify GRO policy and ordered forwarding Xin Xie
2026-11-08 22:08 ` netdev-bot+sashiko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®