* [RFC PATCH net-next 0/4] tcp: carry a BPF-chosen identifier in TSval low bits
@ 2026-10-13 10:39 Willem Ferguson
2026-10-13 10:39 ` [RFC PATCH net-next 1/4] tcp: add plumbing to carry an " Willem Ferguson
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Willem Ferguson @ 2026-10-13 10:39 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, ncardwell, bpf, ast, daniel,
andrii, eddyz87, memxor, linux-doc, corbet, linux-kselftest,
shuah, linux-kernel, kernel-team
This series lets a listener encode a small per-connection identifier in
the low bits of the TCP timestamp values it sends. The peer echoes TSval
back in TSecr, so every segment the peer sends after the handshake
carries the identifier in its TSecr.
In Anycast networks, mid-stream path changes caused by BGP route flapping
or ECMP shifts frequently deliver active TCP packets to an ingress Point
of Presence (PoP) that lacks the flow's socket state, resulting in abrupt
connection resets (RST). While protocols like QUIC avoid this using explicit
Connection IDs, TCP lacks an in-band routing token to help ingress
nodes identify where a flow actually lives. To address this problem,
we propose embedding an N-bit data center ID into the lower-order bits
of the TCP Timestamp option (TSval, RFC 7323), allowing edge load balancers
to recognize misdirected packets and steer them to the correct origin data
center holding the active session.
How it works:
- tcp_ts_id_bits (per netns, 0..8, default 0) sets how many low TSval
bits are reserved. The millisecond clock is shifted up by that many
bits.
- A sock_ops program opts a listener in with
BPF_SOCK_OPS_TCP_TS_ID_CB_FLAG and is called with
BPF_SOCK_OPS_TCP_TS_ID_CB for each SYN, before the SYN-ACK is sent.
It sees the SYN and optionally returns the identifier
- The identifier and width are stored in the request socket, then in
the full and timewait sockets. Incoming TSecr values are decoded
before RTT estimation and PAWS use them.
- TCP_TS_ID reads back the identifier, and /proc/net/netstat gains
TCPTSIdAssigned, TCPTSIdDeclined, TCPTSIdSkipped and
TCPTSIdEchoMismatch.
RTT estimation on the encoding host sees the same decoded timestamps
as before, so they are unchanged. On the peer, PAWS keeps working too;
the only effects are a shorter idle limit and a narrower tolerance for
reordered segments, both described below.
Scope and limitations:
- Passive opens only. SYN cookies, microsecond timestamps and peers
without timestamps are left unencoded.
- The peer sees a clock running 2^bits times faster. That narrows the
real-time span of its PAWS window, so the width should be no larger
than the identifiers need.
- Peers that keep ts_recent for the RFC 7323 24 days (e.g. FreeBSD,
macOS) stall connections that stay idle longer than 24.8/2^bits days
(about 37 hours at 4 bits). Linux peers stop enforcing PAWS after
TCP_PAWS_WRAP and are unaffected.
- TCP_REPAIR cannot restore the identifier, so hosts that migrate
established connections should leave the feature off.
All of this is documented under tcp_ts_id_bits in ip-sysctl.rst. With
the sysctl at 0 (the default) the only new cost on the established fast
path is one likely() branch when parsing TSecr.
Questions for reviewers:
1. Is encoding an identifier in the low bits of TSval acceptable in
principle, given the trade-offs above?
2. If so, what feedback do you have on the implementation?
Patch 1 adds the encoding and decoding, which stay inert until patch 2
lets BPF set an identifier. Patch 3 adds the TCP_TS_ID read-back, and
patch 4 adds a BPF selftest.
Testing:
- A kernel with this series and tcp_ts_id_bits=7 is serving production
traffic on a few Cloudflare hosts and assigning ids to all CDN
connections. Key TCP metrics on those hosts show no difference from
hosts without the series.
Willem Ferguson (4):
tcp: add plumbing to carry an identifier in TSval low bits
tcp: add BPF_SOCK_OPS_TCP_TS_ID_CB to assign a TSval identifier
tcp: add TCP_TS_ID socket option
selftests/bpf: add tcp_ts_id test
Documentation/networking/ip-sysctl.rst | 38 +++++
.../net_cachelines/netns_ipv4_sysctl.rst | 1 +
.../networking/net_cachelines/tcp_sock.rst | 2 +
include/linux/tcp.h | 8 +
include/net/netns/ipv4.h | 1 +
include/net/tcp.h | 58 ++++++-
include/uapi/linux/bpf.h | 15 +-
include/uapi/linux/snmp.h | 4 +
include/uapi/linux/tcp.h | 8 +
net/core/filter.c | 2 +
net/ipv4/proc.c | 4 +
net/ipv4/syncookies.c | 2 +
net/ipv4/sysctl_net_ipv4.c | 10 ++
net/ipv4/tcp.c | 16 ++
net/ipv4/tcp_input.c | 61 ++++++-
net/ipv4/tcp_ipv4.c | 1 +
net/ipv4/tcp_minisocks.c | 27 +++-
net/ipv4/tcp_output.c | 15 +-
tools/include/uapi/linux/bpf.h | 15 +-
.../selftests/bpf/prog_tests/tcp_ts_id.c | 150 ++++++++++++++++++
.../selftests/bpf/prog_tests/tcpbpf_user.c | 2 +-
.../selftests/bpf/progs/test_tcp_ts_id_kern.c | 32 ++++
.../selftests/bpf/progs/test_tcpbpf_kern.c | 2 +-
23 files changed, 459 insertions(+), 15 deletions(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/tcp_ts_id.c
create mode 100644 tools/testing/selftests/bpf/progs/test_tcp_ts_id_kern.c
base-commit: d8674294aefef02266c4d47ad10131f1bffbe534
--
2.47.3
^ permalink raw reply [flat|nested] 5+ messages in thread
* [RFC PATCH net-next 1/4] tcp: add plumbing to carry an identifier in TSval low bits
2026-10-13 10:39 [RFC PATCH net-next 0/4] tcp: carry a BPF-chosen identifier in TSval low bits Willem Ferguson
@ 2026-10-13 10:39 ` Willem Ferguson
2026-10-13 10:39 ` [RFC PATCH net-next 2/4] tcp: add BPF_SOCK_OPS_TCP_TS_ID_CB to assign a TSval identifier Willem Ferguson
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Willem Ferguson @ 2026-10-13 10:39 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, ncardwell, bpf, ast, daniel,
andrii, eddyz87, memxor, linux-doc, corbet, linux-kselftest,
shuah, linux-kernel, kernel-team
Add per-connection state that lets the low bits of locally generated
TSvals carry a small identifier, and teach the receive side to strip it
from echoed TSecr values:
- tcp_request_sock, tcp_sock and tcp_timewait_sock gain ts_id and
ts_id_bits. They are copied from the request socket to the child and
from the child to the timewait socket, and cleared on every path that
creates a request socket without going through tcp_openreq_init()
(SYN cookies and bpf_sk_assign_tcp_reqsk()).
- tcp_tsval_encode() shifts the timestamp clock left by ts_id_bits and
ORs in ts_id. It is used for SYN-ACK, established, request socket and
timewait TSvals. The SYN-ACK path keeps the identifier when it
replaces a zero TSval with ~0U.
- tcp_tsecr_decode() undoes the encoding. The shift drops ts_id_bits of
clock range, so the decoded echo is re-centred on the current clock
with sign_extend32() before RTT estimation and PAWS consume it.
A BUILD_BUG_ON() checks that, at the widest identifier allowed, an
encoded TSval cannot advance by 2^31 within TCP_PAWS_WRAP, the idle
period after which Linux stops enforcing PAWS against ts_recent.
Nothing sets ts_id_bits yet, so this patch does not change behaviour.
With ts_id_bits == 0, tcp_tsval_encode() returns its input and the
established receive path adds a single likely() branch before the
existing tsoffset subtraction.
Signed-off-by: Willem Ferguson <wferguson@cloudflare.com>
---
.../networking/net_cachelines/tcp_sock.rst | 2 +
include/linux/tcp.h | 8 +++
include/net/tcp.h | 50 ++++++++++++++++++-
net/core/filter.c | 2 +
net/ipv4/syncookies.c | 2 +
net/ipv4/tcp.c | 13 +++++
net/ipv4/tcp_input.c | 11 ++--
net/ipv4/tcp_minisocks.c | 18 ++++++-
net/ipv4/tcp_output.c | 15 ++++--
9 files changed, 110 insertions(+), 11 deletions(-)
diff --git a/Documentation/networking/net_cachelines/tcp_sock.rst b/Documentation/networking/net_cachelines/tcp_sock.rst
index 103328dc409e..91a00194ad3a 100644
--- a/Documentation/networking/net_cachelines/tcp_sock.rst
+++ b/Documentation/networking/net_cachelines/tcp_sock.rst
@@ -32,6 +32,8 @@ u32 lsndtime read_write
u32 last_oow_ack_time
u32 compressed_ack_rcv_nxt
u32 tsoffset read_mostly read_mostly tcp_established_options(tx);tcp_fast_parse_options(rx)
+u8 ts_id read_mostly read_mostly tcp_established_options(tx);tcp_fast_parse_options(rx)
+u8 ts_id_bits read_mostly read_mostly tcp_established_options(tx);tcp_fast_parse_options(rx)
struct list_head tsq_node
struct list_head tsorted_sent_queue read_write tcp_update_skb_after_send
u32 snd_wl1 read_mostly tcp_may_update_window
diff --git a/include/linux/tcp.h b/include/linux/tcp.h
index e4d1720f49f0..6de7ace4b016 100644
--- a/include/linux/tcp.h
+++ b/include/linux/tcp.h
@@ -145,6 +145,8 @@ static inline void tcp_clear_options(struct tcp_options_received *rx_opt)
* only four options will fit in a standard TCP header */
#define TCP_NUM_SACKS 4
+#define TCP_TS_ID_BITS_MAX 8
+
struct tcp_request_sock_ops;
struct tcp_request_sock {
@@ -154,6 +156,8 @@ struct tcp_request_sock {
bool tfo_listener;
bool is_mptcp;
bool req_usec_ts;
+ u8 ts_id;
+ u8 ts_id_bits;
#if IS_ENABLED(CONFIG_MPTCP)
bool drop_req;
#endif
@@ -220,6 +224,8 @@ struct tcp_sock {
/* TXRX read-mostly hotpath cache lines */
__cacheline_group_begin(tcp_sock_read_txrx);
u32 tsoffset; /* timestamp offset */
+ u8 ts_id; /* identifier encoded in TSval low bits */
+ u8 ts_id_bits; /* number of identifier bits in TSval */
u32 snd_wnd; /* The window we expect to receive */
u32 mss_cache; /* Cached effective mss, not including SACKS */
u32 snd_cwnd; /* Sending congestion window */
@@ -575,6 +581,8 @@ struct tcp_timewait_sock {
u32 tw_rcv_wnd;
u32 tw_ts_offset;
u32 tw_ts_recent;
+ u8 tw_ts_id;
+ u8 tw_ts_id_bits;
/* The time we sent the last out-of-window ACK: */
u32 tw_last_oow_ack_time;
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 0265c27a5fd7..85d3038f41af 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -1074,14 +1074,60 @@ static inline u32 tcp_skb_timestamp_ts(bool usec_ts, const struct sk_buff *skb)
return div_u64(skb->skb_mstamp_ns, NSEC_PER_MSEC);
}
+static inline u32 tcp_ts_id_mask(u8 ts_id_bits)
+{
+ return BIT(ts_id_bits) - 1;
+}
+
+static inline u32 tcp_ts_clock_mask(u8 ts_id_bits)
+{
+ return U32_MAX >> ts_id_bits;
+}
+
+static inline u32 tcp_tsval_encode(u32 tsval, u8 ts_id, u8 ts_id_bits)
+{
+ if (!ts_id_bits)
+ return tsval;
+
+ return ((tsval & tcp_ts_clock_mask(ts_id_bits)) << ts_id_bits) |
+ ts_id;
+}
+
+static inline u32 tcp_tsecr_decode(u32 tsecr, u32 tsoffset, u8 ts_id_bits,
+ u32 now)
+{
+ if (!ts_id_bits)
+ return tsecr - tsoffset;
+
+ tsecr = ((tsecr >> ts_id_bits) - tsoffset) &
+ tcp_ts_clock_mask(ts_id_bits);
+
+ return now + sign_extend32(tsecr - now, 31 - ts_id_bits);
+}
+
+/* Keeps the timestamp clock read behind the branch: it is only needed to
+ * re-center an echo that was truncated to make room for an identifier.
+ */
+static inline u32 tcp_tsecr_decode_tp(u32 tsecr, const struct tcp_sock *tp)
+{
+ if (likely(!tp->ts_id_bits))
+ return tsecr - tp->tsoffset;
+
+ return tcp_tsecr_decode(tsecr, tp->tsoffset, tp->ts_id_bits,
+ tcp_time_stamp_ts(tp));
+}
+
static inline u32 tcp_tw_tsval(const struct tcp_timewait_sock *tcptw)
{
- return tcp_clock_ts(tcptw->tw_sk.tw_usec_ts) + tcptw->tw_ts_offset;
+ return tcp_tsval_encode(tcp_clock_ts(tcptw->tw_sk.tw_usec_ts) +
+ tcptw->tw_ts_offset, tcptw->tw_ts_id,
+ tcptw->tw_ts_id_bits);
}
static inline u32 tcp_rsk_tsval(const struct tcp_request_sock *treq)
{
- return tcp_clock_ts(treq->req_usec_ts) + treq->ts_off;
+ return tcp_tsval_encode(tcp_clock_ts(treq->req_usec_ts) + treq->ts_off,
+ treq->ts_id, treq->ts_id_bits);
}
#define tcp_flag_byte(th) (((u_int8_t *)th)[13])
diff --git a/net/core/filter.c b/net/core/filter.c
index 70dc621672f2..a0a74c7e2e3d 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -12579,6 +12579,8 @@ __bpf_kfunc int bpf_sk_assign_tcp_reqsk(struct __sk_buff *s, struct sock *sk,
treq->req_usec_ts = !!attrs->usec_ts_ok;
treq->ts_off = tsoff;
+ treq->ts_id = 0;
+ treq->ts_id_bits = 0;
skb_orphan(skb);
skb->sk = req_to_sk(req);
diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c
index 73e129768184..f0d164846a5d 100644
--- a/net/ipv4/syncookies.c
+++ b/net/ipv4/syncookies.c
@@ -280,6 +280,8 @@ static int cookie_tcp_reqsk_init(struct sock *sk, struct sk_buff *skb,
treq->snt_synack = 0;
treq->snt_tsval_first = 0;
treq->tfo_listener = false;
+ treq->ts_id = 0;
+ treq->ts_id_bits = 0;
treq->rcv_isn = ntohl(th->seq) - 1;
treq->snt_isn = ntohl(th->ack_seq) - 1;
/* The request socket was freed after the SYN-ACK; use the cookie
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index f4c9101e5996..af994edde7f7 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -462,6 +462,8 @@ void tcp_init_sock(struct sock *sk)
tcp_assign_congestion_control(sk);
tp->tsoffset = 0;
+ tp->ts_id = 0;
+ tp->ts_id_bits = 0;
tp->rack.reo_wnd_steps = 1;
sk->sk_write_space = sk_stream_write_space;
@@ -3429,6 +3431,8 @@ int tcp_disconnect(struct sock *sk, int flags)
tp->window_clamp = 0;
tp->delivered = 0;
tp->delivered_ce = 0;
+ tp->ts_id = 0;
+ tp->ts_id_bits = 0;
tp->accecn_fail_mode = 0;
tp->saw_accecn_opt = TCP_ACCECN_OPT_NOT_SEEN;
tcp_accecn_init_counters(tp);
@@ -5212,6 +5216,8 @@ static void __init tcp_struct_check(void)
/* TXRX read-mostly hotpath cache lines */
CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_read_txrx, tsoffset);
+ CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_read_txrx, ts_id);
+ CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_read_txrx, ts_id_bits);
CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_read_txrx, snd_wnd);
CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_read_txrx, mss_cache);
CACHELINE_ASSERT_GROUP_MEMBER(struct tcp_sock, tcp_sock_read_txrx, snd_cwnd);
@@ -5302,6 +5308,13 @@ void __init tcp_init(void)
BUILD_BUG_ON(sizeof(struct tcp_skb_cb) >
sizeof_field(struct sk_buff, cb));
+ /* An encoded TSval must not advance by 2^31 within the window that
+ * Linux PAWS keeps ts_recent valid; peers using RFC 7323's 24 day
+ * window are not covered (see tcp_ts_id_bits in ip-sysctl.rst).
+ */
+ BUILD_BUG_ON(((u64)TCP_PAWS_WRAP * MSEC_PER_SEC) <<
+ TCP_TS_ID_BITS_MAX >= (1ULL << 31));
+
tcp_struct_check();
percpu_counter_init(&tcp_sockets_allocated, 0, GFP_KERNEL);
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 1ddce9c4b7e8..68af0283742e 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -4717,7 +4717,8 @@ static bool tcp_parse_aligned_timestamp(struct tcp_sock *tp, const struct tcphdr
tp->rx_opt.rcv_tsval = ntohl(*ptr);
++ptr;
if (*ptr)
- tp->rx_opt.rcv_tsecr = ntohl(*ptr) - tp->tsoffset;
+ tp->rx_opt.rcv_tsecr = tcp_tsecr_decode_tp(ntohl(*ptr),
+ tp);
else
tp->rx_opt.rcv_tsecr = 0;
return true;
@@ -4749,7 +4750,8 @@ static bool tcp_fast_parse_options(const struct net *net,
tcp_parse_options(net, skb, &tp->rx_opt, 1, NULL);
if (tp->rx_opt.saw_tstamp && tp->rx_opt.rcv_tsecr)
- tp->rx_opt.rcv_tsecr -= tp->tsoffset;
+ tp->rx_opt.rcv_tsecr =
+ tcp_tsecr_decode_tp(tp->rx_opt.rcv_tsecr, tp);
return true;
}
@@ -6881,7 +6883,8 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
tcp_parse_options(sock_net(sk), skb, &tp->rx_opt, 0, &foc);
if (tp->rx_opt.saw_tstamp && tp->rx_opt.rcv_tsecr)
- tp->rx_opt.rcv_tsecr -= tp->tsoffset;
+ tp->rx_opt.rcv_tsecr =
+ tcp_tsecr_decode_tp(tp->rx_opt.rcv_tsecr, tp);
if (th->ack) {
/* rfc793:
@@ -7512,6 +7515,8 @@ static void tcp_openreq_init(struct request_sock *req,
tcp_rsk(req)->rcv_nxt = TCP_SKB_CB(skb)->seq + 1;
tcp_rsk(req)->snt_synack = 0;
tcp_rsk(req)->snt_tsval_first = 0;
+ tcp_rsk(req)->ts_id = 0;
+ tcp_rsk(req)->ts_id_bits = 0;
tcp_rsk(req)->last_oow_ack_time = 0;
tcp_rsk(req)->accecn_ok = 0;
tcp_rsk(req)->saw_accecn_opt = TCP_ACCECN_OPT_NOT_SEEN;
diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c
index 0ddfd5af6e58..43580b395bff 100644
--- a/net/ipv4/tcp_minisocks.c
+++ b/net/ipv4/tcp_minisocks.c
@@ -122,7 +122,11 @@ tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb,
if (tmp_opt.saw_tstamp) {
if (tmp_opt.rcv_tsecr)
- tmp_opt.rcv_tsecr -= tcptw->tw_ts_offset;
+ tmp_opt.rcv_tsecr =
+ tcp_tsecr_decode(tmp_opt.rcv_tsecr,
+ tcptw->tw_ts_offset,
+ tcptw->tw_ts_id_bits,
+ tcp_clock_ts(tw->tw_usec_ts));
tmp_opt.ts_recent = READ_ONCE(tcptw->tw_ts_recent);
tmp_opt.ts_recent_stamp = ts_recent_stamp;
paws_reject = tcp_paws_reject(&tmp_opt, th->rst);
@@ -347,6 +351,8 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
tcptw->tw_ts_recent = tp->rx_opt.ts_recent;
tcptw->tw_ts_recent_stamp = tp->rx_opt.ts_recent_stamp;
tcptw->tw_ts_offset = tp->tsoffset;
+ tcptw->tw_ts_id = tp->ts_id;
+ tcptw->tw_ts_id_bits = tp->ts_id_bits;
tw->tw_usec_ts = tp->tcp_usec_ts;
tcptw->tw_last_oow_ack_time = 0;
tcptw->tw_tx_delay = tp->tcp_tx_delay;
@@ -639,6 +645,8 @@ struct sock *tcp_create_openreq_child(const struct sock *sk,
newtp->total_rto_recoveries = 1;
}
newtp->tsoffset = treq->ts_off;
+ newtp->ts_id = treq->ts_id;
+ newtp->ts_id_bits = treq->ts_id_bits;
#ifdef CONFIG_TCP_MD5SIG
newtp->md5sig_info = NULL; /*XXX*/
#endif
@@ -705,11 +713,17 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
if (tmp_opt.saw_tstamp) {
tmp_opt.ts_recent = req->ts_recent;
if (tmp_opt.rcv_tsecr) {
+ u32 now = tcp_clock_ts(tcp_rsk(req)->req_usec_ts);
+
if (inet_rsk(req)->tstamp_ok && !fastopen)
tsecr_reject = !between(tmp_opt.rcv_tsecr,
tcp_rsk(req)->snt_tsval_first,
READ_ONCE(tcp_rsk(req)->snt_tsval_last));
- tmp_opt.rcv_tsecr -= tcp_rsk(req)->ts_off;
+ tmp_opt.rcv_tsecr =
+ tcp_tsecr_decode(tmp_opt.rcv_tsecr,
+ tcp_rsk(req)->ts_off,
+ tcp_rsk(req)->ts_id_bits,
+ now);
}
/* We do not store true stamp, but it is not required,
* it can be estimated (approximately)
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index c8865205cdc8..966adebc9154 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -1101,11 +1101,15 @@ static unsigned int tcp_synack_options(const struct sock *sk,
opts->options |= OPTION_TS;
opts->tsval = tcp_skb_timestamp_ts(tcp_rsk(req)->req_usec_ts, skb) +
tcp_rsk(req)->ts_off;
- if (!tcp_rsk(req)->snt_tsval_first) {
- if (!opts->tsval)
- opts->tsval = ~0U;
+ opts->tsval = tcp_tsval_encode(opts->tsval,
+ tcp_rsk(req)->ts_id,
+ tcp_rsk(req)->ts_id_bits);
+ if (!tcp_rsk(req)->snt_tsval_first && !opts->tsval)
+ opts->tsval = tcp_tsval_encode(~0U,
+ tcp_rsk(req)->ts_id,
+ tcp_rsk(req)->ts_id_bits);
+ if (!tcp_rsk(req)->snt_tsval_first)
tcp_rsk(req)->snt_tsval_first = opts->tsval;
- }
WRITE_ONCE(tcp_rsk(req)->snt_tsval_last, opts->tsval);
opts->tsecr = req->ts_recent;
remaining -= TCPOLEN_TSTAMP_ALIGNED;
@@ -1171,6 +1175,9 @@ static unsigned int tcp_established_options(struct sock *sk, struct sk_buff *skb
opts->options |= OPTION_TS;
opts->tsval = skb ? tcp_skb_timestamp_ts(tp->tcp_usec_ts, skb) +
tp->tsoffset : 0;
+ if (skb)
+ opts->tsval = tcp_tsval_encode(opts->tsval, tp->ts_id,
+ tp->ts_id_bits);
opts->tsecr = tp->rx_opt.ts_recent;
size += TCPOLEN_TSTAMP_ALIGNED;
}
--
2.47.3
^ permalink raw reply [flat|nested] 5+ messages in thread
* [RFC PATCH net-next 2/4] tcp: add BPF_SOCK_OPS_TCP_TS_ID_CB to assign a TSval identifier
2026-10-13 10:39 [RFC PATCH net-next 0/4] tcp: carry a BPF-chosen identifier in TSval low bits Willem Ferguson
2026-10-13 10:39 ` [RFC PATCH net-next 1/4] tcp: add plumbing to carry an " Willem Ferguson
@ 2026-10-13 10:39 ` Willem Ferguson
2026-10-13 10:39 ` [RFC PATCH net-next 3/4] tcp: add TCP_TS_ID socket option Willem Ferguson
2026-10-13 10:39 ` [RFC PATCH net-next 4/4] selftests/bpf: add tcp_ts_id test Willem Ferguson
3 siblings, 0 replies; 5+ messages in thread
From: Willem Ferguson @ 2026-10-13 10:39 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, ncardwell, bpf, ast, daniel,
andrii, eddyz87, memxor, linux-doc, corbet, linux-kselftest,
shuah, linux-kernel, kernel-team
Let a sock_ops program pick the identifier that the previous patch
encodes in the low bits of TSval, for passively opened connections.
The width is set per network namespace by the new tcp_ts_id_bits sysctl
(0 to 8, default 0). A listener opts in by setting
BPF_SOCK_OPS_TCP_TS_ID_CB_FLAG, typically from BPF_SOCK_OPS_TCP_LISTEN_CB.
tcp_conn_request() then runs BPF_SOCK_OPS_TCP_TS_ID_CB once the request
socket exists and before the SYN-ACK is built, so the SYN-ACK already
carries the identifier. The program sees the request socket, the SYN
(bpf_load_hdr_opt() works on it) and the available width in args[1]. A
reply that fits in that width is stored with the width in the request
socket and is inherited by the child, including Fast Open children; any
other reply leaves the connection unencoded.
The callback is not run for SYN cookies, which have nowhere to keep the
identifier, or when the peer did not negotiate timestamps. It is skipped
for connections using microsecond timestamps and while the sysctl is 0.
The width is captured per connection, so changing the sysctl does not
affect existing connections.
New /proc/net/netstat counters report how often an identifier was
assigned, declined by the program or skipped. TCPTSIdEchoMismatch counts
third ACKs whose TSecr no longer carries the identifier, which shows the
echo was rewritten in flight.
BPF_SOCK_OPS_ALL_CB_FLAGS grows from 0x7f to 0xff. The tcpbpf selftest
used 0x80 as an example of an unsupported flag, so move it to 0x100.
Encoding makes the timestamp clock appear 2^bits times faster to the
peer. That shrinks the real-time span of the peer's PAWS window, and
peers that keep ts_recent for the RFC 7323 24 days stall connections
idle for longer than 24.8/2^bits days. ip-sysctl.rst documents both,
along with the lack of TCP_REPAIR support.
Signed-off-by: Willem Ferguson <wferguson@cloudflare.com>
---
Documentation/networking/ip-sysctl.rst | 37 ++++++++++++++
.../net_cachelines/netns_ipv4_sysctl.rst | 1 +
include/net/netns/ipv4.h | 1 +
include/net/tcp.h | 8 +++
include/uapi/linux/bpf.h | 15 +++++-
include/uapi/linux/snmp.h | 4 ++
net/ipv4/proc.c | 4 ++
net/ipv4/sysctl_net_ipv4.c | 10 ++++
net/ipv4/tcp_input.c | 50 +++++++++++++++++++
net/ipv4/tcp_ipv4.c | 1 +
net/ipv4/tcp_minisocks.c | 9 ++++
tools/include/uapi/linux/bpf.h | 15 +++++-
.../selftests/bpf/prog_tests/tcpbpf_user.c | 2 +-
.../selftests/bpf/progs/test_tcpbpf_kern.c | 2 +-
14 files changed, 155 insertions(+), 4 deletions(-)
diff --git a/Documentation/networking/ip-sysctl.rst b/Documentation/networking/ip-sysctl.rst
index 685c84cf543d..ce669f282250 100644
--- a/Documentation/networking/ip-sysctl.rst
+++ b/Documentation/networking/ip-sysctl.rst
@@ -1096,6 +1096,43 @@ tcp_timestamps - INTEGER
Default: 1
+tcp_ts_id_bits - INTEGER
+ Number of low bits in locally generated TCP timestamp values that may
+ contain an identifier selected by BPF_SOCK_OPS_TCP_TS_ID_CB. The upper
+ bits contain the millisecond timestamp clock. Valid values are 0 to 8.
+ A value of 0 disables the callback and timestamp identifier encoding.
+
+ The value is captured when the callback selects an identifier. Changing
+ it does not alter existing connections. Identifier encoding is disabled
+ for connections using microsecond timestamps and for SYN cookies.
+
+ Encoding shifts the timestamp clock up by this many bits, so the TSval
+ wrap time falls from 49.7 days to 49.7/2^bits days and peers see an
+ apparently faster clock. Peers that size their PAWS replay window in
+ timestamp units (Linux does) therefore tolerate less real time before
+ rejecting a reordered pure ACK. The window shrinks with every extra
+ bit, so use the narrowest width that holds the identifiers needed.
+
+ The faster clock also limits how long a connection may stay idle.
+ TSval advances by 2^31 after 24.8/2^bits days (about 2.3 hours at 8
+ bits, 37 hours at 4 bits). Linux peers stop enforcing PAWS after 2147
+ seconds of idle, so they are unaffected. Peers that follow RFC 7323 and
+ keep ts_recent for 24 days (e.g. FreeBSD, macOS) will drop every
+ segment once that point is passed. The connection then stalls until it
+ times out. Traffic in either direction, including TCP keepalives, more
+ often than this interval prevents it.
+
+ Progress is visible through the TCPTSId* counters in
+ /proc/net/netstat.
+
+ Checkpoint/restore is not supported. The identifier is selected by BPF
+ when the SYN arrives and cannot be restored onto a new socket, so a
+ connection carrying one will emit unencoded timestamp values after a
+ TCP_REPAIR restore and be rejected by the peer's PAWS check. Leave
+ this at 0 on hosts that migrate established connections.
+
+ Default: 0 (disabled)
+
tcp_min_tso_segs - INTEGER
Minimal number of segments per TSO frame.
diff --git a/Documentation/networking/net_cachelines/netns_ipv4_sysctl.rst b/Documentation/networking/net_cachelines/netns_ipv4_sysctl.rst
index 3dc03bff739e..cbb6e6f85f17 100644
--- a/Documentation/networking/net_cachelines/netns_ipv4_sysctl.rst
+++ b/Documentation/networking/net_cachelines/netns_ipv4_sysctl.rst
@@ -87,6 +87,7 @@ unsigned_int sysctl_tcp_notsent_lowat rea
u8 sysctl_tcp_sack tcp_syn_options
u8 sysctl_tcp_window_scaling tcp_syn_options,tcp_parse_options
u8 sysctl_tcp_timestamps
+u8 sysctl_tcp_ts_id_bits bpf_skops_tcp_ts_id
u8 sysctl_tcp_early_retrans read_mostly tcp_schedule_loss_probe(tcp_write_xmit)
u32 sysctl_tcp_rto_max_ms
u8 sysctl_tcp_recovery tcp_fastretrans_alert
diff --git a/include/net/netns/ipv4.h b/include/net/netns/ipv4.h
index cb7f8bf15671..4806906276c2 100644
--- a/include/net/netns/ipv4.h
+++ b/include/net/netns/ipv4.h
@@ -204,6 +204,7 @@ struct netns_ipv4 {
u8 sysctl_tcp_sack;
u8 sysctl_tcp_window_scaling;
u8 sysctl_tcp_timestamps;
+ u8 sysctl_tcp_ts_id_bits;
int sysctl_tcp_rto_min_us;
int sysctl_tcp_rto_max_ms;
u8 sysctl_tcp_recovery;
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 85d3038f41af..970757eccf25 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -1105,6 +1105,14 @@ static inline u32 tcp_tsecr_decode(u32 tsecr, u32 tsoffset, u8 ts_id_bits,
return now + sign_extend32(tsecr - now, 31 - ts_id_bits);
}
+/* A TSecr echoing one of our TSvals must still carry our identifier in its
+ * low bits; anything else means the echo was rewritten in flight.
+ */
+static inline bool tcp_tsecr_id_ok(u32 tsecr, u8 ts_id, u8 ts_id_bits)
+{
+ return (tsecr & tcp_ts_id_mask(ts_id_bits)) == ts_id;
+}
+
/* Keeps the timestamp clock read behind the branch: it is only needed to
* re-center an echo that was truncated to make room for an identifier.
*/
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 732b35cc08d1..b440a3fd96d0 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -7062,6 +7062,7 @@ struct bpf_sock_ops {
* header has not been written.
* BPF_SOCK_OPS_WRITE_HDR_OPT_CB: The header and options have
* been written so far.
+ * BPF_SOCK_OPS_TCP_TS_ID_CB: The SYN received by a listener.
* BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB: The SYNACK that concludes
* the 3WHS.
* BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB: The ACK that concludes
@@ -7137,8 +7138,12 @@ enum {
* options first before the BPF program does.
*/
BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAG = (1<<6),
+ /* Call bpf to select a TCP timestamp connection ID for a passive
+ * connection before sending the SYN-ACK.
+ */
+ BPF_SOCK_OPS_TCP_TS_ID_CB_FLAG = (1 << 7),
/* Mask of all currently supported cb flags */
- BPF_SOCK_OPS_ALL_CB_FLAGS = 0x7F,
+ BPF_SOCK_OPS_ALL_CB_FLAGS = 0xFF,
};
enum {
@@ -7282,6 +7287,14 @@ enum {
* sendmsg timestamp with corresponding
* tskey.
*/
+ BPF_SOCK_OPS_TCP_TS_ID_CB, /* Called after a passive SYN created a
+ * request socket and before the SYN-ACK is
+ * sent. Not called for SYN cookies. Arg2 is
+ * the number of low timestamp bits available
+ * for an identifier. Set reply to an identifier
+ * that fits in Arg2 bits. Replies are ignored
+ * for usec timestamps.
+ */
};
/* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
diff --git a/include/uapi/linux/snmp.h b/include/uapi/linux/snmp.h
index 49f5640092a0..b214dcc38736 100644
--- a/include/uapi/linux/snmp.h
+++ b/include/uapi/linux/snmp.h
@@ -306,6 +306,10 @@ enum
LINUX_MIB_TCPAOKEYNOTFOUND, /* TCPAOKeyNotFound */
LINUX_MIB_TCPAOGOOD, /* TCPAOGood */
LINUX_MIB_TCPAODROPPEDICMPS, /* TCPAODroppedIcmps */
+ LINUX_MIB_TCPTSIDASSIGNED, /* TCPTSIdAssigned */
+ LINUX_MIB_TCPTSIDDECLINED, /* TCPTSIdDeclined */
+ LINUX_MIB_TCPTSIDSKIPPED, /* TCPTSIdSkipped */
+ LINUX_MIB_TCPTSIDECHOMISMATCH, /* TCPTSIdEchoMismatch */
__LINUX_MIB_MAX
};
diff --git a/net/ipv4/proc.c b/net/ipv4/proc.c
index bfc06d1713ec..6f92714693aa 100644
--- a/net/ipv4/proc.c
+++ b/net/ipv4/proc.c
@@ -302,6 +302,10 @@ static const struct snmp_mib snmp4_net_list[] = {
SNMP_MIB_ITEM("TCPAOKeyNotFound", LINUX_MIB_TCPAOKEYNOTFOUND),
SNMP_MIB_ITEM("TCPAOGood", LINUX_MIB_TCPAOGOOD),
SNMP_MIB_ITEM("TCPAODroppedIcmps", LINUX_MIB_TCPAODROPPEDICMPS),
+ SNMP_MIB_ITEM("TCPTSIdAssigned", LINUX_MIB_TCPTSIDASSIGNED),
+ SNMP_MIB_ITEM("TCPTSIdDeclined", LINUX_MIB_TCPTSIDDECLINED),
+ SNMP_MIB_ITEM("TCPTSIdSkipped", LINUX_MIB_TCPTSIDSKIPPED),
+ SNMP_MIB_ITEM("TCPTSIdEchoMismatch", LINUX_MIB_TCPTSIDECHOMISMATCH),
};
static void icmpmsg_put_line(struct seq_file *seq, unsigned long *vals,
diff --git a/net/ipv4/sysctl_net_ipv4.c b/net/ipv4/sysctl_net_ipv4.c
index e3760daa3470..ce9bad196795 100644
--- a/net/ipv4/sysctl_net_ipv4.c
+++ b/net/ipv4/sysctl_net_ipv4.c
@@ -55,6 +55,7 @@ static int tcp_min_rcvbuf = 4096;
/* obsolete */
static int sysctl_tcp_low_latency __read_mostly;
+static const unsigned int tcp_ts_id_bits_max = TCP_TS_ID_BITS_MAX;
/* Update system visible IP port range */
static void set_local_port_range(struct net *net, unsigned int low, unsigned int high)
@@ -1227,6 +1228,15 @@ static const struct ctl_table ipv4_net_table[] = {
.mode = 0644,
.proc_handler = proc_dou8vec_minmax,
},
+ {
+ .procname = "tcp_ts_id_bits",
+ .data = &init_net.ipv4.sysctl_tcp_ts_id_bits,
+ .maxlen = sizeof(u8),
+ .mode = 0644,
+ .proc_handler = proc_dou8vec_minmax,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = (void *)&tcp_ts_id_bits_max,
+ },
{
.procname = "tcp_early_retrans",
.data = &init_net.ipv4.sysctl_tcp_early_retrans,
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 68af0283742e..9ec48b9ca01f 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -144,6 +144,49 @@ EXPORT_SYMBOL_GPL(clean_acked_data_flush);
#endif
#ifdef CONFIG_CGROUP_BPF
+static void bpf_skops_tcp_ts_id(struct sock *sk, struct request_sock *req,
+ struct sk_buff *skb)
+{
+ struct bpf_sock_ops_kern sock_ops;
+ u8 ts_id_bits;
+ int err;
+
+ /* Cheapest and most selective test first: it keeps the netns sysctl
+ * out of the SYN path for listeners that never opted in.
+ */
+ if (!BPF_SOCK_OPS_TEST_FLAG(tcp_sk(sk),
+ BPF_SOCK_OPS_TCP_TS_ID_CB_FLAG))
+ return;
+
+ if (tcp_rsk(req)->req_usec_ts) {
+ NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPTSIDSKIPPED);
+ return;
+ }
+
+ ts_id_bits = READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_ts_id_bits);
+ if (!ts_id_bits) {
+ NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPTSIDSKIPPED);
+ return;
+ }
+
+ memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp));
+ sock_ops.op = BPF_SOCK_OPS_TCP_TS_ID_CB;
+ sock_ops.sk = req_to_sk(req);
+ sock_ops.syn_skb = skb;
+ sock_ops.args[1] = ts_id_bits;
+ sock_ops.reply = U32_MAX;
+ bpf_skops_init_skb(&sock_ops, skb, tcp_hdrlen(skb));
+
+ err = BPF_CGROUP_RUN_PROG_SOCK_OPS_SK(&sock_ops, sk);
+ if (!err && sock_ops.reply <= tcp_ts_id_mask(ts_id_bits)) {
+ tcp_rsk(req)->ts_id = sock_ops.reply;
+ tcp_rsk(req)->ts_id_bits = ts_id_bits;
+ NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPTSIDASSIGNED);
+ } else {
+ NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPTSIDDECLINED);
+ }
+}
+
static void bpf_skops_parse_hdr(struct sock *sk, struct sk_buff *skb)
{
bool unknown_opt = tcp_sk(sk)->rx_opt.saw_unknown &&
@@ -198,6 +241,11 @@ static void bpf_skops_established(struct sock *sk, int bpf_op,
BPF_CGROUP_RUN_PROG_SOCK_OPS(&sock_ops);
}
#else
+static void bpf_skops_tcp_ts_id(struct sock *sk, struct request_sock *req,
+ struct sk_buff *skb)
+{
+}
+
static void bpf_skops_parse_hdr(struct sock *sk, struct sk_buff *skb)
{
}
@@ -7781,6 +7829,8 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops,
sk_rx_queue_set(req_to_sk(req), skb);
if (!want_cookie) {
tcp_reqsk_record_syn(sk, req, skb);
+ if (inet_rsk(req)->tstamp_ok)
+ bpf_skops_tcp_ts_id(sk, req, skb);
fastopen_sk = tcp_try_fastopen(sk, skb, req, &foc, dst);
}
if (fastopen_sk) {
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index ebe5aa70374a..4987f6b4bfc3 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -3473,6 +3473,7 @@ static int __net_init tcp_sk_init(struct net *net)
net->ipv4.sysctl_tcp_sack = 1;
net->ipv4.sysctl_tcp_window_scaling = 1;
net->ipv4.sysctl_tcp_timestamps = 1;
+ net->ipv4.sysctl_tcp_ts_id_bits = 0; /* Disabled by default */
net->ipv4.sysctl_tcp_early_retrans = 3;
net->ipv4.sysctl_tcp_recovery = TCP_RACK_LOSS_DETECTION;
net->ipv4.sysctl_tcp_slow_start_after_idle = 1; /* By default, RFC2861 behavior. */
diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c
index 43580b395bff..79fd943c6a1d 100644
--- a/net/ipv4/tcp_minisocks.c
+++ b/net/ipv4/tcp_minisocks.c
@@ -719,6 +719,15 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
tsecr_reject = !between(tmp_opt.rcv_tsecr,
tcp_rsk(req)->snt_tsval_first,
READ_ONCE(tcp_rsk(req)->snt_tsval_last));
+ /* First echo of a TSval we encoded: a mismatch
+ * here means TSecr was rewritten in flight.
+ */
+ if (tcp_rsk(req)->ts_id_bits &&
+ !tcp_tsecr_id_ok(tmp_opt.rcv_tsecr,
+ tcp_rsk(req)->ts_id,
+ tcp_rsk(req)->ts_id_bits))
+ NET_INC_STATS(sock_net(sk),
+ LINUX_MIB_TCPTSIDECHOMISMATCH);
tmp_opt.rcv_tsecr =
tcp_tsecr_decode(tmp_opt.rcv_tsecr,
tcp_rsk(req)->ts_off,
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 732b35cc08d1..b440a3fd96d0 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -7062,6 +7062,7 @@ struct bpf_sock_ops {
* header has not been written.
* BPF_SOCK_OPS_WRITE_HDR_OPT_CB: The header and options have
* been written so far.
+ * BPF_SOCK_OPS_TCP_TS_ID_CB: The SYN received by a listener.
* BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB: The SYNACK that concludes
* the 3WHS.
* BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB: The ACK that concludes
@@ -7137,8 +7138,12 @@ enum {
* options first before the BPF program does.
*/
BPF_SOCK_OPS_WRITE_HDR_OPT_CB_FLAG = (1<<6),
+ /* Call bpf to select a TCP timestamp connection ID for a passive
+ * connection before sending the SYN-ACK.
+ */
+ BPF_SOCK_OPS_TCP_TS_ID_CB_FLAG = (1 << 7),
/* Mask of all currently supported cb flags */
- BPF_SOCK_OPS_ALL_CB_FLAGS = 0x7F,
+ BPF_SOCK_OPS_ALL_CB_FLAGS = 0xFF,
};
enum {
@@ -7282,6 +7287,14 @@ enum {
* sendmsg timestamp with corresponding
* tskey.
*/
+ BPF_SOCK_OPS_TCP_TS_ID_CB, /* Called after a passive SYN created a
+ * request socket and before the SYN-ACK is
+ * sent. Not called for SYN cookies. Arg2 is
+ * the number of low timestamp bits available
+ * for an identifier. Set reply to an identifier
+ * that fits in Arg2 bits. Replies are ignored
+ * for usec timestamps.
+ */
};
/* List of TCP states. There is a build check in net/ipv4/tcp.c to detect
diff --git a/tools/testing/selftests/bpf/prog_tests/tcpbpf_user.c b/tools/testing/selftests/bpf/prog_tests/tcpbpf_user.c
index 7e8fe1bad03f..08f667705291 100644
--- a/tools/testing/selftests/bpf/prog_tests/tcpbpf_user.c
+++ b/tools/testing/selftests/bpf/prog_tests/tcpbpf_user.c
@@ -26,7 +26,7 @@ static void verify_result(struct tcpbpf_globals *result)
ASSERT_EQ(result->bytes_acked, 1002, "bytes_acked");
ASSERT_EQ(result->data_segs_in, 1, "data_segs_in");
ASSERT_EQ(result->data_segs_out, 1, "data_segs_out");
- ASSERT_EQ(result->bad_cb_test_rv, 0x80, "bad_cb_test_rv");
+ ASSERT_EQ(result->bad_cb_test_rv, 0x100, "bad_cb_test_rv");
ASSERT_EQ(result->good_cb_test_rv, 0, "good_cb_test_rv");
ASSERT_EQ(result->num_listen, 1, "num_listen");
diff --git a/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c b/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c
index 6935f32eeb8f..3c011f4684c4 100644
--- a/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c
@@ -92,7 +92,7 @@ int bpf_testcb(struct bpf_sock_ops *skops)
break;
case BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB:
/* Test failure to set largest cb flag (assumes not defined) */
- global.bad_cb_test_rv = bpf_sock_ops_cb_flags_set(skops, 0x80);
+ global.bad_cb_test_rv = bpf_sock_ops_cb_flags_set(skops, 0x100);
/* Set callback */
global.good_cb_test_rv = bpf_sock_ops_cb_flags_set(skops,
BPF_SOCK_OPS_STATE_CB_FLAG);
--
2.47.3
^ permalink raw reply [flat|nested] 5+ messages in thread
* [RFC PATCH net-next 3/4] tcp: add TCP_TS_ID socket option
2026-10-13 10:39 [RFC PATCH net-next 0/4] tcp: carry a BPF-chosen identifier in TSval low bits Willem Ferguson
2026-10-13 10:39 ` [RFC PATCH net-next 1/4] tcp: add plumbing to carry an " Willem Ferguson
2026-10-13 10:39 ` [RFC PATCH net-next 2/4] tcp: add BPF_SOCK_OPS_TCP_TS_ID_CB to assign a TSval identifier Willem Ferguson
@ 2026-10-13 10:39 ` Willem Ferguson
2026-10-13 10:39 ` [RFC PATCH net-next 4/4] selftests/bpf: add tcp_ts_id test Willem Ferguson
3 siblings, 0 replies; 5+ messages in thread
From: Willem Ferguson @ 2026-10-13 10:39 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, ncardwell, bpf, ast, daniel,
andrii, eddyz87, memxor, linux-doc, corbet, linux-kselftest,
shuah, linux-kernel, kernel-team
Add a read-only TCP_TS_ID option so applications and tests can see
which TSval identifier, if any, was assigned to a connection. The value
packs the identifier width into bits 8..15 and the identifier into bits
0..7. Zero means no identifier is encoded.
The option cannot be set. The identifier is chosen by
BPF_SOCK_OPS_TCP_TS_ID_CB when the SYN arrives, and there is no way to
restore it onto a new socket, so connections that carry one do not
survive TCP_REPAIR. ip-sysctl.rst already documents that limitation.
Signed-off-by: Willem Ferguson <wferguson@cloudflare.com>
---
Documentation/networking/ip-sysctl.rst | 3 ++-
include/uapi/linux/tcp.h | 8 ++++++++
net/ipv4/tcp.c | 3 +++
3 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/Documentation/networking/ip-sysctl.rst b/Documentation/networking/ip-sysctl.rst
index ce669f282250..d5e992623326 100644
--- a/Documentation/networking/ip-sysctl.rst
+++ b/Documentation/networking/ip-sysctl.rst
@@ -1123,7 +1123,8 @@ tcp_ts_id_bits - INTEGER
often than this interval prevents it.
Progress is visible through the TCPTSId* counters in
- /proc/net/netstat.
+ /proc/net/netstat, and a connection's identifier can be read with the
+ TCP_TS_ID socket option.
Checkpoint/restore is not supported. The identifier is selected by BPF
when the SYN arrives and cannot be restored onto a new socket, so a
diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index 03772dd4d399..5d7f37b6f8d9 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -142,6 +142,14 @@ enum {
#define TCP_RTO_MAX_MS 44 /* max rto time in ms */
#define TCP_RTO_MIN_US 45 /* min rto time in us */
#define TCP_DELACK_MAX_US 46 /* max delayed ack time in us */
+#define TCP_TS_ID 47 /* Read TSval identifier and its width */
+
+/* TCP_TS_ID value layout: identifier width in bits 8..15, identifier in
+ * bits 0..7. A value of 0 means no identifier is encoded. Read-only: the
+ * identifier is chosen by BPF_SOCK_OPS_TCP_TS_ID_CB and cannot be restored,
+ * so connections with an identifier do not survive TCP_REPAIR.
+ */
+#define TCP_TS_ID_BITS_SHIFT 8
#define TCP_REPAIR_ON 1
#define TCP_REPAIR_OFF 0
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index af994edde7f7..5991b16396b9 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -4711,6 +4711,9 @@ int do_tcp_getsockopt(struct sock *sk, int level,
else
val &= ~1;
break;
+ case TCP_TS_ID:
+ val = ((u32)tp->ts_id_bits << TCP_TS_ID_BITS_SHIFT) | tp->ts_id;
+ break;
case TCP_NOTSENT_LOWAT:
val = READ_ONCE(tp->notsent_lowat);
break;
--
2.47.3
^ permalink raw reply [flat|nested] 5+ messages in thread
* [RFC PATCH net-next 4/4] selftests/bpf: add tcp_ts_id test
2026-10-13 10:39 [RFC PATCH net-next 0/4] tcp: carry a BPF-chosen identifier in TSval low bits Willem Ferguson
` (2 preceding siblings ...)
2026-10-13 10:39 ` [RFC PATCH net-next 3/4] tcp: add TCP_TS_ID socket option Willem Ferguson
@ 2026-10-13 10:39 ` Willem Ferguson
3 siblings, 0 replies; 5+ messages in thread
From: Willem Ferguson @ 2026-10-13 10:39 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, ncardwell, bpf, ast, daniel,
andrii, eddyz87, memxor, linux-doc, corbet, linux-kselftest,
shuah, linux-kernel, kernel-team
Exercise BPF_SOCK_OPS_TCP_TS_ID_CB end to end over IPv6 loopback.
A cgroup sock_ops program sets BPF_SOCK_OPS_TCP_TS_ID_CB_FLAG on the
listener and answers the callback. With tcp_timestamps=1 and
tcp_ts_id_bits=4 the test checks that:
- the callback runs exactly once, on the SYN, with args[1] equal to
the configured width;
- the accepted socket reports the identifier and width through
TCP_TS_ID;
- the connecting socket reports no identifier, as only passive opens
are covered;
- a reply that does not fit in the width ("decline" subtest) leaves the
accepted socket without an identifier.
The test skips when tcp_ts_id_bits does not exist and restores both
sysctls afterwards. It defines TCP_TS_ID locally when the libc headers
do not have it yet.
Signed-off-by: Willem Ferguson <wferguson@cloudflare.com>
---
.../selftests/bpf/prog_tests/tcp_ts_id.c | 150 ++++++++++++++++++
.../selftests/bpf/progs/test_tcp_ts_id_kern.c | 32 ++++
2 files changed, 182 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/tcp_ts_id.c
create mode 100644 tools/testing/selftests/bpf/progs/test_tcp_ts_id_kern.c
diff --git a/tools/testing/selftests/bpf/prog_tests/tcp_ts_id.c b/tools/testing/selftests/bpf/prog_tests/tcp_ts_id.c
new file mode 100644
index 000000000000..25c1f6d8dd43
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/tcp_ts_id.c
@@ -0,0 +1,150 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <errno.h>
+
+#include <test_progs.h>
+#include <network_helpers.h>
+
+#include "test_tcp_ts_id_kern.skel.h"
+
+#define LO_ADDR6 "::1"
+#define CG_NAME "/tcp-ts-id-test"
+
+#ifndef TCP_TS_ID
+#define TCP_TS_ID 47
+#define TCP_TS_ID_BITS_SHIFT 8
+#endif
+
+#define TS_ID_BITS_SYSCTL "/proc/sys/net/ipv4/tcp_ts_id_bits"
+#define TIMESTAMPS_SYSCTL "/proc/sys/net/ipv4/tcp_timestamps"
+#define EXPECTED_TS_ID 7
+#define EXPECTED_TS_ID_BITS 4
+#define EXPECTED_TS_ID_BITS_STR "4"
+
+static char saved_ts_id_bits[16];
+static char saved_timestamps[16];
+
+static int save_sysctl(const char *path, char *buf)
+{
+ int err = 0;
+ FILE *f;
+
+ f = fopen(path, "r");
+ if (!f)
+ return -errno;
+
+ if (fscanf(f, "%15s", buf) != 1)
+ err = -EINVAL;
+ fclose(f);
+
+ return err;
+}
+
+static void restore_sysctl(const char *path, const char *buf)
+{
+ if (buf[0])
+ write_sysctl(path, buf);
+}
+
+static void run_ts_id_test(int cg_fd, bool decline)
+{
+ int listen_fd = -1, cli_fd = -1, accept_fd = -1;
+ struct test_tcp_ts_id_kern *skel;
+ socklen_t optlen;
+ int ts_id;
+
+ skel = test_tcp_ts_id_kern__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "open_and_load"))
+ return;
+
+ skel->bss->ts_id_reply = EXPECTED_TS_ID;
+ skel->bss->ts_id_decline = decline;
+
+ skel->links.tcp_ts_id_sockops =
+ bpf_program__attach_cgroup(skel->progs.tcp_ts_id_sockops,
+ cg_fd);
+ if (!ASSERT_OK_PTR(skel->links.tcp_ts_id_sockops, "attach_cgroup"))
+ goto out;
+
+ /* The listener has to be created after the attach to get LISTEN_CB. */
+ listen_fd = start_server(AF_INET6, SOCK_STREAM, LO_ADDR6, 0, 0);
+ if (!ASSERT_NEQ(listen_fd, -1, "start_server"))
+ goto out;
+
+ cli_fd = connect_to_fd(listen_fd, 0);
+ if (!ASSERT_NEQ(cli_fd, -1, "connect_to_fd"))
+ goto out;
+
+ accept_fd = accept(listen_fd, NULL, NULL);
+ if (!ASSERT_NEQ(accept_fd, -1, "accept"))
+ goto out;
+
+ ASSERT_EQ(skel->bss->ts_id_cb_count, 1, "cb_count");
+ ASSERT_EQ(skel->bss->ts_id_cb_syn, 1, "cb_on_syn");
+ ASSERT_EQ(skel->bss->ts_id_cb_bits, EXPECTED_TS_ID_BITS, "cb_bits");
+
+ /* The identifier must reach the child socket created by the SYN. */
+ optlen = sizeof(ts_id);
+ if (ASSERT_OK(getsockopt(accept_fd, IPPROTO_TCP, TCP_TS_ID, &ts_id,
+ &optlen), "getsockopt(accept_fd, TCP_TS_ID)"))
+ ASSERT_EQ(ts_id, decline ? 0 :
+ (EXPECTED_TS_ID_BITS << TCP_TS_ID_BITS_SHIFT) |
+ EXPECTED_TS_ID, "server_ts_id");
+
+ /* Identifiers are passive-open only, so the opener must have none. */
+ optlen = sizeof(ts_id);
+ if (ASSERT_OK(getsockopt(cli_fd, IPPROTO_TCP, TCP_TS_ID, &ts_id,
+ &optlen), "getsockopt(cli_fd, TCP_TS_ID)"))
+ ASSERT_EQ(ts_id, 0, "client_ts_id");
+out:
+ if (accept_fd != -1)
+ close(accept_fd);
+ if (cli_fd != -1)
+ close(cli_fd);
+ if (listen_fd != -1)
+ close(listen_fd);
+ test_tcp_ts_id_kern__destroy(skel);
+}
+
+void test_tcp_ts_id(void)
+{
+ int cg_fd = -1;
+ int err;
+
+ err = save_sysctl(TS_ID_BITS_SYSCTL, saved_ts_id_bits);
+ if (err == -ENOENT) {
+ test__skip();
+ return;
+ }
+ if (!ASSERT_OK(err, "read " TS_ID_BITS_SYSCTL))
+ return;
+
+ if (!ASSERT_OK(save_sysctl(TIMESTAMPS_SYSCTL, saved_timestamps),
+ "read " TIMESTAMPS_SYSCTL))
+ goto out;
+
+ /* The callback only runs when timestamps are negotiated. */
+ if (!ASSERT_OK(write_sysctl(TIMESTAMPS_SYSCTL, "1"),
+ "write " TIMESTAMPS_SYSCTL))
+ goto out;
+
+ /* Identifiers are disabled by default, so the test has to opt in. */
+ if (!ASSERT_OK(write_sysctl(TS_ID_BITS_SYSCTL, EXPECTED_TS_ID_BITS_STR),
+ "write " TS_ID_BITS_SYSCTL))
+ goto out;
+
+ cg_fd = test__join_cgroup(CG_NAME);
+ if (!ASSERT_GE(cg_fd, 0, "test__join_cgroup(" CG_NAME ")"))
+ goto out;
+
+ if (test__start_subtest("assign"))
+ run_ts_id_test(cg_fd, false);
+
+ /* Only one sockops program may be attached to the cgroup at a time. */
+ if (test__start_subtest("decline"))
+ run_ts_id_test(cg_fd, true);
+out:
+ if (cg_fd != -1)
+ close(cg_fd);
+ restore_sysctl(TIMESTAMPS_SYSCTL, saved_timestamps);
+ restore_sysctl(TS_ID_BITS_SYSCTL, saved_ts_id_bits);
+}
diff --git a/tools/testing/selftests/bpf/progs/test_tcp_ts_id_kern.c b/tools/testing/selftests/bpf/progs/test_tcp_ts_id_kern.c
new file mode 100644
index 000000000000..07e4e56c4fb1
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/test_tcp_ts_id_kern.c
@@ -0,0 +1,32 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+
+__u32 ts_id_cb_count;
+__u32 ts_id_cb_syn;
+__u32 ts_id_cb_bits;
+__u32 ts_id_reply;
+__u32 ts_id_decline; /* set by userspace to exercise the decline path */
+
+SEC("sockops")
+int tcp_ts_id_sockops(struct bpf_sock_ops *skops)
+{
+ if (skops->op == BPF_SOCK_OPS_TCP_LISTEN_CB) {
+ bpf_sock_ops_cb_flags_set(skops,
+ BPF_SOCK_OPS_TCP_TS_ID_CB_FLAG);
+ return 1;
+ }
+
+ if (skops->op != BPF_SOCK_OPS_TCP_TS_ID_CB)
+ return 1;
+
+ ts_id_cb_count++;
+ ts_id_cb_syn = !!(skops->skb_tcp_flags & (1U << 1));
+ ts_id_cb_bits = skops->args[1];
+ /* -1 does not fit in Arg2 bits, so the kernel assigns no ID. */
+ skops->reply = ts_id_decline ? -1 : ts_id_reply;
+
+ return 1;
+}
+
+char _license[] SEC("license") = "GPL";
--
2.47.3
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-10-13 10:39 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-10-13 10:39 [RFC PATCH net-next 0/4] tcp: carry a BPF-chosen identifier in TSval low bits Willem Ferguson
2026-10-13 10:39 ` [RFC PATCH net-next 1/4] tcp: add plumbing to carry an " Willem Ferguson
2026-10-13 10:39 ` [RFC PATCH net-next 2/4] tcp: add BPF_SOCK_OPS_TCP_TS_ID_CB to assign a TSval identifier Willem Ferguson
2026-10-13 10:39 ` [RFC PATCH net-next 3/4] tcp: add TCP_TS_ID socket option Willem Ferguson
2026-10-13 10:39 ` [RFC PATCH net-next 4/4] selftests/bpf: add tcp_ts_id test Willem Ferguson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®