[PATCH net-next v4 0/2] tcp: annotate lockless access to sk->sk_err

[PATCH net-next v4 0/2] tcp: annotate lockless access to sk->sk_err

[Quanye Yang via B4 Relay]

do_recvmmsg() and getsockopt(SO_ERROR) call sock_error() without the
socket lock and clear sk_err with xchg(). Annotate the matching
lockless peeks with READ_ONCE().

Patch 1 covers TCP. Patch 2 covers MPTCP, including the remaining
unmarked subflow peeks.

This does not consume sk_err. The check-then-sock_error() TOCTOU on
the no-data paths is a separate issue.

---
Changes in v4:
- split the annotation change, one points to tcp and another points to
  mptcp.
- annotate the remaining unmarked MPTCP sk_err peeks on the
  subflow error-report path.
- Link to v3: https://patch.msgid.link/20260912-mptcp-sk-err-net-v3-1-c41383878bde@proton.me

Changes in v3:
- drop the Fixes tag and retarget to net-next
- annotate the remaining unmarked sk_err peeks on the TCP/MPTCP
  send, recv and splice paths
- Link to v2: https://patch.msgid.link/20260909-mptcp-sk-err-net-v2-1-5044abecac90@proton.me

Changes in v2:
- add Reported-by and Closes for the MPTCP syzkaller report
- Link to v1: https://patch.msgid.link/20260908-mptcp-sk-err-net-v1-1-da71aaec9afd@proton.me

---
Quanye Yang (2):
      tcp: annotate lockless access to sk->sk_err
      mptcp: annotate lockless access to sk->sk_err

 include/net/tcp.h    | 3 ++-
 net/core/stream.c    | 2 +-
 net/ipv4/tcp.c       | 6 +++---
 net/ipv4/tcp_bpf.c   | 7 +++----
 net/mptcp/protocol.c | 8 ++++----
 net/mptcp/subflow.c  | 4 ++--
 6 files changed, 15 insertions(+), 15 deletions(-)
---
base-commit: 211f2a875f6f447745d80d5762d6d614503ac84a
change-id: 20260908-mptcp-sk-err-net-7ff88ef05044

Best regards,
--  
Quanye Yang <quanyeyang@proton.me>

[PATCH net-next v4 1/2] tcp: annotate lockless access to sk->sk_err

[Quanye Yang via B4 Relay]

From: Quanye Yang <quanyeyang@proton.me>

BUG: KCSAN: data-race in do_recvmmsg / mptcp_recvmsg

read-write (marked) to 0xffff8880134d391c of 4 bytes by task 2619 on cpu 1:
 instrument_atomic_read_write include/linux/instrumented.h:113 [inline]
 sock_error include/net/sock.h:2565 [inline]
 do_recvmmsg+0x50c/0x580 net/socket.c:3049
 __sys_recvmmsg net/socket.c:3144 [inline]
 __do_sys_recvmmsg net/socket.c:3167 [inline]
 __se_sys_recvmmsg net/socket.c:3160 [inline]
 __x64_sys_recvmmsg+0x161/0x180 net/socket.c:3160
 x64_sys_call+0x19c7/0x1ca0 arch/x86/include/generated/asm/syscalls_64.h:300
 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
 do_syscall_64+0xde/0x3d0 arch/x86/entry/syscall_64.c:84
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

read to 0xffff8880134d391c of 4 bytes by task 2620 on cpu 0:
 tcp_recv_should_stop include/net/tcp.h:3086 [inline]
 mptcp_recvmsg+0x54d/0xd50 net/mptcp/protocol.c:2466
 inet_recvmsg+0x204/0x210 net/ipv4/af_inet.c:894
 sock_recvmsg_nosec net/socket.c:1151 [inline]
 sock_recvmsg+0x11a/0x140 net/socket.c:1173
 ____sys_recvmsg+0x14b/0x3c0 net/socket.c:2933
 ___sys_recvmsg+0x116/0x160 net/socket.c:2975
 __sys_recvmsg net/socket.c:3008 [inline]
 __do_sys_recvmsg net/socket.c:3014 [inline]
 __se_sys_recvmsg net/socket.c:3011 [inline]
 __x64_sys_recvmsg+0xeb/0x160 net/socket.c:3011
 x64_sys_call+0x1319/0x1ca0 arch/x86/include/generated/asm/syscalls_64.h:48
 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
 do_syscall_64+0xde/0x3d0 arch/x86/entry/syscall_64.c:84
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

value changed: 0x0000006b -> 0x00000000

Reported by Kernel Concurrency Sanitizer on:
CPU: 0 UID: 0 PID: 2620 Comm: syz.2.33 Not tainted 7.2.0-g39d4f32c5d53 #76 PREEMPT(full)
Hardware name: QEMU Ubuntu 26.04 PC (i440FX + PIIX, 1996), BIOS 1.17.0-debian-1.17.0-1ubuntu1 04/01/2014

do_recvmmsg() and getsockopt(SO_ERROR) call sock_error() without the
socket lock. sock_error() clears sk_err with xchg(), which races with
unmarked loads of the same field.

KCSAN reported the unmarked peek in tcp_recv_should_stop(). The same
lockless writer races with the other plain sk_err reads on the TCP
send, recv and splice paths.

Annotate those peeks with READ_ONCE(). No extra ordering is needed:
the value is only used to decide whether I/O should stop. This does
not consume sk_err; the check-then-sock_error() TOCTOU on the no-data
paths is a separate issue. MPTCP peeks are handled in the next patch.

Suggested-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Link: https://lore.kernel.org/netdev/8bbee583-6f21-4817-bfeb-2d60057380a3@linux.dev/
Reported-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/632
Signed-off-by: Quanye Yang <quanyeyang@proton.me>
---
 include/net/tcp.h  | 3 ++-
 net/core/stream.c  | 2 +-
 net/ipv4/tcp.c     | 6 +++---
 net/ipv4/tcp_bpf.c | 7 +++----
 4 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/include/net/tcp.h b/include/net/tcp.h
index 5e5f5f9b89a3..6ee513bdb503 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -3093,7 +3093,8 @@ enum skb_drop_reason tcp_inbound_hash(struct sock *sk,
 
 static inline int tcp_recv_should_stop(struct sock *sk)
 {
-	return sk->sk_err ||
+	/* sk_err can be cleared locklessly by sock_error(). */
+	return READ_ONCE(sk->sk_err) ||
 	       sk->sk_state == TCP_CLOSE ||
 	       (sk->sk_shutdown & RCV_SHUTDOWN) ||
 	       signal_pending(current);
diff --git a/net/core/stream.c b/net/core/stream.c
index 2d748581862d..5c0adc3077f0 100644
--- a/net/core/stream.c
+++ b/net/core/stream.c
@@ -133,7 +133,7 @@ int sk_stream_wait_memory(struct sock *sk, long *timeo_p)
 	while (1) {
 		sk_set_bit(SOCKWQ_ASYNC_NOSPACE, sk);
 
-		if (sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN))
+		if (READ_ONCE(sk->sk_err) || (sk->sk_shutdown & SEND_SHUTDOWN))
 			goto do_error;
 		if (!*timeo_p)
 			goto do_eagain;
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 1c867a302444..9e7fa7e39726 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -843,7 +843,7 @@ ssize_t tcp_splice_read(struct socket *sock, loff_t *ppos,
 				break;
 			if (sock_flag(sk, SOCK_DONE))
 				break;
-			if (sk->sk_err) {
+			if (READ_ONCE(sk->sk_err)) {
 				ret = sock_error(sk);
 				break;
 			}
@@ -1228,7 +1228,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
 	mss_now = tcp_send_mss(sk, &size_goal, flags);
 
 	err = -EPIPE;
-	if (sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN))
+	if (READ_ONCE(sk->sk_err) || (sk->sk_shutdown & SEND_SHUTDOWN))
 		goto do_error;
 
 	while (msg_data_left(msg)) {
@@ -2760,7 +2760,7 @@ static int tcp_recvmsg_locked(struct sock *sk, struct msghdr *msg, size_t len,
 			if (sock_flag(sk, SOCK_DONE))
 				break;
 
-			if (sk->sk_err) {
+			if (READ_ONCE(sk->sk_err)) {
 				copied = sock_error(sk);
 				break;
 			}
diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c
index 2e234d155b5e..a96323e65758 100644
--- a/net/ipv4/tcp_bpf.c
+++ b/net/ipv4/tcp_bpf.c
@@ -286,7 +286,7 @@ static int tcp_bpf_recvmsg_parser(struct sock *sk,
 		if (sock_flag(sk, SOCK_DONE))
 			goto out;
 
-		if (sk->sk_err) {
+		if (READ_ONCE(sk->sk_err)) {
 			copied = sock_error(sk);
 			goto out;
 		}
@@ -553,10 +553,9 @@ static int tcp_bpf_sendmsg(struct sock *sk, struct msghdr *msg, size_t size)
 		bool enospc = false;
 		u32 copy, osize;
 
-		if (sk->sk_err) {
-			err = -sk->sk_err;
+		err = -READ_ONCE(sk->sk_err);
+		if (err)
 			goto out_err;
-		}
 
 		copy = msg_data_left(msg);
 		if (!sk_stream_memory_free(sk))

-- 
2.55.0

Re: [PATCH net-next v4 1/2] tcp: annotate lockless access to sk->sk_err

[David Laight]

On Thu, 17 Sep 2026 06:41:22 -0700
Quanye Yang via B4 Relay <devnull+quanyeyang.proton.me@kernel.org> wrote:

> From: Quanye Yang <quanyeyang@proton.me>
> 
> BUG: KCSAN: data-race in do_recvmmsg / mptcp_recvmsg
> 
> read-write (marked) to 0xffff8880134d391c of 4 bytes by task 2619 on cpu 1:
>  instrument_atomic_read_write include/linux/instrumented.h:113 [inline]
>  sock_error include/net/sock.h:2565 [inline]
>  do_recvmmsg+0x50c/0x580 net/socket.c:3049
>  __sys_recvmmsg net/socket.c:3144 [inline]
>  __do_sys_recvmmsg net/socket.c:3167 [inline]
>  __se_sys_recvmmsg net/socket.c:3160 [inline]
>  __x64_sys_recvmmsg+0x161/0x180 net/socket.c:3160
>  x64_sys_call+0x19c7/0x1ca0 arch/x86/include/generated/asm/syscalls_64.h:300
>  do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
>  do_syscall_64+0xde/0x3d0 arch/x86/entry/syscall_64.c:84
>  entry_SYSCALL_64_after_hwframe+0x77/0x7f
> 
> read to 0xffff8880134d391c of 4 bytes by task 2620 on cpu 0:
>  tcp_recv_should_stop include/net/tcp.h:3086 [inline]
>  mptcp_recvmsg+0x54d/0xd50 net/mptcp/protocol.c:2466
>  inet_recvmsg+0x204/0x210 net/ipv4/af_inet.c:894
>  sock_recvmsg_nosec net/socket.c:1151 [inline]
>  sock_recvmsg+0x11a/0x140 net/socket.c:1173
>  ____sys_recvmsg+0x14b/0x3c0 net/socket.c:2933
>  ___sys_recvmsg+0x116/0x160 net/socket.c:2975
>  __sys_recvmsg net/socket.c:3008 [inline]
>  __do_sys_recvmsg net/socket.c:3014 [inline]
>  __se_sys_recvmsg net/socket.c:3011 [inline]
>  __x64_sys_recvmsg+0xeb/0x160 net/socket.c:3011
>  x64_sys_call+0x1319/0x1ca0 arch/x86/include/generated/asm/syscalls_64.h:48
>  do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
>  do_syscall_64+0xde/0x3d0 arch/x86/entry/syscall_64.c:84
>  entry_SYSCALL_64_after_hwframe+0x77/0x7f
> 
> value changed: 0x0000006b -> 0x00000000
> 
> Reported by Kernel Concurrency Sanitizer on:
> CPU: 0 UID: 0 PID: 2620 Comm: syz.2.33 Not tainted 7.2.0-g39d4f32c5d53 #76 PREEMPT(full)
> Hardware name: QEMU Ubuntu 26.04 PC (i440FX + PIIX, 1996), BIOS 1.17.0-debian-1.17.0-1ubuntu1 04/01/2014
> 
> do_recvmmsg() and getsockopt(SO_ERROR) call sock_error() without the
> socket lock. sock_error() clears sk_err with xchg(), which races with
> unmarked loads of the same field.
> 
> KCSAN reported the unmarked peek in tcp_recv_should_stop(). The same
> lockless writer races with the other plain sk_err reads on the TCP
> send, recv and splice paths.
> 
> Annotate those peeks with READ_ONCE(). No extra ordering is needed:
> the value is only used to decide whether I/O should stop. This does
> not consume sk_err; the check-then-sock_error() TOCTOU on the no-data
> paths is a separate issue. MPTCP peeks are handled in the next patch.

Don't they need annotating with data_race() ?

David

Re: [PATCH net-next v4 1/2] tcp: annotate lockless access to sk->sk_err

[quanyeyang]

On Friday, September 18th, 2026 at AM 2:51, David Laight <david.laight.linux@gmail.com> wrote:

> Don't they need annotating with data_race() ?

I think READ_ONCE() is the intended marker here.

These loads are part of the I/O control flow (stop send/recv/splice),
not diagnostic or heuristic reads. data_race() would silence KCSAN
but would not restrict compiler optimizations, so the load could still
be torn, fused or reissued. READ_ONCE() is a single-copy atomic load
and pairs with the existing WRITE_ONCE()/xchg() updates of sk_err,
same as tcp_poll() after e13ec3da05d1.

sock_error() uses data_race() only on the fast-path "is it zero?"
check, which is then validated by xchg(). That pattern does not
apply to these peeks.

Quanye

Re: [PATCH net-next v4 1/2] tcp: annotate lockless access to sk->sk_err

[David Laight]

On Fri, 18 Sep 2026 10:33:04 +0000
quanyeyang <quanyeyang@proton.me> wrote:

> On Friday, September 18th, 2026 at AM 2:51, David Laight <david.laight.linux@gmail.com> wrote:
> 
> > Don't they need annotating with data_race() ?  
> 
> I think READ_ONCE() is the intended marker here.

If the race is with locked accesses I think you need both.

David

> 
> These loads are part of the I/O control flow (stop send/recv/splice),
> not diagnostic or heuristic reads. data_race() would silence KCSAN
> but would not restrict compiler optimizations, so the load could still
> be torn, fused or reissued. READ_ONCE() is a single-copy atomic load
> and pairs with the existing WRITE_ONCE()/xchg() updates of sk_err,
> same as tcp_poll() after e13ec3da05d1.
> 
> sock_error() uses data_race() only on the fast-path "is it zero?"
> check, which is then validated by xchg(). That pattern does not
> apply to these peeks.
> 
> Quanye
> 

Re: [PATCH net-next v4 1/2] tcp: annotate lockless access to sk->sk_err

[quanyeyang]

On Friday, September 18th, 2026 at AM 5:27, David Laight <david.laight.linux@gmail.com> wrote:

> If the race is with locked accesses I think you need both.

Thanks — I think I see the case you mean, and I may have been looking
at a different pairing.

If the stores stay unmarked because they run under a lock, and the
reader is the lockless side, then yes: READ_ONCE() alone is not enough
for KCSAN.  The watchpoint is still the plain locked store, so the
report is marked vs unmarked.  data_race(READ_ONCE()) is the usual
answer there: READ_ONCE() for the compiler, data_race() so KCSAN does
not treat those remaining unmarked locked stores as bugs.

The splat this series is trying to fix looks the other way around,
at least as I read it.  The access KCSAN caught on the write side is
the lockless xchg() in sock_error(), from getsockopt(SO_ERROR) /
recvmmsg.  Those paths do not take lock_sock(), so the peek under
lock_sock() is not actually exclusive against the consumer — the two
sides really do run concurrently.  The peek is a plain load, so that
is the unmarked side of the report.

TCP's own stores of sk_err were already converted to WRITE_ONCE() in
e13ec3da05d1, because tcp_poll() already peeks without the socket
lock.  The consume path is xchg().  Adding READ_ONCE() on the peeks
is meant to mark both sides of that race, same as tcp_poll() already
does.  Wrapping them in data_race() would also silence any leftover
plain stores instead of leaving KCSAN able to point at them.  These
loads also feed send/recv/splice control flow rather than a
diagnostic or heuristic, so READ_ONCE() seemed the closer match.

If you had a specific locked store in mind that we should still treat
as unmarked, or a reason the lock_sock() peek should stay a plain
access, I would like to understand it.

Thanks,
Quanye

[PATCH net-next v4 2/2] mptcp: annotate lockless access to sk->sk_err

[Quanye Yang via B4 Relay]

From: Quanye Yang <quanyeyang@proton.me>

sock_error() can clear sk_err with xchg() without the socket lock.
Annotate the remaining MPTCP peeks on the msk send/recv/splice paths
and on the subflow error-report path with READ_ONCE().

This does not consume sk_err; the check-then-sock_error() TOCTOU on
the no-data paths is a separate issue.

Signed-off-by: Quanye Yang <quanyeyang@proton.me>
---
 net/mptcp/protocol.c | 8 ++++----
 net/mptcp/subflow.c  | 4 ++--
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index e1f08f71cdb1..91637d1ed070 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -969,7 +969,7 @@ static bool move_skbs_to_msk(struct mptcp_sock *msk, struct sock *ssk)
 
 	moved = __mptcp_move_skbs_from_subflow(msk, ssk, true);
 	__mptcp_ofo_queue(msk);
-	if (unlikely(ssk->sk_err))
+	if (unlikely(READ_ONCE(ssk->sk_err)))
 		__mptcp_subflow_error_report(sk, ssk);
 
 	/* If the moves have caught up with the DATA_FIN sequence number
@@ -2028,7 +2028,7 @@ static int mptcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	}
 
 	ret = -EPIPE;
-	if (unlikely(sk->sk_err || (sk->sk_shutdown & SEND_SHUTDOWN)))
+	if (unlikely(READ_ONCE(sk->sk_err) || (sk->sk_shutdown & SEND_SHUTDOWN)))
 		goto do_error;
 
 	pfrag = sk_page_frag(sk);
@@ -2429,7 +2429,7 @@ static int mptcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
 			    !timeo)
 				break;
 		} else {
-			if (sk->sk_err) {
+			if (READ_ONCE(sk->sk_err)) {
 				copied = sock_error(sk);
 				break;
 			}
@@ -4655,7 +4655,7 @@ static ssize_t mptcp_splice_read(struct socket *sock, loff_t *ppos,
 				break;
 			if (sock_flag(sk, SOCK_DONE))
 				break;
-			if (sk->sk_err) {
+			if (READ_ONCE(sk->sk_err)) {
 				ret = sock_error(sk);
 				break;
 			}
diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c
index af81ad5e699d..ec9668fe8d5b 100644
--- a/net/mptcp/subflow.c
+++ b/net/mptcp/subflow.c
@@ -1541,7 +1541,7 @@ static void subflow_data_ready(struct sock *sk)
 		if (mptcp_data_avail(msk) < parent->sk_rcvlowat &&
 		    (tcp_sk(sk)->rcv_nxt - tcp_sk(sk)->rcv_wup) > inet_csk(sk)->icsk_ack.rcv_mss)
 			inet_csk(sk)->icsk_ack.pending |= ICSK_ACK_NOW;
-	} else if (unlikely(sk->sk_err)) {
+	} else if (unlikely(READ_ONCE(sk->sk_err))) {
 		subflow_error_report(sk);
 	}
 }
@@ -1889,7 +1889,7 @@ static void subflow_state_change(struct sock *sk)
 	 */
 	if (mptcp_subflow_data_available(sk))
 		mptcp_data_ready(parent, sk);
-	else if (unlikely(sk->sk_err))
+	else if (unlikely(READ_ONCE(sk->sk_err)))
 		subflow_error_report(sk);
 
 	subflow_sched_work_if_closed(mptcp_sk(parent), sk);

-- 
2.55.0

Re: [PATCH net-next v4 2/2] mptcp: annotate lockless access to sk->sk_err

[Matthieu Baerts]

Hi Quanye,

On 17/09/2026 15:41, Quanye Yang via B4 Relay wrote:
> From: Quanye Yang <quanyeyang@proton.me>
> 
> sock_error() can clear sk_err with xchg() without the socket lock.
> Annotate the remaining MPTCP peeks on the msk send/recv/splice paths
> and on the subflow error-report path with READ_ONCE().

This looks good to me, thanks!

Reviewed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.