[PATCH bpf] bpf: mask speculative out-of-bounds skb helper loads

[PATCH bpf] bpf: mask speculative out-of-bounds skb helper loads

[Daniël Trujillo via B4 Relay]

From: Daniël Trujillo <datrujillo@nvidia.com>

The fast paths in bpf_skb_load_helper_{8,16,32}() check whether the
requested bytes fit before loading from data + offset. For indirect loads,
packet data can determine the offset. A mispredicted check can cause a
transient out-of-bounds load, as Jin et al. independently demonstrated
for bpf_skb_load_helper_32() in BeeBox (USENIX Security 2024).

A classic BPF filter can shift the loaded value and use it as another
indirect load’s offset (BPF_LSH, BPF_TAX, then BPF_IND),
creating a data-dependent cache footprint.

Leakage was demonstrated with instrumentation that delays branch
resolution with a CLFLUSH before the mispredicted branch and makes the
leaked location deterministic for reproducible measurement. In that
setup, an unprivileged process that attached a classic BPF socket filter
was able to recover kernel values from userspace using Prime+Probe. A
real-world attacker would likely need to delay branch resolution, which
was not demonstrated end-to-end without the added CLFLUSH.

Mask the load address when the converted offset falls outside the
permitted range. This removed the leakage in the described setup.
Derive the lower bound from the network and MAC header
positions. An out-of-bounds transient offset dereferences NULL instead
of a potential secret.

Masking increased the time for a filter with
64 indirect byte loads by roughly 36% in a direct in-kernel test
(280% with LFENCE). In a loopback UDP test with the same filter, the
overhead was roughly 6% (21% with LFENCE).

Fixes: e0cea7ce988c ("bpf: implement ld_abs/ld_ind in native bpf")
Signed-off-by: Daniël Trujillo <datrujillo@nvidia.com>
---
Requesting fix for stable.
---
 net/core/filter.c | 36 +++++++++++++++++++++++++++++++++---
 1 file changed, 33 insertions(+), 3 deletions(-)

diff --git a/net/core/filter.c b/net/core/filter.c
index 1a60a2a70e2d..c50e94df79b3 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -28,6 +28,7 @@
 #include <linux/in.h>
 #include <linux/inet.h>
 #include <linux/netdevice.h>
+#include <linux/nospec.h>
 #include <linux/if_packet.h>
 #include <linux/if_arp.h>
 #include <linux/gfp.h>
@@ -240,6 +241,32 @@ static int bpf_skb_load_helper_convert_offset(const struct sk_buff *skb, int off
 	return INT_MIN;
 }
 
+static __always_inline const void *
+bpf_skb_load_nospec(const struct sk_buff *skb, const void *data,
+		    int headlen, int offset, int len)
+{
+	/* An unset mac_header is maximal and cannot lower the bound. */
+	int header = min(skb->network_header, skb->mac_header);
+	int lower = min(0, header - (int)skb_headroom(skb));
+	unsigned long index, size, mask;
+
+	/* Normalize [lower, headlen) to a zero-based range. */
+	index = (unsigned long)(long)offset - (unsigned long)(long)lower;
+	size = (unsigned long)(long)headlen - (unsigned long)(long)lower;
+
+	OPTIMIZER_HIDE_VAR(index);
+	OPTIMIZER_HIDE_VAR(size);
+
+	/* Check the start and the number of bytes remaining. */
+	mask = array_index_mask_nospec(index, size);
+	/* Each caller passes a constant width, so inlining eliminates this branch. */
+	if (len > 1)
+		mask &= array_index_mask_nospec(len - 1, size - index);
+
+	/* Out-of-bounds transient offsets select NULL as the load address. */
+	return (const void *)(((unsigned long)data + (long)offset) & mask);
+}
+
 BPF_CALL_4(bpf_skb_load_helper_8, const struct sk_buff *, skb, const void *,
 	   data, int, headlen, int, offset)
 {
@@ -251,7 +278,8 @@ BPF_CALL_4(bpf_skb_load_helper_8, const struct sk_buff *, skb, const void *,
 		return -EFAULT;
 
 	if (headlen - offset >= len)
-		return *(u8 *)(data + offset);
+		return *(const u8 *)bpf_skb_load_nospec(skb, data, headlen,
+							  offset, len);
 	if (!skb_copy_bits(skb, offset, &tmp, sizeof(tmp)))
 		return tmp;
 	else
@@ -276,7 +304,8 @@ BPF_CALL_4(bpf_skb_load_helper_16, const struct sk_buff *, skb, const void *,
 		return -EFAULT;
 
 	if (headlen - offset >= len)
-		return get_unaligned_be16(data + offset);
+		return get_unaligned_be16(bpf_skb_load_nospec(skb, data,
+							 headlen, offset, len));
 	if (!skb_copy_bits(skb, offset, &tmp, sizeof(tmp)))
 		return be16_to_cpu(tmp);
 	else
@@ -301,7 +330,8 @@ BPF_CALL_4(bpf_skb_load_helper_32, const struct sk_buff *, skb, const void *,
 		return -EFAULT;
 
 	if (headlen - offset >= len)
-		return get_unaligned_be32(data + offset);
+		return get_unaligned_be32(bpf_skb_load_nospec(skb, data,
+							 headlen, offset, len));
 	if (!skb_copy_bits(skb, offset, &tmp, sizeof(tmp)))
 		return be32_to_cpu(tmp);
 	else

---
base-commit: 7f1e33230b7c7a55318765862b255b41e5c97cfb
change-id: 20261113-cbpf-ind-nospec-v1-bpf-369dd47fc8af

Best regards,
--  
Daniël Trujillo <datrujillo@nvidia.com>

Re: [PATCH bpf] bpf: mask speculative out-of-bounds skb helper loads

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2525 bytes --]

> bpf: mask speculative out-of-bounds skb helper loads

TL;DR: The nospec masking may be incomplete: inline classic BPF ABS/MSH loads
from convert_bpf_ld_abs() on efficient-unaligned-access architectures bypass
the helpers, so a short packet could still transiently read out of bounds.

> diff --git a/net/core/filter.c b/net/core/filter.c
> --- a/net/core/filter.c
> +++ b/net/core/filter.c

[ ... ]

> +static __always_inline const void *
> +bpf_skb_load_nospec(const struct sk_buff *skb, const void *data,
> +		    int headlen, int offset, int len)
> +{

[ ... ]

> +	/* Out-of-bounds transient offsets select NULL as the load address. */
> +	return (const void *)(((unsigned long)data + (long)offset) & mask);
> +}
> +
>  BPF_CALL_4(bpf_skb_load_helper_8, const struct sk_buff *, skb, const void *,
>  	   data, int, headlen, int, offset)
>  {

[ ... ]

>  	if (headlen - offset >= len)
> -		return *(u8 *)(data + offset);
> +		return *(const u8 *)bpf_skb_load_nospec(skb, data, headlen,
> +							  offset, len);

This isn't a bug in the hunk itself, but is the mitigation complete?

On architectures with HAVE_EFFICIENT_UNALIGNED_ACCESS, a classic
BPF_LD|BPF_ABS load with k >= 0, and BPF_LDX|BPF_MSH|BPF_B, do not reach
these helpers.  convert_bpf_ld_abs() emits an inline check and load
instead:

    TMP = H - k;
    if (TMP s< size) goto helper;
    A = *(size *)(D + k);   /* BPF_REG_D == cached skb->data */

This is the same bounds-check-then-load pattern described in the commit
message, and nothing masks it.  An unprivileged SO_ATTACH_FILTER program
chooses k, and packet size controls the linear length.  After training the
branch with long packets, a short packet could transiently load from
skb->data + k past the linear area.  The value could then go through
BPF_LSH/BPF_TAX into a BPF_IND load, which reaches
bpf_skb_load_helper_*() and passes the new mask because the shifted index
is in range.

The commit message says "For indirect loads, packet data can determine the
offset", yet the patch also masks ABS loads that go through the helpers.

Should convert_bpf_ld_abs() get the same treatment (for example, always
calling the helpers, or masking D + k in the emitted code), or should the
changelog explain why constant-offset loads are out of scope?

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/43500835329