From: Tobias Philipp <tphilipp@gmail.com>
To: bpf@vger.kernel.org
Cc: Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Andrii Nakryiko <andrii@kernel.org>,
Eduard Zingerman <eddyz87@gmail.com>,
Kumar Kartikeya Dwivedi <memxor@gmail.com>,
Martin KaFai Lau <martin.lau@linux.dev>,
Daniel Mack <daniel@zonque.org>,
linux-kernel@vger.kernel.org, Tobias Philipp <tphilipp@gmail.com>
Subject: [PATCH bpf] bpf: Ensure 8-byte alignment for lpm_trie map values
Date: Thu, 19 Nov 2026 16:53:17 +0100 [thread overview]
Message-ID: <20261119155317.11469-1-tphilipp@gmail.com> (raw)
The BPF verifier validates the alignment of PTR_TO_MAP_VALUE accesses in
check_ptr_alignment() only against the offset relative to the start of
the map value, assuming that the map value base address itself is at
least 8-byte aligned.
While hash maps enforce this invariant by rounding the key size up to 8
bytes (l->key + round_up(key_size, 8)), lpm_trie places the map value
immediately after the key bytes at node->data + trie->data_size without
padding. Because trie->data_size (key_size - 4) can be any value from 1
to 256, the value pointer returned by trie_lookup_elem() is only aligned
to (trie->data_size % 8).
In particular, for an IPv4 LPM trie (key_size = 8, data_size = 4), the
map value base address is always 4-byte aligned (4 mod 8, and either 4
or 12 mod 16). The verifier accepts 8-byte atomic operations
(BPF_ATOMIC | BPF_DW) at 8-byte aligned offsets within the value (e.g.
offset 0 and offset 8), but at runtime on arm64 the JITed 64-bit atomic
(stadd) faults with an unhandled hardware alignment fault (ESR.FSC =
0x21, crossing a 16-byte granule boundary on FEAT_LSE2 CPUs, or on any
4-byte aligned address without FEAT_LSE2) and panics the kernel:
Unable to handle kernel paging request at virtual address
0xffff00000159db6c
Mem abort info:
ESR = 0x0000000096000021
EC = 0x25: DABT (current EL), IL = 32 bits
FSC = 0x21: alignment fault
Internal error: Oops: 0x96000021 [#1] SMP
pc : bpf_prog_cc9155b98965fc5e+0x64/0x90
Call trace:
bpf_prog_cc9155b98965fc5e+0x64/0x90 (P)
bpf_dispatcher_xdp_func+0x14/0x28
bpf_test_run+0x11c/0x360
bpf_prog_test_run_xdp+0x358/0x548
Kernel panic - not syncing: Oops: Fatal exception in interrupt
This issue was found while formally specifying eBPF map-value alignment
invariants on Linux 7.3.5.
Fix this by marking struct lpm_trie_node's data[] flexible array as
__aligned(8) and rounding the value offset up to 8 bytes with
round_up(trie->data_size, 8) in trie_alloc(), lpm_trie_node_alloc(),
trie_lookup_elem(), and trie_mem_usage(), matching hashtab. Verified on
Linux 7.3.5 (arm64 under QEMU) with an XDP program executing 64-bit
atomic adds at offsets 0 and 8 of an IPv4 lpm_trie map value.
Fixes: b95a5c4db09b ("bpf: add a longest prefix match trie map implementation")
Assisted-by: LLM
Signed-off-by: Tobias Philipp <tphilipp@gmail.com>
---
A standalone reproducer is available on request.
kernel/bpf/lpm_trie.c | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c
index 4ef9c8c9f3a2..9b4e38d16f21 100644
--- a/kernel/bpf/lpm_trie.c
+++ b/kernel/bpf/lpm_trie.c
@@ -27,7 +27,7 @@ struct lpm_trie_node {
struct lpm_trie_node __rcu *child[2];
u32 prefixlen;
u32 flags;
- u8 data[];
+ u8 data[] __aligned(8);
};
struct lpm_trie {
@@ -286,7 +286,7 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key)
if (!found)
return NULL;
- return found->data + trie->data_size;
+ return found->data + round_up(trie->data_size, 8);
}
static struct lpm_trie_node *lpm_trie_node_alloc(struct lpm_trie *trie,
@@ -302,7 +302,7 @@ static struct lpm_trie_node *lpm_trie_node_alloc(struct lpm_trie *trie,
node->flags = 0;
if (value)
- memcpy(node->data + trie->data_size, value,
+ memcpy(node->data + round_up(trie->data_size, 8), value,
trie->map.value_size);
return node;
@@ -598,7 +598,8 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr)
raw_res_spin_lock_init(&trie->lock);
/* Allocate intermediate and leaf nodes from the same allocator */
- leaf_size = sizeof(struct lpm_trie_node) + trie->data_size +
+ leaf_size = sizeof(struct lpm_trie_node) +
+ round_up(trie->data_size, 8) +
trie->map.value_size;
err = bpf_mem_alloc_init(&trie->ma, leaf_size, false);
if (err)
@@ -766,8 +767,9 @@ static u64 trie_mem_usage(const struct bpf_map *map)
struct lpm_trie *trie = container_of(map, struct lpm_trie, map);
u64 elem_size;
- elem_size = sizeof(struct lpm_trie_node) + trie->data_size +
- trie->map.value_size;
+ elem_size = sizeof(struct lpm_trie_node) +
+ round_up(trie->data_size, 8) +
+ trie->map.value_size;
return elem_size * READ_ONCE(trie->n_entries);
}
--
2.46.0
reply other threads:[~2026-11-19 15:54 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119155317.11469-1-tphilipp@gmail.com \
--to=tphilipp@gmail.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=daniel@zonque.org \
--cc=eddyz87@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=martin.lau@linux.dev \
--cc=memxor@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®