[PATCH bpf 0/2] bpf: fix per-cpu initialization of a BPF_F_CPU created hash element

[PATCH bpf 0/2] bpf: fix per-cpu initialization of a BPF_F_CPU created hash element

[Donggeun Yoo]

A BPF_F_CPU update that creates a [lru_]percpu_hash element writes the
named CPU's slot and leaves the others holding the values of whatever
element was recycled into it, so a lookup of the new key returns a
deleted key's per-cpu values.

Patch 1 zero-fills the other CPUs.  Patch 2 adds the selftest: the
existing cpu_flag subtests always prime a key with BPF_F_ALL_CPUS
first, so the element always exists by the time the flag is used and
the create path is not covered today.

Tested on x86_64 under QEMU/KVM, 4 vCPUs, on bpf/master a11212910cf0,
configured from the selftests' own fragments (config, config.vm,
config.x86_64):

  test_progs -t percpu_alloc,map_init,htab_update,htab_mem,for_each,
    map_kptr,map_ptr,map_in_map,lru_bug,map_lookup_percpu_elem,
    map_percpu_stats,batch_map_ops,htab_reuse,timer,task_work

  without patch 1   31/112 PASSED, 0 SKIPPED, 1/3 FAILED
  with patch 1      32/115 PASSED, 0 SKIPPED, 0/0 FAILED

The three failures are the new subtests, one per allocation mode:

  #304/18  percpu_alloc/cpu_flag_create_percpu_hash:FAIL
  #304/19  percpu_alloc/cpu_flag_create_percpu_hash_malloc:FAIL
  #304/20  percpu_alloc/cpu_flag_create_lru_percpu_hash:FAIL
  test_percpu_map_cpu_flag_create:FAIL:value on specified cpu
    unexpected value on specified cpu: actual 3735929054 != expected 0

3735929054 is 0xdeadc0de, the value the deleted key left behind.  The
BPF_F_NO_PREALLOC case reproduces as reliably as the preallocated ones:
five runs of each arm, five failures without the patch and five passes
with it.

Donggeun Yoo (2):
  bpf: Zero-fill other CPUs when BPF_F_CPU creates a per-cpu hash
    element
  selftests/bpf: Test per-cpu initialization of a BPF_F_CPU created
    element

 kernel/bpf/hashtab.c                          |  6 +-
 .../selftests/bpf/prog_tests/percpu_alloc.c   | 74 +++++++++++++++++++
 2 files changed, 77 insertions(+), 3 deletions(-)

-- 
2.53.0

[PATCH bpf 1/2] bpf: Zero-fill other CPUs when BPF_F_CPU creates a per-cpu hash element

[Donggeun Yoo]

pcpu_init_value() initializes the per-cpu area of a newly created
[lru_]percpu_hash element.  That area is recycled and still holds the
values of whatever element occupied it before, so when the value comes
from a BPF program (onallcpus == false) the function writes the running
CPU's slot and zeroes the rest.

bpf_percpu_hash_update() always passes onallcpus == true, and that arm
calls pcpu_copy_value(), which used to write every CPU.  That changed in
commit c6936161fd55 ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags
support for percpu_hash and lru_percpu_hash maps"): with BPF_F_CPU it
writes the one CPU named in map_flags and returns.  On the create
path the remaining slots are left as they were, and a lookup of the new
key hands back the recycled element's values:

  update(k1, 0xdeadc0de, BPF_F_ALL_CPUS)  every CPU holds 0xdeadc0de
  delete(k1)                              element back on the freelist
  update(k2, 0xc0ffee, BPF_F_CPU | 0)     creates, writes CPU 0 only
  lookup(k2)                              CPU 0 0xc0ffee, rest 0xdeadc0de

Commit d3bec0138bfb ("bpf: Zero-fill re-used per-cpu map element")
established that a re-used element must not return the previous
tenant's values.  BPF_F_CPU is the first way to reach pcpu_init_value()
writing a single CPU with onallcpus set, so extend the zero-filling arm
to cover it, with map_flags >> 32 naming the CPU that receives the
value.

Only creation is affected: pcpu_init_value() is reached from the two
create branches, while an update of an existing element goes straight to
pcpu_copy_value(), where writing one CPU and leaving the others is the
point of the flag.

Fixes: c6936161fd55 ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_hash and lru_percpu_hash maps")
Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
---
Tested on x86_64 under QEMU/KVM against bpf/master a11212910cf0: with
this patch the selftest in 2/2 passes on all three allocation modes,
and without it all three read 0xdeadc0de where they expect 0.  Numbers
in the cover letter.

The merged arm no longer calls bpf_obj_cancel_fields() on the named
CPU.  That call is inert on this path: it acts only on BPF_TIMER,
BPF_WORKQUEUE and BPF_TASK_WORK, and map_check_btf() rejects all three
for [lru_]percpu_hash.

 kernel/bpf/hashtab.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index 4f495dcbf670c..c4683d0e4c149 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -1056,12 +1056,12 @@ static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr,
 	 * known initial values for cpus other than current one
 	 * (onallcpus=false always when coming from bpf prog).
 	 */
-	if (!onallcpus) {
-		int current_cpu = raw_smp_processor_id();
+	if (!onallcpus || (map_flags & BPF_F_CPU)) {
+		int init_cpu = onallcpus ? map_flags >> 32 : raw_smp_processor_id();
 		int cpu;
 
 		for_each_possible_cpu(cpu) {
-			if (cpu == current_cpu)
+			if (cpu == init_cpu)
 				copy_map_value(&htab->map, per_cpu_ptr(pptr, cpu), value);
 			else /* Since elem is preallocated, we cannot touch special fields */
 				zero_map_value(&htab->map, per_cpu_ptr(pptr, cpu));
-- 
2.53.0

Re: [PATCH bpf 1/2] bpf: Zero-fill other CPUs when BPF_F_CPU creates a per-cpu hash element

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2095 bytes --]

> diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
> index 4f495dcbf670c..c4683d0e4c149 100644
> --- a/kernel/bpf/hashtab.c
> +++ b/kernel/bpf/hashtab.c
> @@ -1056,12 +1056,12 @@ static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr,
>  	 * known initial values for cpus other than current one
>  	 * (onallcpus=false always when coming from bpf prog).
>  	 */
> -	if (!onallcpus) {
> -		int current_cpu = raw_smp_processor_id();
> +	if (!onallcpus || (map_flags & BPF_F_CPU)) {
> +		int init_cpu = onallcpus ? map_flags >> 32 : raw_smp_processor_id();
>  		int cpu;
>
>  		for_each_possible_cpu(cpu) {
> -			if (cpu == current_cpu)
> +			if (cpu == init_cpu)
>  				copy_map_value(&htab->map, per_cpu_ptr(pptr, cpu), value);
>  			else /* Since elem is preallocated, we cannot touch special fields */
>  				zero_map_value(&htab->map, per_cpu_ptr(pptr, cpu));

Does the block comment above this condition need updating? It says the value
goes to the "current one" and explains the whole branch with "onallcpus=false
always when coming from bpf prog."

With the patch, the branch is also taken when onallcpus is true and BPF_F_CPU
is set. In that case:

    if (!onallcpus || (map_flags & BPF_F_CPU)) {
        int init_cpu = onallcpus ? map_flags >> 32 : raw_smp_processor_id();

the value goes to the CPU named in map_flags >> 32, while the current CPU
might be one of the CPUs that gets zero-filled.

The rationale also seems specific to the BPF program case. For BPF_F_CPU
updates there is no BPF program involved (htab_map_check_update_flags()
returns -EINVAL for !onallcpus && map_flags > BPF_EXIST), so the reason to
zero the other slots is that the per-cpu area is recycled and must not hand
back the previous tenant's values, as the commit message explains.

Could the comment be updated to cover both entry conditions?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/35502758763

Re: [PATCH bpf 1/2] bpf: Zero-fill other CPUs when BPF_F_CPU creates a per-cpu hash element

[Donggeun Yoo]

On Sun, 20 Sep 2026 10:19:00 +0000 (UTC), bot+bpf-ci@kernel.org wrote:
> Does the block comment above this condition need updating? It says the value
> goes to the "current one" and explains the whole branch with "onallcpus=false
> always when coming from bpf prog."
[...]
> Could the comment be updated to cover both entry conditions?

Yes, it should. What is there stays correct for the !onallcpus arm, where the
value does go to the current cpu and the reason is the one given, so v2 adds
the second arm rather than rewording the first:

	/* When not setting the initial value on all cpus, zero-fill element
	 * values for other cpus. Otherwise, bpf program has no way to ensure
	 * known initial values for cpus other than current one
	 * (onallcpus=false always when coming from bpf prog).  A BPF_F_CPU
	 * update also sets a single cpu, and the element may be recycled, so
	 * the other cpus must not keep what the previous key left there.
	 */

Thanks,
Donggeun

Re: [PATCH bpf 1/2] bpf: Zero-fill other CPUs when BPF_F_CPU creates a per-cpu hash element

[Alexei Starovoitov]

On Sun, Sep 20, 2026 at 06:31 PM Donggeun Yoo <donggeunyoo.kernel@gmail.com> wrote:
> Fixes: c6936161fd55 ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_hash and lru_percpu_hash maps")
> Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>

Pls cc Leon on respin.

[...]

> @@ -1056,12 +1056,12 @@ static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr,
>  	 * known initial values for cpus other than current one
>  	 * (onallcpus=false always when coming from bpf prog).
>  	 */
> -	if (!onallcpus) {
> -		int current_cpu = raw_smp_processor_id();
> +	if (!onallcpus || (map_flags & BPF_F_CPU)) {
> +		int init_cpu = onallcpus ? map_flags >> 32 : raw_smp_processor_id();

The comment above covers only the bpf prog case now.
Pls update it, but one short sentence about BPF_F_CPU is enough.
What you proposed in reply to the bot is too verbose.

pw-bot: cr

[PATCH bpf 2/2] selftests/bpf: Test per-cpu initialization of a BPF_F_CPU created element

[Donggeun Yoo]

The existing cpu_flag subtests always prime a key with BPF_F_ALL_CPUS
before any BPF_F_CPU write, so the element always exists by the time the
flag is used and the create path is never covered.

Add a subtest that creates the element with BPF_F_CPU on a map whose
max_entries is 1, so the key can only reuse the element the previous key
released, and check that the CPUs the update did not name read back zero
rather than the previous key's value.  Run it for PERCPU_HASH both
preallocated and BPF_F_NO_PREALLOC, whose per-cpu areas are recycled by
different allocators, and for LRU_PERCPU_HASH, which is always
preallocated.

Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
---
 .../selftests/bpf/prog_tests/percpu_alloc.c   | 74 +++++++++++++++++++
 1 file changed, 74 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/percpu_alloc.c b/tools/testing/selftests/bpf/prog_tests/percpu_alloc.c
index a72ae0b29f6e9..d0084a2405bed 100644
--- a/tools/testing/selftests/bpf/prog_tests/percpu_alloc.c
+++ b/tools/testing/selftests/bpf/prog_tests/percpu_alloc.c
@@ -350,6 +350,74 @@ static void test_lru_percpu_hash_cpu_flag(void)
 	test_percpu_map_cpu_flag(BPF_MAP_TYPE_LRU_PERCPU_HASH);
 }
 
+/* A BPF_F_CPU update that creates an element must zero the value on the other
+ * cpus, rather than leave them holding whatever the recycled element last
+ * contained.  max_entries is 1 so the second key can only reuse the element
+ * the first one released.
+ */
+static void test_percpu_map_cpu_flag_create(enum bpf_map_type map_type, __u32 map_flags)
+{
+	LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = map_flags);
+	const u32 stale = 0xDEADC0DE, fresh = 0xC0FFEE;
+	int nr_cpus, cpu, map_fd, err, key;
+	u32 value;
+	u64 flags;
+
+	nr_cpus = libbpf_num_possible_cpus();
+	if (!ASSERT_GT(nr_cpus, 1, "libbpf_num_possible_cpus"))
+		return;
+
+	map_fd = bpf_map_create(map_type, "cpu_flag_create", sizeof(key), sizeof(value), 1, &opts);
+	if (!ASSERT_GE(map_fd, 0, "bpf_map_create"))
+		return;
+
+	key = 1;
+	value = stale;
+	err = bpf_map_update_elem(map_fd, &key, &value, BPF_F_ALL_CPUS);
+	if (!ASSERT_OK(err, "bpf_map_update_elem all_cpus"))
+		goto out;
+
+	err = bpf_map_delete_elem(map_fd, &key);
+	if (!ASSERT_OK(err, "bpf_map_delete_elem"))
+		goto out;
+
+	key = 2;
+	value = fresh;
+	flags = BPF_F_CPU;
+	err = bpf_map_update_elem(map_fd, &key, &value, flags);
+	if (!ASSERT_OK(err, "bpf_map_update_elem specified cpu"))
+		goto out;
+
+	for (cpu = 0; cpu < nr_cpus; cpu++) {
+		value = 0;
+		flags = (u64)cpu << 32 | BPF_F_CPU;
+		err = bpf_map_lookup_elem_flags(map_fd, &key, &value, flags);
+		if (!ASSERT_OK(err, "bpf_map_lookup_elem_flags specified cpu"))
+			goto out;
+		if (!ASSERT_EQ(value, cpu ? 0 : fresh, "value on specified cpu"))
+			goto out;
+	}
+
+out:
+	close(map_fd);
+}
+
+static void test_percpu_hash_cpu_flag_create(void)
+{
+	test_percpu_map_cpu_flag_create(BPF_MAP_TYPE_PERCPU_HASH, 0);
+}
+
+static void test_percpu_hash_cpu_flag_create_malloc(void)
+{
+	test_percpu_map_cpu_flag_create(BPF_MAP_TYPE_PERCPU_HASH, BPF_F_NO_PREALLOC);
+}
+
+static void test_lru_percpu_hash_cpu_flag_create(void)
+{
+	/* lru without prealloc is -ENOTSUPP, so there is no malloc variant. */
+	test_percpu_map_cpu_flag_create(BPF_MAP_TYPE_LRU_PERCPU_HASH, 0);
+}
+
 static void test_percpu_cgroup_storage_cpu_flag(void)
 {
 	struct percpu_alloc_array *skel = NULL;
@@ -454,6 +522,12 @@ void test_percpu_alloc(void)
 		test_percpu_hash_cpu_flag();
 	if (test__start_subtest("cpu_flag_lru_percpu_hash"))
 		test_lru_percpu_hash_cpu_flag();
+	if (test__start_subtest("cpu_flag_create_percpu_hash"))
+		test_percpu_hash_cpu_flag_create();
+	if (test__start_subtest("cpu_flag_create_percpu_hash_malloc"))
+		test_percpu_hash_cpu_flag_create_malloc();
+	if (test__start_subtest("cpu_flag_create_lru_percpu_hash"))
+		test_lru_percpu_hash_cpu_flag_create();
 	if (test__start_subtest("cpu_flag_percpu_cgroup_storage"))
 		test_percpu_cgroup_storage_cpu_flag();
 	if (test__start_subtest("cpu_flag_array"))
-- 
2.53.0

Re: [PATCH bpf 2/2] selftests/bpf: Test per-cpu initialization of a BPF_F_CPU created element

[Donggeun Yoo]

On Sun, 20 Sep 2026 09:58:58 +0000, sashiko-bot@kernel.org wrote:
> [Severity: Medium]
> Does this force a hard failure on uniprocessor systems instead of correctly
> skipping the test?
>
> Since UP is a valid hardware configuration for running selftests, if a test
> requires multiple CPUs, shouldn't it gracefully skip using test__skip()
> rather than asserting a failure?

Right, that is a bug in the test. With one possible CPU there is no second
CPU that could hold a stale value, so the case this subtest exists for
cannot occur; booted with -smp 1, all three of the new subtests fail rather
than skip. I will use test__skip() in v2.

> [Severity: Low]
> This isn't a bug, but the BPF subsystem strictly requires multi-line comments
> to have the opening /* on its own line. Could this be reformatted to match
> the required style?

It is not a requirement, but it is the documented preference, so I will
follow it. Commit 82b8000c28b5 ("net: drop special comment style") removed
the netdev comment rule, which leaves the general preference in
Documentation/process/coding-style.rst, and that is the form you describe.
checkpatch checks neither form. I will reformat the comment in v2.

Thanks,
Donggeun