* [PATCH bpf-next 0/2] bpftool: Batch bounded hash map dumps
@ 2026-09-06 17:08 Tianyi Chen
2026-09-06 17:08 ` [PATCH bpf-next 1/2] bpftool: Use batch lookups for " Tianyi Chen
2026-09-06 17:08 ` [PATCH bpf-next 2/2] selftests/bpf: Check bpftool batch map dump contents Tianyi Chen
0 siblings, 2 replies; 5+ messages in thread
From: Tianyi Chen @ 2026-09-06 17:08 UTC (permalink / raw)
To: bpf
Cc: Tianyi Chen, Quentin Monnet, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Martin KaFai Lau, Song Liu, Yonghong Song, Jiri Olsa,
Emil Tsalapatis, Ihor Solodrai, Shuah Khan, linux-kernel,
linux-kselftest
This series uses lookup batches for ordinary hash maps whose maximum
key/value storage fits within a 4 MiB budget. It retains the existing
formatting and falls back to individual lookups if the initial batch
operation is unsupported. Other map types keep individual lookups.
Hash batch lookups need room for a complete bucket, so ENOSPC grows the
buffer without advancing the cursor. The memory eligibility check makes
that growth bounded even after output has begun. Non-ENOENT errors do
not expose untrusted counts or trigger a duplicate dump restart.
Related request:
https://github.com/libbpf/bpftool/issues/63
The series is based on bpf-next and does not depend on my recursive
map-dump series or the flags/ring-buffer series sent alongside it.
Validation on x86-64, Linux 7.3.0-rc1 in KVM, LLVM 20:
- Full bpftool build and focused BPF selftests build.
- All 11 bpftool_map_batch subtests passed against a bpftool built with
only this series: complete unordered plain, JSON and pretty JSON
contents, batch boundaries, short keys, odd-sized values and BTF.
- strace fault injection checked initial EINVAL/EOPNOTSUPP/ENOTSUPP
fallback; ENOSPC growth initially and after progress; fatal EFAULT,
ENOMEM and post-progress EIO/EINVAL; no duplicates and valid JSON.
- A map exceeding the memory eligibility budget used individual lookups.
- On the same static 100,000-entry hash map, BPF syscall counts dropped
from 200,004 to 395. Five untraced runs had median elapsed times of
0.774 s before and 0.734 s after; text formatting still dominates.
Fault injection simulates error returns; it is not a real collision
stress test. Only focused selftests were run. Strict checkpatch has no
errors or checks; the new test file is covered by existing MAINTAINERS
patterns.
Integration check: these three independent series also applied and built
together with my previously posted recursive map-dump v2 series. The
combined 64 subtests passed with no skips or failures in the same guest.
Tianyi Chen (2):
bpftool: Use batch lookups for bounded hash map dumps
selftests/bpf: Check bpftool batch map dump contents
tools/bpf/bpftool/map.c | 113 ++++++++++-
.../bpf/prog_tests/bpftool_map_batch.c | 186 ++++++++++++++++++
2 files changed, 291 insertions(+), 8 deletions(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
--
2.55.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH bpf-next 1/2] bpftool: Use batch lookups for bounded hash map dumps
2026-09-06 17:08 [PATCH bpf-next 0/2] bpftool: Batch bounded hash map dumps Tianyi Chen
@ 2026-09-06 17:08 ` Tianyi Chen
2026-09-06 18:17 ` bot+bpf-ci
2026-09-06 17:08 ` [PATCH bpf-next 2/2] selftests/bpf: Check bpftool batch map dump contents Tianyi Chen
1 sibling, 1 reply; 5+ messages in thread
From: Tianyi Chen @ 2026-09-06 17:08 UTC (permalink / raw)
To: bpf
Cc: Tianyi Chen, Quentin Monnet, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Martin KaFai Lau, Song Liu, Yonghong Song, Jiri Olsa,
Emil Tsalapatis, Ihor Solodrai, Shuah Khan, linux-kernel,
linux-kselftest
Use BPF_MAP_LOOKUP_BATCH when dumping hash maps to reduce the number
of BPF syscalls. Share element formatting with individual lookups to
preserve plain, JSON and BTF output.
Start with up to 256 entries and grow on ENOSPC without advancing the
input cursor. Restrict the optimization to maps whose maximum key and
value storage fits in 4 MiB, so even a worst-case bucket can fit without
restarting a partially printed dump. Preserve aligned element buffers
for formatting keys and values with odd sizes.
Fall back to individual lookups only if the initial batch operation is
unsupported. Process the final partial batch on ENOENT, but never use
count or output buffers after other errors. Report errors after batch
traversal starts without restarting and duplicating output.
Assisted-by: Codex:GPT-6
Signed-off-by: Tianyi Chen <hi@tychen.cc>
---
tools/bpf/bpftool/map.c | 113 +++++++++++++++++++++++++++++++++++++---
1 file changed, 105 insertions(+), 8 deletions(-)
diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index 684a8fb7241..1eccdc0d1e9 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -740,15 +740,10 @@ static int do_show(int argc, char **argv)
return errno == ENOENT ? 0 : -1;
}
-static int dump_map_elem(int fd, void *key, void *value,
- struct bpf_map_info *map_info, struct btf *btf,
- json_writer_t *btf_wtr)
+static void print_map_elem(void *key, void *value,
+ struct bpf_map_info *map_info, struct btf *btf,
+ json_writer_t *btf_wtr)
{
- if (bpf_map_lookup_elem(fd, key, value)) {
- print_entry_error(map_info, key, errno);
- return -1;
- }
-
if (json_output) {
print_entry_json(map_info, key, value, btf);
} else if (btf) {
@@ -762,10 +757,108 @@ static int dump_map_elem(int fd, void *key, void *value,
} else {
print_entry_plain(map_info, key, value);
}
+}
+
+static int dump_map_elem(int fd, void *key, void *value,
+ struct bpf_map_info *map_info, struct btf *btf,
+ json_writer_t *btf_wtr)
+{
+ if (bpf_map_lookup_elem(fd, key, value)) {
+ print_entry_error(map_info, key, errno);
+ return -1;
+ }
+ print_map_elem(key, value, map_info, btf, btf_wtr);
return 0;
}
+#define MAP_DUMP_BATCH_SIZE 256U
+#define MAP_DUMP_BATCH_MAX_BYTES (4 * 1024 * 1024)
+
+/* Return 1 to use individual lookups, but only before batch traversal starts. */
+static int dump_map_batch(int fd, void *key, void *value,
+ struct bpf_map_info *info, struct btf *btf,
+ json_writer_t *wtr, unsigned int *num_elems)
+{
+ __u32 capacity, count, batch = 0, next_batch = 0, i;
+ void *keys = NULL, *values = NULL, *buf;
+ bool first = true, can_fallback = true;
+ int err;
+
+ /* Hash lookup batches must accommodate a whole bucket. Restrict the
+ * optimization to maps whose worst-case bucket fits the memory budget,
+ * so a later ENOSPC never forces a restart after printing some entries.
+ * Division also bounds the allocation multiplications on 32-bit hosts.
+ */
+ if (info->type != BPF_MAP_TYPE_HASH || !info->max_entries ||
+ (__u64)info->key_size + info->value_size >
+ MAP_DUMP_BATCH_MAX_BYTES / info->max_entries)
+ return 1;
+
+ capacity = min(info->max_entries, MAP_DUMP_BATCH_SIZE);
+resize:
+ buf = realloc(keys, (size_t)capacity * info->key_size);
+ if (!buf) {
+ err = ENOMEM;
+ goto error;
+ }
+ keys = buf;
+ buf = realloc(values, (size_t)capacity * info->value_size);
+ if (!buf) {
+ err = ENOMEM;
+ goto error;
+ }
+ values = buf;
+
+ while (true) {
+ count = capacity;
+ err = bpf_map_lookup_batch(fd, first ? NULL : &batch,
+ &next_batch, keys, values, &count, NULL);
+ err = err ? errno : 0;
+ /* Older kernels reject the command before updating count. Do not
+ * inspect the buffers on these errors, or fall back after progress.
+ */
+ if (can_fallback && (err == EINVAL || err == EOPNOTSUPP ||
+ err == 524 /* ENOTSUPP */)) {
+ err = 1;
+ goto out;
+ }
+ can_fallback = false;
+ if (err == ENOSPC) {
+ if (capacity == info->max_entries)
+ goto error;
+ capacity += min(capacity, info->max_entries - capacity);
+ /* Preserve the input cursor: the oversized bucket was not read. */
+ goto resize;
+ }
+ /* In particular, EFAULT can leave count and the buffers invalid. */
+ if (err && err != ENOENT)
+ goto error;
+ for (i = 0; i < count; i++) {
+ /* Keep the alignment provided by individual lookups, including
+ * for BTF types whose map key/value size is not aligned.
+ */
+ memcpy(key, keys + (size_t)i * info->key_size, info->key_size);
+ memcpy(value, values + (size_t)i * info->value_size, info->value_size);
+ print_map_elem(key, value, info, btf, wtr);
+ (*num_elems)++;
+ }
+ if (err == ENOENT) {
+ err = 0;
+ goto out;
+ }
+ first = false;
+ batch = next_batch;
+ }
+error:
+ p_err("can't lookup map batch: %s", strerror(err));
+ err = -1;
+out:
+ free(keys);
+ free(values);
+ return err;
+}
+
static int maps_have_btf(int *fds, int nb_fds)
{
struct bpf_map_info info = {};
@@ -869,6 +962,9 @@ map_dump(int fd, struct bpf_map_info *info, json_writer_t *wtr,
p_info("Warning: cannot read values from %s map with value_size != 8",
map_type_str);
}
+ err = dump_map_batch(fd, key, value, info, btf, wtr, &num_elems);
+ if (err != 1)
+ goto end_dump;
while (true) {
err = bpf_map_get_next_key(fd, prev_key, key);
if (err) {
@@ -881,6 +977,7 @@ map_dump(int fd, struct bpf_map_info *info, json_writer_t *wtr,
prev_key = key;
}
+end_dump:
if (wtr) {
jsonw_end_array(wtr); /* elements */
if (show_header)
--
2.55.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH bpf-next 2/2] selftests/bpf: Check bpftool batch map dump contents
2026-09-06 17:08 [PATCH bpf-next 0/2] bpftool: Batch bounded hash map dumps Tianyi Chen
2026-09-06 17:08 ` [PATCH bpf-next 1/2] bpftool: Use batch lookups for " Tianyi Chen
@ 2026-09-06 17:08 ` Tianyi Chen
2026-09-06 18:01 ` bot+bpf-ci
1 sibling, 1 reply; 5+ messages in thread
From: Tianyi Chen @ 2026-09-06 17:08 UTC (permalink / raw)
To: bpf
Cc: Tianyi Chen, Quentin Monnet, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Eduard Zingerman, Kumar Kartikeya Dwivedi,
Martin KaFai Lau, Song Liu, Yonghong Song, Jiri Olsa,
Emil Tsalapatis, Ihor Solodrai, Shuah Khan, linux-kernel,
linux-kselftest
Exercise hash map dumps around the initial batch size and across
multiple batches, including empty and single-entry maps. Compare each
complete unordered key/value set with the input data in plain, JSON
and pretty JSON output.
Cover one-byte keys, three-byte values and BTF-formatted maps to catch
cursor sizing, buffer alignment and formatting regressions.
Assisted-by: Codex:GPT-6
Signed-off-by: Tianyi Chen <hi@tychen.cc>
---
.../bpf/prog_tests/bpftool_map_batch.c | 186 ++++++++++++++++++
1 file changed, 186 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
diff --git a/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
new file mode 100644
index 00000000000..139d13a49dc
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
@@ -0,0 +1,186 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+#include <bpftool_helpers.h>
+#include <bpf/btf.h>
+#include <ctype.h>
+
+#define MAX_ENTRIES 1025
+#define RECORD_SIZE 256
+#define OUTPUT_SIZE (MAX_ENTRIES * RECORD_SIZE + 1024)
+
+struct dump_case {
+ const char *name;
+ unsigned int count;
+ unsigned int key_size;
+ unsigned int value_size;
+ bool btf;
+};
+
+static void hex_bytes(char *out, const void *data, unsigned int size, bool json)
+{
+ const unsigned char *bytes = data;
+ unsigned int i;
+
+ if (json)
+ *out++ = '[';
+ for (i = 0; i < size; i++) {
+ if (json && i)
+ *out++ = ',';
+ out += sprintf(out, json ? "\"0x%02x\"" : "%02x", bytes[i]);
+ }
+ if (json)
+ *out++ = ']';
+ *out = '\0';
+}
+
+static void expected_record(char *record, const struct dump_case *test,
+ unsigned int index, bool json)
+{
+ __u32 key = index, value = index * 37 + 11;
+ unsigned char short_key = index;
+ char key_hex[64], value_hex[64], formatted[96];
+
+ hex_bytes(key_hex, test->key_size == 1 ? (void *)&short_key : &key,
+ test->key_size, json);
+ hex_bytes(value_hex, &value, test->value_size, json);
+ snprintf(formatted, sizeof(formatted), "{\"key\":%u,\"value\":%u}",
+ key, value);
+ if (json && test->btf)
+ snprintf(record, RECORD_SIZE,
+ "{\"key\":%s,\"value\":%s,\"formatted\":%s}",
+ key_hex, value_hex, formatted);
+ else if (json)
+ snprintf(record, RECORD_SIZE, "{\"key\":%s,\"value\":%s}",
+ key_hex, value_hex);
+ else if (test->btf)
+ snprintf(record, RECORD_SIZE, "%s", formatted);
+ else
+ snprintf(record, RECORD_SIZE, "key:%svalue:%s", key_hex, value_hex);
+}
+
+static void check_dump(const struct dump_case *test, __u32 id, bool json, bool pretty)
+{
+ bool array = json || test->btf;
+ char command[MAX_BPFTOOL_CMD_LEN], expected[RECORD_SIZE], footer[64];
+ bool seen[MAX_ENTRIES] = {};
+ char *output, *src, *dst, *cursor;
+ unsigned int i, n;
+ int err;
+
+ output = calloc(1, OUTPUT_SIZE);
+ if (!ASSERT_OK_PTR(output, "alloc_output"))
+ return;
+ snprintf(command, sizeof(command), "%smap dump id %u",
+ pretty ? "-p " : json ? "-j " : "", id);
+ err = get_bpftool_command_output(command, output, OUTPUT_SIZE);
+ if (!ASSERT_OK(err, "map_dump"))
+ goto out;
+ /* Ignore presentation whitespace, but compare complete records and all
+ * punctuation. Expected contents come only from the input data, never
+ * from another map walk or bpftool invocation.
+ */
+ for (src = output, dst = output; *src; src++)
+ if (!isspace((unsigned char)*src))
+ *dst++ = *src;
+ *dst = '\0';
+ cursor = output;
+ if (array) {
+ if (!ASSERT_EQ(*cursor, '[', "array_start"))
+ goto out;
+ cursor++;
+ }
+ for (n = 0; n < test->count; n++) {
+ if (array && n) {
+ if (!ASSERT_EQ(*cursor, ',', "record_separator"))
+ goto out;
+ cursor++;
+ }
+ for (i = 0; i < test->count; i++) {
+ if (seen[i])
+ continue;
+ expected_record(expected, test, i, json);
+ if (!strncmp(cursor, expected, strlen(expected)))
+ break;
+ }
+ if (!ASSERT_LT(i, test->count, "unique_expected_record"))
+ goto out;
+ seen[i] = true;
+ cursor += strlen(expected);
+ }
+ if (array) {
+ ASSERT_STREQ(cursor, "]", "array_end_and_count");
+ } else {
+ snprintf(footer, sizeof(footer), "Found%uelement%s", test->count,
+ test->count == 1 ? "" : "s");
+ ASSERT_STREQ(cursor, footer, "plain_count");
+ }
+out:
+ free(output);
+}
+
+static void run_dump_case(const struct dump_case *test)
+{
+ LIBBPF_OPTS(bpf_map_create_opts, opts);
+ struct bpf_map_info info = {};
+ __u32 info_len = sizeof(info);
+ struct btf *btf = NULL;
+ unsigned int i;
+ int fd = -1;
+
+ if (test->btf) {
+ btf = btf__new_empty();
+ if (!ASSERT_OK_PTR(btf, "btf_new"))
+ return;
+ if (!ASSERT_EQ(btf__add_int(btf, "unsigned int", 4, 0), 1,
+ "btf_int") ||
+ !ASSERT_OK(btf__load_into_kernel(btf), "btf_load"))
+ goto out;
+ opts.btf_fd = btf__fd(btf);
+ opts.btf_key_type_id = 1;
+ opts.btf_value_type_id = 1;
+ }
+ fd = bpf_map_create(BPF_MAP_TYPE_HASH, "dump_batch", test->key_size,
+ test->value_size, test->count ?: 1, &opts);
+ if (!ASSERT_OK_FD(fd, "map_create"))
+ goto out;
+ for (i = 0; i < test->count; i++) {
+ __u32 key = i, value = i * 37 + 11;
+ unsigned char short_key = i;
+ void *key_ptr = test->key_size == 1 ? (void *)&short_key : &key;
+
+ if (!ASSERT_OK(bpf_map_update_elem(fd, key_ptr, &value, BPF_ANY),
+ "map_update"))
+ goto out;
+ }
+ if (!ASSERT_OK(bpf_map_get_info_by_fd(fd, &info, &info_len), "map_info"))
+ goto out;
+ check_dump(test, info.id, false, false);
+ check_dump(test, info.id, true, false);
+ check_dump(test, info.id, true, true);
+out:
+ if (fd >= 0)
+ close(fd);
+ btf__free(btf);
+}
+
+void test_bpftool_map_batch(void)
+{
+ static const struct dump_case cases[] = {
+ { "empty", 0, 4, 4 },
+ { "single", 1, 4, 4 },
+ { "below_batch", 255, 4, 4 },
+ { "exact_batch", 256, 4, 4 },
+ { "above_batch", 257, 4, 4 },
+ { "multiple_batches", 1025, 4, 4 },
+ { "one_byte_key", 256, 1, 4 },
+ { "odd_value_size", 257, 4, 3 },
+ { "btf_empty", 0, 4, 4, true },
+ { "btf_single", 1, 4, 4, true },
+ { "btf_multiple_batches", 1025, 4, 4, true },
+ };
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(cases); i++)
+ if (test__start_subtest(cases[i].name))
+ run_dump_case(&cases[i]);
+}
--
2.55.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH bpf-next 2/2] selftests/bpf: Check bpftool batch map dump contents
2026-09-06 17:08 ` [PATCH bpf-next 2/2] selftests/bpf: Check bpftool batch map dump contents Tianyi Chen
@ 2026-09-06 18:01 ` bot+bpf-ci
0 siblings, 0 replies; 5+ messages in thread
From: bot+bpf-ci @ 2026-09-06 18:01 UTC (permalink / raw)
To: hi, bpf
Cc: hi, qmo, ast, daniel, andrii, eddyz87, memxor, martin.lau, song,
yonghong.song, jolsa, emil, ihor.solodrai, shuah, linux-kernel,
linux-kselftest, ast, andrii, daniel, martin.lau, eddyz87,
yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 5292 bytes --]
> selftests/bpf: Check bpftool batch map dump contents
>
> Exercise hash map dumps around the initial batch size and across
> multiple batches, including empty and single-entry maps. Compare each
> complete unordered key/value set with the input data in plain, JSON
> and pretty JSON output.
>
> Cover one-byte keys, three-byte values and BTF-formatted maps to catch
> cursor sizing, buffer alignment and formatting regressions.
The changelog, the file name and the subtest names all describe a batched
map-dump implementation that does not exist in the code under test.
bpftool map dump is implemented by map_dump() in tools/bpf/bpftool/map.c,
which walks the map one element at a time:
while (true) {
err = bpf_map_get_next_key(fd, prev_key, key);
if (err) { ... break; }
if (!dump_map_elem(fd, key, value, info, btf, wtr))
num_elems++;
prev_key = key;
}
and dump_map_elem() calls bpf_map_lookup_elem() per key.
grep -rn 'batch\|BPF_MAP_LOOKUP_BATCH\|bpf_map_lookup_batch'
tools/bpf/bpftool/ returns only the unrelated bpftool batch file FILE
subcommand in main.c. There is no bpf_map_lookup_batch() call, no batch
cursor (in_batch/out_batch), no per-batch buffer and no batch-size
constant anywhere in bpftool.
What does this mean for the test?
- "around the initial batch size", "across multiple batches" and "cursor
sizing, buffer alignment" describe nothing in the tested code. The
counts 255/256/257/1025 and the subtest names below_batch, exact_batch,
above_batch, multiple_batches are arbitrary; 256 is not a boundary of
any code path.
- The file name bpftool_map_batch.c and the test name bpftool_map_batch
collide with bpftool's real, unrelated batch subcommand (bpftool batch
file FILE, main.c:334 do_batch()), so a reader looking for coverage of
that subcommand will find a map-dump test instead.
- The series range 407ec415f849..1dcddb9351e8 contains no commit
converting map dump to the batch API (the only other change is
map_perf_ring.c ring-buffer support in c23648f76644), so this is not
infrastructure landing ahead of its user.
What the test actually does is valid and useful - it verifies the exact
plain/JSON/pretty-JSON rendering of map dump for various key/value sizes
and element counts. Should the changelog and naming describe that (e.g.
"check map dump output contents") instead of claiming batch-boundary
coverage, so a future maintainer doesn't believe the batch paths are
covered when they are not?
> diff --git a/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
> new file mode 100644
> index 000000000000..139d13a49dce
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
[ ... ]
> @@ -0,0 +1,186 @@
[ ... ]
> +void test_bpftool_map_batch(void)
> +{
> + static const struct dump_case cases[] = {
> + { "empty", 0, 4, 4 },
> + { "single", 1, 4, 4 },
> + { "below_batch", 255, 4, 4 },
> + { "exact_batch", 256, 4, 4 },
> + { "above_batch", 257, 4, 4 },
> + { "multiple_batches", 1025, 4, 4 },
> + { "one_byte_key", 256, 1, 4 },
> + { "odd_value_size", 257, 4, 3 },
> + { "btf_empty", 0, 4, 4, true },
> + { "btf_single", 1, 4, 4, true },
> + { "btf_multiple_batches", 1025, 4, 4, true },
> + };
A subsystem pattern flags this as potentially concerning: the new test
file is structured around an internal batching strategy rather than the
interface it actually checks. Four of the eleven cases exist only to
straddle a supposed 256-element batch boundary (below_batch, exact_batch,
above_batch, multiple_batches), and the changelog names "cursor sizing"
and "buffer alignment" as the regressions being guarded.
map_dump() in tools/bpf/bpftool/map.c uses bpf_map_get_next_key() +
bpf_map_lookup_elem() per element, so no such boundary exists and these
cases are indistinguishable from any other element count. The
interface-level part of the test (exact plain/JSON/pretty-JSON record
text, element count footer, BTF formatted field) is sound and is what the
case table should be organised around.
The guide directive notes: a new selftest file added for a narrow variant
(a specific size, a specific internal code path) of behavior an existing
test in the same directory already covers more generally should be
reported as a concern - asking whether it should be a new case in the
existing test instead. Tests written against specific implementation
details (an internal batching strategy, a particular code path taken to
reach a result) break or need a rewrite whenever that implementation
changes.
That said, the assertions themselves are written against observable
bpftool output, not against internal state, so the test will not break if
bpftool's enumeration strategy changes - only the case names and the
changelog would become misleading. Adding extra element counts is also
harmless coverage.
Does the case table organization based on a non-existent batch boundary
make the test harder to maintain or extend?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34048930508
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH bpf-next 1/2] bpftool: Use batch lookups for bounded hash map dumps
2026-09-06 17:08 ` [PATCH bpf-next 1/2] bpftool: Use batch lookups for " Tianyi Chen
@ 2026-09-06 18:17 ` bot+bpf-ci
0 siblings, 0 replies; 5+ messages in thread
From: bot+bpf-ci @ 2026-09-06 18:17 UTC (permalink / raw)
To: hi, bpf
Cc: hi, qmo, ast, daniel, andrii, eddyz87, memxor, martin.lau, song,
yonghong.song, jolsa, emil, ihor.solodrai, shuah, linux-kernel,
linux-kselftest, ast, andrii, daniel, martin.lau, eddyz87,
yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 5757 bytes --]
> diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
> index 684a8fb72414..1eccdc0d1e96 100644
> --- a/tools/bpf/bpftool/map.c
> +++ b/tools/bpf/bpftool/map.c
[ ... ]
> @@ -762,10 +757,108 @@ static int dump_map_elem(int fd, void *key, void *value,
> } else {
> print_entry_plain(map_info, key, value);
> }
> +}
> +
> +static int dump_map_elem(int fd, void *key, void *value,
> + struct bpf_map_info *map_info, struct btf *btf,
> + json_writer_t *btf_wtr)
> +{
> + if (bpf_map_lookup_elem(fd, key, value)) {
> + print_entry_error(map_info, key, errno);
> + return -1;
> + }
>
> + print_map_elem(key, value, map_info, btf, btf_wtr);
> return 0;
> }
>
> +#define MAP_DUMP_BATCH_SIZE 256U
> +#define MAP_DUMP_BATCH_MAX_BYTES (4 * 1024 * 1024)
> +
> +/* Return 1 to use individual lookups, but only before batch traversal starts. */
> +static int dump_map_batch(int fd, void *key, void *value,
> + struct bpf_map_info *info, struct btf *btf,
> + json_writer_t *wtr, unsigned int *num_elems)
> +{
> + __u32 capacity, count, batch = 0, next_batch = 0, i;
> + void *keys = NULL, *values = NULL, *buf;
> + bool first = true, can_fallback = true;
> + int err;
> +
> + /* Hash lookup batches must accommodate a whole bucket. Restrict the
> + * optimization to maps whose worst-case bucket fits the memory budget,
> + * so a later ENOSPC never forces a restart after printing some entries.
> + * Division also bounds the allocation multiplications on 32-bit hosts.
> + */
> + if (info->type != BPF_MAP_TYPE_HASH || !info->max_entries ||
> + (__u64)info->key_size + info->value_size >
> + MAP_DUMP_BATCH_MAX_BYTES / info->max_entries)
> + return 1;
> +
> + capacity = min(info->max_entries, MAP_DUMP_BATCH_SIZE);
> +resize:
> + buf = realloc(keys, (size_t)capacity * info->key_size);
> + if (!buf) {
> + err = ENOMEM;
> + goto error;
> + }
> + keys = buf;
> + buf = realloc(values, (size_t)capacity * info->value_size);
> + if (!buf) {
> + err = ENOMEM;
> + goto error;
> + }
> + values = buf;
> +
> + while (true) {
> + count = capacity;
> + err = bpf_map_lookup_batch(fd, first ? NULL : &batch,
> + &next_batch, keys, values, &count, NULL);
> + err = err ? errno : 0;
> + /* Older kernels reject the command before updating count. Do not
> + * inspect the buffers on these errors, or fall back after progress.
> + */
> + if (can_fallback && (err == EINVAL || err == EOPNOTSUPP ||
> + err == 524 /* ENOTSUPP */)) {
> + err = 1;
> + goto out;
> + }
> + can_fallback = false;
> + if (err == ENOSPC) {
> + if (capacity == info->max_entries)
> + goto error;
> + capacity += min(capacity, info->max_entries - capacity);
> + /* Preserve the input cursor: the oversized bucket was not read. */
> + goto resize;
> + }
> + /* In particular, EFAULT can leave count and the buffers invalid. */
> + if (err && err != ENOENT)
> + goto error;
> + for (i = 0; i < count; i++) {
> + /* Keep the alignment provided by individual lookups, including
> + * for BTF types whose map key/value size is not aligned.
> + */
> + memcpy(key, keys + (size_t)i * info->key_size, info->key_size);
> + memcpy(value, values + (size_t)i * info->value_size, info->value_size);
> + print_map_elem(key, value, info, btf, wtr);
> + (*num_elems)++;
> + }
> + if (err == ENOENT) {
> + err = 0;
> + goto out;
> + }
> + first = false;
> + batch = next_batch;
> + }
> +error:
> + p_err("can't lookup map batch: %s", strerror(err));
> + err = -1;
Could this p_err() call produce malformed JSON output when json_output is
set?
Looking at the call chain from do_dump():
do_dump()
assigns wtr = json_wtr when json_output is set
calls map_dump()
opens the elements array with jsonw_start_array(wtr)
calls dump_map_batch()
can reach p_err() here while the array is still open
closes the array at end_dump: with jsonw_end_array(wtr)
When p_err() runs with json_output set, it emits:
tools/bpf/bpftool/common.c:p_err() {
if (json_output) {
jsonw_start_object(json_wtr);
jsonw_name(json_wtr, "error");
jsonw_vprintf_enquote(json_wtr, fmt, ap);
jsonw_end_object(json_wtr);
}
}
So the error object would be inserted as a member of the elements array,
giving consumers an object with no .key or .value fields mixed in with
normal element objects.
This differs from the existing in-array error handling. The pre-existing
print_entry_error() deliberately emits a well-formed element:
{"key": [...], "value": {"error": "..."}}
And p_info() returns early when json_output is set, which is why the
existing REUSEPORT_SOCKARRAY warning inside the array is safe.
Would it be better to emit the error after jsonw_end_array() to keep the
array schema homogeneous, or to use the print_entry_error() format with
a key?
> +out:
> + free(keys);
> + free(values);
> + return err;
> +}
[ ... ]
> @@ -869,6 +962,9 @@ map_dump(int fd, struct bpf_map_info *info, json_writer_t *wtr,
> p_info("Warning: cannot read values from %s map with value_size != 8",
> map_type_str);
> }
> + err = dump_map_batch(fd, key, value, info, btf, wtr, &num_elems);
> + if (err != 1)
> + goto end_dump;
> while (true) {
> err = bpf_map_get_next_key(fd, prev_key, key);
> if (err) {
> @@ -881,6 +977,7 @@ map_dump(int fd, struct bpf_map_info *info, json_writer_t *wtr,
> prev_key = key;
> }
>
> +end_dump:
> if (wtr) {
> jsonw_end_array(wtr); /* elements */
> if (show_header)
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34048943300
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-09-06 18:17 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-09-06 17:08 [PATCH bpf-next 0/2] bpftool: Batch bounded hash map dumps Tianyi Chen
2026-09-06 17:08 ` [PATCH bpf-next 1/2] bpftool: Use batch lookups for " Tianyi Chen
2026-09-06 18:17 ` bot+bpf-ci
2026-09-06 17:08 ` [PATCH bpf-next 2/2] selftests/bpf: Check bpftool batch map dump contents Tianyi Chen
2026-09-06 18:01 ` bot+bpf-ci
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®