* [PATCHv2 0/2] kprobes: Reduce registration lookup costs
@ 2026-11-22 20:02 Jiri Olsa
2026-11-22 20:02 ` [PATCHv2 1/2] kprobes: Increase the kprobe hash table size Jiri Olsa
2026-11-22 20:02 ` [PATCHv2 2/2] x86/kprobes: Use ftrace_location_range in __recover_probed_insn Jiri Olsa
0 siblings, 2 replies; 3+ messages in thread
From: Jiri Olsa @ 2026-11-22 20:02 UTC (permalink / raw)
To: Masami Hiramatsu, Naveen N Rao, David S. Miller
Cc: Peter Zijlstra, Steven Rostedt, Ingo Molnar, x86,
linux-trace-kernel, linux-kernel, bpf
hi,
this is a smaller follow up to the v1 series [1]. This version does
only partial fixes of what v1 addressed. Instead of the rbtree and
iterators, it makes the existing per-address lookups cheaper:
- increase the kprobe hash table from 64 to 4096 buckets, so the
get_kprobe() calls done for every byte or decoded instruction
around the new probe walk short chains instead of 300+ entries
per bucket with 20000 probes
- use ftrace_location_range() in x86 __recover_probed_insn(), so we
skip the symbol lookup ftrace_location() does for every decoded
instruction that is not an ftrace location
Registering kprobes with the kprobe_many_example sample module [2] from
v1 in a KVM guest goes from 4.71s to 1.30s for 20000 probes and from
29.99s to 3.64s for 50000 probes with the hash table change, and the
ftrace change brings 20587 probes from 4.02s to 3.70s. Please check the
changelogs for more details, the first patch has numbers for different
hash table sizes.
v2 changes:
- dropped the static_call/jump_label indexes and the kprobe rbtree
and iterators [will be addressed later]
- added hash table size increase and the ftrace_location_range
change for __recover_probed_insn
thanks,
jirka
[1] https://lore.kernel.org/bpf/20261109194916.1089703-1-jolsa@kernel.org/
[2] https://lore.kernel.org/bpf/20261109195827.1091410-1-jolsa@kernel.org/
---
Jiri Olsa (2):
kprobes: Increase the kprobe hash table size
x86/kprobes: Use ftrace_location_range in __recover_probed_insn
arch/x86/kernel/kprobes/core.c | 6 +++++-
include/linux/ftrace.h | 4 ++++
kernel/kprobes.c | 2 +-
3 files changed, 10 insertions(+), 2 deletions(-)
^ permalink raw reply [flat|nested] 3+ messages in thread* [PATCHv2 1/2] kprobes: Increase the kprobe hash table size
2026-11-22 20:02 [PATCHv2 0/2] kprobes: Reduce registration lookup costs Jiri Olsa
@ 2026-11-22 20:02 ` Jiri Olsa
2026-11-22 20:02 ` [PATCHv2 2/2] x86/kprobes: Use ftrace_location_range in __recover_probed_insn Jiri Olsa
1 sibling, 0 replies; 3+ messages in thread
From: Jiri Olsa @ 2026-11-22 20:02 UTC (permalink / raw)
To: Masami Hiramatsu, Naveen N Rao, David S. Miller
Cc: Peter Zijlstra, Steven Rostedt, Ingo Molnar, x86,
linux-trace-kernel, linux-kernel, bpf
The kprobe hash table has 64 buckets, so with many probes registered
each get_kprobe() call walks a long chain. With 20000 probes that is
more than 300 entries per bucket on average, and every entry is a
separate struct kprobe, so the walk is mostly cache misses.
get_kprobe() is not called just once per registered probe. On x86 the
registration path calls it for each address it has to check around
the new probe: get_optimized_kprobe(), __recover_probed_insn(),
__recover_optprobed_insn() and arch_check_optimized_kprobe() all do
one lookup per byte or per decoded instruction, and most of these
lookups miss, which means walking the whole chain. It is also called
from the breakpoint and ftrace handlers on every probe hit.
Increase KPROBE_HASH_BITS from 6 to 12, which gives 4096 buckets and
cuts the chains to about 5 entries for 20000 probes and about 12
entries for 50000 probes.
Time to register the probes with the kprobe_many_example sample
module [1] (insmod with nr_functions=20000 and nr_functions=50000)
in a KVM guest, mean of 5 runs, for different KPROBE_HASH_BITS
values. The table memory is the size of the static kprobe_table on
64-bit:
bits buckets table memory 20000 probes 50000 probes
6 64 512 B 4.71 s 29.99 s
8 256 2 KB 2.05 s 9.90 s
10 1024 8 KB 1.39 s 4.63 s
12 4096 32 KB 1.30 s 3.64 s
14 16384 128 KB 1.28 s 3.41 s
16 65536 512 KB 1.28 s 3.38 s
With 12 bits the registration of 20000 probes is 3.6 times faster
and the registration of 50000 probes is 8.2 times faster than with
the current 6 bits. 12 bits is where the curve flattens: compared
to 10 bits it is still 21% faster for 50000 probes (6% for 20000),
while going further to 16 bits gains only 1.5% for 20000 probes and
7% for 50000 probes, for a table 16 times bigger (512KB vs 32KB).
The cost is the size of the static table, which grows from 64 to
4096 hlist heads, that is from 512 bytes to 32KB on 64-bit. The
functions that walk the whole table (module notifier, arming or
optimizing all probes, the debugfs list) are slow paths that take
kprobe_mutex and now scan 4096 buckets.
The sample module was posted as the last patch of the kprobes
lookup cost series [1] only to demonstrate the testing, it is not
merged and is not going to be.
[1] https://lore.kernel.org/bpf/20261109195827.1091410-1-jolsa@kernel.org/
Suggested-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
kernel/kprobes.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/kprobes.c b/kernel/kprobes.c
index f14f7ff45961..47ce0bfb7298 100644
--- a/kernel/kprobes.c
+++ b/kernel/kprobes.c
@@ -50,7 +50,7 @@
#include <asm/errno.h>
#include <linux/uaccess.h>
-#define KPROBE_HASH_BITS 6
+#define KPROBE_HASH_BITS 12
#define KPROBE_TABLE_SIZE (1 << KPROBE_HASH_BITS)
#if !defined(CONFIG_OPTPROBES) || !defined(CONFIG_SYSCTL)
--
2.56.1
^ permalink raw reply [flat|nested] 3+ messages in thread* [PATCHv2 2/2] x86/kprobes: Use ftrace_location_range in __recover_probed_insn
2026-11-22 20:02 [PATCHv2 0/2] kprobes: Reduce registration lookup costs Jiri Olsa
2026-11-22 20:02 ` [PATCHv2 1/2] kprobes: Increase the kprobe hash table size Jiri Olsa
@ 2026-11-22 20:02 ` Jiri Olsa
1 sibling, 0 replies; 3+ messages in thread
From: Jiri Olsa @ 2026-11-22 20:02 UTC (permalink / raw)
To: Masami Hiramatsu, Naveen N Rao, David S. Miller
Cc: Peter Zijlstra, Steven Rostedt, Ingo Molnar, x86,
linux-trace-kernel, linux-kernel, bpf
The __recover_probed_insn function is called for every instruction
decoded on the way to the probed address and it uses ftrace_location
to find out whether the instruction is the ftrace location.
When the address is not ftrace location, ftrace_location does also
the symbol lookup to map the sym+0 address to its ftrace location.
But __recover_probed_insn only checks if the returned location is
the address itself, and that can't be the result of the mapping
after the lookup of the address failed, so the symbol lookup is
done for nothing for almost every decoded instruction.
Using ftrace_location_range for the address instead, which does
just the lookup of the address.
Adding the missing ftrace_location_range stub for the config
without dynamic ftrace.
Registering 20587 kprobes (as disabled) with the kprobe_many_example
sample module [1] in qemu with 8 CPUs goes from 4.02s to 3.70s.
The sample module was posted as the last patch of the kprobes
lookup cost series [1] only to demonstrate the testing, it is not
merged and is not going to be.
[1] https://lore.kernel.org/bpf/20261109195827.1091410-1-jolsa@kernel.org/
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
arch/x86/kernel/kprobes/core.c | 6 +++++-
include/linux/ftrace.h | 4 ++++
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c
index 133ff20caccd..34d71907e6d1 100644
--- a/arch/x86/kernel/kprobes/core.c
+++ b/arch/x86/kernel/kprobes/core.c
@@ -207,7 +207,11 @@ __recover_probed_insn(kprobe_opcode_t *buf, unsigned long addr)
bool faddr;
kp = get_kprobe((void *)addr);
- faddr = ftrace_location(addr) == addr;
+ /*
+ * Only the exact ftrace location matters, so skip the symbol
+ * lookup ftrace_location() does to map sym+0 to its location.
+ */
+ faddr = ftrace_location_range(addr, addr) == addr;
/*
* Use the current code if it is not modified by Kprobe
* and it cannot be modified by ftrace.
diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h
index 1c2056a1a68b..255751899682 100644
--- a/include/linux/ftrace.h
+++ b/include/linux/ftrace.h
@@ -1075,6 +1075,10 @@ static inline unsigned long ftrace_location(unsigned long ip)
{
return 0;
}
+static inline unsigned long ftrace_location_range(unsigned long start, unsigned long end)
+{
+ return 0;
+}
/*
* Again users of functions that have ftrace_ops may not
--
2.56.1
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-11-22 20:03 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-22 20:02 [PATCHv2 0/2] kprobes: Reduce registration lookup costs Jiri Olsa
2026-11-22 20:02 ` [PATCHv2 1/2] kprobes: Increase the kprobe hash table size Jiri Olsa
2026-11-22 20:02 ` [PATCHv2 2/2] x86/kprobes: Use ftrace_location_range in __recover_probed_insn Jiri Olsa
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®