* [PATCH v18 01/11] x86/hw_breakpoints: Add arch_modify_local_hw_breakpoint_addr() API
2026-09-25 11:10 [PATCH v18 00/11] tracing: wprobe: x86: Add wprobe for watchpoint Masami Hiramatsu (Google)
@ 2026-09-25 11:11 ` Masami Hiramatsu (Google)
2026-09-25 11:11 ` [PATCH v18 03/11] perf/hw_breakpoint: Add register_wide_hw_breakpoint_cpuslocked() API Masami Hiramatsu (Google)
` (8 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-09-25 11:11 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users, x86
From: Jinchao Wang <wangjinchao600@gmail.com>
Wprobe needs to move an active per-CPU watchpoint without releasing and
reserving its hardware slot.
Add arch_modify_local_hw_breakpoint_addr() as the architecture backend for
modify_local_hw_breakpoint_addr(). It validates the new address, finds the
installed local slot, and updates the hardware debug register and per-CPU
shadow register. The core layer (modify_local_hw_breakpoint_addr()) handles
updating bp->attr.bp_addr and counter_arch_bp(bp)->address.
The caller must provide an installed local event and a valid address.
Slot ownership, breakpoint type, length, mask and DR7 remain unchanged.
Link: https://lore.kernel.org/all/59637b96946653393a7ad3c7de094094796b39c2.1785067572.git.wangjinchao600@gmail.com/
Assisted-by: LLM
Signed-off-by: Jinchao Wang <wangjinchao600@gmail.com>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v18:
- Drop perf_guest_in_guest() check and cpu_dr7_seq sequence loop
since wprobe does not support updating DR7 in NMI context.
Changes in v17:
- In arch_modify_local_hw_breakpoint_addr(), check perf_guest_in_guest()
instead of cpu_dr_in_guest.
Changes in v16:
- Clear breakpoint in shadow cpu_dr7 before setting hardware DR7 so
an intervening NMI cannot re-enable the breakpoint via
local_db_restore() while the address register is being updated,
preventing spurious #DB exceptions.
- Check cpu_dr_in_guest and return -EBUSY to reject modifying local
breakpoints while executing a KVM guest.
Changes in v15:
- Temporarily clear the breakpoint enable bit in DR7 before updating the
debug address register, and re-enable it afterward, to comply with the
x86 hardware specification and avoid spurious #DB exceptions.
Changes in v13:
- Add compiler barrier (barrier()) before checking cpu_dr7_seq to
prevent hoisting the sequence check before set_debugreg().
- Clarify in commit log and code comment that the core layer manages
updating bp->attr.bp_addr and counter_arch_bp(bp)->address.
Changes in v11:
- Return int error code instead of void.
- Validate the new address using hw_breakpoint_arch_parse() before
updating registers.
- Increment cpu_dr7_seq in a sequence loop for NMI protection.
- Return -ENOENT if the breakpoint slot is not found on the local CPU.
---
arch/x86/include/asm/hw_breakpoint.h | 2 +
arch/x86/kernel/hw_breakpoint.c | 51 ++++++++++++++++++++++++++++++++++
2 files changed, 53 insertions(+)
diff --git a/arch/x86/include/asm/hw_breakpoint.h b/arch/x86/include/asm/hw_breakpoint.h
index 0bc931cd0698..f35ec491f6dd 100644
--- a/arch/x86/include/asm/hw_breakpoint.h
+++ b/arch/x86/include/asm/hw_breakpoint.h
@@ -59,6 +59,8 @@ extern int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
int arch_install_hw_breakpoint(struct perf_event *bp);
+int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr);
void arch_uninstall_hw_breakpoint(struct perf_event *bp);
void hw_breakpoint_pmu_read(struct perf_event *bp);
void hw_breakpoint_pmu_unthrottle(struct perf_event *bp);
diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c
index f846c15f21ca..3f63f7899f95 100644
--- a/arch/x86/kernel/hw_breakpoint.c
+++ b/arch/x86/kernel/hw_breakpoint.c
@@ -133,6 +133,57 @@ int arch_install_hw_breakpoint(struct perf_event *bp)
return 0;
}
+/*
+ * Modify the address of an installed perf counter breakpoint on local CPU.
+ *
+ * This updates the debug address register and per-CPU shadow state without
+ * changing slot allocation or DR7. The core layer (modify_local_hw_breakpoint_addr())
+ * manages updating bp->attr.bp_addr and counter_arch_bp(bp)->address.
+ *
+ * Atomic: called with IRQs disabled and handles variables and registers
+ * local to this CPU.
+ */
+int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr)
+{
+ struct arch_hw_breakpoint *info = counter_arch_bp(bp);
+ struct arch_hw_breakpoint hw;
+ struct perf_event_attr attr = bp->attr;
+ unsigned long dr7;
+ int i, ret;
+
+ lockdep_assert_irqs_disabled();
+
+ attr.bp_addr = addr;
+ ret = hw_breakpoint_arch_parse(bp, &attr, &hw);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < HBP_NUM; i++) {
+ if (this_cpu_read(bp_per_reg[i]) == bp)
+ break;
+ }
+
+ if (WARN_ONCE(i == HBP_NUM, "Can't find any breakpoint slot"))
+ return -ENOENT;
+
+ /*
+ * Temporarily clear the breakpoint enable bit in DR7 before updating the
+ * debug address register, and re-enable it afterward, to comply with the
+ * x86 hardware specification and avoid spurious #DB exceptions.
+ */
+ dr7 = this_cpu_read(cpu_dr7);
+ set_debugreg(dr7 & ~__encode_dr7(i, info->len, info->type), 7);
+
+ this_cpu_write(cpu_debugreg[i], addr);
+ barrier();
+ set_debugreg(addr, i);
+
+ set_debugreg(dr7, 7);
+
+ return 0;
+}
+
/*
* Uninstall the breakpoint contained in the given counter.
*
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v18 03/11] perf/hw_breakpoint: Add register_wide_hw_breakpoint_cpuslocked() API
2026-09-25 11:10 [PATCH v18 00/11] tracing: wprobe: x86: Add wprobe for watchpoint Masami Hiramatsu (Google)
2026-09-25 11:11 ` [PATCH v18 01/11] x86/hw_breakpoints: Add arch_modify_local_hw_breakpoint_addr() API Masami Hiramatsu (Google)
@ 2026-09-25 11:11 ` Masami Hiramatsu (Google)
2026-09-25 11:11 ` [PATCH v18 04/11] tracing/wprobe: Add wprobe (watchpoint probe) trace event support Masami Hiramatsu (Google)
` (7 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-09-25 11:11 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users, x86
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Add register_wide_hw_breakpoint_cpuslocked() which registers a wide
hardware breakpoint across all online CPUs while caller holds
cpus_read_lock(). This allows callers who need to perform additional
setup under cpus_read_lock() without releasing it to avoid races with
CPU hotplug and without triggering recursive lockdep warnings.
Refactor register_wide_hw_breakpoint() as a wrapper around
register_wide_hw_breakpoint_cpuslocked().
Assisted-by: LLM
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
include/linux/hw_breakpoint.h | 9 +++++++++
kernel/events/hw_breakpoint.c | 38 ++++++++++++++++++++++++++++++++------
2 files changed, 41 insertions(+), 6 deletions(-)
diff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h
index bf65c7fffd99..65efe55fef14 100644
--- a/include/linux/hw_breakpoint.h
+++ b/include/linux/hw_breakpoint.h
@@ -76,6 +76,11 @@ register_wide_hw_breakpoint_cpu(struct perf_event_attr *attr,
void *context,
int cpu);
+extern struct perf_event * __percpu *
+register_wide_hw_breakpoint_cpuslocked(struct perf_event_attr *attr,
+ perf_overflow_handler_t triggered,
+ void *context);
+
extern struct perf_event * __percpu *
register_wide_hw_breakpoint(struct perf_event_attr *attr,
perf_overflow_handler_t triggered,
@@ -123,6 +128,10 @@ register_wide_hw_breakpoint_cpu(struct perf_event_attr *attr,
void *context,
int cpu) { return NULL; }
static inline struct perf_event * __percpu *
+register_wide_hw_breakpoint_cpuslocked(struct perf_event_attr *attr,
+ perf_overflow_handler_t triggered,
+ void *context) { return NULL; }
+static inline struct perf_event * __percpu *
register_wide_hw_breakpoint(struct perf_event_attr *attr,
perf_overflow_handler_t triggered,
void *context) { return NULL; }
diff --git a/kernel/events/hw_breakpoint.c b/kernel/events/hw_breakpoint.c
index 69bfb17b87c4..5627ad97600a 100644
--- a/kernel/events/hw_breakpoint.c
+++ b/kernel/events/hw_breakpoint.c
@@ -832,27 +832,30 @@ void unregister_hw_breakpoint(struct perf_event *bp)
EXPORT_SYMBOL_GPL(unregister_hw_breakpoint);
/**
- * register_wide_hw_breakpoint - register a wide breakpoint in the kernel
+ * register_wide_hw_breakpoint_cpuslocked - register a wide breakpoint with cpus locked
* @attr: breakpoint attributes
* @triggered: callback to trigger when we hit the breakpoint
* @context: context data could be used in the triggered callback
*
+ * Same as register_wide_hw_breakpoint(), but caller must hold cpus_read_lock().
+ *
* @return a set of per_cpu pointers to perf events
*/
struct perf_event * __percpu *
-register_wide_hw_breakpoint(struct perf_event_attr *attr,
- perf_overflow_handler_t triggered,
- void *context)
+register_wide_hw_breakpoint_cpuslocked(struct perf_event_attr *attr,
+ perf_overflow_handler_t triggered,
+ void *context)
{
struct perf_event * __percpu *cpu_events, *bp;
long err = 0;
int cpu;
+ lockdep_assert_cpus_held();
+
cpu_events = alloc_percpu(typeof(*cpu_events));
if (!cpu_events)
return ERR_PTR_PCPU(-ENOMEM);
- cpus_read_lock();
for_each_online_cpu(cpu) {
bp = perf_event_create_kernel_counter(attr, cpu, NULL,
triggered, context);
@@ -863,7 +866,6 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
per_cpu(*cpu_events, cpu) = bp;
}
- cpus_read_unlock();
if (likely(!err))
return cpu_events;
@@ -871,6 +873,30 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
unregister_wide_hw_breakpoint(cpu_events);
return ERR_PTR_PCPU(err);
}
+EXPORT_SYMBOL_GPL(register_wide_hw_breakpoint_cpuslocked);
+
+/**
+ * register_wide_hw_breakpoint - register a wide breakpoint in the kernel
+ * @attr: breakpoint attributes
+ * @triggered: callback to trigger when we hit the breakpoint
+ * @context: context data could be used in the triggered callback
+ *
+ * @return a set of per_cpu pointers to perf events
+ */
+struct perf_event * __percpu *
+register_wide_hw_breakpoint(struct perf_event_attr *attr,
+ perf_overflow_handler_t triggered,
+ void *context)
+{
+ struct perf_event * __percpu *cpu_events;
+
+ cpus_read_lock();
+ cpu_events = register_wide_hw_breakpoint_cpuslocked(attr, triggered,
+ context);
+ cpus_read_unlock();
+
+ return cpu_events;
+}
EXPORT_SYMBOL_GPL(register_wide_hw_breakpoint);
/**
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v18 04/11] tracing/wprobe: Add wprobe (watchpoint probe) trace event support
2026-09-25 11:10 [PATCH v18 00/11] tracing: wprobe: x86: Add wprobe for watchpoint Masami Hiramatsu (Google)
2026-09-25 11:11 ` [PATCH v18 01/11] x86/hw_breakpoints: Add arch_modify_local_hw_breakpoint_addr() API Masami Hiramatsu (Google)
2026-09-25 11:11 ` [PATCH v18 03/11] perf/hw_breakpoint: Add register_wide_hw_breakpoint_cpuslocked() API Masami Hiramatsu (Google)
@ 2026-09-25 11:11 ` Masami Hiramatsu (Google)
2026-09-25 11:12 ` [PATCH v18 05/11] x86: hw_breakpoint: Add a kconfig to clarify when a breakpoint fires Masami Hiramatsu (Google)
` (6 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-09-25 11:11 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users, x86
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Add hardware-breakpoint-based dynamic trace event support (wprobe).
Wprobe creates a dynamic event on data read/write accesses using
hardware breakpoints and logs the access context and fetchargs.
Link: https://lore.kernel.org/all/59637b96946653393a7ad3c7de094094796b39c2.1785067572.git.wangjinchao600@gmail.com/
Assisted-by: LLM
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v16:
- Remove non-existent wprobe_events file description from readme_msg
in kernel/trace/trace.c as wprobe events are created via
dynamic_events.
Changes in v15:
- Introduce trace_wprobe_is_valid_addr() to validate target address
and natural alignment for wprobe length during event creation.
- Move local variable declarations to function entry in
trace_wprobe_show() to comply with kernel C coding standards.
- Fix indentation in kernel/trace/trace.c to use tabs instead of spaces.
Changes in v14:
- In parse_address_spec(), reject address 0 and user-space addresses
by changing the check to '_addr < TASK_SIZE'.
- In trace_wprobe_show(), format symbol and offset explicitly instead
of using '%pS' to ensure dynamic event restoration works properly.
- Add offset field to struct trace_wprobe and pass it to
alloc_trace_wprobe().
Changes in v13:
- Fix documentation notation to SYMBOL[+|-OFFS] as same as others.
- Fix documentation indents and add default value.
- Fix README file to add [<args>].
- Add a note about module unloading.
Changes in v12:
- Fix syntax comment typo ('b' to 'w') and remove dead 'ret = 0'
initialization in __trace_wprobe_create().
- Fix documentation notation to SYMBOL[[+|-]OFFS].
Changes in v11:
- Add trace_wprobe_is_busy() to prevent releasing busy events.
Changes in v9:
- Use kzalloc_flex for alloc_trace_wprobe.
---
Documentation/trace/index.rst | 1
Documentation/trace/wprobetrace.rst | 78 ++++
include/linux/trace_events.h | 2
kernel/trace/Kconfig | 13 +
kernel/trace/Makefile | 1
kernel/trace/trace.c | 5
kernel/trace/trace.h | 5
kernel/trace/trace_probe.c | 21 +
kernel/trace/trace_probe.h | 10
kernel/trace/trace_wprobe.c | 775 +++++++++++++++++++++++++++++++++++
10 files changed, 907 insertions(+), 4 deletions(-)
create mode 100644 Documentation/trace/wprobetrace.rst
create mode 100644 kernel/trace/trace_wprobe.c
diff --git a/Documentation/trace/index.rst b/Documentation/trace/index.rst
index f4058e8e92e3..4f3fa60e4ecd 100644
--- a/Documentation/trace/index.rst
+++ b/Documentation/trace/index.rst
@@ -36,6 +36,7 @@ the Linux kernel.
kprobes
kprobetrace
fprobetrace
+ wprobetrace
eprobetrace
fprobe
ring-buffer-design
diff --git a/Documentation/trace/wprobetrace.rst b/Documentation/trace/wprobetrace.rst
new file mode 100644
index 000000000000..31cc68b464e9
--- /dev/null
+++ b/Documentation/trace/wprobetrace.rst
@@ -0,0 +1,78 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=======================================
+Watchpoint probe (wprobe) Event Tracing
+=======================================
+
+.. Author: Masami Hiramatsu <mhiramat@kernel.org>
+
+Overview
+--------
+
+Wprobe event is a dynamic event based on the hardware breakpoint, which is
+similar to other probe events, but it is for watching data access. It allows
+you to trace which code accesses a specified data.
+
+As same as other dynamic events, wprobe events are defined via
+`dynamic_events` interface file on tracefs.
+
+Synopsis of wprobe-events
+-------------------------
+::
+
+ w:[GRP/][EVENT] SPEC [FETCHARGS] : Probe on data access
+
+ GRP : Group name for wprobe. If omitted, use "wprobes" for it.
+ EVENT : Event name for wprobe. If omitted, an event name is
+ generated based on the address or symbol.
+ SPEC : Breakpoint specification.
+ [r|w|rw]@<ADDRESS|SYMBOL[+|-OFFS]>[:LENGTH]
+
+ r|w|rw : Access type, r for read, w for write, and rw for both.
+ Default is rw if omitted.
+ ADDRESS : Address to watch (hexadecimal). MUST be in kernel space.
+ SYMBOL[+|-OFFS] : Symbol name to watch. (Optional positive/negative offset)
+ LENGTH : Length of the data to watch in bytes. (1, 2, 4, or 8)
+ Default is 4.
+
+ FETCHARGS : Arguments. Each probe can have up to 128 args.
+ $addr : Fetch the accessing address.
+ $value : Fetch the memory value at the accessing address (same as +0($addr)).
+ @ADDR : Fetch memory at ADDR (ADDR should be in kernel)
+ @SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symbol)
+ +|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*1)(\*2)
+ \IMM : Store an immediate value to the argument.
+ NAME=FETCHARG : Set NAME as the argument name of FETCHARG.
+ FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types
+ (u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types
+ (x8/x16/x32/x64), "char", "string", "ustring", "symbol", "symstr"
+ and bitfield are supported.
+
+ (\*1) This is useful for fetching a field of data structures.
+ (\*2) "u" means user-space dereference.
+
+For the details of TYPE, see :ref:`kprobetrace documentation <kprobetrace_types>`.
+
+Usage examples
+--------------
+Here is an example to add a wprobe event on a variable `jiffies`.
+::
+
+ # echo 'w:my_jiffies w@jiffies' >> dynamic_events
+ # cat dynamic_events
+ w:wprobes/my_jiffies w@jiffies
+ # echo 1 > events/wprobes/enable
+ # cat trace | head
+ # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
+ # | | | ||||| | |
+ <idle>-0 [000] d.Z1. 717.026259: my_jiffies: (tick_do_update_jiffies64+0xbe/0x130)
+ <idle>-0 [000] d.Z1. 717.026373: my_jiffies: (tick_do_update_jiffies64+0xbe/0x130)
+
+You can see the code which writes to `jiffies` is `tick_do_update_jiffies64()`.
+
+Notes
+-----
+Wprobe event does not disable itself even if the module is unloaded.
+For example, if you add a wprobe event on a module variable, and then
+unload the module, the wprobe event will still be enabled. This is for
+watching the address is used unexpectedly after the module is unloaded.
diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
index 5cbd09c8be8d..43ffd9a76d88 100644
--- a/include/linux/trace_events.h
+++ b/include/linux/trace_events.h
@@ -337,6 +337,7 @@ enum {
TRACE_EVENT_FL_UPROBE_BIT,
TRACE_EVENT_FL_EPROBE_BIT,
TRACE_EVENT_FL_FPROBE_BIT,
+ TRACE_EVENT_FL_WPROBE_BIT,
TRACE_EVENT_FL_CUSTOM_BIT,
TRACE_EVENT_FL_TEST_STR_BIT,
};
@@ -367,6 +368,7 @@ enum {
TRACE_EVENT_FL_UPROBE = (1 << TRACE_EVENT_FL_UPROBE_BIT),
TRACE_EVENT_FL_EPROBE = (1 << TRACE_EVENT_FL_EPROBE_BIT),
TRACE_EVENT_FL_FPROBE = (1 << TRACE_EVENT_FL_FPROBE_BIT),
+ TRACE_EVENT_FL_WPROBE = (1 << TRACE_EVENT_FL_WPROBE_BIT),
TRACE_EVENT_FL_CUSTOM = (1 << TRACE_EVENT_FL_CUSTOM_BIT),
TRACE_EVENT_FL_TEST_STR = (1 << TRACE_EVENT_FL_TEST_STR_BIT),
};
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 0ab5916575a9..b58c2565024f 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -862,6 +862,19 @@ config EPROBE_EVENTS
convert the type of an event field. For example, turn an
address into a string.
+config WPROBE_EVENTS
+ bool "Enable wprobe-based dynamic events"
+ depends on TRACING
+ depends on HAVE_HW_BREAKPOINT
+ select PROBE_EVENTS
+ select DYNAMIC_EVENTS
+ help
+ This allows the user to add watchpoint tracing events based on
+ hardware breakpoints on the fly via the ftrace interface.
+
+ Those events can be inserted wherever hardware breakpoints can be
+ set, and record accessed memory address and values.
+
config BPF_EVENTS
depends on BPF_SYSCALL
depends on (KPROBE_EVENTS || UPROBE_EVENTS) && PERF_EVENTS
diff --git a/kernel/trace/Makefile b/kernel/trace/Makefile
index f934ff586bd4..141c8323de20 100644
--- a/kernel/trace/Makefile
+++ b/kernel/trace/Makefile
@@ -126,6 +126,7 @@ obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o
obj-$(CONFIG_FPROBE) += fprobe.o
obj-$(CONFIG_RETHOOK) += rethook.o
obj-$(CONFIG_FPROBE_EVENTS) += trace_fprobe.o
+obj-$(CONFIG_WPROBE_EVENTS) += trace_wprobe.o
obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
obj-$(CONFIG_RV) += rv/
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index a946e0183fd1..da85e2568c41 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4295,7 +4295,7 @@ static const char readme_msg[] =
"\t\t\t Write into this file to define/undefine new trace events.\n"
#endif
#if defined(CONFIG_KPROBE_EVENTS) || defined(CONFIG_UPROBE_EVENTS) || \
- defined(CONFIG_FPROBE_EVENTS)
+ defined(CONFIG_FPROBE_EVENTS) || defined(CONFIG_WPROBE_EVENTS)
"\t accepts: event-definitions (one definition per line)\n"
#if defined(CONFIG_KPROBE_EVENTS) || defined(CONFIG_UPROBE_EVENTS)
"\t Format: p[:[<group>/][<event>]] <place> [<args>]\n"
@@ -4305,6 +4305,9 @@ static const char readme_msg[] =
"\t f[:[<group>/][<event>]] <func-name>[%return] [<args>]\n"
"\t t[:[<group>/][<event>]] <tracepoint> [<args>]\n"
#endif
+#ifdef CONFIG_WPROBE_EVENTS
+ "\t w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>] [<args>]\n"
+#endif
#ifdef CONFIG_HIST_TRIGGERS
"\t s:[synthetic/]<event> <field> [<field>]\n"
#endif
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 74a7a50d1e78..18588c550a98 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -179,6 +179,11 @@ struct fexit_trace_entry_head {
unsigned long ret_ip;
};
+struct wprobe_trace_entry_head {
+ struct trace_entry ent;
+ unsigned long ip;
+};
+
#define TRACE_BUF_SIZE 1024
struct trace_array;
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index 804442b2f7d2..64e7e363e8e2 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -1403,6 +1403,23 @@ static int parse_probe_vars(char *orig_arg, const struct fetch_type *t,
return 0;
}
+ /* wprobe only support "$addr" and "$value" variable */
+ if (ctx->flags & TPARG_FL_WPROBE) {
+ if (!strcmp(arg, "addr")) {
+ code->op = FETCH_OP_BADDR;
+ return 0;
+ }
+ if (!strcmp(arg, "value")) {
+ code->op = FETCH_OP_BADDR;
+ code++;
+ code->op = FETCH_OP_DEREF;
+ code->offset = 0;
+ *pcode = code;
+ return 0;
+ }
+ goto inval;
+ }
+
if (strcmp(arg, "comm") == 0 || strcmp(arg, "COMM") == 0) {
code->op = FETCH_OP_COMM;
return 0;
@@ -1462,8 +1479,8 @@ static int parse_probe_arg_register(char *arg, struct fetch_insn *code,
{
int ret;
- if (ctx->flags & (TPARG_FL_TEVENT | TPARG_FL_FPROBE)) {
- /* eprobe and fprobe do not handle registers */
+ if (ctx->flags & (TPARG_FL_TEVENT | TPARG_FL_FPROBE | TPARG_FL_WPROBE)) {
+ /* eprobe, fprobe and wprobe do not handle registers */
trace_probe_log_err(ctx->offset, BAD_VAR);
return -EINVAL;
}
diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h
index d1fb3520700f..f20cafb87cc6 100644
--- a/kernel/trace/trace_probe.h
+++ b/kernel/trace/trace_probe.h
@@ -90,6 +90,7 @@ typedef int (*print_type_func_t)(struct trace_seq *, void *, void *);
FETCH_OP(STACK, param), /* Stack: .param = index */ \
FETCH_OP(STACKP, none), /* Stack pointer */ \
FETCH_OP(RETVAL, none), /* Return value */ \
+ FETCH_OP(BADDR, none), /* Break address */ \
FETCH_OP(IMM, imm), /* Immediate: .immediate */ \
FETCH_OP(COMM, none), /* Current comm */ \
FETCH_OP(CURRENT, none), /* Current task_struct address */\
@@ -420,6 +421,7 @@ static inline int traceprobe_get_entry_data_size(struct trace_probe *tp)
#define TPARG_FL_USER BIT(4)
#define TPARG_FL_FPROBE BIT(5)
#define TPARG_FL_TPOINT BIT(6)
+#define TPARG_FL_WPROBE BIT(7)
#define TPARG_FL_LOC_MASK GENMASK(4, 0)
static inline bool tparg_is_function_entry(unsigned int flags)
@@ -546,6 +548,10 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
C(ARG_TOO_LONG, "Argument expression is too long"), \
C(ARRAY_NO_CLOSE, "Array is not closed"), \
C(ARRAY_TOO_BIG, "Array number is too big"), \
+ C(BAD_ACCESS_ADDR, "Invalid access memory address"), \
+ C(BAD_ACCESS_FMT, "Access memory address requires @"), \
+ C(BAD_ACCESS_LEN, "This memory access length is not supported"), \
+ C(BAD_ACCESS_TYPE, "Bad memory access type"), \
C(BAD_ADDR_SUFFIX, "Invalid probed address suffix"), \
C(BAD_ARG_NAME, "Argument name must follow the same rules as C identifiers"), \
C(BAD_ARG_NUM, "Invalid argument number"), \
@@ -628,7 +634,9 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
C(TYPECAST_NOT_EVENT, "Typecasts are only for eprobe fields"), \
C(TYPECAST_REQ_FIELD, "Typecast requires a field access"), \
C(TYPECAST_SYM_OFFSET, "@SYM+/-OFFSET with typecast needs parentheses"), \
- C(USED_ARG_NAME, "This argument name is already used"),
+ C(USED_ARG_NAME, "This argument name is already used"), \
+ C(WPROBE_NO_MAXACT, "Watchpoint probe does not support maxactive"), \
+ C(WPROBE_NO_SIBLING, "Watchpoint probe does not support sibling probes"),
#undef C
#define C(a, b) TP_ERR_##a
diff --git a/kernel/trace/trace_wprobe.c b/kernel/trace/trace_wprobe.c
new file mode 100644
index 000000000000..9c6100b63e8d
--- /dev/null
+++ b/kernel/trace/trace_wprobe.c
@@ -0,0 +1,775 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hardware-breakpoint-based tracing events
+ *
+ * Copyright (C) 2023, Masami Hiramatsu <mhiramat@kernel.org>
+ */
+#define pr_fmt(fmt) "trace_wprobe: " fmt
+
+#include <linux/compiler.h>
+#include <linux/hw_breakpoint.h>
+#include <linux/kallsyms.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/perf_event.h>
+#include <linux/rculist.h>
+#include <linux/security.h>
+#include <linux/tracepoint.h>
+#include <linux/uaccess.h>
+
+#include <asm/ptrace.h>
+
+#include "trace_dynevent.h"
+#include "trace_probe.h"
+#include "trace_probe_kernel.h"
+#include "trace_probe_tmpl.h"
+#include "trace_output.h"
+
+#define WPROBE_EVENT_SYSTEM "wprobes"
+
+static int trace_wprobe_create(const char *raw_command);
+static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev);
+static int trace_wprobe_release(struct dyn_event *ev);
+static bool trace_wprobe_is_busy(struct dyn_event *ev);
+static bool trace_wprobe_match(const char *system, const char *event,
+ int argc, const char **argv, struct dyn_event *ev);
+
+static struct dyn_event_operations trace_wprobe_ops = {
+ .create = trace_wprobe_create,
+ .show = trace_wprobe_show,
+ .is_busy = trace_wprobe_is_busy,
+ .free = trace_wprobe_release,
+ .match = trace_wprobe_match,
+};
+
+struct trace_wprobe {
+ struct dyn_event devent;
+ struct perf_event * __percpu *bp_event;
+ unsigned long addr;
+ int offset;
+ int len;
+ int type;
+ const char *symbol;
+ struct trace_probe tp;
+};
+
+static bool is_trace_wprobe(struct dyn_event *ev)
+{
+ return ev->ops == &trace_wprobe_ops;
+}
+
+static struct trace_wprobe *to_trace_wprobe(struct dyn_event *ev)
+{
+ return container_of(ev, struct trace_wprobe, devent);
+}
+
+#define for_each_trace_wprobe(pos, dpos) \
+ for_each_dyn_event(dpos) \
+ if (is_trace_wprobe(dpos) && (pos = to_trace_wprobe(dpos)))
+
+static bool trace_wprobe_is_busy(struct dyn_event *ev)
+{
+ struct trace_wprobe *tw = to_trace_wprobe(ev);
+
+ return trace_probe_is_enabled(&tw->tp);
+}
+
+static bool trace_wprobe_match(const char *system, const char *event,
+ int argc, const char **argv, struct dyn_event *ev)
+{
+ struct trace_wprobe *tw = to_trace_wprobe(ev);
+
+ if (event[0] != '\0' && strcmp(trace_probe_name(&tw->tp), event))
+ return false;
+
+ if (system && strcmp(trace_probe_group_name(&tw->tp), system))
+ return false;
+
+ return trace_probe_match_command_args(&tw->tp, argc, argv);
+}
+
+/*
+ * Note that we don't verify the fetch_insn code, since it does not come
+ * from user space.
+ */
+static int
+process_fetch_insn(struct fetch_insn *code, void *rec, void *edata,
+ void *dest, void *base)
+{
+ void *baddr = rec;
+ unsigned long val;
+ int ret;
+
+retry:
+ /* 1st stage: get value from context */
+ switch (code->op) {
+ case FETCH_OP_BADDR:
+ val = (unsigned long)baddr;
+ break;
+ case FETCH_NOP_SYMBOL: /* Ignore a place holder */
+ code++;
+ goto retry;
+ default:
+ ret = process_common_fetch_insn(code, &val);
+ if (ret < 0)
+ return ret;
+ }
+ code++;
+
+ return process_fetch_insn_bottom(code, val, dest, base);
+}
+NOKPROBE_SYMBOL(process_fetch_insn)
+
+static void wprobe_trace_handler(struct trace_wprobe *tw,
+ unsigned long addr,
+ struct pt_regs *regs,
+ struct trace_event_file *trace_file)
+{
+ struct wprobe_trace_entry_head *entry;
+ struct trace_event_call *call = trace_probe_event_call(&tw->tp);
+ struct trace_event_buffer fbuffer;
+ int dsize;
+
+ if (WARN_ON_ONCE(call != trace_file->event_call))
+ return;
+
+ if (trace_trigger_soft_disabled(trace_file))
+ return;
+
+ if (READ_ONCE(tw->addr) != addr)
+ return;
+
+ dsize = __get_data_size(&tw->tp, (void *)addr, NULL);
+
+ entry = trace_event_buffer_reserve(&fbuffer, trace_file,
+ sizeof(*entry) + tw->tp.size + dsize);
+ if (!entry)
+ return;
+
+ entry->ip = instruction_pointer(regs);
+ store_trace_args(&entry[1], &tw->tp, (void *)addr, NULL, sizeof(*entry), dsize);
+
+ fbuffer.regs = regs;
+ trace_event_buffer_commit(&fbuffer);
+}
+
+static void wprobe_perf_handler(struct perf_event *bp,
+ struct perf_sample_data *data,
+ struct pt_regs *regs)
+{
+ struct trace_wprobe *tw = bp->overflow_handler_context;
+ struct event_file_link *link;
+ unsigned long addr = bp->attr.bp_addr;
+
+ trace_probe_for_each_link_rcu(link, &tw->tp)
+ wprobe_trace_handler(tw, addr, regs, link->file);
+}
+
+static int __register_trace_wprobe(struct trace_wprobe *tw)
+{
+ struct perf_event_attr attr;
+ int i, ret;
+
+ if (tw->bp_event)
+ return -EINVAL;
+
+ for (i = 0; i < tw->tp.nr_args; i++) {
+ ret = traceprobe_update_arg(&tw->tp.args[i]);
+ if (ret)
+ return ret;
+ }
+
+ hw_breakpoint_init(&attr);
+ attr.bp_addr = tw->addr;
+ attr.bp_len = tw->len;
+ attr.bp_type = tw->type;
+
+ tw->bp_event = register_wide_hw_breakpoint(&attr, wprobe_perf_handler, tw);
+ if (IS_ERR_PCPU(tw->bp_event)) {
+ int ret = PTR_ERR_PCPU(tw->bp_event);
+
+ tw->bp_event = NULL;
+ return ret;
+ }
+
+ return 0;
+}
+
+static void __unregister_trace_wprobe(struct trace_wprobe *tw)
+{
+ if (tw->bp_event) {
+ unregister_wide_hw_breakpoint(tw->bp_event);
+ tw->bp_event = NULL;
+ }
+}
+
+static void free_trace_wprobe(struct trace_wprobe *tw)
+{
+ if (tw) {
+ trace_probe_cleanup(&tw->tp);
+ kfree(tw->symbol);
+ kfree(tw);
+ }
+}
+DEFINE_FREE(free_trace_wprobe, struct trace_wprobe *,
+ if (!IS_ERR_OR_NULL(_T))
+ free_trace_wprobe(_T))
+
+
+static struct trace_wprobe *alloc_trace_wprobe(const char *group,
+ const char *event,
+ const char *symbol,
+ int offset,
+ unsigned long addr,
+ int len, int type, int nargs)
+{
+ struct trace_wprobe *tw __free(free_trace_wprobe) = NULL;
+ int ret;
+
+ tw = kzalloc_flex(*tw, tp.args, nargs);
+ if (!tw)
+ return ERR_PTR(-ENOMEM);
+
+ if (symbol) {
+ tw->symbol = kstrdup(symbol, GFP_KERNEL);
+ if (!tw->symbol)
+ return ERR_PTR(-ENOMEM);
+ }
+ tw->offset = offset;
+ tw->addr = addr;
+ tw->len = len;
+ tw->type = type;
+
+ ret = trace_probe_init(&tw->tp, event, group, false, nargs);
+ if (ret < 0)
+ return ERR_PTR(ret);
+
+ dyn_event_init(&tw->devent, &trace_wprobe_ops);
+ return_ptr(tw);
+}
+
+static struct trace_wprobe *find_trace_wprobe(const char *event,
+ const char *group)
+{
+ struct dyn_event *pos;
+ struct trace_wprobe *tw;
+
+ for_each_trace_wprobe(tw, pos)
+ if (strcmp(trace_probe_name(&tw->tp), event) == 0 &&
+ strcmp(trace_probe_group_name(&tw->tp), group) == 0)
+ return tw;
+ return NULL;
+}
+
+static enum print_line_t
+print_wprobe_event(struct trace_iterator *iter, int flags,
+ struct trace_event *event)
+{
+ struct wprobe_trace_entry_head *field;
+ struct trace_seq *s = &iter->seq;
+ struct trace_probe *tp;
+
+ field = (struct wprobe_trace_entry_head *)iter->ent;
+ tp = trace_probe_primary_from_call(
+ container_of(event, struct trace_event_call, event));
+ if (WARN_ON_ONCE(!tp))
+ goto out;
+
+ trace_seq_printf(s, "%s: (", trace_probe_name(tp));
+
+ if (!seq_print_ip_sym_offset(s, field->ip, flags))
+ goto out;
+
+ trace_seq_putc(s, ')');
+
+ if (trace_probe_print_args(s, tp->args, tp->nr_args,
+ (u8 *)&field[1], field) < 0)
+ goto out;
+
+ trace_seq_putc(s, '\n');
+out:
+ return trace_handle_return(s);
+}
+
+static int wprobe_event_define_fields(struct trace_event_call *event_call)
+{
+ int ret;
+ struct wprobe_trace_entry_head field;
+ struct trace_probe *tp;
+
+ tp = trace_probe_primary_from_call(event_call);
+ if (WARN_ON_ONCE(!tp))
+ return -ENOENT;
+
+ DEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0);
+
+ return traceprobe_define_arg_fields(event_call, sizeof(field), tp);
+}
+
+static struct trace_event_functions wprobe_funcs = {
+ .trace = print_wprobe_event
+};
+
+static struct trace_event_fields wprobe_fields_array[] = {
+ { .type = TRACE_FUNCTION_TYPE,
+ .define_fields = wprobe_event_define_fields },
+ {}
+};
+
+static int wprobe_register(struct trace_event_call *event,
+ enum trace_reg type, void *data);
+
+static inline void init_trace_event_call(struct trace_wprobe *tw)
+{
+ struct trace_event_call *call = trace_probe_event_call(&tw->tp);
+
+ call->event.funcs = &wprobe_funcs;
+ call->class->fields_array = wprobe_fields_array;
+ call->flags = TRACE_EVENT_FL_WPROBE;
+ call->class->reg = wprobe_register;
+}
+
+static int register_wprobe_event(struct trace_wprobe *tw)
+{
+ init_trace_event_call(tw);
+ return trace_probe_register_event_call(&tw->tp);
+}
+
+static int register_trace_wprobe_event(struct trace_wprobe *tw)
+{
+ struct trace_wprobe *old_tw;
+ int ret;
+
+ guard(mutex)(&event_mutex);
+
+ old_tw = find_trace_wprobe(trace_probe_name(&tw->tp),
+ trace_probe_group_name(&tw->tp));
+ if (old_tw) {
+ /*
+ * Wprobe does not support sibling probes because the event
+ * trigger (set_wprobe/clear_wprobe) identifies the target
+ * wprobe by its event name. Having multiple wprobes sharing
+ * the same event name would make the target ambiguous.
+ */
+ trace_probe_log_set_index(0);
+ trace_probe_log_err(0, WPROBE_NO_SIBLING);
+ return -EBUSY;
+ }
+
+ ret = register_wprobe_event(tw);
+ if (ret) {
+ trace_probe_log_set_index(0);
+ if (ret == -EEXIST)
+ trace_probe_log_err(0, EVENT_EXIST);
+ else if (ret != -ENOMEM)
+ trace_probe_log_err(0, FAIL_REG_PROBE);
+ return ret;
+ }
+
+ dyn_event_add(&tw->devent, trace_probe_event_call(&tw->tp));
+ return 0;
+}
+static int unregister_wprobe_event(struct trace_wprobe *tw)
+{
+ return trace_probe_unregister_event_call(&tw->tp);
+}
+
+static int unregister_trace_wprobe(struct trace_wprobe *tw)
+{
+ if (trace_probe_has_sibling(&tw->tp))
+ goto unreg;
+
+ if (trace_probe_is_enabled(&tw->tp))
+ return -EBUSY;
+
+ if (trace_event_dyn_busy(trace_probe_event_call(&tw->tp)))
+ return -EBUSY;
+
+ if (unregister_wprobe_event(tw))
+ return -EBUSY;
+
+unreg:
+ __unregister_trace_wprobe(tw);
+ dyn_event_remove(&tw->devent);
+ trace_probe_unlink(&tw->tp);
+
+ return 0;
+}
+
+static int enable_trace_wprobe(struct trace_event_call *call,
+ struct trace_event_file *file)
+{
+ struct trace_probe *tp;
+ struct trace_wprobe *tw;
+ bool enabled;
+ int ret = 0;
+
+ tp = trace_probe_primary_from_call(call);
+ if (WARN_ON_ONCE(!tp))
+ return -ENODEV;
+ enabled = trace_probe_is_enabled(tp);
+
+ if (file) {
+ ret = trace_probe_add_file(tp, file);
+ if (ret)
+ return ret;
+ } else {
+ trace_probe_set_flag(tp, TP_FLAG_PROFILE);
+ }
+
+ if (!enabled) {
+ list_for_each_entry(tw, trace_probe_probe_list(tp), tp.list) {
+ ret = __register_trace_wprobe(tw);
+ if (ret < 0) {
+ struct trace_wprobe *tmp;
+
+ list_for_each_entry(tmp, trace_probe_probe_list(tp), tp.list) {
+ if (tmp == tw)
+ break;
+ __unregister_trace_wprobe(tmp);
+ }
+ if (file)
+ trace_probe_remove_file(tp, file);
+ else
+ trace_probe_clear_flag(tp, TP_FLAG_PROFILE);
+ return ret;
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int disable_trace_wprobe(struct trace_event_call *call,
+ struct trace_event_file *file)
+{
+ struct trace_wprobe *tw;
+ struct trace_probe *tp;
+
+ tp = trace_probe_primary_from_call(call);
+ if (WARN_ON_ONCE(!tp))
+ return -ENODEV;
+
+ if (file) {
+ if (!trace_probe_get_file_link(tp, file))
+ return -ENOENT;
+ if (!trace_probe_has_single_file(tp))
+ goto out;
+ trace_probe_clear_flag(tp, TP_FLAG_TRACE);
+ } else {
+ trace_probe_clear_flag(tp, TP_FLAG_PROFILE);
+ }
+
+ if (!trace_probe_is_enabled(tp)) {
+ list_for_each_entry(tw, trace_probe_probe_list(tp), tp.list) {
+ __unregister_trace_wprobe(tw);
+ }
+ }
+
+out:
+ if (file)
+ trace_probe_remove_file(tp, file);
+
+ return 0;
+}
+
+static int wprobe_register(struct trace_event_call *event,
+ enum trace_reg type, void *data)
+{
+ struct trace_event_file *file = data;
+
+ switch (type) {
+ case TRACE_REG_REGISTER:
+ return enable_trace_wprobe(event, file);
+ case TRACE_REG_UNREGISTER:
+ return disable_trace_wprobe(event, file);
+
+#ifdef CONFIG_PERF_EVENTS
+ case TRACE_REG_PERF_REGISTER:
+ case TRACE_REG_PERF_UNREGISTER:
+ case TRACE_REG_PERF_OPEN:
+ case TRACE_REG_PERF_CLOSE:
+ case TRACE_REG_PERF_ADD:
+ case TRACE_REG_PERF_DEL:
+ return -EOPNOTSUPP;
+#endif
+ }
+ return 0;
+}
+
+static bool trace_wprobe_is_valid_addr(unsigned long addr, int len)
+{
+ if (addr < TASK_SIZE)
+ return false;
+
+ if (addr & (len - 1))
+ return false;
+
+ return true;
+}
+
+static int parse_address_spec(const char *spec, unsigned long *addr, int *type,
+ int *len, char **symbol, int *offset)
+{
+ char *_spec __free(kfree) = NULL;
+ int _len = HW_BREAKPOINT_LEN_4;
+ int _type = HW_BREAKPOINT_RW;
+ unsigned long _addr = 0;
+ int _offset = 0;
+ char *at, *col;
+
+ _spec = kstrdup(spec, GFP_KERNEL);
+ if (!_spec)
+ return -ENOMEM;
+
+ at = strchr(_spec, '@');
+ col = strchr(_spec, ':');
+
+ if (!at) {
+ trace_probe_log_err(0, BAD_ACCESS_FMT);
+ return -EINVAL;
+ }
+
+ if (at != _spec) {
+ *at = '\0';
+
+ if (strcmp(_spec, "r") == 0)
+ _type = HW_BREAKPOINT_R;
+ else if (strcmp(_spec, "w") == 0)
+ _type = HW_BREAKPOINT_W;
+ else if (strcmp(_spec, "rw") == 0)
+ _type = HW_BREAKPOINT_RW;
+ else {
+ trace_probe_log_err(0, BAD_ACCESS_TYPE);
+ return -EINVAL;
+ }
+ }
+
+ if (col) {
+ *col = '\0';
+ if (kstrtoint(col + 1, 0, &_len)) {
+ trace_probe_log_err(col + 1 - _spec, BAD_ACCESS_LEN);
+ return -EINVAL;
+ }
+
+ switch (_len) {
+ case 1:
+ _len = HW_BREAKPOINT_LEN_1;
+ break;
+ case 2:
+ _len = HW_BREAKPOINT_LEN_2;
+ break;
+ case 4:
+ _len = HW_BREAKPOINT_LEN_4;
+ break;
+ case 8:
+ _len = HW_BREAKPOINT_LEN_8;
+ break;
+ default:
+ trace_probe_log_err(col + 1 - _spec, BAD_ACCESS_LEN);
+ return -EINVAL;
+ }
+ }
+
+ if (kstrtoul(at + 1, 0, &_addr) != 0) {
+ char *off_str = strpbrk(at + 1, "+-");
+
+ if (off_str) {
+ if (kstrtoint(off_str, 0, &_offset) != 0) {
+ trace_probe_log_err(off_str - _spec, BAD_PROBE_ADDR);
+ return -EINVAL;
+ }
+ *off_str = '\0';
+ }
+ _addr = kallsyms_lookup_name(at + 1);
+ if (!_addr) {
+ trace_probe_log_err(at + 1 - _spec, BAD_ACCESS_ADDR);
+ return -ENOENT;
+ }
+ _addr += _offset;
+ *symbol = kstrdup(at + 1, GFP_KERNEL);
+ if (!*symbol)
+ return -ENOMEM;
+ }
+
+ if (!trace_wprobe_is_valid_addr(_addr, _len)) {
+ trace_probe_log_err(at + 1 - _spec, BAD_ACCESS_ADDR);
+ return -EINVAL;
+ }
+
+ *addr = _addr;
+ *offset = _offset;
+ *type = _type;
+ *len = _len;
+ return 0;
+}
+
+static int __trace_wprobe_create(int argc, const char *argv[])
+{
+ /*
+ * Argument syntax:
+ * w[:[GRP/][EVENT]] SPEC
+ *
+ * SPEC:
+ * [r|w|rw]@[ADDR|SYMBOL[+OFFS]][:LEN]
+ */
+ struct traceprobe_parse_context *ctx __free(traceprobe_parse_context) = NULL;
+ struct trace_wprobe *tw __free(free_trace_wprobe) = NULL;
+ const char *event = NULL, *group = WPROBE_EVENT_SYSTEM;
+ const char *tplog __free(trace_probe_log_clear) = NULL;
+ char *symbol __free(kfree) = NULL;
+ char *gbuf __free(kfree) = NULL;
+ char *ebuf __free(kfree) = NULL;
+ unsigned long addr;
+ int len, type, offset, i;
+ int ret;
+
+ if (argv[0][0] != 'w')
+ return -ECANCELED;
+
+ tplog = trace_probe_log_init("wprobe", argc, argv);
+
+ if (argc < 2) {
+ trace_probe_log_set_index(0);
+ trace_probe_log_err(0, NO_ARG_BODY);
+ return -EINVAL;
+ }
+
+ if (argv[0][1] != '\0') {
+ if (argv[0][1] != ':') {
+ trace_probe_log_set_index(0);
+ trace_probe_log_err(1, WPROBE_NO_MAXACT);
+ return -EINVAL;
+ }
+ event = &argv[0][2];
+ }
+
+ trace_probe_log_set_index(1);
+ ret = parse_address_spec(argv[1], &addr, &type, &len, &symbol, &offset);
+ if (ret < 0)
+ return ret;
+
+ trace_probe_log_set_index(0);
+ if (event) {
+ gbuf = kmalloc(MAX_EVENT_NAME_LEN, GFP_KERNEL);
+ if (!gbuf)
+ return -ENOMEM;
+ ret = traceprobe_parse_event_name(&event, &group, gbuf,
+ event - argv[0]);
+ if (ret)
+ return ret;
+ }
+
+ if (!event) {
+ /* Make a new event name */
+ ebuf = kmalloc(MAX_EVENT_NAME_LEN, GFP_KERNEL);
+ if (!ebuf)
+ return -ENOMEM;
+ if (symbol)
+ snprintf(ebuf, MAX_EVENT_NAME_LEN, "%s", symbol);
+ else
+ snprintf(ebuf, MAX_EVENT_NAME_LEN, "w_0x%lx", addr);
+ sanitize_event_name(ebuf);
+ event = ebuf;
+ }
+
+ argc -= 2; argv += 2;
+ if (argc > MAX_TRACE_ARGS) {
+ trace_probe_log_set_index(2);
+ trace_probe_log_err(0, TOO_MANY_ARGS);
+ return -E2BIG;
+ }
+ tw = alloc_trace_wprobe(group, event, symbol, offset, addr, len, type, argc);
+ if (IS_ERR(tw))
+ return PTR_ERR(tw);
+
+ ctx = kzalloc_obj(*ctx);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->flags = TPARG_FL_KERNEL | TPARG_FL_WPROBE;
+
+ /* parse arguments */
+ for (i = 0; i < argc; i++) {
+ trace_probe_log_set_index(i + 2);
+ ctx->offset = 0;
+ ret = traceprobe_parse_probe_arg(&tw->tp, i, argv[i], ctx);
+ if (ret)
+ return ret; /* This can be -ENOMEM */
+ }
+
+ ret = traceprobe_set_print_fmt(&tw->tp, PROBE_PRINT_NORMAL);
+ if (ret < 0)
+ return ret;
+
+ ret = register_trace_wprobe_event(tw);
+ if (!ret)
+ tw = NULL; /* To avoid free */
+
+ return ret;
+}
+
+static int trace_wprobe_create(const char *raw_command)
+{
+ return trace_probe_create(raw_command, __trace_wprobe_create);
+}
+
+static int trace_wprobe_release(struct dyn_event *ev)
+{
+ struct trace_wprobe *tw = to_trace_wprobe(ev);
+ int ret = unregister_trace_wprobe(tw);
+
+ if (!ret)
+ free_trace_wprobe(tw);
+ return ret;
+}
+
+static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev)
+{
+ struct trace_wprobe *tw = to_trace_wprobe(ev);
+ const char *type_str;
+ int i, len;
+
+ seq_printf(m, "w:%s/%s", trace_probe_group_name(&tw->tp),
+ trace_probe_name(&tw->tp));
+
+ if (tw->type == HW_BREAKPOINT_R)
+ type_str = "r";
+ else if (tw->type == HW_BREAKPOINT_W)
+ type_str = "w";
+ else
+ type_str = "rw";
+
+ if (tw->len == HW_BREAKPOINT_LEN_1)
+ len = 1;
+ else if (tw->len == HW_BREAKPOINT_LEN_2)
+ len = 2;
+ else if (tw->len == HW_BREAKPOINT_LEN_4)
+ len = 4;
+ else
+ len = 8;
+
+ if (tw->symbol) {
+ if (tw->offset)
+ seq_printf(m, " %s@%s%+d:%d", type_str, tw->symbol,
+ tw->offset, len);
+ else
+ seq_printf(m, " %s@%s:%d", type_str, tw->symbol, len);
+ } else {
+ seq_printf(m, " %s@0x%lx:%d", type_str, tw->addr, len);
+ }
+
+ for (i = 0; i < tw->tp.nr_args; i++)
+ seq_printf(m, " %s=%s", tw->tp.args[i].name, tw->tp.args[i].comm);
+ seq_putc(m, '\n');
+
+ return 0;
+}
+
+static __init int init_wprobe_trace(void)
+{
+ return dyn_event_register(&trace_wprobe_ops);
+}
+fs_initcall(init_wprobe_trace);
+
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v18 05/11] x86: hw_breakpoint: Add a kconfig to clarify when a breakpoint fires
2026-09-25 11:10 [PATCH v18 00/11] tracing: wprobe: x86: Add wprobe for watchpoint Masami Hiramatsu (Google)
` (2 preceding siblings ...)
2026-09-25 11:11 ` [PATCH v18 04/11] tracing/wprobe: Add wprobe (watchpoint probe) trace event support Masami Hiramatsu (Google)
@ 2026-09-25 11:12 ` Masami Hiramatsu (Google)
2026-09-25 11:12 ` [PATCH v18 06/11] selftests: tracing: Add a basic testcase for wprobe Masami Hiramatsu (Google)
` (5 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-09-25 11:12 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users, x86
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Add CONFIG_HAVE_POST_BREAKPOINT_HOOK which indicates the hw_breakpoint
on that architecture fires after the target memory has been modified.
This is currently x86 only behavior.
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Reviewed-by: Jinchao Wang <wangjinchao600@gmail.com>
---
Changes in v11:
- Move select to alphabetically sorted place.
---
arch/Kconfig | 10 ++++++++++
arch/x86/Kconfig | 1 +
kernel/trace/Kconfig | 1 +
3 files changed, 12 insertions(+)
diff --git a/arch/Kconfig b/arch/Kconfig
index d021bc31ef14..5292bd05d5d4 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -474,6 +474,16 @@ config HAVE_MIXED_BREAKPOINTS_REGS
Select this option if your arch implements breakpoints under the
latter fashion.
+config HAVE_POST_BREAKPOINT_HOOK
+ bool
+ depends on HAVE_HW_BREAKPOINT
+ help
+ Depending on the arch implementation of hardware breakpoints,
+ some of them provide breakpoint hook after the target memory
+ is modified.
+ Select this option if your arch implements breakpoints overflow
+ handler hooks after the target memory is modified.
+
config HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
bool
depends on HAVE_HW_BREAKPOINT
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 38e29e465022..e4eaf0b607d3 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -289,6 +289,7 @@ config X86
select MMU_GATHER_RCU_TABLE_FREE
select MMU_GATHER_MERGE_VMAS
select HAVE_POSIX_CPU_TIMERS_TASK_WORK
+ select HAVE_POST_BREAKPOINT_HOOK
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_RELIABLE_STACKTRACE if UNWINDER_ORC || STACK_VALIDATION
select HAVE_FUNCTION_ARG_ACCESS_API
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index b58c2565024f..d9b6fa5c35d9 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -866,6 +866,7 @@ config WPROBE_EVENTS
bool "Enable wprobe-based dynamic events"
depends on TRACING
depends on HAVE_HW_BREAKPOINT
+ depends on HAVE_POST_BREAKPOINT_HOOK
select PROBE_EVENTS
select DYNAMIC_EVENTS
help
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v18 06/11] selftests: tracing: Add a basic testcase for wprobe
2026-09-25 11:10 [PATCH v18 00/11] tracing: wprobe: x86: Add wprobe for watchpoint Masami Hiramatsu (Google)
` (3 preceding siblings ...)
2026-09-25 11:12 ` [PATCH v18 05/11] x86: hw_breakpoint: Add a kconfig to clarify when a breakpoint fires Masami Hiramatsu (Google)
@ 2026-09-25 11:12 ` Masami Hiramatsu (Google)
2026-09-25 11:12 ` [PATCH v18 07/11] selftests: tracing: Add syntax " Masami Hiramatsu (Google)
` (4 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-09-25 11:12 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users, x86
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Add 'add_remove_wprobe.tc' testcase for testing wprobe event that
tests adding and removing operations of the wprobe event.
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v17:
- Catch failures when enabling wprobe and call exit_unresolved
in case hardware watchpoints are exhausted.
Changes in v9:
- Fix command check logic to prevent early exit under 'set -e'
(errexit) when grep or test fails.
- Simplify enable/disable status checks by removing cat pipes.
Changes in v8:
- Fixed silently test failure path.
---
tools/testing/selftests/ftrace/config | 1
.../ftrace/test.d/dynevent/add_remove_wprobe.tc | 66 ++++++++++++++++++++
2 files changed, 67 insertions(+)
create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc
diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftests/ftrace/config
index 544de0db5f58..d2f503722020 100644
--- a/tools/testing/selftests/ftrace/config
+++ b/tools/testing/selftests/ftrace/config
@@ -27,3 +27,4 @@ CONFIG_STACK_TRACER=y
CONFIG_TRACER_SNAPSHOT=y
CONFIG_UPROBES=y
CONFIG_UPROBE_EVENTS=y
+CONFIG_WPROBE_EVENTS=y
diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc
new file mode 100644
index 000000000000..3c622fb07e7f
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc
@@ -0,0 +1,66 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Generic dynamic event - add/remove wprobe events
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README
+
+echo 0 > events/enable
+echo > dynamic_events
+
+# Use jiffies as a variable that is frequently written to.
+TARGET=jiffies
+
+echo "w:my_wprobe w@$TARGET" >> dynamic_events
+
+if ! grep -q my_wprobe dynamic_events; then
+ echo "Failed to create wprobe event"
+ exit_fail
+fi
+
+if [ ! -d events/wprobes/my_wprobe ]; then
+ echo "Failed to create wprobe event directory"
+ exit_fail
+fi
+
+if ! echo 1 > events/wprobes/my_wprobe/enable; then
+ echo "Failed to enable wprobe (hardware watchpoints may be exhausted)"
+ exit_unresolved
+fi
+
+# Check if the event is enabled
+if ! grep -q 1 events/wprobes/my_wprobe/enable; then
+ echo "Failed to enable wprobe event"
+ exit_fail
+fi
+
+# Let some time pass to trigger the breakpoint
+sleep 1
+
+# Check if we got any trace output
+if ! grep -q my_wprobe trace; then
+ echo "wprobe event was not triggered"
+ exit_fail
+fi
+
+echo 0 > events/wprobes/my_wprobe/enable
+
+# Check if the event is disabled
+if ! grep -q 0 events/wprobes/my_wprobe/enable; then
+ echo "Failed to disable wprobe event"
+ exit_fail
+fi
+
+echo "-:my_wprobe" >> dynamic_events
+
+if grep -q my_wprobe dynamic_events; then
+ echo "Failed to remove wprobe event"
+ exit_fail
+fi
+
+if [ -d events/wprobes/my_wprobe ]; then
+ echo "Failed to remove wprobe event directory"
+ exit_fail
+fi
+
+clear_trace
+
+exit 0
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v18 07/11] selftests: tracing: Add syntax testcase for wprobe
2026-09-25 11:10 [PATCH v18 00/11] tracing: wprobe: x86: Add wprobe for watchpoint Masami Hiramatsu (Google)
` (4 preceding siblings ...)
2026-09-25 11:12 ` [PATCH v18 06/11] selftests: tracing: Add a basic testcase for wprobe Masami Hiramatsu (Google)
@ 2026-09-25 11:12 ` Masami Hiramatsu (Google)
2026-09-25 11:12 ` [PATCH v18 08/11] tracing/wprobe: Add set_wprobe and clear_wprobe event triggers Masami Hiramatsu (Google)
` (3 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-09-25 11:12 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users, x86
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Add "wprobe_syntax_errors.tc" testcase for testing syntax errors
of the watch probe events.
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v14:
- Update comment for invalid offset syntax check to BAD_PROBE_ADDR.
- Add testcase for invalid address 0 error check (BAD_ACCESS_ADDR).
Changes in v12:
- Add NO_ARG_BODY error case.
Changes in v11:
- Add WPROBE_NO_MAXACT syntax error case.
---
.../test.d/dynevent/wprobes_syntax_errors.tc | 23 ++++++++++++++++++++
1 file changed, 23 insertions(+)
create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc
diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc b/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc
new file mode 100644
index 000000000000..e0df58fe7aa3
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc
@@ -0,0 +1,23 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Watch probe event parser error log check
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README
+
+check_error() { # command-with-error-pos-by-^
+ ftrace_errlog_check 'wprobe' "$1" 'dynamic_events'
+}
+
+check_error '^w' # NO_ARG_BODY
+check_error 'w^10 w@jiffies' # WPROBE_NO_MAXACT
+check_error 'w ^symbol' # BAD_ACCESS_FMT
+check_error 'w ^a@symbol' # BAD_ACCESS_TYPE
+check_error 'w w@^symbol' # BAD_ACCESS_ADDR
+check_error 'w w@^0:4' # BAD_ACCESS_ADDR
+check_error 'w w@jiffies^+offset' # BAD_PROBE_ADDR
+check_error 'w w@jiffies:^100' # BAD_ACCESS_LEN
+check_error 'w w@jiffies ^$arg1' # BAD_VAR
+check_error 'w w@jiffies ^$retval' # BAD_VAR
+check_error 'w w@jiffies ^$stack' # BAD_VAR
+check_error 'w w@jiffies ^%ax' # BAD_VAR
+
+exit 0
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v18 08/11] tracing/wprobe: Add set_wprobe and clear_wprobe event triggers
2026-09-25 11:10 [PATCH v18 00/11] tracing: wprobe: x86: Add wprobe for watchpoint Masami Hiramatsu (Google)
` (5 preceding siblings ...)
2026-09-25 11:12 ` [PATCH v18 07/11] selftests: tracing: Add syntax " Masami Hiramatsu (Google)
@ 2026-09-25 11:12 ` Masami Hiramatsu (Google)
2026-09-25 11:13 ` [PATCH v18 09/11] selftests: tracing: Add wprobe trigger testcases Masami Hiramatsu (Google)
` (2 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-09-25 11:12 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users, x86
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Add set_wprobe and clear_wprobe event triggers to dynamically attach
and detach hardware breakpoint address monitoring based on event field
contents.
Link: https://lore.kernel.org/all/59637b96946653393a7ad3c7de094094796b39c2.1785067572.git.wangjinchao600@gmail.com/
Assisted-by: LLM
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v18:
- Use register_wide_hw_breakpoint_cpuslocked() under cpus_read_lock()
in __register_trace_wprobe() to avoid CPU hotplug race and lockdep
recursive locking warnings.
- Return early on in_nmi() in wprobe_trigger() before taking tw->lock
to prevent NMI deadlock.
- Always evaluate __wprobe_count_func() under tw->lock in
wprobe_trigger() to fix TOCTOU race for counted triggers.
- Drop redundant in_nmi() check from __wprobe_count_func().
- Defer data->count decrement to wprobe_trigger() under tw->lock after
successful hardware update to avoid consuming count on missed triggers
and prevent counter underflow.
Changes in v17:
- Add dynamic CPU hotplug callbacks to handle CPU offline/online.
- Restore EVENT_FILE_FL_SOFT_DISABLED_BIT in wprobe_trigger_free().
- Check whether trigger event has "unlimited" field before skipping
"unlimited" in clear_wprobe trigger parsing.
- Call tracepoint_synchronize_unregister() before freeing trigger
data via data->cmd_ops->free() on registration failure.
- Implement __wprobe_count_func()/wprobe_count_func() helpers to
consolidate precondition checks and use it for count_func() and
inside wprobe_trigger() function.
Changes in v16:
- Immediately update the hardware breakpoint on the local CPU via
trace_wprobe_update_local() in wprobe_trigger() to avoid workqueue
latency, and only kick irq_work for other CPUs.
- Skip redundant updates in trace_wprobe_update_local() if
bp->attr.bp_addr already matches the target address.
- Use strcmp() instead of strncmp() for "unlimited" to avoid
misinterpreting field names starting with "unlimited" as count.
- Normalize "unlimited" and "count=" parameters before creating
trigger data so trigger parsing handles count keywords correctly.
Changes in v15:
- Use trace_wprobe_is_valid_addr() in wprobe_trigger() to validate
address and natural alignment for wprobe length.
- Record SMP local update failures in tw->missed counter.
- Expose missed event count as a trailing comment in trigger file.
- Add set_wprobe and clear_wprobe triggers to readme_msg in trace.c.
- Fix line length exceeding 100 columns in wprobe_trigger_log_err
macro.
- Initialize irq_work and work structs immediately after allocation
in alloc_trace_wprobe() and check tw->work.func in
free_trace_wprobe() to prevent WARN_ON(!work->func) on early
allocation failure.
Changes in v14:
- Allow '-1' as a dummy address in parse_address_spec() to set initial
address to WPROBE_DEFAULT_CLEAR_ADDRESS for trigger-only wprobes.
- In trace_wprobe_show(), format WPROBE_DEFAULT_CLEAR_ADDRESS as '-1'.
- Update documentation and examples to use '-1' instead of '0'.
- Skip field parsing in clear_wprobe if parameter is a count keyword.
- Set tw->addr to WPROBE_DEFAULT_CLEAR_ADDRESS before soft-enabling
event and roll back on failure.
- Fix memory and refcount leak of trigger_data/wprobe_data on error
path.
Changes in v13:
- Align WPROBE_DEFAULT_CLEAR_ADDRESS to 8 bytes using dedicated u64.
- Remove manual count decrement to rely on core event_trigger_count().
- Parse optional FIELD argument in clear_wprobe command.
- Fix trigger reference count leak and error cleanup in command
parsing.
- Update document to add [:COUNT] to trigger syntax and fix adjust
forms.
- Add notes about the exclusive trigger setting behavior.
- Remove unused dyn_event_ops_mutex.
Changes in v12:
- Decrement trigger data->count only when watchpoint state is actually
changed.
- Remove dyn_event_ops_mutex to fix lockdep circular dependency
deadlock.
- Add event_trigger_init() call to prevent premature freeing of
trigger_data.
---
Documentation/trace/wprobetrace.rst | 107 +++++
include/linux/trace_events.h | 1
kernel/trace/Kconfig | 10
kernel/trace/trace.c | 4
kernel/trace/trace.h | 1
kernel/trace/trace_events_trigger.c | 2
kernel/trace/trace_probe.c | 2
kernel/trace/trace_probe.h | 9
kernel/trace/trace_wprobe.c | 788 +++++++++++++++++++++++++++++++++++
9 files changed, 919 insertions(+), 5 deletions(-)
diff --git a/Documentation/trace/wprobetrace.rst b/Documentation/trace/wprobetrace.rst
index 31cc68b464e9..df8985283312 100644
--- a/Documentation/trace/wprobetrace.rst
+++ b/Documentation/trace/wprobetrace.rst
@@ -76,3 +76,110 @@ Wprobe event does not disable itself even if the module is unloaded.
For example, if you add a wprobe event on a module variable, and then
unload the module, the wprobe event will still be enabled. This is for
watching the address is used unexpectedly after the module is unloaded.
+
+Combination with trigger action
+-------------------------------
+The event trigger action can extend the utilization of this wprobe.
+
+- set_wprobe:WPEVENT:FIELD[+|-ADJUST][:COUNT]
+- clear_wprobe:WPEVENT[:FIELD[+|-ADJUST][:COUNT]]
+
+Set these triggers to the target event, then the WPROBE event will be
+setup to trace the memory access at FIELD[+|-ADJUST] address.
+When clear_wprobe is hit, if FIELD is NOT specified, the WPEVENT is
+forcibly cleared. If FIELD[+|-ADJUST] is set, it clears WPEVENT only
+if its watching address is the same as the FIELD[+|-ADJUST] value.
+If COUNT is specified, it will set/clear WPEVENT only if it hits COUNT
+times.
+
+Notes:
+- set_wprobe only works on the wprobe which is NOT set a valid address yet,
+ and it must be enabled after the trigger is set.
+- clear_wprobe only works on the wprobe which is set a valid address, and it
+ will be soft-disabled after the trigger is cleared.
+- Therefore, if a trigger sets/clears a wprobe, other/same trigger events
+ will not work (on the same event) while the wprobe is set.
+
+The set_wprobe trigger does not change the type and length, these
+must be set when creating a new wprobe.
+
+The WPROBE event must be disabled when setting the new trigger
+and it will be busy afterwards. Recommended usage is to add a new
+wprobe at invalid dummy address (-1) and keep disabled.
+
+Wprobe triggers only support target addresses in kernel memory. If a
+set_wprobe trigger evaluates to a user-space memory address or NULL
+pointer, the trigger action ignores the update and skips setting the
+watchpoint.
+
+Wprobe triggers are not supported on kprobe_events, because kprobes
+themselves can use software breakpoints which conflicts with wprobe
+operation.
+
+
+For example, trace the first 8 bytes of the dentry data structure passed
+to do_truncate() until it is deleted by dentry_kill().
+(Note: all tracefs setup uses '>>' so that it does not kick do_truncate())
+::
+
+ # echo 'w:watch rw@-1:8 address=$addr value=+0($addr)' >> dynamic_events
+ # echo 'f:truncate do_truncate dentry=$arg2' >> dynamic_events
+ # echo 'set_wprobe:watch:dentry' >> events/fprobes/truncate/trigger
+ # echo 'f:dentry_kill dentry_kill dentry=$arg1' >> dynamic_events
+ # echo 'clear_wprobe:watch:dentry' >> events/fprobes/dentry_kill/trigger
+ # echo 1 >> events/fprobes/truncate/enable
+ # echo 1 >> events/fprobes/dentry_kill/enable
+
+ # echo aaa > /tmp/hoge
+ # echo bbb > /tmp/hoge
+ # echo ccc > /tmp/hoge
+ # rm /tmp/hoge
+
+Then, the trace data will show::
+
+ # tracer: nop
+ #
+ # entries-in-buffer/entries-written: 32/32 #P:8
+ #
+ # _-----=> irqs-off/BH-disabled
+ # / _----=> need-resched
+ # | / _---=> hardirq/softirq
+ # || / _--=> preempt-depth
+ # ||| / _-=> migrate-disable
+ # |||| / delay
+ # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
+ # | | | ||||| | |
+ sh-107 [004] ...1. 9.990418: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+ sh-107 [004] ...1. 9.990914: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b3de78
+ sh-107 [004] ...1. 9.993175: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049ddd40
+ sh-107 [004] ..... 9.995198: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
+ sh-107 [004] ...1. 9.995389: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db998
+ sh-107 [004] ..Zff 9.997503: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..Zff 9.997509: watch: (path_openat+0x211/0xda0) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..Zff 9.997514: watch: (path_openat+0xa56/0xda0) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..Zff 9.997518: watch: (path_openat+0xae2/0xda0) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..... 9.997521: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
+ sh-107 [004] ...1. 9.997582: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004808270
+ sh-107 [004] ...1. 9.999365: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db728
+ sh-107 [004] ...1. 9.999388: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b1c000
+ rm-113 [005] ..Zff 10.000965: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.000971: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.000984: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.000988: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001010: watch: (lookup_one_qstr_excl+0x28/0x140) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001014: watch: (lookup_one_qstr_excl+0xd1/0x140) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001018: watch: (may_delete_dentry+0x1c/0x200) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001021: watch: (may_delete_dentry+0x195/0x200) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001031: watch: (vfs_unlink+0x5e/0x260) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] d.Z.. 10.001067: watch: (d_make_discardable+0x1b/0x40) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] d.Z.. 10.001071: watch: (d_make_discardable+0x29/0x40) address=0xffff8880048083a8 value=0x200080
+ rm-113 [005] ...1. 10.001072: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
+ rm-113 [005] ...1. 10.001218: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
+ sh-107 [004] ...1. 10.001416: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db110
+ sh-107 [004] ...1. 10.001444: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db248
+ sh-107 [004] ...1. 10.001500: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+ sh-107 [004] ...1. 10.002067: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
+ sh-107 [004] ...1. 10.904920: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
+ sh-107 [004] ...1. 10.905129: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+
+You can see the watch event is correctly configured on the dentry.
diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
index 43ffd9a76d88..f81a5308c116 100644
--- a/include/linux/trace_events.h
+++ b/include/linux/trace_events.h
@@ -738,6 +738,7 @@ enum event_trigger_type {
ETT_EVENT_HIST = (1 << 4),
ETT_HIST_ENABLE = (1 << 5),
ETT_EVENT_EPROBE = (1 << 6),
+ ETT_EVENT_WPROBE = (1 << 7),
};
extern int filter_match_preds(struct event_filter *filter, void *rec);
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index d9b6fa5c35d9..5fd8ed63c516 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -876,6 +876,16 @@ config WPROBE_EVENTS
Those events can be inserted wherever hardware breakpoints can be
set, and record accessed memory address and values.
+config WPROBE_TRIGGERS
+ depends on WPROBE_EVENTS
+ depends on HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
+ bool
+ default y
+ help
+ This adds an event trigger which will set the wprobe on a specific
+ field of an event. This allows user to trace the memory access of
+ an address pointed by the event field.
+
config BPF_EVENTS
depends on BPF_SYSCALL
depends on (KPROBE_EVENTS || UPROBE_EVENTS) && PERF_EVENTS
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index da85e2568c41..61d7d8bfb157 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4362,6 +4362,10 @@ static const char readme_msg[] =
"\t trigger: traceon, traceoff\n"
"\t enable_event:<system>:<event>\n"
"\t disable_event:<system>:<event>\n"
+#ifdef CONFIG_WPROBE_TRIGGERS
+ "\t set_wprobe:<wprobe-event>:<field>[+|-<offset>]\n"
+ "\t clear_wprobe:<wprobe-event>[:<field>[+|-<offset>]]\n"
+#endif
#ifdef CONFIG_HIST_TRIGGERS
"\t enable_hist:<system>:<event>\n"
"\t disable_hist:<system>:<event>\n"
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 18588c550a98..a61f840b18bd 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -1983,6 +1983,7 @@ trigger_data_alloc(struct event_command *cmd_ops, char *cmd, char *param,
void *private_data);
extern void trigger_data_free(struct event_trigger_data *data);
extern int event_trigger_init(struct event_trigger_data *data);
+extern void event_trigger_free(struct event_trigger_data *data);
extern int trace_event_trigger_enable_disable(struct trace_event_file *file,
int trigger_enable);
extern void update_cond_flag(struct trace_event_file *file);
diff --git a/kernel/trace/trace_events_trigger.c b/kernel/trace/trace_events_trigger.c
index 149300cc5e8a..120e6ec634b7 100644
--- a/kernel/trace/trace_events_trigger.c
+++ b/kernel/trace/trace_events_trigger.c
@@ -589,7 +589,7 @@ int event_trigger_init(struct event_trigger_data *data)
* Usually used directly as the @free method in event trigger
* implementations.
*/
-static void
+void
event_trigger_free(struct event_trigger_data *data)
{
if (WARN_ON_ONCE(data->ref <= 0))
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index 64e7e363e8e2..77266e042418 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -20,7 +20,7 @@
#undef C
#define C(a, b) b
-static const char *trace_probe_err_text[] = { ERRORS };
+const char *trace_probe_err_text[] = { ERRORS };
static const char *reserved_field_names[] = {
"common_type",
diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h
index f20cafb87cc6..1e11077ead67 100644
--- a/kernel/trace/trace_probe.h
+++ b/kernel/trace/trace_probe.h
@@ -636,7 +636,12 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
C(TYPECAST_SYM_OFFSET, "@SYM+/-OFFSET with typecast needs parentheses"), \
C(USED_ARG_NAME, "This argument name is already used"), \
C(WPROBE_NO_MAXACT, "Watchpoint probe does not support maxactive"), \
- C(WPROBE_NO_SIBLING, "Watchpoint probe does not support sibling probes"),
+ C(WPROBE_NO_SIBLING, "Watchpoint probe does not support sibling probes"), \
+ C(WPROBE_ON_KPROBE, "Wprobe trigger is not supported on kprobe event"), \
+ C(WPROBE_NOT_FOUND, "Target wprobe event is not found"), \
+ C(WPROBE_BUSY, "Target wprobe event is already enabled"), \
+ C(WPROBE_NEED_FIELD, "Wprobe trigger requires a target field"), \
+ C(WPROBE_BAD_FIELD, "Target field must be pointer size"),
#undef C
#define C(a, b) TP_ERR_##a
@@ -660,6 +665,8 @@ void __trace_probe_log_err(int offset, int err);
DEFINE_FREE(trace_probe_log_clear, const char *, if (_T) trace_probe_log_clear())
+extern const char *trace_probe_err_text[];
+
#define trace_probe_log_err(offs, err) \
__trace_probe_log_err(offs, TP_ERR_##err)
diff --git a/kernel/trace/trace_wprobe.c b/kernel/trace/trace_wprobe.c
index 9c6100b63e8d..31de7dd5bfd9 100644
--- a/kernel/trace/trace_wprobe.c
+++ b/kernel/trace/trace_wprobe.c
@@ -6,7 +6,11 @@
*/
#define pr_fmt(fmt) "trace_wprobe: " fmt
+#include <linux/atomic.h>
#include <linux/compiler.h>
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/errno.h>
#include <linux/hw_breakpoint.h>
#include <linux/kallsyms.h>
#include <linux/list.h>
@@ -15,11 +19,16 @@
#include <linux/perf_event.h>
#include <linux/rculist.h>
#include <linux/security.h>
+#include <linux/spinlock.h>
#include <linux/tracepoint.h>
#include <linux/uaccess.h>
+#include <linux/workqueue.h>
+#include <linux/irq_work.h>
+#include <linux/preempt.h>
#include <asm/ptrace.h>
+#include "trace.h"
#include "trace_dynevent.h"
#include "trace_probe.h"
#include "trace_probe_kernel.h"
@@ -45,12 +54,24 @@ static struct dyn_event_operations trace_wprobe_ops = {
struct trace_wprobe {
struct dyn_event devent;
+ struct list_head active_list;
struct perf_event * __percpu *bp_event;
unsigned long addr;
int offset;
int len;
int type;
const char *symbol;
+ raw_spinlock_t lock;
+ struct irq_work irq_work;
+ struct work_struct work;
+ atomic_t missed;
+ /*
+ * work_pending is set to 1 before irq_work_queue() and cleared to 0
+ * after wprobe_work_func() finishes on_each_cpu(). This prevents a
+ * new trigger from overwriting tw->addr while the work is propagating
+ * the old address to per-CPU debug registers via IPI.
+ */
+ atomic_t work_pending;
struct trace_probe tp;
};
@@ -64,6 +85,9 @@ static struct trace_wprobe *to_trace_wprobe(struct dyn_event *ev)
return container_of(ev, struct trace_wprobe, devent);
}
+static DEFINE_MUTEX(wprobe_mutex);
+static LIST_HEAD(active_wprobes);
+
#define for_each_trace_wprobe(pos, dpos) \
for_each_dyn_event(dpos) \
if (is_trace_wprobe(dpos) && (pos = to_trace_wprobe(dpos)))
@@ -185,28 +209,143 @@ static int __register_trace_wprobe(struct trace_wprobe *tw)
attr.bp_len = tw->len;
attr.bp_type = tw->type;
- tw->bp_event = register_wide_hw_breakpoint(&attr, wprobe_perf_handler, tw);
+ cpus_read_lock();
+ tw->bp_event = register_wide_hw_breakpoint_cpuslocked(&attr,
+ wprobe_perf_handler,
+ tw);
if (IS_ERR_PCPU(tw->bp_event)) {
int ret = PTR_ERR_PCPU(tw->bp_event);
tw->bp_event = NULL;
+ cpus_read_unlock();
return ret;
}
+ mutex_lock(&wprobe_mutex);
+ list_add(&tw->active_list, &active_wprobes);
+ mutex_unlock(&wprobe_mutex);
+ cpus_read_unlock();
+
return 0;
}
static void __unregister_trace_wprobe(struct trace_wprobe *tw)
{
if (tw->bp_event) {
+ mutex_lock(&wprobe_mutex);
+ list_del_init(&tw->active_list);
+ mutex_unlock(&wprobe_mutex);
+
+ irq_work_sync(&tw->irq_work);
+ cancel_work_sync(&tw->work);
unregister_wide_hw_breakpoint(tw->bp_event);
tw->bp_event = NULL;
}
}
+static int wprobe_cpu_online(unsigned int cpu)
+{
+ struct trace_wprobe *tw;
+
+ mutex_lock(&wprobe_mutex);
+ list_for_each_entry(tw, &active_wprobes, active_list) {
+ if (tw->bp_event && !per_cpu(*tw->bp_event, cpu)) {
+ struct perf_event_attr attr;
+ struct perf_event *bp;
+
+ hw_breakpoint_init(&attr);
+ attr.bp_addr = READ_ONCE(tw->addr);
+ attr.bp_len = tw->len;
+ attr.bp_type = tw->type;
+
+ bp = perf_event_create_kernel_counter(&attr, cpu, NULL,
+ wprobe_perf_handler, tw);
+ if (!IS_ERR(bp))
+ per_cpu(*tw->bp_event, cpu) = bp;
+ }
+ }
+ mutex_unlock(&wprobe_mutex);
+
+ return 0;
+}
+
+static int wprobe_cpu_offline(unsigned int cpu)
+{
+ struct trace_wprobe *tw;
+
+ mutex_lock(&wprobe_mutex);
+ list_for_each_entry(tw, &active_wprobes, active_list) {
+ if (tw->bp_event) {
+ struct perf_event *bp = per_cpu(*tw->bp_event, cpu);
+
+ if (bp) {
+ per_cpu(*tw->bp_event, cpu) = NULL;
+ unregister_hw_breakpoint(bp);
+ }
+ }
+ }
+ mutex_unlock(&wprobe_mutex);
+
+ return 0;
+}
+
+static int trace_wprobe_update_local(struct trace_wprobe *tw, unsigned long addr)
+{
+ struct perf_event * __percpu *pevent;
+ struct perf_event *bp;
+
+ pevent = tw->bp_event;
+ if (!pevent)
+ return -EINVAL;
+
+ bp = *this_cpu_ptr(pevent);
+ if (!bp)
+ return -EINVAL;
+
+ if (bp->attr.bp_addr == addr)
+ return 0;
+
+ return modify_local_hw_breakpoint_addr(bp, addr);
+}
+
+static void wprobe_smp_update_func(void *info)
+{
+ struct trace_wprobe *tw = info;
+ unsigned long addr = READ_ONCE(tw->addr);
+
+ if (trace_wprobe_update_local(tw, addr))
+ atomic_inc(&tw->missed);
+}
+
+static void wprobe_work_func(struct work_struct *work)
+{
+ struct trace_wprobe *tw = container_of(work, struct trace_wprobe, work);
+
+ cpus_read_lock();
+ on_each_cpu(wprobe_smp_update_func, tw, true);
+ cpus_read_unlock();
+ /*
+ * Clear work_pending after all CPUs have updated their local debug
+ * registers. A new trigger may now update tw->addr and queue a new
+ * irq_work.
+ */
+ atomic_set(&tw->work_pending, 0);
+}
+
+static void wprobe_irq_work_func(struct irq_work *irq_work)
+{
+ struct trace_wprobe *tw = container_of(irq_work, struct trace_wprobe, irq_work);
+
+ schedule_work(&tw->work);
+}
+
static void free_trace_wprobe(struct trace_wprobe *tw)
{
if (tw) {
+ if (tw->work.func) {
+ irq_work_sync(&tw->irq_work);
+ cancel_work_sync(&tw->work);
+ }
trace_probe_cleanup(&tw->tp);
kfree(tw->symbol);
kfree(tw);
@@ -231,6 +370,13 @@ static struct trace_wprobe *alloc_trace_wprobe(const char *group,
if (!tw)
return ERR_PTR(-ENOMEM);
+ raw_spin_lock_init(&tw->lock);
+ INIT_LIST_HEAD(&tw->active_list);
+ init_irq_work(&tw->irq_work, wprobe_irq_work_func);
+ INIT_WORK(&tw->work, wprobe_work_func);
+ atomic_set(&tw->missed, 0);
+ atomic_set(&tw->work_pending, 0);
+
if (symbol) {
tw->symbol = kstrdup(symbol, GFP_KERNEL);
if (!tw->symbol)
@@ -509,6 +655,11 @@ static bool trace_wprobe_is_valid_addr(unsigned long addr, int len)
return true;
}
+#ifdef CONFIG_WPROBE_TRIGGERS
+static u64 wprobe_trigger_clear_target __aligned(8);
+#define WPROBE_DEFAULT_CLEAR_ADDRESS ((unsigned long)&wprobe_trigger_clear_target)
+#endif
+
static int parse_address_spec(const char *spec, unsigned long *addr, int *type,
int *len, char **symbol, int *offset)
{
@@ -572,7 +723,13 @@ static int parse_address_spec(const char *spec, unsigned long *addr, int *type,
}
}
+#ifdef CONFIG_WPROBE_TRIGGERS
+ if (strcmp(at + 1, "-1") == 0) {
+ _addr = WPROBE_DEFAULT_CLEAR_ADDRESS;
+ } else if (kstrtoul(at + 1, 0, &_addr) != 0) {
+#else
if (kstrtoul(at + 1, 0, &_addr) != 0) {
+#endif
char *off_str = strpbrk(at + 1, "+-");
if (off_str) {
@@ -756,6 +913,10 @@ static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev)
tw->offset, len);
else
seq_printf(m, " %s@%s:%d", type_str, tw->symbol, len);
+#ifdef CONFIG_WPROBE_TRIGGERS
+ } else if (tw->addr == WPROBE_DEFAULT_CLEAR_ADDRESS) {
+ seq_printf(m, " %s@-1:%d", type_str, len);
+#endif
} else {
seq_printf(m, " %s@0x%lx:%d", type_str, tw->addr, len);
}
@@ -767,9 +928,632 @@ static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev)
return 0;
}
+static enum cpuhp_state wprobe_hp_state __ro_after_init;
+
static __init int init_wprobe_trace(void)
{
- return dyn_event_register(&trace_wprobe_ops);
+ int ret;
+
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "trace/wprobe:online",
+ wprobe_cpu_online, wprobe_cpu_offline);
+ if (ret < 0)
+ return ret;
+
+ wprobe_hp_state = ret;
+
+ ret = dyn_event_register(&trace_wprobe_ops);
+ if (ret)
+ cpuhp_remove_state(wprobe_hp_state);
+
+ return ret;
}
fs_initcall(init_wprobe_trace);
+#ifdef CONFIG_WPROBE_TRIGGERS
+
+#define SET_WPROBE_STR "set_wprobe"
+#define CLEAR_WPROBE_STR "clear_wprobe"
+#define wprobe_trigger_log_err(file, glob, offs, err) \
+ tracing_log_err((file)->tr, "wprobe_trigger", glob, \
+ trace_probe_err_text, TP_ERR_##err, offs)
+
+struct wprobe_trigger_data {
+ struct rcu_head rcu;
+ struct trace_event_file *file;
+ struct trace_wprobe *tw;
+ int offset;
+ long adjust;
+ const char *field;
+ bool clear;
+};
+
+static bool __wprobe_count_func(struct event_trigger_data *data,
+ unsigned long addr)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ struct trace_wprobe *tw = wprobe_data->tw;
+ unsigned long cur_addr;
+
+ if (atomic_read(&tw->work_pending))
+ return false;
+
+ cur_addr = READ_ONCE(tw->addr);
+
+ if (!wprobe_data->clear) {
+ if (cur_addr != WPROBE_DEFAULT_CLEAR_ADDRESS)
+ return false;
+ if (!trace_wprobe_is_valid_addr(addr, tw->len))
+ return false;
+ } else {
+ if (cur_addr == WPROBE_DEFAULT_CLEAR_ADDRESS)
+ return false;
+ if (wprobe_data->field && cur_addr != addr)
+ return false;
+ }
+
+ return true;
+}
+
+static bool wprobe_count_func(struct event_trigger_data *data,
+ struct trace_buffer *buffer, void *rec,
+ struct ring_buffer_event *event)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ struct trace_wprobe *tw = wprobe_data->tw;
+ unsigned long addr = 0;
+
+ if (in_nmi())
+ return false;
+
+ if (!data->count)
+ return false;
+
+ if (wprobe_data->field) {
+ if (!rec)
+ return false;
+ addr = *(unsigned long *)((char *)rec + wprobe_data->offset);
+ addr += wprobe_data->adjust;
+ }
+
+ /*
+ * Pre-check locklessly whether the wprobe can accept this trigger.
+ * If the probe is already busy, record it as missed and do not invoke
+ * wprobe_trigger(). Note that data->count is not decremented here;
+ * it is decremented under tw->lock in wprobe_trigger() to ensure
+ * count budget is only consumed when the hardware update succeeds.
+ */
+ if (!__wprobe_count_func(data, addr)) {
+ atomic_inc(&tw->missed);
+ return false;
+ }
+
+ return true;
+}
+
+static void wprobe_trigger(struct event_trigger_data *data,
+ struct trace_buffer *buffer, void *rec,
+ struct ring_buffer_event *event)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ struct trace_wprobe *tw = wprobe_data->tw;
+ unsigned long target_addr, addr = 0, flags;
+
+ if (in_nmi())
+ return;
+
+ if (wprobe_data->field) {
+ if (!rec)
+ return;
+ addr = *(unsigned long *)((char *)rec + wprobe_data->offset);
+ addr += wprobe_data->adjust;
+ }
+
+ raw_spin_lock_irqsave(&tw->lock, flags);
+
+ if (!__wprobe_count_func(data, addr)) {
+ atomic_inc(&tw->missed);
+ goto out;
+ }
+
+ /*
+ * For counted triggers, verify data->count is still available under
+ * tw->lock to prevent concurrent executions from exceeding the limit.
+ */
+ if ((data->flags & EVENT_TRIGGER_FL_COUNT) && !data->count)
+ goto out;
+
+ target_addr = wprobe_data->clear ? WPROBE_DEFAULT_CLEAR_ADDRESS : addr;
+
+ /*
+ * Try updating the hardware breakpoint on the local CPU first.
+ * If this fails, skip updating tw->addr and do not queue irq_work.
+ * Note that this wprobe will not be traced until tw_addr is updated.
+ * (e.g. NMI happens on this CPU.)
+ */
+ if (trace_wprobe_update_local(tw, target_addr)) {
+ atomic_inc(&tw->missed);
+ goto out;
+ }
+
+ /*
+ * Decrement data->count here under tw->lock only after the local
+ * hardware breakpoint has been successfully updated. This avoids:
+ * 1) Consuming count on trigger attempts that failed/missed.
+ * 2) Race windows where concurrent CPUs decrement count and underflow
+ * to -1 (unlimited).
+ */
+ if (data->flags & EVENT_TRIGGER_FL_COUNT) {
+ if (data->count > 0)
+ (data->count)--;
+ }
+
+ WRITE_ONCE(tw->addr, target_addr);
+ if (!wprobe_data->clear)
+ clear_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
+ else
+ set_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
+
+ /*
+ * For other CPUs, kick irq_work to asynchronously propagate
+ * the new address.
+ */
+ if (num_online_cpus() > 1) {
+ atomic_set(&tw->work_pending, 1);
+ irq_work_queue(&tw->irq_work);
+ }
+
+out:
+ raw_spin_unlock_irqrestore(&tw->lock, flags);
+}
+
+static void free_wprobe_trigger_data(struct wprobe_trigger_data *wprobe_data)
+{
+ if (wprobe_data) {
+ kfree(wprobe_data->field);
+ kfree(wprobe_data);
+ }
+}
+DEFINE_FREE(free_wprobe_trigger_data, struct wprobe_trigger_data *, free_wprobe_trigger_data(_T));
+
+static void free_private_wprobe_trigger_data(struct event_trigger_data *data)
+{
+ free_wprobe_trigger_data(data->private_data);
+}
+
+static int wprobe_trigger_print(struct seq_file *m,
+ struct event_trigger_data *data)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ int missed = atomic_read(&wprobe_data->tw->missed);
+
+ if (wprobe_data->clear) {
+ seq_printf(m, "%s:%s", CLEAR_WPROBE_STR,
+ trace_event_name(wprobe_data->file->event_call));
+ if (wprobe_data->field) {
+ seq_printf(m, ":%s%+ld",
+ wprobe_data->field, wprobe_data->adjust);
+ }
+ } else {
+ seq_printf(m, "%s:%s:%s%+ld", SET_WPROBE_STR,
+ trace_event_name(wprobe_data->file->event_call),
+ wprobe_data->field, wprobe_data->adjust);
+ }
+
+ if (data->count == -1)
+ seq_puts(m, ":unlimited");
+ else
+ seq_printf(m, ":count=%ld", data->count);
+
+ if (data->filter_str)
+ seq_printf(m, " if %s", data->filter_str);
+
+ if (missed)
+ seq_printf(m, " # missed: %d", missed);
+
+ seq_putc(m, '\n');
+
+ return 0;
+}
+
+static struct wprobe_trigger_data *
+wprobe_trigger_alloc(struct trace_wprobe *tw, struct trace_event_file *file,
+ bool clear)
+{
+ struct wprobe_trigger_data *wprobe_data;
+
+ wprobe_data = kzalloc_obj(*wprobe_data);
+ if (!wprobe_data)
+ return NULL;
+
+ wprobe_data->tw = tw;
+ wprobe_data->clear = clear;
+ wprobe_data->file = file;
+
+ return wprobe_data;
+}
+
+static void wprobe_trigger_free(struct event_trigger_data *data)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+
+ if (WARN_ON_ONCE(data->ref <= 0))
+ return;
+
+ data->ref--;
+ if (!data->ref) {
+ /*
+ * Ensure SOFT_DISABLED flag is restored before clearing soft
+ * mode so that if the user did not explicitly enable this
+ * event, __ftrace_event_enable_disable() unregisters it.
+ */
+ set_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
+ /* Remove the SOFT_MODE flag */
+ trace_event_enable_disable(wprobe_data->file, 0, 1);
+ trace_event_put_ref(wprobe_data->file->event_call);
+ trigger_data_free(data);
+ }
+}
+
+static int wprobe_trigger_cmd_parse(struct event_command *cmd_ops,
+ struct trace_event_file *file,
+ char *glob, char *cmd,
+ char *param_and_filter)
+{
+ /*
+ * set_wprobe:EVENT:FIELD[+OFFS]
+ * clear_wprobe:EVENT[:FIELD[+OFFS]]
+ */
+ struct wprobe_trigger_data *wprobe_data = NULL;
+ struct event_trigger_data *trigger_data = NULL;
+ struct trace_event_file *wprobe_file;
+ struct trace_array *tr = file->tr;
+ char *event_str, *field_str, *comment;
+ struct ftrace_event_field *field;
+ struct trace_event_call *event;
+ bool remove, clear = false;
+ unsigned long orig_addr = 0;
+ struct trace_wprobe *tw;
+ char *param, *filter;
+ int ret;
+
+ remove = event_trigger_check_remove(glob);
+
+ if (!strcmp(cmd, CLEAR_WPROBE_STR))
+ clear = true;
+
+ if (param_and_filter) {
+ /* Recover original trigger string to show the error log correctly. */
+ if (*(param_and_filter - 1) == '\0')
+ *(param_and_filter - 1) = ':';
+ comment = strchr(param_and_filter, '#');
+ if (comment)
+ *comment = '\0';
+ }
+
+ if (event_trigger_empty_param(param_and_filter)) {
+ wprobe_trigger_log_err(file, glob, strlen(cmd) + 1, WPROBE_NOT_FOUND);
+ return -EINVAL;
+ }
+
+ ret = event_trigger_separate_filter(param_and_filter, ¶m, &filter, true);
+ if (ret)
+ return ret;
+
+ if (file->event_call->flags & TRACE_EVENT_FL_KPROBE) {
+ wprobe_trigger_log_err(file, glob, 0, WPROBE_ON_KPROBE);
+ return -EOPNOTSUPP;
+ }
+
+ event_str = strsep(¶m, ":");
+
+ /* Find target wprobe */
+ tw = find_trace_wprobe(event_str, WPROBE_EVENT_SYSTEM);
+ if (!tw) {
+ wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_NOT_FOUND);
+ return -ENOENT;
+ }
+ /* The target wprobe must not be used (unless clear) */
+ if (!remove && !clear && trace_probe_is_enabled(&tw->tp)) {
+ wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_BUSY);
+ return -EBUSY;
+ }
+
+ wprobe_file = find_event_file(tr, WPROBE_EVENT_SYSTEM, event_str);
+ if (!wprobe_file) {
+ wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_NOT_FOUND);
+ return -EINVAL;
+ }
+
+ wprobe_data = wprobe_trigger_alloc(tw, wprobe_file, clear);
+ if (!wprobe_data)
+ return -ENOMEM;
+
+ /* clear_wprobe does not need field, but can have optional field. */
+ if (!clear) {
+ char *offs;
+
+ /* Find target field, which must be equivalent to "void *" */
+ field_str = strsep(¶m, ":");
+ if (!field_str) {
+ wprobe_trigger_log_err(file, glob, strlen(glob), WPROBE_NEED_FIELD);
+ ret = -EINVAL;
+ goto out_free;
+ }
+
+ offs = strpbrk(field_str, "+-");
+ if (offs) {
+ long val;
+
+ if (kstrtol(offs, 0, &val) < 0) {
+ wprobe_trigger_log_err(file, glob, offs - glob, BAD_DEREF_OFFS);
+ ret = -EINVAL;
+ goto out_free;
+ }
+ wprobe_data->adjust = val;
+ *offs = '\0';
+ }
+
+ event = file->event_call;
+ field = trace_find_event_field(event, field_str);
+ if (!field) {
+ wprobe_trigger_log_err(file, glob, field_str - glob, NO_EVENT_FIELD);
+ ret = -ENOENT;
+ goto out_free;
+ }
+
+ if (field->size != sizeof(void *)) {
+ wprobe_trigger_log_err(file, glob, field_str - glob, WPROBE_BAD_FIELD);
+ ret = -ENOEXEC;
+ goto out_free;
+ }
+ wprobe_data->offset = field->offset;
+ wprobe_data->field = kstrdup(field_str, GFP_KERNEL);
+ if (!wprobe_data->field) {
+ ret = -ENOMEM;
+ goto out_free;
+ }
+ } else if (param && (isalpha(param[0]) || param[0] == '_')) {
+ if (strncmp(param, "count=", 6) != 0 &&
+ (strcmp(param, "unlimited") != 0 ||
+ trace_find_event_field(file->event_call, "unlimited"))) {
+ char *offs;
+
+ field_str = strsep(¶m, ":");
+ offs = strpbrk(field_str, "+-");
+ if (offs) {
+ long val;
+
+ if (kstrtol(offs, 0, &val) < 0) {
+ wprobe_trigger_log_err(file, glob,
+ offs - glob,
+ BAD_DEREF_OFFS);
+ ret = -EINVAL;
+ goto out_free;
+ }
+ wprobe_data->adjust = val;
+ *offs = '\0';
+ }
+
+ event = file->event_call;
+ field = trace_find_event_field(event, field_str);
+ if (!field) {
+ wprobe_trigger_log_err(file, glob,
+ field_str - glob,
+ NO_EVENT_FIELD);
+ ret = -ENOENT;
+ goto out_free;
+ }
+
+ if (field->size != sizeof(void *)) {
+ wprobe_trigger_log_err(file, glob,
+ field_str - glob,
+ WPROBE_BAD_FIELD);
+ ret = -ENOEXEC;
+ goto out_free;
+ }
+ wprobe_data->offset = field->offset;
+ wprobe_data->field = kstrdup(field_str, GFP_KERNEL);
+ if (!wprobe_data->field) {
+ ret = -ENOMEM;
+ goto out_free;
+ }
+ }
+ }
+
+ if (param) {
+ if (!strcmp(param, "unlimited"))
+ param = NULL;
+ else if (!strncmp(param, "count=", 6))
+ param += 6;
+ }
+
+ trigger_data = trigger_data_alloc(cmd_ops, cmd, param, wprobe_data);
+ if (!trigger_data) {
+ ret = -ENOMEM;
+ goto out_free;
+ }
+
+ trigger_data->private_data_free = free_private_wprobe_trigger_data;
+
+ if (remove) {
+ event_trigger_unregister(cmd_ops, file, glob+1, trigger_data);
+ trigger_data_free(trigger_data);
+ return 0;
+ }
+
+ /* Up the trigger_data count to make sure nothing frees it on failure */
+ event_trigger_init(trigger_data);
+
+ ret = event_trigger_parse_num(param, trigger_data);
+ if (ret) {
+ wprobe_trigger_log_err(file, glob, param - glob, BAD_IMM);
+ goto out_free_trigger;
+ }
+
+ ret = event_trigger_set_filter(cmd_ops, file, filter, trigger_data);
+ if (ret < 0)
+ goto out_free_trigger;
+
+ /* Soft-enable (register) wprobe event on WPROBE_DEFAULT_CLEAR_ADDRESS */
+ if (!trace_event_try_get_ref(wprobe_file->event_call)) {
+ ret = -ENODEV;
+ goto out_free_trigger;
+ }
+
+ if (!clear) {
+ orig_addr = tw->addr;
+ WRITE_ONCE(tw->addr, WPROBE_DEFAULT_CLEAR_ADDRESS);
+ }
+
+ ret = trace_event_enable_disable(wprobe_file, 1, 1);
+ if (ret < 0) {
+ if (!clear)
+ WRITE_ONCE(tw->addr, orig_addr);
+ goto out_put;
+ }
+
+ ret = event_trigger_register(cmd_ops, file, glob, trigger_data);
+ if (ret)
+ goto out_disable;
+
+ /* Balance the event_trigger_init() at registration start */
+ event_trigger_free(trigger_data);
+ return 0;
+
+out_disable:
+ if (!clear)
+ WRITE_ONCE(tw->addr, orig_addr);
+ trace_event_enable_disable(wprobe_file, 0, 1);
+out_put:
+ trace_event_put_ref(wprobe_file->event_call);
+out_free_trigger:
+ event_trigger_reset_filter(cmd_ops, trigger_data);
+ trigger_data_free(trigger_data);
+ return ret;
+
+out_free:
+ free_wprobe_trigger_data(wprobe_data);
+ return ret;
+}
+
+/* Return event_trigger_data if there is a trigger which points the same wprobe */
+static struct event_trigger_data *
+wprobe_trigger_find_same(struct event_trigger_data *test,
+ struct trace_event_file *file)
+{
+ struct wprobe_trigger_data *test_wprobe_data = test->private_data;
+ struct wprobe_trigger_data *wprobe_data;
+ struct event_trigger_data *iter;
+
+ list_for_each_entry(iter, &file->triggers, list) {
+ wprobe_data = iter->private_data;
+ if (!wprobe_data ||
+ iter->cmd_ops->trigger_type !=
+ test->cmd_ops->trigger_type)
+ continue;
+ if (wprobe_data->tw == test_wprobe_data->tw &&
+ wprobe_data->clear == test_wprobe_data->clear)
+ return iter;
+ }
+ return NULL;
+}
+
+static int wprobe_register_trigger(char *glob,
+ struct event_trigger_data *data,
+ struct trace_event_file *file)
+{
+ int ret = 0;
+
+ lockdep_assert_held(&event_mutex);
+
+ /* The same wprobe is not accept on the same file (event) */
+ if (wprobe_trigger_find_same(data, file))
+ return -EEXIST;
+
+ if (data->cmd_ops->init) {
+ ret = data->cmd_ops->init(data);
+ if (ret < 0)
+ return ret;
+ }
+
+ list_add_rcu(&data->list, &file->triggers);
+
+ update_cond_flag(file);
+ ret = trace_event_trigger_enable_disable(file, 1);
+ if (ret < 0) {
+ list_del_rcu(&data->list);
+ update_cond_flag(file);
+ tracepoint_synchronize_unregister();
+ if (data->cmd_ops->free)
+ data->cmd_ops->free(data);
+ }
+ return ret;
+}
+
+static void wprobe_unregister_trigger(char *glob,
+ struct event_trigger_data *test,
+ struct trace_event_file *file)
+{
+ struct event_trigger_data *data;
+
+ lockdep_assert_held(&event_mutex);
+
+ data = wprobe_trigger_find_same(test, file);
+ if (!data)
+ return;
+
+ list_del_rcu(&data->list);
+ trace_event_trigger_enable_disable(file, 0);
+ update_cond_flag(file);
+ tracepoint_synchronize_unregister();
+ if (data->cmd_ops->free)
+ data->cmd_ops->free(data);
+}
+
+static struct event_command trigger_wprobe_set_cmd = {
+ .name = SET_WPROBE_STR,
+ .trigger_type = ETT_EVENT_WPROBE,
+ /* This triggers after when the event is recorded. */
+ .flags = EVENT_CMD_FL_NEEDS_REC,
+ .parse = wprobe_trigger_cmd_parse,
+ .reg = wprobe_register_trigger,
+ .unreg = wprobe_unregister_trigger,
+ .set_filter = set_trigger_filter,
+ .trigger = wprobe_trigger,
+ .count_func = wprobe_count_func,
+ .print = wprobe_trigger_print,
+ .init = event_trigger_init,
+ .free = wprobe_trigger_free,
+};
+
+static struct event_command trigger_wprobe_clear_cmd = {
+ .name = CLEAR_WPROBE_STR,
+ .trigger_type = ETT_EVENT_WPROBE,
+ /* This triggers after when the event is recorded. */
+ .flags = EVENT_CMD_FL_NEEDS_REC,
+ .parse = wprobe_trigger_cmd_parse,
+ .reg = wprobe_register_trigger,
+ .unreg = wprobe_unregister_trigger,
+ .set_filter = set_trigger_filter,
+ .trigger = wprobe_trigger,
+ .count_func = wprobe_count_func,
+ .print = wprobe_trigger_print,
+ .init = event_trigger_init,
+ .free = wprobe_trigger_free,
+};
+
+static __init int init_trigger_wprobe_cmds(void)
+{
+ int ret;
+
+ ret = register_event_command(&trigger_wprobe_set_cmd);
+ if (WARN_ON(ret < 0))
+ return ret;
+ ret = register_event_command(&trigger_wprobe_clear_cmd);
+ if (WARN_ON(ret < 0))
+ unregister_event_command(&trigger_wprobe_set_cmd);
+
+ return ret;
+}
+fs_initcall(init_trigger_wprobe_cmds);
+#endif /* CONFIG_WPROBE_TRIGGERS */
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v18 09/11] selftests: tracing: Add wprobe trigger testcases
2026-09-25 11:10 [PATCH v18 00/11] tracing: wprobe: x86: Add wprobe for watchpoint Masami Hiramatsu (Google)
` (6 preceding siblings ...)
2026-09-25 11:12 ` [PATCH v18 08/11] tracing/wprobe: Add set_wprobe and clear_wprobe event triggers Masami Hiramatsu (Google)
@ 2026-09-25 11:13 ` Masami Hiramatsu (Google)
2026-09-25 11:13 ` [PATCH v18 10/11] tracing/wprobe: Support BTF typecast in fetchargs Masami Hiramatsu (Google)
2026-09-25 11:13 ` [PATCH v18 11/11] tracing/wprobe: Support BTF struct offset resolution in set_wprobe trigger Masami Hiramatsu (Google)
9 siblings, 0 replies; 11+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-09-25 11:13 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users, x86
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Add testcases for checking wprobe triggers. This sets set_wprobe and
clear_wprobe triggers on fprobe event and static trace event to
monitor memory accesses within the trace-events-sample kernel module.
Also add a testcase for verifying wprobe triggers across CPU hotplug,
and a testcase for verifying wprobe trigger syntax error logging.
Link: https://lore.kernel.org/all/59637b96946653393a7ad3c7de094094796b39c2.1785067572.git.wangjinchao600@gmail.com/
Assisted-by: LLM
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v18:
- Add "set_wprobe:<wprobe-event>":README to requires line in wprobe trigger
tests so they are skipped if CONFIG_WPROBE_TRIGGERS is disabled.
- Clear trace buffer after taking CPU offline in trigger-wprobe-hotplug.tc
to avoid matching watchpoint events logged prior to offlining.
Changes in v17:
- Add trigger-wprobe-hotplug.tc to verify wprobe triggers across
CPU offline and online transitions.
Changes in v15:
- Add explicit failure checks with fail helper in trigger-wprobe.tc.
- Declare fprobe README requirement in trigger-wprobe.tc.
- Add trigger file check to trigger-wprobe-syntax-errors.tc requires line.
- Fix subject tag to tracing.
Changes in v14:
- Update dummy wprobe event definition to use '-1' instead of '0'.
- Add BTF argument dependency requirement in trigger-wprobe.tc.
- Add delay before clear_trace in trigger-wprobe.tc to avoid race with
clear_wprobe trigger execution.
Changes in v13:
- Look up target function dynamically in trigger-wprobe-syntax-errors.tc.
- Disable set_wprobe trigger before verifying clear_wprobe in
trigger-wprobe.tc to avoid race condition with sample_timer_cb.
Changes in v12:
- Add trigger-wprobe-syntax-errors.tc for verifying wprobe trigger syntax
error logging.
- Fix requires line in trigger-wprobe-syntax-errors.tc so test is not
evaluated as unsupported prior to execution.
- Define dfd=$arg1 fetcharg on testevent for trigger syntax checks.
Changes in v11:
- Update testcase to use trace-events-sample kernel module instead of
VFS file operations.
---
tools/testing/selftests/ftrace/config | 1
.../test.d/trigger/trigger-wprobe-hotplug.tc | 153 ++++++++++++++++++++
.../test.d/trigger/trigger-wprobe-syntax-errors.tc | 37 +++++
.../ftrace/test.d/trigger/trigger-wprobe.tc | 107 ++++++++++++++
4 files changed, 298 insertions(+)
create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc
create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc
create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc
diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftests/ftrace/config
index d2f503722020..ecdee77f360f 100644
--- a/tools/testing/selftests/ftrace/config
+++ b/tools/testing/selftests/ftrace/config
@@ -28,3 +28,4 @@ CONFIG_TRACER_SNAPSHOT=y
CONFIG_UPROBES=y
CONFIG_UPROBE_EVENTS=y
CONFIG_WPROBE_EVENTS=y
+CONFIG_WPROBE_TRIGGERS=y
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc
new file mode 100644
index 000000000000..e7cfe5a4b259
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc
@@ -0,0 +1,153 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger across cpu hotplug
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "[(structname[,field])]<argname>[->field[->field|.field...]]":README "set_wprobe:<wprobe-event>":README
+
+fail() { #msg
+ echo "$1"
+ exit_fail
+}
+
+if ! which nproc >/dev/null 2>&1 ; then
+ nproc() {
+ ls -d /sys/devices/system/cpu/cpu[0-9]* | wc -l
+ }
+fi
+
+NP=`nproc`
+if [ $NP -le 1 ] ; then
+ echo "We cannot test cpu hotplug in UP environment"
+ exit_unresolved
+fi
+
+# Find an online CPU that can be offlined
+for i in /sys/devices/system/cpu/cpu[1-9]*; do
+ if [ -f $i/online ] && [ "$(cat $i/online)" = "1" ]; then
+ cpu=$i
+ break
+ fi
+done
+
+if [ -z "$cpu" ]; then
+ echo "We cannot test cpu hotplug without a hotpluggable online cpu"
+ exit_unresolved
+fi
+
+rmmod trace-events-sample ||:
+if ! modprobe trace-events-sample ; then
+ echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m"
+ exit_unresolved
+fi
+
+cleanup_wprobe_triggers() {
+ [ -n "$cpu" ] && [ -f "$cpu/online" ] && echo 1 > $cpu/online || true
+ if [ -f events/fprobes/testevent/trigger ]; then
+ reset_trigger_file events/fprobes/testevent/trigger || true
+ fi
+ if [ -f events/sample-trace/foo_bar_with_fn/trigger ]; then
+ reset_trigger_file events/sample-trace/foo_bar_with_fn/trigger || true
+ fi
+ echo 0 > events/enable 2>/dev/null || true
+ echo > dynamic_events 2>/dev/null || true
+ sleep 1
+ rmmod trace-events-sample 2>/dev/null || true
+ return 0
+}
+
+trap cleanup_wprobe_triggers EXIT
+
+echo 0 > tracing_on
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "sample_timer_cb" /proc/kallsyms; then
+ echo "UNRESOLVED: sample_timer_cb not found"
+ exit_unresolved
+fi
+
+:;: "Add a wprobe event used by trigger" ;:
+echo 'w:watch rw@-1:8 address=$addr value=$value' > dynamic_events
+
+:;: "Add events for triggering wprobe" ;:
+echo 'f:fprobes/testevent sample_timer_cb timer=t' >> dynamic_events
+
+:;: "Enable all events before setting triggers" ;:
+echo 1 > tracing_on
+echo 1 >> events/fprobes/testevent/enable
+echo 1 >> events/sample-trace/foo_bar_with_fn/enable
+
+:;: "Set set_wprobe trigger on testevent" ;:
+echo 'set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to set set_wprobe trigger"
+fi
+
+:;: "Take target CPU offline" ;:
+echo 0 > $cpu/online
+sleep 1
+
+# Clear trace before waiting for events while CPU is offline
+clear_trace
+
+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate while CPU is offline
+sleep 2
+
+:;: "Check set_wprobe trigger activated the watchpoint" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint while CPU is offline"
+fi
+
+:;: "Bring target CPU back online" ;:
+echo 1 > $cpu/online
+sleep 1
+
+# Clear trace and verify that watchpoint triggers after CPU is onlined
+clear_trace
+sleep 2
+
+:;: "Verify watchpoint triggers after CPU came back online" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint after CPU was onlined"
+fi
+
+:;: "Set clear_wprobe trigger on foo_bar_with_fn" ;:
+echo 'clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger
+if ! grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then
+ fail "Failed to set clear_wprobe trigger"
+fi
+
+# Disable set_wprobe to prevent sample_timer_cb from re-arming the watchpoint
+echo '!set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+
+# Wait for foo_bar_with_fn to fire and clear_wprobe trigger to deactivate watchpoint
+sleep 2
+
+# Clear trace and wait to ensure no new watchpoint events are generated
+clear_trace
+sleep 1
+
+:;: "Ensure clear_wprobe trigger deactivated the watchpoint" ;:
+if grep -q watch trace; then
+ fail "Failed to clear watchpoint"
+fi
+
+:;: "Remove wprobe triggers" ;:
+echo '!clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger
+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to remove set_wprobe trigger"
+fi
+if grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then
+ fail "Failed to remove clear_wprobe trigger"
+fi
+
+:;: "Disable events and remove dynamic events" ;:
+echo 0 > events/enable
+echo > dynamic_events
+clear_trace
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc
new file mode 100644
index 000000000000..87170a66e882
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc
@@ -0,0 +1,37 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger syntax errors
+# requires: dynamic_events error_log "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "set_wprobe:<wprobe-event>":README
+
+check_error() { # command-with-error-pos-by-^
+ ftrace_errlog_check "wprobe_trigger" "$1" "events/fprobes/testevent/trigger"
+}
+
+TARGET_FUNC=$(grep -m 1 -E -w "[tT] (vfs_read|do_sys_openat2|do_sys_open)" /proc/kallsyms | awk '{print $3}')
+if [ -z "$TARGET_FUNC" ]; then
+ echo "UNRESOLVED: target function not found"
+ exit_unresolved
+fi
+
+# Add a dummy fprobe event to attach triggers to
+echo "f:fprobes/testevent $TARGET_FUNC dfd=\$arg1" > dynamic_events
+
+# Add a target wprobe event
+echo 'w:watch rw@-1:8' >> dynamic_events
+
+# Test errors on trigger syntax
+check_error 'set_wprobe:^non_exist_wprobe:dfd' # WPROBE_NOT_FOUND
+check_error 'set_wprobe:^' # WPROBE_NOT_FOUND
+check_error 'set_wprobe:watch^' # WPROBE_NEED_FIELD
+check_error 'set_wprobe:watch:^non_exist_field' # NO_EVENT_FIELD
+
+# Enable target wprobe event and test WPROBE_BUSY error
+echo 1 > events/wprobes/watch/enable
+check_error 'set_wprobe:^watch:dfd' # WPROBE_BUSY
+echo 0 > events/wprobes/watch/enable
+
+# Cleanup
+echo '-:watch' >> dynamic_events
+echo '-:testevent' >> dynamic_events
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc
new file mode 100644
index 000000000000..a0565b80932c
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc
@@ -0,0 +1,107 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "[(structname[,field])]<argname>[->field[->field|.field...]]":README "set_wprobe:<wprobe-event>":README
+
+fail() { #msg
+ echo "$1"
+ exit_fail
+}
+
+rmmod trace-events-sample ||:
+if ! modprobe trace-events-sample ; then
+ echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m"
+ exit_unresolved
+fi
+
+cleanup_wprobe_triggers() {
+ if [ -f events/fprobes/testevent/trigger ]; then
+ reset_trigger_file events/fprobes/testevent/trigger || true
+ fi
+ if [ -f events/sample-trace/foo_bar_with_fn/trigger ]; then
+ reset_trigger_file events/sample-trace/foo_bar_with_fn/trigger || true
+ fi
+ echo 0 > events/enable 2>/dev/null || true
+ echo > dynamic_events 2>/dev/null || true
+ sleep 1
+ rmmod trace-events-sample 2>/dev/null || true
+ return 0
+}
+
+trap cleanup_wprobe_triggers EXIT
+
+echo 0 > tracing_on
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "sample_timer_cb" /proc/kallsyms; then
+ echo "UNRESOLVED: sample_timer_cb not found"
+ exit_unresolved
+fi
+
+:;: "Add a wprobe event used by trigger" ;:
+echo 'w:watch rw@-1:8 address=$addr value=$value' > dynamic_events
+
+:;: "Add events for triggering wprobe" ;:
+echo 'f:fprobes/testevent sample_timer_cb timer=t' >> dynamic_events
+
+:;: "Enable all events before setting triggers" ;:
+echo 1 > tracing_on
+echo 1 >> events/fprobes/testevent/enable
+echo 1 >> events/sample-trace/foo_bar_with_fn/enable
+
+:;: "Set set_wprobe trigger on testevent" ;:
+echo 'set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to set set_wprobe trigger"
+fi
+
+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate
+sleep 2
+
+:;: "Check set_wprobe trigger activated the watchpoint" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint"
+fi
+
+:;: "Set clear_wprobe trigger on foo_bar_with_fn" ;:
+echo 'clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger
+if ! grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then
+ fail "Failed to set clear_wprobe trigger"
+fi
+
+# Disable set_wprobe to prevent sample_timer_cb from re-arming the watchpoint
+echo '!set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+
+# Wait for foo_bar_with_fn to fire and clear_wprobe trigger to deactivate watchpoint
+sleep 2
+
+# Clear trace and wait to ensure no new watchpoint events are generated
+clear_trace
+sleep 1
+
+:;: "Ensure clear_wprobe trigger deactivated the watchpoint" ;:
+if grep -q watch trace; then
+ fail "Failed to clear watchpoint"
+fi
+
+:;: "Remove wprobe triggers" ;:
+echo '!clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger
+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to remove set_wprobe trigger"
+fi
+if grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then
+ fail "Failed to remove clear_wprobe trigger"
+fi
+
+:;: "Disable events and remove dynamic events" ;:
+echo 0 > events/enable
+echo > dynamic_events
+clear_trace
+
+exit 0
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v18 10/11] tracing/wprobe: Support BTF typecast in fetchargs
2026-09-25 11:10 [PATCH v18 00/11] tracing: wprobe: x86: Add wprobe for watchpoint Masami Hiramatsu (Google)
` (7 preceding siblings ...)
2026-09-25 11:13 ` [PATCH v18 09/11] selftests: tracing: Add wprobe trigger testcases Masami Hiramatsu (Google)
@ 2026-09-25 11:13 ` Masami Hiramatsu (Google)
2026-09-25 11:13 ` [PATCH v18 11/11] tracing/wprobe: Support BTF struct offset resolution in set_wprobe trigger Masami Hiramatsu (Google)
9 siblings, 0 replies; 11+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-09-25 11:13 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users, x86
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Allow BTF typecast syntax (STRUCT)FETCHARG->MEMBER in wprobe event
fetchargs. Previously, handle_typecast() rejected any probe context
that was not a function entry/return or tracepoint event probe.
Wprobe events use $addr (the accessed address) and $value (the value
at that address). By enabling BTF typecast, users can now cast these
to a concrete struct type and access its fields directly. For example:
echo 'w:watch rw@-1:8 dflag=(dentry)$addr->d_flags' >> dynamic_events
With a set_wprobe trigger pointing the watchpoint at a dentry address,
the resulting trace shows d_flags being accessed at that location.
Note that $addr and $value are restricted to kernel-space memory,
which is consistent with the existing TPARG_FL_KERNEL flag used when
parsing wprobe fetchargs.
Assisted-by: LLM
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v18:
- Add "set_wprobe:<wprobe-event>":README to requires line in
trigger-wprobe-btf-typecast.tc so the test is skipped if
CONFIG_WPROBE_TRIGGERS is disabled.
Changes in v15:
- Document BTF typecast syntax for wprobe fetchargs in wprobetrace.rst.
- Add explicit failure checks with fail helper in
trigger-wprobe-btf-typecast.tc.
- Declare fprobe README requirement in trigger-wprobe-btf-typecast.tc.
Changes in v14:
- Update dummy wprobe event definition to use '-1' instead of '0'.
Changes in v11:
- Update trigger-wprobe-btf-typecast.tc to use trace-events-sample
kernel module.
- Fix commit comment.
Changes in v9:
- Newly added.
---
Documentation/trace/wprobetrace.rst | 8 ++
kernel/trace/trace_probe.c | 13 +++
kernel/trace/trace_probe.h | 5 +
tools/testing/selftests/ftrace/config | 1
.../test.d/trigger/trigger-wprobe-btf-typecast.tc | 85 ++++++++++++++++++++
5 files changed, 111 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc
diff --git a/Documentation/trace/wprobetrace.rst b/Documentation/trace/wprobetrace.rst
index df8985283312..20a11443c6db 100644
--- a/Documentation/trace/wprobetrace.rst
+++ b/Documentation/trace/wprobetrace.rst
@@ -47,6 +47,14 @@ Synopsis of wprobe-events
(u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types
(x8/x16/x32/x64), "char", "string", "ustring", "symbol", "symstr"
and bitfield are supported.
+ (STRUCT[,ASGN])FETCHARG->MEMBER[->MEMBER] : If BTF is supported, typecast
+ FETCHARG to a pointer to STRUCT and then dereference the
+ pointer defined by ->MEMBER. ASGN can be specified optionally.
+ If ASGN is specified, FETCHARG will be cast to the same offset
+ position as the ASGN member, rather than to the beginning of
+ the STRUCT.
+ (STRUCT[,ASGN])(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above
+ can also be used with another FETCHARG.
(\*1) This is useful for fetching a field of data structures.
(\*2) "u" means user-space dereference.
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index 77266e042418..ccf489635ad7 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -890,6 +890,16 @@ static int query_btf_struct(const char *sname, struct traceprobe_parse_context *
ctx->struct_btf = NULL;
}
+ if (ctx->btf) {
+ id = btf_find_by_name_kind(ctx->btf, sname, BTF_KIND_STRUCT);
+ if (id > 0) {
+ btf_get(ctx->btf);
+ ctx->struct_btf = ctx->btf;
+ ctx->last_struct = btf_type_by_id(ctx->struct_btf, id);
+ return 0;
+ }
+ }
+
id = bpf_find_btf_id(sname, BTF_KIND_STRUCT, &btf);
if (id < 0)
return id;
@@ -965,7 +975,8 @@ static int handle_typecast(char *arg, struct traceprobe_parse_context *ctx)
if (!(tparg_is_event_probe(ctx->flags) ||
tparg_is_function_entry(ctx->flags) ||
- tparg_is_function_return(ctx->flags))) {
+ tparg_is_function_return(ctx->flags) ||
+ tparg_is_wprobe(ctx->flags))) {
trace_probe_log_err(ctx->offset, NOSUP_BTFARG);
return -EOPNOTSUPP;
}
diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h
index 1e11077ead67..c54c554b1949 100644
--- a/kernel/trace/trace_probe.h
+++ b/kernel/trace/trace_probe.h
@@ -439,6 +439,11 @@ static inline bool tparg_is_event_probe(unsigned int flags)
return !!(flags & TPARG_FL_TEVENT);
}
+static inline bool tparg_is_wprobe(unsigned int flags)
+{
+ return !!(flags & TPARG_FL_WPROBE);
+}
+
/* Each typecast consumes nested level. So the max number of typecast is 8. */
#define TRACEPROBE_MAX_NESTED_LEVEL 8
diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftests/ftrace/config
index ecdee77f360f..f067874902ed 100644
--- a/tools/testing/selftests/ftrace/config
+++ b/tools/testing/selftests/ftrace/config
@@ -1,5 +1,6 @@
CONFIG_BPF_SYSCALL=y
CONFIG_DEBUG_INFO_BTF=y
+CONFIG_DEBUG_INFO_BTF_MODULES=y
CONFIG_DEBUG_INFO_DWARF4=y
CONFIG_EPROBE_EVENTS=y
CONFIG_FPROBE=y
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc
new file mode 100644
index 000000000000..94831706238a
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc
@@ -0,0 +1,85 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger with BTF typecast fetchargs
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "[(structname[,field])]<argname>[->field[->field|.field...]]":README "set_wprobe:<wprobe-event>":README
+
+fail() { #msg
+ echo "$1"
+ exit_fail
+}
+
+rmmod trace-events-sample ||:
+if ! modprobe trace-events-sample ; then
+ echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m"
+ exit_unresolved
+fi
+
+cleanup_wprobe_triggers() {
+ if [ -f events/fprobes/testevent/trigger ]; then
+ reset_trigger_file events/fprobes/testevent/trigger || true
+ fi
+ echo 0 > events/enable 2>/dev/null || true
+ echo > dynamic_events 2>/dev/null || true
+ sleep 1
+ rmmod trace-events-sample 2>/dev/null || true
+ return 0
+}
+
+trap cleanup_wprobe_triggers EXIT
+
+echo 0 > tracing_on
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "sample_timer_cb" /proc/kallsyms; then
+ echo "UNRESOLVED: sample_timer_cb not found"
+ exit_unresolved
+fi
+
+:;: "Add a wprobe event with BTF typecast fetchargs" ;:
+# (foo_timer_data,timer)$addr->counter reads counter from struct foo_timer_data via BTF typecast
+echo 'w:watch rw@-1:8 address=$addr counter=(foo_timer_data,timer)$addr->counter' >> dynamic_events
+
+:;: "Check the wprobe event is registered with counter field" ;:
+if ! grep -q "counter" dynamic_events; then
+ fail "Failed to register counter field in dynamic_events"
+fi
+
+:;: "Add fprobe event for sample_timer_cb" ;:
+echo 'f:fprobes/testevent sample_timer_cb timer=t' >> dynamic_events
+
+:;: "Enable all events before setting triggers" ;:
+echo 1 > tracing_on
+echo 1 >> events/fprobes/testevent/enable
+
+:;: "Set set_wprobe trigger on testevent" ;:
+echo 'set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to set set_wprobe trigger"
+fi
+
+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate
+sleep 3
+
+:;: "Check set_wprobe trigger activated the watchpoint" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint"
+fi
+
+:;: "Remove wprobe triggers" ;:
+echo '!set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to remove set_wprobe trigger"
+fi
+
+:;: "Disable events and remove dynamic events" ;:
+echo 0 > events/enable
+echo > dynamic_events
+clear_trace
+
+exit 0
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v18 11/11] tracing/wprobe: Support BTF struct offset resolution in set_wprobe trigger
2026-09-25 11:10 [PATCH v18 00/11] tracing: wprobe: x86: Add wprobe for watchpoint Masami Hiramatsu (Google)
` (8 preceding siblings ...)
2026-09-25 11:13 ` [PATCH v18 10/11] tracing/wprobe: Support BTF typecast in fetchargs Masami Hiramatsu (Google)
@ 2026-09-25 11:13 ` Masami Hiramatsu (Google)
9 siblings, 0 replies; 11+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-09-25 11:13 UTC (permalink / raw)
To: Steven Rostedt, Peter Zijlstra, Ingo Molnar
Cc: Jinchao Wang, Mathieu Desnoyers, Masami Hiramatsu,
Thomas Gleixner, Borislav Petkov, Dave Hansen, H . Peter Anvin,
Alexander Shishkin, Ian Rogers, linux-kernel, linux-trace-kernel,
linux-doc, linux-perf-users, x86
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Allow set_wprobe trigger to use BTF struct offset resolution to specify
the target address field.
Link: https://lore.kernel.org/all/59637b96946653393a7ad3c7de094094796b39c2.1785067572.git.wangjinchao600@gmail.com/
Assisted-by: LLM
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v18:
- In get_offset_of_field(), rename mtype to container and document that
it represents the containing structure/union type (handling anonymous
structures/unions) rather than member's own type.
- In get_offset_of_field(), verify bit offset is byte-aligned before
returning byte offset.
- Add "set_wprobe:<wprobe-event>":README to requires line in
trigger-wprobe-btf-offset.tc so the test is skipped if
CONFIG_WPROBE_TRIGGERS is disabled.
Changes in v15:
- Duplicate field string to avoid modifying glob in place in
wprobe_trigger_typecast_parse() so tracing_log_err() prints pristine
command string.
- Remove leftover offset and adjust debug print in wprobe_trigger_print().
- Document BTF struct offset resolution syntax in wprobetrace.rst.
- Add explicit failure checks with fail helper in
trigger-wprobe-btf-offset.tc.
- Declare fprobe README requirement in trigger-wprobe-btf-offset.tc.
Changes in v14:
- Update dummy wprobe event definition to use '-1' instead of '0'.
- Pass member_type to btf_find_struct_member() and check its kflag in
get_offset_of_field().
Changes in v13:
- Check for field token before calling wprobe_trigger_field_parse() in
clear_wprobe to avoid spurious error log entries on numeric counts.
Changes in v12:
- Refactor field parsing logic into wprobe_trigger_field_parse().
- Remove unused variable count_str in wprobe_trigger_cmd_parse().
---
Documentation/trace/wprobetrace.rst | 7
kernel/trace/trace_wprobe.c | 304 +++++++++++++++-----
.../test.d/trigger/trigger-wprobe-btf-offset.tc | 85 ++++++
3 files changed, 318 insertions(+), 78 deletions(-)
create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc
diff --git a/Documentation/trace/wprobetrace.rst b/Documentation/trace/wprobetrace.rst
index 20a11443c6db..1d9b32d02d53 100644
--- a/Documentation/trace/wprobetrace.rst
+++ b/Documentation/trace/wprobetrace.rst
@@ -90,10 +90,17 @@ Combination with trigger action
The event trigger action can extend the utilization of this wprobe.
- set_wprobe:WPEVENT:FIELD[+|-ADJUST][:COUNT]
+- set_wprobe:WPEVENT:(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST][:COUNT]
- clear_wprobe:WPEVENT[:FIELD[+|-ADJUST][:COUNT]]
+- clear_wprobe:WPEVENT[:(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST][:COUNT]]
Set these triggers to the target event, then the WPROBE event will be
setup to trace the memory access at FIELD[+|-ADJUST] address.
+If BTF is supported, the target address can also be resolved using BTF
+struct offset resolution: `(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST]`.
+Here, EVENT_FIELD is an event field containing a pointer to STRUCT (or to
+the ASGN member if ASGN is specified), and MEMBER is the struct member whose
+offset is resolved automatically via BTF.
When clear_wprobe is hit, if FIELD is NOT specified, the WPEVENT is
forcibly cleared. If FIELD[+|-ADJUST] is set, it clears WPEVENT only
if its watching address is the same as the FIELD[+|-ADJUST] value.
diff --git a/kernel/trace/trace_wprobe.c b/kernel/trace/trace_wprobe.c
index 31de7dd5bfd9..a79439cbfe6b 100644
--- a/kernel/trace/trace_wprobe.c
+++ b/kernel/trace/trace_wprobe.c
@@ -29,6 +29,7 @@
#include <asm/ptrace.h>
#include "trace.h"
+#include "trace_btf.h"
#include "trace_dynevent.h"
#include "trace_probe.h"
#include "trace_probe_kernel.h"
@@ -1194,6 +1195,223 @@ static void wprobe_trigger_free(struct event_trigger_data *data)
}
}
+#ifdef CONFIG_PROBE_EVENTS_BTF_ARGS
+
+static int get_offset_of_field(struct btf *btf, const struct btf_type *type, char *field_name)
+{
+ const struct btf_member *field;
+ const struct btf_type *container;
+ int bitoffs = 0;
+ u32 anon_offs;
+ char *next;
+
+ do {
+ next = strchr(field_name, '.');
+ if (next)
+ *next++ = '\0';
+
+ anon_offs = 0;
+ /*
+ * btf_find_struct_member() returns the actual containing
+ * structure/union type in @container (which can be an
+ * inner anonymous struct/union rather than @type).
+ */
+ field = btf_find_struct_member(btf, type, field_name, &anon_offs, &container);
+ if (IS_ERR_OR_NULL(field))
+ return -ENOENT;
+
+ if (btf_type_kflag(container)) {
+ /* Reject bitfield member access */
+ if (BTF_MEMBER_BITFIELD_SIZE(field->offset))
+ return -EINVAL;
+ bitoffs += anon_offs + BTF_MEMBER_BIT_OFFSET(field->offset);
+ } else {
+ bitoffs += anon_offs + field->offset;
+ }
+
+ field_name = next;
+ if (next) {
+ type = btf_type_skip_modifiers(btf, field->type, NULL);
+ if (!type)
+ return -ENOENT;
+ }
+ } while (next);
+
+ if (bitoffs % BITS_PER_BYTE)
+ return -EINVAL;
+
+ return bitoffs / BITS_PER_BYTE;
+}
+
+/* btf_put(NULL) is acceptable. */
+DEFINE_FREE(btf_put, struct btf *, btf_put(_T))
+
+/* parse typecast: (TYPE[,ASGN])EVENT_FIELD->FIELD[.SUBFIELD...][+-OFFS] and set adjust. */
+static int wprobe_trigger_typecast_parse(char *field_str,
+ struct trace_event_file *file,
+ struct wprobe_trigger_data *wprobe_data,
+ const char *glob)
+{
+ struct btf *btf __free(btf_put) = NULL;
+ char *buf __free(kfree) = NULL;
+ struct ftrace_event_field *field;
+ const struct btf_type *type;
+ char *assign_field;
+ char *event_field;
+ char *type_field;
+ char *type_name;
+ char *offs;
+ long val = 0;
+ int base_offset = field_str - glob;
+ int event_field_offset;
+ int id, adjust;
+
+ buf = kstrdup(field_str, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ type_name = buf + 1;
+ event_field = strchr(type_name, ')');
+ if (!event_field) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (type_name - buf),
+ DEREF_OPEN_BRACE);
+ return -EINVAL;
+ }
+ *event_field++ = '\0';
+
+ /* Check the optional assign field. */
+ assign_field = strchr(type_name, ',');
+ if (assign_field)
+ *assign_field++ = '\0';
+
+ /* Get the type field name. */
+ type_field = strstr(event_field, "->");
+ if (!type_field) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (event_field - buf),
+ TYPECAST_REQ_FIELD);
+ return -EINVAL;
+ }
+ *type_field = '\0';
+ type_field += 2;
+
+ offs = strpbrk(type_field, "+-");
+ if (offs) {
+ if (kstrtol(offs, 0, &val) < 0) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (offs - buf),
+ BAD_DEREF_OFFS);
+ return -EINVAL;
+ }
+ *offs = '\0';
+ }
+
+ /* find type from BTF */
+ id = bpf_find_btf_id(type_name, BTF_KIND_STRUCT, &btf);
+ if (id < 0) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (type_name - buf),
+ BAD_BTF_TID);
+ return id;
+ }
+
+ type = btf_type_by_id(btf, id);
+ if (!type) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (type_name - buf),
+ BAD_BTF_TID);
+ return -EINVAL;
+ }
+
+ adjust = get_offset_of_field(btf, type, type_field);
+ if (adjust < 0) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (type_field - buf),
+ NO_BTF_FIELD);
+ return adjust;
+ }
+ wprobe_data->adjust = adjust + val;
+
+ if (assign_field) {
+ /* assign_field should be a struct field */
+ adjust = get_offset_of_field(btf, type, assign_field);
+ if (adjust < 0) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (assign_field - buf),
+ NO_BTF_FIELD);
+ return adjust;
+ }
+ wprobe_data->adjust -= adjust;
+ }
+
+ event_field_offset = base_offset + (event_field - buf);
+ field = trace_find_event_field(file->event_call, event_field);
+ if (!field) {
+ wprobe_trigger_log_err(file, glob, event_field_offset, NO_EVENT_FIELD);
+ return -ENOENT;
+ }
+ if (field->size != sizeof(void *)) {
+ wprobe_trigger_log_err(file, glob, event_field_offset, WPROBE_BAD_FIELD);
+ return -ENOEXEC;
+ }
+ wprobe_data->offset = field->offset;
+ wprobe_data->field = kstrdup(event_field, GFP_KERNEL);
+ if (!wprobe_data->field)
+ return -ENOMEM;
+
+ return 0;
+}
+#else
+static int wprobe_trigger_typecast_parse(char *field_str,
+ struct trace_event_file *file,
+ struct wprobe_trigger_data *wprobe_data,
+ const char *glob)
+{
+ wprobe_trigger_log_err(file, glob, field_str - glob, NOSUP_BTFARG);
+ return -EOPNOTSUPP;
+}
+#endif /* CONFIG_PROBE_EVENTS_BTF_ARGS */
+
+static int wprobe_trigger_field_parse(char *field_str, struct trace_event_file *file,
+ struct wprobe_trigger_data *wprobe_data,
+ const char *glob)
+{
+ struct ftrace_event_field *field;
+ char *offs;
+
+ if (field_str[0] == '(')
+ return wprobe_trigger_typecast_parse(field_str, file, wprobe_data, glob);
+
+ offs = strpbrk(field_str, "+-");
+ if (offs) {
+ long val;
+
+ if (kstrtol(offs, 0, &val) < 0) {
+ wprobe_trigger_log_err(file, glob, offs - glob, BAD_DEREF_OFFS);
+ return -EINVAL;
+ }
+ wprobe_data->adjust = val;
+ *offs = '\0';
+ }
+
+ field = trace_find_event_field(file->event_call, field_str);
+ if (!field) {
+ wprobe_trigger_log_err(file, glob, field_str - glob, NO_EVENT_FIELD);
+ return -ENOENT;
+ }
+ if (field->size != sizeof(void *)) {
+ wprobe_trigger_log_err(file, glob, field_str - glob, WPROBE_BAD_FIELD);
+ return -ENOEXEC;
+ }
+ wprobe_data->offset = field->offset;
+ wprobe_data->field = kstrdup(field_str, GFP_KERNEL);
+ if (!wprobe_data->field)
+ return -ENOMEM;
+
+ return 0;
+}
+
static int wprobe_trigger_cmd_parse(struct event_command *cmd_ops,
struct trace_event_file *file,
char *glob, char *cmd,
@@ -1206,10 +1424,8 @@ static int wprobe_trigger_cmd_parse(struct event_command *cmd_ops,
struct wprobe_trigger_data *wprobe_data = NULL;
struct event_trigger_data *trigger_data = NULL;
struct trace_event_file *wprobe_file;
+ char *event_str, *comment;
struct trace_array *tr = file->tr;
- char *event_str, *field_str, *comment;
- struct ftrace_event_field *field;
- struct trace_event_call *event;
bool remove, clear = false;
unsigned long orig_addr = 0;
struct trace_wprobe *tw;
@@ -1270,93 +1486,25 @@ static int wprobe_trigger_cmd_parse(struct event_command *cmd_ops,
/* clear_wprobe does not need field, but can have optional field. */
if (!clear) {
- char *offs;
+ char *field_str = strsep(¶m, ":");
- /* Find target field, which must be equivalent to "void *" */
- field_str = strsep(¶m, ":");
if (!field_str) {
wprobe_trigger_log_err(file, glob, strlen(glob), WPROBE_NEED_FIELD);
ret = -EINVAL;
goto out_free;
}
-
- offs = strpbrk(field_str, "+-");
- if (offs) {
- long val;
-
- if (kstrtol(offs, 0, &val) < 0) {
- wprobe_trigger_log_err(file, glob, offs - glob, BAD_DEREF_OFFS);
- ret = -EINVAL;
- goto out_free;
- }
- wprobe_data->adjust = val;
- *offs = '\0';
- }
-
- event = file->event_call;
- field = trace_find_event_field(event, field_str);
- if (!field) {
- wprobe_trigger_log_err(file, glob, field_str - glob, NO_EVENT_FIELD);
- ret = -ENOENT;
- goto out_free;
- }
-
- if (field->size != sizeof(void *)) {
- wprobe_trigger_log_err(file, glob, field_str - glob, WPROBE_BAD_FIELD);
- ret = -ENOEXEC;
- goto out_free;
- }
- wprobe_data->offset = field->offset;
- wprobe_data->field = kstrdup(field_str, GFP_KERNEL);
- if (!wprobe_data->field) {
- ret = -ENOMEM;
+ ret = wprobe_trigger_field_parse(field_str, file, wprobe_data, glob);
+ if (ret < 0)
goto out_free;
- }
- } else if (param && (isalpha(param[0]) || param[0] == '_')) {
+ } else if (param && (isalpha(param[0]) || param[0] == '_' || param[0] == '(')) {
if (strncmp(param, "count=", 6) != 0 &&
(strcmp(param, "unlimited") != 0 ||
trace_find_event_field(file->event_call, "unlimited"))) {
- char *offs;
-
- field_str = strsep(¶m, ":");
- offs = strpbrk(field_str, "+-");
- if (offs) {
- long val;
-
- if (kstrtol(offs, 0, &val) < 0) {
- wprobe_trigger_log_err(file, glob,
- offs - glob,
- BAD_DEREF_OFFS);
- ret = -EINVAL;
- goto out_free;
- }
- wprobe_data->adjust = val;
- *offs = '\0';
- }
-
- event = file->event_call;
- field = trace_find_event_field(event, field_str);
- if (!field) {
- wprobe_trigger_log_err(file, glob,
- field_str - glob,
- NO_EVENT_FIELD);
- ret = -ENOENT;
- goto out_free;
- }
+ char *field_str = strsep(¶m, ":");
- if (field->size != sizeof(void *)) {
- wprobe_trigger_log_err(file, glob,
- field_str - glob,
- WPROBE_BAD_FIELD);
- ret = -ENOEXEC;
+ ret = wprobe_trigger_field_parse(field_str, file, wprobe_data, glob);
+ if (ret < 0)
goto out_free;
- }
- wprobe_data->offset = field->offset;
- wprobe_data->field = kstrdup(field_str, GFP_KERNEL);
- if (!wprobe_data->field) {
- ret = -ENOMEM;
- goto out_free;
- }
}
}
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc
new file mode 100644
index 000000000000..eda6584c286f
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc
@@ -0,0 +1,85 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test set_wprobe trigger with BTF struct offset
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "[(structname[,field])]<argname>[->field[->field|.field...]]":README "set_wprobe:<wprobe-event>":README
+
+fail() { #msg
+ echo "$1"
+ exit_fail
+}
+
+rmmod trace-events-sample ||:
+if ! modprobe trace-events-sample ; then
+ echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m"
+ exit_unresolved
+fi
+
+cleanup_wprobe_triggers() {
+ if [ -f events/fprobes/testevent/trigger ]; then
+ reset_trigger_file events/fprobes/testevent/trigger || true
+ fi
+ echo 0 > events/enable 2>/dev/null || true
+ echo > dynamic_events 2>/dev/null || true
+ sleep 1
+ rmmod trace-events-sample 2>/dev/null || true
+ return 0
+}
+
+trap cleanup_wprobe_triggers EXIT
+
+echo 0 > tracing_on
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "sample_timer_cb" /proc/kallsyms; then
+ echo "UNRESOLVED: sample_timer_cb not found"
+ exit_unresolved
+fi
+
+:;: "Add a wprobe event watching 8 bytes" ;:
+echo 'w:watch rw@-1:8 address=$addr value=$value' >> dynamic_events
+
+:;: "Add fprobe event for sample_timer_cb" ;:
+# sample_timer_cb(struct timer_list *t)
+# container_of(t, struct foo_timer_data, timer)
+echo 'f:fprobes/testevent sample_timer_cb timer=t' >> dynamic_events
+
+:;: "Enable all events before setting triggers" ;:
+echo 1 > tracing_on
+echo 1 >> events/fprobes/testevent/enable
+
+:;: "Set set_wprobe trigger using BTF struct offset resolution" ;:
+# Syntax: set_wprobe:WPEVENT:(STRUCT,FIELD)EVENT_FIELD->MEMBER
+# (foo_timer_data,timer) is the BTF struct type and field name
+# timer->expires is the struct member whose offset is resolved automatically via BTF
+echo 'set_wprobe:watch:(foo_timer_data,timer)timer->timer.expires' >> events/fprobes/testevent/trigger
+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to set set_wprobe trigger"
+fi
+
+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate
+sleep 3
+
+:;: "Check set_wprobe trigger activated the watchpoint" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint"
+fi
+
+:;: "Remove wprobe triggers" ;:
+# Since we don't know actual offset of timer->expires in foo_timer_data, we use reset_trigger_file
+reset_trigger_file events/fprobes/testevent/trigger
+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to remove set_wprobe trigger"
+fi
+
+:;: "Disable events and remove dynamic events" ;:
+echo 0 > events/enable
+echo > dynamic_events
+clear_trace
+
+exit 0
^ permalink raw reply [flat|nested] 11+ messages in thread