[PATCH] arm64: Add user and kernel page-fault tracepoints

[PATCH] arm64: Add user and kernel page-fault tracepoints

[Justinien Bouron]

Those tracepoints were made generic in commit 06aa9378df01
("x86/tracing, x86/mm: Move page fault tracepoints to generic"), call
them from arm64's page fault handling routine.

Signed-off-by: Justinien Bouron <jbouron@amazon.com>
Reviewed-by: Gunnar Kudrjavets <gunnarku@amazon.com>
---
Benchmarking
============

Since the added code sits in the page-fault handling hot-path, I've also
ran some benchmarks to make sure this change is not accidentally
introducing a regression. All testing below was done on a c6g.metal EC2
instance (i.e. bare-metal) which is an ARM64 CPU with 64 cores and
128GiB of RAM.

First, a microbenchmark that maps anonymous memory and write to the
first four bytes of each page, thus running into page-fault for each
one:
```
#include <stdio.h>
#include <sys/mman.h>
#include <stdint.h>
#include <stdlib.h>
#include <unistd.h>
#include <assert.h>

int main(int argc, char **argv) {
    int npages = atoi(argv[1]);
    size_t size = npages * 4096;
    printf("pid = %d\n", getpid());
    char *p = mmap((void*)0xcafe0000,
            size,
            PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED,
            -1, 0);
    assert(p != MAP_FAILED);
    for (int i = 0; i < npages; i++)
        *(uint32_t *)(p + i * 4096) = 0xdeadbeef;
    munmap(p, size);
}
```

Compiled with: `gcc microbench.c -no-pie -o microbench` (GCC 15.2.0)

Before the change:
    $ hyperfine --runs 50 './microbench 100000'
    Benchmark 1: ./microbench 100000
      Time (mean +/- stddev):     174.3 ms +/-   3.4 ms    [User: 5.9 ms, System: 168.3 ms]
      Range (min ... max):   168.7 ms ... 185.1 ms    50 runs
After the change:
    $ hyperfine --runs 50 './microbench_orig 100000'
    Benchmark 1: ./microbench 100000
      Time (mean +/- stddev):     177.9 ms +/-   3.1 ms    [User: 6.3 ms, System: 171.6 ms]
      Range (min ... max):   173.3 ms ... 186.0 ms    50 runs

Second benchmark, linux kernel compilation benchmark with defconfig (on
commit 70390501):

Before the change:
    $ hyperfine --runs 3 --prepare 'make clean && sync && echo 3 | sudo tee /proc/sys/vm/drop_caches' 'make -j65'
    Benchmark 1: make -j65
      Time (mean +/- stddev):     155.744 s +/-  2.767 s    [User: 6812.900 s, System: 1156.095 s]
      Range (min ... max):   153.709 s ... 158.895 s    3 runs

After the change:
    $ hyperfine --runs 3 --prepare 'make clean && sync && echo 3 | sudo tee /proc/sys/vm/drop_caches' 'make -j65'
    Benchmark 1: make -j65
      Time (mean +/- stddev):     153.315 s +/-  0.179 s    [User: 6799.948 s, System: 1148.976 s]
      Range (min ... max):   153.119 s ... 153.471 s    3 runs

Testing
=======

Re-using the microbenchmark above, we can see the page-faults generated
by writing to each mmap'ed page (mmap'ed region starts at address
0xcafe0000):
```
$ sudo perf record -e exceptions:page_fault_user -- ./microbench 16
pid = 307352
[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote 0.015 MB perf.data (63 samples) ]
$ sudo perf script
      microbench  307352 [034] 29395.753387: exceptions:page_fault_user: address=0xfffff7fda640 ip=0xfffff7fda640 error_code=0x82000007
      microbench  307352 [034] 29395.753406: exceptions:page_fault_user: address=0xfffff7fff066 ip=0xfffff7fd60c4 error_code=0x92000007
      microbench  307352 [034] 29395.753407: exceptions:page_fault_user: address=0xfffff7fff066 ip=0xfffff7fd60cc error_code=0x9200004f
      microbench  307352 [034] 29395.753410: exceptions:page_fault_user: address=0xfffff7ffde28 ip=0xfffff7fd60d0 error_code=0x92000007
      microbench  307352 [034] 29395.753412: exceptions:page_fault_user: address=0xfffff7ffde70 ip=0xfffff7fd619c error_code=0x9200004f
      microbench  307352 [034] 29395.753416: exceptions:page_fault_user: address=0xfffff7fbeb38 ip=0xfffff7fd6304 error_code=0x92000007
      microbench  307352 [034] 29395.753419: exceptions:page_fault_user: address=0xfffff7ffcd28 ip=0xfffff7fd62f8 error_code=0x9200004f
      microbench  307352 [034] 29395.753422: exceptions:page_fault_user: address=0xffffffffeff0 ip=0xfffff7fcfa90 error_code=0x92000047
      microbench  307352 [034] 29395.753428: exceptions:page_fault_user: address=0xfffff7fe1b90 ip=0xfffff7fdc9b0 error_code=0x92000007
      microbench  307352 [034] 29395.753431: exceptions:page_fault_user: address=0x400040 ip=0xfffff7fd7160 error_code=0x92000007
      microbench  307352 [034] 29395.753432: exceptions:page_fault_user: address=0x41fdf8 ip=0xfffff7fd72f0 error_code=0x92000007
      microbench  307352 [034] 29395.753434: exceptions:page_fault_user: address=0xfffff7ffa020 ip=0xfffff7fd7548 error_code=0x92000007
      microbench  307352 [034] 29395.753437: exceptions:page_fault_user: address=0xfffff7ffb000 ip=0xfffff7fd7640 error_code=0x92000007
      microbench  307352 [034] 29395.753443: exceptions:page_fault_user: address=0xfffff7ff4008 ip=0xfffff7fc4848 error_code=0x92000047
      microbench  307352 [034] 29395.753446: exceptions:page_fault_user: address=0x41fec0 ip=0xfffff7fd7cd0 error_code=0x9200004f
      microbench  307352 [034] 29395.753464: exceptions:page_fault_user: address=0xfffff7fed000 ip=0xfffff7fd11c4 error_code=0x92000007
      microbench  307352 [034] 29395.753502: exceptions:page_fault_user: address=0xfffff7fa14d8 ip=0xfffff7fdcbe0 error_code=0x92000047
      microbench  307352 [034] 29395.753509: exceptions:page_fault_user: address=0xfffff7f9fa88 ip=0xfffff7fc54c0 error_code=0x92000007
      microbench  307352 [034] 29395.753512: exceptions:page_fault_user: address=0xfffff7f9fb20 ip=0xfffff7fc5534 error_code=0x9200004f
      microbench  307352 [034] 29395.753515: exceptions:page_fault_user: address=0xfffff7de02e0 ip=0xfffff7fc5704 error_code=0x92000006
      microbench  307352 [034] 29395.753519: exceptions:page_fault_user: address=0xfffff7f887c8 ip=0xfffff7fc4bb0 error_code=0x92000007
      microbench  307352 [034] 29395.753522: exceptions:page_fault_user: address=0xfffff7dffcef ip=0xfffff7fdc358 error_code=0x92000007
      microbench  307352 [034] 29395.753526: exceptions:page_fault_user: address=0xfffff7e01650 ip=0xfffff7fd027c error_code=0x92000007
      microbench  307352 [034] 29395.753530: exceptions:page_fault_user: address=0xfffff7ff5610 ip=0xfffff7fdcc10 error_code=0x92000047
      microbench  307352 [034] 29395.753534: exceptions:page_fault_user: address=0xfffff7f9d080 ip=0xfffff7fcb6a4 error_code=0x9200004f
      microbench  307352 [034] 29395.753538: exceptions:page_fault_user: address=0xfffff7f9e000 ip=0xfffff7fcb6fc error_code=0x9200004f
      microbench  307352 [034] 29395.753542: exceptions:page_fault_user: address=0xfffff7fa02d8 ip=0xfffff7fcb6a4 error_code=0x9200004f
      microbench  307352 [034] 29395.753568: exceptions:page_fault_user: address=0xfffff7e821c0 ip=0xfffff7e821c0 error_code=0x82000007
      microbench  307352 [034] 29395.753585: exceptions:page_fault_user: address=0xfffff7feb000 ip=0xfffff7fc1a40 error_code=0x92000047
      microbench  307352 [034] 29395.753594: exceptions:page_fault_user: address=0xfffff7f2b1e0 ip=0xfffff7f2b1e0 error_code=0x82000007
      microbench  307352 [034] 29395.753596: exceptions:page_fault_user: address=0xfffff7e1188c ip=0xfffff7e1188c error_code=0x82000007
      microbench  307352 [034] 29395.753597: exceptions:page_fault_user: address=0xfffff7fa7458 ip=0xfffff7f2b21c error_code=0x92000047
      microbench  307352 [034] 29395.753600: exceptions:page_fault_user: address=0xfffff7fade39 ip=0xfffff7f2b224 error_code=0x92000047
      microbench  307352 [034] 29395.753601: exceptions:page_fault_user: address=0xfffff7ecdd20 ip=0xfffff7ecdd20 error_code=0x82000007
      microbench  307352 [034] 29395.753603: exceptions:page_fault_user: address=0xfffff7fa6518 ip=0xfffff7f2b2a0 error_code=0x92000047
      microbench  307352 [034] 29395.753605: exceptions:page_fault_user: address=0xfffff7e6ec50 ip=0xfffff7e6ec50 error_code=0x82000007
      microbench  307352 [034] 29395.753607: exceptions:page_fault_user: address=0xfffff7fa2180 ip=0xfffff7e1c918 error_code=0x92000047
      microbench  307352 [034] 29395.753609: exceptions:page_fault_user: address=0xfffff7e7dac8 ip=0xfffff7e7dac8 error_code=0x82000007
      microbench  307352 [034] 29395.753623: exceptions:page_fault_user: address=0xfffff7ed6720 ip=0xfffff7ed6720 error_code=0x82000007
      microbench  307352 [034] 29395.753626: exceptions:page_fault_user: address=0xfffff7e28220 ip=0xfffff7e28220 error_code=0x82000007
      microbench  307352 [034] 29395.753627: exceptions:page_fault_user: address=0xfffff7f349e2 ip=0xfffff7e282b8 error_code=0x92000007
      microbench  307352 [034] 29395.753629: exceptions:page_fault_user: address=0xfffff7ead1c0 ip=0xfffff7ead1c0 error_code=0x82000007
      microbench  307352 [034] 29395.753631: exceptions:page_fault_user: address=0xfffff7e39e40 ip=0xfffff7e39e40 error_code=0x82000007
      microbench  307352 [034] 29395.753633: exceptions:page_fault_user: address=0xfffff7f453c4 ip=0xfffff7e37c58 error_code=0x92000007
      microbench  307352 [034] 29395.753635: exceptions:page_fault_user: address=0xfffff7f53412 ip=0xfffff7e2e788 error_code=0x92000007
      microbench  307352 [034] 29395.753636: exceptions:page_fault_user: address=0xfffff7e53bc0 ip=0xfffff7e53bc0 error_code=0x82000007
      microbench  307352 [034] 29395.753643: exceptions:page_fault_user: address=0x421008 ip=0xfffff7e7bbc8 error_code=0x92000047
      microbench  307352 [034] 29395.753658: exceptions:page_fault_user: address=0xcafe0000 ip=0x4008e4 error_code=0x92000045
      microbench  307352 [034] 29395.753663: exceptions:page_fault_user: address=0xcafe1000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753664: exceptions:page_fault_user: address=0xcafe2000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753666: exceptions:page_fault_user: address=0xcafe3000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753675: exceptions:page_fault_user: address=0xcafe4000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753676: exceptions:page_fault_user: address=0xcafe5000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753678: exceptions:page_fault_user: address=0xcafe6000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753679: exceptions:page_fault_user: address=0xcafe7000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753681: exceptions:page_fault_user: address=0xcafe8000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753682: exceptions:page_fault_user: address=0xcafe9000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753684: exceptions:page_fault_user: address=0xcafea000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753686: exceptions:page_fault_user: address=0xcafeb000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753687: exceptions:page_fault_user: address=0xcafec000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753689: exceptions:page_fault_user: address=0xcafed000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753691: exceptions:page_fault_user: address=0xcafee000 ip=0x4008e4 error_code=0x92000047
      microbench  307352 [034] 29395.753692: exceptions:page_fault_user: address=0xcafef000 ip=0x4008e4 error_code=0x92000047
```

IP 0x4008e4 corresponds to the instruction where the write to each page
is performed:
```
   4008c8:       b9402be0        ldr     w0, [sp, #40]
   4008cc:       53144c00        lsl     w0, w0, #12
   4008d0:       93407c00        sxtw    x0, w0
   4008d4:       f9401fe1        ldr     x1, [sp, #56]
   4008d8:       8b000020        add     x0, x1, x0
   4008dc:       5297dde1        mov     w1, #0xbeef                     // #48879
   4008e0:       72bbd5a1        movk    w1, #0xdead, lsl #16
=> 4008e4:       b9000001        str     w1, [x0]
   4008e8:       b9402be0        ldr     w0, [sp, #40]
   4008ec:       11000400        add     w0, w0, #0x1
   4008f0:       b9002be0        str     w0, [sp, #40]
   4008f4:       b9402be1        ldr     w1, [sp, #40]
   4008f8:       b9402fe0        ldr     w0, [sp, #44]
   4008fc:       6b00003f        cmp     w1, w0
   400900:       54fffe4b        b.lt    4008c8 <main+0x9c>  // b.tstop
```

First error_code when writing to the mmap'ed region is 0x92000045 which,
as expected, corresponds to user-mode, write fault, at page-table level
1 (see https://esr.arm64.dev/#0x92000045). Following faults have similar
error_code but for page-table level 3 fault
(https://esr.arm64.dev/#0x92000047).

For kernel page-faults, tested using tracepoints directly:
```
# echo 1 > events/exceptions/page_fault_kernel/enable
# echo test
test
# cat trace_pipe
    ps-307392  [028] ..... 29802.331281: page_fault_kernel: address=0xaaaaaaae0280 ip=__arch_clear_user error_code=0x96000044
    ps-307392  [028] ..... 29802.331303: page_fault_kernel: address=0xfffff7ffe8e4 ip=__arch_clear_user error_code=0x96000046
    ps-307392  [028] ..... 29802.331851: page_fault_kernel: address=0xfffff7f13360 ip=strncpy_from_user error_code=0x96000007
    ps-307392  [028] ..... 29802.349787: page_fault_kernel: address=0xaaaaaab23000 ip=__arch_copy_to_user error_code=0x96000047
    ...
```

---
 arch/arm64/mm/fault.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 0f3c5c7ca054..e75cc0348e74 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -45,6 +45,9 @@
 #include <asm/traps.h>
 #include <asm/virt.h>
 
+#define CREATE_TRACE_POINTS
+#include <trace/events/exceptions.h>
+
 struct fault_info {
 	int	(*fn)(unsigned long far, unsigned long esr,
 		      struct pt_regs *regs);
@@ -606,6 +609,11 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr,
 	int si_code;
 	int pkey = -1;
 
+	if (user_mode(regs))
+		trace_page_fault_user(addr, regs, esr);
+	else
+		trace_page_fault_kernel(addr, regs, esr);
+
 	if (kprobe_page_fault(regs, esr))
 		return 0;
 
-- 
2.53.0

Re: [PATCH] arm64: Add user and kernel page-fault tracepoints

[Leo Yan]

On Tue, May 19, 2026 at 09:55:24PM -0700, Justinien Bouron wrote:

[...]

> @@ -606,6 +609,11 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr,
>  	int si_code;
>  	int pkey = -1;
>  
> +	if (user_mode(regs))
> +		trace_page_fault_user(addr, regs, esr);
> +	else
> +		trace_page_fault_kernel(addr, regs, esr);

Based on the discussion [1], Arm64 has already supported perf sw event
for page-faults:

  perf record -e page-faults ...

Seems there have a plan to consolidate perf event and tracepoints but I
have no idea how it is going. I would leave this to maintainers.

> +
>  	if (kprobe_page_fault(regs, esr))
>  		return 0;

tracepoints should be after kprobe_page_fault(), as explained [2] by Mark.

Thanks,
Leo

[1] https://lore.kernel.org/all/20250520140453.GA18711@willie-the-truck/
[2] https://lore.kernel.org/all/aCtZfiU8bgkSAgLh@J2N7QTR9R3/

Re: [PATCH] arm64: Add user and kernel page-fault tracepoints

[Justinien Bouron]

On Wed, May 20, 2026 at 08:36:06AM +0100, Leo Yan wrote:
> On Tue, May 19, 2026 at 09:55:24PM -0700, Justinien Bouron wrote:
> 
> [...]
> 
> > @@ -606,6 +609,11 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr,
> >  	int si_code;
> >  	int pkey = -1;
> >  
> > +	if (user_mode(regs))
> > +		trace_page_fault_user(addr, regs, esr);
> > +	else
> > +		trace_page_fault_kernel(addr, regs, esr);
> 
> Based on the discussion [1], Arm64 has already supported perf sw event
> for page-faults:
> 
>   perf record -e page-faults ...
> 
> Seems there have a plan to consolidate perf event and tracepoints but I
> have no idea how it is going.
This refactor/consolidation is not specific to arm64 right? I see that both x86
and riscv also have both the tracepoints and the perf event, presumably they
will need to be refactored as well?

> I would leave this to maintainers.
Agreed.

> 
> > +
> >  	if (kprobe_page_fault(regs, esr))
> >  		return 0;
> 
> tracepoints should be after kprobe_page_fault(), as explained [2] by Mark.
Interesting. The reason I put them before the kprobe_page_fault is because this
is how x86 is doing it. x86 calls kprobe_page_fault from do_kern_addr_fault /
do_user_addr_fault which are called _after_ the trace_page_fault_{user,kernel}.
Is there a reason why this is allowed in x86 but not arm64?

Also I did not realize that there already was an attempt to add the page-fault
tracepoints in the past!

Best,
Justinien
> 
> Thanks,
> Leo
> 
> [1] https://lore.kernel.org/all/20250520140453.GA18711@willie-the-truck/
> [2] https://lore.kernel.org/all/aCtZfiU8bgkSAgLh@J2N7QTR9R3/

Re: [PATCH] arm64: Add user and kernel page-fault tracepoints

[Bouron, Justinien]

Leo,

Thank you for the feedback. I've posted a second revision at
https://lore.kernel.org/lkml/20260902115216.49158-1-jbouron@amazon.com/, though
it doesn't seem to be getting much traction.

Thanks,
Justinien

Re: [PATCH] arm64: Add user and kernel page-fault tracepoints

[Leo Yan]

Hi Justinien,

On Wed, Sep 09, 2026 at 01:13:42PM +0000, Bouron, Justinien wrote:
> Leo,
> 
> Thank you for the feedback. I've posted a second revision at
> https://lore.kernel.org/lkml/20260902115216.49158-1-jbouron@amazon.com/, though
> it doesn't seem to be getting much traction.

This should be in the arm64 maintainers' scope (v1 caught my attention
due to it mentioned perf related things).

I think the main question is these tracepoints overlap with perf sw
events. Could you elaborate on why ftrace tracepoints are necessary?
('perf record' and 'perf script' can provide similar info and we can
avoid extra latency).

If the maintainers think this patch is reasonable, I'd be happy to help
test it.

Thanks,
Leo