* [PATCH v7 01/11] arm64, unwind: build kernel with sframe V3 info
2026-09-18 22:41 [PATCH v7 00/11] unwind, arm64: add sframe unwinder for kernel Dylan Hatch
@ 2026-09-18 22:41 ` Dylan Hatch
2026-09-18 22:41 ` [PATCH v7 02/11] arm64/sframe: Read vmlinux .sframe header Dylan Hatch
` (9 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Dylan Hatch @ 2026-09-18 22:41 UTC (permalink / raw)
To: Roman Gushchin, Weinan Liu, Will Deacon, Josh Poimboeuf,
Indu Bhagat, Peter Zijlstra, Steven Rostedt, Catalin Marinas,
Jiri Kosina, Mark Rutland, Jens Remus
Cc: Dylan Hatch, Prasanna Kumar T S M, Puranjay Mohan, Song Liu,
joe.lawrence, linux-toolchains, linux-kernel, live-patching,
linux-arm-kernel, Randy Dunlap, Mostafa Saleh, Herbert Xu,
David S. Miller
To generate a .sframe section, build with -Wa,--gsframe-3 flags. Also,
add unwind-table options to ensure the necessary CFI directives are
emitted by the compiler, but discard .eh_frame unless
CONFIG_UNWIND_TABLES is explicitly enabled.
This will be used for in-kernel reliable stacktrace in
cases where the frame pointer alone is insufficient.
Currently, the sframe format only supports arm64, x86_64 and s390x
architectures.
Co-developed-by: Weinan Liu <wnliu@google.com>
Signed-off-by: Weinan Liu <wnliu@google.com>
Reviewed-by: Prasanna Kumar T S M <ptsm@linux.microsoft.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Dylan Hatch <dylanbhatch@google.com>
---
Changes since v6:
- Moved to beginning of series
- Squashed in AS_SFRAME, AS_SFRAME3, and UNWIND_SFRAME_LOOKUP config
definitions.
- Squashed in STACK UNWINDING change in MAINTAINERS file.
- Dropped SFRAME_REG_SP / SFRAME_REG_FP definitions.
- Added -fasynchronous-unwind-tables so CFI directives are emitted by
compiler.
- Discard .eh_frame sections when CONFIG_UNWIND_TABLES is not set.
---
MAINTAINERS | 3 ++-
Makefile | 8 ++++++++
arch/Kconfig | 31 +++++++++++++++++++++++++++++
arch/arm64/Kconfig | 1 +
arch/arm64/Makefile | 2 +-
arch/arm64/include/asm/module.lds.h | 2 ++
arch/arm64/kernel/vdso/Makefile | 2 +-
arch/arm64/kernel/vmlinux.lds.S | 12 +++++++++++
include/asm-generic/sections.h | 4 ++++
include/asm-generic/vmlinux.lds.h | 15 ++++++++++++++
10 files changed, 77 insertions(+), 3 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index c2414447892c2..6c1c95979e437 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -28514,10 +28514,11 @@ F: Documentation/driver-api/uio-howto.rst
F: drivers/uio/
F: include/linux/uio_driver.h
-USERSPACE STACK UNWINDING
+STACK UNWINDING
M: Josh Poimboeuf <jpoimboe@kernel.org>
M: Steven Rostedt <rostedt@goodmis.org>
S: Maintained
+F: arch/*/include/asm/unwind_sframe.h
F: include/linux/unwind*.h
F: kernel/unwind/
diff --git a/Makefile b/Makefile
index 0f1b80100b479..5ed4a59e71aac 100644
--- a/Makefile
+++ b/Makefile
@@ -1197,6 +1197,14 @@ CFLAGS_GCOV += $(call try-run,\
endif
export CFLAGS_GCOV
+# build with sframe table
+ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME
+CC_FLAGS_SFRAME := -Wa,--gsframe-3
+KBUILD_CFLAGS += $(CC_FLAGS_SFRAME)
+KBUILD_AFLAGS += $(CC_FLAGS_SFRAME)
+export CC_FLAGS_SFRAME
+endif
+
# change __FILE__ to the relative path to the source directory
ifdef building_out_of_srctree
KBUILD_CPPFLAGS += -fmacro-prefix-map=$(srcroot)/=
diff --git a/arch/Kconfig b/arch/Kconfig
index 45c6577723623..cc59a5da80915 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -507,6 +507,37 @@ config HAVE_UNWIND_USER_FP
bool
select UNWIND_USER
+config AS_SFRAME
+ bool
+
+config AS_SFRAME3
+ def_bool $(as-instr,.cfi_startproc\n.cfi_endproc,-Wa$(comma)--gsframe-3)
+ select AS_SFRAME
+
+config UNWIND_SFRAME_LOOKUP
+ bool
+
+config ARCH_SUPPORTS_UNWIND_KERNEL_SFRAME
+ bool
+ help
+ An architecture can select this if it enables the SFrame (Simple
+ Frame) unwinder for unwinding kernel stack traces. It uses an unwind
+ table that is directly generated by the toolchain based on DWARF CFI
+ information.
+
+config HAVE_UNWIND_KERNEL_SFRAME
+ bool "Sframe unwinder"
+ depends on AS_SFRAME3
+ depends on 64BIT
+ depends on ARCH_SUPPORTS_UNWIND_KERNEL_SFRAME
+ select UNWIND_SFRAME_LOOKUP
+ help
+ This option enables the SFrame (Simple Frame) unwinder for unwinding
+ kernel stack traces. It uses an unwind table that is directly
+ generated by the toolchain based on DWARF CFI information. In
+ practice, this can provide more reliable stacktrace results than
+ unwinding with frame pointers alone.
+
config HAVE_PERF_REGS
bool
help
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index b5a51b0ef9440..041de63a659d1 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -87,6 +87,7 @@ config ARM64
select ARCH_SUPPORTS_SCHED_SMT
select ARCH_SUPPORTS_SCHED_CLUSTER
select ARCH_SUPPORTS_SCHED_MC
+ select ARCH_SUPPORTS_UNWIND_KERNEL_SFRAME
select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH
select ARCH_WANT_COMPAT_IPC_PARSE_VERSION if COMPAT
select ARCH_WANT_DEFAULT_BPF_JIT
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 6b005c8fef706..ae1c23579ad2f 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -55,7 +55,7 @@ KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
# Avoid generating .eh_frame* sections.
-ifneq ($(CONFIG_UNWIND_TABLES),y)
+ifeq ($(filter y, $(CONFIG_UNWIND_TABLES) $(CONFIG_HAVE_UNWIND_KERNEL_SFRAME)),)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables -fno-unwind-tables
KBUILD_AFLAGS += -fno-asynchronous-unwind-tables -fno-unwind-tables
KBUILD_RUSTFLAGS += -Cforce-unwind-tables=n
diff --git a/arch/arm64/include/asm/module.lds.h b/arch/arm64/include/asm/module.lds.h
index 603f92a8a73ed..034cb635e8bcc 100644
--- a/arch/arm64/include/asm/module.lds.h
+++ b/arch/arm64/include/asm/module.lds.h
@@ -10,5 +10,7 @@ SECTIONS {
* put it into the .init allocation.
*/
.init.eh_frame 0 : { *(.eh_frame) }
+#elif defined(CONFIG_HAVE_UNWIND_KERNEL_SFRAME)
+ /DISCARD/ : { *(.eh_frame) }
#endif
}
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index 7dec05dd33b70..c60ef921956fb 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -38,7 +38,7 @@ ccflags-y += -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO
CC_FLAGS_REMOVE_VDSO := $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) \
$(RANDSTRUCT_CFLAGS) $(KSTACK_ERASE_CFLAGS) \
$(GCC_PLUGINS_CFLAGS) \
- $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) \
+ $(CC_FLAGS_LTO) $(CC_FLAGS_CFI) $(CC_FLAGS_SFRAME) \
-Wmissing-prototypes -Wmissing-declarations
CC_FLAGS_ADD_VDSO := -O2 -mcmodel=tiny -fasynchronous-unwind-tables
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index af1d720209764..eb1f54829503c 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -145,14 +145,25 @@ jiffies = jiffies_64;
#endif
#ifdef CONFIG_UNWIND_TABLES
+
#define UNWIND_DATA_SECTIONS \
.eh_frame : { \
__pi___eh_frame_start = .; \
*(.eh_frame) \
__pi___eh_frame_end = .; \
}
+#define SFRAME_DISCARDS
+
+#elif defined(CONFIG_HAVE_UNWIND_KERNEL_SFRAME)
+
+#define UNWIND_DATA_SECTIONS
+#define SFRAME_DISCARDS *(.eh_frame)
+
#else
+
#define UNWIND_DATA_SECTIONS
+#define SFRAME_DISCARDS
+
#endif
/*
@@ -184,6 +195,7 @@ SECTIONS
*(.interp .dynamic)
*(.dynsym .dynstr .hash .gnu.hash)
*(.ARM.attributes)
+ SFRAME_DISCARDS
}
. = KIMAGE_VADDR;
diff --git a/include/asm-generic/sections.h b/include/asm-generic/sections.h
index 0755bc39b0d80..336d27011a580 100644
--- a/include/asm-generic/sections.h
+++ b/include/asm-generic/sections.h
@@ -31,6 +31,7 @@
* __irqentry_text_start, __irqentry_text_end
* __softirqentry_text_start, __softirqentry_text_end
* __start_opd, __end_opd
+ * __start_sframe, __end_sframe
*/
extern char _text[], _stext[], _etext[];
extern char _data[], _sdata[], _edata[];
@@ -53,6 +54,9 @@ extern char __ctors_start[], __ctors_end[];
/* Start and end of .opd section - used for function descriptors. */
extern char __start_opd[], __end_opd[];
+/* Start and end of .sframe section - used for stack unwinding. */
+extern char __start_sframe[], __end_sframe[];
+
/* Start and end of instrumentation protected text section */
extern char __noinstr_text_start[], __noinstr_text_end[];
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index b2988aa12f664..278f5764d990d 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -491,6 +491,8 @@
*(.rodata1) \
} \
\
+ SFRAME \
+ \
/* PCI quirks */ \
.pci_fixup : AT(ADDR(.pci_fixup) - LOAD_OFFSET) { \
BOUNDED_SECTION_PRE_LABEL(.pci_fixup_early, _pci_fixups_early, __start, __end) \
@@ -912,6 +914,19 @@
#define TRACEDATA
#endif
+#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME
+#define SFRAME \
+ /* sframe */ \
+ .sframe : AT(ADDR(.sframe) - LOAD_OFFSET) { \
+ __start_sframe = .; \
+ KEEP(*(.sframe)) \
+ KEEP(*(.init.sframe)) \
+ __end_sframe = .; \
+ }
+#else
+#define SFRAME
+#endif
+
#ifdef CONFIG_PRINTK_INDEX
#define PRINTK_INDEX \
.printk_index : AT(ADDR(.printk_index) - LOAD_OFFSET) { \
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH v7 02/11] arm64/sframe: Read vmlinux .sframe header
2026-09-18 22:41 [PATCH v7 00/11] unwind, arm64: add sframe unwinder for kernel Dylan Hatch
2026-09-18 22:41 ` [PATCH v7 01/11] arm64, unwind: build kernel with sframe V3 info Dylan Hatch
@ 2026-09-18 22:41 ` Dylan Hatch
2026-09-18 22:41 ` [PATCH v7 03/11] sframe: Add support for reading vmlinux .sframe contents Dylan Hatch
` (8 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Dylan Hatch @ 2026-09-18 22:41 UTC (permalink / raw)
To: Roman Gushchin, Weinan Liu, Will Deacon, Josh Poimboeuf,
Indu Bhagat, Peter Zijlstra, Steven Rostedt, Catalin Marinas,
Jiri Kosina, Mark Rutland, Jens Remus
Cc: Dylan Hatch, Prasanna Kumar T S M, Puranjay Mohan, Song Liu,
joe.lawrence, linux-toolchains, linux-kernel, live-patching,
linux-arm-kernel, Randy Dunlap, Mostafa Saleh, Herbert Xu,
David S. Miller
From: Josh Poimboeuf <jpoimboe@kernel.org>
In preparation for unwinding kernel space stacks with sframe, add basic
sframe compile infrastructure and support for reading the .sframe
section header. Add init_sframe_table() to read the vmlinux .sframe
header at boot.
[ Jens Remus: Add support for SFrame V3. Add support for PC-relative
FDE function start offset. Cleanup includes and indentation. ]
[ Dylan Hatch: Repurpose patch for arm64 vmlinux. ]
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Reviewed-by: Indu Bhagat <ibhagatgnu@gmail.com>
Co-developed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
Co-developed-by: Dylan Hatch <dylanbhatch@google.com>
Signed-off-by: Dylan Hatch <dylanbhatch@google.com>
---
This patch is adapted from commit c7db518447c6 ("unwind_user/sframe: Add
support for reading .sframe headers"), and includes the header/init
portions of v6 patch "sframe: Allow kernelspace sframe sections" in this
series.
Changes include:
- kernel/unwind/sframe* is repurposed exclusively for kernel .sframe
sections, under CONFIG_UNWIND_SFRAME_LOOKUP.
- Replaced copy_from_user() with direct memory pointer access when
reading the SFrame header.
- (sashiko) Moved header size check to take place before any header
fields are checked, to ensure safe memory access. Also dropped the
(header + auxiliary) size helper, since non-zero auxiliary size is
not currently supported.
- Dropped unused userspace section tracking functions
sframe_add_section(), sframe_remove_section(), free_section()
- Dropped text_start/text_end section fields, since kernel/module
section boundaries can be examined directly without saving.
- Squashed in changes adding vmlinux-tracking kernel_sfsec and
init_sframe_table(), to read vmlinux .sframe header at boot time.
---
MAINTAINERS | 1 +
arch/arm64/kernel/setup.c | 2 +
include/linux/sframe.h | 32 ++++++++++++++++
kernel/unwind/Makefile | 3 +-
kernel/unwind/sframe.c | 80 +++++++++++++++++++++++++++++++++++++++
kernel/unwind/sframe.h | 78 ++++++++++++++++++++++++++++++++++++++
6 files changed, 195 insertions(+), 1 deletion(-)
create mode 100644 include/linux/sframe.h
create mode 100644 kernel/unwind/sframe.c
create mode 100644 kernel/unwind/sframe.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 6c1c95979e437..eaa691abd7a2e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -28519,6 +28519,7 @@ M: Josh Poimboeuf <jpoimboe@kernel.org>
M: Steven Rostedt <rostedt@goodmis.org>
S: Maintained
F: arch/*/include/asm/unwind_sframe.h
+F: include/linux/sframe.h
F: include/linux/unwind*.h
F: kernel/unwind/
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index 29c6100f0c50b..84983df605ed5 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -32,6 +32,7 @@
#include <linux/sched/task.h>
#include <linux/scs.h>
#include <linux/mm.h>
+#include <linux/sframe.h>
#include <asm/acpi.h>
#include <asm/fixmap.h>
@@ -373,6 +374,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
"This indicates a broken bootloader or old kernel\n",
boot_args[1], boot_args[2], boot_args[3]);
}
+ init_sframe_table();
}
static inline bool cpu_can_disable(unsigned int cpu)
diff --git a/include/linux/sframe.h b/include/linux/sframe.h
new file mode 100644
index 0000000000000..54e5cb76cb92c
--- /dev/null
+++ b/include/linux/sframe.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_SFRAME_H
+#define _LINUX_SFRAME_H
+
+#include <linux/init.h>
+
+#ifdef CONFIG_UNWIND_SFRAME_LOOKUP
+
+struct sframe_section {
+ unsigned long sframe_start;
+ unsigned long sframe_end;
+
+ unsigned long fdes_start;
+ unsigned long fres_start;
+ unsigned long fres_end;
+ unsigned int num_fdes;
+
+ signed char ra_off;
+ signed char fp_off;
+};
+
+extern struct sframe_section kernel_sfsec __ro_after_init;
+
+void __init init_sframe_table(void);
+
+#else /* !CONFIG_UNWIND_SFRAME_LOOKUP */
+
+static inline void __init init_sframe_table(void) {}
+
+#endif /* CONFIG_UNWIND_SFRAME_LOOKUP */
+
+#endif /* _LINUX_SFRAME_H */
diff --git a/kernel/unwind/Makefile b/kernel/unwind/Makefile
index eae37bea54fdb..c5f9f8124564e 100644
--- a/kernel/unwind/Makefile
+++ b/kernel/unwind/Makefile
@@ -1 +1,2 @@
- obj-$(CONFIG_UNWIND_USER) += user.o deferred.o
+ obj-$(CONFIG_UNWIND_USER) += user.o deferred.o
+ obj-$(CONFIG_UNWIND_SFRAME_LOOKUP) += sframe.o
diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
new file mode 100644
index 0000000000000..5f49f05e6226d
--- /dev/null
+++ b/kernel/unwind/sframe.c
@@ -0,0 +1,80 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Kernel sframe access functions
+ */
+
+#define pr_fmt(fmt) "sframe: " fmt
+
+#include <linux/mm.h>
+#include <linux/string_helpers.h>
+#include <linux/sframe.h>
+#include <asm/sections.h>
+
+#include "sframe.h"
+
+static bool sframe_init __ro_after_init;
+struct sframe_section kernel_sfsec __ro_after_init;
+
+static int sframe_read_header(struct sframe_section *sec)
+{
+ unsigned long header_end, fdes_start, fdes_end, fres_start, fres_end;
+ struct sframe_header *shdr;
+ unsigned int num_fdes;
+
+ header_end = sec->sframe_start + sizeof(struct sframe_header);
+ if (header_end >= sec->sframe_end) {
+ pr_debug("header struct doesn't fit in section\n");
+ return -EINVAL;
+ }
+
+ shdr = (struct sframe_header *)sec->sframe_start;
+
+ if (shdr->preamble.magic != SFRAME_MAGIC ||
+ shdr->preamble.version != SFRAME_VERSION_3 ||
+ !(shdr->preamble.flags & SFRAME_F_FDE_SORTED) ||
+ !(shdr->preamble.flags & SFRAME_F_FDE_FUNC_START_PCREL) ||
+ shdr->auxhdr_len) {
+ pr_debug("bad/unsupported sframe header\n");
+ return -EINVAL;
+ }
+
+ if (!shdr->num_fdes || !shdr->num_fres) {
+ pr_debug("no fde/fre entries\n");
+ return -EINVAL;
+ }
+
+ num_fdes = shdr->num_fdes;
+ fdes_start = header_end + shdr->fdes_off;
+ fdes_end = fdes_start + (num_fdes * sizeof(struct sframe_fde_v3));
+
+ fres_start = header_end + shdr->fres_off;
+ fres_end = fres_start + shdr->fre_len;
+
+ if (fres_start < fdes_end || fres_end > sec->sframe_end) {
+ pr_debug("inconsistent fde/fre offsets\n");
+ return -EINVAL;
+ }
+
+ sec->num_fdes = num_fdes;
+ sec->fdes_start = fdes_start;
+ sec->fres_start = fres_start;
+ sec->fres_end = fres_end;
+
+ sec->ra_off = shdr->cfa_fixed_ra_offset;
+ sec->fp_off = shdr->cfa_fixed_fp_offset;
+
+ return 0;
+}
+
+void __init init_sframe_table(void)
+{
+ kernel_sfsec.sframe_start = (unsigned long)__start_sframe;
+ kernel_sfsec.sframe_end = (unsigned long)__end_sframe;
+
+ if (sframe_read_header(&kernel_sfsec)) {
+ pr_warn("invalid vmlinux SFrame header\n");
+ return;
+ }
+
+ sframe_init = true;
+}
diff --git a/kernel/unwind/sframe.h b/kernel/unwind/sframe.h
new file mode 100644
index 0000000000000..bf895107c2070
--- /dev/null
+++ b/kernel/unwind/sframe.h
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * From https://www.sourceware.org/binutils/docs/sframe-spec.html
+ */
+#ifndef _SFRAME_H
+#define _SFRAME_H
+
+#include <linux/types.h>
+
+#define SFRAME_VERSION_1 1
+#define SFRAME_VERSION_2 2
+#define SFRAME_VERSION_3 3
+#define SFRAME_MAGIC 0xdee2
+
+#define SFRAME_F_FDE_SORTED 0x1
+#define SFRAME_F_FRAME_POINTER 0x2
+#define SFRAME_F_FDE_FUNC_START_PCREL 0x4
+
+#define SFRAME_ABI_AARCH64_ENDIAN_BIG 1
+#define SFRAME_ABI_AARCH64_ENDIAN_LITTLE 2
+#define SFRAME_ABI_AMD64_ENDIAN_LITTLE 3
+
+struct sframe_preamble {
+ u16 magic;
+ u8 version;
+ u8 flags;
+} __packed;
+
+struct sframe_header {
+ struct sframe_preamble preamble;
+ u8 abi_arch;
+ s8 cfa_fixed_fp_offset;
+ s8 cfa_fixed_ra_offset;
+ u8 auxhdr_len;
+ u32 num_fdes;
+ u32 num_fres;
+ u32 fre_len;
+ u32 fdes_off;
+ u32 fres_off;
+} __packed;
+
+struct sframe_fde_v3 {
+ s64 func_start_off;
+ u32 func_size;
+ u32 fres_off;
+} __packed;
+
+struct sframe_fda_v3 {
+ u16 fres_num;
+ u8 info;
+ u8 info2;
+ u8 rep_size;
+} __packed;
+
+#define SFRAME_FDE_PCTYPE_INC 0
+#define SFRAME_FDE_PCTYPE_MASK 1
+
+#define SFRAME_AARCH64_PAUTH_KEY_A 0
+#define SFRAME_AARCH64_PAUTH_KEY_B 1
+
+#define SFRAME_V3_FDE_FRE_TYPE(info) ((info) & 0xf)
+#define SFRAME_V3_FDE_PCTYPE(info) (((info) >> 4) & 0x1)
+#define SFRAME_V3_AARCH64_FDE_PAUTH_KEY(info) (((info) >> 5) & 0x1)
+
+#define SFRAME_FDE_TYPE_DEFAULT 0
+
+#define SFRAME_V3_FDE_TYPE_MASK 0x1f
+#define SFRAME_V3_FDE_TYPE(info2) ((info2) & SFRAME_V3_FDE_TYPE_MASK)
+
+#define SFRAME_BASE_REG_FP 0
+#define SFRAME_BASE_REG_SP 1
+
+#define SFRAME_V3_FRE_CFA_BASE_REG_ID(info) ((info) & 0x1)
+#define SFRAME_V3_FRE_DATAWORD_COUNT(info) (((info) >> 1) & 0xf)
+#define SFRAME_V3_FRE_DATAWORD_SIZE(info) (((info) >> 5) & 0x3)
+#define SFRAME_V3_AARCH64_FRE_MANGLED_RA_P(info) (((info) >> 7) & 0x1)
+
+#endif /* _SFRAME_H */
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH v7 03/11] sframe: Add support for reading vmlinux .sframe contents
2026-09-18 22:41 [PATCH v7 00/11] unwind, arm64: add sframe unwinder for kernel Dylan Hatch
2026-09-18 22:41 ` [PATCH v7 01/11] arm64, unwind: build kernel with sframe V3 info Dylan Hatch
2026-09-18 22:41 ` [PATCH v7 02/11] arm64/sframe: Read vmlinux .sframe header Dylan Hatch
@ 2026-09-18 22:41 ` Dylan Hatch
2026-09-18 22:41 ` [PATCH v7 04/11] sframe: Separate reading of FRE from reading of FRE data words Dylan Hatch
` (7 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Dylan Hatch @ 2026-09-18 22:41 UTC (permalink / raw)
To: Roman Gushchin, Weinan Liu, Will Deacon, Josh Poimboeuf,
Indu Bhagat, Peter Zijlstra, Steven Rostedt, Catalin Marinas,
Jiri Kosina, Mark Rutland, Jens Remus
Cc: Dylan Hatch, Prasanna Kumar T S M, Puranjay Mohan, Song Liu,
joe.lawrence, linux-toolchains, linux-kernel, live-patching,
linux-arm-kernel, Randy Dunlap, Mostafa Saleh, Herbert Xu,
David S. Miller
From: Josh Poimboeuf <jpoimboe@kernel.org>
In preparation for using sframe to unwind kernel space stacks, add an
sframe_find() interface for finding the sframe information associated
with a given vmlinux text address.
Treat a RA offset from CFA of zero as indication that the RA is not
saved (on the stack). For the topmost frame treat it as indication that
the RA is in the link/RA register, such as on arm64 and s390, and obtain
it from there. For non-topmost frames treat it as error, as the RA must
be saved.
[ Jens Remus: Add initial support for SFrame V3 (limited to regular
FDEs). Add support for PC-relative FDE function start offset. Simplify
logic by using an internal FDE representation. Rename struct sframe_fre
to sframe_fre_internal to align with struct sframe_fde_internal.
Cleanup includes. Fix checkpatch errors "spaces required around that
':'". ]
[ Dylan Hatch: Adapt for in-kernel unwinding. Squash parts of patches by
Jens Remus <jremus@linux.ibm.com>:
- "unwind_user: Enable archs that pass RA in a register"
- "unwind_user: Flexible FP/RA recovery rules"
- "unwind_user: Flexible CFA recovery rules"
- "unwind_user/sframe: Add support for outermost frame indication" ]
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Co-developed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
Reviewed-by: Indu Bhagat <ibhagatgnu@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Co-developed-by: Dylan Hatch <dylanbhatch@google.com>
Signed-off-by: Dylan Hatch <dylanbhatch@google.com>
---
This patch is adaped from commit 9f0daa3a841e ("unwind_user/sframe: Add
support for reading .sframe contents"), and squashes in part of:
- 79313210e44e ("unwind_user/sframe: Add support for outermost frame
indication")
- 70b5d8a3ddc0 ("unwind_user: Enable archs that pass RA in a register")
- 29774da9d30f ("unwind_user: Flexible FP/RA recovery rules")
- 9c9b953fe83c ("unwind_user: Flexible CFA recovery rules")
all of which are from Steven's sframe/core branch.
The patch also squash the relevant parts of the following two patching
from v6 of this series:
- sframe: Allow kernelspace sframe sections
- sframe: Provide PC lookup for vmlinux .sframe section
Changes include:
- Replace user-access sections with direct pointer reads.
- (sashiko) use get_unaligned() when reading packed .sframe contents.
- Create include/linux/unwind_types.h to define SFrame unwind rules.
- Implement sframe_find() to perform an SFrame IP lookup on
kernel_sfsec.
- Squash in SFRAME_V3_FRE_RA_UNDEFINED_P(fre->info) to support Arm64 RA
behavior.
- Omit kernel/unwind/sframe_debug.h, added in later patch.
---
include/linux/sframe.h | 4 +-
include/linux/unwind_types.h | 34 ++++
kernel/unwind/sframe.c | 349 +++++++++++++++++++++++++++++++++++
kernel/unwind/sframe.h | 1 +
4 files changed, 387 insertions(+), 1 deletion(-)
create mode 100644 include/linux/unwind_types.h
diff --git a/include/linux/sframe.h b/include/linux/sframe.h
index 54e5cb76cb92c..c1fe8ac7df7b3 100644
--- a/include/linux/sframe.h
+++ b/include/linux/sframe.h
@@ -3,6 +3,7 @@
#define _LINUX_SFRAME_H
#include <linux/init.h>
+#include <linux/unwind_types.h>
#ifdef CONFIG_UNWIND_SFRAME_LOOKUP
@@ -22,11 +23,12 @@ struct sframe_section {
extern struct sframe_section kernel_sfsec __ro_after_init;
void __init init_sframe_table(void);
+extern int sframe_find(unsigned long ip, struct unwind_frame *frame);
#else /* !CONFIG_UNWIND_SFRAME_LOOKUP */
static inline void __init init_sframe_table(void) {}
-
+static inline int sframe_find(unsigned long ip, struct unwind_frame *frame) { return -EINVAL; }
#endif /* CONFIG_UNWIND_SFRAME_LOOKUP */
#endif /* _LINUX_SFRAME_H */
diff --git a/include/linux/unwind_types.h b/include/linux/unwind_types.h
new file mode 100644
index 0000000000000..d0e69189af2ab
--- /dev/null
+++ b/include/linux/unwind_types.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_UNWIND_TYPES_H
+#define _LINUX_UNWIND_TYPES_H
+
+enum unwind_cfa_rule {
+ UNWIND_CFA_RULE_SP_OFFSET, /* CFA = SP + offset */
+ UNWIND_CFA_RULE_FP_OFFSET, /* CFA = FP + offset */
+ /* REG and DEREF variants not implemented -- flex FDEs not supported. */
+};
+
+struct unwind_cfa_rule_data {
+ enum unwind_cfa_rule rule;
+ s32 offset;
+ unsigned int regnum;
+};
+
+enum unwind_rule {
+ UNWIND_RULE_RETAIN, /* entity = entity */
+ UNWIND_RULE_CFA_OFFSET_DEREF, /* entity = *(CFA + offset) */
+};
+
+struct unwind_rule_data {
+ enum unwind_rule rule;
+ s32 offset;
+};
+
+struct unwind_frame {
+ struct unwind_cfa_rule_data cfa;
+ struct unwind_rule_data ra;
+ struct unwind_rule_data fp;
+ bool outermost;
+};
+
+#endif /* _LINUX_UNWIND_TYPES_H */
diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
index 5f49f05e6226d..f3975c7e89493 100644
--- a/kernel/unwind/sframe.c
+++ b/kernel/unwind/sframe.c
@@ -8,6 +8,9 @@
#include <linux/mm.h>
#include <linux/string_helpers.h>
#include <linux/sframe.h>
+#include <linux/unaligned.h>
+#include <linux/unwind_types.h>
+#include <linux/kallsyms.h>
#include <asm/sections.h>
#include "sframe.h"
@@ -15,6 +18,352 @@
static bool sframe_init __ro_after_init;
struct sframe_section kernel_sfsec __ro_after_init;
+struct sframe_fde_internal {
+ unsigned long func_addr;
+ u32 func_size;
+ u32 fda_off;
+ u32 fres_off;
+ u32 fres_num;
+ u8 info;
+ u8 info2;
+ u8 rep_size;
+};
+
+struct sframe_fre_internal {
+ unsigned int size;
+ u32 ip_off;
+ s32 cfa_off;
+ s32 ra_off;
+ s32 fp_off;
+ u8 info;
+};
+
+static __always_inline unsigned char fre_type_to_size(unsigned char fre_type)
+{
+ if (fre_type > 2)
+ return 0;
+ return 1 << fre_type;
+}
+
+static __always_inline unsigned char dataword_size_enum_to_size(unsigned char dataword_size)
+{
+ if (dataword_size > 2)
+ return 0;
+ return 1 << dataword_size;
+}
+
+static __always_inline int __read_fde(struct sframe_section *sec,
+ unsigned int fde_num,
+ struct sframe_fde_internal *fde)
+{
+ unsigned long fde_addr, fda_addr, func_addr;
+ struct sframe_fde_v3 *_fde;
+ struct sframe_fda_v3 *_fda;
+
+ fde_addr = sec->fdes_start + (fde_num * sizeof(struct sframe_fde_v3));
+ _fde = (struct sframe_fde_v3 *)fde_addr;
+
+ func_addr = fde_addr + _fde->func_start_off;
+
+ fda_addr = sec->fres_start + _fde->fres_off;
+ if (fda_addr + sizeof(struct sframe_fda_v3) > sec->fres_end ||
+ fda_addr + sizeof(struct sframe_fda_v3) < sec->fres_start)
+ return -EINVAL;
+ _fda = (struct sframe_fda_v3 *)fda_addr;
+
+ fde->func_addr = func_addr;
+ fde->func_size = _fde->func_size;
+ fde->fda_off = _fde->fres_off;
+ fde->fres_off = _fde->fres_off + sizeof(struct sframe_fda_v3);
+ fde->fres_num = _fda->fres_num;
+ fde->info = _fda->info;
+ fde->info2 = _fda->info2;
+ fde->rep_size = _fda->rep_size;
+
+ return 0;
+}
+
+static __always_inline int __find_fde(struct sframe_section *sec,
+ unsigned long ip,
+ struct sframe_fde_internal *fde)
+{
+ unsigned long func_addr_low = 0, func_addr_high = ULONG_MAX;
+ struct sframe_fde_v3 *first, *low, *high, *found = NULL;
+ int ret;
+
+ first = (void *)sec->fdes_start;
+ low = first;
+ high = first + sec->num_fdes - 1;
+
+ while (low <= high) {
+ struct sframe_fde_v3 *mid;
+ s64 func_off;
+ unsigned long func_addr;
+
+ mid = low + ((high - low) / 2);
+
+ func_off = get_unaligned((s64 *)mid);
+ func_addr = (unsigned long)mid + func_off;
+
+ if (ip >= func_addr) {
+ if (func_addr < func_addr_low)
+ return -EFAULT;
+
+ func_addr_low = func_addr;
+
+ found = mid;
+ low = mid + 1;
+ } else {
+ if (func_addr > func_addr_high)
+ return -EFAULT;
+
+ func_addr_high = func_addr;
+
+ high = mid - 1;
+ }
+ }
+
+ if (!found)
+ return -EINVAL;
+
+ ret = __read_fde(sec, found - first, fde);
+ if (ret)
+ return ret;
+
+ /* make sure it's not in a gap */
+ if (ip < fde->func_addr || ip >= fde->func_addr + fde->func_size)
+ return -EINVAL;
+
+ return 0;
+}
+
+#define ____GET_AND_INC(to, from, type) \
+({ \
+ type __to; \
+ (__to) = get_unaligned((type *)(from)); \
+ from += sizeof(__to); \
+ to = __to; \
+})
+
+#define __GET_AND_INC(to, from, size, u_or_s) \
+({ \
+ switch (size) { \
+ case 1: \
+ ____GET_AND_INC(to, from, u_or_s##8); \
+ break; \
+ case 2: \
+ ____GET_AND_INC(to, from, u_or_s##16); \
+ break; \
+ case 4: \
+ ____GET_AND_INC(to, from, u_or_s##32); \
+ break; \
+ default: \
+ return -EFAULT; \
+ } \
+})
+
+#define GET_UNSIGNED_AND_INC(to, from, size) \
+ __GET_AND_INC(to, from, size, u)
+
+#define GET_SIGNED_AND_INC(to, from, size) \
+ __GET_AND_INC(to, from, size, s)
+
+#define GET_AND_INC(to, from, size) \
+ _Generic(to, \
+ u8 : GET_UNSIGNED_AND_INC(to, from, size), \
+ u16 : GET_UNSIGNED_AND_INC(to, from, size), \
+ u32 : GET_UNSIGNED_AND_INC(to, from, size), \
+ u64 : GET_UNSIGNED_AND_INC(to, from, size), \
+ s8 : GET_SIGNED_AND_INC(to, from, size), \
+ s16 : GET_SIGNED_AND_INC(to, from, size), \
+ s32 : GET_SIGNED_AND_INC(to, from, size), \
+ s64 : GET_SIGNED_AND_INC(to, from, size))
+
+static __always_inline int __read_fre(struct sframe_section *sec,
+ struct sframe_fde_internal *fde,
+ unsigned long fre_addr,
+ struct sframe_fre_internal *fre)
+{
+ unsigned char fde_type = SFRAME_V3_FDE_TYPE(fde->info2);
+ unsigned char fde_pctype = SFRAME_V3_FDE_PCTYPE(fde->info);
+ unsigned char fre_type = SFRAME_V3_FDE_FRE_TYPE(fde->info);
+ unsigned char dataword_count, dataword_size;
+ s32 cfa_off, ra_off, fp_off;
+ unsigned long cur = fre_addr;
+ unsigned char addr_size;
+ u32 ip_off;
+ u8 info;
+
+ addr_size = fre_type_to_size(fre_type);
+ if (!addr_size)
+ return -EFAULT;
+
+ if (fre_addr + addr_size + 1 > sec->fres_end)
+ return -EFAULT;
+
+ GET_AND_INC(ip_off, cur, addr_size);
+ if (fde_pctype == SFRAME_FDE_PCTYPE_INC && ip_off > fde->func_size)
+ return -EFAULT;
+
+ GET_AND_INC(info, cur, 1);
+ dataword_count = SFRAME_V3_FRE_DATAWORD_COUNT(info);
+ dataword_size = dataword_size_enum_to_size(SFRAME_V3_FRE_DATAWORD_SIZE(info));
+ if (!dataword_size || !dataword_count)
+ return -EFAULT;
+
+ if (cur + (dataword_count * dataword_size) > sec->fres_end)
+ return -EFAULT;
+
+ /* Flexible FDEs not supported */
+ if (fde_type != SFRAME_FDE_TYPE_DEFAULT)
+ return -EFAULT;
+
+ fre->size = addr_size + 1 + (dataword_count * dataword_size);
+
+ GET_AND_INC(cfa_off, cur, dataword_size);
+ dataword_count--;
+
+ ra_off = sec->ra_off;
+ if (!ra_off && dataword_count) {
+ dataword_count--;
+ GET_AND_INC(ra_off, cur, dataword_size);
+ }
+
+ fp_off = sec->fp_off;
+ if (!fp_off && dataword_count) {
+ dataword_count--;
+ GET_AND_INC(fp_off, cur, dataword_size);
+ }
+
+ if (dataword_count)
+ return -EFAULT;
+
+ fre->ip_off = ip_off;
+ fre->cfa_off = cfa_off;
+ fre->ra_off = ra_off;
+ fre->fp_off = fp_off;
+ fre->info = info;
+
+ return 0;
+}
+
+static __always_inline int
+sframe_init_cfa_rule_data(struct unwind_cfa_rule_data *cfa_rule_data,
+ u8 info, s32 offset)
+{
+ switch (SFRAME_V3_FRE_CFA_BASE_REG_ID(info)) {
+ case SFRAME_BASE_REG_SP:
+ cfa_rule_data->rule = UNWIND_CFA_RULE_SP_OFFSET;
+ break;
+ case SFRAME_BASE_REG_FP:
+ cfa_rule_data->rule = UNWIND_CFA_RULE_FP_OFFSET;
+ break;
+ default:
+ /* Flex FDEs not supported; base can't be another register */
+ return -EINVAL;
+ }
+
+ cfa_rule_data->offset = offset;
+
+ return 0;
+}
+
+static __always_inline void
+sframe_init_rule_data(struct unwind_rule_data *rule_data, s32 offset)
+{
+ if (!offset) {
+ rule_data->rule = UNWIND_RULE_RETAIN;
+ return;
+ }
+ /* Flex FDEs not supported; always deref */
+ rule_data->rule = UNWIND_RULE_CFA_OFFSET_DEREF;
+ rule_data->offset = offset;
+}
+
+static __always_inline int __find_fre(struct sframe_section *sec,
+ struct sframe_fde_internal *fde,
+ unsigned long ip,
+ struct unwind_frame *frame)
+{
+ unsigned char fde_pctype = SFRAME_V3_FDE_PCTYPE(fde->info);
+ struct sframe_fre_internal *fre, *prev_fre = NULL;
+ struct sframe_fre_internal fres[2];
+ unsigned long fre_addr;
+ bool which = false;
+ unsigned int i;
+ u32 ip_off;
+
+ ip_off = ip - fde->func_addr;
+
+ if (fde_pctype == SFRAME_FDE_PCTYPE_MASK) {
+ if (!fde->rep_size)
+ return -EFAULT;
+ ip_off %= fde->rep_size;
+ }
+
+ fre_addr = sec->fres_start + fde->fres_off;
+
+ for (i = 0; i < fde->fres_num; i++) {
+ int ret;
+
+ /*
+ * Alternate between the two fre_addr[] entries for 'fre' and
+ * 'prev_fre'.
+ */
+ fre = which ? fres : fres + 1;
+ which = !which;
+
+ ret = __read_fre(sec, fde, fre_addr, fre);
+ if (ret)
+ return ret;
+
+ fre_addr += fre->size;
+
+ if (prev_fre && fre->ip_off <= prev_fre->ip_off)
+ return -EFAULT;
+
+ if (fre->ip_off > ip_off)
+ break;
+
+ prev_fre = fre;
+ }
+
+ if (!prev_fre)
+ return -EINVAL;
+ fre = prev_fre;
+
+ if (sframe_init_cfa_rule_data(&frame->cfa, fre->info, fre->cfa_off))
+ return -EINVAL;
+
+ sframe_init_rule_data(&frame->ra, fre->ra_off);
+ sframe_init_rule_data(&frame->fp, fre->fp_off);
+ frame->outermost = SFRAME_V3_FRE_RA_UNDEFINED_P(fre->info);
+
+ return 0;
+}
+
+static __always_inline int __sframe_find(struct sframe_section *sec,
+ unsigned long ip,
+ struct unwind_frame *frame)
+{
+ struct sframe_fde_internal fde;
+ int ret;
+
+ ret = __find_fde(sec, ip, &fde);
+ if (ret)
+ return ret;
+
+ return __find_fre(sec, &fde, ip, frame);
+}
+
+int sframe_find(unsigned long ip, struct unwind_frame *frame)
+{
+ if (!frame || !sframe_init)
+ return -EINVAL;
+
+ return __sframe_find(&kernel_sfsec, ip, frame);
+}
+
static int sframe_read_header(struct sframe_section *sec)
{
unsigned long header_end, fdes_start, fdes_end, fres_start, fres_end;
diff --git a/kernel/unwind/sframe.h b/kernel/unwind/sframe.h
index bf895107c2070..17daf68309e61 100644
--- a/kernel/unwind/sframe.h
+++ b/kernel/unwind/sframe.h
@@ -74,5 +74,6 @@ struct sframe_fda_v3 {
#define SFRAME_V3_FRE_DATAWORD_COUNT(info) (((info) >> 1) & 0xf)
#define SFRAME_V3_FRE_DATAWORD_SIZE(info) (((info) >> 5) & 0x3)
#define SFRAME_V3_AARCH64_FRE_MANGLED_RA_P(info) (((info) >> 7) & 0x1)
+#define SFRAME_V3_FRE_RA_UNDEFINED_P(info) (SFRAME_V3_FRE_DATAWORD_COUNT(info) == 0)
#endif /* _SFRAME_H */
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH v7 04/11] sframe: Separate reading of FRE from reading of FRE data words
2026-09-18 22:41 [PATCH v7 00/11] unwind, arm64: add sframe unwinder for kernel Dylan Hatch
` (2 preceding siblings ...)
2026-09-18 22:41 ` [PATCH v7 03/11] sframe: Add support for reading vmlinux .sframe contents Dylan Hatch
@ 2026-09-18 22:41 ` Dylan Hatch
2026-09-18 22:41 ` [PATCH v7 05/11] arm64/module, sframe: Add sframe support for modules Dylan Hatch
` (6 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Dylan Hatch @ 2026-09-18 22:41 UTC (permalink / raw)
To: Roman Gushchin, Weinan Liu, Will Deacon, Josh Poimboeuf,
Indu Bhagat, Peter Zijlstra, Steven Rostedt, Catalin Marinas,
Jiri Kosina, Mark Rutland, Jens Remus
Cc: Dylan Hatch, Prasanna Kumar T S M, Puranjay Mohan, Song Liu,
joe.lawrence, linux-toolchains, linux-kernel, live-patching,
linux-arm-kernel, Randy Dunlap, Mostafa Saleh, Herbert Xu,
David S. Miller
From: Jens Remus <jremus@linux.ibm.com>
__find_fre() performs linear search for a matching SFrame FRE for a
given IP. For that purpose it uses __read_fre(), which reads the whole
FRE. That is the variable-size FRE structure as well as the trailing
variable-length array of variable-size data words. For the search logic
to skip over the FRE it would be sufficient to read the variable-size
FRE structure only, which includes the count and size of data words.
Add fields to struct sframe_fre_internal to store the FRE data word's
address, count, and size. Change __read_fre() to read the variable-
size FRE structure only and populate those new fields. Change
__read_fre_datawords() to use those new fields. Change __find_fre()
to use __read_fre_datawords() to read the FRE data words only after a
matching FRE has been found.
[ Dylan Hatch: Adapt for in-kernel unwinding without flexible FDEs. ]
Reviewed-by: Indu Bhagat <ibhagatgnu@gmail.com>
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Dylan Hatch <dylanbhatch@google.com>
---
This patch is adapted from commit 745489faa10c ("unwind_user/sframe:
Separate reading of FRE from reading of FRE data words") and squashes
changes from:
- fdaf91d4fc0d ("unwind_user/sframe: Add support for SFrame V3
flexible FDEs")
- 79313210e44e ("unwind_user/sframe: Add support for outermost frame
indication")
all of which are from Steven's sframe/core branch.
Changes include:
- Drop support for flexible FDEs (SFRAME_FDE_TYPE_FLEX).
- Sqaush in creation of __read_default_fre_datawords() and
__read_fre_datawords().
- Squash in outermost frame handling logic.
- Deferred sframe_validate_section() changes to a later patch.
---
kernel/unwind/sframe.c | 97 +++++++++++++++++++++++++++++++-----------
1 file changed, 73 insertions(+), 24 deletions(-)
diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
index f3975c7e89493..9aac703f7ce6e 100644
--- a/kernel/unwind/sframe.c
+++ b/kernel/unwind/sframe.c
@@ -36,6 +36,9 @@ struct sframe_fre_internal {
s32 ra_off;
s32 fp_off;
u8 info;
+ unsigned long dw_addr;
+ unsigned char dw_count;
+ unsigned char dw_size;
};
static __always_inline unsigned char fre_type_to_size(unsigned char fre_type)
@@ -179,6 +182,67 @@ static __always_inline int __find_fde(struct sframe_section *sec,
s32 : GET_SIGNED_AND_INC(to, from, size), \
s64 : GET_SIGNED_AND_INC(to, from, size))
+static __always_inline int
+__read_default_fre_datawords(struct sframe_section *sec,
+ struct sframe_fde_internal *fde,
+ struct sframe_fre_internal *fre)
+{
+ unsigned char dataword_count = fre->dw_count;
+ unsigned char dataword_size = fre->dw_size;
+ unsigned long cur = fre->dw_addr;
+ s32 cfa_off, ra_off, fp_off;
+
+ GET_AND_INC(cfa_off, cur, dataword_size);
+ dataword_count--;
+
+ ra_off = sec->ra_off;
+ if (!ra_off && dataword_count) {
+ dataword_count--;
+ GET_AND_INC(ra_off, cur, dataword_size);
+ }
+
+ fp_off = sec->fp_off;
+ if (!fp_off && dataword_count) {
+ dataword_count--;
+ GET_AND_INC(fp_off, cur, dataword_size);
+ }
+
+ if (dataword_count)
+ return -EFAULT;
+
+ fre->cfa_off = cfa_off;
+ fre->ra_off = ra_off;
+ fre->fp_off = fp_off;
+
+ return 0;
+}
+
+static __always_inline int
+__read_fre_datawords(struct sframe_section *sec,
+ struct sframe_fde_internal *fde,
+ struct sframe_fre_internal *fre)
+{
+ unsigned char fde_type = SFRAME_V3_FDE_TYPE(fde->info2);
+ unsigned char dataword_count = fre->dw_count;
+
+ if (!dataword_count) {
+ /* A FRE without data words indicates an outermost frame. */
+ fre->cfa_off = 0;
+ fre->ra_off = 0;
+ fre->fp_off = 0;
+
+ return 0;
+ }
+
+ switch (fde_type) {
+ case SFRAME_FDE_TYPE_DEFAULT:
+ return __read_default_fre_datawords(sec, fde, fre);
+ /* Flexible FDEs not supported */
+ default:
+ return -EFAULT;
+ }
+}
+
static __always_inline int __read_fre(struct sframe_section *sec,
struct sframe_fde_internal *fde,
unsigned long fre_addr,
@@ -188,7 +252,6 @@ static __always_inline int __read_fre(struct sframe_section *sec,
unsigned char fde_pctype = SFRAME_V3_FDE_PCTYPE(fde->info);
unsigned char fre_type = SFRAME_V3_FDE_FRE_TYPE(fde->info);
unsigned char dataword_count, dataword_size;
- s32 cfa_off, ra_off, fp_off;
unsigned long cur = fre_addr;
unsigned char addr_size;
u32 ip_off;
@@ -208,7 +271,7 @@ static __always_inline int __read_fre(struct sframe_section *sec,
GET_AND_INC(info, cur, 1);
dataword_count = SFRAME_V3_FRE_DATAWORD_COUNT(info);
dataword_size = dataword_size_enum_to_size(SFRAME_V3_FRE_DATAWORD_SIZE(info));
- if (!dataword_size || !dataword_count)
+ if (!dataword_size)
return -EFAULT;
if (cur + (dataword_count * dataword_size) > sec->fres_end)
@@ -219,30 +282,11 @@ static __always_inline int __read_fre(struct sframe_section *sec,
return -EFAULT;
fre->size = addr_size + 1 + (dataword_count * dataword_size);
-
- GET_AND_INC(cfa_off, cur, dataword_size);
- dataword_count--;
-
- ra_off = sec->ra_off;
- if (!ra_off && dataword_count) {
- dataword_count--;
- GET_AND_INC(ra_off, cur, dataword_size);
- }
-
- fp_off = sec->fp_off;
- if (!fp_off && dataword_count) {
- dataword_count--;
- GET_AND_INC(fp_off, cur, dataword_size);
- }
-
- if (dataword_count)
- return -EFAULT;
-
fre->ip_off = ip_off;
- fre->cfa_off = cfa_off;
- fre->ra_off = ra_off;
- fre->fp_off = fp_off;
fre->info = info;
+ fre->dw_addr = cur;
+ fre->dw_count = dataword_count;
+ fre->dw_size = dataword_size;
return 0;
}
@@ -292,6 +336,7 @@ static __always_inline int __find_fre(struct sframe_section *sec,
bool which = false;
unsigned int i;
u32 ip_off;
+ int ret;
ip_off = ip - fde->func_addr;
@@ -332,6 +377,10 @@ static __always_inline int __find_fre(struct sframe_section *sec,
return -EINVAL;
fre = prev_fre;
+ ret = __read_fre_datawords(sec, fde, fre);
+ if (ret)
+ return ret;
+
if (sframe_init_cfa_rule_data(&frame->cfa, fre->info, fre->cfa_off))
return -EINVAL;
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH v7 05/11] arm64/module, sframe: Add sframe support for modules
2026-09-18 22:41 [PATCH v7 00/11] unwind, arm64: add sframe unwinder for kernel Dylan Hatch
` (3 preceding siblings ...)
2026-09-18 22:41 ` [PATCH v7 04/11] sframe: Separate reading of FRE from reading of FRE data words Dylan Hatch
@ 2026-09-18 22:41 ` Dylan Hatch
2026-09-18 22:54 ` sashiko-bot
2026-09-18 22:41 ` [PATCH v7 06/11] arm64/sframe: Validate IP addresses Dylan Hatch
` (5 subsequent siblings)
10 siblings, 1 reply; 15+ messages in thread
From: Dylan Hatch @ 2026-09-18 22:41 UTC (permalink / raw)
To: Roman Gushchin, Weinan Liu, Will Deacon, Josh Poimboeuf,
Indu Bhagat, Peter Zijlstra, Steven Rostedt, Catalin Marinas,
Jiri Kosina, Mark Rutland, Jens Remus
Cc: Dylan Hatch, Prasanna Kumar T S M, Puranjay Mohan, Song Liu,
joe.lawrence, linux-toolchains, linux-kernel, live-patching,
linux-arm-kernel, Randy Dunlap, Mostafa Saleh, Herbert Xu,
David S. Miller
Add sframe table to mod_arch_specific and support sframe PC lookups when
an .sframe section can be found on incoming modules. SFRAME_F_FDE_SORTED
is not set for module .sframe, so FDEs are sorted right after the sframe
header is read.
Co-developed-by: Weinan Liu <wnliu@google.com>
Signed-off-by: Weinan Liu <wnliu@google.com>
Suggested-by: Jens Remus <jremus@linux.ibm.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Dylan Hatch <dylanbhatch@google.com>
---
Changes since v6:
- (sashiko) Check section flag SHF_ALLOC before reading in module
.sframe.
- (sashiko) Use READ_ONCE/WRITE_ONCE to enforce memory barrier when
checking mod->arch.sframe_init.
- Change IP range checks to strictly check text sections of
vmlinux/module (instead of is_ksym_addr(ip) / __module_address(ip)).
- (sashiko) Check kernel_sfsec.fdes_sorted during init, since
SFRAME_F_FDE_SORTED is no longer required (in order to support
modules).
- (sashiko) Drop WARN_ON to pr_warn() when module sframe header read
fails.
- Drop sec->sec_type check in sframe_sort_fdes(), return void.
---
arch/arm64/include/asm/module.h | 6 ++
arch/arm64/kernel/module.c | 5 ++
include/linux/sframe.h | 7 +++
kernel/unwind/sframe.c | 105 ++++++++++++++++++++++++++++++--
4 files changed, 119 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/module.h
index fb9b88eebeb15..07f309c51eee6 100644
--- a/arch/arm64/include/asm/module.h
+++ b/arch/arm64/include/asm/module.h
@@ -6,6 +6,7 @@
#define __ASM_MODULE_H
#include <asm-generic/module.h>
+#include <linux/sframe.h>
struct mod_plt_sec {
int plt_shndx;
@@ -17,6 +18,11 @@ struct mod_arch_specific {
struct mod_plt_sec core;
struct mod_plt_sec init;
+#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME
+ struct sframe_section sframe_sec;
+ bool sframe_init;
+#endif
+
/* for CONFIG_DYNAMIC_FTRACE */
struct plt_entry *ftrace_trampolines;
struct plt_entry *init_ftrace_trampolines;
diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c
index 24adb581af0eb..69334e22be34e 100644
--- a/arch/arm64/kernel/module.c
+++ b/arch/arm64/kernel/module.c
@@ -18,6 +18,7 @@
#include <linux/moduleloader.h>
#include <linux/random.h>
#include <linux/scs.h>
+#include <linux/sframe.h>
#include <asm/alternative.h>
#include <asm/insn.h>
@@ -515,5 +516,9 @@ int module_finalize(const Elf_Ehdr *hdr,
}
}
+ s = find_section(hdr, sechdrs, ".sframe");
+ if (s && (s->sh_flags & SHF_ALLOC))
+ sframe_module_init(me, (void *)s->sh_addr, s->sh_size);
+
return module_init_ftrace_plt(hdr, sechdrs, me);
}
diff --git a/include/linux/sframe.h b/include/linux/sframe.h
index c1fe8ac7df7b3..58a68de1396af 100644
--- a/include/linux/sframe.h
+++ b/include/linux/sframe.h
@@ -15,6 +15,7 @@ struct sframe_section {
unsigned long fres_start;
unsigned long fres_end;
unsigned int num_fdes;
+ bool fdes_sorted;
signed char ra_off;
signed char fp_off;
@@ -23,12 +24,18 @@ struct sframe_section {
extern struct sframe_section kernel_sfsec __ro_after_init;
void __init init_sframe_table(void);
+void sframe_module_init(struct module *mod, void *sframe, size_t sframe_size);
+
extern int sframe_find(unsigned long ip, struct unwind_frame *frame);
#else /* !CONFIG_UNWIND_SFRAME_LOOKUP */
static inline void __init init_sframe_table(void) {}
+static inline void sframe_module_init(struct module *mod, void *sframe,
+ size_t sframe_size) {}
+
static inline int sframe_find(unsigned long ip, struct unwind_frame *frame) { return -EINVAL; }
+
#endif /* CONFIG_UNWIND_SFRAME_LOOKUP */
#endif /* _LINUX_SFRAME_H */
diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
index 9aac703f7ce6e..503d4a2beb50e 100644
--- a/kernel/unwind/sframe.c
+++ b/kernel/unwind/sframe.c
@@ -8,6 +8,7 @@
#include <linux/mm.h>
#include <linux/string_helpers.h>
#include <linux/sframe.h>
+#include <linux/sort.h>
#include <linux/unaligned.h>
#include <linux/unwind_types.h>
#include <linux/kallsyms.h>
@@ -94,6 +95,9 @@ static __always_inline int __find_fde(struct sframe_section *sec,
struct sframe_fde_v3 *first, *low, *high, *found = NULL;
int ret;
+ if (!sec->fdes_sorted)
+ return -EINVAL;
+
first = (void *)sec->fdes_start;
low = first;
high = first + sec->num_fdes - 1;
@@ -405,12 +409,43 @@ static __always_inline int __sframe_find(struct sframe_section *sec,
return __find_fre(sec, &fde, ip, frame);
}
+#ifdef CONFIG_MODULES
+
+static __always_inline int __sframe_find_module(unsigned long ip,
+ struct unwind_frame *frame)
+{
+ struct module *mod;
+
+ mod = __module_text_address(ip);
+ if (!mod || !READ_ONCE(mod->arch.sframe_init))
+ return -EINVAL;
+
+ return __sframe_find(&mod->arch.sframe_sec, ip, frame);
+}
+
+#else
+
+static __always_inline int __sframe_find_module(unsigned long ip,
+ struct unwind_frame *frame)
+{
+ return -EINVAL;
+}
+
+#endif
+
int sframe_find(unsigned long ip, struct unwind_frame *frame)
{
- if (!frame || !sframe_init)
+ if (!frame)
return -EINVAL;
- return __sframe_find(&kernel_sfsec, ip, frame);
+ if (is_kernel_text(ip) || is_kernel_inittext(ip)) {
+ if (!sframe_init)
+ return -EINVAL;
+
+ return __sframe_find(&kernel_sfsec, ip, frame);
+ }
+
+ return __sframe_find_module(ip, frame);
}
static int sframe_read_header(struct sframe_section *sec)
@@ -429,7 +464,6 @@ static int sframe_read_header(struct sframe_section *sec)
if (shdr->preamble.magic != SFRAME_MAGIC ||
shdr->preamble.version != SFRAME_VERSION_3 ||
- !(shdr->preamble.flags & SFRAME_F_FDE_SORTED) ||
!(shdr->preamble.flags & SFRAME_F_FDE_FUNC_START_PCREL) ||
shdr->auxhdr_len) {
pr_debug("bad/unsupported sframe header\n");
@@ -453,6 +487,7 @@ static int sframe_read_header(struct sframe_section *sec)
return -EINVAL;
}
+ sec->fdes_sorted = shdr->preamble.flags & SFRAME_F_FDE_SORTED;
sec->num_fdes = num_fdes;
sec->fdes_start = fdes_start;
sec->fres_start = fres_start;
@@ -469,10 +504,72 @@ void __init init_sframe_table(void)
kernel_sfsec.sframe_start = (unsigned long)__start_sframe;
kernel_sfsec.sframe_end = (unsigned long)__end_sframe;
- if (sframe_read_header(&kernel_sfsec)) {
+ if (sframe_read_header(&kernel_sfsec) || !kernel_sfsec.fdes_sorted) {
pr_warn("invalid vmlinux SFrame header\n");
return;
}
sframe_init = true;
}
+
+#ifdef CONFIG_MODULES
+
+static int sframe_sort_cmp_fde(const void *a, const void *b)
+{
+ const struct sframe_fde_v3 *fde_a = a, *fde_b = b;
+ unsigned long func_start_a, func_start_b;
+
+ func_start_a = (unsigned long)fde_a + fde_a->func_start_off;
+ func_start_b = (unsigned long)fde_b + fde_b->func_start_off;
+
+ return cmp_int(func_start_a, func_start_b);
+}
+
+static void sframe_sort_swap_fde(void *a, void *b, int size)
+{
+ struct sframe_fde_v3 *fde_a = a, *fde_b = b;
+ struct sframe_fde_v3 temp;
+ long delta;
+
+ /* Swap potentially unaligned FDE */
+ memcpy(&temp, fde_a, sizeof(struct sframe_fde_v3));
+ memcpy(fde_a, fde_b, sizeof(struct sframe_fde_v3));
+ memcpy(fde_b, &temp, sizeof(struct sframe_fde_v3));
+
+ /* Adjust FDE function start offset from FDE */
+ delta = (long)((unsigned long)fde_b - (unsigned long)fde_a);
+ fde_a->func_start_off += delta;
+ fde_b->func_start_off -= delta;
+}
+
+static void sframe_sort_fdes(struct sframe_section *sec)
+{
+ void *fdes = (void *)sec->fdes_start;
+ size_t num_fdes = sec->num_fdes;
+
+ if (sec->fdes_sorted)
+ return;
+
+ sort(fdes, num_fdes, sizeof(struct sframe_fde_v3),
+ sframe_sort_cmp_fde, sframe_sort_swap_fde);
+ sec->fdes_sorted = true;
+}
+
+void sframe_module_init(struct module *mod, void *sframe, size_t sframe_size)
+{
+ struct sframe_section *sec = &mod->arch.sframe_sec;
+
+ sec->sframe_start = (unsigned long)sframe;
+ sec->sframe_end = (unsigned long)sframe + sframe_size;
+
+ if (sframe_read_header(sec)) {
+ pr_warn("invalid SFrame header in module %s\n", mod->name);
+ return;
+ }
+ sframe_sort_fdes(sec);
+
+ /* Ensure SFrame is initialized when sframe_find() happens */
+ WRITE_ONCE(mod->arch.sframe_init, true);
+}
+
+#endif
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v7 05/11] arm64/module, sframe: Add sframe support for modules
2026-09-18 22:41 ` [PATCH v7 05/11] arm64/module, sframe: Add sframe support for modules Dylan Hatch
@ 2026-09-18 22:54 ` sashiko-bot
0 siblings, 0 replies; 15+ messages in thread
From: sashiko-bot @ 2026-09-18 22:54 UTC (permalink / raw)
To: Jens Remus, Peter Zijlstra, Jiri Kosina, Roman Gushchin,
Indu Bhagat, Dylan Hatch, Steven Rostedt, Catalin Marinas,
Weinan Liu, Josh Poimboeuf, Mark Rutland, Will Deacon
Cc: Prasanna Kumar T S M, Puranjay Mohan, joe.lawrence,
linux-arm-kernel, Randy Dunlap, Mostafa Saleh, Song Liu,
linux-kernel, live-patching, linux-toolchains, Herbert Xu,
David S. Miller
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] sframe: UAF and lockdep warning due to missing RCU lock around module lookup
- [Medium] sframe: architecture abstraction violation in generic module code
--
commit d8442279cd580755fb98f0f72b910fcd83585a23
Author: Dylan Hatch <dylanbhatch@google.com>
arm64/module, sframe: Add sframe support for modules
This commit adds SFrame support for kernel modules on ARM64 by extending
the architecture-specific module structure. It parses the .sframe section
from incoming modules during load time and registers it for stack unwinding.
> diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/module.h
> index fb9b88eebeb15..07f309c51eee6 100644
> --- a/arch/arm64/include/asm/module.h
> +++ b/arch/arm64/include/asm/module.h
[ ... ]
> @@ -17,6 +18,11 @@ struct mod_arch_specific {
> struct mod_plt_sec core;
> struct mod_plt_sec init;
>
> +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME
> + struct sframe_section sframe_sec;
> + bool sframe_init;
> +#endif
> +
[Severity: Medium]
Will this break cross-architecture builds if other architectures attempt to
adopt SFrame?
By placing these generically named fields into the ARM64-specific
mod_arch_specific struct, generic unwinder code in kernel/unwind/sframe.c
is now tightly coupled to an architecture-private layout.
Should these generic module extensions reside directly in struct module
(in include/linux/module.h) under the appropriate config guards instead?
[ ... ]
> diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
> index 9aac703f7ce6e..503d4a2beb50e 100644
> --- a/kernel/unwind/sframe.c
> +++ b/kernel/unwind/sframe.c
[ ... ]
> @@ -405,12 +409,43 @@ static __always_inline int __sframe_find(struct sframe_section *sec,
> return __find_fre(sec, &fde, ip, frame);
> }
>
> +#ifdef CONFIG_MODULES
> +
> +static __always_inline int __sframe_find_module(unsigned long ip,
> + struct unwind_frame *frame)
> +{
> + struct module *mod;
> +
> + mod = __module_text_address(ip);
[Severity: High]
Does this code introduce a use-after-free and a lockdep warning?
Stack unwinding can be initiated from preemptible contexts, but
__module_text_address() requires preemption to be disabled or the RCU read
lock to be held. If a module is concurrently unloaded via sys_delete_module()
immediately after __module_text_address() returns, the returned pointer
becomes dangling.
The subsequent call to __sframe_find() would then dereference pointers within
the freed memory. Should this lookup and data access be wrapped in
guard(rcu)(); or an rcu_read_lock() / rcu_read_unlock() block?
> + if (!mod || !READ_ONCE(mod->arch.sframe_init))
> + return -EINVAL;
> +
> + return __sframe_find(&mod->arch.sframe_sec, ip, frame);
[Severity: Medium]
Does this generic lookup code directly reach into the arch-specific
abstraction? This creates a tight coupling between the generic module unwinder
and the ARM64-specific layout.
[ ... ]
> +void sframe_module_init(struct module *mod, void *sframe, size_t sframe_size)
> +{
> + struct sframe_section *sec = &mod->arch.sframe_sec;
[Severity: Medium]
Is it safe for this generic initialization code to assume the presence of
the sframe_sec field in the architecture-private mod_arch_specific structure?
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260918224157.1471085-1-dylanbhatch@google.com?part=5
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v7 06/11] arm64/sframe: Validate IP addresses
2026-09-18 22:41 [PATCH v7 00/11] unwind, arm64: add sframe unwinder for kernel Dylan Hatch
` (4 preceding siblings ...)
2026-09-18 22:41 ` [PATCH v7 05/11] arm64/module, sframe: Add sframe support for modules Dylan Hatch
@ 2026-09-18 22:41 ` Dylan Hatch
2026-09-18 22:41 ` [PATCH v7 07/11] sframe: Add debug helpers with object name Dylan Hatch
` (4 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Dylan Hatch @ 2026-09-18 22:41 UTC (permalink / raw)
To: Roman Gushchin, Weinan Liu, Will Deacon, Josh Poimboeuf,
Indu Bhagat, Peter Zijlstra, Steven Rostedt, Catalin Marinas,
Jiri Kosina, Mark Rutland, Jens Remus
Cc: Dylan Hatch, Prasanna Kumar T S M, Puranjay Mohan, Song Liu,
joe.lawrence, linux-toolchains, linux-kernel, live-patching,
linux-arm-kernel, Randy Dunlap, Mostafa Saleh, Herbert Xu,
David S. Miller
Validate the given IP and computed function start address against the
known vmlinux and module text address ranges.
This requires arch-specific validation for vmlinux. This is because
arm64 keeps .exit.text (normally discarded) and .rodata.text, both of
both of which lie outside the bounds of .text and .init.text. Note that
.rodata.text contains code that is never executed by the kernel mapping,
but for which the toolchain nonetheless generates sframe data, and needs
to be considered valid at lookup time.
Suggested-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Dylan Hatch <dylanbhatch@google.com>
---
This patch is split out from v6 patch "sframe: Introduce in-kernel
SFRAME_VALIDATION", and includes just the IP validation logic without
the SFRAME_VALIDATION option.
---
arch/arm64/include/asm/sections.h | 1 +
arch/arm64/include/asm/unwind_sframe.h | 32 +++++++++++++++++++
arch/arm64/kernel/vmlinux.lds.S | 2 ++
kernel/unwind/sframe.c | 43 +++++++++++++++++++++++++-
4 files changed, 77 insertions(+), 1 deletion(-)
create mode 100644 arch/arm64/include/asm/unwind_sframe.h
diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h
index 51b0d594239eb..5edb4304f661e 100644
--- a/arch/arm64/include/asm/sections.h
+++ b/arch/arm64/include/asm/sections.h
@@ -23,6 +23,7 @@ extern char __irqentry_text_start[], __irqentry_text_end[];
extern char __mmuoff_data_start[], __mmuoff_data_end[];
extern char __entry_tramp_text_start[], __entry_tramp_text_end[];
extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[];
+extern char _srodatatext[], _erodatatext[];
static inline size_t entry_tramp_text_size(void)
{
diff --git a/arch/arm64/include/asm/unwind_sframe.h b/arch/arm64/include/asm/unwind_sframe.h
new file mode 100644
index 0000000000000..8eb720f59434c
--- /dev/null
+++ b/arch/arm64/include/asm/unwind_sframe.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_ARM64_UNWIND_SFRAME_H
+#define _ASM_ARM64_UNWIND_SFRAME_H
+
+#include <linux/module.h>
+#include <linux/sframe.h>
+#include <asm/sections.h>
+
+static inline bool sframe_is_kernel_ip_valid(unsigned long ip)
+{
+ if (is_kernel_text(ip) || is_kernel_inittext(ip))
+ return true;
+
+ /* .exit.text is retained in vmlinux on arm64. */
+ if (ip >= (unsigned long)__exittext_begin &&
+ ip < (unsigned long)__exittext_end)
+ return true;
+
+ /*
+ * .rodata.text is never executed from the kernel mapping, but
+ * still has sframe data
+ */
+ if (ip >= (unsigned long)_srodatatext &&
+ ip < (unsigned long)_erodatatext)
+ return true;
+
+ return false;
+}
+
+#define sframe_is_kernel_ip_valid sframe_is_kernel_ip_valid
+
+#endif /* _ASM_ARM64_UNWIND_SFRAME_H */
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index eb1f54829503c..82fd20ccb7c43 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -237,12 +237,14 @@ SECTIONS
/* code sections that are never executed via the kernel mapping */
.rodata.text : {
+ _srodatatext = .;
TRAMP_TEXT
HIBERNATE_TEXT
KEXEC_TEXT
IDMAP_TEXT
. = ALIGN(PAGE_SIZE);
}
+ _erodatatext = .;
idmap_pg_dir = .;
. += PAGE_SIZE;
diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
index 503d4a2beb50e..e6542bb678585 100644
--- a/kernel/unwind/sframe.c
+++ b/kernel/unwind/sframe.c
@@ -42,6 +42,45 @@ struct sframe_fre_internal {
unsigned char dw_size;
};
+#ifndef sframe_is_kernel_ip_valid
+
+static __always_inline bool sframe_is_kernel_ip_valid(unsigned long ip)
+{
+ return is_kernel_text(ip) || is_kernel_inittext(ip);
+}
+
+#endif
+
+#ifdef CONFIG_MODULES
+
+static __always_inline bool sframe_is_sec_module_ip_valid(struct sframe_section *sec,
+ unsigned long ip)
+{
+ struct module *mod = container_of(sec, struct module, arch.sframe_sec);
+
+ return within_module_mem_type(ip, mod, MOD_TEXT) ||
+ within_module_mem_type(ip, mod, MOD_INIT_TEXT);
+}
+
+#else
+
+static __always_inline bool sframe_is_sec_module_ip_valid(struct sframe_section *sec,
+ unsigned long ip)
+{
+ return false;
+}
+
+#endif
+
+static __always_inline bool is_sec_ip_valid(struct sframe_section *sec,
+ unsigned long ip)
+{
+ if (sec == &kernel_sfsec)
+ return sframe_is_kernel_ip_valid(ip);
+
+ return sframe_is_sec_module_ip_valid(sec, ip);
+}
+
static __always_inline unsigned char fre_type_to_size(unsigned char fre_type)
{
if (fre_type > 2)
@@ -68,6 +107,8 @@ static __always_inline int __read_fde(struct sframe_section *sec,
_fde = (struct sframe_fde_v3 *)fde_addr;
func_addr = fde_addr + _fde->func_start_off;
+ if (!is_sec_ip_valid(sec, func_addr))
+ return -EINVAL;
fda_addr = sec->fres_start + _fde->fres_off;
if (fda_addr + sizeof(struct sframe_fda_v3) > sec->fres_end ||
@@ -438,7 +479,7 @@ int sframe_find(unsigned long ip, struct unwind_frame *frame)
if (!frame)
return -EINVAL;
- if (is_kernel_text(ip) || is_kernel_inittext(ip)) {
+ if (sframe_is_kernel_ip_valid(ip)) {
if (!sframe_init)
return -EINVAL;
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH v7 07/11] sframe: Add debug helpers with object name
2026-09-18 22:41 [PATCH v7 00/11] unwind, arm64: add sframe unwinder for kernel Dylan Hatch
` (5 preceding siblings ...)
2026-09-18 22:41 ` [PATCH v7 06/11] arm64/sframe: Validate IP addresses Dylan Hatch
@ 2026-09-18 22:41 ` Dylan Hatch
2026-09-18 22:41 ` [PATCH v7 08/11] sframe: Add .sframe validation option Dylan Hatch
` (3 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Dylan Hatch @ 2026-09-18 22:41 UTC (permalink / raw)
To: Roman Gushchin, Weinan Liu, Will Deacon, Josh Poimboeuf,
Indu Bhagat, Peter Zijlstra, Steven Rostedt, Catalin Marinas,
Jiri Kosina, Mark Rutland, Jens Remus
Cc: Dylan Hatch, Prasanna Kumar T S M, Puranjay Mohan, Song Liu,
joe.lawrence, linux-toolchains, linux-kernel, live-patching,
linux-arm-kernel, Randy Dunlap, Mostafa Saleh, Herbert Xu,
David S. Miller
Include the object name for kernel .sframe sections, indicated by the
format "(<module-name>)".
Suggested-by: Jens Remus <jremus@linux.ibm.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Dylan Hatch <dylanbhatch@google.com>
---
Changes since v6:
- Compute objname of sec on the fly to avoid allocating/storing an
extra string in struct sframe_section for the name.
- Squash dbg_print_header() definition, and add calls from
init_sframe_table() and sframe_module_init().
---
kernel/unwind/sframe.c | 11 +++++++----
kernel/unwind/sframe_debug.h | 34 ++++++++++++++++++++++++++++++++++
2 files changed, 41 insertions(+), 4 deletions(-)
create mode 100644 kernel/unwind/sframe_debug.h
diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
index e6542bb678585..c4715befb59b5 100644
--- a/kernel/unwind/sframe.c
+++ b/kernel/unwind/sframe.c
@@ -15,6 +15,7 @@
#include <asm/sections.h>
#include "sframe.h"
+#include "sframe_debug.h"
static bool sframe_init __ro_after_init;
struct sframe_section kernel_sfsec __ro_after_init;
@@ -497,7 +498,7 @@ static int sframe_read_header(struct sframe_section *sec)
header_end = sec->sframe_start + sizeof(struct sframe_header);
if (header_end >= sec->sframe_end) {
- pr_debug("header struct doesn't fit in section\n");
+ dbg_sec(sec, "header struct doesn't fit in section\n");
return -EINVAL;
}
@@ -507,12 +508,12 @@ static int sframe_read_header(struct sframe_section *sec)
shdr->preamble.version != SFRAME_VERSION_3 ||
!(shdr->preamble.flags & SFRAME_F_FDE_FUNC_START_PCREL) ||
shdr->auxhdr_len) {
- pr_debug("bad/unsupported sframe header\n");
+ dbg_sec(sec, "bad/unsupported sframe header\n");
return -EINVAL;
}
if (!shdr->num_fdes || !shdr->num_fres) {
- pr_debug("no fde/fre entries\n");
+ dbg_sec(sec, "no fde/fre entries\n");
return -EINVAL;
}
@@ -524,7 +525,7 @@ static int sframe_read_header(struct sframe_section *sec)
fres_end = fres_start + shdr->fre_len;
if (fres_start < fdes_end || fres_end > sec->sframe_end) {
- pr_debug("inconsistent fde/fre offsets\n");
+ dbg_sec(sec, "inconsistent fde/fre offsets\n");
return -EINVAL;
}
@@ -551,6 +552,7 @@ void __init init_sframe_table(void)
}
sframe_init = true;
+ dbg_print_header(&kernel_sfsec);
}
#ifdef CONFIG_MODULES
@@ -611,6 +613,7 @@ void sframe_module_init(struct module *mod, void *sframe, size_t sframe_size)
/* Ensure SFrame is initialized when sframe_find() happens */
WRITE_ONCE(mod->arch.sframe_init, true);
+ dbg_print_header(sec);
}
#endif
diff --git a/kernel/unwind/sframe_debug.h b/kernel/unwind/sframe_debug.h
new file mode 100644
index 0000000000000..114a4ff4899ef
--- /dev/null
+++ b/kernel/unwind/sframe_debug.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _SFRAME_DEBUG_H
+#define _SFRAME_DEBUG_H
+
+#include <linux/sframe.h>
+#include "sframe.h"
+
+#ifdef CONFIG_MODULES
+
+#define objname(sec) ((sec) == &kernel_sfsec \
+ ? "vmlinux" \
+ : container_of((sec), struct module, arch.sframe_sec)->name)
+
+#else
+
+#define objname(...) "vmlinux"
+
+#endif
+
+#define dbg_sec(sec, fmt, ...) \
+ pr_debug("(%s): " fmt, objname(sec), ##__VA_ARGS__)
+
+static __always_inline void dbg_print_header(struct sframe_section *sec)
+{
+ unsigned long fdes_end;
+
+ fdes_end = sec->fdes_start + (sec->num_fdes * sizeof(struct sframe_fde_v3));
+
+ dbg_sec(sec, "SEC:0x%lx-0x%lx fdes:0x%lx-0x%lx fres:0x%lx-0x%lx ra_off:%d fp_off:%d\n",
+ sec->sframe_start, sec->sframe_end, sec->fdes_start, fdes_end, sec->fres_start,
+ sec->fres_end, sec->ra_off, sec->fp_off);
+}
+
+#endif /* _SFRAME_DEBUG_H */
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH v7 08/11] sframe: Add .sframe validation option
2026-09-18 22:41 [PATCH v7 00/11] unwind, arm64: add sframe unwinder for kernel Dylan Hatch
` (6 preceding siblings ...)
2026-09-18 22:41 ` [PATCH v7 07/11] sframe: Add debug helpers with object name Dylan Hatch
@ 2026-09-18 22:41 ` Dylan Hatch
2026-09-18 22:41 ` [PATCH v7 09/11] arm64: entry: add unwind info for call_on_irq_stack() Dylan Hatch
` (2 subsequent siblings)
10 siblings, 0 replies; 15+ messages in thread
From: Dylan Hatch @ 2026-09-18 22:41 UTC (permalink / raw)
To: Roman Gushchin, Weinan Liu, Will Deacon, Josh Poimboeuf,
Indu Bhagat, Peter Zijlstra, Steven Rostedt, Catalin Marinas,
Jiri Kosina, Mark Rutland, Jens Remus
Cc: Dylan Hatch, Prasanna Kumar T S M, Puranjay Mohan, Song Liu,
joe.lawrence, linux-toolchains, linux-kernel, live-patching,
linux-arm-kernel, Randy Dunlap, Mostafa Saleh, Herbert Xu,
David S. Miller
From: Josh Poimboeuf <jpoimboe@kernel.org>
Add a debug feature to validate all .sframe sections when first loading
the file rather than on demand.
[ Jens Remus: Add support for SFrame V3. Add support for PC-relative
FDE function start offset. Adjust to rename of struct sframe_fre to
sframe_fre_internal. Use %#x/%#lx format specifiers. ]
[ Dylan Hatch: Adapt this patch for in-kernel sframe sections. ]
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Reviewed-by: Indu Bhagat <ibhagatgnu@gmail.com>
Co-developed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Co-developed-by: Dylan Hatch <dylanbhatch@google.com>
Signed-off-by: Dylan Hatch <dylanbhatch@google.com>
---
Adapted from commit 4b30214c7896 ("unwind_user/sframe: Add .sframe
validation option") from Steven's sframe/core branch, and squashed in v6
patch "sframe: Introduce in-kernel SFRAME_VALIDATION".
Changes include:
- Squash in __read_fre_datawords() validation, given the patch
reordering.
- Drop safe_read_* user-access wrappers.
- Call sframe_validate_section() when vmlinux/kernel sections load.
---
arch/Kconfig | 18 +++++++++
kernel/unwind/sframe.c | 83 +++++++++++++++++++++++++++++++++++++++---
2 files changed, 96 insertions(+), 5 deletions(-)
diff --git a/arch/Kconfig b/arch/Kconfig
index cc59a5da80915..20aa63c93a0e2 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -538,6 +538,24 @@ config HAVE_UNWIND_KERNEL_SFRAME
practice, this can provide more reliable stacktrace results than
unwinding with frame pointers alone.
+config SFRAME_VALIDATION
+ bool "Enable .sframe section debugging"
+ depends on UNWIND_SFRAME_LOOKUP
+ depends on DYNAMIC_DEBUG
+ help
+ When adding an .sframe section for an object, validate the entire
+ section immediately rather than on demand.
+
+ This is a debug feature which is helpful for rooting out .sframe
+ section issues. If the .sframe section is corrupt, it will fail to
+ load immediately, with more information provided in dynamic printks.
+
+ This has a significant page cache footprint due to its reading of the
+ entire .sframe section for every loaded module. Not recommended for
+ general use.
+
+ If unsure, say N.
+
config HAVE_PERF_REGS
bool
help
diff --git a/kernel/unwind/sframe.c b/kernel/unwind/sframe.c
index c4715befb59b5..3386af16fc275 100644
--- a/kernel/unwind/sframe.c
+++ b/kernel/unwind/sframe.c
@@ -13,6 +13,7 @@
#include <linux/unwind_types.h>
#include <linux/kallsyms.h>
#include <asm/sections.h>
+#include <asm/unwind_sframe.h>
#include "sframe.h"
#include "sframe_debug.h"
@@ -294,7 +295,6 @@ static __always_inline int __read_fre(struct sframe_section *sec,
unsigned long fre_addr,
struct sframe_fre_internal *fre)
{
- unsigned char fde_type = SFRAME_V3_FDE_TYPE(fde->info2);
unsigned char fde_pctype = SFRAME_V3_FDE_PCTYPE(fde->info);
unsigned char fre_type = SFRAME_V3_FDE_FRE_TYPE(fde->info);
unsigned char dataword_count, dataword_size;
@@ -323,10 +323,6 @@ static __always_inline int __read_fre(struct sframe_section *sec,
if (cur + (dataword_count * dataword_size) > sec->fres_end)
return -EFAULT;
- /* Flexible FDEs not supported */
- if (fde_type != SFRAME_FDE_TYPE_DEFAULT)
- return -EFAULT;
-
fre->size = addr_size + 1 + (dataword_count * dataword_size);
fre->ip_off = ip_off;
fre->info = info;
@@ -490,6 +486,73 @@ int sframe_find(unsigned long ip, struct unwind_frame *frame)
return __sframe_find_module(ip, frame);
}
+#ifdef CONFIG_SFRAME_VALIDATION
+
+static int sframe_validate_section(struct sframe_section *sec)
+{
+ unsigned long prev_ip = 0;
+ unsigned int i;
+
+ for (i = 0; i < sec->num_fdes; i++) {
+ struct sframe_fre_internal *fre, *prev_fre = NULL;
+ unsigned long ip, fre_addr;
+ struct sframe_fde_internal fde;
+ struct sframe_fre_internal fres[2];
+ bool which = false;
+ unsigned int j;
+ int ret;
+
+ ret = __read_fde(sec, i, &fde);
+ if (ret) {
+ dbg_sec(sec, "__read_fde(%u) failed\n", i);
+ return ret;
+ }
+
+ ip = fde.func_addr;
+ if (ip <= prev_ip) {
+ dbg_sec(sec, "FDE %u not sorted\n", i);
+ return -EFAULT;
+ }
+ prev_ip = ip;
+
+ fre_addr = sec->fres_start + fde.fres_off;
+ for (j = 0; j < fde.fres_num; j++) {
+ int ret;
+
+ fre = which ? fres : fres + 1;
+ which = !which;
+
+ ret = __read_fre(sec, &fde, fre_addr, fre);
+ if (ret) {
+ dbg_sec(sec, "FDE %u: __read_fre(%u) failed\n", i, j);
+ return ret;
+ }
+ ret = __read_fre_datawords(sec, &fde, fre);
+ if (ret) {
+ dbg_sec(sec, "FDE %u: __read_fre_datawords(%u) failed\n", i, j);
+ return ret;
+ }
+
+ fre_addr += fre->size;
+
+ if (prev_fre && fre->ip_off <= prev_fre->ip_off) {
+ dbg_sec(sec, "FDE %u: FRE %u not sorted\n", i, j);
+ return -EFAULT;
+ }
+
+ prev_fre = fre;
+ }
+ }
+
+ return 0;
+}
+
+#else /* !CONFIG_SFRAME_VALIDATION */
+
+static int sframe_validate_section(struct sframe_section *sec) { return 0; }
+
+#endif /* !CONFIG_SFRAME_VALIDATION */
+
static int sframe_read_header(struct sframe_section *sec)
{
unsigned long header_end, fdes_start, fdes_end, fres_start, fres_end;
@@ -551,6 +614,11 @@ void __init init_sframe_table(void)
return;
}
+ if (sframe_validate_section(&kernel_sfsec)) {
+ pr_warn("invalid vmlinux SFrame section\n");
+ return;
+ }
+
sframe_init = true;
dbg_print_header(&kernel_sfsec);
}
@@ -611,6 +679,11 @@ void sframe_module_init(struct module *mod, void *sframe, size_t sframe_size)
}
sframe_sort_fdes(sec);
+ if (sframe_validate_section(sec)) {
+ pr_warn("invalid SFrame section in module %s\n", mod->name);
+ return;
+ }
+
/* Ensure SFrame is initialized when sframe_find() happens */
WRITE_ONCE(mod->arch.sframe_init, true);
dbg_print_header(sec);
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH v7 09/11] arm64: entry: add unwind info for call_on_irq_stack()
2026-09-18 22:41 [PATCH v7 00/11] unwind, arm64: add sframe unwinder for kernel Dylan Hatch
` (7 preceding siblings ...)
2026-09-18 22:41 ` [PATCH v7 08/11] sframe: Add .sframe validation option Dylan Hatch
@ 2026-09-18 22:41 ` Dylan Hatch
2026-09-18 22:41 ` [PATCH v7 10/11] arm64, crypto/lib: Annotate leaf functions with CFI info Dylan Hatch
2026-09-18 22:41 ` [PATCH v7 11/11] unwind: arm64: Use sframe to unwind interrupt frames Dylan Hatch
10 siblings, 0 replies; 15+ messages in thread
From: Dylan Hatch @ 2026-09-18 22:41 UTC (permalink / raw)
To: Roman Gushchin, Weinan Liu, Will Deacon, Josh Poimboeuf,
Indu Bhagat, Peter Zijlstra, Steven Rostedt, Catalin Marinas,
Jiri Kosina, Mark Rutland, Jens Remus
Cc: Dylan Hatch, Prasanna Kumar T S M, Puranjay Mohan, Song Liu,
joe.lawrence, linux-toolchains, linux-kernel, live-patching,
linux-arm-kernel, Randy Dunlap, Mostafa Saleh, Herbert Xu,
David S. Miller
From: Weinan Liu <wnliu@google.com>
DWARF CFI (Call Frame Information) specifies how to recover the return
address and callee-saved registers at each PC in a given function.
Compilers are able to generate the CFI annotations when they compile
the code to assembly language. For handcrafted assembly, we need to
annotate them by hand.
Frame pointers alone are usually sufficient to recover stack frames
(without CFI), except at the exception boundary, where more information
is needed to determine if the LR is live.
Since an exception can be taken from call_on_irq_stack(), annotate it
with CFI. The actual entry assembly functions are left untouched, since
they are not expected to take exceptions themselves.
Signed-off-by: Weinan Liu <wnliu@google.com>
Suggested-by: Jens Remus <jremus@linux.ibm.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Dylan Hatch <dylanbhatch@google.com>
---
arch/arm64/kernel/entry.S | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index f63049ac32dc6..3703acbeb16a5 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -30,6 +30,12 @@
#include <asm/asm-uaccess.h>
#include <asm/unistd.h>
+/*
+ * Do not generate .eh_frame. Only generate .debug_frame and optionally
+ * .sframe (via assembler option --gsframe[-N]).
+ */
+ .cfi_sections .debug_frame
+
.macro clear_gp_regs
.irp n,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29
mov x\n, xzr
@@ -866,6 +872,7 @@ NOKPROBE(ret_from_fork)
* Calls func(regs) using this CPU's irq stack and shadow irq stack.
*/
SYM_FUNC_START(call_on_irq_stack)
+ .cfi_startproc
save_and_disable_daif x9
#ifdef CONFIG_SHADOW_CALL_STACK
get_current_task x16
@@ -876,6 +883,9 @@ SYM_FUNC_START(call_on_irq_stack)
/* Create a frame record to save our LR and SP (implicit in FP) */
stp x29, x30, [sp, #-16]!
mov x29, sp
+ .cfi_def_cfa 29, 16
+ .cfi_offset 29, -16
+ .cfi_offset 30, -8
ldr_this_cpu x16, irq_stack_ptr, x17
@@ -891,9 +901,13 @@ SYM_FUNC_START(call_on_irq_stack)
*/
mov sp, x29
ldp x29, x30, [sp], #16
+ .cfi_restore 29
+ .cfi_restore 30
+ .cfi_def_cfa 31, 0
scs_load_current
restore_irq x9
ret
+ .cfi_endproc
SYM_FUNC_END(call_on_irq_stack)
NOKPROBE(call_on_irq_stack)
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH v7 10/11] arm64, crypto/lib: Annotate leaf functions with CFI info
2026-09-18 22:41 [PATCH v7 00/11] unwind, arm64: add sframe unwinder for kernel Dylan Hatch
` (8 preceding siblings ...)
2026-09-18 22:41 ` [PATCH v7 09/11] arm64: entry: add unwind info for call_on_irq_stack() Dylan Hatch
@ 2026-09-18 22:41 ` Dylan Hatch
2026-09-18 22:58 ` sashiko-bot
2026-09-18 22:41 ` [PATCH v7 11/11] unwind: arm64: Use sframe to unwind interrupt frames Dylan Hatch
10 siblings, 1 reply; 15+ messages in thread
From: Dylan Hatch @ 2026-09-18 22:41 UTC (permalink / raw)
To: Roman Gushchin, Weinan Liu, Will Deacon, Josh Poimboeuf,
Indu Bhagat, Peter Zijlstra, Steven Rostedt, Catalin Marinas,
Jiri Kosina, Mark Rutland, Jens Remus
Cc: Dylan Hatch, Prasanna Kumar T S M, Puranjay Mohan, Song Liu,
joe.lawrence, linux-toolchains, linux-kernel, live-patching,
linux-arm-kernel, Randy Dunlap, Mostafa Saleh, Herbert Xu,
David S. Miller
DWARF CFI (Call Frame Information) specifies how to recover return
address and callee-saved registers for annotated functions. These
annotations are generated by the compiler, but for assembly, they must
be annotated by hand.
Add simple CFI annotations to assembly leaf functions so that the LR can
be recovered by the unwinder when an exception is taken from one of them.
Signed-off-by: Dylan Hatch <dylanbhatch@google.com>
---
Changes since v6:
- (sashiko) Fix non-leaf annotions in arch/arm64/crypto/.
---
arch/arm64/crypto/aes-ce-ccm-core.S | 12 +++----
arch/arm64/crypto/aes-neonbs-core.S | 16 +++++-----
arch/arm64/crypto/ghash-ce-core.S | 12 +++----
arch/arm64/crypto/sm4-ce-ccm-core.S | 16 +++++-----
arch/arm64/crypto/sm4-ce-cipher-core.S | 4 +--
arch/arm64/crypto/sm4-ce-core.S | 44 +++++++++++++-------------
arch/arm64/crypto/sm4-ce-gcm-core.S | 16 +++++-----
arch/arm64/crypto/sm4-neon-core.S | 12 +++----
arch/arm64/include/asm/linkage.h | 29 +++++++++++++++++
arch/arm64/lib/clear_page.S | 4 +--
arch/arm64/lib/clear_user.S | 4 +--
arch/arm64/lib/copy_from_user.S | 4 +--
arch/arm64/lib/copy_page.S | 4 +--
arch/arm64/lib/copy_to_user.S | 4 +--
arch/arm64/lib/memchr.S | 4 +--
arch/arm64/lib/memcmp.S | 4 +--
arch/arm64/lib/memcpy.S | 8 ++---
arch/arm64/lib/memset.S | 8 ++---
arch/arm64/lib/mte.S | 28 ++++++++--------
arch/arm64/lib/strchr.S | 4 +--
arch/arm64/lib/strcmp.S | 4 +--
arch/arm64/lib/strlen.S | 4 +--
arch/arm64/lib/strncmp.S | 4 +--
arch/arm64/lib/strnlen.S | 4 +--
arch/arm64/lib/tishift.S | 12 +++----
25 files changed, 147 insertions(+), 118 deletions(-)
diff --git a/arch/arm64/crypto/aes-ce-ccm-core.S b/arch/arm64/crypto/aes-ce-ccm-core.S
index f2624238fd954..519309c886b99 100644
--- a/arch/arm64/crypto/aes-ce-ccm-core.S
+++ b/arch/arm64/crypto/aes-ce-ccm-core.S
@@ -80,7 +80,7 @@ CPU_LE( rev x8, x8 )
ret
.endm
-SYM_FUNC_START_LOCAL(ce_aes_ccm_crypt_tail)
+SYM_LEAF_FUNC_START_LOCAL(ce_aes_ccm_crypt_tail)
eor v0.16b, v0.16b, v5.16b /* final round mac */
eor v1.16b, v1.16b, v5.16b /* final round enc */
@@ -113,7 +113,7 @@ SYM_INNER_LABEL(ce_aes_ccm_final, SYM_L_LOCAL)
eor v0.16b, v0.16b, v1.16b /* en-/decrypt the mac */
0: st1 {v0.16b}, [x5] /* store result */
ret
-SYM_FUNC_END(ce_aes_ccm_crypt_tail)
+SYM_LEAF_FUNC_END(ce_aes_ccm_crypt_tail)
/*
* void ce_aes_ccm_encrypt(u8 out[], u8 const in[], u32 cbytes,
@@ -123,15 +123,15 @@ SYM_FUNC_END(ce_aes_ccm_crypt_tail)
* u8 const rk[], u32 rounds, u8 mac[],
* u8 ctr[], u8 const final_iv[]);
*/
-SYM_FUNC_START(ce_aes_ccm_encrypt)
+SYM_LEAF_FUNC_START(ce_aes_ccm_encrypt)
movi v22.16b, #255
aes_ccm_do_crypt 1
-SYM_FUNC_END(ce_aes_ccm_encrypt)
+SYM_LEAF_FUNC_END(ce_aes_ccm_encrypt)
-SYM_FUNC_START(ce_aes_ccm_decrypt)
+SYM_LEAF_FUNC_START(ce_aes_ccm_decrypt)
movi v22.16b, #0
aes_ccm_do_crypt 0
-SYM_FUNC_END(ce_aes_ccm_decrypt)
+SYM_LEAF_FUNC_END(ce_aes_ccm_decrypt)
.section ".rodata", "a"
.align 6
diff --git a/arch/arm64/crypto/aes-neonbs-core.S b/arch/arm64/crypto/aes-neonbs-core.S
index baf450717b24b..01cd6c0aa10ae 100644
--- a/arch/arm64/crypto/aes-neonbs-core.S
+++ b/arch/arm64/crypto/aes-neonbs-core.S
@@ -381,7 +381,7 @@ ISRM0: .octa 0x0306090c00070a0d01040b0e0205080f
/*
* void aesbs_convert_key(u8 out[], u32 const rk[], int rounds)
*/
-SYM_FUNC_START(aesbs_convert_key)
+SYM_LEAF_FUNC_START(aesbs_convert_key)
ld1 {v7.4s}, [x1], #16 // load round 0 key
ld1 {v17.4s}, [x1], #16 // load round 1 key
@@ -426,10 +426,10 @@ SYM_FUNC_START(aesbs_convert_key)
eor v17.16b, v17.16b, v7.16b
str q17, [x0]
ret
-SYM_FUNC_END(aesbs_convert_key)
+SYM_LEAF_FUNC_END(aesbs_convert_key)
.align 4
-SYM_FUNC_START_LOCAL(aesbs_encrypt8)
+SYM_LEAF_FUNC_START_LOCAL(aesbs_encrypt8)
ldr q9, [bskey], #16 // round 0 key
ldr q8, M0SR
ldr q24, SR
@@ -489,10 +489,10 @@ SYM_FUNC_START_LOCAL(aesbs_encrypt8)
eor v2.16b, v2.16b, v12.16b
eor v5.16b, v5.16b, v12.16b
ret
-SYM_FUNC_END(aesbs_encrypt8)
+SYM_LEAF_FUNC_END(aesbs_encrypt8)
.align 4
-SYM_FUNC_START_LOCAL(aesbs_decrypt8)
+SYM_LEAF_FUNC_START_LOCAL(aesbs_decrypt8)
lsl x9, rounds, #7
add bskey, bskey, x9
@@ -554,7 +554,7 @@ SYM_FUNC_START_LOCAL(aesbs_decrypt8)
eor v3.16b, v3.16b, v12.16b
eor v5.16b, v5.16b, v12.16b
ret
-SYM_FUNC_END(aesbs_decrypt8)
+SYM_LEAF_FUNC_END(aesbs_decrypt8)
/*
* aesbs_ecb_encrypt(u8 out[], u8 const in[], u8 const rk[], int rounds,
@@ -735,7 +735,7 @@ SYM_FUNC_END(aesbs_cbc_decrypt)
* aesbs_xts_decrypt(u8 out[], u8 const in[], u8 const rk[], int rounds,
* int blocks, u8 iv[])
*/
-SYM_FUNC_START_LOCAL(__xts_crypt8)
+SYM_LEAF_FUNC_START_LOCAL(__xts_crypt8)
movi v18.2s, #0x1
movi v19.2s, #0x87
uzp1 v18.4s, v18.4s, v19.4s
@@ -766,7 +766,7 @@ SYM_FUNC_START_LOCAL(__xts_crypt8)
mov bskey, x2
mov rounds, x3
br x16
-SYM_FUNC_END(__xts_crypt8)
+SYM_LEAF_FUNC_END(__xts_crypt8)
.macro __xts_crypt, do8, o0, o1, o2, o3, o4, o5, o6, o7
frame_push 0, 32
diff --git a/arch/arm64/crypto/ghash-ce-core.S b/arch/arm64/crypto/ghash-ce-core.S
index 33772d8fe6b5e..8773678e13df5 100644
--- a/arch/arm64/crypto/ghash-ce-core.S
+++ b/arch/arm64/crypto/ghash-ce-core.S
@@ -66,7 +66,7 @@
* void pmull_ghash_update_p64(int blocks, u64 dg[], const char *src,
* u64 const h[4][2], const char *head)
*/
-SYM_FUNC_START(pmull_ghash_update_p64)
+SYM_LEAF_FUNC_START(pmull_ghash_update_p64)
ld1 {SHASH.2d}, [x3]
ld1 {XL.2d}, [x1]
@@ -173,7 +173,7 @@ CPU_LE( rev64 T1.16b, T1.16b )
5: st1 {XL.2d}, [x1]
ret
-SYM_FUNC_END(pmull_ghash_update_p64)
+SYM_LEAF_FUNC_END(pmull_ghash_update_p64)
KS0 .req v8
KS1 .req v9
@@ -431,7 +431,7 @@ SYM_FUNC_START(pmull_gcm_decrypt)
pmull_gcm_do_crypt 0
SYM_FUNC_END(pmull_gcm_decrypt)
-SYM_FUNC_START_LOCAL(pmull_gcm_ghash_4x)
+SYM_LEAF_FUNC_START_LOCAL(pmull_gcm_ghash_4x)
movi MASK.16b, #0xe1
shl MASK.2d, MASK.2d, #57
@@ -512,9 +512,9 @@ SYM_FUNC_START_LOCAL(pmull_gcm_ghash_4x)
eor XL.16b, XL.16b, T2.16b
ret
-SYM_FUNC_END(pmull_gcm_ghash_4x)
+SYM_LEAF_FUNC_END(pmull_gcm_ghash_4x)
-SYM_FUNC_START_LOCAL(pmull_gcm_enc_4x)
+SYM_LEAF_FUNC_START_LOCAL(pmull_gcm_enc_4x)
ld1 {KS0.16b}, [x5] // load upper counter
sub w10, w8, #4
sub w11, w8, #3
@@ -577,7 +577,7 @@ SYM_FUNC_START_LOCAL(pmull_gcm_enc_4x)
eor INP3.16b, INP3.16b, KS3.16b
ret
-SYM_FUNC_END(pmull_gcm_enc_4x)
+SYM_LEAF_FUNC_END(pmull_gcm_enc_4x)
.section ".rodata", "a"
.align 6
diff --git a/arch/arm64/crypto/sm4-ce-ccm-core.S b/arch/arm64/crypto/sm4-ce-ccm-core.S
index fa85856f33cee..20a8853609e04 100644
--- a/arch/arm64/crypto/sm4-ce-ccm-core.S
+++ b/arch/arm64/crypto/sm4-ce-ccm-core.S
@@ -37,7 +37,7 @@
.align 3
-SYM_FUNC_START(sm4_ce_cbcmac_update)
+SYM_LEAF_FUNC_START(sm4_ce_cbcmac_update)
/* input:
* x0: round key array, CTX
* x1: mac
@@ -81,10 +81,10 @@ SYM_FUNC_START(sm4_ce_cbcmac_update)
.Lcbcmac_end:
st1 {RMAC.16b}, [x1]
ret
-SYM_FUNC_END(sm4_ce_cbcmac_update)
+SYM_LEAF_FUNC_END(sm4_ce_cbcmac_update)
.align 3
-SYM_FUNC_START(sm4_ce_ccm_final)
+SYM_LEAF_FUNC_START(sm4_ce_ccm_final)
/* input:
* x0: round key array, CTX
* x1: ctr0 (big endian, 128 bit)
@@ -102,10 +102,10 @@ SYM_FUNC_START(sm4_ce_ccm_final)
st1 {RMAC.16b}, [x2]
ret
-SYM_FUNC_END(sm4_ce_ccm_final)
+SYM_LEAF_FUNC_END(sm4_ce_ccm_final)
.align 3
-SYM_TYPED_FUNC_START(sm4_ce_ccm_enc)
+SYM_TYPED_LEAF_FUNC_START(sm4_ce_ccm_enc)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -214,10 +214,10 @@ SYM_TYPED_FUNC_START(sm4_ce_ccm_enc)
.Lccm_enc_ret:
ret
-SYM_FUNC_END(sm4_ce_ccm_enc)
+SYM_LEAF_FUNC_END(sm4_ce_ccm_enc)
.align 3
-SYM_TYPED_FUNC_START(sm4_ce_ccm_dec)
+SYM_TYPED_LEAF_FUNC_START(sm4_ce_ccm_dec)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -326,4 +326,4 @@ SYM_TYPED_FUNC_START(sm4_ce_ccm_dec)
.Lccm_dec_ret:
ret
-SYM_FUNC_END(sm4_ce_ccm_dec)
+SYM_LEAF_FUNC_END(sm4_ce_ccm_dec)
diff --git a/arch/arm64/crypto/sm4-ce-cipher-core.S b/arch/arm64/crypto/sm4-ce-cipher-core.S
index 4ac6cfbc57970..7aea346cb14cd 100644
--- a/arch/arm64/crypto/sm4-ce-cipher-core.S
+++ b/arch/arm64/crypto/sm4-ce-cipher-core.S
@@ -15,7 +15,7 @@
* void sm4_ce_do_crypt(const u32 *rk, u32 *out, const u32 *in);
*/
.text
-SYM_FUNC_START(sm4_ce_do_crypt)
+SYM_LEAF_FUNC_START(sm4_ce_do_crypt)
ld1 {v8.4s}, [x2]
ld1 {v0.4s-v3.4s}, [x0], #64
CPU_LE( rev32 v8.16b, v8.16b )
@@ -33,4 +33,4 @@ CPU_LE( rev32 v8.16b, v8.16b )
CPU_LE( rev32 v8.16b, v8.16b )
st1 {v8.4s}, [x1]
ret
-SYM_FUNC_END(sm4_ce_do_crypt)
+SYM_LEAF_FUNC_END(sm4_ce_do_crypt)
diff --git a/arch/arm64/crypto/sm4-ce-core.S b/arch/arm64/crypto/sm4-ce-core.S
index 1f3625c2c67e4..6af5b10859b88 100644
--- a/arch/arm64/crypto/sm4-ce-core.S
+++ b/arch/arm64/crypto/sm4-ce-core.S
@@ -40,7 +40,7 @@
.align 3
-SYM_FUNC_START(sm4_ce_expand_key)
+SYM_LEAF_FUNC_START(sm4_ce_expand_key)
/* input:
* x0: 128-bit key
* x1: rkey_enc
@@ -86,10 +86,10 @@ SYM_FUNC_START(sm4_ce_expand_key)
st1 {v20.16b-v23.16b}, [x2]
ret;
-SYM_FUNC_END(sm4_ce_expand_key)
+SYM_LEAF_FUNC_END(sm4_ce_expand_key)
.align 3
-SYM_FUNC_START(sm4_ce_crypt_block)
+SYM_LEAF_FUNC_START(sm4_ce_crypt_block)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -102,10 +102,10 @@ SYM_FUNC_START(sm4_ce_crypt_block)
st1 {v0.16b}, [x1];
ret;
-SYM_FUNC_END(sm4_ce_crypt_block)
+SYM_LEAF_FUNC_END(sm4_ce_crypt_block)
.align 3
-SYM_FUNC_START(sm4_ce_crypt)
+SYM_LEAF_FUNC_START(sm4_ce_crypt)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -153,10 +153,10 @@ SYM_FUNC_START(sm4_ce_crypt)
.Lcrypt_end:
ret;
-SYM_FUNC_END(sm4_ce_crypt)
+SYM_LEAF_FUNC_END(sm4_ce_crypt)
.align 3
-SYM_FUNC_START(sm4_ce_cbc_enc)
+SYM_LEAF_FUNC_START(sm4_ce_cbc_enc)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -208,10 +208,10 @@ SYM_FUNC_START(sm4_ce_cbc_enc)
st1 {RIV.16b}, [x3]
ret
-SYM_FUNC_END(sm4_ce_cbc_enc)
+SYM_LEAF_FUNC_END(sm4_ce_cbc_enc)
.align 3
-SYM_FUNC_START(sm4_ce_cbc_dec)
+SYM_LEAF_FUNC_START(sm4_ce_cbc_dec)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -306,10 +306,10 @@ SYM_FUNC_START(sm4_ce_cbc_dec)
st1 {RIV.16b}, [x3]
ret
-SYM_FUNC_END(sm4_ce_cbc_dec)
+SYM_LEAF_FUNC_END(sm4_ce_cbc_dec)
.align 3
-SYM_FUNC_START(sm4_ce_cbc_cts_enc)
+SYM_LEAF_FUNC_START(sm4_ce_cbc_cts_enc)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -354,10 +354,10 @@ SYM_FUNC_START(sm4_ce_cbc_cts_enc)
st1 {v1.16b}, [x1]
ret
-SYM_FUNC_END(sm4_ce_cbc_cts_enc)
+SYM_LEAF_FUNC_END(sm4_ce_cbc_cts_enc)
.align 3
-SYM_FUNC_START(sm4_ce_cbc_cts_dec)
+SYM_LEAF_FUNC_START(sm4_ce_cbc_cts_dec)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -400,10 +400,10 @@ SYM_FUNC_START(sm4_ce_cbc_cts_dec)
st1 {v0.16b}, [x1]
ret
-SYM_FUNC_END(sm4_ce_cbc_cts_dec)
+SYM_LEAF_FUNC_END(sm4_ce_cbc_cts_dec)
.align 3
-SYM_FUNC_START(sm4_ce_ctr_enc)
+SYM_LEAF_FUNC_START(sm4_ce_ctr_enc)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -506,7 +506,7 @@ SYM_FUNC_START(sm4_ce_ctr_enc)
stp x7, x8, [x3]
ret
-SYM_FUNC_END(sm4_ce_ctr_enc)
+SYM_LEAF_FUNC_END(sm4_ce_ctr_enc)
#define tweak_next(vt, vin, RTMP) \
@@ -517,7 +517,7 @@ SYM_FUNC_END(sm4_ce_ctr_enc)
eor vt.16b, vt.16b, RTMP.16b;
.align 3
-SYM_FUNC_START(sm4_ce_xts_enc)
+SYM_LEAF_FUNC_START(sm4_ce_xts_enc)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -681,10 +681,10 @@ SYM_FUNC_START(sm4_ce_xts_enc)
.Lxts_enc_ret:
ret
-SYM_FUNC_END(sm4_ce_xts_enc)
+SYM_LEAF_FUNC_END(sm4_ce_xts_enc)
.align 3
-SYM_FUNC_START(sm4_ce_xts_dec)
+SYM_LEAF_FUNC_START(sm4_ce_xts_dec)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -848,10 +848,10 @@ SYM_FUNC_START(sm4_ce_xts_dec)
.Lxts_dec_ret:
ret
-SYM_FUNC_END(sm4_ce_xts_dec)
+SYM_LEAF_FUNC_END(sm4_ce_xts_dec)
.align 3
-SYM_FUNC_START(sm4_ce_mac_update)
+SYM_LEAF_FUNC_START(sm4_ce_mac_update)
/* input:
* x0: round key array, CTX
* x1: digest
@@ -917,7 +917,7 @@ SYM_FUNC_START(sm4_ce_mac_update)
.Lmac_ret:
st1 {RMAC.16b}, [x1]
ret
-SYM_FUNC_END(sm4_ce_mac_update)
+SYM_LEAF_FUNC_END(sm4_ce_mac_update)
.section ".rodata", "a"
diff --git a/arch/arm64/crypto/sm4-ce-gcm-core.S b/arch/arm64/crypto/sm4-ce-gcm-core.S
index 347f25d757279..dac6db8160f28 100644
--- a/arch/arm64/crypto/sm4-ce-gcm-core.S
+++ b/arch/arm64/crypto/sm4-ce-gcm-core.S
@@ -259,7 +259,7 @@
#define RH4 v19
.align 3
-SYM_FUNC_START(sm4_ce_pmull_ghash_setup)
+SYM_LEAF_FUNC_START(sm4_ce_pmull_ghash_setup)
/* input:
* x0: round key array, CTX
* x1: ghash table
@@ -293,10 +293,10 @@ SYM_FUNC_START(sm4_ce_pmull_ghash_setup)
st1 {RH1.16b-RH4.16b}, [x1]
ret
-SYM_FUNC_END(sm4_ce_pmull_ghash_setup)
+SYM_LEAF_FUNC_END(sm4_ce_pmull_ghash_setup)
.align 3
-SYM_FUNC_START(pmull_ghash_update)
+SYM_LEAF_FUNC_START(pmull_ghash_update)
/* input:
* x0: ghash table
* x1: ghash result
@@ -368,10 +368,10 @@ SYM_FUNC_START(pmull_ghash_update)
st1 {RHASH.2d}, [x1]
ret
-SYM_FUNC_END(pmull_ghash_update)
+SYM_LEAF_FUNC_END(pmull_ghash_update)
.align 3
-SYM_TYPED_FUNC_START(sm4_ce_pmull_gcm_enc)
+SYM_TYPED_LEAF_FUNC_START(sm4_ce_pmull_gcm_enc)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -534,7 +534,7 @@ SYM_TYPED_FUNC_START(sm4_ce_pmull_gcm_enc)
st1 {RHASH.2d}, [x5]
ret
-SYM_FUNC_END(sm4_ce_pmull_gcm_enc)
+SYM_LEAF_FUNC_END(sm4_ce_pmull_gcm_enc)
#undef RR1
#undef RR3
@@ -582,7 +582,7 @@ SYM_FUNC_END(sm4_ce_pmull_gcm_enc)
#define RH3 v20
.align 3
-SYM_TYPED_FUNC_START(sm4_ce_pmull_gcm_dec)
+SYM_TYPED_LEAF_FUNC_START(sm4_ce_pmull_gcm_dec)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -726,7 +726,7 @@ SYM_TYPED_FUNC_START(sm4_ce_pmull_gcm_dec)
st1 {RHASH.2d}, [x5]
ret
-SYM_FUNC_END(sm4_ce_pmull_gcm_dec)
+SYM_LEAF_FUNC_END(sm4_ce_pmull_gcm_dec)
.section ".rodata", "a"
.align 4
diff --git a/arch/arm64/crypto/sm4-neon-core.S b/arch/arm64/crypto/sm4-neon-core.S
index 734dc7193610c..d1fe37fce13ad 100644
--- a/arch/arm64/crypto/sm4-neon-core.S
+++ b/arch/arm64/crypto/sm4-neon-core.S
@@ -257,7 +257,7 @@
.align 3
-SYM_FUNC_START(sm4_neon_crypt)
+SYM_LEAF_FUNC_START(sm4_neon_crypt)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -318,10 +318,10 @@ SYM_FUNC_START(sm4_neon_crypt)
.Lcrypt_end:
ret
-SYM_FUNC_END(sm4_neon_crypt)
+SYM_LEAF_FUNC_END(sm4_neon_crypt)
.align 3
-SYM_FUNC_START(sm4_neon_cbc_dec)
+SYM_LEAF_FUNC_START(sm4_neon_cbc_dec)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -435,10 +435,10 @@ SYM_FUNC_START(sm4_neon_cbc_dec)
st1 {RIV.16b}, [x3]
ret
-SYM_FUNC_END(sm4_neon_cbc_dec)
+SYM_LEAF_FUNC_END(sm4_neon_cbc_dec)
.align 3
-SYM_FUNC_START(sm4_neon_ctr_crypt)
+SYM_LEAF_FUNC_START(sm4_neon_ctr_crypt)
/* input:
* x0: round key array, CTX
* x1: dst
@@ -563,4 +563,4 @@ SYM_FUNC_START(sm4_neon_ctr_crypt)
stp x7, x8, [x3]
ret
-SYM_FUNC_END(sm4_neon_ctr_crypt)
+SYM_LEAF_FUNC_END(sm4_neon_ctr_crypt)
diff --git a/arch/arm64/include/asm/linkage.h b/arch/arm64/include/asm/linkage.h
index d1f7a16729d25..c4377e6ba0a67 100644
--- a/arch/arm64/include/asm/linkage.h
+++ b/arch/arm64/include/asm/linkage.h
@@ -3,6 +3,12 @@
#ifdef __ASSEMBLER__
#include <asm/assembler.h>
+
+/*
+ * Do not generate .eh_frame. Only generate .debug_frame and optionally
+ * .sframe (via assembler option --gsframe[-N]).
+ */
+ .cfi_sections .debug_frame
#endif
#define __ALIGN .balign CONFIG_FUNCTION_ALIGNMENT
@@ -47,4 +53,27 @@
#define __bss_pgtbl __section(".bss..pgtbl") __aligned(PAGE_SIZE)
+/*
+ * SYM_[TYPED_]LEAF_FUNC_[START|END] macros add CFI minimal CFI directives
+ * allowing .sframe data to be generated for functions which do not modify the
+ * LR (x30). Unwind data will not be correct if these macros are used on
+ * non-leaf functions, as additional CFI directives would be necessary in such
+ * cases.
+ */
+#define SYM_LEAF_FUNC_START(name) \
+ .cfi_startproc ; \
+ SYM_FUNC_START(name)
+
+#define SYM_LEAF_FUNC_END(name) \
+ .cfi_endproc ; \
+ SYM_FUNC_END(name)
+
+#define SYM_LEAF_FUNC_START_LOCAL(name) \
+ .cfi_startproc ; \
+ SYM_FUNC_START_LOCAL(name)
+
+#define SYM_TYPED_LEAF_FUNC_START(name) \
+ .cfi_startproc ; \
+ SYM_TYPED_FUNC_START(name)
+
#endif
diff --git a/arch/arm64/lib/clear_page.S b/arch/arm64/lib/clear_page.S
index bd6f7d5eb6eb6..fceb875c35702 100644
--- a/arch/arm64/lib/clear_page.S
+++ b/arch/arm64/lib/clear_page.S
@@ -14,7 +14,7 @@
* Parameters:
* x0 - dest
*/
-SYM_FUNC_START(__pi_clear_page)
+SYM_LEAF_FUNC_START(__pi_clear_page)
#ifdef CONFIG_AS_HAS_MOPS
.arch_extension mops
alternative_if_not ARM64_HAS_MOPS
@@ -48,6 +48,6 @@ alternative_else_nop_endif
tst x0, #(PAGE_SIZE - 1)
b.ne 2b
ret
-SYM_FUNC_END(__pi_clear_page)
+SYM_LEAF_FUNC_END(__pi_clear_page)
SYM_FUNC_ALIAS(clear_page, __pi_clear_page)
EXPORT_SYMBOL(clear_page)
diff --git a/arch/arm64/lib/clear_user.S b/arch/arm64/lib/clear_user.S
index de9a303b6ad0e..cf07c010ca927 100644
--- a/arch/arm64/lib/clear_user.S
+++ b/arch/arm64/lib/clear_user.S
@@ -17,7 +17,7 @@
* Alignment fixed up by hardware.
*/
-SYM_FUNC_START(__arch_clear_user)
+SYM_LEAF_FUNC_START(__arch_clear_user)
add x2, x0, x1
#ifdef CONFIG_AS_HAS_MOPS
@@ -68,5 +68,5 @@ USER(7f, sttrb wzr, [x2, #-1])
8: add x0, x0, #4 // ...or the second word of the 4-7 byte case
9: sub x0, x2, x0
ret
-SYM_FUNC_END(__arch_clear_user)
+SYM_LEAF_FUNC_END(__arch_clear_user)
EXPORT_SYMBOL(__arch_clear_user)
diff --git a/arch/arm64/lib/copy_from_user.S b/arch/arm64/lib/copy_from_user.S
index 400057d607ecd..e9bb9c2dd8e12 100644
--- a/arch/arm64/lib/copy_from_user.S
+++ b/arch/arm64/lib/copy_from_user.S
@@ -61,7 +61,7 @@
end .req x5
srcin .req x15
-SYM_FUNC_START(__arch_copy_from_user)
+SYM_LEAF_FUNC_START(__arch_copy_from_user)
add end, x0, x2
mov srcin, x1
#include "copy_template.S"
@@ -79,5 +79,5 @@ USER(9998f, ldtrb tmp1w, [srcin])
strb tmp1w, [dst], #1
9998: sub x0, end, dst // bytes not copied
ret
-SYM_FUNC_END(__arch_copy_from_user)
+SYM_LEAF_FUNC_END(__arch_copy_from_user)
EXPORT_SYMBOL(__arch_copy_from_user)
diff --git a/arch/arm64/lib/copy_page.S b/arch/arm64/lib/copy_page.S
index e6374e7e55110..b6048d6483064 100644
--- a/arch/arm64/lib/copy_page.S
+++ b/arch/arm64/lib/copy_page.S
@@ -17,7 +17,7 @@
* x0 - dest
* x1 - src
*/
-SYM_FUNC_START(__pi_copy_page)
+SYM_LEAF_FUNC_START(__pi_copy_page)
#ifdef CONFIG_AS_HAS_MOPS
.arch_extension mops
alternative_if_not ARM64_HAS_MOPS
@@ -77,6 +77,6 @@ alternative_else_nop_endif
stnp x16, x17, [x0, #112 - 256]
ret
-SYM_FUNC_END(__pi_copy_page)
+SYM_LEAF_FUNC_END(__pi_copy_page)
SYM_FUNC_ALIAS(copy_page, __pi_copy_page)
EXPORT_SYMBOL(copy_page)
diff --git a/arch/arm64/lib/copy_to_user.S b/arch/arm64/lib/copy_to_user.S
index 819f2e3fc7a93..aea10a5b3cbac 100644
--- a/arch/arm64/lib/copy_to_user.S
+++ b/arch/arm64/lib/copy_to_user.S
@@ -60,7 +60,7 @@
end .req x5
srcin .req x15
-SYM_FUNC_START(__arch_copy_to_user)
+SYM_LEAF_FUNC_START(__arch_copy_to_user)
add end, x0, x2
mov srcin, x1
#include "copy_template.S"
@@ -79,5 +79,5 @@ USER(9998f, sttrb tmp1w, [dst])
add dst, dst, #1
9998: sub x0, end, dst // bytes not copied
ret
-SYM_FUNC_END(__arch_copy_to_user)
+SYM_LEAF_FUNC_END(__arch_copy_to_user)
EXPORT_SYMBOL(__arch_copy_to_user)
diff --git a/arch/arm64/lib/memchr.S b/arch/arm64/lib/memchr.S
index 37a9f2a4f7f4b..909599bba5bf7 100644
--- a/arch/arm64/lib/memchr.S
+++ b/arch/arm64/lib/memchr.S
@@ -38,7 +38,7 @@
.p2align 4
nop
-SYM_FUNC_START(__pi_memchr)
+SYM_LEAF_FUNC_START(__pi_memchr)
and chrin, chrin, #0xff
lsr wordcnt, cntin, #3
cbz wordcnt, L(byte_loop)
@@ -71,6 +71,6 @@ CPU_LE( rev tmp, tmp)
L(not_found):
mov result, #0
ret
-SYM_FUNC_END(__pi_memchr)
+SYM_LEAF_FUNC_END(__pi_memchr)
SYM_FUNC_ALIAS_WEAK(memchr, __pi_memchr)
EXPORT_SYMBOL_NOKASAN(memchr)
diff --git a/arch/arm64/lib/memcmp.S b/arch/arm64/lib/memcmp.S
index a5ccf2c55f911..91ee3a00e6644 100644
--- a/arch/arm64/lib/memcmp.S
+++ b/arch/arm64/lib/memcmp.S
@@ -32,7 +32,7 @@
#define tmp1 x7
#define tmp2 x8
-SYM_FUNC_START(__pi_memcmp)
+SYM_LEAF_FUNC_START(__pi_memcmp)
subs limit, limit, 8
b.lo L(less8)
@@ -134,6 +134,6 @@ L(byte_loop):
b.eq L(byte_loop)
sub result, data1w, data2w
ret
-SYM_FUNC_END(__pi_memcmp)
+SYM_LEAF_FUNC_END(__pi_memcmp)
SYM_FUNC_ALIAS_WEAK(memcmp, __pi_memcmp)
EXPORT_SYMBOL_NOKASAN(memcmp)
diff --git a/arch/arm64/lib/memcpy.S b/arch/arm64/lib/memcpy.S
index 9b99106fb95f1..90caf402ea7d9 100644
--- a/arch/arm64/lib/memcpy.S
+++ b/arch/arm64/lib/memcpy.S
@@ -57,7 +57,7 @@
The loop tail is handled by always copying 64 bytes from the end.
*/
-SYM_FUNC_START_LOCAL(__pi_memcpy_generic)
+SYM_LEAF_FUNC_START_LOCAL(__pi_memcpy_generic)
add srcend, src, count
add dstend, dstin, count
cmp count, 128
@@ -238,11 +238,11 @@ L(copy64_from_start):
stp B_l, B_h, [dstin, 16]
stp C_l, C_h, [dstin]
ret
-SYM_FUNC_END(__pi_memcpy_generic)
+SYM_LEAF_FUNC_END(__pi_memcpy_generic)
#ifdef CONFIG_AS_HAS_MOPS
.arch_extension mops
-SYM_FUNC_START(__pi_memcpy)
+SYM_LEAF_FUNC_START(__pi_memcpy)
alternative_if_not ARM64_HAS_MOPS
b __pi_memcpy_generic
alternative_else_nop_endif
@@ -252,7 +252,7 @@ alternative_else_nop_endif
cpym [dst]!, [src]!, count!
cpye [dst]!, [src]!, count!
ret
-SYM_FUNC_END(__pi_memcpy)
+SYM_LEAF_FUNC_END(__pi_memcpy)
#else
SYM_FUNC_ALIAS(__pi_memcpy, __pi_memcpy_generic)
#endif
diff --git a/arch/arm64/lib/memset.S b/arch/arm64/lib/memset.S
index 97157da65ec6b..8ee307f5891b4 100644
--- a/arch/arm64/lib/memset.S
+++ b/arch/arm64/lib/memset.S
@@ -43,7 +43,7 @@ dst .req x8
tmp3w .req w9
tmp3 .req x9
-SYM_FUNC_START_LOCAL(__pi_memset_generic)
+SYM_LEAF_FUNC_START_LOCAL(__pi_memset_generic)
mov dst, dstin /* Preserve return value. */
and A_lw, val, #255
orr A_lw, A_lw, A_lw, lsl #8
@@ -202,11 +202,11 @@ SYM_FUNC_START_LOCAL(__pi_memset_generic)
ands count, count, zva_bits_x
b.ne .Ltail_maybe_long
ret
-SYM_FUNC_END(__pi_memset_generic)
+SYM_LEAF_FUNC_END(__pi_memset_generic)
#ifdef CONFIG_AS_HAS_MOPS
.arch_extension mops
-SYM_FUNC_START(__pi_memset)
+SYM_LEAF_FUNC_START(__pi_memset)
alternative_if_not ARM64_HAS_MOPS
b __pi_memset_generic
alternative_else_nop_endif
@@ -216,7 +216,7 @@ alternative_else_nop_endif
setm [dst]!, count!, val_x
sete [dst]!, count!, val_x
ret
-SYM_FUNC_END(__pi_memset)
+SYM_LEAF_FUNC_END(__pi_memset)
#else
SYM_FUNC_ALIAS(__pi_memset, __pi_memset_generic)
#endif
diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S
index 5018ac03b6bf3..442202cd02a32 100644
--- a/arch/arm64/lib/mte.S
+++ b/arch/arm64/lib/mte.S
@@ -27,14 +27,14 @@
* Clear the tags in a page
* x0 - address of the page to be cleared
*/
-SYM_FUNC_START(mte_clear_page_tags)
+SYM_LEAF_FUNC_START(mte_clear_page_tags)
multitag_transfer_size x1, x2
1: stgm xzr, [x0]
add x0, x0, x1
tst x0, #(PAGE_SIZE - 1)
b.ne 1b
ret
-SYM_FUNC_END(mte_clear_page_tags)
+SYM_LEAF_FUNC_END(mte_clear_page_tags)
/*
* Zero the page and tags at the same time
@@ -42,7 +42,7 @@ SYM_FUNC_END(mte_clear_page_tags)
* Parameters:
* x0 - address to the beginning of the page
*/
-SYM_FUNC_START(mte_zero_clear_page_tags)
+SYM_LEAF_FUNC_START(mte_zero_clear_page_tags)
and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag
mrs x1, dczid_el0
tbnz x1, #4, 2f // Branch if DC GZVA is prohibited
@@ -60,14 +60,14 @@ SYM_FUNC_START(mte_zero_clear_page_tags)
tst x0, #(PAGE_SIZE - 1)
b.ne 2b
ret
-SYM_FUNC_END(mte_zero_clear_page_tags)
+SYM_LEAF_FUNC_END(mte_zero_clear_page_tags)
/*
* Copy the tags from the source page to the destination one
* x0 - address of the destination page
* x1 - address of the source page
*/
-SYM_FUNC_START(mte_copy_page_tags)
+SYM_LEAF_FUNC_START(mte_copy_page_tags)
mov x2, x0
mov x3, x1
multitag_transfer_size x5, x6
@@ -78,7 +78,7 @@ SYM_FUNC_START(mte_copy_page_tags)
tst x2, #(PAGE_SIZE - 1)
b.ne 1b
ret
-SYM_FUNC_END(mte_copy_page_tags)
+SYM_LEAF_FUNC_END(mte_copy_page_tags)
/*
* Read tags from a user buffer (one tag per byte) and set the corresponding
@@ -89,7 +89,7 @@ SYM_FUNC_END(mte_copy_page_tags)
* Returns:
* x0 - number of tags read/set
*/
-SYM_FUNC_START(mte_copy_tags_from_user)
+SYM_LEAF_FUNC_START(mte_copy_tags_from_user)
mov x3, x1
cbz x2, 2f
1:
@@ -103,7 +103,7 @@ USER(2f, ldtrb w4, [x1])
// exception handling and function return
2: sub x0, x1, x3 // update the number of tags set
ret
-SYM_FUNC_END(mte_copy_tags_from_user)
+SYM_LEAF_FUNC_END(mte_copy_tags_from_user)
/*
* Get the tags from a kernel address range and write the tag values to the
@@ -114,7 +114,7 @@ SYM_FUNC_END(mte_copy_tags_from_user)
* Returns:
* x0 - number of tags read/set
*/
-SYM_FUNC_START(mte_copy_tags_to_user)
+SYM_LEAF_FUNC_START(mte_copy_tags_to_user)
mov x3, x0
cbz x2, 2f
1:
@@ -129,14 +129,14 @@ USER(2f, sttrb w4, [x0])
// exception handling and function return
2: sub x0, x0, x3 // update the number of tags copied
ret
-SYM_FUNC_END(mte_copy_tags_to_user)
+SYM_LEAF_FUNC_END(mte_copy_tags_to_user)
/*
* Save the tags in a page
* x0 - page address
* x1 - tag storage, MTE_PAGE_TAG_STORAGE bytes
*/
-SYM_FUNC_START(mte_save_page_tags)
+SYM_LEAF_FUNC_START(mte_save_page_tags)
multitag_transfer_size x7, x5
1:
mov x2, #0
@@ -153,14 +153,14 @@ SYM_FUNC_START(mte_save_page_tags)
b.ne 1b
ret
-SYM_FUNC_END(mte_save_page_tags)
+SYM_LEAF_FUNC_END(mte_save_page_tags)
/*
* Restore the tags in a page
* x0 - page address
* x1 - tag storage, MTE_PAGE_TAG_STORAGE bytes
*/
-SYM_FUNC_START(mte_restore_page_tags)
+SYM_LEAF_FUNC_START(mte_restore_page_tags)
multitag_transfer_size x7, x5
1:
ldr x2, [x1], #8
@@ -174,4 +174,4 @@ SYM_FUNC_START(mte_restore_page_tags)
b.ne 1b
ret
-SYM_FUNC_END(mte_restore_page_tags)
+SYM_LEAF_FUNC_END(mte_restore_page_tags)
diff --git a/arch/arm64/lib/strchr.S b/arch/arm64/lib/strchr.S
index 94ee67a6b212c..455582efd07ae 100644
--- a/arch/arm64/lib/strchr.S
+++ b/arch/arm64/lib/strchr.S
@@ -18,7 +18,7 @@
* Returns:
* x0 - address of first occurrence of 'c' or 0
*/
-SYM_FUNC_START(__pi_strchr)
+SYM_LEAF_FUNC_START(__pi_strchr)
and w1, w1, #0xff
1: ldrb w2, [x0], #1
cmp w2, w1
@@ -28,7 +28,7 @@ SYM_FUNC_START(__pi_strchr)
cmp w2, w1
csel x0, x0, xzr, eq
ret
-SYM_FUNC_END(__pi_strchr)
+SYM_LEAF_FUNC_END(__pi_strchr)
SYM_FUNC_ALIAS_WEAK(strchr, __pi_strchr)
EXPORT_SYMBOL_NOKASAN(strchr)
diff --git a/arch/arm64/lib/strcmp.S b/arch/arm64/lib/strcmp.S
index 9b89b45336074..d0ce2040a32b9 100644
--- a/arch/arm64/lib/strcmp.S
+++ b/arch/arm64/lib/strcmp.S
@@ -53,7 +53,7 @@
NUL too in big-endian, byte-reverse the data before the NUL check. */
-SYM_FUNC_START(__pi_strcmp)
+SYM_LEAF_FUNC_START(__pi_strcmp)
sub off2, src2, src1
mov zeroones, REP8_01
and tmp, src1, 7
@@ -185,6 +185,6 @@ L(tail):
L(done):
sub result, data1, data2
ret
-SYM_FUNC_END(__pi_strcmp)
+SYM_LEAF_FUNC_END(__pi_strcmp)
SYM_FUNC_ALIAS_WEAK(strcmp, __pi_strcmp)
EXPORT_SYMBOL_NOKASAN(strcmp)
diff --git a/arch/arm64/lib/strlen.S b/arch/arm64/lib/strlen.S
index 4919fe81ae540..a5d4151548b5f 100644
--- a/arch/arm64/lib/strlen.S
+++ b/arch/arm64/lib/strlen.S
@@ -79,7 +79,7 @@
whether the first fetch, which may be misaligned, crosses a page
boundary. */
-SYM_FUNC_START(__pi_strlen)
+SYM_LEAF_FUNC_START(__pi_strlen)
and tmp1, srcin, MIN_PAGE_SIZE - 1
mov zeroones, REP8_01
cmp tmp1, MIN_PAGE_SIZE - 16
@@ -208,6 +208,6 @@ L(page_cross):
csel data1, data1, tmp4, eq
csel data2, data2, tmp2, eq
b L(page_cross_entry)
-SYM_FUNC_END(__pi_strlen)
+SYM_LEAF_FUNC_END(__pi_strlen)
SYM_FUNC_ALIAS_WEAK(strlen, __pi_strlen)
EXPORT_SYMBOL_NOKASAN(strlen)
diff --git a/arch/arm64/lib/strncmp.S b/arch/arm64/lib/strncmp.S
index fe7bbc0b42a78..8fd5c5d7dc2ad 100644
--- a/arch/arm64/lib/strncmp.S
+++ b/arch/arm64/lib/strncmp.S
@@ -58,7 +58,7 @@
#define LS_BK lsl
#endif
-SYM_FUNC_START(__pi_strncmp)
+SYM_LEAF_FUNC_START(__pi_strncmp)
cbz limit, L(ret0)
eor tmp1, src1, src2
mov zeroones, #REP8_01
@@ -305,6 +305,6 @@ L(syndrome_check):
L(ret0):
mov result, #0
ret
-SYM_FUNC_END(__pi_strncmp)
+SYM_LEAF_FUNC_END(__pi_strncmp)
SYM_FUNC_ALIAS_WEAK(strncmp, __pi_strncmp)
EXPORT_SYMBOL_NOKASAN(strncmp)
diff --git a/arch/arm64/lib/strnlen.S b/arch/arm64/lib/strnlen.S
index d5ac0e10a01db..9f3f02e3f7e2c 100644
--- a/arch/arm64/lib/strnlen.S
+++ b/arch/arm64/lib/strnlen.S
@@ -47,7 +47,7 @@ limit_wd .req x14
#define REP8_7f 0x7f7f7f7f7f7f7f7f
#define REP8_80 0x8080808080808080
-SYM_FUNC_START(__pi_strnlen)
+SYM_LEAF_FUNC_START(__pi_strnlen)
cbz limit, .Lhit_limit
mov zeroones, #REP8_01
bic src, srcin, #15
@@ -156,7 +156,7 @@ CPU_LE( lsr tmp2, tmp2, tmp4 ) /* Shift (tmp1 & 63). */
.Lhit_limit:
mov len, limit
ret
-SYM_FUNC_END(__pi_strnlen)
+SYM_LEAF_FUNC_END(__pi_strnlen)
SYM_FUNC_ALIAS_WEAK(strnlen, __pi_strnlen)
EXPORT_SYMBOL_NOKASAN(strnlen)
diff --git a/arch/arm64/lib/tishift.S b/arch/arm64/lib/tishift.S
index a88613834fb07..b12d7f6a60031 100644
--- a/arch/arm64/lib/tishift.S
+++ b/arch/arm64/lib/tishift.S
@@ -7,7 +7,7 @@
#include <asm/assembler.h>
-SYM_FUNC_START(__ashlti3)
+SYM_LEAF_FUNC_START(__ashlti3)
cbz x2, 1f
mov x3, #64
sub x3, x3, x2
@@ -26,10 +26,10 @@ SYM_FUNC_START(__ashlti3)
lsl x1, x0, x1
mov x0, x2
ret
-SYM_FUNC_END(__ashlti3)
+SYM_LEAF_FUNC_END(__ashlti3)
EXPORT_SYMBOL(__ashlti3)
-SYM_FUNC_START(__ashrti3)
+SYM_LEAF_FUNC_START(__ashrti3)
cbz x2, 1f
mov x3, #64
sub x3, x3, x2
@@ -48,10 +48,10 @@ SYM_FUNC_START(__ashrti3)
asr x0, x1, x0
mov x1, x2
ret
-SYM_FUNC_END(__ashrti3)
+SYM_LEAF_FUNC_END(__ashrti3)
EXPORT_SYMBOL(__ashrti3)
-SYM_FUNC_START(__lshrti3)
+SYM_LEAF_FUNC_START(__lshrti3)
cbz x2, 1f
mov x3, #64
sub x3, x3, x2
@@ -70,5 +70,5 @@ SYM_FUNC_START(__lshrti3)
lsr x0, x1, x0
mov x1, x2
ret
-SYM_FUNC_END(__lshrti3)
+SYM_LEAF_FUNC_END(__lshrti3)
EXPORT_SYMBOL(__lshrti3)
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v7 10/11] arm64, crypto/lib: Annotate leaf functions with CFI info
2026-09-18 22:41 ` [PATCH v7 10/11] arm64, crypto/lib: Annotate leaf functions with CFI info Dylan Hatch
@ 2026-09-18 22:58 ` sashiko-bot
0 siblings, 0 replies; 15+ messages in thread
From: sashiko-bot @ 2026-09-18 22:58 UTC (permalink / raw)
To: Peter Zijlstra, Roman Gushchin, Indu Bhagat, Dylan Hatch,
Jens Remus, Catalin Marinas, Steven Rostedt, Jiri Kosina,
Weinan Liu, Mark Rutland, Josh Poimboeuf, Will Deacon
Cc: Mostafa Saleh, Herbert Xu, David S. Miller, linux-toolchains,
live-patching, joe.lawrence, Prasanna Kumar T S M,
Puranjay Mohan, linux-kernel, linux-arm-kernel, Randy Dunlap,
Song Liu
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] arm64: CFI start directive placed before function alignment corrupts DWARF FDE bounds
--
commit 498fccd9732a9f4edcb2b385432263d6681de459
Author: Dylan Hatch <dylanbhatch@google.com>
arm64, crypto/lib: Annotate leaf functions with CFI info
This commit adds simple CFI annotations to assembly leaf functions so that
the link register can be recovered by the unwinder. It introduces the
SYM_LEAF_FUNC_START macro family for these annotations.
> diff --git a/arch/arm64/include/asm/linkage.h b/arch/arm64/include/asm/linkage.h
> index d1f7a16729d25..c4377e6ba0a67 100644
> --- a/arch/arm64/include/asm/linkage.h
> +++ b/arch/arm64/include/asm/linkage.h
> @@ -47,4 +53,27 @@
>
> #define __bss_pgtbl __section(".bss..pgtbl") __aligned(PAGE_SIZE)
>
> +/*
> + * SYM_[TYPED_]LEAF_FUNC_[START|END] macros add CFI minimal CFI directives
> + * allowing .sframe data to be generated for functions which do not modify the
> + * LR (x30). Unwind data will not be correct if these macros are used on
> + * non-leaf functions, as additional CFI directives would be necessary in such
> + * cases.
> + */
> +#define SYM_LEAF_FUNC_START(name) \
> + .cfi_startproc ; \
> + SYM_FUNC_START(name)
[Severity: Medium]
Will placing .cfi_startproc before SYM_FUNC_START() cause the DWARF FDE bounds
to include alignment padding bytes?
Since SYM_FUNC_START() emits the .balign directive for the function,
placing .cfi_startproc before it captures this padding. This can misalign
the FDE start address relative to the function symbol, which could confuse
stack unwinders like the sframe unwinder.
Should the .cfi_startproc directive be emitted after the alignment within
SYM_FUNC_START() completes?
> +
> +#define SYM_LEAF_FUNC_END(name) \
> + .cfi_endproc ; \
> + SYM_FUNC_END(name)
> +
> +#define SYM_LEAF_FUNC_START_LOCAL(name) \
> + .cfi_startproc ; \
> + SYM_FUNC_START_LOCAL(name)
[Severity: Medium]
Does the same alignment issue apply to SYM_LEAF_FUNC_START_LOCAL()? The
.cfi_startproc directive is emitted before SYM_FUNC_START_LOCAL(), which
also handles function alignment.
> +
> +#define SYM_TYPED_LEAF_FUNC_START(name) \
> + .cfi_startproc ; \
> + SYM_TYPED_FUNC_START(name)
[Severity: Medium]
Could this also lead to misaligned FDE start addresses for typed leaf
functions, since .cfi_startproc is placed before SYM_TYPED_FUNC_START()
alignment?
> +
> #endif
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260918224157.1471085-1-dylanbhatch@google.com?part=10
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v7 11/11] unwind: arm64: Use sframe to unwind interrupt frames
2026-09-18 22:41 [PATCH v7 00/11] unwind, arm64: add sframe unwinder for kernel Dylan Hatch
` (9 preceding siblings ...)
2026-09-18 22:41 ` [PATCH v7 10/11] arm64, crypto/lib: Annotate leaf functions with CFI info Dylan Hatch
@ 2026-09-18 22:41 ` Dylan Hatch
2026-09-18 23:00 ` sashiko-bot
10 siblings, 1 reply; 15+ messages in thread
From: Dylan Hatch @ 2026-09-18 22:41 UTC (permalink / raw)
To: Roman Gushchin, Weinan Liu, Will Deacon, Josh Poimboeuf,
Indu Bhagat, Peter Zijlstra, Steven Rostedt, Catalin Marinas,
Jiri Kosina, Mark Rutland, Jens Remus
Cc: Dylan Hatch, Prasanna Kumar T S M, Puranjay Mohan, Song Liu,
joe.lawrence, linux-toolchains, linux-kernel, live-patching,
linux-arm-kernel, Randy Dunlap, Mostafa Saleh, Herbert Xu,
David S. Miller
Add kunwind_next_regs_sframe() function to unwind by sframe info if
present. Use this method at exception boundaries, falling back to
frame-pointer unwind only on failure. In such failure cases, the
stacktrace is considered unreliable. During normal unwind, prefer frame
pointer unwind (for better performance).
This change restores the LR behavior originally introduced in commit
c2c6b27b5aa14fa2 ("arm64: stacktrace: unwind exception boundaries"),
But later removed in commit 32ed1205682e ("arm64: stacktrace: Skip
reporting LR at exception boundaries")
This can be done because the sframe data can be used to determine
whether the LR is current for the PC value recovered from pt_regs at the
exception boundary.
Co-developed-by: Weinan Liu <wnliu@google.com>
Signed-off-by: Weinan Liu <wnliu@google.com>
Reviewed-by: Prasanna Kumar T S M <ptsm@linux.microsoft.com>
Reviewed-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Dylan Hatch <dylanbhatch@google.com>
---
Changes since v6:
- (sashiko) Fix cfa alignment check.
- (sashiko) Fix kunwind_next_regs_sframe() stub name.
- Switch sframe_find_kernel() call to sframe_find() after API change.
- Simplify FP and RA recovery logic to reflect simpler API and lack of
flexible FDE support.
- (sashiko) Simlify arch_stack_walk_reliable(), dropping unnecessary
data struct and dead code.
---
arch/arm64/kernel/stacktrace.c | 186 ++++++++++++++++++++++++++++++---
1 file changed, 174 insertions(+), 12 deletions(-)
diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c
index 3ebcf8c53fb04..5aaeac9611015 100644
--- a/arch/arm64/kernel/stacktrace.c
+++ b/arch/arm64/kernel/stacktrace.c
@@ -13,6 +13,7 @@
#include <linux/sched.h>
#include <linux/sched/debug.h>
#include <linux/sched/task_stack.h>
+#include <linux/sframe.h>
#include <linux/stacktrace.h>
#include <asm/efi.h>
@@ -26,6 +27,7 @@ enum kunwind_source {
KUNWIND_SOURCE_CALLER,
KUNWIND_SOURCE_TASK,
KUNWIND_SOURCE_REGS_PC,
+ KUNWIND_SOURCE_REGS_LR,
};
union unwind_flags {
@@ -45,6 +47,7 @@ union unwind_flags {
* @kr_cur: When KRETPROBES is selected, holds the kretprobe instance
* associated with the most recently encountered replacement lr
* value.
+ * @unreliable: Stacktrace is unreliable.
*/
struct kunwind_state {
struct unwind_state common;
@@ -56,6 +59,7 @@ struct kunwind_state {
enum kunwind_source source;
union unwind_flags flags;
struct pt_regs *regs;
+ bool unreliable;
};
static __always_inline void
@@ -181,7 +185,6 @@ int kunwind_next_regs_pc(struct kunwind_state *state)
state->regs = regs;
state->common.pc = regs->pc;
state->common.fp = regs->regs[29];
- state->regs = NULL;
state->source = KUNWIND_SOURCE_REGS_PC;
return 0;
}
@@ -244,6 +247,162 @@ kunwind_next_frame_record(struct kunwind_state *state)
return 0;
}
+#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME
+
+static __always_inline struct stack_info *
+get_word(struct unwind_state *state, unsigned long *word)
+{
+ unsigned long addr = *word;
+ struct stack_info *info;
+
+ info = unwind_find_stack(state, addr, sizeof(addr));
+ if (!info)
+ return info;
+
+ *word = READ_ONCE(*(unsigned long *)addr);
+
+ return info;
+}
+
+static __always_inline int
+get_consume_word(struct unwind_state *state, unsigned long *word)
+{
+ struct stack_info *info;
+ unsigned long addr = *word;
+
+ info = get_word(state, word);
+ if (!info)
+ return -EINVAL;
+
+ unwind_consume_stack(state, info, addr, sizeof(addr));
+ return 0;
+}
+
+/*
+ * Unwind from a pt_regs according to sframe.
+ */
+static __always_inline int
+kunwind_next_regs_sframe(struct kunwind_state *state)
+{
+ struct unwind_frame frame;
+ unsigned long cfa, fp, ra;
+ enum kunwind_source source = KUNWIND_SOURCE_FRAME;
+ struct pt_regs *regs = state->regs;
+
+ int err;
+
+ if (WARN_ON_ONCE(state->source != KUNWIND_SOURCE_REGS_PC))
+ return -EINVAL;
+ if (WARN_ON_ONCE(!state->regs))
+ return -EINVAL;
+
+ /* FP/SP alignment 8 bytes */
+ if (state->common.fp & 0x7)
+ return -EINVAL;
+
+ err = sframe_find(state->common.pc, &frame);
+ if (err)
+ return -EINVAL;
+
+ /*
+ * A kernel unwind should always end at a FRAME_META_TYPE_FINAL
+ * frame. There should be no outermost frames within the kernel.
+ */
+ if (frame.outermost)
+ return -EINVAL;
+
+ /* Get the Canonical Frame Address (CFA) */
+ switch (frame.cfa.rule) {
+ case UNWIND_CFA_RULE_SP_OFFSET:
+ cfa = state->regs->sp;
+ break;
+ case UNWIND_CFA_RULE_FP_OFFSET:
+ if (state->common.fp < state->regs->sp)
+ return -EINVAL;
+ cfa = state->common.fp;
+ break;
+ /*
+ * UNWIND_CFA_RULE_REG_OFFSET and UNWIND_CFA_RULE_REG_OFFSET_DEREF not
+ * implemented -- flexible FDEs are not currently generated by assembler
+ * for arm64.
+ */
+ default:
+ WARN_ON_ONCE(1);
+ return -EINVAL;
+ }
+ cfa += frame.cfa.offset;
+
+ /* CFA alignment 16 bytes */
+ if (cfa & 0xf)
+ return -EINVAL;
+
+ /*
+ * Consume RA and FP from the stack. The frame record puts FP at a lower
+ * address than RA, so we read FP first.
+ */
+ switch (frame.fp.rule) {
+ case UNWIND_RULE_RETAIN:
+ fp = state->common.fp;
+ break;
+ /*
+ * UNWIND_RULE_CFA_OFFSET is currently not used for FP
+ * (e.g. SFrame cannot represent this rule).
+ */
+ case UNWIND_RULE_CFA_OFFSET_DEREF:
+ fp = cfa + frame.fp.offset;
+ if (!get_word(&state->common, &fp))
+ return -EINVAL;
+ break;
+ /*
+ * UNWIND_RULE_REG_OFFSET and UNWIND_RULE_REG_OFFSET_DEREF not
+ * implemented -- flexible FDEs are not currently generated by assembler
+ * for arm64.
+ */
+ default:
+ WARN_ON_ONCE(1);
+ return -EINVAL;
+ }
+
+ switch (frame.ra.rule) {
+ case UNWIND_RULE_RETAIN:
+ ra = regs->regs[30];
+ source = KUNWIND_SOURCE_REGS_LR;
+ break;
+
+ /*
+ * UNWIND_RULE_CFA_OFFSET doesn't make sense for RA.
+ * The return address cannot legitimately be a stack address.
+ */
+ case UNWIND_RULE_CFA_OFFSET_DEREF:
+ ra = cfa + frame.ra.offset;
+ if (get_consume_word(&state->common, &ra))
+ return -EINVAL;
+ break;
+ /*
+ * UNWIND_RULE_REG_OFFSET and UNWIND_RULE_REG_OFFSET_DEREF not
+ * implemented -- flexible FDEs are not currently generated by assembler
+ * for arm64.
+ */
+ default:
+ WARN_ON_ONCE(1);
+ return -EINVAL;
+ }
+
+ state->common.pc = ra;
+ state->common.fp = fp;
+
+ state->source = source;
+
+ return 0;
+}
+
+#else /* !CONFIG_HAVE_UNWIND_KERNEL_SFRAME */
+
+static __always_inline int
+kunwind_next_regs_sframe(struct kunwind_state *state) { return -EINVAL; }
+
+#endif /* !CONFIG_HAVE_UNWIND_KERNEL_SFRAME*/
+
/*
* Unwind from one frame record (A) to the next frame record (B).
*
@@ -259,10 +418,20 @@ kunwind_next(struct kunwind_state *state)
state->flags.all = 0;
switch (state->source) {
+ case KUNWIND_SOURCE_REGS_PC:
+ err = kunwind_next_regs_sframe(state);
+
+ if (err) {
+ /* Fallback to FP based unwinder */
+ err = kunwind_next_frame_record(state);
+ state->unreliable = true;
+ }
+ state->regs = NULL;
+ break;
case KUNWIND_SOURCE_FRAME:
case KUNWIND_SOURCE_CALLER:
case KUNWIND_SOURCE_TASK:
- case KUNWIND_SOURCE_REGS_PC:
+ case KUNWIND_SOURCE_REGS_LR:
err = kunwind_next_frame_record(state);
break;
default:
@@ -393,21 +562,13 @@ noinline noinstr void arch_stack_walk(stack_trace_consume_fn consume_entry,
static __always_inline bool
arch_reliable_kunwind_consume_entry(const struct kunwind_state *state, void *cookie)
{
- /*
- * At an exception boundary we can reliably consume the saved PC. We do
- * not know whether the LR was live when the exception was taken, and
- * so we cannot perform the next unwind step reliably.
- *
- * All that matters is whether the *entire* unwind is reliable, so give
- * up as soon as we hit an exception boundary.
- */
- if (state->source == KUNWIND_SOURCE_REGS_PC)
+ if (state->unreliable)
return false;
return arch_kunwind_consume_entry(state, cookie);
}
-noinline noinstr int arch_stack_walk_reliable(stack_trace_consume_fn consume_entry,
+noinline notrace int arch_stack_walk_reliable(stack_trace_consume_fn consume_entry,
void *cookie,
struct task_struct *task)
{
@@ -452,6 +613,7 @@ static const char *state_source_string(const struct kunwind_state *state)
case KUNWIND_SOURCE_CALLER: return "C";
case KUNWIND_SOURCE_TASK: return "T";
case KUNWIND_SOURCE_REGS_PC: return "P";
+ case KUNWIND_SOURCE_REGS_LR: return "L";
default: return "U";
}
}
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH v7 11/11] unwind: arm64: Use sframe to unwind interrupt frames
2026-09-18 22:41 ` [PATCH v7 11/11] unwind: arm64: Use sframe to unwind interrupt frames Dylan Hatch
@ 2026-09-18 23:00 ` sashiko-bot
0 siblings, 0 replies; 15+ messages in thread
From: sashiko-bot @ 2026-09-18 23:00 UTC (permalink / raw)
To: Mark Rutland, Jens Remus, Roman Gushchin, Catalin Marinas,
Peter Zijlstra, Josh Poimboeuf, Will Deacon, Weinan Liu,
Indu Bhagat, Steven Rostedt, Jiri Kosina, Dylan Hatch
Cc: Puranjay Mohan, Herbert Xu, linux-kernel, Mostafa Saleh,
Prasanna Kumar T S M, Randy Dunlap, linux-toolchains,
joe.lawrence, David S. Miller, linux-arm-kernel, live-patching,
Song Liu
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] arm64: noinstr violation by calling sframe_find() in stack unwinder
--
commit 13ca4d55a43f451bf08442b7e6af494a6c23a78e
Author: Dylan Hatch <dylanbhatch@google.com>
unwind: arm64: Use sframe to unwind interrupt frames
This patch adds kunwind_next_regs_sframe() to unwind interrupt frames using
sframe info, falling back to frame-pointer unwinding on failure. It updates
the exception boundary unwinding logic to recover the return address more
reliably when sframe data is present.
> diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c
> index 3ebcf8c53fb04..5aaeac9611015 100644
> --- a/arch/arm64/kernel/stacktrace.c
> +++ b/arch/arm64/kernel/stacktrace.c
[ ... ]
> @@ -244,6 +247,162 @@ kunwind_next_frame_record(struct kunwind_state *state)
> return 0;
> }
>
> +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME
[ ... ]
> +/*
> + * Unwind from a pt_regs according to sframe.
> + */
> +static __always_inline int
> +kunwind_next_regs_sframe(struct kunwind_state *state)
> +{
> + struct unwind_frame frame;
> + unsigned long cfa, fp, ra;
> + enum kunwind_source source = KUNWIND_SOURCE_FRAME;
> + struct pt_regs *regs = state->regs;
> +
> + int err;
> +
> + if (WARN_ON_ONCE(state->source != KUNWIND_SOURCE_REGS_PC))
> + return -EINVAL;
> + if (WARN_ON_ONCE(!state->regs))
> + return -EINVAL;
> +
> + /* FP/SP alignment 8 bytes */
> + if (state->common.fp & 0x7)
> + return -EINVAL;
> +
> + err = sframe_find(state->common.pc, &frame);
[Severity: High]
Does calling sframe_find() here violate the noinstr guarantee of the stack
unwinding path?
The unwind path is called via arch_stack_walk() -> kunwind_stack_walk() ->
kunwind_next() -> kunwind_next_regs_sframe().
Since arch_stack_walk() is explicitly marked noinstr to guarantee safe
execution in uninstrumentable contexts (like nmi), calling instrumentable
or rcu-protected functions like sframe_find() and __module_text_address()
could cause a recursive fault or kernel panic if tracing or kasan is active.
Additionally, could accessing rcu-protected module structures without rcu
watching lead to use-after-free bugs when stack unwinding is routinely
triggered by tools like perf or bpf tracing?
> + if (err)
> + return -EINVAL;
> +
--
Sashiko AI review · https://sashiko.dev/#/patchset/20260918224157.1471085-1-dylanbhatch@google.com?part=11
^ permalink raw reply [flat|nested] 15+ messages in thread