[PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[Ian Rogers]

Commit a339671db64b ("vdso: Switch get/put_unaligned() from packed struct
to memcpy()") switched __get_unaligned_t() and __put_unaligned_t() from
using a __packed struct to __builtin_memcpy() to avoid strict aliasing
undefined behavior.

On ARMv5 with GCC, __builtin_memcpy() may not be inlined and instead emit
a much slower out-of-line memcpy() call. Add CONFIG_VDSO_UNALIGNED_PACKED,
enabled on ARMv5 (CPU_32v5), to use the __packed struct implementation on
ARMv5 while keeping __builtin_memcpy() elsewhere.

Fixes: a339671db64b ("vdso: Switch get/put_unaligned() from packed struct to memcpy()")
Reported-by: Stefan Kerkmann <s.kerkmann@pengutronix.de>
Closes: https://lore.kernel.org/lkml/47f8c30b-e326-4ff0-a2bc-5683209a9e2d@pengutronix.de/
Signed-off-by: Ian Rogers <irogers@google.com>
---
 include/vdso/unaligned.h | 16 ++++++++++++++++
 lib/vdso/Kconfig         |  8 ++++++++
 2 files changed, 24 insertions(+)

diff --git a/include/vdso/unaligned.h b/include/vdso/unaligned.h
index 9076483c9fbb..726fd15efc82 100644
--- a/include/vdso/unaligned.h
+++ b/include/vdso/unaligned.h
@@ -4,6 +4,20 @@
 
 #include <linux/compiler_types.h>
 
+#ifdef CONFIG_VDSO_UNALIGNED_PACKED
+
+#define __get_unaligned_t(type, ptr) ({							\
+	const struct { type x; } __packed * __get_pptr = (typeof(__get_pptr))(ptr);	\
+	__get_pptr->x;									\
+})
+
+#define __put_unaligned_t(type, val, ptr) do {						\
+	struct { type x; } __packed * __put_pptr = (typeof(__put_pptr))(ptr);		\
+	__put_pptr->x = (val);								\
+} while (0)
+
+#else /* !CONFIG_VDSO_UNALIGNED_PACKED */
+
 /**
  * __get_unaligned_t - read an unaligned value from memory.
  * @type:	the type to load from the pointer.
@@ -41,4 +55,6 @@
 			 sizeof(__put_unaligned_val));			\
 } while (0)
 
+#endif /* CONFIG_VDSO_UNALIGNED_PACKED */
+
 #endif /* __VDSO_UNALIGNED_H */
diff --git a/lib/vdso/Kconfig b/lib/vdso/Kconfig
index 597f5f0f9681..709eb193999f 100644
--- a/lib/vdso/Kconfig
+++ b/lib/vdso/Kconfig
@@ -23,3 +23,11 @@ config VDSO_GETRANDOM
 	select VDSO_DATASTORE
 	help
 	  Selected by architectures that support vDSO getrandom().
+
+config VDSO_UNALIGNED_PACKED
+	bool
+	default CPU_32v5
+	help
+	  Use packed structs rather than memcpy() in include/vdso/unaligned.h
+	  for unaligned memory accesses. Enabled on ARMv5 where GCC may emit
+	  out-of-line memcpy() calls instead of inlining byte accesses.
-- 
2.56.0.212.g129ede20d9-goog

[PATCH v1 2/2] tools headers: Update the vdso/unaligned.h copy with the kernel sources

[Ian Rogers]

To pick up the changes in:

  vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/include/vdso/unaligned.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/tools/include/vdso/unaligned.h b/tools/include/vdso/unaligned.h
index 9076483c9fbb..726fd15efc82 100644
--- a/tools/include/vdso/unaligned.h
+++ b/tools/include/vdso/unaligned.h
@@ -4,6 +4,20 @@
 
 #include <linux/compiler_types.h>
 
+#ifdef CONFIG_VDSO_UNALIGNED_PACKED
+
+#define __get_unaligned_t(type, ptr) ({							\
+	const struct { type x; } __packed * __get_pptr = (typeof(__get_pptr))(ptr);	\
+	__get_pptr->x;									\
+})
+
+#define __put_unaligned_t(type, val, ptr) do {						\
+	struct { type x; } __packed * __put_pptr = (typeof(__put_pptr))(ptr);		\
+	__put_pptr->x = (val);								\
+} while (0)
+
+#else /* !CONFIG_VDSO_UNALIGNED_PACKED */
+
 /**
  * __get_unaligned_t - read an unaligned value from memory.
  * @type:	the type to load from the pointer.
@@ -41,4 +55,6 @@
 			 sizeof(__put_unaligned_val));			\
 } while (0)
 
+#endif /* CONFIG_VDSO_UNALIGNED_PACKED */
+
 #endif /* __VDSO_UNALIGNED_H */
-- 
2.56.0.212.g129ede20d9-goog

Re: [PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[Arnd Bergmann]

On Mon, Oct 19, 2026, at 22:12, Ian Rogers wrote:
> +
> +config VDSO_UNALIGNED_PACKED
> +	bool
> +	default CPU_32v5
> +	help
> +	  Use packed structs rather than memcpy() in include/vdso/unaligned.h
> +	  for unaligned memory accesses. Enabled on ARMv5 where GCC may emit
> +	  out-of-line memcpy() calls instead of inlining byte accesses.

As explained, I think this misrepresents the problem as Arm specific and
isn't the correct method for -munaligned-access targets on Arm, which
include CPU_32v3, CPU_32v4 and CPU_32v4T along with CPU_32v5.

I still hope we can use a version that just produces the intended code
on both -munaligned-access and -mno-unaligned-access targets like the
old version did.

     Arnd

Re: [PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]

On 20.10.2026 00:12:57, Arnd Bergmann wrote:
> On Mon, Oct 19, 2026, at 22:12, Ian Rogers wrote:
> > +
> > +config VDSO_UNALIGNED_PACKED
> > +	bool
> > +	default CPU_32v5
> > +	help
> > +	  Use packed structs rather than memcpy() in include/vdso/unaligned.h
> > +	  for unaligned memory accesses. Enabled on ARMv5 where GCC may emit
> > +	  out-of-line memcpy() calls instead of inlining byte accesses.
>
> As explained, I think this misrepresents the problem as Arm specific and
> isn't the correct method for -munaligned-access targets on Arm, which
> include CPU_32v3, CPU_32v4 and CPU_32v4T along with CPU_32v5.
>
> I still hope we can use a version that just produces the intended code
> on both -munaligned-access and -mno-unaligned-access targets like the
> old version did.

FYI: there is the kconfig Symbol HAVE_EFFICIENT_UNALIGNED_ACCESS. Can we
make use of it?

regards,
Marc

-- 
Pengutronix e.K.                 | Marc Kleine-Budde          |
Embedded Linux                   | https://www.pengutronix.de |
Vertretung Nürnberg              | Phone: +49-5121-206917-129 |
Amtsgericht Hildesheim, HRA 2686 | Fax:   +49-5121-206917-9   |

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[Arnd Bergmann]

On Tue, Oct 20, 2026, at 08:44, Marc Kleine-Budde wrote:
> On 20.10.2026 00:12:57, Arnd Bergmann wrote:
>> On Mon, Oct 19, 2026, at 22:12, Ian Rogers wrote:
>> > +
>> > +config VDSO_UNALIGNED_PACKED
>> > +	bool
>> > +	default CPU_32v5
>> > +	help
>> > +	  Use packed structs rather than memcpy() in include/vdso/unaligned.h
>> > +	  for unaligned memory accesses. Enabled on ARMv5 where GCC may emit
>> > +	  out-of-line memcpy() calls instead of inlining byte accesses.
>>
>> As explained, I think this misrepresents the problem as Arm specific and
>> isn't the correct method for -munaligned-access targets on Arm, which
>> include CPU_32v3, CPU_32v4 and CPU_32v4T along with CPU_32v5.
>>
>> I still hope we can use a version that just produces the intended code
>> on both -munaligned-access and -mno-unaligned-access targets like the
>> old version did.
>
> FYI: there is the kconfig Symbol HAVE_EFFICIENT_UNALIGNED_ACCESS. Can we
> make use of it?

That would be better, but I still hope we can just get back to the
portable code across all architectures for the in-kernel version.

I have one more request for you: can you check /proc/cpu/alignment
on the system that shows the performance regression with and without
the patch? In your original report, you have shown that Ian's patch
caused the drop in performance and that it caused the added function
calls, but I don't see any sufficiently hot code path that would
actually explain the scale of the regression, unless there is now
also a case where get_unaligned() ends up doing the wrong case and
running into a fixup handler. If this happens, you should see a
nontrivial number of fixups listed in /proc/cpu/alignment. On
a working system, the numbers should all be low but likely nonzero
because there tend to be callers that still lack get_unaligned()
wrappers and just access misaligned pointers directly.

      Arnd

Re: [PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 2701 bytes --]

On 20.10.2026 10:24:51, Arnd Bergmann wrote:
> On Tue, Oct 20, 2026, at 08:44, Marc Kleine-Budde wrote:
> > On 20.10.2026 00:12:57, Arnd Bergmann wrote:
> >> On Mon, Oct 19, 2026, at 22:12, Ian Rogers wrote:
> >> > +
> >> > +config VDSO_UNALIGNED_PACKED
> >> > +	bool
> >> > +	default CPU_32v5
> >> > +	help
> >> > +	  Use packed structs rather than memcpy() in include/vdso/unaligned.h
> >> > +	  for unaligned memory accesses. Enabled on ARMv5 where GCC may emit
> >> > +	  out-of-line memcpy() calls instead of inlining byte accesses.
> >>
> >> As explained, I think this misrepresents the problem as Arm specific and
> >> isn't the correct method for -munaligned-access targets on Arm, which
> >> include CPU_32v3, CPU_32v4 and CPU_32v4T along with CPU_32v5.
> >>
> >> I still hope we can use a version that just produces the intended code
> >> on both -munaligned-access and -mno-unaligned-access targets like the
> >> old version did.
> >
> > FYI: there is the kconfig Symbol HAVE_EFFICIENT_UNALIGNED_ACCESS. Can we
> > make use of it?
>
> That would be better, but I still hope we can just get back to the
> portable code across all architectures for the in-kernel version.
>
> I have one more request for you: can you check /proc/cpu/alignment
> on the system that shows the performance regression with and without
> the patch?

I asked Stefan Kerkmann (who wrote the original regression report) the
same question, but we haven't had time for it yet. We'll get around to
it.

> In your original report, you have shown that Ian's patch
> caused the drop in performance and that it caused the added function
> calls, but I don't see any sufficiently hot code path that would
> actually explain the scale of the regression,

Don't underestimate the overhead of function calls on those old CPUs
with tiny I-caches.

Little side note: IIRC, even on ARMv7 you can measure the performance
overhead of the built-in, but disabled, function tracer.

> unless there is now
> also a case where get_unaligned() ends up doing the wrong case and
> running into a fixup handler. If this happens, you should see a
> nontrivial number of fixups listed in /proc/cpu/alignment. On
> a working system, the numbers should all be low but likely nonzero
> because there tend to be callers that still lack get_unaligned()
> wrappers and just access misaligned pointers directly.

regards,
Marc

-- 
Pengutronix e.K.                 | Marc Kleine-Budde          |
Embedded Linux                   | https://www.pengutronix.de |
Vertretung Nürnberg              | Phone: +49-5121-206917-129 |
Amtsgericht Hildesheim, HRA 2686 | Fax:   +49-5121-206917-9   |

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v1 1/2] vdso: Add VDSO_UNALIGNED_PACKED config option for ARMv5

[Arnd Bergmann]

On Tue, Oct 20, 2026, at 10:41, Marc Kleine-Budde wrote:
> On 20.10.2026 10:24:51, Arnd Bergmann wrote:
>> On Tue, Oct 20, 2026, at 08:44, Marc Kleine-Budde wrote:
>>
>> That would be better, but I still hope we can just get back to the
>> portable code across all architectures for the in-kernel version.
>>
>> I have one more request for you: can you check /proc/cpu/alignment
>> on the system that shows the performance regression with and without
>> the patch?
>
> I asked Stefan Kerkmann (who wrote the original regression report) the
> same question, but we haven't had time for it yet. We'll get around to
> it.

Ok, thanks!

>> In your original report, you have shown that Ian's patch
>> caused the drop in performance and that it caused the added function
>> calls, but I don't see any sufficiently hot code path that would
>> actually explain the scale of the regression,
>
> Don't underestimate the overhead of function calls on those old CPUs
> with tiny I-caches.

What I meant is that I tried to look for calls to get_unaligned()
in the code path you'd hit with UBIFS on nand flash, and I only
found a possible call once per page written
(ebu_nand_write_page_hwecc), which should have a much lower
overhead than what you observe.

While ubifs does use crc32, it uses the lib/crc/ version rather
than the crypto/crc32c.c that you listed, and there is no
get_unaligned() in the lib/ one. If you have NOR flash, the
caller is likely do_write_buffer() calling map_word_load().
That one could explain the problem and could also be
optimized further if it turns out to be the bottleneck.

     Arnd