[PATCH] lib/crypto: arm64: Fix lost Poly1305 carry when resuming NEON state

[PATCH] lib/crypto: arm64: Fix lost Poly1305 carry when resuming NEON state

[Jérémy Jean]

When poly1305_blocks_neon() resumes from a radix-2^26 state with an odd
number of input blocks, it converts the accumulator back to radix-2^64
before consuming the first block. The final ADC stores the carry into d2,
but the following accumulation and poly1305_mult() use h2.  If the
conversion overflows across bit 128, the carry is dropped and the emitted
tag is wrong.

Store the carry back into h2. This matches the other conversion paths and
preserves the represented accumulator.

Fixes: f569ca164751 ("crypto: arm64/poly1305 - incorporate OpenSSL/CRYPTOGAMS NEON implementation")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Jérémy Jean <Jeremy.Jean@oss.cyber.gouv.fr>
---
 lib/crypto/arm64/poly1305-armv8.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/crypto/arm64/poly1305-armv8.pl b/lib/crypto/arm64/poly1305-armv8.pl
index f1930c6..234398f 100644
--- a/lib/crypto/arm64/poly1305-armv8.pl
+++ b/lib/crypto/arm64/poly1305-armv8.pl
@@ -375,7 +375,7 @@ poly1305_blocks_neon:
 	adc	$h1,$h1,xzr
 	lsr	$h2,x14,#24
 	adds	$h1,$h1,x14,lsl#40
-	adc	$d2,$h2,xzr		// can be partially reduced...
+	adc	$h2,$h2,xzr		// preserve carry into top limb
 
 	ldp	$d0,$d1,[$inp],#16	// load input
 	sub	$len,$len,#16
-- 
2.47.3

Re: [PATCH] lib/crypto: arm64: Fix lost Poly1305 carry when resuming NEON state

[Eric Biggers]

On Wed, Oct 07, 2026 at 10:02:36PM +0000, Jérémy Jean wrote:
> When poly1305_blocks_neon() resumes from a radix-2^26 state with an odd
> number of input blocks, it converts the accumulator back to radix-2^64
> before consuming the first block. The final ADC stores the carry into d2,
> but the following accumulation and poly1305_mult() use h2.  If the
> conversion overflows across bit 128, the carry is dropped and the emitted
> tag is wrong.
> 
> Store the carry back into h2. This matches the other conversion paths and
> preserves the represented accumulator.
> 
> Fixes: f569ca164751 ("crypto: arm64/poly1305 - incorporate OpenSSL/CRYPTOGAMS NEON implementation")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Jérémy Jean <Jeremy.Jean@oss.cyber.gouv.fr>
> ---
>  lib/crypto/arm64/poly1305-armv8.pl | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/lib/crypto/arm64/poly1305-armv8.pl b/lib/crypto/arm64/poly1305-armv8.pl
> index f1930c6..234398f 100644
> --- a/lib/crypto/arm64/poly1305-armv8.pl
> +++ b/lib/crypto/arm64/poly1305-armv8.pl
> @@ -375,7 +375,7 @@ poly1305_blocks_neon:
>  	adc	$h1,$h1,xzr
>  	lsr	$h2,x14,#24
>  	adds	$h1,$h1,x14,lsl#40
> -	adc	$d2,$h2,xzr		// can be partially reduced...
> +	adc	$h2,$h2,xzr		// preserve carry into top limb

This needs a regression test in lib/crypto/tests/poly1305_kunit.c.

Please include more details in the commit message about under what
circumstances that carry bit could be nonzero.

It seems this was introduced by commit 03dc4adf91c8bc of
https://github.com/dot-asm/cryptogams just before the kernel imported
the code.  The bug never reached the copy of this file in OpenSSL.  Do
you agree?  If so, please mention this information in the commit message
too, and also open an issue at https://github.com/dot-asm/cryptogams so
that it can be fixed there as well.

- Eric