[PATCH] ecdsa: Fix incorrect usage of vli_cmp

[PATCH] ecdsa: Fix incorrect usage of vli_cmp

[Stefan Berger]

Fix incorrect usage of vli_cmp when calculating the value of res.x. For
signature verification to succeed, res.x must be the same as the r
component of the signature which is in the range of [1..n-1] with 'n'
being the order of the curve. Therefore, when res.x equals n calculate
res.x = res.x - n as well. Signature verification could have previously
unnecessarily failed in extremely rare cases.

Fixes: 4e6602916bc6 ("crypto: ecdsa - Add support for ECDSA signature verification")
Cc: <stable@vger.kernel.org>
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
---
 crypto/ecdsa.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/crypto/ecdsa.c b/crypto/ecdsa.c
index b3a8a6b572ba..674ab9275366 100644
--- a/crypto/ecdsa.c
+++ b/crypto/ecdsa.c
@@ -120,8 +120,8 @@ static int _ecdsa_verify(struct ecc_ctx *ctx, const u64 *hash, const u64 *r, con
 	/* res = u1*G + u2 * pub_key */
 	ecc_point_mult_shamir(&res, u1, &curve->g, u2, &ctx->pub_key, curve);
 
-	/* res.x = res.x mod n (if res.x > order) */
-	if (unlikely(vli_cmp(res.x, curve->n, ndigits) == 1))
+	/* res.x = res.x mod n (if res.x >= order) */
+	if (unlikely(vli_cmp(res.x, curve->n, ndigits) >= 0))
 		/* faster alternative for NIST p384, p256 & p192 */
 		vli_sub(res.x, res.x, curve->n, ndigits);
 
-- 
2.34.1

Re: [PATCH] ecdsa: Fix incorrect usage of vli_cmp

[Stefan Berger]

On 4/21/22 14:57, Stefan Berger wrote:
> Fix incorrect usage of vli_cmp when calculating the value of res.x. For
> signature verification to succeed, res.x must be the same as the r
> component of the signature which is in the range of [1..n-1] with 'n'
> being the order of the curve. Therefore, when res.x equals n calculate
> res.x = res.x - n as well. Signature verification could have previously
> unnecessarily failed in extremely rare cases.

Actually, I am withdrawing this patch. Before this patch res.x could 
equal 'n' and then wouldn't match r due to the range of r being 
[1..n-1]. Now if res.x equals 'n' then res.x - n will be 0 and again 
will not match 'r' due to the range of r being [1..n-1]. So it makes no 
difference whether vli_cmp() == 1 or vli_cmp() >= 1 and the concern 
above about rare cases not verifying the signature is wrong.

    Stefan

> 
> Fixes: 4e6602916bc6 ("crypto: ecdsa - Add support for ECDSA signature verification")
> Cc: <stable@vger.kernel.org>
> Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
> ---
>   crypto/ecdsa.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/crypto/ecdsa.c b/crypto/ecdsa.c
> index b3a8a6b572ba..674ab9275366 100644
> --- a/crypto/ecdsa.c
> +++ b/crypto/ecdsa.c
> @@ -120,8 +120,8 @@ static int _ecdsa_verify(struct ecc_ctx *ctx, const u64 *hash, const u64 *r, con
>   	/* res = u1*G + u2 * pub_key */
>   	ecc_point_mult_shamir(&res, u1, &curve->g, u2, &ctx->pub_key, curve);
>   
> -	/* res.x = res.x mod n (if res.x > order) */
> -	if (unlikely(vli_cmp(res.x, curve->n, ndigits) == 1))
> +	/* res.x = res.x mod n (if res.x >= order) */
> +	if (unlikely(vli_cmp(res.x, curve->n, ndigits) >= 0))
>   		/* faster alternative for NIST p384, p256 & p192 */
>   		vli_sub(res.x, res.x, curve->n, ndigits);
>