Re: [PATCH] crypto: rsa: add debug message if leading zero byte is missing

[Kepplinger-Novakovic Martin <Martin.Kepplinger-Novakovic@ginzinger.com>]

Am Donnerstag, dem 12.02.2026 um 11:15 +0000 schrieb Ignat Korchagin:
> Hi,
> 
> On Thu, Feb 12, 2026 at 10:39 AM Martin Kepplinger-Novakovic
> <martin.kepplinger-novakovic@ginzinger.com> wrote:
> > 
> > When debugging RSA certificate validation it can be valuable to see
> > why the RSA verify() callback returns -EINVAL.
> 
> Not sure if this would be some kind of an information leak (depending
> on a subsystem using RSA). Also what makes this case so special?
> Should we then annotate every other validation check in the code?
> 
> > Signed-off-by: Martin Kepplinger-Novakovic
> > <martin.kepplinger-novakovic@ginzinger.com>
> > ---
> > 
> > hi,
> > 
> > my real issue is: When using a certificate based on an RSA-key,
> > I sometimes see signature-verify errors and (via dm-verity)
> > rootfs signature-verify errors, all triggered by "no leading 0
> > byte".
> > 
> > key/cert generation:
> > openssl req -x509 -newkey rsa:4096 -keyout ca_key.pem -out ca.pem -
> > nodes -days 365 -set_serial 01 -subj /CN=ginzinger.com
> > and simply used as trusted built-in key and rootfs hash sign
> > appended
> > to rootfs (squashfs).
> > 
> > I'm on imx6ul. The thing is: Using the same certificate/key, works
> > on
> > old v5.4-based kernels, up to v6.6!
> > 
> > Starting with commit 2f1f34c1bf7b309 ("crypto: ahash - optimize
> > performance
> > when wrapping shash") it starts to break. it is not a commit on
> > it's own I
> > can revert and move on.
> > 
> > What happended since v6.6 ? On v6.7 I see
> > [    2.978722] caam_jr 2142000.jr: 40000013: DECO: desc idx 0:
> > Header Error. Invalid length or parity, or certain other problems.
> > 
> > and later the above -EINVAL from the RSA verify callback, where I
> > add
> > the debug printing I see.
> > 
> > What's the deal with this "leading 0 byte"?
> 
> See RFC 2313, p 8.1

hi Ignat,

thanks for your time, the problem is *sometimes* rsa verify fails.
there seems to be a race condition:

in the failure-case after crypto_akcipher_encrypt() and
crypto_wait_req() the *same* data as before is still at out_buf! that
has not yet been written to.

It's not that obvious to be yet because msleep(1000) doesn't change
much and 00, 01, ff, ff... is *still* not yet written to out_buf!

is there a reason why crypto_akcipher_sync_{en,de}crypt() is not used?
Can you imagine what could go wrong here?

*maybe* commit 1e562deacecca1f1bec7d23da526904a1e87525e that did a lot
of things in parallel (in order to keep functionality similar) got
something wrong?

sidenote: when I use an elliptic curve key instead of rsa, everything
works.

also, the auto-free for child_req looks a bit dangerous when using
out_buf, but ok :)

maybe this rings a bell, I'll keep debugging,

                            martin


> 
> > 
> > thank you!
> > 
> >                                     martin
> > 
> > 
> > 
> >  crypto/rsa-pkcs1pad.c | 5 +++--
> >  crypto/rsassa-pkcs1.c | 5 +++--
> >  2 files changed, 6 insertions(+), 4 deletions(-)
> > 
> > diff --git a/crypto/rsa-pkcs1pad.c b/crypto/rsa-pkcs1pad.c
> > index 50bdb18e7b483..65a4821e9758b 100644
> > --- a/crypto/rsa-pkcs1pad.c
> > +++ b/crypto/rsa-pkcs1pad.c
> > @@ -191,9 +191,10 @@ static int pkcs1pad_decrypt_complete(struct
> > akcipher_request *req, int err)
> > 
> >         out_buf = req_ctx->out_buf;
> >         if (dst_len == ctx->key_size) {
> > -               if (out_buf[0] != 0x00)
> > -                       /* Decrypted value had no leading 0 byte */
> > +               if (out_buf[0] != 0x00) {
> > +                       pr_debug("Decrypted value had no leading 0
> > byte\n");
> >                         goto done;
> > +               }
> > 
> >                 dst_len--;
> >                 out_buf++;
> > diff --git a/crypto/rsassa-pkcs1.c b/crypto/rsassa-pkcs1.c
> > index 94fa5e9600e79..22919728ea1c8 100644
> > --- a/crypto/rsassa-pkcs1.c
> > +++ b/crypto/rsassa-pkcs1.c
> > @@ -263,9 +263,10 @@ static int rsassa_pkcs1_verify(struct
> > crypto_sig *tfm,
> >                 return -EINVAL;
> > 
> >         if (dst_len == ctx->key_size) {
> > -               if (out_buf[0] != 0x00)
> > -                       /* Encrypted value had no leading 0 byte */
> > +               if (out_buf[0] != 0x00) {
> > +                       pr_debug("Encrypted value had no leading 0
> > byte\n");
> >                         return -EINVAL;
> > +               }
> > 
> >                 dst_len--;
> >                 out_buf++;
> > --
> > 2.47.3
> > 
> 
> Ignat