Re: [PATCH] crypto: rsa: add debug message if leading zero byte is missing

[Kepplinger-Novakovic Martin <Martin.Kepplinger-Novakovic@ginzinger.com>]

Am Mittwoch, dem 18.02.2026 um 13:53 +0000 schrieb Kepplinger-Novakovic Martin:
> Am Mittwoch, dem 18.02.2026 um 09:42 +0000 schrieb Kepplinger-Novakovic
> Martin:
> > Am Mittwoch, dem 18.02.2026 um 09:22 +0000 schrieb Kepplinger-
> > Novakovic
> > Martin:
> > > Am Mittwoch, dem 18.02.2026 um 10:06 +0100 schrieb Ignat Korchagin:
> > > > On Wed, Feb 18, 2026 at 9:36 AM Kepplinger-Novakovic Martin
> > > > <Martin.Kepplinger-Novakovic@ginzinger.com> wrote:
> > > > > 
> > > > > Am Donnerstag, dem 12.02.2026 um 11:15 +0000 schrieb Ignat
> > > > > Korchagin:
> > > > > > Hi,
> > > > > > 
> > > > > > On Thu, Feb 12, 2026 at 10:39 AM Martin Kepplinger-Novakovic
> > > > > > <martin.kepplinger-novakovic@ginzinger.com> wrote:
> > > > > > > 
> > > > > > > When debugging RSA certificate validation it can be
> > > > > > > valuable
> > > > > > > to
> > > > > > > see
> > > > > > > why the RSA verify() callback returns -EINVAL.
> > > > > > 
> > > > > > Not sure if this would be some kind of an information leak
> > > > > > (depending
> > > > > > on a subsystem using RSA). Also what makes this case so
> > > > > > special?
> > > > > > Should we then annotate every other validation check in the
> > > > > > code?
> > > > > > 
> > > > > > > Signed-off-by: Martin Kepplinger-Novakovic
> > > > > > > <martin.kepplinger-novakovic@ginzinger.com>
> > > > > > > ---
> > > > > > > 
> > > > > > > hi,
> > > > > > > 
> > > > > > > my real issue is: When using a certificate based on an RSA-
> > > > > > > key,
> > > > > > > I sometimes see signature-verify errors and (via dm-verity)
> > > > > > > rootfs signature-verify errors, all triggered by "no
> > > > > > > leading
> > > > > > > 0
> > > > > > > byte".
> > > > > > > 
> > > > > > > key/cert generation:
> > > > > > > openssl req -x509 -newkey rsa:4096 -keyout ca_key.pem -out
> > > > > > > ca.pem -
> > > > > > > nodes -days 365 -set_serial 01 -subj /CN=ginzinger.com
> > > > > > > and simply used as trusted built-in key and rootfs hash
> > > > > > > sign
> > > > > > > appended
> > > > > > > to rootfs (squashfs).
> > > > > > > 
> > > > > > > I'm on imx6ul. The thing is: Using the same
> > > > > > > certificate/key,
> > > > > > > works
> > > > > > > on
> > > > > > > old v5.4-based kernels, up to v6.6!
> > > > > > > 
> > > > > > > Starting with commit 2f1f34c1bf7b309 ("crypto: ahash -
> > > > > > > optimize
> > > > > > > performance
> > > > > > > when wrapping shash") it starts to break. it is not a
> > > > > > > commit
> > > > > > > on
> > > > > > > it's own I
> > > > > > > can revert and move on.
> > > > > > > 
> > > > > > > What happended since v6.6 ? On v6.7 I see
> > > > > > > [    2.978722] caam_jr 2142000.jr: 40000013: DECO: desc idx
> > > > > > > 0:
> > > > > > > Header Error. Invalid length or parity, or certain other
> > > > > > > problems.
> > > > > > > 
> > > > > > > and later the above -EINVAL from the RSA verify callback,
> > > > > > > where
> > > > > > > I
> > > > > > > add
> > > > > > > the debug printing I see.
> > > > > > > 
> > > > > > > What's the deal with this "leading 0 byte"?
> > > > > > 
> > > > > > See RFC 2313, p 8.1
> > > > > 
> > > > > hi Ignat,
> > > > > 
> > > > > thanks for your time, the problem is *sometimes* rsa verify
> > > > > fails.
> > > > > there seems to be a race condition:
> > > > 
> > > > Can you clarify the failure case a bit? Is this the same
> > > > signature
> > > > that fails? (That is, you just verify a fixed signature in a
> > > > loop)
> > > > Or
> > > > are these different signatures? (some reliably verify and some
> > > > reliably fail)
> > > > 
> > > 
> > > different signuatures but nothing special: I add ca.pem (output of
> > > "openssl req -x509 -newkey rsa:4096 -keyout ca_key.pem -out ca.pem
> > > -
> > > nodes -days 365 -set_serial 01 -subj /CN=ginzinger.com") to
> > > CONFIG_SYSTEM_TRUSTED_KEYS
> > > 
> > > during boot, asymmetric_key_preparse() is called, first on this,
> > > and
> > > after that, "cfg80211: Loading compiled-in X.509 certificates for
> > > regulatory database" does the same thing for Chen-Yu, Seth's keys
> > > in
> > > mainline net/wireless/certs where I also added Ben's Debian
> > > certificate.
> > > 
> > > The above verifications of 5 keys fail randomly.
> > > 
> > 
> > to clarify: no verification reliably fails or succeeds. the first
> > one,
> > my own cert, mostly (always?) succeeds, for the 4 regdb certs I see
> > no
> > pattern at all. Chen-Yu's "wens" cert kind of fails more often that
> > than the others maybe.
> > 
> > There is almost never a boot where all certs verifications succeed,
> > although I've seen at least one already.
> > 
> > 
> > > In the end I (want to) use my own cert for dm-verity rootfs loading
> > > (which also randomly fails).
> > > 
> > > on old kernels, most likely up to v6.6, there was no problem.
> > > 
> > > thank you for asking,
> > > 
> > >                                 martin
> > > 
> > > 
> > > > > in the failure-case after crypto_akcipher_encrypt() and
> > > > > crypto_wait_req() the *same* data as before is still at
> > > > > out_buf!
> > > > > that
> > > > > has not yet been written to.
> > > > > 
> > > > > It's not that obvious to be yet because msleep(1000) doesn't
> > > > > change
> > > > > much and 00, 01, ff, ff... is *still* not yet written to
> > > > > out_buf!
> 
> oh, I might have been on a slightly wrong path: I'm on imx6ul and
> disabling CONFIG_CRYPTO_DEV_FSL_CAAM indeed is a workaround, so it's
> probably drivers/crypto/caam/ where enqueue + dequeue properly run, but
> still the CPU doesn't see updated data.
> 
> I added Horia, Pankaj and Gaurav and with luck they see what could go
> wrong here with CAAM-DMA/CPU syncing.


I wrote a more accurate and detailed bugreport for this, see
https://lore.kernel.org/linux-crypto/6029acc0f0ddfe25e2537c2866d54fd7f54bc182.camel@ginzinger.com/T/#u

thanks,

                                   martin