mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Eric Biggers <ebiggers@kernel.org>
To: David Laight <david.laight.linux@gmail.com>
Cc: linux-crypto@vger.kernel.org, Ard Biesheuvel <ardb@kernel.org>,
	"Jason A . Donenfeld" <Jason@zx2c4.com>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	x86@kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] lib/crypto: x86/sha2: Drop one-off implementations for Sandy/Ivy Bridge
Date: Sun, 22 Nov 2026 11:55:51 -0800	[thread overview]
Message-ID: <20261122195551.GB2122@sol> (raw)
In-Reply-To: <20261122113934.48a9ac42@pumpkin>

On Sun, Nov 22, 2026 at 11:39:34AM +0000, David Laight wrote:
> On Sun, 15 Nov 2026 00:09:00 -0800
> Eric Biggers <ebiggers@kernel.org> wrote:
> 
> > [ This is proposed to supersede my patch
> >   "lib/crypto: x86/sha512: Partially roll up sha512_transform_avx()"
> >   which was starting to clean up one of the files being dropped. ]
> > 
> > Currently, the kernel has SSSE3, AVX, and AVX2 implementations of each
> > of SHA-256 and SHA-512.
> > 
> > The AVX implementations of SHA-256 and SHA-512 run only on CPUs that
> > have AVX but do *not* have AVX2.  These are all the ones I could find:
> > 
> >     - Intel Sandy Bridge and Ivy Bridge
> >     - AMD Bulldozer, Piledriver, Steamroller, Jaguar, and Puma
> >     - Zhaoxin KX-5000 and KX-6000 (possibly SHA-512 only, as these CPUs
> >       might be able to run the PHE optimized SHA-256 code)
> >     - Maybe some oddly configured VMs
> > 
> > Oddly enough, this "AVX" code doesn't actually have much to do with AVX.
> > The only AVX capability it uses is support for 3-operand VEX-coded
> > instructions, which is unlikely to make a huge difference.
> 
> Especially if they are the only VEX encoded instructions.
> Switching back and forth is horribly expensive and the change isn't
> 'sticky' on some later Intel cpu and you get the 'hit' for every
> instruction.
> (kernel_fpu_begin() might need a vzeroupper to migitage this.)
> 
> > 
> > Its actual purpose is mostly unrelated to AVX: it replaces the 'ror'
> > instruction with 'shld'.  This is entirely to work around a quirk
> > specific to Sandy Bridge and Ivy Bridge where the 'ror' instruction was
> > very slow for microarchitectural reasons, and 'shld' worked better.
> > 
> > This is documented in the contemporaneous paper "Speeding up SHA-1,
> > SHA-256 and SHA-512 on the 2nd Generation Intel Core Processors"
> > (https://ieeexplore.ieee.org/document/6209073).  There, the 'shld'
> > workaround was claimed to speed up SHA-256 and SHA-512 by about 20% on
> > Sandy Bridge.
> > 
> > However, that was a one-off quirk.  And even the paper itself warns
> > about using the 'shld' approach on other microarchitectures.
> > 
> > On AMD CPUs, and on Intel CPUs after Sandy and Ivy Bridge, the exact
> > opposite applies: ror is a fast single-cycle and single µop instruction,
> > while shld is either a slow microcoded or multi-µop instruction.
> > 
> > From what I can find, this is true even on the older AMD CPUs mentioned
> > above that run this code.  Thus this code has likely always been harmful
> > on AMD.  (I don't have such a CPU available to test, but for what it's
> > worth, on Zen 5 the AVX SHA-512 code is 58% as fast as the SSSE3 code.
> > Clearly, the 'shld' hack can be quite slow on CPUs it wasn't meant for.)
> 
> I've got a bulldozer (or similar) I can run a test on.
> I might just to a quick one for shld v ror.
> I've rewritten my 'rep movs/stos' benchmark and plan to run it on the
> older cpu I have.
> (I need to get something newer than an i7-7 and older than a zen5.)

Thanks.  If you do want to benchmark it, you can use sha512_kunit with
CONFIG_CRYPTO_LIB_BENCHMARK=y enabled.

After that, you could also check the performance with the
sha512-ssse3-asm.S optimization patchset applied on top of it
(https://lore.kernel.org/linux-crypto/20261122022030.296468-1-ebiggers@kernel.org/)

My prediction is that performance on Bulldozer goes up slightly when
sha512-avx-asm.S is removed (due to sha512-ssse3-asm.S being faster),
then up slightly more due to the sha512-ssse3-asm.S optimizations.

(Of course, even if the results are more mixed, both patches seem like
the clear way to go anyway for other reasons: reducing the code size,
deleting an entire asm file that we'll no longer have to maintain,
eliminating the huge on-stack buffer that was never cleared, etc...)

- Eric

      reply	other threads:[~2026-11-22 19:55 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-15  8:09 Eric Biggers
2026-11-21 19:51 ` Eric Biggers
2026-11-22 11:39 ` David Laight
2026-11-22 19:55   ` Eric Biggers [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261122195551.GB2122@sol \
    --to=ebiggers@kernel.org \
    --cc=Jason@zx2c4.com \
    --cc=ardb@kernel.org \
    --cc=david.laight.linux@gmail.com \
    --cc=herbert@gondor.apana.org.au \
    --cc=linux-crypto@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®