From: Miroslav Cimerman <mc@raidlab.org>
To: Song Liu <song@kernel.org>, Yu Kuai <yukuai@fygo.io>,
Andrew Morton <akpm@linux-foundation.org>
Cc: Li Nan <magiclinan@didiglobal.com>, Xiao Ni <xiao@kernel.org>,
Eric Biggers <ebiggers@kernel.org>,
Christoph Hellwig <hch@lst.de>,
David Laight <david.laight.linux@gmail.com>,
Markus Stockhausen <markus.stockhausen@collogia.de>,
linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH 3/3] raid6: prefetch Q with prefetcht0 in the x86 x4 xor_syndrome functions
Date: Sun, 22 Nov 2026 03:29:33 +0000 [thread overview]
Message-ID: <20261122032933.2306875-4-mc@raidlab.org> (raw)
In-Reply-To: <20261122032933.2306875-1-mc@raidlab.org>
The x4 xor_syndrome() functions for SSE2, AVX2 and AVX-512 prefetch Q
with prefetchnta. That hint is meant for data that is used once. Q is
read, updated and written back with normal stores, and md calls
xor_syndrome() twice on the same P/Q page for each write.
Prefetch Q with prefetcht0, which keeps it in all cache levels. The data
blocks are read once and keep prefetchnta.
Time per write measured in a userspace harness in md's read-modify-write
pattern (two xor_syndrome() calls on the same P/Q page), avx512x4 on a
Xeon Gold 6130, 2-24 data disks, write to the first, middle and last
data disk, median of 10 passes:
P and Q in DRAM: -2.2% (geometric mean), up to -5.7%
P and Q in L3: +0.2%
P and Q in L1/L2: +0.3%
The gain comes with 2-8 data disks, where Q is a larger part of the
work. The SSE2 and AVX2 versions were not measured.
Suggested-by: Eric Biggers <ebiggers@kernel.org>
Link: https://lore.kernel.org/linux-raid/20261116210440.GE229817@google.com/
Signed-off-by: Miroslav Cimerman <mc@raidlab.org>
---
lib/raid/raid6/x86/avx2.c | 4 ++--
lib/raid/raid6/x86/avx512.c | 4 ++--
lib/raid/raid6/x86/sse2.c | 4 ++--
3 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/lib/raid/raid6/x86/avx2.c b/lib/raid/raid6/x86/avx2.c
index d3f488a9c1a5..5a49079ace68 100644
--- a/lib/raid/raid6/x86/avx2.c
+++ b/lib/raid/raid6/x86/avx2.c
@@ -408,8 +408,8 @@ static void raid6_avx24_xor_syndrome(int disks, int start, int stop,
asm volatile("vpxor %ymm13,%ymm12,%ymm12");
asm volatile("vpxor %ymm15,%ymm14,%ymm14");
}
- asm volatile("prefetchnta %0" :: "m" (q[d]));
- asm volatile("prefetchnta %0" :: "m" (q[d+64]));
+ asm volatile("prefetcht0 %0" :: "m" (q[d]));
+ asm volatile("prefetcht0 %0" :: "m" (q[d+64]));
/* P/Q left side optimization */
for (z = start-1 ; z >= 0 ; z--) {
asm volatile("vpxor %ymm5,%ymm5,%ymm5");
diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c
index 33d7639da5b3..38077109ab0f 100644
--- a/lib/raid/raid6/x86/avx512.c
+++ b/lib/raid/raid6/x86/avx512.c
@@ -476,8 +476,8 @@ static void raid6_avx5124_xor_syndrome(int disks, int start, int stop,
"m" (dptr[z][d+128]),
"m" (dptr[z][d+192]));
}
- asm volatile("prefetchnta %0\n\t"
- "prefetchnta %1\n\t"
+ asm volatile("prefetcht0 %0\n\t"
+ "prefetcht0 %1\n\t"
:
: "m" (q[d]), "m" (q[d+128]));
/* P/Q left side optimization */
diff --git a/lib/raid/raid6/x86/sse2.c b/lib/raid/raid6/x86/sse2.c
index a0709916289c..c27a4c8254ba 100644
--- a/lib/raid/raid6/x86/sse2.c
+++ b/lib/raid/raid6/x86/sse2.c
@@ -405,8 +405,8 @@ static void raid6_sse24_xor_syndrome(int disks, int start, int stop,
asm volatile("pxor %xmm13,%xmm12");
asm volatile("pxor %xmm15,%xmm14");
}
- asm volatile("prefetchnta %0" :: "m" (q[d]));
- asm volatile("prefetchnta %0" :: "m" (q[d+32]));
+ asm volatile("prefetcht0 %0" :: "m" (q[d]));
+ asm volatile("prefetcht0 %0" :: "m" (q[d+32]));
/* P/Q left side optimization */
for ( z = start-1 ; z >= 0 ; z-- ) {
asm volatile("pxor %xmm5,%xmm5");
--
2.53.0
prev parent reply other threads:[~2026-11-22 3:29 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-22 3:29 [PATCH 0/3] raid6: x86: asm constraints, sfence and prefetch in xor_syndrome Miroslav Cimerman
2026-11-22 3:29 ` [PATCH 1/3] raid6: declare the AVX-512 P/Q stores as asm outputs Miroslav Cimerman
2026-11-22 3:29 ` [PATCH 2/3] raid6: remove the sfence from the x86 xor_syndrome functions Miroslav Cimerman
2026-11-22 3:29 ` Miroslav Cimerman [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261122032933.2306875-4-mc@raidlab.org \
--to=mc@raidlab.org \
--cc=akpm@linux-foundation.org \
--cc=david.laight.linux@gmail.com \
--cc=ebiggers@kernel.org \
--cc=hch@lst.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-raid@vger.kernel.org \
--cc=magiclinan@didiglobal.com \
--cc=markus.stockhausen@collogia.de \
--cc=song@kernel.org \
--cc=xiao@kernel.org \
--cc=yukuai@fygo.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®