From: Huang Shijie <shijie@os.amperecomputing.com>
To: catalin.marinas@arm.com
Cc: will@kernel.org, mark.rutland@arm.com, suzuki.poulose@arm.com,
broonie@kernel.org, linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, anshuman.khandual@arm.com,
robh@kernel.org, oliver.upton@linux.dev, maz@kernel.org,
patches@amperecomputing.com,
Huang Shijie <shijie@os.amperecomputing.com>
Subject: [PATCH 3/4] arm64: copy_template.S: add loop_for_copy_128_bytes macro
Date: Wed, 22 Nov 2023 17:28:54 +0800 [thread overview]
Message-ID: <20231122092855.4440-4-shijie@os.amperecomputing.com> (raw)
In-Reply-To: <20231122092855.4440-1-shijie@os.amperecomputing.com>
Add the loop_for_copy_128_bytes macro, to make the code clean.
And make preparation for the next patch.
Signed-off-by: Huang Shijie <shijie@os.amperecomputing.com>
---
arch/arm64/lib/copy_template.S | 58 ++++++++++++++++++----------------
1 file changed, 31 insertions(+), 27 deletions(-)
diff --git a/arch/arm64/lib/copy_template.S b/arch/arm64/lib/copy_template.S
index 488df234c49a..79b32569260c 100644
--- a/arch/arm64/lib/copy_template.S
+++ b/arch/arm64/lib/copy_template.S
@@ -10,6 +10,36 @@
* files/head:/src/aarch64/
*/
+.macro loop_for_copy_128_bytes extra_ops
+ /* pre-get 64 bytes data. */
+ ldp1 A_l, A_h, src, #16
+ ldp1 B_l, B_h, src, #16
+ ldp1 C_l, C_h, src, #16
+ ldp1 D_l, D_h, src, #16
+1:
+ \extra_ops
+ /*
+ * interlace the load of next 64 bytes data block with store of the last
+ * loaded 64 bytes data.
+ */
+ stp1 A_l, A_h, dst, #16
+ ldp1 A_l, A_h, src, #16
+ stp1 B_l, B_h, dst, #16
+ ldp1 B_l, B_h, src, #16
+ stp1 C_l, C_h, dst, #16
+ ldp1 C_l, C_h, src, #16
+ stp1 D_l, D_h, dst, #16
+ ldp1 D_l, D_h, src, #16
+ subs count, count, #64
+ b.ge 1b
+ stp1 A_l, A_h, dst, #16
+ stp1 B_l, B_h, dst, #16
+ stp1 C_l, C_h, dst, #16
+ stp1 D_l, D_h, dst, #16
+
+ tst count, #0x3f
+ b.ne .Ltail63
+.endm
/*
* Copy a buffer from src to dest (alignment handled by the hardware)
@@ -151,31 +181,5 @@ D_h .req x14
*/
.p2align L1_CACHE_SHIFT
.Lcpy_body_large:
- /* pre-get 64 bytes data. */
- ldp1 A_l, A_h, src, #16
- ldp1 B_l, B_h, src, #16
- ldp1 C_l, C_h, src, #16
- ldp1 D_l, D_h, src, #16
-1:
- /*
- * interlace the load of next 64 bytes data block with store of the last
- * loaded 64 bytes data.
- */
- stp1 A_l, A_h, dst, #16
- ldp1 A_l, A_h, src, #16
- stp1 B_l, B_h, dst, #16
- ldp1 B_l, B_h, src, #16
- stp1 C_l, C_h, dst, #16
- ldp1 C_l, C_h, src, #16
- stp1 D_l, D_h, dst, #16
- ldp1 D_l, D_h, src, #16
- subs count, count, #64
- b.ge 1b
- stp1 A_l, A_h, dst, #16
- stp1 B_l, B_h, dst, #16
- stp1 C_l, C_h, dst, #16
- stp1 D_l, D_h, dst, #16
-
- tst count, #0x3f
- b.ne .Ltail63
+ loop_for_copy_128_bytes
.Lexitfunc:
--
2.40.1
next prev parent reply other threads:[~2023-11-22 9:30 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-11-22 9:28 [PATCH 0/4] arm64: an optimization for AmpereOne Huang Shijie
2023-11-22 9:28 ` [PATCH 1/4] extable: add __sort_main_extable Huang Shijie
2023-11-22 9:28 ` [PATCH 2/4] arm64: alternative: handle the kernel exception table Huang Shijie
2023-11-22 9:28 ` Huang Shijie [this message]
2023-11-22 9:28 ` [PATCH 4/4] arm64: add software prefetches for AmpereOne Huang Shijie
2023-11-22 11:34 ` Robin Murphy
2023-11-22 9:48 ` [PATCH 0/4] arm64: an optimization " Will Deacon
2023-11-22 11:40 ` Mark Rutland
2023-11-22 12:11 ` Marc Zyngier
2023-11-23 7:59 ` Linus Walleij
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231122092855.4440-4-shijie@os.amperecomputing.com \
--to=shijie@os.amperecomputing.com \
--cc=anshuman.khandual@arm.com \
--cc=broonie@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=maz@kernel.org \
--cc=oliver.upton@linux.dev \
--cc=patches@amperecomputing.com \
--cc=robh@kernel.org \
--cc=suzuki.poulose@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®