Re: [PATCH v2] powerpc/lib: prefetch ahead in __csum_partial()

["Christophe Leroy (CS GROUP)" <chleroy@kernel.org>]

Le 12/10/2026 à 20:28, Rosen Penev a écrit :
> On Mon, Oct 12, 2026 at 11:18 AM Christophe Leroy (CS GROUP)
> <chleroy@kernel.org> wrote:
>>
>>
>>
>> Le 12/10/2026 à 20:00, Rosen Penev a écrit :
>>> __csum_partial() walks the buffer linearly and reads each cache line
>>> exactly once. On cores without a hardware prefetcher, such as the 464FP
>>> in the APM82181, every new line is therefore a demand miss that stalls
>>> the loop. Packets received by non-coherent DMA are never in the cache,
>>> so this is the common case for GRO checksum validation: when routing
>>> TCP with GRO enabled on a Meraki MX60, __csum_partial() takes about 30%
>>> of all cycles and is the top entry in the profile.
>>>
>>> Issue a dcbt four cache lines (128 bytes) ahead in the main loop.
>>> dcbt is only a hint and never faults, so prefetching past the end of
>>> the buffer is harmless. Lines that are already cached, for example when
>>> checksumming data the CPU just wrote, are left untouched.
>>>
>>> Measured on a Meraki MX60 (APM82181 at 800 MHz), single TCP stream
>>> routed through a qca8k DSA switch, iperf3 median of 5. Only the dcbt
>>> was toggled at runtime, in the same boot:
>>>
>>>                 LAN->WAN               WAN->LAN
>>>                 Mbit/s  csum cycles    Mbit/s  csum cycles
>>>     before      495     30.5%          439     27.2%
>>>     after       599     15.3%          520     13.7%
>>>
>>> That is 58% less time spent in __csum_partial() per byte.
>>>
>>> The prefetch distance was chosen with an earlier, slower driver
>>> configuration (median of 3, Mbit/s):
>>>
>>>     distance    LAN->WAN   WAN->LAN
>>>     none        370        341
>>>     64 bytes    430        381
>>>     96 bytes    438        397
>>>     128 bytes   438        385
>>>     160 bytes   434        386
>>>     256 bytes   423        383
>>>
>>> Prefetching the first lines before entering the loop made no
>>> measurable difference and is not done.
>>>
>>> Assisted-by: LLM
>>> Signed-off-by: Rosen Penev <rosenp@gmail.com>
>>
>> csum_partial_copy_generic() does the same, so your patch makes sense.
>>
>> Wondering if you should use MAX_COPY_PREFETCH instead of hardcoding value 4.
> li      r9,MAX_COPY_PREFETCH*L1_CACHE_BYTES /* prefetch distance */
> 
> LLM comment:
> 
> What changes per platform: MAX_COPY_PREFETCH is 4 on every 32-bit
> config except 8xx, where it is 1. So 44x, 47x and e500mc behave
> exactly as before; on the 44x (canyonlands) build the instruction is
> still li r9, 128. The one difference is 8xx, which now prefetches 16
> bytes ahead instead of 64. That fits how the copy routine already
> treats 8xx, but I didn't benchmark it.
> 
> My comment:
> I don't have an 8xx device so I don't know if this works properly there or not.

I have an 885 and might do the test one day, but I won't have time 
during the coming weeks. Nevertheless I have the feeling it makes sense 
to implement something based on MAX_COPY_PREFETCH to resemble 
csum_partial_copy_generic()

>>
>> Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
> I'll keep this in v3.
>>
>>> ---
>>>    v2: reword commit message and retest numbers.
>>>    arch/powerpc/lib/checksum_32.S | 4 +++-
>>>    1 file changed, 3 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/arch/powerpc/lib/checksum_32.S b/arch/powerpc/lib/checksum_32.S
>>> index cd00b9bdd772..5a14d45c7b9d 100644
>>> --- a/arch/powerpc/lib/checksum_32.S
>>> +++ b/arch/powerpc/lib/checksum_32.S
>>> @@ -43,6 +43,7 @@ _GLOBAL(__csum_partial)
>>>        bdnz    2b
>>>    21: srwi.   r6,r4,4         /* # blocks of 4 words to do */
>>>        beq     3f
>>> +     li      r9,4*L1_CACHE_BYTES     /* prefetch distance */
>>>        lwz     r0,4(r3)
>>>        mtctr   r6
>>>        lwz     r6,8(r3)
>>> @@ -52,7 +53,8 @@ _GLOBAL(__csum_partial)
>>>        lwzu    r8,16(r3)
>>>        adde    r5,r5,r7
>>>        bdz     23f
>>> -22:  lwz     r0,4(r3)
>>> +22:  dcbt    r3,r9
>>> +     lwz     r0,4(r3)
>>>        adde    r5,r5,r8
>>>        lwz     r6,8(r3)
>>>        adde    r5,r5,r0
>>