Re: [PATCH v10 00/14] Allow preemption during IPI completion waiting to improve real-time performance

["Chuyi Zhou" <zhouchuyi@bytedance.com>]

On 2026-07-24 3:26 a.m., Borislav Petkov wrote:
> On Thu, Jul 09, 2026 at 08:29:19PM +0800, Chuyi Zhou wrote:
>> In our production environments, latency-sensitive workloads (DPDK) are
>> configured with the highest priority to preempt lower-priority tasks at any
>> time. We discovered that DPDK's wake-up latency is primarily caused by the
>> current CPU having preemption disabled. Therefore, we collected the maximum
>> preemption disabled events within every 30-second interval and then
>> calculated the P50/P99 of these max preemption disabled events:
> 
> Out of curiosity, can you run your workloads on AMD Zen3 and newer which have
> TLBI support (this does away with the TLB flush IPIs). Do you see any
> improvement there?
> 
> Thx.
> 

Hi Boris,

Thanks for the suggestion.

If I get access to suitable AMD Zen 3 or newer production machines, I
would be happy to evaluate this series there.

On a system where X86_FEATURE_INVLPGB is enabled, I would expect the
direct benefit on the TLB flush paths to be smaller. For an mm which
already has a global ASID, flush_tlb_mm_range() uses
broadcast_tlb_flush(). arch_tlbbatch_flush() also uses
invlpgb_flush_all_nonglobals() directly for pending unmaps. Neither path
waits for remote TLB flush IPIs.

Those paths still have to remain on the CPU which issued INVLPGB until
TLBSYNC completes. Consequently, this series does not make the
INVLPGB/TLBSYNC section preemptible.

The benefit is not necessarily zero, though. A global ASID is assigned
opportunistically, currently for an mm active on at least four CPUs.
Before that assignment, flush_tlb_mm_range() can still fall back to the
IPI path. The same applies when INVLPGB is unavailable or disabled.

There is also an architecture-independent effect. The generic
smp_call_function*() changes keep the caller pinned only while selecting
the target CPUs, preparing the per-CPU state, queueing the callbacks and
sending the IPIs. The synchronous completion wait can then be 
preemptible when the caller's context otherwise permits it.

In particular, smp_call_function_many() and its wrappers no longer rely
on an outer preempt_disable() merely to protect the call-function
machinery. This benefits non-TLB synchronous IPI users as well. It also 
provides a basis for auditing other call-function IPI users and removing 
outer preemption-disabled regions which only existed because of the old
requirement. Callers which protect their own per-CPU state would, of
course, still need their local protection.

The data in the cover letter came from real production workloads. The
long-tail latency depends on the workload mix, interrupt-disabled
sections on remote CPUs and the number of target CPUs, so a short local
or synthetic test may not reproduce it accurately.

A more meaningful experiment would be a controlled, long-running
comparison between the baseline and patched kernels on comparable
production AMD Zen 3 or newer machines. We could then collect the same
30-second maximum preemption-disabled distributions and DPDK wake-up
latency over a sufficiently long period. That would also help separate
the remaining INVLPGB/TLBSYNC latency from latency caused by other
synchronous IPI users.

If I have the opportunity to perform such a production comparison, I
will report the results.

Thanks.