Re: [PATCH v3 10/12] x86/mm: Move flush_tlb_info back to the stack

["Chuyi Zhou" <zhouchuyi@bytedance.com>]

Hi Sebastian,

On 2026-03-19 6:58 p.m., Sebastian Andrzej Siewior wrote:
> On 2026-03-19 12:37:27 [+0200], Nadav Amit wrote:
>>>> Further thinking about it and looking at the rest of the series: wouldn’t it be
>>>> simpler to put flush_tlb_info and smp_call_function_many_cond()’s
>>>> cpumask on thread_struct? It would allow to support CONFIG_CPUMASK_OFFSTACK=y
>>>> case by preallocating cpumask on thread creation.
>>>>
>>>> I’m not sure whether the memory overhead is prohibitive.
>>>
>>> My Debian config has CONFIG_NR_CPUS=8192 which would add 1KiB if we add
>>> a plain cpumask_t. The allocation based on cpumask_size() would add just
>>> 8 bytes/ pointer to the struct which should be fine. We could even stash
>>> the mask in the pointer for CPUs <= 64 on 64bit.
>>> On RT it would be desired to have the memory and not to fallback to
>>> waiting with disabled preemption if the allocation fails.
>>>
>>> The flush_tlb_info are around 40 bytes + alignment. Maybe we could try
>>> stack first if this gets us to acceptable performance.
>>
>> I know it 1KB sounds a lot, but considering fpu size and other per-task
>> structures, it’s already almost 6KB. Just raising the option.
> 
> Sure. We might try allocating just the cpumask while allocating the
> task_struct so we won't do the whole 1KiB but just what is required and
> we might avoid allocation if it fits in the pointer size.
> 
> Sebastian

IIUC, you mean something like the following?


diff --git a/include/linux/sched.h b/include/linux/sched.h
index 5a5d3dbc9cdf..4222114cd34c 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -927,6 +927,8 @@ struct task_struct {
         unsigned short                  migration_disabled;
         unsigned short                  migration_flags;

+       cpumask_t                       *ipi_cpus;
+
  #ifdef CONFIG_PREEMPT_RCU
         int                             rcu_read_lock_nesting;
         union rcu_special               rcu_read_unlock_special;
diff --git a/kernel/smp.c b/kernel/smp.c
index 47c3b057f57f..f2bbd9b87f7f 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -112,6 +112,29 @@ void __init call_function_init(void)
         smpcfd_prepare_cpu(smp_processor_id());
  }

+static inline bool can_inline_cpumask(void)
+{
+       return cpumask_size() <= sizeof(cpumask_t *);
+}
+
+void alloc_ipi_cpumask(struct task_struct *task)
+{
+       if (can_inline_cpumask())
+               return;
+       /*
+        * Fallback to the default smp_call_function_many_cond
+        * logic if the allocation fails.
+        */
+       task->ipi_cpus = kmalloc(cpumask_size(), GFP_KERNEL);
+}
+
+static inline cpumask_t *get_local_cpumask(struct task_struct *cur)
+{
+       if (can_inline_cpumask())
+               return (cpumask_t *)&cur->ipi_cpus;
+       return cur->ipi_cpus;
+}
+
  static __always_inline void
  send_call_function_single_ipi(int cpu)
  {