[PATCH 0/3] irq_work: CPU-hotplug improvements on PREEMPT_RT

[PATCH 0/3] irq_work: CPU-hotplug improvements on PREEMPT_RT

[Sebastian Andrzej Siewior]

There was an unrelated thread which made me look into how
!IRQ_WORK_HARD_IRQ is processed on PREEMPT_RT. Turns out the queue of
callbacks gets never flushed on CPU shutdown. !PREEMPT_RT has a flush
but on PREEMPT_RT the thread context is required.
This almost never happens but if it happens it is a pain.

A small fix is for the smpboot thread to invoke the thread-function
before parking. This is an improvement. There is still a small window if
callbacks are added after the irq_work thread has been shutdown.
As a fix I added an explicit flush on the control CPU once the CPU is
dead.
From what I've seen there are two users that don't behave as expected if
invoked from the "wrong" CPU:
- cgrp_dead_tasks_iwork()
  This shouldn't be a problem because no task terminates after the
  smpboot thread parked. Everythng that was queued up before will be
  flushed during parking.
  Regardless posted
     https://lore.kernel.org/all/20260911101900.984420-1-bigeasy@linutronix.de/

- wake_up_klogd_work()
  This will not cause data curruption but console printing will be
  delayed until the following printk. Not pretty, posted
     https://lore.kernel.org/all/20260911103832.w6C8cT4L@linutronix.de/

Sebastian Andrzej Siewior (3):
  irq_work: Update a comment regarding CPU hotplug invocation
  irq_work: Flush lazy work CPU down on PREEMPT_RT
  smpboot: Don't park the thread if work is pending

 include/linux/irq_work.h |  2 ++
 kernel/irq_work.c        | 16 ++++++++++++----
 kernel/smp.c             |  1 +
 kernel/smpboot.c         |  6 ++++--
 4 files changed, 19 insertions(+), 6 deletions(-)

-- 
2.55.0

[PATCH 1/3] irq_work: Update a comment regarding CPU hotplug invocation

[Sebastian Andrzej Siewior]

hotplug_cfd() no longer exists and the flow changed a bit since the
rework of CPU hotplug as of commit 31487f8328f20 ("smp/cfd: Convert core
to hotplug state machine").

Update the comment to reflect reality as of today.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 kernel/irq_work.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/kernel/irq_work.c b/kernel/irq_work.c
index f7e2dc2c30c62..9f19c05c9962c 100644
--- a/kernel/irq_work.c
+++ b/kernel/irq_work.c
@@ -252,10 +252,7 @@ static void irq_work_run_list(struct llist_head *list)
 		irq_work_single(work);
 }
 
-/*
- * hotplug calls this through:
- *  hotplug_cfd() -> flush_smp_call_function_queue()
- */
+/* CPU hotplug calls this through smpcfd_dying_cpu() */
 void irq_work_run(void)
 {
 	irq_work_run_list(this_cpu_ptr(&raised_list));
-- 
2.55.0

[PATCH 2/3] irq_work: Flush lazy work CPU down on PREEMPT_RT

[Sebastian Andrzej Siewior]

PREEMPT_RT invokes IRQ_WORK_LAZY callbacks and callbacks which are not
explicitly marked IRQ_WORK_HARD_IRQ from thread context.  If a CPU gets
shutdown, the irq_work is flushed during smpcfd_dying_cpu() which is
invoked on the target CPU with disabled interrupts.

The callbacks enqueued on lazy_list on PREEMPT_RT are not flushed because
they require thread context. The callbacks remain and get processed once
the CPU gets back online.

per-CPU irq_work items continue to work for enqueues on their local-CPU,
only the work on the "offline" CPU get stuck. "Single" irq_work are
worse because they remain "claimed" and can not be used again. One
"symptom" is the late printk() in CPU down path which enqueues the
irq_work for printing but it never gets scheduled. Further prints don't
print to the console because the irq_work is "pending" of the offline
CPU.
Another problem is the canceling/ flushing of the irq_work which is
stuck. Since the irq_work will not continue, the task waiting for its
completion will block waiting.

While looking through the users of irq_work, it does not matter if the
callback is enqueued on another CPU. This makes it possible to invoke
the callbacks from a remote CPU by smpcfd_dead_cpu(). This is invoked
from the control CPU with enabled interrupts shortly after the
hotplugged CPU is shutdown so the llist can be accessed safely.

Add irq_work_run_cpu() which invokes the irq_work callbacks of the
specified dead CPU. Invoke it from smpcfd_dead_cpu().

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 include/linux/irq_work.h |  2 ++
 kernel/irq_work.c        | 11 +++++++++++
 kernel/smp.c             |  1 +
 3 files changed, 14 insertions(+)

diff --git a/include/linux/irq_work.h b/include/linux/irq_work.h
index c5afd053ae32c..7184e12739653 100644
--- a/include/linux/irq_work.h
+++ b/include/linux/irq_work.h
@@ -59,6 +59,7 @@ void irq_work_sync(struct irq_work *work);
 #include <asm/irq_work.h>
 
 void irq_work_run(void);
+void irq_work_run_cpu(unsigned int cpu);
 bool irq_work_needs_cpu(void);
 void irq_work_single(void *arg);
 
@@ -67,6 +68,7 @@ void arch_irq_work_raise(void);
 #else
 static inline bool irq_work_needs_cpu(void) { return false; }
 static inline void irq_work_run(void) { }
+static inline void irq_work_run_cpu(unsigned int cpu) { }
 static inline void irq_work_single(void *arg) { }
 #endif
 
diff --git a/kernel/irq_work.c b/kernel/irq_work.c
index 9f19c05c9962c..73eabcbdcd50c 100644
--- a/kernel/irq_work.c
+++ b/kernel/irq_work.c
@@ -263,6 +263,17 @@ void irq_work_run(void)
 }
 EXPORT_SYMBOL_GPL(irq_work_run);
 
+void irq_work_run_cpu(unsigned int cpu)
+{
+	if (WARN_ON_ONCE(!cpumask_test_cpu(cpu, cpu_dying_mask)))
+		return;
+
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		return;
+
+	irq_work_run_list(per_cpu_ptr(&lazy_list, cpu));
+}
+
 void irq_work_tick(void)
 {
 	struct llist_head *raised = this_cpu_ptr(&raised_list);
diff --git a/kernel/smp.c b/kernel/smp.c
index b696bcc60c08f..8f530a094c304 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -87,6 +87,7 @@ int smpcfd_dead_cpu(unsigned int cpu)
 
 	free_cpumask_var(cfd->cpumask);
 	free_cpumask_var(cfd->cpumask_ipi);
+	irq_work_run_cpu(cpu);
 	return 0;
 }
 
-- 
2.55.0

[PATCH 3/3] smpboot: Don't park the thread if work is pending

[Sebastian Andrzej Siewior]

Once a smpboot thread gets work assigned it receives a wake up. If the
thread had no chance to run and receives a parking requesst during a
CPU-hotplug event then it is parked without running the callback first.
As a result the enqueued work remains stuck until the CPU gets back
online.

There is nothing wrong with invoking the thread function first and
parking in the following iteration. This would ensure that the callbacks
are processed before the shutdown.

Honor the park request once thread function does not need to run.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 kernel/smpboot.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kernel/smpboot.c b/kernel/smpboot.c
index 4503b60ce9bd2..3f60e8c6dd301 100644
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -103,6 +103,7 @@ static int smpboot_thread_fn(void *data)
 {
 	struct smpboot_thread_data *td = data;
 	struct smp_hotplug_thread *ht = td->ht;
+	bool should_run;
 
 	while (1) {
 		set_current_state(TASK_INTERRUPTIBLE);
@@ -117,7 +118,8 @@ static int smpboot_thread_fn(void *data)
 			return 0;
 		}
 
-		if (kthread_should_park()) {
+		should_run = td->status == HP_THREAD_ACTIVE && ht->thread_should_run(td->cpu);
+		if (kthread_should_park() && !should_run) {
 			__set_current_state(TASK_RUNNING);
 			preempt_enable();
 			if (ht->park && td->status == HP_THREAD_ACTIVE) {
@@ -151,7 +153,7 @@ static int smpboot_thread_fn(void *data)
 			continue;
 		}
 
-		if (!ht->thread_should_run(td->cpu)) {
+		if (!should_run) {
 			preempt_enable_no_resched();
 			schedule();
 		} else {
-- 
2.55.0

Re: [PATCH 2/3] irq_work: Flush lazy work CPU down on PREEMPT_RT

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Executing a dying CPU's lazy irq_work callbacks on the control CPU violates `this_cpu_*()` assumptions, leading to memory leaks and state corruption.
- [High] Use-after-free of `irq_work` structure due to missing `rcu_read_lock()` in `irq_work_run_cpu()`.
--

commit c2c4af12bef2f74291db156f37cadb81d93b8598
Author: Sebastian Andrzej Siewior <bigeasy@linutronix.de>

irq_work: Flush lazy work CPU down on PREEMPT_RT

This commit adds support for flushing lazy IRQ work for a dead CPU when
PREEMPT_RT is enabled. It introduces irq_work_run_cpu() to process the
dying CPU's lazy list from the control CPU during smpcfd_dead_cpu().

> While looking through the users of irq_work, it does not matter if the
> callback is enqueued on another CPU. This makes it possible to invoke
> the callbacks from a remote CPU by smpcfd_dead_cpu().

[Severity: High]
Is this statement accurate?

Several lazy irq_work callbacks (e.g., cgrp_dead_tasks_iwork_fn in
kernel/cgroup/cgroup.c) rely on this_cpu_*() accessors to process local queues:

kernel/cgroup/cgroup.c:cgrp_dead_tasks_iwork_fn() {
    ...
    lnode = llist_del_all(this_cpu_ptr(&cgrp_dead_tasks));
    ...
}

Executing them remotely on the control CPU causes them to access and drain the
control CPU's per-CPU lists instead of the dying CPU's. Could this permanently
leak task structs and pin cgroups on the dead CPU indefinitely, while
incorrectly altering the state of the control CPU?

> diff --git a/kernel/irq_work.c b/kernel/irq_work.c
> index 9f19c05c9962c..73eabcbdcd50c 100644
> --- a/kernel/irq_work.c
> +++ b/kernel/irq_work.c
[ ... ]
> @@ -263,6 +263,17 @@ void irq_work_run(void)
>  }
>  EXPORT_SYMBOL_GPL(irq_work_run);
>  
> +void irq_work_run_cpu(unsigned int cpu)
> +{
> +	if (WARN_ON_ONCE(!cpumask_test_cpu(cpu, cpu_dying_mask)))
> +		return;
> +
> +	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
> +		return;
> +
> +	irq_work_run_list(per_cpu_ptr(&lazy_list, cpu));
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[Severity: High]
Could this result in a use-after-free of the irq_work structure?

irq_work_sync() protects against concurrent execution by waiting for
IRQ_WORK_BUSY to clear and then calling synchronize_rcu(). This assumes the work
callback executes inside an RCU read-side critical section.

However, irq_work_run_cpu() runs from preemptible thread context in
smpcfd_dead_cpu() without holding the RCU read lock. irq_work_single() can be
preempted after clearing IRQ_WORK_BUSY but before calling rcuwait_wake_up().

If irq_work_sync() on another CPU sees the BUSY bit clear, it will call
synchronize_rcu() which returns immediately, and the caller may then free the
irq_work structure. When the preempted work function resumes, could it access
freed memory in rcuwait_wake_up()?

> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260911143815.997254-1-bigeasy@linutronix.de?part=2

Re: [PATCH 2/3] irq_work: Flush lazy work CPU down on PREEMPT_RT

[Sebastian Andrzej Siewior]

On 2026-09-11 14:55:55 [+0000], sashiko-bot@kernel.org wrote:
> [Severity: High]
> Is this statement accurate?
> 
> Several lazy irq_work callbacks (e.g., cgrp_dead_tasks_iwork_fn in
> kernel/cgroup/cgroup.c) rely on this_cpu_*() accessors to process local queues:
> 
> kernel/cgroup/cgroup.c:cgrp_dead_tasks_iwork_fn() {
>     ...
>     lnode = llist_del_all(this_cpu_ptr(&cgrp_dead_tasks));
>     ...
> }
> 
> Executing them remotely on the control CPU causes them to access and drain the
> control CPU's per-CPU lists instead of the dying CPU's. Could this permanently
> leak task structs and pin cgroups on the dead CPU indefinitely, while
> incorrectly altering the state of the control CPU?

The cover letter explained why this shouldn't happen. Additionally the
referenced cgroup patch is already staged for 7.4. 

> > diff --git a/kernel/irq_work.c b/kernel/irq_work.c
> > index 9f19c05c9962c..73eabcbdcd50c 100644
> > --- a/kernel/irq_work.c
> > +++ b/kernel/irq_work.c
> [ ... ]
> > @@ -263,6 +263,17 @@ void irq_work_run(void)
> >  }
> >  EXPORT_SYMBOL_GPL(irq_work_run);
> >  
> > +void irq_work_run_cpu(unsigned int cpu)
> > +{
> > +	if (WARN_ON_ONCE(!cpumask_test_cpu(cpu, cpu_dying_mask)))
> > +		return;
> > +
> > +	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
> > +		return;
> > +
> > +	irq_work_run_list(per_cpu_ptr(&lazy_list, cpu));
>         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> [Severity: High]
> Could this result in a use-after-free of the irq_work structure?
> 
> irq_work_sync() protects against concurrent execution by waiting for
> IRQ_WORK_BUSY to clear and then calling synchronize_rcu(). This assumes the work
> callback executes inside an RCU read-side critical section.
> 
> However, irq_work_run_cpu() runs from preemptible thread context in
> smpcfd_dead_cpu() without holding the RCU read lock. irq_work_single() can be
> preempted after clearing IRQ_WORK_BUSY but before calling rcuwait_wake_up().
> 
> If irq_work_sync() on another CPU sees the BUSY bit clear, it will call
> synchronize_rcu() which returns immediately, and the caller may then free the
> irq_work structure. When the preempted work function resumes, could it access
> freed memory in rcuwait_wake_up()?

Okay, this is a bummer.

> > +}
> 

Sebastian

Re: [PATCH 0/3] irq_work: CPU-hotplug improvements on PREEMPT_RT

[Sebastian Andrzej Siewior]

On 2026-09-11 16:38:11 [+0200], To linux-kernel@vger.kernel.org wrote:
> There was an unrelated thread which made me look into how
> !IRQ_WORK_HARD_IRQ is processed on PREEMPT_RT. Turns out the queue of
> callbacks gets never flushed on CPU shutdown. !PREEMPT_RT has a flush
> but on PREEMPT_RT the thread context is required.
> This almost never happens but if it happens it is a pain.
 
Something else that almost never happens: printk() has this
__printk_percpu_data_ready() thingy. I interpreted this wrongly last
Friday: If someone does prinkt() before per-CPU pages are setup then the
work item has the IRQ_WORK_CLAIMED bit set and this work item is copied
to every CPU's view of the per-CPU data. It is not protecting itself
from setting bits but the irq_work API.
irq_work has its own per-CPU llist_head which gets copied during per-CPU
setup. As a result, once interrupts are enabled the IRQ-work will be
invoked on each CPU passing the "original" irq_work pointer (from the
per-CPU-data-init sample). The actual per-CPU data (after setup) never
gets cleared.

We should either disallow irq_work_queue() until per-CPU data is setup
or flush the irq-work queue before the setup (so it has no items while
the data is copied).
While the printk workaround feels wrong, it gets the job done for now. I
am a bit worried that if its usage spreads around and we get other early
users…

Sebastian