mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Marek Szyprowski <m.szyprowski@samsung.com>
To: Thomas Gleixner <tglx@linutronix.de>,
	Frederic Weisbecker <frederic@kernel.org>
Cc: LKML <linux-kernel@vger.kernel.org>,
	Marco Crivellari <marco.crivellari@suse.com>,
	Waiman Long <llong@redhat.com>,
	cgroups@vger.kernel.org
Subject: Re: [PATCH 1/2] genirq: Fix IRQ threads affinity VS cpuset isolated partitions
Date: Thu, 20 Nov 2025 17:34:43 +0100	[thread overview]
Message-ID: <13ba4125-8972-4ddb-b630-96e89bdd7248@samsung.com> (raw)
In-Reply-To: <87tsyokaug.ffs@tglx>

On 20.11.2025 16:00, Thomas Gleixner wrote:
> On Thu, Nov 20 2025 at 14:20, Frederic Weisbecker wrote:
>> Le Thu, Nov 20, 2025 at 12:51:31PM +0100, Marek Szyprowski a écrit :
>>> On 18.11.2025 15:30, Frederic Weisbecker wrote:
>>>> When a cpuset isolated partition is created / updated or destroyed,
>>>> the IRQ threads are affine blindly to all the non-isolated CPUs. And
>>>> this happens without taking into account the IRQ thread initial
>>>> affinity that becomes ignored.
>>>>
>>>> For example in a system with 8 CPUs, if an IRQ and its kthread are
>>>> initially affine to CPU 5, creating an isolated partition with only
>>>> CPU 2 inside will eventually end up affining the IRQ kthread to all
>>>> CPUs but CPU 2 (that is CPUs 0,1,3-7), losing the kthread preference for
>>>> CPU 5.
>>>>
>>>> Besides the blind re-affinity, this doesn't take care of the actual
>>>> low level interrupt which isn't migrated. As of today the only way to
>>>> isolate non managed interrupts, along with their kthreads, is to
>>>> overwrite their affinity separately, for example through /proc/irq/
>>>>
>>>> To avoid doing that manually, future development should focus on
>>>> updating the IRQs affinity whenever cpuset isolated partitions are
>>>> updated.
>>>>
>>>> In the meantime, cpuset shouldn't fiddle with IRQ threads directly.
>>>> To prevent from that, set the PF_NO_SETAFFINITY flag to them.
>>>>
>>>> Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
>>> This patch landed in today's linux-next as commit 844dcacab287 ("genirq:
>>> Fix interrupt threads affinity vs. cpuset isolated partitions"). In my
>>> tests I found that it triggers a warnings on some of my test systems.
>>> This is example of such warning:
>>>
>>> ------------[ cut here ]------------
>>> WARNING: CPU: 0 PID: 1 at kernel/kthread.c:599 kthread_bind_mask+0x2c/0x84
>> Erm, does this means that the IRQ thread got awaken before the first official
>> wakeup in wake_up_and_wait_for_irq_thread_ready()? This looks wrong...
>>
>> irq_startup() may be called on a few occcasions before. So perhaps
>> the IRQ already fired and woke up the kthread once before the "official"
>> first wake up?
>>
>> There seem to be some initialization ordering issue here...
> That's unavoidable because of locking and hen and egg ordering problems.
>
> When the thread is created the interrupt is not yet started up and
> therefore effective affinity is not known. So doing a bind there is
> pointless.
>
> The action has to be made visible _before_ starting it up as once the
> interrupt is unmasked it might fire. That's even more true for shared
> interrupts.
>
> The wakeup/bind/... muck cannot be invoked with the descriptor lock
> held, so it has to be done _after_ the thing went live.
>
> So yes an interrupt which fires before kthread_bind() is invoked might
> exactly have that effect. It wakes it from the kthread() wait and it
> goes straight through to the thread function.
>
> That's why the original code just set the affinity bit and let the
> thread itself handle it.
>
> There are three options to solve that:
>
>        1) Bind the thread right after kthread_create() to a random
>           housekeeping task and let move itself over to the real place
>           once it came up (at this point the affinity is established)
>
>        2) Serialize the setup so that the thread (even, when woken up
>           early) get's stuck in UNINTERRUPTIBLE state _before_ it can
>           reach wait_for_interrupt() which waits with INTERRUPTIBLE
>
>        3) Teach kthread_create() that the thread is subject to being
>           bound to something later on and implement that serialization
>           there.
>
> #1 is pretty straight forward. See untested below.

This fixes the observed issue. The only problem with that patch is 
incorrect set of arguments to kthread_create_on_cpu() (it doesn't 
support printf-style args).

If that matters:

Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>


> #2 is ugly (I tried)
>
> #3 could be useful in general, but that needs more thoughts
>
> Thanks
>
>          tglx
> ---
> --- a/kernel/irq/handle.c
> +++ b/kernel/irq/handle.c
> @@ -133,7 +133,15 @@ void __irq_wake_thread(struct irq_desc *
>   	 */
>   	atomic_inc(&desc->threads_active);
>   
> -	wake_up_process(action->thread);
> +	/*
> +	 * This might be a premature wakeup before the thread reached the
> +	 * thread function and set the IRQTF_READY bit. It's waiting in
> +	 * kthread code with state UNINTERRUPTIBLE. Once it reaches the
> +	 * thread function it waits with INTERRUPTIBLE. The wakeup is not
> +	 * lost in that case because the thread is guaranteed to observe
> +	 * the RUN flag before it goes to sleep in wait_for_interrupt().
> +	 */
> +	wake_up_state(action->thread, TASK_INTERRUPTIBLE);
>   }
>   
>   static DEFINE_STATIC_KEY_FALSE(irqhandler_duration_check_enabled);
> --- a/kernel/irq/manage.c
> +++ b/kernel/irq/manage.c
> @@ -1026,16 +1026,8 @@ static void irq_thread_check_affinity(st
>   	set_cpus_allowed_ptr(current, mask);
>   	free_cpumask_var(mask);
>   }
> -
> -static inline void irq_thread_set_affinity(struct task_struct *t,
> -					   struct irq_desc *desc)
> -{
> -	kthread_bind_mask(t, irq_data_get_effective_affinity_mask(&desc->irq_data));
> -}
>   #else
>   static inline void irq_thread_check_affinity(struct irq_desc *desc, struct irqaction *action) { }
> -static inline void irq_thread_set_affinity(struct task_struct *t,
> -					   struct irq_desc *desc) { }
>   #endif
>   
>   static int irq_wait_for_interrupt(struct irq_desc *desc,
> @@ -1220,7 +1212,6 @@ static void wake_up_and_wait_for_irq_thr
>   	if (!action || !action->thread)
>   		return;
>   
> -	irq_thread_set_affinity(action->thread, desc);
>   	wake_up_process(action->thread);
>   	wait_event(desc->wait_for_threads,
>   		   test_bit(IRQTF_READY, &action->thread_flags));
> @@ -1389,26 +1380,35 @@ static void irq_nmi_teardown(struct irq_
>   static int
>   setup_irq_thread(struct irqaction *new, unsigned int irq, bool secondary)
>   {
> +	/*
> +	 * At this point interrupt affinity is not known. just assume that
> +	 * the current CPU is not isolated and valid to bring the thread
> +	 * up. The thread will move itself over to the right place once the
> +	 * whole setup is complete.
> +	 */
> +	unsigned int cpu = raw_smp_processor_id();
>   	struct task_struct *t;
>   
> -	if (!secondary) {
> -		t = kthread_create(irq_thread, new, "irq/%d-%s", irq,
> -				   new->name);
> -	} else {
> -		t = kthread_create(irq_thread, new, "irq/%d-s-%s", irq,
> -				   new->name);
> -	}
> +	if (!secondary)
> +		t = kthread_create_on_cpu(irq_thread, new, cpu, "irq/%d-%s", irq, new->name);
> +	else
> +		t = kthread_create_on_cpu(irq_thread, new, cpu, "irq/%d-s-%s", irq, new->name);
>   
>   	if (IS_ERR(t))
>   		return PTR_ERR(t);
>   
>   	/*
> -	 * We keep the reference to the task struct even if
> -	 * the thread dies to avoid that the interrupt code
> -	 * references an already freed task_struct.
> +	 * We keep the reference to the task struct even if the thread dies
> +	 * to avoid that the interrupt code references an already freed
> +	 * task_struct.
>   	 */
>   	new->thread = get_task_struct(t);
>   
> +	/*
> +	 * Ensure the thread adjusts the affinity once it reaches the
> +	 * thread function.
> +	 */
> +	new->thread_flags = BIT(IRQTF_AFFINITY);
>   	return 0;
>   }
>   
>
Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland


  parent reply	other threads:[~2025-11-20 16:34 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-11-18 14:30 [PATCH 0/2] genirq: Fix IRQ threads VS cpuset Frederic Weisbecker
2025-11-18 14:30 ` [PATCH 1/2] genirq: Fix IRQ threads affinity VS cpuset isolated partitions Frederic Weisbecker
2025-11-18 15:26   ` [tip: irq/core] genirq: Fix interrupt threads affinity vs. " tip-bot2 for Frederic Weisbecker
2025-11-18 16:27   ` [PATCH 1/2] genirq: Fix IRQ threads affinity VS " Waiman Long
2025-11-18 17:23     ` Thomas Gleixner
2025-11-18 17:29       ` Waiman Long
     [not found]   ` <CGME20251120115132eucas1p160658ba63831911aadd0b0c2f70b7e28@eucas1p1.samsung.com>
2025-11-20 11:51     ` Marek Szyprowski
2025-11-20 13:20       ` Frederic Weisbecker
2025-11-20 15:00         ` Thomas Gleixner
2025-11-20 15:50           ` Frederic Weisbecker
2025-11-20 19:09             ` Thomas Gleixner
2025-11-20 16:34           ` Marek Szyprowski [this message]
2025-11-20 13:45       ` Thomas Gleixner
2025-11-18 14:30 ` [PATCH 2/2] genirq: Remove cpumask availability check on kthread affinity setting Frederic Weisbecker
2025-11-18 15:26   ` [tip: irq/core] " tip-bot2 for Frederic Weisbecker
2025-11-18 15:14 ` [PATCH 0/2] genirq: Fix IRQ threads VS cpuset Thomas Gleixner
2025-11-18 17:44   ` Frederic Weisbecker

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=13ba4125-8972-4ddb-b630-96e89bdd7248@samsung.com \
    --to=m.szyprowski@samsung.com \
    --cc=cgroups@vger.kernel.org \
    --cc=frederic@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=llong@redhat.com \
    --cc=marco.crivellari@suse.com \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®