Re: [PATCH 4/5] irq_work: Handle some irq_work in SOFTIRQ on PREEMPT_RT

[Peter Zijlstra <peterz@infradead.org>]

On Thu, Sep 30, 2021 at 11:53:48AM +0200, Sebastian Andrzej Siewior wrote:
> On 2021-09-30 11:07:18 [+0200], Peter Zijlstra wrote:
> > 
> > IIRC we have existing problems in -RT due to this irq_work softirq muck.
> 
> We have existing problems in -RT due irq_work being used without knowing
> the consequences.

Obviously :-)

> > I think the problem was something Jolsa found a while ago, where perf
> > defers to an irq_work (from NMI context) and that irq_work wants to
> > deliver signals, which it can't on -RT, so the whole thing gets punted
> > to softirq. With the end-result that if you self-profile RT tasks,
> > things come apart or something.
> 
> For signals (at least on x86) we this ARCH_RT_DELAYS_SIGNAL_SEND thingy
> where the signal is delayed until exit_to_user_mode_loop().

Yeah, I think that is what started much of the entry rework.. the signal
rework is still pending.

> perf_pending_event() is the only non-HARD on RT (on the perf side). I
> think that is due to perf_event_wakeup() where we have wake_up_all() and

Right, and that is exactly the problem, that needs to run at a higher
prio than the task that needs it, but softirq makes that 'difficult'.

One possible 'solution' would be to, instead of softirq, run the thing
as a kthread (worker or otherwise) such that userspace can at least set
the priority and has a small chance of making it work.

Runing them all at the same prio still sucks (much like the single
net-RX thing), but at least a kthread is somewhat controllable.

> read_lock_irqsave().

That one is really vexing, that really is just signal delivery to self
but even when signal stuff is fixed, we're stuck behind that fasync
rwlock :/