RE: sched/fair: Long delays starting RT processes on idle cpu.

[David Laight <David.Laight@ACULAB.COM>]

From Steven Rostedt
> Sent: 27 January 2020 15:50
> On Mon, 27 Jan 2020 15:39:24 +0000
> David Laight <David.Laight@ACULAB.COM> wrote:
> 
> > I'd have thought that the processor should wake up much faster than that.
> > I can't see the memory write that is paired with the monitor/mwait.
> > Does it need a strong barrier?
> 
> You may want to prevent the CPU from going into a deep C state. 90us is
> something I would expect if the CPU is in a deep C state (I've seen
> much longer wake up times due to deep C state).
> 
> Boot the kernel with idle=poll and see if you can trigger the same
> latency. If not, then you know it's the CPU going into a deep C state
> that is causing your latency.

With idle=poll the delays seem to be minimal.

Is there any way to limit the C state entered by mwait?
Or to try the check that monitor/mwait is being triggered properly.
Is there a clfulsh() in the writing side?
If not, might one help??

I see almost repeatable delays eg for 6 processes being scheduled (more or less in sequence).
10us waking cpu 2.
60us waking cpu 1.
40us waking cpu 0 and 40us waking cpu3.
20us waking cpu 2.
20us waking cpu 0.
None of the processes runs for anything like the delays.
The whole thing repeats every 10ms.
Note that the first process is actually signalling a CV that 4 of the other 5
are waiting on.
So there are cumulative delays of around 140us before the 4th is woken.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)