Re: SCHED_RR and kernel threads

[Con Kolivas <kernel@kolivas.org>]

Stephen Warren writes:

>> From: Con Kolivas [mailto:kernel@kolivas.org] 
>> Stephen Warren wrote:
>> > It appears that during times of high application CPU usage, some
>> > *kernel* threads don't get to run.
>> > ...
>> > This appears to be due to the fact that the kernel threads are all
>> > SCHED_OTHER, so our SCHED_RR user-space application trumps them!
>> 
>> Don't run your userspace at SCHED_RR? The kernel threads are 
>> SCHED_NORMAL precisely for the reason that you wont get real time 
>> performance if the kernel threads rear their ugly heads, 
>> albeit rarely.
> 
> We have actually set the kernel threads to priority SCHED_RR 50, and
> most user-space threads to SCHED_RR priority 50. Some critical
> user-space threads are above priority 50.
> 
> Won't this allow the kernel and user space threads to co-operate nicely
> all the time?
> 
> What is it specifically that will make kernel SCHED_RR threads cause
> non-real-time operation? If it's just a bunch of corner cases or odd
> conditions, we may be in an environment we can control so that doesn't
> happen...
> 
> I guess we could have most threads stay at SCHED_NORMAL, and just make
> the few critical threads SCHED_RR, but I'm getting a lot of push-back on
> this, since it makes our thread API a lot more complex.

Your workaround is not suitable for the kernel at large. Preventing 
starvation of the system if you are using SCHED_RR threads is up to your 
userspace apps to provide. SCHED_RR is _not_ designed to use 100% of the cpu 
all the time, but to provide minimum latency preempting everything lower 
priority than itself when scheduled. The kernel threads do not need that 
sort of control and can potentially starve critical userspace threads during 
heavy system stress.

Cheers,
Con