Re: SCHED_RR and kernel threads

[Con Kolivas <kernel@kolivas.org>]

Stephen Warren writes:

>> From: Con Kolivas [mailto:kernel@kolivas.org] 
>> Stephen Warren writes:
>>> I guess we could have most threads stay at SCHED_NORMAL, and just
> make
>>> the few critical threads SCHED_RR, but I'm getting a lot of push-back
> on
>>> this, since it makes our thread API a lot more complex.
>>
>>Your workaround is not suitable for the kernel at large.
> 
> You mean the official kernel.org kernel? I wasn't implying that the
> patch should be part of that!
> 
> In our system we have literally EVERY single thread (kernel, user-space
> daemons, and user-space applications) all setup as SCHED_RR with
> identical priority at present, except a couple higher priority threads.
> We did this initially for user-space by replacing /sbin/init with a
> wrapper that set the scheduler policy and default priority, and verified
> that this was inherited by all daemons & application threads. Then, we
> found that the kernel threads could get starved in some situations,
> hence the kernel change.
> 
> Our threading model dictates that every thread have a priority (so that
> the thread model is portable between Linux, embedded RTOSs etc.), and in
> Linux AFAIK, the only way to implement priorities is to use a real-time
> scheduling policy. Some threads do a lot of calculation. We want to make
> them equal (or probably, lower) priority to the kernel threads, so
> therefore the kernel threads must then be SCHED_RR.
> 
> Can you elaborate on specific conditions that would cause the kernel
> threads to suck up unusual amounts of CPU time?
> 
> In our application, keyboard processing is a real-time requirement, so
> if that is performed in a kernel thread, that kernel thread should be
> real-time. We basically want the control to insert e.g. the keyboard
> processing kernel thread into the middle of our priority hierarchy,
> rather than having it forced as the lowest possible priority.
> 
> I get the impression you're implying that scheduling doesn't work
> correctly in this situation - that if kernel threads are set to
> SCHED_RR, they can still lock out user-space threads of the same or
> higher priority? Is this what you're saying, or do you mean that the
> kernel threads can lock out user-space threads of *lower* priority,
> which is to be expected. In all the RTOS's I've seen, all threads are
> SCHED_RR, thus mimicking the situation we've creating by patching our
> kernel...

If everything is the same priority then you've created a simple round robin 
scheduler out of the kernel and that's fine for your setting. If you're 
looking for another alternative to this, check out the email I posted in the 
last week for implementing a sched bound policy. We will be looking at 
implementing that in the near future.

Cheers,
Con