Re: [patch 1/6] sched, nohz: introduce nohz_flags in the struct rq

[Suresh Siddha <suresh.b.siddha@intel.com>]

On Thu, 2011-11-24 at 02:24 -0800, Peter Zijlstra wrote:
> On Fri, 2011-11-18 at 15:03 -0800, Suresh Siddha wrote:
> > plain text document attachment (introduce_rq_nohz_flags.patch)
> > Introduce nohz_flags in the struct rq, which will track two flags for now.
> > 
> > NOHZ_TICK_STOPPED will convey the stop tick status that gets set when
> > the tick is stopped and cleared during the first busy tick after the tick
> > is restarted.
> > 
> > NOHZ_BALANCE_KICK will track the need for nohz idle load balance
> > on this rq.
> 
> The changelog is missing the crucial part: Why are we doing this? :-)

We already had nohz_balance_kick in the rq, which was not being updated
atomically. So it is quite possible that two cpu's can simultaneously
kick the idle load balancing cpu. NOHZ_BALANCE_KICK bit of the
nohz_flags will track the need for nohz idle load balance on this rq and
will be updated atomically.

When the rq is ready to run a task when coming out of tickless idle,
tick is restarted and data structures used for nohz idle load balancing
are updated immediately. This will result in too many nohz idle load
balancer status updates, if the rq comes out of tickless for running one
(or very few) short running tasks before going back to tickless idle.
This is resulting in scalability issues especially if there are lot of
logical cpu's that enters and exit idle often. There is no need to
update the nohz idle load balancer data structures for the semi-idle cpu
so often (as the information for the need of the idle load balancing and
who needs to be kicked is needed only at the busy cpu's tick).
NOHZ_TICK_STOPPED in rq's nohz_flags is introduced to keep track of the
tick stopped status that gets set when the tick is stopped. It will be
used to update the nohz idle load balancer data structures during the
first busy tick after the tick is restarted. At this first busy tick
after tickless idle, NOHZ_TICK_STOPPED flag will be reset.

I will include this info in the next update.

thanks,
suresh