Re: [PATCH v6] sched: fix init NOHZ_IDLE flag

[Peter Zijlstra <peterz@infradead.org>]

On Fri, 2013-04-19 at 15:10 +0200, Vincent Guittot wrote:
> As suggested by Frederic Weisbecker, another solution is to have the
> same
> rcu lifecycle for both NOHZ_IDLE and sched_domain struct. I have
> introduce
> a new sched_domain_rq struct that is the entry point for both
> sched_domains
> and objects that must follow the same lifecycle like NOHZ_IDLE flags.
> They
> will share the same RCU lifecycle and will be always synchronized.
> 
> The synchronization is done at the cost of :
>  - an additional indirection for accessing the first sched_domain
> level
>  - an additional indirection and a rcu_dereference before accessing to
> the
>    NOHZ_IDLE flag.

> diff --git a/include/linux/sched.h b/include/linux/sched.h
> index d35d2b6..61ad5f1 100644
> --- a/include/linux/sched.h
> +++ b/include/linux/sched.h
> @@ -959,6 +959,18 @@ struct sched_domain {
>         unsigned long span[0];
>  };
>  
> +/*
> + * Some flags must stay synchronized with fields of sched_group_power
> and as a
> + * consequence they must follow the same lifecycle for the lockless
> scheme.
> + * sched_domain_rq encapsulates those flags and sched_domains in one
> RCU
> + * object.
> + */
> +struct sched_domain_rq {
> +       struct sched_domain *sd;
> +       unsigned long flags;
> +       struct rcu_head rcu;    /* used during destruction */
> +};

I'm not quite getting things.. what's wrong with adding this flags
thing to sched_domain itself? That's already RCU destroyed so why add a
second RCU layer?

We also have the root_domain for things that don't need to go in a
hierarchy but are once per cpu -- it sounds like this is one of those
things; iirc the root_domain life-time is the same as the entire
sched_domain tree so adding it to the root_domain is also an option.