Re: [patch] sched: optimize siblings status check logic in wake_idle()

[Nick Piggin <npiggin@suse.de>]

On Fri, Mar 02, 2007 at 08:23:32PM -0800, Suresh B wrote:
> When a logical cpu 'x' already has more than one process running, then most likely
> the siblings of that cpu 'x' must be busy. Otherwise the idle siblings
> would have likely(in most of the scenarios) picked up the extra load making
> the load on 'x' atmost one.

Do you have any stats on this?

> Use this logic to eliminate the siblings status check and minimize the cache
> misses encountered on a heavily loaded system.

Well it does increase the cacheline footprint a bit, but all cachelines
should be local to our L1 cache, presuming you don't have any CPUs where
threads have seperate caches.

What sort of numbers do you have?

> 
> Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
> ---
> 
> diff --git a/kernel/sched.c b/kernel/sched.c
> index 0dc7572..d1ecc56 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -1368,7 +1368,16 @@ static int wake_idle(int cpu, struct task_struct *p)
>  	struct sched_domain *sd;
>  	int i;
>  
> -	if (idle_cpu(cpu))
> +	/*
> +	 * If it is idle, then it is the best cpu to run this task.
> +	 *
> +	 * This cpu is also the best, if it has more than one task already.
> +	 * Siblings must be also busy(in most cases) as they didn't already
> +	 * pickup the extra load from this cpu and hence we need not check
> +	 * sibling runqueue info. This will avoid the checks and cache miss
> +	 * penalities associated with that.
> +	 */
> +	if (idle_cpu(cpu) || cpu_rq(cpu)->nr_running > 1)
>  		return cpu;
>  
>  	for_each_domain(cpu, sd) {