Re: [PATCH v2 5/6] sched/deadline: Make DL capacity-aware

[Dietmar Eggemann <dietmar.eggemann@arm.com>]

On 04/05/2020 05:58, Pavan Kondeti wrote:
> On Fri, May 01, 2020 at 06:12:07PM +0200, Dietmar Eggemann wrote:
>> On 30/04/2020 15:10, Pavan Kondeti wrote:
>>> On Mon, Apr 27, 2020 at 10:37:08AM +0200, Dietmar Eggemann wrote:
>>>> From: Luca Abeni <luca.abeni@santannapisa.it>
>>
>> [...]
>>
>>>> @@ -1653,10 +1654,19 @@ select_task_rq_dl(struct task_struct *p, int cpu, int sd_flag, int flags)
>>>>  	 * other hand, if it has a shorter deadline, we
>>>>  	 * try to make it stay here, it might be important.
>>>>  	 */
>>>> -	if (unlikely(dl_task(curr)) &&
>>>> -	    (curr->nr_cpus_allowed < 2 ||
>>>> -	     !dl_entity_preempt(&p->dl, &curr->dl)) &&
>>>> -	    (p->nr_cpus_allowed > 1)) {
>>>> +	select_rq = unlikely(dl_task(curr)) &&
>>>> +		    (curr->nr_cpus_allowed < 2 ||
>>>> +		     !dl_entity_preempt(&p->dl, &curr->dl)) &&
>>>> +		    p->nr_cpus_allowed > 1;
>>>> +
>>>> +	/*
>>>> +	 * Take the capacity of the CPU into account to
>>>> +	 * ensure it fits the requirement of the task.
>>>> +	 */
>>>> +	if (static_branch_unlikely(&sched_asym_cpucapacity))
>>>> +		select_rq |= !dl_task_fits_capacity(p, cpu);
>>>> +
>>>> +	if (select_rq) {
>>>>  		int target = find_later_rq(p);
>>>
>>> I see that find_later_rq() checks if the previous CPU is part of
>>> later_mask and returns it immediately. So we don't migrate the
>>> task in the case where there previous CPU can't fit the task and
>>> there are no idle CPUs on which the task can fit. LGTM.
>>
>> Hope I understand you here. I don't think that [patch 6/6] provides this
>> already.
>>
>> In case 'later_mask' has no fitting CPUs, 'max_cpu' is set in the
>> otherwise empty 'later_mask'. But 'max_cpu' is not necessary task_cpu(p).
>>
>> Example on Juno [L b b L L L] with thread0-0 (big task)
>>
>>      cpudl_find [thread0-0 2117] orig later_mask=0,3-4 later_mask=0
>>   find_later_rq [thread0-0 2117] task_cpu=2 later_mask=0
>>
>> A tweak could be added favor task_cpu(p) in case it is amongst the CPUs
>> with the maximum capacity in cpudl_find() for the !fit case.
>>
> 
> You are right. max_cpu can be other than task_cpu(p) in which case we
> migrate the task though it won't fit on the new CPU. While introducing
> capacity awareness in RT, Quais made the below change to avoid the
> migration. We can do something similar here also.
> 
> commit b28bc1e002c2 (sched/rt: Re-instate old behavior in select_task_rq_rt())

I'm not sure something like this is necessary here.

With DL capacity awareness we reduce the later_mask returned by
cpudl_find() in find_later_rq() to those idle CPUs which can handle p
or in case there is none to (the/a) 'non-fitting CPU w/ max capacity'.

We just have to favor task_cpu(p) in [patch 6/6] in case it is part
of the initial later_mask and among these 'non-fitting CPUs w/ max
capacity'.

This will make sure that it gets chosen so find_later_rq() returns it
before the 'for_each_domain()' loop.

And I guess we still want to migrate if there is a non-fitting CPU w/ a
higher CPU capacity than task_cpu() (tri-gear).

@@ -137,7 +137,8 @@ int cpudl_find(struct cpudl *cp, struct task_struct *p,
 
                                cap = capacity_orig_of(cpu);
 
-                               if (cap > max_cap) {
+                               if (cap > max_cap ||
+                                   (cpu == task_cpu(p) && cap == max_cap)) {
                                        max_cap = cap;
                                        max_cpu = cpu;

In case task_cpu() is not part of later_cpu, the 'max CPU capacity CPU' is
returned as 'best_cpu'. 

[...]