Re: [RFC PATCH 1/2] sched: Rate limit migrations to 1 per 2ms per task

[Mathieu Desnoyers <mathieu.desnoyers@efficios.com>]

On 9/6/23 16:51, Tim Chen wrote:
> On Wed, 2023-09-06 at 11:47 +0200, Peter Zijlstra wrote:
>> On Tue, Sep 05, 2023 at 03:44:57PM -0700, Tim Chen wrote:
>>
>>> Reading up on sched_clock() documentation and seems like it should
>>> indeed be monotonic.
>>
>> It tries very hard to be monotonic but cannot guarantee. The moment TSC
>> is found unstable it's too late to fix up everything.
>>
> 
> Yes, if TSC becomes unstable and could cause sched_clock to reset and go way backward.
> Perhaps we can add the following check in Mathieu's original
> patch to fix things up:
> 
> +static bool should_migrate_task(struct task_struct *p, int prev_cpu)
>> +{
> 	/* sched_clock reset causing next migration time to be too far ahead */
> 	if (p->se.next_migration_time > sched_clock_cpu(prev_cpu) + SCHED_MIGRATION_RATELIMIT_WINDOW)
> 		p->se.next_migration_time = sched_clock_cpu(prev_cpu) + SCHED_MIGRATION_RATELIMIT_WINDOW;
> 
>> +	/* Rate limit task migration. */
>> +	if (sched_clock_cpu(prev_cpu) < p->se.next_migration_time)
>> +	       return false;
>> +	return true;
>> +}
>> +
> 

Along those lines I think something like this should work:

static bool should_migrate_task(struct task_struct *p, int prev_cpu)
{
         u64 now = sched_clock_cpu(prev_cpu);

         /* sched_clock reset causing next migration time to be too far ahead. */
         if (now + SCHED_MIGRATION_RATELIMIT_WINDOW < p->se.next_migration_time)
                 return true;
         /* Rate limit task migration. */
         if (now >= p->se.next_migration_time)
                return true;
         return false;
}

It will let migrate_task_rq_fair() update se->next_migration_time.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com