Re: [External] Re: [PATCH v4] sched/core: Adapt WARN_DOUBLE_CLOCK machinery for core-sched

[Hao Jia <jiahao.os@bytedance.com>]

On 2023/5/4 Peter Zijlstra wrote:
> On Thu, Apr 06, 2023 at 02:44:15PM +0800, Hao Jia wrote:
>> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
>> index 0d18c3969f90..c6e2c79152ef 100644
>> --- a/kernel/sched/core.c
>> +++ b/kernel/sched/core.c
>> @@ -429,11 +429,32 @@ void sched_core_put(void)
>>   		schedule_work(&_work);
>>   }
>>   
>> +/*
>> + * Now, we have obtained a core-wide rq->lock, then we need to clear
>> + * RQCF_UPDATED of rq->clock_update_flags of the sibiling CPU
>> + * on this core to avoid the WARN_DOUBLE_CLOCK warning.
>> + */
>> +static inline void sched_core_clear_rqcf_updated(struct rq *rq)
>> +{
>> +#ifdef CONFIG_SCHED_DEBUG
>> +	const struct cpumask *smt_mask;
>> +	int i;
>> +
>> +	if (rq->core_enabled) {
>> +		smt_mask = cpu_smt_mask(rq->cpu);
>> +		for_each_cpu(i, smt_mask) {
>> +			if (rq->cpu != i)
>> +				cpu_rq(i)->clock_update_flags &= (RQCF_REQ_SKIP|RQCF_ACT_SKIP);
>> +		}
>> +	}
>> +#endif
>> +}
>>   #else /* !CONFIG_SCHED_CORE */
>>   
>>   static inline void sched_core_enqueue(struct rq *rq, struct task_struct *p) { }
>>   static inline void
>>   sched_core_dequeue(struct rq *rq, struct task_struct *p, int flags) { }
>> +static inline void sched_core_clear_rqcf_updated(struct rq *rq) { }
>>   
>>   #endif /* CONFIG_SCHED_CORE */
>>   
>> @@ -548,6 +569,7 @@ void raw_spin_rq_lock_nested(struct rq *rq, int subclass)
>>   		if (likely(lock == __rq_lockp(rq))) {
>>   			/* preempt_count *MUST* be > 1 */
>>   			preempt_enable_no_resched();
>> +			sched_core_clear_rqcf_updated(rq);
>>   			return;
>>   		}
>>   		raw_spin_unlock(lock);
> 
> This still looks absolutely wrong. The whole RQCF thing is a pin action.

Do you think it is better for us to extend rq_pin_lock() to clean RQCF 
updated than to do it in raw_spin_rq_lock_nested()?

Before doing this, we need to solve the situation where rq_pin_lock() 
and raw_spin_rq_lock() are used separately.

Any suggestion will be very helpful for me.

Thanks,
Hao