[PATCH] nohz: fix jiffies warp while switch to low-res nohz mode

[PATCH] nohz: fix jiffies warp while switch to low-res nohz mode

[Wanpeng Li]

After commit 0ff53d096422 ("tick: sched: Force tick interrupt and get rid 
of softirq magic"), next expire time is overwritten by last_jiffies_update 
while switch to low-res nohz mode which leads to jiffies warp. This patch 
fix it by programming ce device the expire time in the future.

Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
---
 kernel/time/tick-sched.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 9cc20af..cea6a0e 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -993,9 +993,9 @@ static void tick_nohz_switch_to_nohz(void)
 	/* Get the next period */
 	next = tick_init_jiffy_update();
 
-	hrtimer_forward_now(&ts->sched_timer, tick_period);
 	hrtimer_set_expires(&ts->sched_timer, next);
-	tick_program_event(next, 1);
+	hrtimer_forward_now(&ts->sched_timer, tick_period);
+	tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1);
 	tick_nohz_activate(ts, NOHZ_MODE_LOWRES);
 }
 
-- 
1.9.1

Re: [PATCH] nohz: fix jiffies warp while switch to low-res nohz mode

[Thomas Gleixner]

On Thu, 21 Jan 2016, Wanpeng Li wrote:

> After commit 0ff53d096422 ("tick: sched: Force tick interrupt and get rid 
> of softirq magic"), next expire time is overwritten by last_jiffies_update 
> while switch to low-res nohz mode which leads to jiffies warp. This patch 
> fix it by programming ce device the expire time in the future.

> -	hrtimer_forward_now(&ts->sched_timer, tick_period);
>  	hrtimer_set_expires(&ts->sched_timer, next);
> -	tick_program_event(next, 1);
> +	hrtimer_forward_now(&ts->sched_timer, tick_period);
> +	tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1);
>  	tick_nohz_activate(ts, NOHZ_MODE_LOWRES);

I agree, that the ordering forward/set is wrong and needs to be fixed, but how
does that result in a jiffies warp?

tick_nohz_handler()
  tick_sched_do_timer()
    tick_do_update_jiffies64()
        delta = ktime_sub(now, last_jiffies_update);
        if (delta.tv64 < tick_period.tv64)
                return;

So if that timer fires early, tick_do_update_jiffies64() will just return and
do nothing. So what makes jiffies warp?

Unless I'm missing something, this changelog is patently wrong.

Thanks,

	tglx

Re: [PATCH] nohz: fix jiffies warp while switch to low-res nohz mode

[Wanpeng Li]

2016-01-26 23:21 GMT+08:00 Thomas Gleixner <tglx@linutronix.de>:
> On Thu, 21 Jan 2016, Wanpeng Li wrote:
>
>> After commit 0ff53d096422 ("tick: sched: Force tick interrupt and get rid
>> of softirq magic"), next expire time is overwritten by last_jiffies_update
>> while switch to low-res nohz mode which leads to jiffies warp. This patch
>> fix it by programming ce device the expire time in the future.
>
>> -     hrtimer_forward_now(&ts->sched_timer, tick_period);
>>       hrtimer_set_expires(&ts->sched_timer, next);
>> -     tick_program_event(next, 1);
>> +     hrtimer_forward_now(&ts->sched_timer, tick_period);
>> +     tick_program_event(hrtimer_get_expires(&ts->sched_timer), 1);
>>       tick_nohz_activate(ts, NOHZ_MODE_LOWRES);
>
> I agree, that the ordering forward/set is wrong and needs to be fixed, but how
> does that result in a jiffies warp?
>
> tick_nohz_handler()
>   tick_sched_do_timer()
>     tick_do_update_jiffies64()
>         delta = ktime_sub(now, last_jiffies_update);
>         if (delta.tv64 < tick_period.tv64)
>                 return;

Yeah, I know this catch it and that's why no real warp happen, just send out v2,
thanks for your review, Thomas. :-)

Regards,
Wanpeng Li