Question about nohz and sysidle

Question about nohz and sysidle

[Yu Liao]

Hi,

I'm working on an issue about nohz. When NO_HZ_FULL is enabled, CPU 0 
handles the timekeeping duty on behalf of all other CPUs, which means 
CPU 0 never stop tick even in sysidle state. This is a powersaving issue.

I found Frederic's patchset (nohz: Support sysidle) in the below link.
https://lore.kernel.org/all/1406569056-30217-1-git-send-email-fweisbec@gmail.com/
But these patches haven't been merged into mainline yet and sysidle 
state detection has been removed by commit fe5ac724d81a (rcu: Remove 
nohz_full full-system-idle state machine) as well.

I tried Frederic's patches and it does work, why are we no longer 
working on stopping timekeeping duty when all full dynticks CPUs are idle?

Thanks,

Yu

Re: Question about nohz and sysidle

[Yu Liao]

Sorry, I sent to the old email address.

On 2022/2/14 16:28, Yu Liao wrote:
> Hi Frederic,
> 
> I'm working on an issue about nohz. When NO_HZ_FULL is enabled, CPU 0 
> handles the timekeeping duty on behalf of all other CPUs, which means 
> CPU 0 never stop tick even in sysidle state. This is a powersaving issue.
> 
> I found your patchset (nohz: Support sysidle) in the below link.
> https://lore.kernel.org/all/1406569056-30217-1-git-send-email-fweisbec@gmail.com/ 
> 
> But these patches haven't been merged into mainline yet and sysidle 
> state detection has been removed by commit fe5ac724d81a (rcu: Remove 
> nohz_full full-system-idle state machine) as well.
> 
> I tried your patches and it does work, why are we no longer 
> working on stopping timekeeping duty when all full dynticks CPUs are idle?
> 
> Thanks,
> Yu

Re: Question about nohz and sysidle

[Frederic Weisbecker]

On Mon, Feb 14, 2022 at 05:40:55PM +0800, Yu Liao wrote:

Hi Yu Liao,

> 
> On 2022/2/14 16:28, Yu Liao wrote:
> > Hi Frederic,
> > 
> > I'm working on an issue about nohz. When NO_HZ_FULL is enabled, CPU 0
> > handles the timekeeping duty on behalf of all other CPUs, which means
> > CPU 0 never stop tick even in sysidle state. This is a powersaving
> > issue.
> > 
> > I found your patchset (nohz: Support sysidle) in the below link.
> > https://lore.kernel.org/all/1406569056-30217-1-git-send-email-fweisbec@gmail.com/
> > 
> > But these patches haven't been merged into mainline yet and sysidle
> > state detection has been removed by commit fe5ac724d81a (rcu: Remove
> > nohz_full full-system-idle state machine) as well.
> > 
> > I tried your patches and it does work, why are we no longer working on
> > stopping timekeeping duty when all full dynticks CPUs are idle?

Because it was not a priority at that time. There were so many things to handle
first (and we are not even done yet) that we postponed that feature until
someone ever comes up with powersaving issues on nohz_full. We were waiting for
you :)

It's possible to unearth this. I think the first step will be to merge the
RCU dynticks counters into context tracking, something that was on my queue
anyway, and then revive this:

	https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/commit/?h=sysidle.2017.05.11a&id=fe5ac724d81a3c7803e60c2232718f212f3f38d4

Thanks.

Re: Question about nohz and sysidle

[Paul E. McKenney]

On Mon, Feb 14, 2022 at 11:57:44AM +0100, Frederic Weisbecker wrote:
> On Mon, Feb 14, 2022 at 05:40:55PM +0800, Yu Liao wrote:
> 
> Hi Yu Liao,
> 
> > 
> > On 2022/2/14 16:28, Yu Liao wrote:
> > > Hi Frederic,
> > > 
> > > I'm working on an issue about nohz. When NO_HZ_FULL is enabled, CPU 0
> > > handles the timekeeping duty on behalf of all other CPUs, which means
> > > CPU 0 never stop tick even in sysidle state. This is a powersaving
> > > issue.
> > > 
> > > I found your patchset (nohz: Support sysidle) in the below link.
> > > https://lore.kernel.org/all/1406569056-30217-1-git-send-email-fweisbec@gmail.com/
> > > 
> > > But these patches haven't been merged into mainline yet and sysidle
> > > state detection has been removed by commit fe5ac724d81a (rcu: Remove
> > > nohz_full full-system-idle state machine) as well.
> > > 
> > > I tried your patches and it does work, why are we no longer working on
> > > stopping timekeeping duty when all full dynticks CPUs are idle?
> 
> Because it was not a priority at that time. There were so many things to handle
> first (and we are not even done yet) that we postponed that feature until
> someone ever comes up with powersaving issues on nohz_full. We were waiting for
> you :)
> 
> It's possible to unearth this. I think the first step will be to merge the
> RCU dynticks counters into context tracking, something that was on my queue
> anyway, and then revive this:
> 
> 	https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/commit/?h=sysidle.2017.05.11a&id=fe5ac724d81a3c7803e60c2232718f212f3f38d4

Given a valid use case, getting that functionality back would be fine
by me.  But yes, properly merging RCU dynticks into context tracking
would reduce the resulting idle-entry/exit performance penalty, so it
would be good to do that first.

							Thanx, Paul

Re: Question about nohz and sysidle

[Xiongfeng Wang]

Hi,

On 2022/2/14 22:49, Paul E. McKenney wrote:
> On Mon, Feb 14, 2022 at 11:57:44AM +0100, Frederic Weisbecker wrote:
>> On Mon, Feb 14, 2022 at 05:40:55PM +0800, Yu Liao wrote:
>>
>> Hi Yu Liao,
>>
>>>
>>> On 2022/2/14 16:28, Yu Liao wrote:
>>>> Hi Frederic,
>>>>
>>>> I'm working on an issue about nohz. When NO_HZ_FULL is enabled, CPU 0
>>>> handles the timekeeping duty on behalf of all other CPUs, which means
>>>> CPU 0 never stop tick even in sysidle state. This is a powersaving
>>>> issue.
>>>>
>>>> I found your patchset (nohz: Support sysidle) in the below link.
>>>> https://lore.kernel.org/all/1406569056-30217-1-git-send-email-fweisbec@gmail.com/
>>>>
>>>> But these patches haven't been merged into mainline yet and sysidle
>>>> state detection has been removed by commit fe5ac724d81a (rcu: Remove
>>>> nohz_full full-system-idle state machine) as well.
>>>>
>>>> I tried your patches and it does work, why are we no longer working on
>>>> stopping timekeeping duty when all full dynticks CPUs are idle?
>>
>> Because it was not a priority at that time. There were so many things to handle
>> first (and we are not even done yet) that we postponed that feature until
>> someone ever comes up with powersaving issues on nohz_full. We were waiting for
>> you :)
>>
>> It's possible to unearth this. I think the first step will be to merge the
>> RCU dynticks counters into context tracking, something that was on my queue
>> anyway, and then revive this:
>>
>> 	https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/commit/?h=sysidle.2017.05.11a&id=fe5ac724d81a3c7803e60c2232718f212f3f38d4
> 
> Given a valid use case, getting that functionality back would be fine
> by me.  But yes, properly merging RCU dynticks into context tracking
> would reduce the resulting idle-entry/exit performance penalty, so it
> would be good to do that first.

I have came across the same issue. The original email is as below.
 https://www.spinics.net/lists/kernel/msg4131382.html
But unfortunately, I also sent to the old email address.

In my situation, all the CPUs are in the same power domain. When nohz_full is
enabled, if one of the CPUs keep receiving tick interrupts, all the CPUs can not
be put into powerdown state, even though all other CPUs are idle.

Thanks,
Xiongfeng

> 
> 							Thanx, Paul
> .
>