[Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Ming Lei]

Hi,

Looks like it is a bit difficult to understand PM_QOS_CPU_DMA_LATENCY
from the words' meaning.

After searching from google, I don't find some useful information about
the root cause for introducing PM_QOS_CPU_DMA_LATENCY. I understand
it is very similar to interrupt latency. Also from the comment for
omap_pm_set_max_mpu_wakeup_lat in file[1], the description is basically same
with interrupt latency.

>From comments of pm_qos_add_request usages in drivers, it can be understood
as interrupt latency too, IMO.

So, could we think that PM_QOS_CPU_DMA_LATENCY is interrupt latency?


[1], arch/arm/plat-omap/include/plat/omap-pm.h

thanks,
-- 
Ming Lei

Re: [Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Alan Cox]

On Mon, 10 Oct 2011 22:31:34 +0800
Ming Lei <tom.leiming@gmail.com> wrote:

> Hi,
> 
> Looks like it is a bit difficult to understand PM_QOS_CPU_DMA_LATENCY
> from the words' meaning.
> 
> After searching from google, I don't find some useful information about
> the root cause for introducing PM_QOS_CPU_DMA_LATENCY. I understand
> it is very similar to interrupt latency. Also from the comment for
> omap_pm_set_max_mpu_wakeup_lat in file[1], the description is basically same
> with interrupt latency.
> 
> >From comments of pm_qos_add_request usages in drivers, it can be understood
> as interrupt latency too, IMO.
> 
> So, could we think that PM_QOS_CPU_DMA_LATENCY is interrupt latency?

No. Well it may be on some platforms but it isn't the same thing. On some
devices a DMA transfer doesn't need the CPU involved but needs the CPU to
respond within a set timescale (eg for coherency or bus arbitration). It
is not the same thing as IRQ latency.

Re: [Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Ming Lei]

Hi,

On Tue, Oct 11, 2011 at 12:25 AM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> On Mon, 10 Oct 2011 22:31:34 +0800
> Ming Lei <tom.leiming@gmail.com> wrote:
>
>> Hi,
>>
>> Looks like it is a bit difficult to understand PM_QOS_CPU_DMA_LATENCY
>> from the words' meaning.
>>
>> After searching from google, I don't find some useful information about
>> the root cause for introducing PM_QOS_CPU_DMA_LATENCY. I understand
>> it is very similar to interrupt latency. Also from the comment for
>> omap_pm_set_max_mpu_wakeup_lat in file[1], the description is basically same
>> with interrupt latency.
>>
>> >From comments of pm_qos_add_request usages in drivers, it can be understood
>> as interrupt latency too, IMO.
>>
>> So, could we think that PM_QOS_CPU_DMA_LATENCY is interrupt latency?
>
> No. Well it may be on some platforms but it isn't the same thing. On some
> devices a DMA transfer doesn't need the CPU involved but needs the CPU to
> respond within a set timescale (eg for coherency or bus arbitration). It

I understand only the CPU can respond after it is notified by a
interrupt event,
don't I?

Also could you give a example about how the CPU responds to a DMA transfer
within a set timescale if it is required?

> is not the same thing as IRQ latency.


thanks,
-- 
Ming Lei

Re: [Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Shaohua Li]

2011/10/11 Ming Lei <tom.leiming@gmail.com>:
> Hi,
>
> On Tue, Oct 11, 2011 at 12:25 AM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> On Mon, 10 Oct 2011 22:31:34 +0800
>> Ming Lei <tom.leiming@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Looks like it is a bit difficult to understand PM_QOS_CPU_DMA_LATENCY
>>> from the words' meaning.
>>>
>>> After searching from google, I don't find some useful information about
>>> the root cause for introducing PM_QOS_CPU_DMA_LATENCY. I understand
>>> it is very similar to interrupt latency. Also from the comment for
>>> omap_pm_set_max_mpu_wakeup_lat in file[1], the description is basically same
>>> with interrupt latency.
>>>
>>> >From comments of pm_qos_add_request usages in drivers, it can be understood
>>> as interrupt latency too, IMO.
>>>
>>> So, could we think that PM_QOS_CPU_DMA_LATENCY is interrupt latency?
>>
>> No. Well it may be on some platforms but it isn't the same thing. On some
>> devices a DMA transfer doesn't need the CPU involved but needs the CPU to
>> respond within a set timescale (eg for coherency or bus arbitration). It
>
> I understand only the CPU can respond after it is notified by a
> interrupt event,
> don't I?
As Alan explained, PM_QOS_CPU_DMA_LATENCY is for dma snooping. For example,
in x86, cpu snoop dma. when cpu is in idle state, cpu need snoop
device dma activity, there
is latency involved for idle state.

Re: [Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Ming Lei]

Hi,

On Tue, Oct 11, 2011 at 9:49 AM, Shaohua Li <shli@kernel.org> wrote:

> As Alan explained, PM_QOS_CPU_DMA_LATENCY is for dma snooping. For example,
> in x86, cpu snoop dma. when cpu is in idle state, cpu need snoop
> device dma activity, there
> is latency involved for idle state.
>

I see, thanks for your clarification.

I also have two further questions about it:

- Except for dma snooping purpose, are there any other cases in which
PM_QOS_CPU_DMA_LATENCY is required?

- Are all CPUs required to be involved to dma snoop? Or only one CPU
is enough? If one is enough, maybe we can allow other CPUs to reach
deeper idle state.


thanks,
-- 
Ming Lei

Re: [Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Shaohua Li]

2011/10/11 Ming Lei <tom.leiming@gmail.com>:
> Hi,
>
> On Tue, Oct 11, 2011 at 9:49 AM, Shaohua Li <shli@kernel.org> wrote:
>
>> As Alan explained, PM_QOS_CPU_DMA_LATENCY is for dma snooping. For example,
>> in x86, cpu snoop dma. when cpu is in idle state, cpu need snoop
>> device dma activity, there
>> is latency involved for idle state.
>>
>
> I see, thanks for your clarification.
>
> I also have two further questions about it:
>
> - Except for dma snooping purpose, are there any other cases in which
> PM_QOS_CPU_DMA_LATENCY is required?
it's the main motivation, IIRC, don't know other platforms

> - Are all CPUs required to be involved to dma snoop? Or only one CPU
> is enough? If one is enough, maybe we can allow other CPUs to reach
> deeper idle state.
then how can you make cache coherency between the cpus?

Re: [Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Ming Lei]

Hi,

On Tue, Oct 11, 2011 at 10:26 AM, Shaohua Li <shli@kernel.org> wrote:
> 2011/10/11 Ming Lei <tom.leiming@gmail.com>:
>> Hi,
>>
>> On Tue, Oct 11, 2011 at 9:49 AM, Shaohua Li <shli@kernel.org> wrote:
>>
>>> As Alan explained, PM_QOS_CPU_DMA_LATENCY is for dma snooping. For example,
>>> in x86, cpu snoop dma. when cpu is in idle state, cpu need snoop
>>> device dma activity, there
>>> is latency involved for idle state.
>>>
>>
>> I see, thanks for your clarification.
>>
>> I also have two further questions about it:
>>
>> - Except for dma snooping purpose, are there any other cases in which
>> PM_QOS_CPU_DMA_LATENCY is required?
> it's the main motivation, IIRC, don't know other platforms

If so, maybe all device drivers which support DMA transfer should
have used PM_QOS_CPU_DMA_LATENCY, but why only few drivers
did it?

>
>> - Are all CPUs required to be involved to dma snoop? Or only one CPU
>> is enough? If one is enough, maybe we can allow other CPUs to reach
>> deeper idle state.
> then how can you make cache coherency between the cpus?
>

Seems ARM supports cache maintenance operations from software[1],
but I don't know how to do it on x86, :-)


[1], http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dai0228a/index.html


thanks,
-- 
Ming Lei

Re: [Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Shaohua Li]

2011/10/11 Ming Lei <tom.leiming@gmail.com>:
> Hi,
>
> On Tue, Oct 11, 2011 at 10:26 AM, Shaohua Li <shli@kernel.org> wrote:
>> 2011/10/11 Ming Lei <tom.leiming@gmail.com>:
>>> Hi,
>>>
>>> On Tue, Oct 11, 2011 at 9:49 AM, Shaohua Li <shli@kernel.org> wrote:
>>>
>>>> As Alan explained, PM_QOS_CPU_DMA_LATENCY is for dma snooping. For example,
>>>> in x86, cpu snoop dma. when cpu is in idle state, cpu need snoop
>>>> device dma activity, there
>>>> is latency involved for idle state.
>>>>
>>>
>>> I see, thanks for your clarification.
>>>
>>> I also have two further questions about it:
>>>
>>> - Except for dma snooping purpose, are there any other cases in which
>>> PM_QOS_CPU_DMA_LATENCY is required?
>> it's the main motivation, IIRC, don't know other platforms
>
> If so, maybe all device drivers which support DMA transfer should
> have used PM_QOS_CPU_DMA_LATENCY, but why only few drivers
> did it?
depends on your device. Say cpu takes 1s to snoop dma in idle state, so device
dma will cost more than 1s. if your device can work with such latency, no
problem then.

Re: [Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Ming Lei]

Hi,

On Tue, Oct 11, 2011 at 1:32 PM, Shaohua Li <shli@kernel.org> wrote:
> 2011/10/11 Ming Lei <tom.leiming@gmail.com>:
>> Hi,
>>
>> On Tue, Oct 11, 2011 at 10:26 AM, Shaohua Li <shli@kernel.org> wrote:
>>> 2011/10/11 Ming Lei <tom.leiming@gmail.com>:
>>>> Hi,
>>>>
>>>> On Tue, Oct 11, 2011 at 9:49 AM, Shaohua Li <shli@kernel.org> wrote:
>>>>
>>>>> As Alan explained, PM_QOS_CPU_DMA_LATENCY is for dma snooping. For example,
>>>>> in x86, cpu snoop dma. when cpu is in idle state, cpu need snoop
>>>>> device dma activity, there
>>>>> is latency involved for idle state.
>>>>>
>>>>
>>>> I see, thanks for your clarification.
>>>>
>>>> I also have two further questions about it:
>>>>
>>>> - Except for dma snooping purpose, are there any other cases in which
>>>> PM_QOS_CPU_DMA_LATENCY is required?
>>> it's the main motivation, IIRC, don't know other platforms
>>
>> If so, maybe all device drivers which support DMA transfer should
>> have used PM_QOS_CPU_DMA_LATENCY, but why only few drivers
>> did it?
> depends on your device. Say cpu takes 1s to snoop dma in idle state, so device
> dma will cost more than 1s. if your device can work with such latency, no
> problem then.
>

Sorry, I am still a bit confused.

As you said before, for x86, some deep cpu idle states may stop dma
snoop, so introduce PM_QOS_CPU_DMA_LATENCY to prevent cpuidle from
entering such kind of deep idle state if I/O dma is in progress.  I
understand the cpu dma pm qos should always be applied before starting
any dma transfer, Otherwise, cpu local cache may not be updated after
dma is over when cpu is returned from the deep idle state.

If so, the latency is only related with the exit latency of cpu idle
state in which dma snoop is stopped, instead of being related with I/O
devices.

Please correct me if the above is wrong.


thanks,
-- 
Ming Lei

Re: [Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Alan Cox]

> > No. Well it may be on some platforms but it isn't the same thing. On some
> > devices a DMA transfer doesn't need the CPU involved but needs the CPU to
> > respond within a set timescale (eg for coherency or bus arbitration). It
> 
> I understand only the CPU can respond after it is notified by a
> interrupt event, don't I?

The instruction stream being executed maybe, but not things like the cache

> Also could you give a example about how the CPU responds to a DMA transfer
> within a set timescale if it is required?

The kind of thing you are dealing with is

	DMA engine requests a cache line of data
	CPU wakes out of sleep, completes bus transaction
	CPU goes back to sleep
	DMA engine starts outputting data bits over SPI bus or similar

	repeat until done

so it's not instruction level stuff, merely bus traffic.

Re: [Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Ming Lei]

On Tue, Oct 11, 2011 at 6:07 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> > No. Well it may be on some platforms but it isn't the same thing. On some
>> > devices a DMA transfer doesn't need the CPU involved but needs the CPU to
>> > respond within a set timescale (eg for coherency or bus arbitration). It
>>
>> I understand only the CPU can respond after it is notified by a
>> interrupt event, don't I?
>
> The instruction stream being executed maybe, but not things like the cache
>
>> Also could you give a example about how the CPU responds to a DMA transfer
>> within a set timescale if it is required?
>
> The kind of thing you are dealing with is
>
>        DMA engine requests a cache line of data
>        CPU wakes out of sleep, completes bus transaction

I think the CPU should be woken up by interrupt from DMA engine, so
it is still a kind of interrupt latency?

Also looks like it is a bit odd that why CPU is involved to complete the
bus transaction which should have been done by DMA engine only.
Is there a practical example about this?

>        CPU goes back to sleep
>        DMA engine starts outputting data bits over SPI bus or similar
>
>        repeat until done
>
> so it's not instruction level stuff, merely bus traffic.
>


thanks,
-- 
Ming Lei

Re: [Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[mark gross]

On Mon, Oct 10, 2011 at 10:31:34PM +0800, Ming Lei wrote:
> Hi,
> 
> Looks like it is a bit difficult to understand PM_QOS_CPU_DMA_LATENCY
> from the words' meaning.
> 
> After searching from google, I don't find some useful information about
> the root cause for introducing PM_QOS_CPU_DMA_LATENCY. I understand
> it is very similar to interrupt latency. Also from the comment for
> omap_pm_set_max_mpu_wakeup_lat in file[1], the description is basically same
> with interrupt latency.

its the amount of time the CPU can take to wake up and feed a DMA
engine.  Its a bit more general than interrupt latency.

> From comments of pm_qos_add_request usages in drivers, it can be understood
> as interrupt latency too, IMO.

The original 2 issues that drove its predecessor implementation where
DMA'ing data to the original intel wifi device.  If the CPU snoozed too
long the wifi device would get lost and network would suffer.

The other one was something to do with generating audio pops if the cpu
didn't keep some audio buffer that was getting DMA'd to something from
underflowing.


> 
> So, could we think that PM_QOS_CPU_DMA_LATENCY is interrupt latency?
nope.

--mark
> 
> [1], arch/arm/plat-omap/include/plat/omap-pm.h
> 
> thanks,
> -- 
> Ming Lei
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Question] PM-QoS: PM_QOS_CPU_DMA_LATENCY == interrupt latency?

[Ming Lei]

Hi,

On Thu, Oct 13, 2011 at 12:06 PM, mark gross <markgross@thegnar.org> wrote:
> On Mon, Oct 10, 2011 at 10:31:34PM +0800, Ming Lei wrote:
>> Hi,
>>
>> Looks like it is a bit difficult to understand PM_QOS_CPU_DMA_LATENCY
>> from the words' meaning.
>>
>> After searching from google, I don't find some useful information about
>> the root cause for introducing PM_QOS_CPU_DMA_LATENCY. I understand
>> it is very similar to interrupt latency. Also from the comment for
>> omap_pm_set_max_mpu_wakeup_lat in file[1], the description is basically same
>> with interrupt latency.
>
> its the amount of time the CPU can take to wake up and feed a DMA
> engine.  Its a bit more general than interrupt latency.

Maybe interrupt or wake up latency is more general than cpu dma
latency, since interrupt/wake up latency should include this kind of
DMA case.

But the latency name does not matter, what maters is below idea.

>> From comments of pm_qos_add_request usages in drivers, it can be understood
>> as interrupt latency too, IMO.
>
> The original 2 issues that drove its predecessor implementation where
> DMA'ing data to the original intel wifi device.  If the CPU snoozed too
> long the wifi device would get lost and network would suffer.

If the original 2 issues are only related slow cpu snooping in deep idle
state, maybe it is enough to keep one cpu(suppose CPU0) to obey the
constraint, then only let CPU0 handle IRQ from the devices, so the
other CPUs may be allowed to enter deep idle state.

>
> The other one was something to do with generating audio pops if the cpu
> didn't keep some audio buffer that was getting DMA'd to something from
> underflowing.

>From the old comment for audio case[1], we can see the below:

- * An example announcer of latency is an audio driver that knowns it
- * will get an interrupt when the hardware has 200 usec of samples
- * left in the DMA buffer; in that case the driver can set a latency
- * constraint of, say, 150 usec.

so the audio case may be a wake up or interrupt latency, and maybe
it is possible to apply the constraint on only one CPU which will be
responsible for handling audio irq.

Anyway, it is just a rough idea for discussion. If it is doable, I think
it should make sense because multiple core CPU is very popular
in current application from server, laptop, desktop to tablet.

>
>>
>> So, could we think that PM_QOS_CPU_DMA_LATENCY is interrupt latency?
> nope.

[1], commit f011e2e2df3393c16b0fdc48e855e909b7e021ee
Author: Mark Gross <mgross@linux.intel.com>
Date:   Mon Feb 4 22:30:09 2008 -0800

    latency.c: use QoS infrastructure



thanks,
-- 
Ming Lei