Re: Reworking suspend-resume sequence (was: Re: PCI PM: Restore standard config registers of all devices early)

[Ingo Molnar <mingo@elte.hu>]

* Ingo Molnar <mingo@elte.hu> wrote:

[...]
> That principle works both for networking and for other IO transports - but 
> we have little support for it yet. It would work really well for workloads 
> where one physical device is shared by many CPUs.

Which is really what we have in _practice_ - only benchmarketing sets up one 
physical device per CPU and avoids all the ugly consequences of 
gathering/spreadig a channel of information from/to a single device to 
multiple CPUs.

> (A lesser method that approximates this is the use of lots of 
> submission/completion rings per device and their binding to cpus - but 
> that can never really approach the number of CPUs really possible in a 
> system.)

btw., more advanced device IRQ models was one of the thinking behind sparse 
IRQ support: defining a really large NR_IRQS limit on x86 by default, on all 
form factors, and making it really easy and cheap to have a _ton_ of IRQs in 
a Linux system might give hw designers ideas to create such hardware.

/me dreams on ;-)

> And in this most advanced mode of MSI IRQs, and if MSI devices had the 
> ability to direct IRQs to a specific CPU (they dont have that right now 
> AFAICT), we'd run into the overhead scenarios you describe above, and your 
> edge-triggered flow is the most performant one.

So i'd still like your tentative Signed-off-by for your patch - it's i think 
not v2.6.29 material but if it stays problem free in testing we can try it 
in v2.6.30. If it causes problem it will be clearly bisectable and clearly 
revertable.

Maybe we could split it in two: and for MSI we could introduce a 'simpler 
and faster' edge flow as well - and keep the legacy handler untouched. That 
way it's low-risk in its entirety. (and avoids the MSI ->mask complication 
as well.)

	Ingo