From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756757AbYDYFss (ORCPT ); Fri, 25 Apr 2008 01:48:48 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1754690AbYDYFsi (ORCPT ); Fri, 25 Apr 2008 01:48:38 -0400 Received: from out02.mta.xmission.com ([166.70.13.232]:53089 "EHLO out02.mta.xmission.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753199AbYDYFsg (ORCPT ); Fri, 25 Apr 2008 01:48:36 -0400 From: ebiederm@xmission.com (Eric W. Biederman) To: Jeff Garzik Cc: Roland Dreier , Linus Torvalds , Rene Herman , Adrian Bunk , Andrew Morton , LKML , rmk@arm.linux.org.uk, Thomas Gleixner , Ingo Molnar References: <20080422221733.GA16260@havoc.gtf.org> <480E6DE5.6010103@garzik.org> <480E796E.4070001@garzik.org> <20080423000529.GG28933@cs181133002.pp.htv.fi> <480F3ECC.1090809@keyaccess.nl> <480FEC1B.6040102@garzik.org> <481066C1.1030805@garzik.org> <4810A9E6.1080201@garzik.org> <4810C3B5.3020605@garzik.org> <481150EE.3040103@garzik.org> <48115F90.1030100@garzik.org> Date: Thu, 24 Apr 2008 22:48:24 -0700 In-Reply-To: <48115F90.1030100@garzik.org> (Jeff Garzik's message of "Fri, 25 Apr 2008 00:35:28 -0400") Message-ID: User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/21.4 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-SA-Exim-Connect-IP: 24.130.11.59 X-SA-Exim-Mail-From: ebiederm@xmission.com X-Spam-DCC: XMission; sa03 1397; Body=1 Fuz1=1 Fuz2=1 X-Spam-Report: * -1.8 ALL_TRUSTED Passed through trusted hosts only via SMTP * 0.0 T_TM2_M_HEADER_IN_MSG BODY: T_TM2_M_HEADER_IN_MSG * -2.6 BAYES_00 BODY: Bayesian spam probability is 0 to 1% * [score: 0.0000] * -0.0 DCC_CHECK_NEGATIVE Not listed in DCC * [sa03 1397; Body=1 Fuz1=1 Fuz2=1] * 2.2 XMSubMetaSxObfu_04 Obfuscated Sexy Noun-Thing * 1.6 XMSubMetaSx_00 1+ Sexy Words * 0.0 XM_SPF_Neutral SPF-Neutral Subject: Re: MSI, fun for the whole family X-SA-Exim-Version: 4.2 (built Thu, 03 Mar 2005 10:44:12 +0100) X-SA-Exim-Scanned: Yes (on mgr1.xmission.com) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Jeff Garzik writes: > Roland Dreier wrote: >> I think you've fundamentally misunderstood what the PCI spec for MSI >> multi message means. It is true that if the whole system agrees, then > > You seem to be ignoring a key usage... > > >> an MSI-capable device that supports multiple messages might be allocated >> a range of vectors (MSI is kind of stupid because it only allows >> multiple messages to be generated by varying the low order bits -- MSI-X >> fixes this limitation). However, the way that these different messages >> are handled is that they are all independent interrupt vectors. >> >> Now, it is true that the kernel could do something crazy and collapse >> all these interrupt vectors into a single "IRQ" and then tell the >> interrupt handler which vector it was by passing some "metadata" in, but >> why not just give each MSI message it's own IRQ? > > The answer is: the driver might prefer to see the message as it arrived, rather > than dividing it up into independent vectors. The message itself is a unit of > data consistency, and there is value in letting the driver see the bounds of > that unit. On x86 the driver very much sees the message as it arrived at the cpu. If you want the message as it was sent we would need to deduce it from which cpu we are on and which interrupt descriptors function was called. As for the bounds of data consistency currently no information is lost. > As it stands now, we only a spray of $N function calls for each message, with no > notion of "we started processing this set of messages" and "we ended > processing[...]" Hopefully my other messages explains it better but there is no spray of function calls. Nor can there be. There is not a bitmask in your MSI message there is instead a base 2 number encoding which port had traffic. You don't have 32 bits but 5. > Don't assume that the way Linux supports this stuff today is the best, or the > only way to do things. It's not "collapsing all these interrupt vectors" -- > remember that an expansion occurred, and /avoiding expansion/ into multiple > vectors for multiple messages may be an optimal path for a specific driver > application. If we were receiving a bitmask I would find this more persuasive. To implement what you suggest currently feels like a lot of work digging through each architectures implementation of MSI and seeing if it possible and maintainable on each architecture, and refactoring a lot of the interrupt handling infrastructure to get there. All to support hardware vendors that were too lazy to either implement MSI-X or implement a DMAing a status value to a known spot before sending the MSI. I don't see it being worth it at this time. Eric