From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754507AbcFTNhq (ORCPT ); Mon, 20 Jun 2016 09:37:46 -0400 Received: from mail-bn1on0087.outbound.protection.outlook.com ([157.56.110.87]:35961 "EHLO na01-bn1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751865AbcFTNha (ORCPT ); Mon, 20 Jun 2016 09:37:30 -0400 Authentication-Results: spf=pass (sender IP is 63.163.107.21) smtp.mailfrom=sandisk.com; fb.com; dkim=none (message not signed) header.d=none;fb.com; dmarc=bestguesspass action=none header.from=sandisk.com; X-AuditID: ac160c69-33fff70000000bcf-5f-5767ecbdf72b Subject: Re: [PATCH 02/13] irq: Introduce IRQD_AFFINITY_MANAGED flag To: Christoph Hellwig References: <1465934346-20648-3-git-send-email-hch@lst.de> <0412b942-ea0d-d4eb-c724-8243d12ff6f3@sandisk.com> <20160615102311.GA16619@lst.de> <67ef7a1c-56e1-db2c-b038-f9784fc1f52f@sandisk.com> <20160615151415.GA1919@localhost.localdomain> <7f0b16bd-b39f-99e6-c1c1-6a508bf9bbbf@sandisk.com> <20160615160316.GB1919@localhost.localdomain> <86aa652b-48d0-a7bb-683e-bf43939aa811@sandisk.com> <20160616152006.GA13615@lst.de> <20160620122219.GA29059@lst.de> CC: Keith Busch , "tglx@linutronix.de" , "axboe@fb.com" , "linux-block@vger.kernel.org" , "linux-pci@vger.kernel.org" , "linux-nvme@lists.infradead.org" , "linux-kernel@vger.kernel.org" From: Bart Van Assche Message-ID: <066e6b9d-34a8-5917-4ad8-cb466d3389c3@sandisk.com> Date: Mon, 20 Jun 2016 15:21:47 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160620122219.GA29059@lst.de> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupmkeLIzCtJLcpLzFFi42JZI8azSPfgm/Rwg52X2Cz+7znGZrFy9VEm i3tHvzBZ7L2lbXF51xw2i/nLnrJbnJ13nM1i86apzA4cHhOb37F7LN7zksnj3blz7B6bl9R7 7L7ZwObxeZNcAFsUl01Kak5mWWqRvl0CV8b9dStYC67wVsxfv5elgfEPVxcjJ4eEgInEvrtP mboYuTiEBOYySVz4tZkdwtnBKHH8xBommKoDfc9YIBJLGSX+n53NDpIQFnCRaLz2lRnEFhFQ knj66iwjRFEni8S+7fdYQRxmgc9MEj927gPrYBMwkvj2fiYLiM0rYCex4kMTWJxFQFXi5j+I SaICERK9v5ZA1QhKnJz5BMzmFNCRePihDaiGA2iovcSDrWUgYWYBeYntb+cwg+ySELjKKjHp 6lQ2kISQgLrEySXzmSYwCs9CMmoWQvssJO0LGJlXMYrlZuYU56anFhia6hUn5qVkFmfrJefn bmKERFHmDsa7T7wPMQpwMCrx8FqcTQsXYk0sK67MPcQowcGsJMJ7FBiDQrwpiZVVqUX58UWl OanFhxilOViUxHnd7n0JExJITyxJzU5NLUgtgskycXBKNTBWBp5LnPnN/CtThvlU76LFaVfU eSvKmC+s+7JrRdtBve9BwvsW5n/2OJC39vic7uYErYKSynbxD+svpazKifj1K2mWteirpeUi Du03n8xpNq67aHrxneYe89esubcLAywu1G841+GStyB5Hv/Fe01HtBp3rNKMrZ3A5e7xY0Hn 47+inF2bJ3sosRRnJBpqMRcVJwIALmuqaZ4CAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmpiluLIzCtJLcpLzFFi42Lh2siRovvxbXq4weOF/Bb/9xxjs1i5+iiT xb2jX5gs9t7Stri8aw6bxfxlT9ktzs47zmaxedNUZgcOj4nN79g9Fu95yeTx7tw5do/NS+o9 dt9sYPP4vEkugC2KyyYlNSezLLVI3y6BK+P+uhWsBVd4K+av38vSwPiHq4uRk0NCwETiQN8z li5GLg4hgcWMEn0XzrCCJIQFXCQar31lBrFFBJQknr46ywhR1MkisW/7PVYQh1ngM5NE//NW RpAqNgEjiW/vZ7KA2LwCdhIrPjSxg9gsAqoSN/9BTBIViJDo/bUEqkZQ4uTMJ2A2p4COxMMP bWA1zAK2Enfm7oay5SW2v53DPIGRbxaSlllIymYhKVvAyLyKUSw3M6c4Nz2zwNBQrzgxLyWz OFsvOT93EyM4lDkjdzA+nWh+iJGJg1OqgVHh0YOqclvRjwpLjl1xr9L49q9Z6EXby8i3q23r j7t6tjPO5eAy37OxdWLjRjkvzWtbApNS8msuLHOcKGJyzm+68DGeyVPudfMXBmwtW9vTFl0o 3mW+giswfc/BnjZeQa9dbya+rHly3+vh9gVKG/l4RRg3+vf9PV0WzrT3aIrfafHVd6fM61Vi Kc5INNRiLipOBABUAsNEFQIAAA== X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:63.163.107.21;IPV:NLI;CTRY:US;EFV:NLI;SFV:NSPM;SFS:(10009020)(6009001)(7916002)(2980300002)(438002)(189002)(24454002)(377454003)(199003)(6806005)(110136002)(31696002)(76176999)(92566002)(4326007)(2906002)(54356999)(83506001)(47776003)(11100500001)(65806001)(106466001)(5003600100002)(77096005)(93886004)(97736004)(586003)(50466002)(36756003)(4001350100001)(65956001)(33646002)(8936002)(69596002)(189998001)(23746002)(81156014)(2950100001)(68736007)(356003)(64126003)(230700001)(2270400002)(53416004)(87936001)(50986999)(8676002)(7846002)(31686004)(81166006)(86362001)(65826006);DIR:OUT;SFP:1101;SCL:1;SRVR:BY2PR0201MB1767;H:milsmgep15.sandisk.com;FPR:;SPF:Pass;PTR:InfoDomainNonexistent;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: 1;BY2FFO11FD021;1:46wOak7GHiQCHriAxKsDuN6qahry6XelKrD+B45/IlSFJiFmcGcbM9xWTa5U/A+xL/GFiwEu3gnr+E18Z60CennmIxI5akA41ZORnHok2yVZbCqWXYB2oqrlSDZ0vpOANvMBfNJYLIvQTD3qrMnF36MABDHp0Ac39DyBXzLN5vAxg0h4EHv+KE+sLmdmCYpDf9kqpFwVnS3UELsvjoj5V16GtI7y/+0AaaKAWcCuiNE8uIwRJ/R2cYwftHjflDkFwh1PhlYvMkJCFHyGUdCUqgQ28OuAHcz1BlI1V7Uw9AuAJPdqISu2bEAGJqJ9ritCQbHd2KeN1dNgpS5lismhMczP/bDpMBeQ5QJtAKgEpbhcjI8FZNVGll5ruTPb2vVceGGso+OsccWPSDoMlZ9/FeWz+OvHCgc/iFBxBShSt/mpbxv9WhtcAPgmcaKOqWtPfnZwEvMTpa5dxpsHzfw2ArV/tvlZHJ9qnMIa8dUjQG00MKUoXyqAlRFZFWOBFdKU X-MS-Office365-Filtering-Correlation-Id: 0d23a4e1-0dcb-452a-69bf-08d3990ddad0 X-Microsoft-Exchange-Diagnostics: 1;BY2PR0201MB1767;2:ozvHL/x91BeCAKuNw1Q9US1iwAROV1qfHQro0/tuTWcHz71nhbqSfrsPgc2INRljxIz4MIkfdgQ/Nmra7sn2sgBnS6Bzd0U7Iuc0CQkM2UqynYZ8d80gZJaDMxTEQDX2LnEvARK2sKvW8qjJadpDOD9TOtdXPjyfJn50wzPrH5Fkcl54N24KhNIM+KNy1bGm;3:Ggl4Oto5FkrCrlyRQOTD7Tq+TqnGj7T6whw84NsACgnu3+k1znCEmicwRZgxVQDeKq8ONL0YyEVJjYEq/Qj7lf9n+8eBDhyDRk/E/uZw9dYXpjLz2nNHi9qO0zzHWr8sTob6mWBeEtSyJQOJMk4F7iBrQN+r4vlAN5LGp0piszr1QLo1mtWNTcX92VCOiPEb+/NY1Ax6MxzlYN0B9Q1kBmJMO3uwTqsUR1l7i2GkK0MclptWHESLZIxfbIpYjoznSKviZ6rBsfHe8iUhi+Y3WQ== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(8251501002);SRVR:BY2PR0201MB1767; X-Microsoft-Exchange-Diagnostics: 1;BY2PR0201MB1767;25:H0I8wU9DSYns/NxUjnNnek+ZK5alqPSAHzcO6rgPbrYodMOfwscJaEC/Apt9ZgE8Ejjj0CpjbhQruW3I33Dv/2HX6HrJ0aktulNUr8p9iNxoblxkAIi8Xp3OpulY/6RRsV3rBcxzP/9o0J11ArvoCwerm23WLpYAELoSIkKfJlsDF7rJr0zXVwItXVE4VVDobV8mkNpwvq0KDh7N22+1XHReREr35MxmSwbkQ4Y5BngyHNsRpyZsq7Ey3buFgMn7SCZA7xBxlZ9xOLzzxTrZn5R7QdeftSINxkx6EzGwDtC+6WaPiEGHxIudhEfrgZb3VpsRtJW2LKAw0zaU4f42ddhOPyCUHsNEPjAhIm92x/3QCrR+NajqNbMzf+SGKWFl3lqSSdI9NqzGpwsYR79ZYR0DrulkhAOOGlwjQw1/V8DVNx2AVya86L462RbtwJHGAHJwHqPGsbk4ALBIcBeokMRlBAdsOchiS/6P7bIelsoW/51DPDE6eXs950Ms8eXt88RSVoSZITpfVy14c4Jw/5350nX3sZpPLsqAU35UkJF8KmxwK0uR27uiD1aaK22Dl5iFuKiMgZG+H231wXX7JuMgomIU25siQhtiMw5FHdiswHV8Y6tU3FirECxhVz1rjEP+AjH8vbdmfHh+KLgQbCw9ERmPaXC1Ip1SzRIb2giJF5bFuqRLcdB14M9UXjxWn9rKfvFCmnVBqW/hsnAyOqtFJPCdiGZRec+skVLFYWXDEz2kmpLDjKLNitRecz6HK/i8toBK6TyDDAU5DYFk4iYEC1iff24uHVjlMbtoNQA= X-Microsoft-Exchange-Diagnostics: 1;BY2PR0201MB1767;20:35B180eMVQnghL+bBjy9NL3cvl25rl+m7sJJRwHfDXnN2YPGVvBuFlvUfRbNzV8RPbaMcmB5P8ONLDI8yY/zCyE4aSweONwFhcyFEd4F/dilD7NBjPFrTNPdtn0qMnl3Q6ZU6TV1UiPh1jKyfI7vj4GVOu5iih19RZjSesMqzIrNoeqEURWJHycPrXq11mA2exBtS3vRmIoPL7z8kUggNrW1ssqkvvlvweP3tOPFUeH0XuMv7wvJlD8y2cuETcL0JWXDnPdkwv1M7kArCDa4NpA9t9jkF8mywwznci0DNuBrwVSSbYfA8MnSfa2YVq0CIJdoCZLnAaFbvcNc7/0SjDhakSlzF17bv6941/vjdKzanDRhbE1SXm0g4DrQJs7+ArFMkFJq/hH5CIWnDyttDShFNNDJvzaLdE2G20b9QQnS/RRh+eoldXSlKmzwvqlGj4WSn8qFkWSjPqGauPQLOfTXPl5nry9a6leDv8rEBC3IIXtS59aqH+sGwRxIqEEW X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(20558992708506); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(2401047)(13016025)(13018025)(5005006)(8121501046)(10201501046)(3002001)(6055026);SRVR:BY2PR0201MB1767;BCL:0;PCL:0;RULEID:;SRVR:BY2PR0201MB1767; X-Microsoft-Exchange-Diagnostics: 1;BY2PR0201MB1767;4:PGuKY46DliNf6YcsbmN6yl+kfhGWvpVzMg28rBjq+Eab+g96A7+YTJEvDF8Znvg8eUUrJ5/SXArX+y02QFs28tg6WoTdPXUGB4RfS1IhaFAAw7t7H6OG3A9AVrEf4mLM47RcHvBbLGxR75R2ubMLQcTtYWJv1S4R//x6lc4r57H8Cay7JOBhTmVLxQIM/Cw467627pQXaj0VMToaZhoy7sIQu+BFOaMTsSSvRxlXkcqjntOFAXEQLCoYleydSFIx1DVhMIy/QyMtuE33BBjt8CEBwPKAQtDzgfSJjXZ5N+Wo61LuMIDTx4cwmeby0ELtu0GEasjigfWdQ0YWLISsLeUcBEPLwHqCNqsO1Q8kcfLdz1IMe+o9Qg+NwhW+DgDw/LIUcozp9Lvd8K0mi8K7cNaR9hDEaqdu0CmnfbshX5YELpUPWlwskfqYJ8GS2ys4M5a6qwKmUzRmwmrubGA3hlS9Wd/D4yexghx6N20Wy4U= X-Forefront-PRVS: 09796A1B83 X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;BY2PR0201MB1767;23:RLPzXU2qs6cz3kHY9NYjTNo++DNwhItXSET?= =?Windows-1252?Q?9wdISIP0uPkQZ94nrjW9Su0m0N3hOdogowGrZM+96ogpbzn6q65idHLb?= =?Windows-1252?Q?bF3dHUco+D6hw9CM4VTBm0d1W+0Y8IL4LoHXeto9pmkCtLINc2r3xxe9?= =?Windows-1252?Q?aPs09GlyGcuajUe6nqbV+2cbAysSUH1dc/kHo76c1cAgXQnbH5Mz/tyc?= =?Windows-1252?Q?IX/eU13x5KHXYCWKuQYnUwalyjE+Lp+4qbcKnEjveCGywOP+PR62TECV?= =?Windows-1252?Q?ugBOnEhH1wDjWFpKBZBrcsLH4VPQ0+4pnM8o4egfX3t9OjQl8Gbo8YU6?= =?Windows-1252?Q?8/4gPFlRrrZEa+c26tIjTgWx3QfEonZnnVbzU4WtEUsGuS5sDA9hQPiz?= =?Windows-1252?Q?+wxRWtQWnHP0CRaKLkir5qTWP1QJUyXYRnX/Or3WWpiq/+mM4Ph8o3o9?= =?Windows-1252?Q?2O6kb8f3JmkBSaw1JDt65+Sg3cgbJbLUvsG4lMUHEeKUEr2l5eDJRVTo?= =?Windows-1252?Q?Rp9u8LftT5XrJZWiBe5Q+MiGXj0UpwIg7x1NoGz08hOtwheZt5jFBZ+h?= =?Windows-1252?Q?8kR4DR8k5O3Y6R1soCZlH84eZNIsPo1dQ3cDhlZTEg5shswhzzNQtKS2?= =?Windows-1252?Q?7vyzv1Vm+6MJ43Va40hmC81m/l2XwQ7OAT02AMvaQGPJBcxq9ZAq/zfL?= =?Windows-1252?Q?/9sNYKezfrDT7wCrCu/ii8xzc1HGRSZ7zw6ZsqUfh6+ljzggS7JfyjNj?= =?Windows-1252?Q?OknyIrB1NRPin5f0OCTXL8Fg/dJlP5UjIBtR8uOmfU03Chm8TtTjqGkE?= =?Windows-1252?Q?hpMrgSXdFsvZ7b4PWthBtEVkF4x5C87SzERsFqnm2881DbsNiH0TqUhA?= =?Windows-1252?Q?HygwpuaZvw5ICw4GUaNYi5re7Y/jWN1nH4MsnyR9naEUBRE8KCbmgCmz?= =?Windows-1252?Q?Yfm+4P/SpdHZCCz3ZJlVoksjBadHlAqTCJlFr0QC08buyEAKE75jhIlb?= =?Windows-1252?Q?EI+841OWRE1x89N2M02UCp6afWe1gk3NxzRKewCtrOTS+D+8UpoEyws3?= =?Windows-1252?Q?FZEvelzYL6EKdfHF69OWH1EglfP6ljS6nk8eM2YscYc1wxM1snxgruNZ?= =?Windows-1252?Q?JfwMZxTkoX5xoaPfEgNmolwzHDAZ0Z9uGoQabnDc78pJxbC54j1X4VfW?= =?Windows-1252?Q?jXS4P9U5/Az/yKiIAxUfHi9Qhmv3Ohl+LehX6O094VoX9ViD3Y0MLvGt?= =?Windows-1252?Q?0e2M2IurxZQQmvbV8XHJDn0NdCQeYberoDw1QZjZZnpp+bgontEWitGC?= =?Windows-1252?Q?N/riP0+tFocuOtaL8CEYGJOUbfA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;BY2PR0201MB1767;6:WoWwt4xnGpfdzbP1yNpV402fjiXr9GZ5i1HfnftEZweFsl3tzNf2MYyd+uBylLJerr4/wJ5jCCYC5YHkisRL4hPIfUNNUvNSXztl0vtHNozMsrYGXybnwxuu7H14t6iz3jjSEqcLp5i2hTfUlqAYLL1TPeV+8N8RooyF/lI/ENk0k2BuRI6vLzrnYvXu9Z5dEXutY/t75RUdAg9MAf/tdKRm4nXx7d0fh2Q4yZkBF9y7A2u+HlEsxELewzF+HBAtWE5x+mx5PJTvhBVkv5H+hhWGguBUMPNHLytaavvVLz3TMjEBcRp0KK0SE/VrE1HhIHfvvgY5/U1a4PSO7gpIdw==;5:IGaxXh8ErzID99V5PTqQU8g5939ZbYkf7IgzdnR/F1Q5Zn2jbLliz7BXjes5q5Ei58drlrWNKGi9NBEoq1fTFzodeTDoEn4JYZV5HRaXntKf4ZHls8+Nk7Bboycm4QZVmUP8AGXc/OZRjfAm/RFwAA==;24:CzWLNBeTpdTcJ8fH1TzyfQo+2Bb07ipVsDFjv0XQNfVB2lsvrLHouv6+/z0Z/sPfghvpDwcU/qYQqSvj/3fn8BE8xPx3GGY/woLPUMIOVrA=;7:PIBJ0oi523fUUvRdKcAsIwJKtVzL/ZzkV5iPIZCf4a+iEhY2Eaz8ApajjfEKGVlDHDeMEn/7Nm/KSFL1gyWhI3h+5oxJ9iJmox4G8bealLiJxMR04e00TTJqijyRgrwhWSi7s/wwj/QcMxfLFXOQYaMoOhrZsg68pV9o531cqjA7yd+uR6iCTPixUh11bWa2hHjsa6lAlSoUCi687FRxHVnEWTFAVgnjOTCD1n4BVNIBVSIQ/6nOFlHVTCQ3Jt9f SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;BY2PR0201MB1767;20:7fR/GvU2qdDDTU/O64+20fmJ5NCoO1aFvQAwbmGtKd70kLZpYV8IdtH/2VVCoN0S7qKaKZO1roczbxWH/xbNiKJpR41OeEnHnxJ0V7HOR7vmOSBKYOc8pCT2TKV0HSxM8sfCiZrAh0ehpEdoJ69Pj7sOmJ9TzyXse31yUwfSCcPgT9OItBf+xOMalbluCiO39T9nq19+qT0DemUOk0+UDZMgDZgFVUjRLmtEj3LhDpbgqzOXCLQxJl3JLec4DpiK X-OriginatorOrg: sandisk.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jun 2016 13:21:57.3152 (UTC) X-MS-Exchange-CrossTenant-Id: fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d;Ip=[63.163.107.21];Helo=[milsmgep15.sandisk.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR0201MB1767 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 06/20/2016 02:22 PM, Christoph Hellwig wrote: > On Thu, Jun 16, 2016 at 05:39:07PM +0200, Bart Van Assche wrote: >> On 06/16/2016 05:20 PM, Christoph Hellwig wrote: >>> On Wed, Jun 15, 2016 at 09:36:54PM +0200, Bart Van Assche wrote: >> My concern >> is that I doubt that there is an interrupt assignment scheme that works >> optimally for all workloads. Hence my request to preserve the ability to >> modify interrupt affinity from user space. > > I'd say let's do such an interface incrementally based on the use > case - especially after we get networking over to use common code > to distribute the interrupts. If you were doing something like this > with the current blk-mq code it wouldn't work very well due to the > fact that you'd have a mismatch between the assigned interrupt and > the blk-mq queue mapping anyway. > > It might be a good idea to start brainstorming how we'd want to handle > this change - we'd basically need a per-device notification that the > interrupt mapping changes so that we can rebuild the queue mapping, > which is somewhat similar to the lib/cpu_rmap.c code used by a few > networking drivers. This would also help with dealing with cpu > hotplug events that change the cpu mapping. A notification mechanism that reports interrupt mapping changes will definitely help. What would also help is an API that allows drivers to query the MSI-X IRQ of an adapter that is nearest given a cpumask, e.g. hctx->cpumask. Another function can then map that IRQ into an index in the range 0..n-1 where n is the number of MSI-X interrupts for that adapter. Every blk-mq/scsi-mq driver will need this functionality to decide which IRQ to associate with a block layer hctx. Bart.