From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752774AbcIOOpa (ORCPT ); Thu, 15 Sep 2016 10:45:30 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:47372 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751800AbcIOOpQ (ORCPT ); Thu, 15 Sep 2016 10:45:16 -0400 Subject: Re: blk-mq: allow passing in an external queue mapping V3 To: Christoph Hellwig References: <1473862739-15032-1-git-send-email-hch@lst.de> <20160915144050.GD16556@localhost.localdomain> <20160915143202.GA26221@lst.de> <20160915144212.GA26430@lst.de> CC: Keith Busch , , , , From: Jens Axboe Message-ID: <1c3794ac-4890-adf4-e953-3d57ab545cd2@fb.com> Date: Thu, 15 Sep 2016 08:44:44 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <20160915144212.GA26430@lst.de> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [216.160.245.98] X-ClientProxiedBy: CO1PR16CA0015.namprd16.prod.outlook.com (10.166.27.25) To CY4PR15MB1189.namprd15.prod.outlook.com (10.172.177.11) X-MS-Office365-Filtering-Correlation-Id: 4277b3f7-3acd-481c-d175-08d3dd76d80c X-Microsoft-Exchange-Diagnostics: 1;CY4PR15MB1189;2:hxIpA/GlCoYdIwEhxaKMhxqw3HsHCXZwNOMV84IX+H7rL44kM02/hDNfLCq7WQIVXQZlQf+94p0zlb9pCso8R6LGxe26/yfQvfyqlJruxfDtdau0mqh/9jli2MM3QPQBx9ZX4ZoKk8sfK27kG6RTuzh9VE5KWKT8snpitnLPaexujcG1p4G833avoyFdJrNV;3:Nuo3GQYEXGDG1dx9yg7gjxCvr9nrdLvr7P5vfSjuMfnBdE0EmCZO0CdTfRPca3tnsmnW2XcuG6Fc7FmCb8jVQGBXRQtQt8rvkRcacqtYX11ijUQh9BhRl5Ohbjr8v4Gh X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY4PR15MB1189; X-Microsoft-Exchange-Diagnostics: 1;CY4PR15MB1189;25:6D5FWZe5YEeVJik6s19ELkpZaDGcblMOperTTCARFV+jYMVCIz9L+NOwGOQVJrfKXV89Qkoem9u18I1javBmaQU/9MI5gA+wpGYXLZbHZbTYbyQfsLbLbyL/AF9+wsX5GyQyokc4tsRVycVugXEaERpNKOCO9cBtdxETEAJGGcbXFm9wFEYe9ZQQA1LR18X65+5Jq/dIo5ZbPxwtz4xzrrN5yt38RLRecGb0APzlQvk3PGVP54ehH0HCfGn4akhiX8fA091pABBXIqxdYh0B1HsP078WhKaBf6C6Pu5lGMpyXU1i5sNxwBuDZP+cfvMvxj7UUvaDsxjXPoPyJ3C+0Q0K3oQFv6onz0xS7zhZuLb49oYXvixhRL+93/3SHIz8kUBAaPgBdJX3CPC90PpjbsDM3DwLQH+vWeFr8O0eQ65IAmDj2jqug9IdS8VOkKuOAk6FW/CUs9qF0VZMfJ1ppTIc/CScZviCzcnRzSj1SgotWZNYUdHDoXIzg5IjjLSg3jmX3QuYWieBQtTZEuLlBRdZ+OZLsyxa1D5y1BUOrW0srYPQWLQe4f9SArfO9r/sbQAnBCYWfA8zPW58jsNJGRJ/474VAEAkjtVk6I11HR1dYHqSPGkzhV0BqNRFuuSM+jC2hxS8iGGieDU1Z2S2yvTXzkiFpo0pCQL6Dv334xpNah+9boRxjwVsC19WtWqG;31:Hz43W0JQvsMjh/it9dOG70u49FOTLpGaPIsTUhJl0sz4EfvT1Nu40z8SZDzMLzgr3QF0eCoqf1O08LWkITqO6L9yhzSPMGbV0GJADLgyEKGLpTZgMBe2+HVngEiCIYTD/DVpklsfV7dR1jAtQimP6wXBmzzEOrms5+n7npeg7aoAQomb68HEFMZJ/TzHt2CzkWBzh8QIJbvoaIeStEva6KyZeEWFz5vQITJrUcfzBc0= X-Microsoft-Exchange-Diagnostics: 1;CY4PR15MB1189;20:R+vJWSvq6HAZY0nSqCYFhftEyD/SBfw/LsHtcJ7fr7j3/OhuekiqoUTJ2JIY7JHmnLOs2VmSvVASjUL5DzcL1L0VdSBhC3KVm3Sz6sidFDXWkfU1zfQ15IW5dQghM0Q9FJa4+GNDjYqfW0dcQIOteoqyqNqY1moD5fo6IULBj2w=;4:p5oDLIUHLo95CE4iPb7eZq1SrQZCBl69/JKXp4SXkZx3nXOmz0SiF3+Ua4TTuotLbX2eODhLiLB2kon3sG1ipshFZRfgL/ZwI9ni5ynyd4/4C/xbg5jsXIRf/F15gjnWOBJvJMkcmkAxBCtSbveidGNdrAtoPMUNO8tXgieaQFY2CZatk3+4J40TYdsttge33RYa2t3xO8pH3YSoNki4z1tbzhPq6g5PEwYAo19rWume8imB/2vSK/KvhzntpqaqBQIegs8ssZQ5J3SHgcq0+JRnnCcr386Y+xYPovNMGIulSJfH050c08JdFj07DKg7JWPAYDa/l6DOcWJ4QS4aFYTI/n/5mYdjHihXDXihACaDfdrwA8EFBfqele5E5/i/NAZrN4M7KWFFOkp25z6cJg== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046);SRVR:CY4PR15MB1189;BCL:0;PCL:0;RULEID:;SRVR:CY4PR15MB1189; X-Forefront-PRVS: 0066D63CE6 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6049001)(7916002)(199003)(189002)(377454003)(24454002)(7846002)(4001350100001)(97736004)(101416001)(31686004)(586003)(305945005)(42186005)(7736002)(31696002)(6116002)(86362001)(8676002)(76176999)(54356999)(81166006)(81156014)(3846002)(50986999)(23746002)(36756003)(50466002)(77096005)(189998001)(110136003)(5660300001)(66066001)(65806001)(92566002)(65956001)(106356001)(64126003)(230700001)(47776003)(68736007)(83506001)(33646002)(93886004)(4326007)(2906002)(2950100001)(65826007)(117156001)(105586002);DIR:OUT;SFP:1102;SCL:1;SRVR:CY4PR15MB1189;H:[192.168.1.153];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;CY4PR15MB1189;23:D17aweJG2SHRgD09fb0rt+u3J+AUkz8QMZvXp?= =?Windows-1252?Q?NYXoeA1XqeZNlHaXMb2gYw+8BBNgCNj12vDI9vr79e5uHKlKtFJw7jT0?= =?Windows-1252?Q?pNaaa9rUOkxz/ttzOuCOue+XiHrXEQNYL2R1AGwB1EIiTuDphnCAv77x?= =?Windows-1252?Q?p9IXzxS+UzZWdmXgmWVSmjWyVILsLvguAazslxBq9W/LlSeADrwacyBB?= =?Windows-1252?Q?eO5YOY75gQ9SoWnggTPNJAlwkfKoOEa9FOay2kbSWgCz1w65lM+NTbB7?= =?Windows-1252?Q?wAD+X9LearCbHNAmZWQcx/SiW8s8BskYkZ7r4+qbk7bwL09gcEmqq46m?= =?Windows-1252?Q?EIP5yXTRWLGv3gfL9MGtmWMb05vN+KcH5BJViYX6Gs54bRLHFvNxqkkP?= =?Windows-1252?Q?hmKUalj3Ie7NYAKMvSZBnHxXn75vsnFIy+nZTTRUOAxIRqgGYfFLCLOz?= =?Windows-1252?Q?bR8IoEKJKNJbUz3jCCunUFLPZfIkPSF9GIQbniWiXuXTDr/xa9JEAbT8?= =?Windows-1252?Q?39HfNpratw+mpoDM/83HOdzuCnvx4MoM/PqsOLQSA6xMNhwShQwpBPNh?= =?Windows-1252?Q?hMDj9I+Vl+R7OPuwjWh6rxtOp6aEK8Klp25d89bPH690B6lwULAbZTBF?= =?Windows-1252?Q?1TQXGFPDavKqPSCwAK8bO0hTzuHc8kT0xuAKmU7QFHwzKiJIRIR3++NZ?= =?Windows-1252?Q?4DFlX6UeB8A/Ed9A/3R7DqJPAHXccEqZILrW67Ij65mnDtVWhvcIJ2Qg?= =?Windows-1252?Q?g6GUNTRpkVz1Kj/kC6jmk4MzMePP1zKoKpVGABRlp6Fgo8ASFiMsIIZL?= =?Windows-1252?Q?X5Cob7WtkXbZtnXrOPtNsX9wWtrkkXXTSRGdkeAmGQ+9HAeCWk8GFaa2?= =?Windows-1252?Q?GyZVG3MqPrj3X8CB7+XHRCFSHQq9jFKRpNafnQ3QTOiVKwVQVGK8AMaL?= =?Windows-1252?Q?SPgSYdAdxyMgHvxp2WbOjwEf5msnlVGMRaen9p1vCvXdCOorg0Hm28tX?= =?Windows-1252?Q?7uoUr7/j32HhjX5KcfRarh6XN8j8SvTqyLf2sI1MXB30upj1PVpUkqDz?= =?Windows-1252?Q?pRb0aVG+rs+rfXGc7xZkNcf0qbVpYEha51rhLkMCUB2LEg7SmX0ERqFw?= =?Windows-1252?Q?hzGwmdcOqmiG7F4HyYc9P/4fjJWSoqYCv6ARmzMH+CptouLfYezTr+q2?= =?Windows-1252?Q?X8o0TXRPIyiPGrA7HkXO3PoO0ewAodFrjfBH6GndUHXRptZcic3mxxKE?= =?Windows-1252?Q?pnkyz55PeQ6g0eAwIVN07RQng0DZKdWGoZFI0WIyuvikWOfmFB0CB4B8?= =?Windows-1252?Q?e8Qr/e0VqilfyWzqCdKNKwjNUEQmqrh6LnijrBXA2rzlu/4qJ0Do+dsk?= =?Windows-1252?Q?Vf9aRKJB5e9pTp9dnkwIsO8pzOm4HGyUw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;CY4PR15MB1189;6:vUXY+anDPlXGYbwo1ILPL9tzpBvDcMKIMjc7E34HsoNLKQhLA+uzc5rYRvWr2SN8EEl6xn+OyiAuQ1dST6HglFPnzlqheS7ghmbQTiBUqCvmBIELwbRYewFr0RE9gEJ6+oXtFFygL/+VK96WwgVH2iDwgQKx8k4hqec3rCyzScOdAfBLdcG0U59pZnSz20i1zygtU2RtGu7Ljd8876WJB5M8eAGwtn4QvigdYM46x2XenqlBTz5MfYx9013T9pVBEWJOHze0nv1dSfu/4RnIC4rjcSjB3fJV3mysUdPatSA=;5:dyB5Ubnzb7b/wNgQhskghf6pch/sZ8rT3us+uygfu11QNqnpK9/ybYyOhDqGa0oS7hxNwsl+KEb2DgDJFqMQFLTf5u5EYrdtHycSnFXbv1ar85H+O8zHI20FVYk4C8CZkRZLe79oMuV3gOzEv2IniQ==;24:W4zQw503Y3lraxXmke/hfMlJdf7lZ7HdEg8ust96lKtv6ax4zS7SMUkTXsF7WyNNUgQ8c6josA1z2dn3hT/VU5eOAAXc0oqzWEfofC4ax+I=;7:D47Mt82sNO3TRAIAnyWbPO2iG5cKRvkG1FfYyD8+2raE0jDMpyyZKNRxGYbAoabcRyZurCN+uUM1GVFb8R+MhqH+DquzsDoaRUXrq9q2SKSA1Uls7JeoyYc5tQyytluqWA/myHDpVbCYxX9BlME2uLMk5+0tcTTt53kvRL4AFanL8JXJNtmwabHlSf9Dc5xaTdzzItmk43iSRx5QpumwQc5A9MReiSi97jXj03VFS2Tq0N00+8vHvhbVq90DMh+b SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;CY4PR15MB1189;20:YBPFY+7GXR8c4IOXpdYt9UKL4AL5bDR0GlXhT9bjFgMHeMiFm90/Yfv4Qq29gsfVHcBsmiwN52HjpWZTNdqH/5e7Xq8O03g7TZoIi98Psmt+mBl07rDtt8+W/t6VgxIGHGFz/a8LjTo+hh3osZkG3wjBeboGw1+I2LbPm/BBsWA= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2016 14:44:48.4635 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR15MB1189 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-09-15_08:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 09/15/2016 08:42 AM, Christoph Hellwig wrote: > On Thu, Sep 15, 2016 at 08:34:42AM -0600, Jens Axboe wrote: >> I was going to ask about splitting it, but that looks fine, I can pull >> that in. >> >> The series looks fine to me. My only real concern is giving drivers the >> flexibility to define mappings, I don't want that to evolve into drivers >> (again) doing stupid things wrt mappings. As long as we keep it strictly >> as a tunnel for passing mappings defined by the (previous blk-mq) core >> code, then that's fine. > > So my earlier versions just passed in the affinity mask and left > all the mapping in the core. This doesn't really work anymore with > the sibling aware code so I had to add a method. That being said there > are some drivers that might want slightly different mappings. > > For example skd (if converted to blk-mq) has MSI-X vectors for it's up > to four queues, but it also has MSI-X vectors for misc book keeping > before those, so we'd need a version of our PCI mapping that adds an > offset to add to queue number when assining the MSI-X vectors. > > That being said I structured the map_queues interface so it can't do > anything crazy - it can just build up the cpu to queue mapping array > so there isn't exactly a whole lot of crazy things a driver could do. By crazy, I mean even things like thinking it knows better and defining mappings differently just because it can. But I'm not too worried about it, it's just something to watch for that we didn't have to care about before, as the mappings were completely mandated by the core code. -- Jens Axboe