From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751106AbdAQCsV (ORCPT ); Mon, 16 Jan 2017 21:48:21 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:54212 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750803AbdAQCsR (ORCPT ); Mon, 16 Jan 2017 21:48:17 -0500 Subject: Re: [PATCHSET v4] blk-mq-scheduling framework To: Paolo Valente References: <1481933536-12844-1-git-send-email-axboe@fb.com> <7A8A5078-E9B8-4EBF-BAB1-9E8EEBF3A043@linaro.org> <37570439-97C8-4009-B143-C5E78789A137@linaro.org> CC: Jens Axboe , , Linux-Kernal , Omar Sandoval , Linus Walleij , Ulf Hansson , Mark Brown From: Jens Axboe Message-ID: Date: Mon, 16 Jan 2017 18:47:15 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <37570439-97C8-4009-B143-C5E78789A137@linaro.org> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-Originating-IP: [50.233.49.230] X-ClientProxiedBy: SN1PR10CA0015.namprd10.prod.outlook.com (10.162.255.153) To MWHPR15MB1199.namprd15.prod.outlook.com (10.175.2.141) X-MS-Office365-Filtering-Correlation-Id: 6ac9dbb4-c08f-4232-a96b-08d43e8328c1 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:MWHPR15MB1199; X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1199;3:oytVhpHrBXqJAY8l3jNeMBM01EmWwfoYJ7RrB4hY9hOfPfQEkY4hCZxGpsxeuLExOHWTHsYfdfXnKoSE+nwx90//Iecaf9FQLKvvUdjwTQGLQ3iZNqrHxFNt0OvFk9roszRyfWKzSD2GmRu1pHp9ZbJ6mn4z7Eg5FyzlCcS0TJa3Ps4zmveiUDAvuPlhQXVSxHgt0cvHTwI0hNMOuYQv6MQhCpF0Wu/afSmJNBYsSmS6VQvHnDPLiqmIW829AoqLuDcRyWE0p7WRP8Njr0ayZw== X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1199;25:h6NB8y903r9Kfj1hVmP0+mtDhNOH+IVe8mxviMJMWibxZyFtl00F/LQntWWIxUJz5+oMbI+fywsGTYSmNWjpTXZMN58FpcGx2LdU6NfsWC9MbjECkC2l5Fh397rhSQeS74BL6ulrWMqocGmoBhEuwTV7IGfJBxUEhZGjncScgUnHtz/Bv38Iv9AicfUqsDyuNlBmAjKoGfNlj9Z+Tm0ABOXLuh9ozo7xjLxD6d82jQKOYorl5NR2KNvUb2GSoUlaiZbldni/jTPCLTU12w9+osg5hOueKtryI1yeU434MiNacCLU1aHZbZuKtW6stgSSZi2g+r84uM/n/xqbwsiRa3BF2h0965/9sm1OKq5kw7AA3HYw4spJdkpGL6Esx0qvz0qgVEinb4V/FichSuWamDAFAtirF7L9w9HyiSJaFAlKbKSRs0ILOBZmQJt3GmdtLOpCsBq7qOf/IuPeLGMO4ZFLKP+FovgqeNx+wb8aTujJJnPcjnQVwnPXzvhDQGxxnh3ovQiILQ6CSvN6SRIFjfWlL/P3BXNRqkT1FJ2KVm+MZ38TQnv3c8T8+NOVwg/kCr8Nx49HaDBXvrD03pepLCF46Qn834aj4nxLmH2tb5Y82bruhSeiNJcLFAliNngdUggUQAei0U1ok/dRfUVSBIvAreOkM93N4HnTChhEgEXfI1PQy+eslPLfpmleOOjdlj0Uoa/QlmkvK36maFwF/LGVFNnPvxz7zc0HHQBzA8O2eEzrLv4IvwzGIy8ppy36Q3pfJRxx0jd3pTNc0AWblWj/WzT3n7GqCOv+oulnoJS8mVFvLLUtRjklSx0nCuDgM6MTNWUN09RCFUynBkfJMi3JhISpqIvy1NERydfIGmo= X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1199;31:E6M1yx0xTuZImaIVz25JZXyxqaL0d6MBYZA4Xd4VHYWQ3+d2nJxzDn9rTcYV75CjIVC74piwMj1MiratzexhfYrJ0mvD7VONTLXem0icp39V3Lt3HzXusi5kXqcpBUJogL6rRHn05cS6NtFe0jMrQAHA2ps1G/eJ3ILAk6TrCocOqHqNGmybE5wIskrMmxawSSgx4jo6VQft4zVHBo4j59uriumezgnaYcvxPMQ22uUkwek/HIQq6PvnWx5z9qjl;20:VEKTiHCV7XbCwSQYXyx25Lj4cO73MQRMrOgS8ZCzSRhINUrlW0RhjTw8KKneYaerZXizaZyhWfI9WafY1BjT3InxPJdCvIwrM2Zn8x1Kra6Dy0OoxwwOWQSzVgopTUkqwR3fgjcIHWHf1sQfi+VD4HIaIX3P5rnLkVC+ULQcNOQA3pUpUD/1XHni2YRFq6b7kBbyIrYPz8HLErqS+lyzRwvAuqvy0YsWaSneal+ckAz1QyDaQV/J9+AUOwls2Uk1vNxZkg+IaQyBZaRvNmbS/eIFp72c1mzZzPUTJfYHwduFu9EltW0Hk1hMTV5li3Atk7frN8Q+jbFVhLNMBYJtP4Xo/B4mxoVmxqNhmwHLU6NpCWNu1IuwHsKsr5FFuhY05QBUDTvdphNJKnMqIEuY1LgvkxJP9fQoe3KPUxktNSePQxgsSicOF8QkMO8K7ak17Aqjdope422PDqtvv8IRy8umuoM2r6tG0ASsqr6MSqWL4f6o7w/6qQwIRxnSMh6D X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(788757137089)(67672495146484); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040375)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6041248)(20161123555025)(20161123564025)(20161123562025)(20161123560025)(6072148);SRVR:MWHPR15MB1199;BCL:0;PCL:0;RULEID:;SRVR:MWHPR15MB1199; X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1199;4:ymQpVggLt86eFOXdkIud/6Ah1WKwQp2jPw7ekWe8ZZjwrokTCCMEiFSgp2q3OvwmXomcvVjPckIxYWoshgrzbglzDx8+PY0sELIJxB+/bvAiPt9lFgZ58eoFB6adTx4gRRzM1MloIUQOtoiC5jg9Z48WBBTK7XHofZh+ZKuTtJkajhLlPCdVVt/+doKgWtGym94ycaCVp/xwa4H0UToKy0u4zP8kfQ6+mhrnCinu02sNRPfFaCnIS5/pf+xeduie45b9SP58TY3sagO2lTY5MC7oHr/V3unPTkxQwuNYbbfDCCC8V6wwEHKRLTO9BXH4GGPeDl5lPqqPuyZM5tkWd/tcyfnNfv8BK0CfmGQWTTFhbghyD+FDk2vF2ug2/Jgev+fcoYZN7s469RsbdKWy5xDNM+oIqJAukWiJM5/2SGOeHN6H8KCT7WsA2X/8Fe4K9Urv8azPfiWKuD/NQ5IWPtv9rgneIkkC0gsxqVPzGqldoshmuYcr6AaqflkdSGDyvR7VDiUW23yAwqj2F4d8n0foLPbJWfwWbJVfIhRMMlBBez/DyjF2ZNvo3QK82k2IzkRIhwllrLpe6Nik96qyQdG3PSFvgNmBkGgf9celOnYZ6ezCnNP3+3SGJovGK7e35YZ8TqQPDO/QFtwJEz8ncQ== X-Forefront-PRVS: 01901B3451 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(6049001)(7916002)(39450400003)(39410400002)(39830400002)(189002)(24454002)(54094003)(377454003)(199003)(5660300001)(64126003)(83506001)(230700001)(33646002)(31686004)(90366009)(92566002)(6666003)(50466002)(6916009)(6486002)(93886004)(54906002)(65826007)(25786008)(117156001)(229853002)(77096006)(2950100002)(38730400001)(6306002)(110136003)(230783001)(76176999)(66066001)(31430400001)(7736002)(97736004)(101416001)(81166006)(81156014)(189998001)(54356999)(68736007)(86362001)(42186005)(31696002)(23746002)(4001350100001)(65956001)(305945005)(106356001)(6116002)(3846002)(8676002)(4326007)(105586002)(47776003)(36756003)(2906002)(65806001)(50986999)(30001)(6606295002);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR15MB1199;H:[192.168.102.217];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;MWHPR15MB1199;23:8CkN4syR5SQKSrWkzgqxaWf7v9PIiosjbbjNn?= =?Windows-1252?Q?1PFFQngRFSCej4BYyp0l0AHK50wW92Pc6LuvodPszgBI8Yarv66UfR/I?= =?Windows-1252?Q?wx03lSRJt9xtU7mO2FptYF2agmEIhCiMXdhwftPkFQYDsTtIEOLjR0S0?= =?Windows-1252?Q?231jscz63gW2MFwv1uY7Qkj4B1KQIo9A+9B0JQFo4tdt3UE7EjKO7pos?= =?Windows-1252?Q?yxByUEM/gpWI7kwh5LPMh2ad74df2ub/35Clw3b9kQXFdb/SkRuuQgzd?= =?Windows-1252?Q?qUGQsHS2hp3hj3hm1ryc64birNVV24Cs/hRZK+UOQzfVLbOFjVf9UQn2?= =?Windows-1252?Q?tiK9hV/xyX2m+miuYmVO2+uZkqpy45Dr2LFv7iDwcxaD9L4MzZKRuvZh?= =?Windows-1252?Q?Pi1L+/afi0e8noP+rZY72rYpGuSPhBcH1VJc2KJp5Gsg1xB+cfis4CCG?= =?Windows-1252?Q?WdVL5cmuXVC1go/LJ7wcRmYZUemQ97xUAYllAaAcFvN1DyMLe8BKxuCC?= =?Windows-1252?Q?ymeieILI4uW++ytx0PJnACPNFfEMBfGRYbqJTrcCJq3dqFdLulBHHkYG?= =?Windows-1252?Q?w8aH81OUQKJia1G2lc1563d9B8EyBL2E51ZRNE9azDh8MKj3/g3OMUV1?= =?Windows-1252?Q?ENIRUTNkgHA60MdyhOo7hf8FupstrOPplfNxnpJV+S4KUTmdFBx/oLje?= =?Windows-1252?Q?4dHx7sH+3VlJbW3780fWhbFvnbaz6x9ax0Ucc0WT1ph+UDldve4E5VV8?= =?Windows-1252?Q?DB956NUVjykc4jtcuTe2ddbXRmt2B2R8UOreqPvD3BLgP8Pxsv0jow2F?= =?Windows-1252?Q?6wJFvbDZhCSR24gEDpRbn7tFl3CsGhPgrSNCk0FN5ot3iPlHYb5zlSNg?= =?Windows-1252?Q?j4UPuTurgkpo3KfjQk9p4mly6u8VNLo2uSqfkm2cvWqkQ/GYZqLL+r7E?= =?Windows-1252?Q?GpqJESl1ZaFRBj4sR6cd67y/JHe9JdqxDRWmR5S74O4R6ijFMrPvuiQm?= =?Windows-1252?Q?QybvuN4mi0RmrlWnQ2Gsvt+QeC6kzw8ovEIu2UnUJ+08v4I8oBUmUGqc?= =?Windows-1252?Q?vPxoRAjr1I4We2RARbiMcjGbCwsctoDg+jloDui4thwgVQ7Oz7yFc0KM?= =?Windows-1252?Q?YkO+6nBa9+XawyEtpW1mP5PH9aMvbkQH31j+QS/Kx/dvJ8I2PSkYpogv?= =?Windows-1252?Q?PwHx9FTieu2zyyecZNz/US11rQawCWSxq+LHZu5pVX6BPNJVPMnVKDvA?= =?Windows-1252?Q?/+BhwKtDPTdzqFcfMQkZBuyxNqmWDebSgbDpAZze8OwSha7p8lFvVhBS?= =?Windows-1252?Q?32/JFBTaCTJsUCcAuw4AyTb2stIMau9X9QgEVgGqFIpMvF1Pbqct16x4?= =?Windows-1252?Q?bBq6TTwVkQjGslCWrzKDbYbR9gINbWfU/5O0+/5b6GAxK79eeBHh+kjk?= =?Windows-1252?Q?d2GnrItv9KY9tRvEBe4T6rK/hvM5BuUmm7r2M1aFDxA03gj6vgEYfE+4?= =?Windows-1252?Q?mfEKR7tkpdRpCn1hYcizJPiZLpvYOEXvfMiZ9DlXe2WAS8c1dOlIUoWs?= =?Windows-1252?Q?NNX58K5/tym6NjldsvL0+C+BQeEm3sFtddF4n8A7nZIwQv3SrP7l5eI/?= =?Windows-1252?Q?49CHEMtpqHJ8Rm+wopjcWlGkLZb9tfYXlgN855i6xw1zHrF2Wti6c0QR?= =?Windows-1252?Q?TzyyO2vbae9MUIy5Gnc946knnUB+lGf74X36JnI9lSZHdf6M/wP?= X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1199;6:obFPfyige6goSmW09Mq8xyGx+9ApaP/gmk8Bq9LO/1/xF9m8oX7d543uLHmu0f90blYzHksUNGa8EXkX6ueZRFJxM/KvzKhaORRc8yrhwRBkpa/LedbT2qw/g/V8pG0lSYwKaJjvHPwJzH2BP99askOw78+cULBkUmMqeNb6y3nsFEfJK5cC0khWNEa6L+xzt3Oe8CCpWssGCq3lE47ZbyihkfqLWGamURJMvopek2g+uVkq15ztOab22opoacJirPngHINCTOZexoYrKSceoIIEwEen2aVox3jBLLWeCM22FZ0JB7WbGbTG/XLHLAgbckgGXxZh6jIRq40RocbTxH1BoOdrUfyudu4b3a7y5TGVubb3jBpeRVjGPentIadm2i8/c8J9gNWP68DBYn9hfQxx6wC5VX+ymPsHulzHabc=;5:oREx3jGUHCvS3p7nMh9Y8jEYq0pZo4DWZwQ3a4Ey04zwXGt/T96deeboSj3RX6ZGtba1Qolj/3ZUTG4+hjnes7LeH2hlIYkUW9MIut45TNMk83JESieNNHvsGp5kZCrlVBCe3vfOtjJcostKUsqwhAHu+xuzPPcYRd5iQ7ed7Sc=;24:vhx9bvPk6Iubf6ghkBfPHZJEUDU797rYP7yL+9/7bLHPaw0sBE/yAHJj8fTcjl5AHqv4KWRLwMZlJGTLmxm7ygPMlrMC1DmyuFetuIS//x4= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1199;7:c5aS0GHmV54oIpO00SKQgcpjBlrjyheOZjn2Y+GKekPNt/0H+Z1dP3w6vSFCV1c7Gah/wOTQQiDEnTyNwbbgB3wR/dBA4R9ki5SixEZUTiS6IGxL3ZTGUl2ATwK+S2DCwRadPvz/F94ixxH0kfNGYUEZcZI5hNom6cAVPG0TqoUJ2xQMvMxLZxjHj0l12wSSWYY+pww65K/pdKT+0lSz7elLd9D4ilO/pkJWNq2dDj4o6gCAqCW6eE4VfLONI54cwW9Go8jq0+80SeC38QqaTBGcQ0XAN7pjlDXL3kjVm5C1hHUVNLCB1N6rNIQ5cwMamCpoSRj2ny6NSmIvdN92hzKfALpU921t7x2+hCJW+etwQS340y+s1Iff/lpChnInX1tTyKFGj+F+Jybbd5lFw3yx0xpOU3ufSkWZJr6D0ZZDvu+Hy9slNYj1DlnsmSCZB/UsTOSSdCDyEPRNZnhg9Q==;20:BT0K6MjubnyvjU0F21tPwTMg2XSVM5n/YBwCYpV9nir49nL+VCkebhgs6pHYk9rDX4FpFISVQzg62TMWqfDof35dOu7AKn4YBAoCDCkDX1i5nRt/ex2dbgO6gxQgqaCnFv9d6ewCU5drnsJUhYpzGX7u28flpmr9TocTmZrWQuU= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jan 2017 02:47:19.9390 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1199 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-01-17_01:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 12/22/2016 08:28 AM, Paolo Valente wrote: > >> Il giorno 19 dic 2016, alle ore 22:05, Jens Axboe ha scritto: >> >> On 12/19/2016 11:21 AM, Paolo Valente wrote: >>> >>>> Il giorno 19 dic 2016, alle ore 16:20, Jens Axboe ha scritto: >>>> >>>> On 12/19/2016 04:32 AM, Paolo Valente wrote: >>>>> >>>>>> Il giorno 17 dic 2016, alle ore 01:12, Jens Axboe ha scritto: >>>>>> >>>>>> This is version 4 of this patchset, version 3 was posted here: >>>>>> >>>>>> https://marc.info/?l=linux-block&m=148178513407631&w=2 >>>>>> >>>>>> From the discussion last time, I looked into the feasibility of having >>>>>> two sets of tags for the same request pool, to avoid having to copy >>>>>> some of the request fields at dispatch and completion time. To do that, >>>>>> we'd have to replace the driver tag map(s) with our own, and augment >>>>>> that with tag map(s) on the side representing the device queue depth. >>>>>> Queuing IO with the scheduler would allocate from the new map, and >>>>>> dispatching would acquire the "real" tag. We would need to change >>>>>> drivers to do this, or add an extra indirection table to map a real >>>>>> tag to the scheduler tag. We would also need a 1:1 mapping between >>>>>> scheduler and hardware tag pools, or additional info to track it. >>>>>> Unless someone can convince me otherwise, I think the current approach >>>>>> is cleaner. >>>>>> >>>>>> I wasn't going to post v4 so soon, but I discovered a bug that led >>>>>> to drastically decreased merging. Especially on rotating storage, >>>>>> this release should be fast, and on par with the merging that we >>>>>> get through the legacy schedulers. >>>>>> >>>>> >>>>> I'm to modifying bfq. You mentioned other missing pieces to come. Do >>>>> you already have an idea of what they are, so that I am somehow >>>>> prepared to what won't work even if my changes are right? >>>> >>>> I'm mostly talking about elevator ops hooks that aren't there in the new >>>> framework, but exist in the old one. There should be no hidden >>>> surprises, if that's what you are worried about. >>>> >>>> On the ops side, the only ones I can think of are the activate and >>>> deactivate, and those can be done in the dispatch_request hook for >>>> activate, and put/requeue for deactivate. >>>> >>> >>> You mean that there is no conceptual problem in moving the code of the >>> activate interface function into the dispatch function, and the code >>> of the deactivate into the put_request? (for a requeue it is a little >>> less clear to me, so one step at a time) Or am I missing >>> something more complex? >> >> Yes, what I mean is that there isn't a 1:1 mapping between the old ops >> and the new ops. So you'll have to consider the cases. >> >> > > Problem: whereas it seems easy and safe to do somewhere else the > simple increment that was done in activate_request, I wonder if it may > happen that a request is deactivate before being completed. In it may > happen, then, without a deactivate_request hook, the increments would > remain unbalanced. Or are request completions always guaranteed till > no hw/sw components breaks? You should be able to do it in get/put_request. But you might need some extra tracking, I'd need to double check. I'm trying to avoid adding hooks that we don't truly need, the old interface had a lot of that. If you find that you need a hook and it isn't there, feel free to add it. activate/deactivate might be a good change. -- Jens Axboe