From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 980B0C282CC for ; Tue, 5 Feb 2019 15:28:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7206D2081B for ; Tue, 5 Feb 2019 15:28:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729868AbfBEP2C (ORCPT ); Tue, 5 Feb 2019 10:28:02 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:3270 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729232AbfBEP2B (ORCPT ); Tue, 5 Feb 2019 10:28:01 -0500 Received: from DGGEMS410-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id A42247F3165B2F119336; Tue, 5 Feb 2019 23:27:58 +0800 (CST) Received: from [127.0.0.1] (10.202.227.238) by DGGEMS410-HUB.china.huawei.com (10.3.19.210) with Microsoft SMTP Server id 14.3.408.0; Tue, 5 Feb 2019 23:27:49 +0800 Subject: Re: Question on handling managed IRQs when hotplugging CPUs To: Hannes Reinecke , Keith Busch References: <20190129154433.GF15302@localhost.localdomain> <757902fc-a9ea-090b-7853-89944a0ce1b5@huawei.com> <20190129172059.GC17132@localhost.localdomain> <3fe63dab-0791-f476-69c4-9866b70e8520@huawei.com> <86d5028d-44ab-3696-f7fe-828d7655faa9@huawei.com> <745609be-b215-dd2d-c31f-0bd84572f49f@suse.de> <42d149c5-0380-c357-8811-81015159ac04@huawei.com> <20190205145244.GB28023@localhost.localdomain> <43b43df0-8c8a-fbf0-69d8-9165a73bffea@suse.de> CC: Thomas Gleixner , Christoph Hellwig , "Marc Zyngier" , "axboe@kernel.dk" , "Peter Zijlstra" , Michael Ellerman , Linuxarm , "linux-kernel@vger.kernel.org" , Hannes Reinecke , "linux-scsi@vger.kernel.org" , "linux-block@vger.kernel.org" From: John Garry Message-ID: <5c9b33c7-ae48-6ae8-b9d9-12ab7c486001@huawei.com> Date: Tue, 5 Feb 2019 15:27:40 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <43b43df0-8c8a-fbf0-69d8-9165a73bffea@suse.de> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.202.227.238] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/02/2019 15:15, Hannes Reinecke wrote: > On 2/5/19 4:09 PM, John Garry wrote: >> On 05/02/2019 14:52, Keith Busch wrote: >>> On Tue, Feb 05, 2019 at 05:24:11AM -0800, John Garry wrote: >>>> On 04/02/2019 07:12, Hannes Reinecke wrote: >>>> >>>> Hi Hannes, >>>> >>>>> >>>>> So, as the user then has to wait for the system to declars 'ready for >>>>> CPU remove', why can't we just disable the SQ and wait for all I/O to >>>>> complete? >>>>> We can make it more fine-grained by just waiting on all outstanding >>>>> I/O >>>>> on that SQ to complete, but waiting for all I/O should be good as an >>>>> initial try. >>>>> With that we wouldn't need to fiddle with driver internals, and could >>>>> make it pretty generic. >>>> >>>> I don't fully understand this idea - specifically, at which layer would >>>> we be waiting for all the IO to complete? >>> >>> Whichever layer dispatched the IO to a CPU specific context should >>> be the one to wait for its completion. That should be blk-mq for most >>> block drivers. >> >> For SCSI devices, unfortunately not all IO sent to the HW originates >> from blk-mq or any other single entity. >> > No, not as such. > But each IO sent to the HW requires a unique identifcation (ie a valid > tag). And as the tagspace is managed by block-mq (minus management > commands, but I'm working on that currently) we can easily figure out if > the device is busy by checking for an empty tag map. That sounds like a reasonable starting solution. Thanks, John > > Should be doable for most modern HBAs. > > Cheers, > > Hannes