From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757520Ab0IXVRK (ORCPT ); Fri, 24 Sep 2010 17:17:10 -0400 Received: from smtp103.sbc.mail.ac4.yahoo.com ([76.13.13.242]:21840 "HELO smtp103.sbc.mail.ac4.yahoo.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1752423Ab0IXVRI (ORCPT ); Fri, 24 Sep 2010 17:17:08 -0400 X-Yahoo-SMTP: fzDSGlOswBCWnIOrNw7KwwK1j9PqyNbe5PtLKiS4dDU.UNl_t6bdEZu9tTLW X-YMail-OSG: EMuZn7kVM1lKwYvFsqTF8VXteQFNPRLAGhS5U2JUEtwKiTX gePO74YZMY2zdxblPGzC2z6BHqxzaRmpOlbjz_kbCEMyNesm1wZbqiU2MZuG X82wg4Hc.Tg8hZTGzuPON_eTCI1egHW3eOeLZoT9fbmBvcC1Gp3_SpcXb1bs fhZvHAln2nwiGDB0EM0lCNZ7JeUYxPngRqcL2f9S2lMD8aCaxizNNhKgZmWd uGTfZmnD8JiLxovpT1AI- X-Yahoo-Newman-Property: ymail-3 Subject: Re: [RFC v3 01/15] scsi: Drop struct Scsi_Host->host_lock usage in scsi_dispatch_cmd() From: "Nicholas A. Bellinger" To: Brian King Cc: linux-scsi , linux-kernel , Vasu Dev , Tim Chen , Andi Kleen , Matthew Wilcox , James Bottomley , Mike Christie , James Smart , Andrew Vasquez , FUJITA Tomonori , Hannes Reinecke , Joe Eykholt , Christoph Hellwig , MPTFusionLinux , "eata.c maintainer" In-Reply-To: <4C9D13B6.5080404@linux.vnet.ibm.com> References: <1285285052-16351-1-git-send-email-nab@linux-iscsi.org> <4C9CAA80.1030702@linux.vnet.ibm.com> <1285361040.1849.235.camel@haakon2.linux-iscsi.org> <4C9D13B6.5080404@linux.vnet.ibm.com> Content-Type: text/plain Date: Fri, 24 Sep 2010 14:12:49 -0700 Message-Id: <1285362769.1849.256.camel@haakon2.linux-iscsi.org> Mime-Version: 1.0 X-Mailer: Evolution 2.22.3.1 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 2010-09-24 at 16:10 -0500, Brian King wrote: > On 09/24/2010 03:44 PM, Nicholas A. Bellinger wrote: > > On Fri, 2010-09-24 at 08:41 -0500, Brian King wrote: > >> On 09/23/2010 06:37 PM, Nicholas A. Bellinger wrote: > >>> @@ -651,7 +655,6 @@ static inline void scsi_cmd_get_serial(struct Scsi_Host *host, struct scsi_cmnd > >>> int scsi_dispatch_cmd(struct scsi_cmnd *cmd) > >>> { > >>> struct Scsi_Host *host = cmd->device->host; > >>> - unsigned long flags = 0; > >>> unsigned long timeout; > >>> int rtn = 0; > >>> > >>> @@ -736,15 +739,11 @@ int scsi_dispatch_cmd(struct scsi_cmnd *cmd) > >>> scsi_done(cmd); > >>> goto out; > >>> } > >>> - > >>> - spin_lock_irqsave(host->host_lock, flags); > >>> /* > >>> - * AK: unlikely race here: for some reason the timer could > >>> - * expire before the serial number is set up below. > >>> - * > >>> - * TODO: kill serial or move to blk layer > >>> + * Note that scsi_cmd_get_serial() used to be called here, but > >>> + * now we expect the legacy SCSI LLDs that actually need this > >>> + * to call it directly within their SHT->queuecommand() caller. > >>> */ > >>> - scsi_cmd_get_serial(host, cmd); > >>> > >>> if (unlikely(host->shost_state == SHOST_DEL)) { > >>> cmd->result = (DID_NO_CONNECT << 16); > >>> @@ -753,7 +752,7 @@ int scsi_dispatch_cmd(struct scsi_cmnd *cmd) > >>> trace_scsi_dispatch_cmd_start(cmd); > >>> rtn = host->hostt->queuecommand(cmd, scsi_done); > >>> } > >>> - spin_unlock_irqrestore(host->host_lock, flags); > >>> + > >>> if (rtn) { > >>> trace_scsi_dispatch_cmd_error(cmd, rtn); > >>> if (rtn != SCSI_MLQUEUE_DEVICE_BUSY && > >> > >> Are you planning a future revision that moves the acquiring of the host lock > >> into the LLDD's queuecommand for all the other drivers you don't currently > >> touch in this patch set? > >> > > > > Hi Brian, > > > > I was under the impression that this would be unnecessary for the vast > > majority of existing LLD drivers, but if you are aware of specific LLDs > > that would still need the struct Scsi_Host->host_lock held in their > > SHT->queuecommand() for whaterver reason please let me know and I would > > be happy to include this into an RFCv4. > > I would think that most drivers might have issues without some pretty careful > auditing. When Christoph did this for the EH handlers, the first step was to > simply move acquiring the host lock into the LLDs. That way we can optimize > drivers one at a time after ensuring they can run lockless in their queuecommand > handler. > > A couple examples of possible issues with drivers I'm familiar with (ibmvfc, ipr): > > * Some drivers will do list manipulation for resources needed to send commands. If > done lockless, this could result in list corruption with multiple readers/writers. > > * Some drivers check the state of the hardware before sending a command. Failing to > do this when the hardware is being reset may result in nasty things like PCI bus > errors or even sending a command to the wrong device. > Indeed, I can very much see some older LLDs making these types of assumptions. > These are all the sorts of errors that will be very difficult to hit but have > pretty bad consequence when they are hit. > I think pretty bad would be an under-statement when running into either of the above two items in ancient LLD code. In that case, I will re-spin a v4 series that contains a legacy host_lock held w/ SHT->queuecomand() for all of the "historically high host_lock optimized in ->queuecommand()" LLDs that are in RFCv3, and include the other specific ones (namely mpt/fusion and mpt2sas) that we know are safe to drop host_lock. Many thanks for your invaluable input on some of the non-obvious items at play here. Best, --nab