Re: 2.5.62-mm2 slow file system writes across multiple disks

[Patrick Mansfield <patmans@us.ibm.com>]

On Wed, Feb 26, 2003 at 12:44:54AM -0800, Andrew Morton wrote:

> Does the other qlogic driver exhibit the same thing?

OK I finally tried out the qlogic driver on the same 10 drives, actually
with scsi-misc-2.5 (2.5.63).

The qlogic is OK performance wise - as Mike pointed out, it sets a lower
queue depth; and even though it sets can_queue higher than the feral, the
qlogic driver software queues where in the same case the feral would give
us a host busy (i.e. queuecomand returns 1).

andmike> Well the qlogic provided driver should exhibit slightly different
andmike> behavior as its per device queue depth is 16 and the request ring count
andmike> is 128.
andmike> 
andmike> The feral driver is currently running a per device queue of 63 and a
andmike> request ring size of 64. (if I am reading the driver correctly?). When
andmike> the request count is exceeded I believe it should return a 1 to the call
andmike> of queuecommand. Can you tell if scsi_queue_insert is being called.

As Mike implies, the feral driver is setting can_queue too high, so in
addition to large queue depth affects, I am also hitting scsi host busy
code paths - yes, it is calling scsi_queue_insert. The host busy code is
not meant to be hit so often, and likely leads to lower performance.

So the feral driver needs lower can_queue (and/or queueing changes) and
lower queue_depth limits.

> Does writeout to a single disk exhibit the same thing?

No, single disk IO performance is OK (with queue depth/TCQ 63 and
can_queue 744), so the too high can_queue with host busy's is probably
hurting performance than the high queue_depth.

> > The larger queue depths can be nice for disk arrays with lots of cache and
> > (more) random IO patterns.
> 
> So says the scsi lore ;)  Have you observed this yourself?  Have you
> any numbers handy?

No and no :(

I'm not sure if the disk arrays we have available have enough memory to
show such affects (I assume standard disk caches are not large enough to
have much of an affect).

-- Patrick Mansfield