Re: [POT] Linux SAN?

Re: [POT] Linux SAN?

[James Bottomley]

> The FC HBA driver put out by Qlogic works well but does a silly thing;
> it  enumerates devices from 0, instead of by the actually loop ID.
> This makes  it impossible to spec absolute paths to the device, as
> everything will shift when devices are moved on the FC loop.

There are several reasons why this is done:

Most modern SANs use soft loop ID which means that the ID is determined at 
login time to the SAN, so changes as the SAN composition changes, therefore 
the loop ID isn't really meaningful anyway.

FC drivers are coming around to the notion of persistent binding, which is 
where you try to identify your devices by WWN instead of loop ID.  This is 
usually implemented as a mapping function which assigns a known SCSI pun to a 
particular WWN regardless of the actual loop ID.

Version 5.x of the qla2x00 driver (in SuSE 7.3 and also on the IBM website but 
not the qlogic website [yet]) does arbitrated loop.  Now, since arbitrated 
loop has two or more paths to the device through different ports with possibly 
different loop IDs, which loop ID would you use as the "actual" one?

James Bottomley

Re: [POT] Linux SAN?

[Dave Cinege]

On Friday 05 October 2001 10:47, you wrote:
> > The FC HBA driver put out by Qlogic works well but does a silly thing;
> > it  enumerates devices from 0, instead of by the actually loop ID.
> > This makes  it impossible to spec absolute paths to the device, as
> > everything will shift when devices are moved on the FC loop.
>
> There are several reasons why this is done:

Note that there is clear distiction between devices that are on the *local 
loop* and those you obtain from fabric login. My bitch, and my patch, fixes 
the logical issue that a local loop devices requesting a hard ID, should be 
accessable by *that ID* from userland if it is available. Very simple, and 
what the FC spec requires, and what the QLogic HBA BIOS does. However the 
Qlogic Linux driver attempts to present all devices as found from unit 0. 
This makes no sense because there are 2 assignment loops in the QLA source,
the second one for the purposes of login/softid resolution. I fixed the first 
loop. The second loop still starts from zero and will catch any conflicts 
from the first...
 
As for persistent binding, I just couldn't make it work with any of the 
QLA2x00 drivers I tried. Their parsing code is just fubar. 
Persistent binding is also not much fun to keep track of, and can run you out 
of cmdline space *really* quick if you try to spec devices at boot.
For a small SAN, devices (bays) with hard id's are usually much more 
desirable.

-- 
The time is now 22:19 (Totalitarian)  -  http://www.ccops.org/clock.html

Re: [POT] Linux SAN?

[Dave Cinege]

On Wednesday 03 October 2001 16:21, you wrote:
> This is fairly off-topic, but would you be willing to describe your
> SAN? I'm getting interested in Linux SAN solutions, and it sounds like
> you've successfully deployed one. If you don't have time, that's fine.

Not much to describe. It's quite straight forward. 3 servers connected to a 
fibre channel hub, connected to a JBOD built using my own product:
	http://www.cinonic.com/

I'm not sharing file systems between host transparently. To do that requires 
an FS like GFS (made by the same guys that handle linux LVM)

All hosts must mirror a single raidtab file for coherency, and you can't use 
things like autoraid starting, as hosts will step on  each others toes. (Use 
md= at cmdline for booting raids) You also want to use devfs in order to keep 
your sanity and access the many devices by actually bus/lun/id and not an 
obsure changing letter. 

If you're not familar with Fibre Channel, it's basically serial SCSI with 
long cable lengths and 126ID's.

FC Drives and HBA's are dirt cheap on ebay right now...have fun and buy soem 
FC2's   : >

Dave

-- 
The time is now 22:19 (Totalitarian)  -  http://www.ccops.org/clock.html

Re: [POT] Linux SAN?

[Alan Cox]

> I'm not sharing file systems between host transparently. To do that requires 
> an FS like GFS (made by the same guys that handle linux LVM)

Originally made. They took their new version proprietary so the open one is now
a community project.

Re: [POT] Linux SAN?

[Dave Cinege]

On Thursday 04 October 2001 16:44, Alan Cox wrote:
> > I'm not sharing file systems between host transparently. To do that
> > requires an FS like GFS (made by the same guys that handle linux LVM)
>
> Originally made. They took their new version proprietary so the open one is
> now a community project.

I thought that was for GFS only, and they were keeping LVM GPL???

Dave

-- 
The time is now 22:19 (Totalitarian)  -  http://www.ccops.org/clock.html

Re: [POT] Linux SAN?

[Alan Cox]

> On Thursday 04 October 2001 16:44, Alan Cox wrote:
> > > I'm not sharing file systems between host transparently. To do that
> > > requires an FS like GFS (made by the same guys that handle linux LVM)
> >
> > Originally made. They took their new version proprietary so the open one is
> > now a community project.
> 
> I thought that was for GFS only, and they were keeping LVM GPL???

I'm told LVM is staying GPL. 

Re: [POT] Linux SAN?

[Dave Cinege]

On Thursday 04 October 2001 22:31, you wrote:

> For the HBA ( fibre channel adapter ) i use the module in the kernel CPQFC
> is for a specific compaq HBA, but you can use use QLOGIC module or EMULEX (
> but the EMULEX driver is not under GPL and you don't have source and it's
> really convinient for the correction on the driver ) with support for FC.

FYI to all interested in playing with Fibre Channel:
The FC HBA driver put out by Qlogic works well but does a silly thing; it 
enumerates devices from 0, instead of by the actually loop ID. This makes 
it impossible to spec absolute paths to the device, as everything will
shift when devices are moved on the FC loop.

This one liner patch to qla2x00.c fixes this problem, by enumerating
by loop id.

ftp://ftp.psychosis.com/linux/fibrechannel/qla_fixid.c

-- 
The time is now 22:19 (Totalitarian)  -  http://www.ccops.org/clock.html

Re: [POT] Linux SAN?

[Wilson]

> On Thursday 04 October 2001 22:31, someone wrote:
>
> > For the HBA ( fibre channel adapter ) i use the module in the kernel
CPQFC
> > is for a specific compaq HBA, but you can use use QLOGIC module or
EMULEX (
> > but the EMULEX driver is not under GPL and you don't have source and
it's
> > really convinient for the correction on the driver ) with support for
FC.
>

Is anyone aware of a project (now that LinuxDisk is gone..) that has the
potential to turn a Linux box into one of these?
I'd like to have a Linux machine running as a "SAN appliance" with a
heterogenous mix of servers (with FC cards) booting from "virtual" slices of
one big array.
I'm envisioning a "roll your own" version of this Winchester Flaskdisk
product:
http://www.winsys.com/products/

Regards,
--Wilson.