3ware Escalade problems

3ware Escalade problems

[Scott Ransom]

Hello,

After months of running a fileserver with an 8 port 3ware escalade card
(kernels 2.4.[3457] using reiserfs and software RAID5) I started getting
problems this weekend.

Over the last three days, when I try to access the drives, after a
couple minutes I get a drive failure (I even heard a "yelp" from the
drive during one of them...).  But the "failure" has happened to 3 of
the 8 drives over 3 days -- so unless there is a hardware problem that
is killing my drives I find it hard to believe that 3 drives really and
truly failed....

Here is a sample from my syslog of a failure:

3w-xxxx: tw_interrupt(): Bad response, status = 0xc7, flags = 0x51, unit
= 0x1.
3w-xxxx: tw_scsi_eh_reset(): Reset succeeded for card 1.
3w-xxxx: tw_interrupt(): Bad response, status = 0xc7, flags = 0x51, unit
= 0x1.
scsi: device set offline - not ready or command retry failed after host
reset: host 1 channel 0 id 1 lun 0
SCSI disk error : host 1 channel 0 id 1 lun 0 return code = 80000
I/O error: dev 08:11, sector 158441712

I've noticed several "issues" with the 3ware cards in the archives.  Has
anyone seen something like this?

Scott

PS:  I'm currently running 2.4.7 with the lm-sensors/i2c patches.

-- 
Scott M. Ransom                   Address:  Harvard-Smithsonian CfA
Phone:  (617) 496-7908                      60 Garden St.  MS 10 
email:  ransom@cfa.harvard.edu              Cambridge, MA  02138
GPG Fingerprint: 06A9 9553 78BE 16DB 407B  FFCA 9BFA B6FF FFD3 2989

Re: 3ware Escalade problems

[Jeff V. Merkey]

On Wed, Aug 01, 2001 at 02:14:44PM -0400, Scott Ransom wrote:


I am also using 8 way escalade adapters, and am seeing a host of problems.
The first and foremore is that the gendisk head in 2.4.X is not being 
initialized properly in the driver.  I have reported these problems to
3-ware, and they are attmepting to get the engineer who owns the drivers
on the line with us.  The problems you are seeing are probably related 
to the same bugs.  This driver requires some rework to get it compliant
with 2.4.X.  At present, several programs fail with it since is is not 
setting up the gendisk head properly.  I do not know if your
problem is related, but this one will get added to the list when I speak 
with this person.

Jeff


> Hello,
> 
> After months of running a fileserver with an 8 port 3ware escalade card
> (kernels 2.4.[3457] using reiserfs and software RAID5) I started getting
> problems this weekend.
> 
> Over the last three days, when I try to access the drives, after a
> couple minutes I get a drive failure (I even heard a "yelp" from the
> drive during one of them...).  But the "failure" has happened to 3 of
> the 8 drives over 3 days -- so unless there is a hardware problem that
> is killing my drives I find it hard to believe that 3 drives really and
> truly failed....
> 
> Here is a sample from my syslog of a failure:
> 
> 3w-xxxx: tw_interrupt(): Bad response, status = 0xc7, flags = 0x51, unit
> = 0x1.
> 3w-xxxx: tw_scsi_eh_reset(): Reset succeeded for card 1.
> 3w-xxxx: tw_interrupt(): Bad response, status = 0xc7, flags = 0x51, unit
> = 0x1.
> scsi: device set offline - not ready or command retry failed after host
> reset: host 1 channel 0 id 1 lun 0
> SCSI disk error : host 1 channel 0 id 1 lun 0 return code = 80000
> I/O error: dev 08:11, sector 158441712
> 
> I've noticed several "issues" with the 3ware cards in the archives.  Has
> anyone seen something like this?
> 
> Scott
> 
> PS:  I'm currently running 2.4.7 with the lm-sensors/i2c patches.
> 
> -- 
> Scott M. Ransom                   Address:  Harvard-Smithsonian CfA
> Phone:  (617) 496-7908                      60 Garden St.  MS 10 
> email:  ransom@cfa.harvard.edu              Cambridge, MA  02138
> GPG Fingerprint: 06A9 9553 78BE 16DB 407B  FFCA 9BFA B6FF FFD3 2989
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: 3ware Escalade problems

[Alan Cox]

> I am also using 8 way escalade adapters, and am seeing a host of problems.

I've seen no problems since the 1.02.00.005 driver. 

> The first and foremore is that the gendisk head in 2.4.X is not being 
> initialized properly in the driver.  I have reported these problems to

The gendisk comes from the scsi midlayer so you want linux-scsi for that

Re: 3ware Escalade problems

[Jeff V. Merkey]

On Thu, Aug 02, 2001 at 01:26:22AM +0100, Alan Cox wrote:
> > I am also using 8 way escalade adapters, and am seeing a host of problems.
> 
> I've seen no problems since the 1.02.00.005 driver. 
> 
> > The first and foremore is that the gendisk head in 2.4.X is not being 
> > initialized properly in the driver.  I have reported these problems to
> 
> The gendisk comes from the scsi midlayer so you want linux-scsi for that

Alan,

Try putting four adapters into a system all at once with 32 drives, and 
you will see all sorts of bugs.  I do not see problems with a single board,
other than gendisk reporting junk.  If it's the scsi layer, then the driver 
must not be calling the sd driver.  I will attempt to get on the phone with
Adam, and get these issues resolved.

Jeff

Re: 3ware Escalade problems

[Alan Cox]

> Try putting four adapters into a system all at once with 32 drives, and 
> you will see all sorts of bugs.  I do not see problems with a single board,
> other than gendisk reporting junk.  If it's the scsi layer, then the driver 
> must not be calling the sd driver.  I will attempt to get on the phone with
> Adam, and get these issues resolved.

The scsi disk layer does the entire gendisk stuff itself. Its actually
very hard for a scsi driver to screw that up - the scsi driver has no
real concept of a 'disk'. sd talks to a disk and the scsi driver just gets
messages posted around.

Alan

Re: 3ware Escalade problems

[Jeff V. Merkey]

On Thu, Aug 02, 2001 at 01:40:55AM +0100, Alan Cox wrote:
> > Try putting four adapters into a system all at once with 32 drives, and 
> > you will see all sorts of bugs.  I do not see problems with a single board,
> > other than gendisk reporting junk.  If it's the scsi layer, then the driver 
> > must not be calling the sd driver.  I will attempt to get on the phone with
> > Adam, and get these issues resolved.
> 
> The scsi disk layer does the entire gendisk stuff itself. Its actually
> very hard for a scsi driver to screw that up - the scsi driver has no
> real concept of a 'disk'. sd talks to a disk and the scsi driver just gets
> messages posted around.
> 
> Alan

Alan,

If you have an adapter, install it, dumop and gendisk head, and take a 
look at what's happening.  I am seeing drives being reported with 0 
block lengths and other wierdness.    

Jeff

Re: 3ware Escalade problems

[Alan Cox]

> If you have an adapter, install it, dumop and gendisk head, and take a 
> look at what's happening.  I am seeing drives being reported with 0 
> block lengths and other wierdness.    

Looks fine to me. However if you are seeing 0 block length drives reported
thats tw_scsiop_read_capacity_complete() causing the sd.c code to do
something daft.

Alan

Re: 3ware Escalade problems

[Jeff V. Merkey]

On Thu, Aug 02, 2001 at 01:22:15PM +0100, Alan Cox wrote:
> > If you have an adapter, install it, dumop and gendisk head, and take a 
> > look at what's happening.  I am seeing drives being reported with 0 
> > block lengths and other wierdness.    
> 
> Looks fine to me. However if you are seeing 0 block length drives reported
> thats tw_scsiop_read_capacity_complete() causing the sd.c code to do
> something daft.
> 
> Alan

Alan,

Thanks.  I will look at this.  I am running into this problem on 2.2.X 
kernels more so than 2.4.X, but occasionally see it on 2.4.X.  It seems
related to RAID recovery during testing.  We noticed it when we were 
triggering RAID failover for testing.

Jeff

Re: 3ware Escalade problems

[Jeff V. Merkey]

Alan,

On a related problem, we are seeing packet checksum corruption with both
the 3Com and Intel Gigabit ethernet adapters when a ping -f -s > 1472 packet
fllods are initiated.  The sniffers report packets with corrupted checksum
information.  I am copying Larry Angus on this email since he has all 
the sniffer traces.  These problems show up on 2.2 and 2.4 kernels.

We noticed several emails in the archive regarding this problem, but 
no fixes reported.  What's disturbing regarding this issue is that 
hardware checksum errors are being generated which results in the 
packets being lost.

Larry - please provide the traces and documentation you have discovered
to Alan regarding this problem.

Jeff

Re: 3ware Escalade problems? Adaptec?

[Michael Rothwell]

I've been pricing out a 3ware-based raid system for my own personal use. Are
the problems wuth the Escalade cards bad enough to consider not using them
with 2.4.7?

It seems that the only ATA-raid cards supported in the kernel tree are the
3ware ones. What's the word on the Adaptec 2400A card?

Thanks,

-M

RE: 3ware Escalade problems? Adaptec?

[Ryan C. Bonham]

I am using an adaptec 2400A card currently. I have 2.4.6 kernel compiled and
the adaptec 2400A driver patched in. I have had no problems with this card.

> -----Original Message-----
> From: Michael Rothwell [mailto:rothwell@holly-springs.nc.us]
> Sent: Thursday, August 02, 2001 10:20 AM
> To: linux-kernel@vger.kernel.org
> Subject: Re: 3ware Escalade problems? Adaptec?
> 
> 
> I've been pricing out a 3ware-based raid system for my own 
> personal use. Are
> the problems wuth the Escalade cards bad enough to consider 
> not using them
> with 2.4.7?
> 
> It seems that the only ATA-raid cards supported in the kernel 
> tree are the
> 3ware ones. What's the word on the Adaptec 2400A card?
> 
> Thanks,
> 
> -M
> 
> -
> To unsubscribe from this list: send the line "unsubscribe 
> linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: 3ware Escalade problems? Adaptec?

[Alan Cox]

> I've been pricing out a 3ware-based raid system for my own personal use. Are
> the problems wuth the Escalade cards bad enough to consider not using them
> with 2.4.7?

Im really attached to my 3ware cards, they are the best ide raid cards I've
used. The newer boxes I built just use software raid 0/1 which is easy now
that everyone throws 4 UDMA100 channels on their motherboards.

I've also done the i2o driver fixups for the Promise SuperTrak100 with a
card provided by Promise and that works in -ac but not yet Linus tree.
I'm more impressed with the 3ware than the promise card right now, although
it will depend on workload. The promise card has onboard caches and raid5 
hardware which the earlier 3ware didn't.