Re: Re: Re: BUG in kernel: Wrong Handling of USB HDD’s in scsiglue(slave_configure) and scsi/sd(sd_read_cache_type)

Re: Re: Re: BUG in kernel: Wrong Handling of USB HDD’s in scsiglue(slave_configure) and scsi/sd(sd_read_cache_type)

[Alan Stern]

On Tue, 4 Oct 2011, Amit Sahrawat wrote:

> Not able to get tools which can show the Caching Information when
> connected USB device to Windows 7(Except one option which allows for
> optimizing Performance by enabling/disabling Cache - But that option
> is available for all the Mass storage device) - Please let me know if
> there is option to view this.

I have no idea.

> So, I have collected USB analyzer logs for Device after connected to
> Windows 7.  Please find the attached logs.

It would be more informative if your logs did not show the initial
connection, but did show what happens when you write a small file to
the drive and then go through the "Safely Remove Hardware" procedure.

The real question is whether or not Windows sends a SYNCHRONIZE CACHE 
command to the drive.

Alan Stern

Re: Re: Re: BUG in kernel: Wrong Handling of USB HDD’s in scsiglue(slave_configure) and scsi/sd(sd_read_cache_type)

[Amit Sahrawat]

On Tue, Oct 4, 2011 at 7:40 PM, Alan Stern <stern@rowland.harvard.edu> wrote:
> On Tue, 4 Oct 2011, Amit Sahrawat wrote:
>
>> Not able to get tools which can show the Caching Information when
>> connected USB device to Windows 7(Except one option which allows for
>> optimizing Performance by enabling/disabling Cache - But that option
>> is available for all the Mass storage device) - Please let me know if
>> there is option to view this.
>
> I have no idea.
>
>> So, I have collected USB analyzer logs for Device after connected to
>> Windows 7.  Please find the attached logs.
>
> It would be more informative if your logs did not show the initial
> connection, but did show what happens when you write a small file to
> the drive and then go through the "Safely Remove Hardware" procedure.
>
> The real question is whether or not Windows sends a SYNCHRONIZE CACHE
> command to the drive.
ok, I will try this out and provide the results tomorrow early morning.
>
> Alan Stern
>
>
Thanks & Regards,
Amit Sahrawat

Re: Re: Re: Re: BUG in kernel: Wrong Handling of USB HDD’s in scsiglue(slave_configure) and scsi/sd(sd_read_cache_ =?windows-1252?Q?type=29

[Alan Stern]

On Wed, 5 Oct 2011, Amit Sahrawat wrote:

> Hi,
> 
> In order to verify the SYNC_CACHE command being passed.
> First I checked on the USB HDD(Seagate - Free Agent) which is gettind
> properly identified on linux. I performed - copy of a file and then
> did 'Safely Remove Hardware" - this resulted in SYNC_CACHE command
> being passed - All logs are captured for this.
> 
> Then, for the same Hard Disk - I checked for the behaviour on Windows.
> There is no SYNC CACHE command being passed in this case.
> 
> Again, checked for a different hard disk(Seagate) - which has cache
> but not shown in Linux. In that case also, there is no SYNC CACHE
> command.
> 
> Please find the logs for the '3' cases attached.

This is a little disappointing.  If Windows doesn't use the SYNCHRONIZE 
CACHE command, it's quite likely that some USB drives will crash when 
they receive it.

This strongly suggests that we should not change the current code.

Alan Stern

Re: Re: Re: Re: BUG in kernel: Wrong Handling of USB HDD’s in scsiglue(slave_configure) and scsi/sd(sd_read_cache_ =?windows-1252?Q?type=29

[James Bottomley]

On Wed, 2011-10-05 at 10:17 -0400, Alan Stern wrote:
> On Wed, 5 Oct 2011, Amit Sahrawat wrote:
> 
> > Hi,
> > 
> > In order to verify the SYNC_CACHE command being passed.
> > First I checked on the USB HDD(Seagate - Free Agent) which is gettind
> > properly identified on linux. I performed - copy of a file and then
> > did 'Safely Remove Hardware" - this resulted in SYNC_CACHE command
> > being passed - All logs are captured for this.
> > 
> > Then, for the same Hard Disk - I checked for the behaviour on Windows.
> > There is no SYNC CACHE command being passed in this case.
> > 
> > Again, checked for a different hard disk(Seagate) - which has cache
> > but not shown in Linux. In that case also, there is no SYNC CACHE
> > command.
> > 
> > Please find the logs for the '3' cases attached.
> 
> This is a little disappointing.  If Windows doesn't use the SYNCHRONIZE 
> CACHE command, it's quite likely that some USB drives will crash when 
> they receive it.
> 
> This strongly suggests that we should not change the current code.

I agree ... it means they go by the SCSI mode pages too, so we'll be no
worse than they are.  Plus we don't risk upsetting the rest of the USB
device universe.

James

Re: Re: Re: BUG in kernel: Wrong Handling of USB HDD’s in scsiglue(slave_configure) and scsi/sd(sd_read_cache_type)

[Amit Sahrawat]

Hi All,
Can anyone suggest what is the way ahead to detect the presence of
Write Cache in these kind of USB HDD and proceed accordingly by
assigning proper flushing method.
Please share some opinions on this.

Thanks & Regards,
Amit Sahrawat

On Wed, Oct 5, 2011 at 3:05 PM, Amit Sahrawat <amit.sahrawat83@gmail.com> wrote:
> Hi,
>
> In order to verify the SYNC_CACHE command being passed.
> First I checked on the USB HDD(Seagate - Free Agent) which is gettind
> properly identified on linux. I performed - copy of a file and then
> did 'Safely Remove Hardware" - this resulted in SYNC_CACHE command
> being passed - All logs are captured for this.
>
> Then, for the same Hard Disk - I checked for the behaviour on Windows.
> There is no SYNC CACHE command being passed in this case.
>
> Again, checked for a different hard disk(Seagate) - which has cache
> but not shown in Linux. In that case also, there is no SYNC CACHE
> command.
>
> Please find the logs for the '3' cases attached.
>
> Thanks & Regards,
> Amit Sahrawat
>
> On Tue, Oct 4, 2011 at 9:04 PM, Amit Sahrawat <amit.sahrawat83@gmail.com> wrote:
>> On Tue, Oct 4, 2011 at 7:40 PM, Alan Stern <stern@rowland.harvard.edu> wrote:
>>> On Tue, 4 Oct 2011, Amit Sahrawat wrote:
>>>
>>>> Not able to get tools which can show the Caching Information when
>>>> connected USB device to Windows 7(Except one option which allows for
>>>> optimizing Performance by enabling/disabling Cache - But that option
>>>> is available for all the Mass storage device) - Please let me know if
>>>> there is option to view this.
>>>
>>> I have no idea.
>>>
>>>> So, I have collected USB analyzer logs for Device after connected to
>>>> Windows 7.  Please find the attached logs.
>>>
>>> It would be more informative if your logs did not show the initial
>>> connection, but did show what happens when you write a small file to
>>> the drive and then go through the "Safely Remove Hardware" procedure.
>>>
>>> The real question is whether or not Windows sends a SYNCHRONIZE CACHE
>>> command to the drive.
>> ok, I will try this out and provide the results tomorrow early morning.
>>>
>>> Alan Stern
>>>
>>>
>> Thanks & Regards,
>> Amit Sahrawat
>>
>

Re: Re: Re: BUG in kernel: Wrong Handling of USB HDD’s in scsiglue(slave_configure) and scsi/sd(sd_read_cache_type)

[James Bottomley]

On Wed, 2011-10-05 at 22:29 +0530, Amit Sahrawat wrote:
> Can anyone suggest what is the way ahead to detect the presence of
> Write Cache in these kind of USB HDD and proceed accordingly by
> assigning proper flushing method.
> Please share some opinions on this.

We're a bit out of ideas.  We've already established that we do exactly
what windows does in this situation, which is usually what we aim for.
Realistically, if the disks lie how do we know when not to believe them?

The best we could probably offer is an interface to turn on the WCE bit
in software (technically, you can do this today
in /scsi/class/scsi_disk/<disk>/cache_type, it's just that it will try
to commit the change as a MODE_SELECT which will presumably fail).

James

Re: Re: Re: BUG in kernel: Wrong Handling of USB HDD’s in scsiglue(slave_configure) and scsi/sd(sd_read_cache_type)

[Amit Sahrawat]

On Wed, Oct 5, 2011 at 10:54 PM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Wed, 2011-10-05 at 22:29 +0530, Amit Sahrawat wrote:
>> Can anyone suggest what is the way ahead to detect the presence of
>> Write Cache in these kind of USB HDD and proceed accordingly by
>> assigning proper flushing method.
>> Please share some opinions on this.
>
> We're a bit out of ideas.  We've already established that we do exactly
> what windows does in this situation, which is usually what we aim for.
> Realistically, if the disks lie how do we know when not to believe them?
I never knew this could turn out to be an issue with such history. But
atleast debugging helped in diagnosis.
>
> The best we could probably offer is an interface to turn on the WCE bit
> in software (technically, you can do this today
> in /scsi/class/scsi_disk/<disk>/cache_type, it's just that it will try
> to commit the change as a MODE_SELECT which will presumably fail).
Thanks James, is this related with SCSI command MODE_SELECT? and this
is to be passed when there is some failure?
or simply doing an "echo <value>" to
/scsi/class/scsi_disk/<disk>/cache_type? can you please elaborate a
little
>
> James
>
Thanks & Regards,
Amit Sahrawat

Re: Re: Re: BUG in kernel: Wrong Handling of USB HDD’s in scsiglue(slave_configure) and scsi/sd(sd_read_cache_type)

[James Bottomley]

On Thu, 2011-10-06 at 01:21 +0530, Amit Sahrawat wrote:
> > The best we could probably offer is an interface to turn on the WCE bit
> > in software (technically, you can do this today
> > in /scsi/class/scsi_disk/<disk>/cache_type, it's just that it will try
> > to commit the change as a MODE_SELECT which will presumably fail).
> Thanks James, is this related with SCSI command MODE_SELECT? and this
> is to be passed when there is some failure?
> or simply doing an "echo <value>" to
> /scsi/class/scsi_disk/<disk>/cache_type? can you please elaborate a
> little

You tell me since you have the device.  What that echo does is that it
does try to make the change permanent with a mode select ... that likely
won't work and the cache change only takes if the revalidated disk says
the write back has been enabled (which I really think it won't).  So I
think you need an additional software bit to flip for the case where the
device lies.

James

Re: Re: Re: BUG in kernel: Wrong Handling of USB HDD’s in scsiglue(slave_configure) and scsi/sd(sd_read_cache_type)

[Amit Sahrawat]

Trying to set the cache type as "write back" through "echo "write
back" > /sys/class/scsi_disk/<disk>/cache_type" does not work.
Few logs :
#> usb 2-1.4: new high speed USB device using ehci-sdp and address 5
usb 2-1.4: New USB device found, idVendor=152d, idProduct=2339
usb 2-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=5
usb 2-1.4: Product: USB to ATA/ATAPI Bridge
usb 2-1.4: Manufacturer: JMicron
usb 2-1.4: SerialNumber: 3446184AA01C
scsi1 : usb-storage 2-1.4:1.0
scsi 1:0:0:0: Direct-Access     SAMSUNG  HM501IX               PQ: 0 ANSI: 2 CCS
sd 1:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB)
sd 1:0:0:0: [sda] Write Protect is off
sd 1:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't
support DPO or FUA
 sda:

#> echo "write back" > /sys/class/scsi_disk/1\:0\:0\:0/cache_type
sd 1:0:0:0: [sda] Sense Key : 0x5 [current]
sd 1:0:0:0: [sda] ASC=0x20 ASCQ=0x0


#> usb 2-1.4: new high speed USB device using ehci-sdp and address 4
usb 2-1.4: New USB device found, idVendor=0bc2, idProduct=2300
usb 2-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 2-1.4: Product: Portable
usb 2-1.4: Manufacturer: Seagate
usb 2-1.4: SerialNumber: 2GHW02GR
scsi0 : usb-storage 2-1.4:1.0

#>
#> scsi 0:0:0:0: Direct-Access     Seagate  Portable         0130 PQ: 0 ANSI: 4
sd 0:0:0:0: [sda] 488397168 512-byte logical blocks: (250 GB/232 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] No Caching mode page present
sd 0:0:0:0: [sda] Assuming drive cache: write through
sd 0:0:0:0: [sda] No Caching mode page present
sd 0:0:0:0: [sda] Assuming drive cache: write through
 sda: sda1 sda2 sda3 sda4
sd 0:0:0:0: [sda] No Caching mode page present
sd 0:0:0:0: [sda] Assuming drive cache: write through
sd 0:0:0:0: [sda] Attached SCSI disk

#> echo "write back" > /sys/class/scsi_disk/0\:0\:0\:0/cache_type
sd 0:0:0:0: [sda] No Caching mode page present
sd 0:0:0:0: [sda] Assuming drive cache: write through
#>


#> usb 2-1.4: new high speed USB device using ehci-sdp and address 7
usb 2-1.4: New USB device found, idVendor=1058, idProduct=070a
usb 2-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 2-1.4: Product: My Passport 070A
usb 2-1.4: Manufacturer: Western Digital
usb 2-1.4: SerialNumber: 575832314132304534373635
scsi3 : usb-storage 2-1.4:1.0
scsi 3:0:0:0: Direct-Access     WD       My Passport 070A 1032 PQ: 0 ANSI: 4
scsi 3:0:0:1: CD-ROM            WD       Virtual CD 070A  1032 PQ: 0 ANSI: 4
scsi 3:0:0:2: Enclosure         WD       SES Device       1032 PQ: 0 ANSI: 4
sd 3:0:0:0: [sda] 623769600 512-byte logical blocks: (319 GB/297 GiB)
sd 3:0:0:0: [sda] Write Protect is off
sd 3:0:0:0: [sda] No Caching mode page present
sd 3:0:0:0: [sda] Assuming drive cache: write through
sd 3:0:0:0: [sda] No Caching mode page present
sd 3:0:0:0: [sda] Assuming drive cache: write through
 sda:
sd 3:0:0:0: [sda] No Caching mode page present
sd 3:0:0:0: [sda] Assuming drive cache: write through
sd 3:0:0:0: [sda] Attached SCSI disk

#> cat /sys/class/scsi_disk/3\:0\:0\:0/cache_type
write through
#> echo "write back" > /sys/class/scsi_disk/3\:0\:0\:0/cache_type
#>
#> echo "write back" > /sys/class/scsi_disk/3\:0\:0\:0/cache_type
#>
#>
#> cat /sys/class/scsi_disk/3\:0\:0\:0/cache_type
write through

Regards,
Amit Sahrawat

On Thu, Oct 6, 2011 at 1:38 AM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Thu, 2011-10-06 at 01:21 +0530, Amit Sahrawat wrote:
>> > The best we could probably offer is an interface to turn on the WCE bit
>> > in software (technically, you can do this today
>> > in /scsi/class/scsi_disk/<disk>/cache_type, it's just that it will try
>> > to commit the change as a MODE_SELECT which will presumably fail).
>> Thanks James, is this related with SCSI command MODE_SELECT? and this
>> is to be passed when there is some failure?
>> or simply doing an "echo <value>" to
>> /scsi/class/scsi_disk/<disk>/cache_type? can you please elaborate a
>> little
>
> You tell me since you have the device.  What that echo does is that it
> does try to make the change permanent with a mode select ... that likely
> won't work and the cache change only takes if the revalidated disk says
> the write back has been enabled (which I really think it won't).  So I
> think you need an additional software bit to flip for the case where the
> device lies.
>
> James
>
>
>

Re: Re: Re: Re: BUG in kernel: Wrong Handling of USB HDD’s in scsiglue(slave_configure) and scsi/sd(sd_read_cache_ =?windows-1252?Q?type=29

[Alan Stern]

On Fri, 7 Oct 2011, Amit Sahrawat wrote:

> Trying to set the cache type as "write back" through "echo "write
> back" > /sys/class/scsi_disk/<disk>/cache_type" does not work.
> Few logs :
> #> usb 2-1.4: new high speed USB device using ehci-sdp and address 5
> usb 2-1.4: New USB device found, idVendor=152d, idProduct=2339
> usb 2-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=5
> usb 2-1.4: Product: USB to ATA/ATAPI Bridge
> usb 2-1.4: Manufacturer: JMicron
> usb 2-1.4: SerialNumber: 3446184AA01C
> scsi1 : usb-storage 2-1.4:1.0
> scsi 1:0:0:0: Direct-Access     SAMSUNG  HM501IX               PQ: 0 ANSI: 2 CCS
> sd 1:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB)
> sd 1:0:0:0: [sda] Write Protect is off
> sd 1:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't
> support DPO or FUA
>  sda:
> 
> #> echo "write back" > /sys/class/scsi_disk/1\:0\:0\:0/cache_type
> sd 1:0:0:0: [sda] Sense Key : 0x5 [current]
> sd 1:0:0:0: [sda] ASC=0x20 ASCQ=0x0

This is what James was talking about.  When you write something to 
/sys/class/scsi_disk/.../cache_type, the value you specify is sent to 
the drive using a MODE SELECT command.  However, if a USB drive doesn't 
report the caching type properly, it's not very likely to support the 
appropriate MODE SELECT command either.

What you need to do is create a variant of the cache_type sysfs 
attribute, one that will set or clear the WCE flag directly, without 
trying to communicate with the drive.

Alan Stern

Re: Re: Re: BUG in kernel: Wrong Handling of USB HDD’s in scsiglue(slave_configure) and scsi/sd(sd_read_cache_type)

[Alan Stern]

On Mon, 3 Oct 2011, Amit Sahrawat wrote:

> Hi,
> 
> Please find the USB Analyzer - logs for '6' USB based Mass Storage
> devices attached.

Too bad the analyzer log presents only the response, not the command.

> So, only for one of the USB HDD(Seagate - Free Agent), the response
> obtained is correct.

I don't understand the format of the Seagate Free Agent MODE SENSE
data.  Maybe someone else can interpret it; here it is for reference:

SCSI Op(3) ADDR(4) Tag(0x00000004) SCSI CDB MODE SENSE(6)
_______| Data(
__ 000: 1C 00 00 00 86 0B 00 02 00 00 12 A1 9E B0 3C 01 00 1A 0A 00 01 00 00 00 00 00 00 0B B8 00 00 00 
__ 032: 00 00 00 00 1A 0A 00 01 00 00 00 00 00 00 0B B8 A2 06 00 00 00 00 00 00 78 20 55 53 42 32 2E 30 
__ 064: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 7F FF 00 00 00 00 80 FA 0B C2 50 21 4E 41 30 42 
__ 096: 36 58 33 4C 00 30 00 00 00 32 43 46 45 39 42 00 70 F5 12 00 74 FF 00 00 00 00 00 00 00 00 00 00 
__ 128: 3C 00 00 0B B8 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 02 50 01 
__ 160: 46 72 65 65 41 67 65 6E 74 20 47 6F 46 6C 65 78 10 50 11 46 72 65 65 41 67 65 6E 74 20 47 6F 46)

For the other devices this seems clear enough.  In most of the cases,
the data doesn't include the Caching page.  The Samsung and Transcend
devices _do_ provide their Caching pages, and in both of them the data
says that write caching is disabled.  (In the Samsung case this is
wrong undoubtedly because of the JMicron bridge -- JMicron's firmware
is notoriously buggy.)

If you can suggest a robust way of determining whether or not a drive 
supports SAT (one which won't cause non-supporting devices to crash), 
it could be added.  In the meantime, I'm not sure what else we can do.

Would it make sense to assume write caching is present and enabled if 
the drive does not provide a Caching page?

Alan Stern