Syncing single filesystem (slow USB writing)

Syncing single filesystem (slow USB writing)

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 645 bytes --]

Mandrake always mounted USB sticks with sync option; it was effectively noop 
except for a patch that implemented limited dsync semantic.

Now, when full sync support for FATis in kernel, moutning with sync became 
real pain. Writing speed dropped from 3MB/s to 30KB/s in my case (and I am 
not alone).

One idea how to improve situation - continue to mount with dsync (having 
basically old case) and do frequent sync of filesystem (this culd be started 
as HAL callout or whatever). Unfortunately, I could not find a way to request 
a sync (flush) of single mount point or block device. Have I missed 
something?

TIA

-andrey

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Syncing single filesystem (slow USB writing)

[Andrew Morton]

Andrey Borzenkov <arvidjaar@mail.ru> wrote:
>
> Mandrake always mounted USB sticks with sync option; it was effectively noop 
> except for a patch that implemented limited dsync semantic.
> 
> Now, when full sync support for FATis in kernel, moutning with sync became 
> real pain. Writing speed dropped from 3MB/s to 30KB/s in my case (and I am 
> not alone).

Unfortunately I think we're just going to have to live with that.  It is
right that fatfs behaves as it does, and unfortunate that some distros will
operate slowly.

For reference: how does mandrake implement this?  Just in /etc/fstab?  How
should we tell other people to fix this?

> One idea how to improve situation - continue to mount with dsync (having 
> basically old case) and do frequent sync of filesystem (this culd be started 
> as HAL callout or whatever). Unfortunately, I could not find a way to request 
> a sync (flush) of single mount point or block device. Have I missed 
> something?

It's trivial to do in-kernel but no, I'm afraid there isn't a userspace
interface for this.

Re: Syncing single filesystem (slow USB writing)

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 1533 bytes --]

On Friday 29 July 2005 07:50, Andrew Morton wrote:
> Andrey Borzenkov <arvidjaar@mail.ru> wrote:
> > Mandrake always mounted USB sticks with sync option; it was effectively
> > noop except for a patch that implemented limited dsync semantic.
> >
> > Now, when full sync support for FATis in kernel, moutning with sync
> > became real pain. Writing speed dropped from 3MB/s to 30KB/s in my case
> > (and I am not alone).
>
> Unfortunately I think we're just going to have to live with that.  It is
> right that fatfs behaves as it does, and unfortunate that some distros will
> operate slowly.
>

Well, I was not going to suggest killing sync support in FAT :)

> For reference: how does mandrake implement this?  Just in /etc/fstab?  How
> should we tell other people to fix this?
>

Yes, just fstab option. It has been "fixed" a couple of days ago by removing 
sync but I am going to test effect of dsync; it should behave more or less as 
before and provide at least some level of fs consistency.

> > One idea how to improve situation - continue to mount with dsync (having
> > basically old case) and do frequent sync of filesystem (this culd be
> > started as HAL callout or whatever). Unfortunately, I could not find a
> > way to request a sync (flush) of single mount point or block device. Have
> > I missed something?
>
> It's trivial to do in-kernel but no, I'm afraid there isn't a userspace
> interface for this.

Oh well, let's do it in kernel to check if it is worth hassle.

Thanks

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Syncing single filesystem (slow USB writing)

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

On Friday 29 July 2005 07:50, Andrew Morton wrote:
> > One idea how to improve situation - continue to mount with dsync (having
> > basically old case) and do frequent sync of filesystem (this culd be
> > started as HAL callout or whatever). Unfortunately, I could not find a
> > way to request a sync (flush) of single mount point or block device. Have
> > I missed something?
>
> It's trivial to do in-kernel but no, I'm afraid there isn't a userspace
> interface for this.

apparently one should not ask such a question at 7 am. Any reason why 
BLKFLSBUF does not suite?

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Syncing single filesystem (slow USB writing)

[Andrew Morton]

Andrey Borzenkov <arvidjaar@mail.ru> wrote:
>
> On Friday 29 July 2005 07:50, Andrew Morton wrote:
> > > One idea how to improve situation - continue to mount with dsync (having
> > > basically old case) and do frequent sync of filesystem (this culd be
> > > started as HAL callout or whatever). Unfortunately, I could not find a
> > > way to request a sync (flush) of single mount point or block device. Have
> > > I missed something?
> >
> > It's trivial to do in-kernel but no, I'm afraid there isn't a userspace
> > interface for this.
> 
> apparently one should not ask such a question at 7 am. Any reason why 
> BLKFLSBUF does not suite?

That'll only write back data associated with /dev/hdXX (ie: filesystem
metadata) and not the data associated with all the files in the filesystem
which is mounted on /dev/hdXX.

Re: Syncing single filesystem (slow USB writing)

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 1157 bytes --]

On Friday 29 July 2005 15:20, Andrew Morton wrote:
> Andrey Borzenkov <arvidjaar@mail.ru> wrote:
> > On Friday 29 July 2005 07:50, Andrew Morton wrote:
> > > > One idea how to improve situation - continue to mount with dsync
> > > > (having basically old case) and do frequent sync of filesystem (this
> > > > culd be started as HAL callout or whatever). Unfortunately, I could
> > > > not find a way to request a sync (flush) of single mount point or
> > > > block device. Have I missed something?
> > >
> > > It's trivial to do in-kernel but no, I'm afraid there isn't a userspace
> > > interface for this.
> >
> > apparently one should not ask such a question at 7 am. Any reason why
> > BLKFLSBUF does not suite?
>
> That'll only write back data associated with /dev/hdXX (ie: filesystem
> metadata) and not the data associated with all the files in the filesystem
> which is mounted on /dev/hdXX.

I am confused. BKLFLSBUF boils down to sync_inodes_sb for mounted device, and 
that appears to write out direty inode pages?

What should be used for in-kernel implementation then? I was going to use the 
same frankly speaking :)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Syncing single filesystem (slow USB writing)

[Andrew Morton]

Andrey Borzenkov <arvidjaar@mail.ru> wrote:
>
> On Friday 29 July 2005 15:20, Andrew Morton wrote:
> > Andrey Borzenkov <arvidjaar@mail.ru> wrote:
> > > On Friday 29 July 2005 07:50, Andrew Morton wrote:
> > > > > One idea how to improve situation - continue to mount with dsync
> > > > > (having basically old case) and do frequent sync of filesystem (this
> > > > > culd be started as HAL callout or whatever). Unfortunately, I could
> > > > > not find a way to request a sync (flush) of single mount point or
> > > > > block device. Have I missed something?
> > > >
> > > > It's trivial to do in-kernel but no, I'm afraid there isn't a userspace
> > > > interface for this.
> > >
> > > apparently one should not ask such a question at 7 am. Any reason why
> > > BLKFLSBUF does not suite?
> >
> > That'll only write back data associated with /dev/hdXX (ie: filesystem
> > metadata) and not the data associated with all the files in the filesystem
> > which is mounted on /dev/hdXX.
> 
> I am confused. BKLFLSBUF boils down to sync_inodes_sb for mounted device, and 
> that appears to write out direty inode pages?

Oh, you're right.  How cunning.  Yes, `blockdev --flushbufs /dev/hda1'
would indeed appear to sync all data associated with that device.

Re: Syncing single filesystem (slow USB writing)

[Jan Engelhardt]

>Now, when full sync support for FATis in kernel, moutning with sync became 
>real pain. Writing speed dropped from 3MB/s to 30KB/s in my case (and I am 
>not alone).

Interesting. I mount all my USB drives and players with async option - and 
they do use vfat. There is no problem on unmount, they complete the sync. 
(Though btw, the drive's led still flashes after umount has returned.)
For USB 1.1 devices, it takes a little longer since the buffer cache or 
something like that is so big.



Jan Engelhardt
-- 
| Alphagate Systems, http://alphagate.hopto.org/

Re: Syncing single filesystem (slow USB writing)

[Chris Wedgwood]

On Fri, Jul 29, 2005 at 07:31:20AM +0400, Andrey Borzenkov wrote:

> Mandrake always mounted USB sticks with sync option; it was
> effectively noop except for a patch that implemented limited dsync
> semantic.

fwiw; various people have reported using flash block devices with
'sync' ruins them as there can be many more (and very frequent)
updates to the device