VFAT truncate performance

VFAT truncate performance

[Mattias Rönnblom]

Hi,

extending files by ftruncate(2) runs very slow on VFAT file
systems. On my USB harddisk w/ VFAT, it takes 14 seconds to extend an
empty file to 1 GB. On a memory stick, it takes well over 4 minutes.

My question is: is this problem on the conceptual level (ie there is
no way of extending files on FAT that doesn't involve many disk
operations) or is the current Linux fs driver suboptimal in this
respect?

The reason for asking is that I run Samba which service files on USB
devices (w/ VFAT for portability) to Windows XP clients. When copying
files to the Samba server, Microsoft SMB clients seem to extend the
file before actually starting to copy the data. This results in
sluggishness and timeouts when copying large files to VFAT
filesystems.

Thanks,
        Mattias

Re: VFAT truncate performance

[Xavier Bestel]

On Tue, 2006-09-05 at 15:52, Mattias Rönnblom wrote:
> Hi,
> 
> extending files by ftruncate(2) runs very slow on VFAT file
> systems. On my USB harddisk w/ VFAT, it takes 14 seconds to extend an
> empty file to 1 GB. On a memory stick, it takes well over 4 minutes.
> 
> My question is: is this problem on the conceptual level (ie there is
> no way of extending files on FAT that doesn't involve many disk
> operations) or is the current Linux fs driver suboptimal in this
> respect?
> 
> The reason for asking is that I run Samba which service files on USB
> devices (w/ VFAT for portability) to Windows XP clients. When copying
> files to the Samba server, Microsoft SMB clients seem to extend the
> file before actually starting to copy the data. This results in
> sluggishness and timeouts when copying large files to VFAT
> filesystems.

Is your USB stick mounted -o sync ? If that's the case, the truncate()
and write() won't be merged so they will take twice as long. -o sync
generally kills performance on USB sticks.

	Xav

Re: VFAT truncate performance

[OGAWA Hirofumi]

Mattias Rönnblom <hofors@lysator.liu.se> writes:

> extending files by ftruncate(2) runs very slow on VFAT file
> systems. On my USB harddisk w/ VFAT, it takes 14 seconds to extend an
> empty file to 1 GB. On a memory stick, it takes well over 4 minutes.
>
> My question is: is this problem on the conceptual level (ie there is
> no way of extending files on FAT that doesn't involve many disk
> operations) or is the current Linux fs driver suboptimal in this
> respect?

Unfortunately FAT doesn't support sparse file, so ftruncate(2) which
extend size needs to fill all clusters with zero, and write data.

This is limitation of FAT filesystem.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

Re: VFAT truncate performance

[Mattias Rönnblom]

Xavier Bestel <xavier.bestel@free.fr> writes:

> On Tue, 2006-09-05 at 15:52, Mattias Rönnblom wrote:
> > Hi,
> > 
> > extending files by ftruncate(2) runs very slow on VFAT file
> > systems. On my USB harddisk w/ VFAT, it takes 14 seconds to extend an
> > empty file to 1 GB. On a memory stick, it takes well over 4 minutes.
> > 
> > My question is: is this problem on the conceptual level (ie there is
> > no way of extending files on FAT that doesn't involve many disk
> > operations) or is the current Linux fs driver suboptimal in this
> > respect?
> > 
> > The reason for asking is that I run Samba which service files on USB
> > devices (w/ VFAT for portability) to Windows XP clients. When copying
> > files to the Samba server, Microsoft SMB clients seem to extend the
> > file before actually starting to copy the data. This results in
> > sluggishness and timeouts when copying large files to VFAT
> > filesystems.
> 
> Is your USB stick mounted -o sync ? If that's the case, the truncate()
> and write() won't be merged so they will take twice as long. -o sync
> generally kills performance on USB sticks.
> 

No, 'async'.

Regards,
        Mattias

Re: VFAT truncate performance

[Bodo Eggert]

Mattias Rönnblom <hofors@lysator.liu.se> wrote:

> extending files by ftruncate(2) runs very slow on VFAT file
> systems. On my USB harddisk w/ VFAT, it takes 14 seconds to extend an
> empty file to 1 GB. On a memory stick, it takes well over 4 minutes.
> 
> My question is: is this problem on the conceptual level (ie there is
> no way of extending files on FAT that doesn't involve many disk
> operations) or is the current Linux fs driver suboptimal in this
> respect?

Linux needs to zero files it truncate-extends because of security guarantees.

You could temporarily ignore the truncate after create if it's followed by
writing the file (defer untill first non-write), but it will be a BAD hack.
It might work.

Default: open(O_WRITE|O_CREAT|O_TRUNC) -> do it, goto State 1
         otherwise -> just do it

State 1: ftruncate -> remember offset instead of executing ftruncate,
                      goto State 2
         otherwise -> goto Default

State 2: write     -> do it, stay in State 2 unless file size increases
                      beyond fake size, then goto Default
         stat      -> return fake size
         otherwise -> really do ftruncate, goto Default

It might cause some operations to be slow you'd expect to be fast, and
I'm not sure how it has to deal with concurrent access.
-- 
Ich danke GMX dafür, die Verwendung meiner Adressen mittels per SPF
verbreiteten Lügen zu sabotieren.

http://david.woodhou.se/why-not-spf.html