Re: ext4 features

Re: ext4 features

[Bodo Eggert]

Petr Tesarik <ptesarik@suse.cz> wrote:

> Salvaging files would be done with a separate tool. Of course, if you
> delete more files with the same name in the same directory, you'd need
> to tell that tool which one of them you want to salvage. Yes, I really
> mean you'd have more than one deleted file with the same name in the
> directory.

> Anyway, I doubt we want such feature for ext4, because to make things
> efficient, you'd need to provide some kind of pointer from the deleted
> (but not yet purged) blocks to the corresponding file. Hard links are
> also problematic and there is a whole lot of other troubles I haven't
> even thought of.

You can add the original directory inode of the deleted file to an attribute.
The name can be stored in the same way, so you can have multiple files with
the same name in the trashcan directory. (Heuristics for the in-trashcan-name
can be e.g. the original inode number or "n-$ORIGINALNAME" if it's unique,
but just using the number will be good enough.)

Using the directory inode you can follow the '..'-chain to the root in order
to present a nice 'original' name, but you should also be able to undelete
files into any directory (even chroots) if both the trashcan and the target
directory are visible and you've got apropiate permissions.

BTW: If you're using the trashcan on linked files, you will need to store
multiple sources for an inode.
-- 
Ich danke GMX dafür, die Verwendung meiner Adressen mittels per SPF
verbreiteten Lügen zu sabotieren.

http://david.woodhou.se/why-not-spf.html

Re: ext4 features

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 523 bytes --]

On Tue, 04 Jul 2006 14:28:05 +0200, Bodo Eggert said:
> BTW: If you're using the trashcan on linked files, you will need to store
> multiple sources for an inode.

No, if it's a multiply linked file, you want all unlinks except the last
to operate the way they always have, and only the LAST link moves to the
trashcan.

Otherwise, if you have a file with two links, and one moves to the trashcan,
it's then subject to reaping and overwriting - which will come as a great
surprise to the still-existing link....

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: ext4 features (salvage)

[Bodo Eggert]

Matthew Frost <artusemrys@sbcglobal.net> wrote:

> We silently keep files around in many filesystems, at least until
> whatever reclamation process runs.  The delete event doesn't itself
> generally purge the data from disk.  However, this is a matter of simple
> tools doing simple things.  Designing an intentional structure around
> not actually deleting deleted files, but keeping them around just in
> case may be lauded as "user-friendly", but it is counter-intuitive.  It
> is cleverness over clarity, good design smothered under feature demand.

[...]

> If you have to add a "really delete, I mean it" command, you're breaking
> fundamental assumptions.

Even without the patch, you can't guarantee that nobody will create a link
just before the supposed-to-be-final unlink(), or copies the file around.
Having undelete() will just make this risk very obvious.

BTW: If you really want to delete files, use shred.


[...]
> Why add non-free space to the free space count, when we're intentionally
> keeping those files?  If you have to be counter-intuitive, why go the
> second counter of hiding it from the user who "wants us to keep and
> index his deleted files"?

It is free space, but it can be unfreed in order to
a) restore the data (on demand)
b) store new data (automatically, no tool needed)

If b happens before a, it's called bad luck.

This concept is fundamentally different from keeping a recycle bin
of 10 % disksize for each of the 20 users (adding up to 200 % of the
disk size), which is well-known from a colorfull "operating" system.
-- 
Ich danke GMX dafür, die Verwendung meiner Adressen mittels per SPF
verbreiteten Lügen zu sabotieren.

http://david.woodhou.se/why-not-spf.html

Re: ext4 features

[Bodo Eggert]

Lew Palm <lew@tzi.de> wrote:
> Jeffrey V. Merkey wrote:

>> The old novell model is simple. When someone unlinks a file, don't
>> delete it, just mv it to another special directory called DELETED.SAV.
>> Then setup the
>> fs space allocation to reuse these files when the drive fills up by
>> oldest files first. It's very simple. Then you have a salvagable file
>> system.
> 
> A complete foolproof car is a car with a maximum speed of 0 mph.
> As a user I give commands to my computer, for example an order to delete a
> file. And this is what I expect it to do.

You don't delete a file but a filename, and that's what your system will
still do. 

> If I want it to move a file to another position in the filesystem, I would
> use another command. I don't want my operating system to josh me, that's why
> I use Linux.
> Stealthy keeping of deleted files somewhere is a security black hole.

Depending on unlinked file to be inaccessible and never have been copied
just because you called unlink is the real security hole.

> But accidents happen. Hardware perishes, users are making mistakes, sometimes
> coffee is pouring...
> That's why we backup important data regulary.

And the salvaging fs will do exactly this whenever you unlink() the final
reference. You could also use a userspace library catching each unlink call,
but it would also have to intercept each write() call for each user and
try to reclaim the backup copies on disk-full and would-have-to-fragment
events. Off cause there are no userspace-visible would-have-to-fragment
events, so besides being ugly a userspace solution would not be able to
completely provide the same service.

> A not-really-deleting-filesystem wouldn't relieve us of that duty, but would
> make a system more insecure and ambiguous.

It's just a marginal shift. If you can't trust yourself, you've lost. If
you can't trust the current root, you're screwed, too. If you can't trust
a future root, the time window in which the file can be recovered will
slightly increase and the needed knowledge will be reduced. Otherwise, there
is no change.
-- 
Ich danke GMX dafür, die Verwendung meiner Adressen mittels per SPF
verbreiteten Lügen zu sabotieren.

http://david.woodhou.se/why-not-spf.html

Re: ext4 features

[Bodo Eggert]

Trond Myklebust <trond.myklebust@fys.uio.no> wrote:
> On Wed, 2006-07-05 at 22:32 -0400, Bill Davidsen wrote:

>> But with timestamps I need remember only one number, the time of my last
>> backup. Skipping over the question of "who's idea of time" inherent in
>> network filesystems. I compare all ctimes with the time of the last
>> backup and do incremental on the newer ones. If we use versioning I have
>> to remember the version for each file! In practice I really question if
>> the benefit justified keeping all that metadata between backups. And if
>> I delete a file and create another by the same name, what is it's version?
> 
> You are completely missing the point. Our background is that all NFS
> clients are required to use the mtime and ctime timestamps in order to
> figure out if their cached data is valid. They need to do this extremely
> frequently (in fact, every time you open() the file).

If the changes to these files are very infrequent compared to nanoseconds,
you'll only need the version during some nanoseconds, and only during
runtime. Having a second-change-within-one-timeframe-flag(*) instead of
versions will be enough to make NFS mostly happy and only penalize your
users for one nanosecond, and it won't force version-keeping into the
filesystem. And besides that, all other filesystems will profit even
without having nanosecond resolution nor versioning (but they'll suffer
for up to a whole second).


*) TODO: Create a nice name
-- 
Ich danke GMX dafür, die Verwendung meiner Adressen mittels per SPF
verbreiteten Lügen zu sabotieren.

http://david.woodhou.se/why-not-spf.html

Re: ext4 features

[Trond Myklebust]

On Fri, 2006-07-07 at 11:38 +0200, Bodo Eggert wrote:

> If the changes to these files are very infrequent compared to nanoseconds,
> you'll only need the version during some nanoseconds, and only during
> runtime. Having a second-change-within-one-timeframe-flag(*) instead of
> versions will be enough to make NFS mostly happy and only penalize your
> users for one nanosecond, and it won't force version-keeping into the
> filesystem. And besides that, all other filesystems will profit even
> without having nanosecond resolution nor versioning (but they'll suffer
> for up to a whole second).

NFS never required file versioning. You're talking to the wrong person.
It does, however, need a change attribute that logs all changes to the
file. The above flag does not suffice to provide that.

Trond

Re: ext4 features

[Bodo Eggert]

On Fri, 7 Jul 2006, Trond Myklebust wrote:
> On Fri, 2006-07-07 at 11:38 +0200, Bodo Eggert wrote:

> > If the changes to these files are very infrequent compared to nanoseconds,
> > you'll only need the version during some nanoseconds, and only during
> > runtime. Having a second-change-within-one-timeframe-flag(*) instead of
> > versions will be enough to make NFS mostly happy and only penalize your
> > users for one nanosecond, and it won't force version-keeping into the
> > filesystem. And besides that, all other filesystems will profit even
> > without having nanosecond resolution nor versioning (but they'll suffer
> > for up to a whole second).
> 
> NFS never required file versioning. You're talking to the wrong person.
> It does, however, need a change attribute that logs all changes to the
> file. The above flag does not suffice to provide that.

ACK, that's why I'm suggesting this doublechange-flag.
-- 
In the beginning, God created the earth and rested.
Then God created Man and rested.
Then God created Woman.
Since then, neither God nor Man has rested.

ext4 features

[Thomas Glanzmann]

Hello,
I would like to know which new features are planed to be incorported by
ext4. So far I only read about supporting bigger filesystems to fit
recent hardware developments. So are there any other big goals for ext4?

What I personally would like to see most in ext4 are

        * checksums for data
        * and snapshots on filesystem basis

But I guess that this is way out of scope for ext4.

        Thomas

Re: ext4 features

[Tomasz Torcz]

[-- Attachment #1: Type: text/plain, Size: 806 bytes --]

On Sat, Jul 01, 2006 at 06:33:01PM +0200, Thomas Glanzmann wrote:
> Hello,
> I would like to know which new features are planed to be incorported by
> ext4. So far I only read about supporting bigger filesystems to fit
> recent hardware developments. So are there any other big goals for ext4?
> 
> What I personally would like to see most in ext4 are
> 
>         * checksums for data

  Checksums are not very useful for themselves. They are useful when we
have other copy of data (think raid mirroring) so data can be
reconstructed from working copy.

>         * and snapshots on filesystem basis

  What's wrong with DM snapshots?

-- 
Tomasz Torcz            There exists no separation between gods and men:
zdzichu@irc.-nie.spam-.pl   one blends softly casual into the other.


[-- Attachment #2: Type: application/pgp-signature, Size: 229 bytes --]

Re: ext4 features

[Thomas Glanzmann]

Hello,

> Checksums are not very useful for themselves. They are useful when we
> have other copy of data (think raid mirroring) so data can be
> reconstructed from working copy.

it would be possible to identify data corruption.

>   What's wrong with DM snapshots?

they're inefficient in matter of disk space consumption because they
don't have a clue of the filesystems that are on top of them.

        Thomas

Re: ext4 features

[Tomasz Torcz]

[-- Attachment #1: Type: text/plain, Size: 608 bytes --]

On Sat, Jul 01, 2006 at 07:47:16PM +0200, Thomas Glanzmann wrote:
> Hello,
> 
> > Checksums are not very useful for themselves. They are useful when we
> > have other copy of data (think raid mirroring) so data can be
> > reconstructed from working copy.
> 
> it would be possible to identify data corruption.
> 

  Yes, but what good is identification? We could only return I/O error.
Ability to fix corruption (like ZFS) is the real killer.

-- 
Tomasz Torcz            There exists no separation between gods and men:
zdzichu@irc.-nie.spam-.pl   one blends softly casual into the other.


[-- Attachment #2: Type: application/pgp-signature, Size: 229 bytes --]

Re: ext4 features

[Helge Hafting]

On Sat, Jul 01, 2006 at 08:17:02PM +0200, Tomasz Torcz wrote:
> On Sat, Jul 01, 2006 at 07:47:16PM +0200, Thomas Glanzmann wrote:
> > Hello,
> > 
> > > Checksums are not very useful for themselves. They are useful when we
> > > have other copy of data (think raid mirroring) so data can be
> > > reconstructed from working copy.
> > 
> > it would be possible to identify data corruption.
> > 
> 
>   Yes, but what good is identification? We could only return I/O error.
> Ability to fix corruption (like ZFS) is the real killer.

Isn't that what we have RAID-1/5/6 for?  

Helge Hafting

Re: ext4 features

[Tomasz Torcz]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

On Mon, Jul 03, 2006 at 10:22:19PM +0200, Helge Hafting wrote:
> On Sat, Jul 01, 2006 at 08:17:02PM +0200, Tomasz Torcz wrote:
> > On Sat, Jul 01, 2006 at 07:47:16PM +0200, Thomas Glanzmann wrote:
> > > Hello,
> > > 
> > > > Checksums are not very useful for themselves. They are useful when we
> > > > have other copy of data (think raid mirroring) so data can be
> > > > reconstructed from working copy.
> > > 
> > > it would be possible to identify data corruption.
> > > 
> > 
> >   Yes, but what good is identification? We could only return I/O error.
> > Ability to fix corruption (like ZFS) is the real killer.
> 
> Isn't that what we have RAID-1/5/6 for?  

  ZFS was already called ,,blatant layering violation''. ;)
Yes,that what RAID is for. And if we want checksums in filesystem,
that's the best way to utilise them.

-- 
Tomasz Torcz                 Morality must always be based on practicality.
zdzichu@irc.-nie.spam-.pl                -- Baron Vladimir Harkonnen


[-- Attachment #2: Type: application/pgp-signature, Size: 229 bytes --]

Re: ext4 features

[Arjan van de Ven]

>   ZFS was already called ,,blatant layering violation''. ;)
> Yes,that what RAID is for. And if we want checksums in filesystem,
> that's the best way to utilise them.


Hi,

checksums have a very different purpose than raid.

checksums are great at detecting corruption. And yes, corruption can
happen even if you have raid, for many many reasons. Detecting means
knowing when to not trust something, when to go for the backup tapes...

raid is great for protecting against individual disks or sectors going
bad. But raid, especially high performance implementations, do not
checksum data or detect corruptions. 

They're different purpose with almost zero overlap in purpose or even
goal...

Re: ext4 features

[Jeff V. Merkey]

Arjan van de Ven wrote:

>>  ZFS was already called ,,blatant layering violation''. ;)
>>Yes,that what RAID is for. And if we want checksums in filesystem,
>>that's the best way to utilise them.
>>    
>>
>
>
>Hi,
>
>checksums have a very different purpose than raid.
>
>checksums are great at detecting corruption. And yes, corruption can
>happen even if you have raid, for many many reasons. Detecting means
>knowing when to not trust something, when to go for the backup tapes...
>
>raid is great for protecting against individual disks or sectors going
>bad. But raid, especially high performance implementations, do not
>checksum data or detect corruptions. 
>
>They're different purpose with almost zero overlap in purpose or even
>goal...
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>  
>
Add a salvagable file system to ext4, i.e. when a file is deleted, you 
just rename it and move it to a directory called DELETED.SAV and recycle 
the files as people allocate new ones.  Easy to do (internal "mv" of 
file to another directory) and modification of the allocation bitmaps.  
Very simple and will pay off big.  If you need help designing it, just 
ask me.

Jeff

Re: ext4 features

[Diego Calleja]

El Mon, 03 Jul 2006 15:46:55 -0600,
"Jeff V. Merkey" <jmerkey@wolfmountaingroup.com> escribió:

> Add a salvagable file system to ext4, i.e. when a file is deleted, you 
> just rename it and move it to a directory called DELETED.SAV and recycle 
> the files as people allocate new ones.  Easy to do (internal "mv" of 


Easily doable in userspace, why bother with kernel programming

Re: ext4 features

[Petr Tesarik]

On Mon, 2006-07-03 at 23:25 +0200, Diego Calleja wrote:
> El Mon, 03 Jul 2006 15:46:55 -0600,
> "Jeff V. Merkey" <jmerkey@wolfmountaingroup.com> escribió:
> 
> > Add a salvagable file system to ext4, i.e. when a file is deleted, you 
> > just rename it and move it to a directory called DELETED.SAV and recycle 
> > the files as people allocate new ones.  Easy to do (internal "mv" of 
> 
> 
> Easily doable in userspace, why bother with kernel programming

Yes and no. A simple mv is better done in userspace, but what I'd
_really_ appreciate would be a true kernel salvage (similar to the way
NetWare does things). That means marking the file as deleted in the
directory, marking its blocks as deleted but avoiding the use of those
blocks. The kernel would then prefer allocating new blocks from
elsewhere but once the filesystem runs out of space, it would start
allocating from the deleted files area and marking the blocks as well as
the corresponding files purged.

Salvaging files would be done with a separate tool. Of course, if you
delete more files with the same name in the same directory, you'd need
to tell that tool which one of them you want to salvage. Yes, I really
mean you'd have more than one deleted file with the same name in the
directory.

Anyway, I doubt we want such feature for ext4, because to make things
efficient, you'd need to provide some kind of pointer from the deleted
(but not yet purged) blocks to the corresponding file. Hard links are
also problematic and there is a whole lot of other troubles I haven't
even thought of.

Just my two cents.

--
Petr Tesarik

Re: ext4 features

[Peter Zijlstra]

On Tue, 2006-07-04 at 11:22 +0200, Petr Tesarik wrote:
> On Mon, 2006-07-03 at 23:25 +0200, Diego Calleja wrote:
> > El Mon, 03 Jul 2006 15:46:55 -0600,
> > "Jeff V. Merkey" <jmerkey@wolfmountaingroup.com> escribió:
> > 
> > > Add a salvagable file system to ext4, i.e. when a file is deleted, you 
> > > just rename it and move it to a directory called DELETED.SAV and recycle 
> > > the files as people allocate new ones.  Easy to do (internal "mv" of 
> > 
> > 
> > Easily doable in userspace, why bother with kernel programming
> 
> Yes and no. A simple mv is better done in userspace, but what I'd
> _really_ appreciate would be a true kernel salvage (similar to the way
> NetWare does things). That means marking the file as deleted in the
> directory, marking its blocks as deleted but avoiding the use of those
> blocks. The kernel would then prefer allocating new blocks from
> elsewhere but once the filesystem runs out of space, it would start
> allocating from the deleted files area and marking the blocks as well as
> the corresponding files purged.
> 
> Salvaging files would be done with a separate tool. Of course, if you
> delete more files with the same name in the same directory, you'd need
> to tell that tool which one of them you want to salvage. Yes, I really
> mean you'd have more than one deleted file with the same name in the
> directory.
> 
> Anyway, I doubt we want such feature for ext4, because to make things
> efficient, you'd need to provide some kind of pointer from the deleted
> (but not yet purged) blocks to the corresponding file. Hard links are
> also problematic and there is a whole lot of other troubles I haven't
> even thought of.

Wouldn't such a scheme interfere with the block allocator algorithms,
and hence increase the risk of fragmentation? Schemes like this realy
put my hairs on end,

  1) if you don't want to lose your data, make backups; 
  2) if I mean to delete a file, I want it gone proper. Silently keeping
     it about is not unix like;
  3) don't aid third parties in recovering your removed data. If I want
     them to have it I'll give it to them.

Peter

Re: ext4 features (salvage)

[Petr Tesarik]

On Tue, 2006-07-04 at 13:35 +0200, Peter Zijlstra wrote:
> On Tue, 2006-07-04 at 11:22 +0200, Petr Tesarik wrote:
> > Yes and no. A simple mv is better done in userspace, but what I'd
> > _really_ appreciate would be a true kernel salvage (similar to the way
> > NetWare does things). That means marking the file as deleted in the
> > directory, marking its blocks as deleted but avoiding the use of those
> > blocks. The kernel would then prefer allocating new blocks from
> > elsewhere but once the filesystem runs out of space, it would start
> > allocating from the deleted files area and marking the blocks as well as
> > the corresponding files purged.
> > 
> > Salvaging files would be done with a separate tool. Of course, if you
> > delete more files with the same name in the same directory, you'd need
> > to tell that tool which one of them you want to salvage. Yes, I really
> > mean you'd have more than one deleted file with the same name in the
> > directory.
> 
> Wouldn't such a scheme interfere with the block allocator algorithms,
> and hence increase the risk of fragmentation? Schemes like this realy
> put my hairs on end,

Yes, they would interfere. That's why I'm not proposing to add them to
ext4 in the first place.

>   1) if you don't want to lose your data, make backups; 

Generally, I agree.

>   2) if I mean to delete a file, I want it gone proper. Silently keeping
>      it about is not unix like;

Yes, this is a problem. Although you would of course have a tool for
purging the files unconditionally, some programs may need the assumption
that an unlinked file is gone forever.

Regarding the second clause, well, Linux is not Unix-like in many
respects and we want it like that. That's a weak argument.

>   3) don't aid third parties in recovering your removed data. If I want
>      them to have it I'll give it to them.

See 2. Explicit purging is of course possible. (Novell Netware also had
a "purge" command.)

Anyway, it seems that there is some functionality which many users want
but which can't be provided in user space: 

  - if files are moved to the recycle-bin-or-whatever-you-call-it, their
size is added to disk free space and
  - automatically purging least recently deleted files.

Regards,
Petr Tesarik

Re: ext4 features (salvage)

[Petr Tesarik]

On Tue, 2006-07-04 at 16:08 +0400, Lex Lyamin wrote:
> you mean that blocks are naturaly free, but we cant use them because
> someone may made them free by accident, but we cant use them...
> 
> hmm...
> great idea!
> 
> wait, its not.
> because of we cant use those blocks we cant optimise way we write one
> disk , and if we have defragmenter we cant  make use of them either.
> and if (just if) this is online defragmenter, it cant use them too. 

Well, the way I saw it done was that you had no guarantee that any
deleted file could be salvaged. Sometimes you even could salvage a file
but not another one which was deleted later. Users seemed to be content
with that, because in most situations it did help them restore files
they deleted and within a few seconds realized that they didn't want to.

This means that the allocator MAY purge any deleted block at any moment,
although it tends to allocate blocks from areas of disk which haven't
been used recently.

And the benefits? The performance of such a filesystem could be better
than snapshots, while allowing to cope with one of the most common human
errors.

Regards,
Petr Tesarik

> for what purpose ?
> are not we trying  play out solution to problem from level 3 on level
> 2 ?
> does the soulion really belong to this level ?
> would people pay with performance for "feature" which *probably* will
> help them to restore their files ? 

Re: ext4 features (salvage)

[Helge Hafting]

On Tue, Jul 04, 2006 at 02:31:56PM +0200, Petr Tesarik wrote:
> On Tue, 2006-07-04 at 16:08 +0400, Lex Lyamin wrote:
> > you mean that blocks are naturaly free, but we cant use them because
> > someone may made them free by accident, but we cant use them...
> > 
> > hmm...
> > great idea!
> > 
> > wait, its not.
> > because of we cant use those blocks we cant optimise way we write one
> > disk , and if we have defragmenter we cant  make use of them either.
> > and if (just if) this is online defragmenter, it cant use them too. 
> 
> Well, the way I saw it done was that you had no guarantee that any
> deleted file could be salvaged. Sometimes you even could salvage a file
> but not another one which was deleted later. Users seemed to be content
> with that, because in most situations it did help them restore files
> they deleted and within a few seconds realized that they didn't want to.
> 
> This means that the allocator MAY purge any deleted block at any moment,
> although it tends to allocate blocks from areas of disk which haven't
> been used recently.
> 
> And the benefits? The performance of such a filesystem could be better
> than snapshots, while allowing to cope with one of the most common human
> errors.

The most common error?  A few years ago I restored a file from
backup, because I deleted it in error.  I can't even remember
the second-last time I had that problem.

I'd say this error is among the easiest to avoid. :-)
Even a little performance loss won't justify it for me.

Now, there may be clumsier users than me, but they tend to
be using GUI "file managers" which do implement a "wastebasket"
for all internal deletion.

Helge Hafting

Re: ext4 features (salvage)

[Matthew Frost]

(Stupid mailer + user error = not sent to list)

Petr Tesarik wrote:
 > On Tue, 2006-07-04 at 13:35 +0200, Peter Zijlstra wrote:
 >> On Tue, 2006-07-04 at 11:22 +0200, Petr Tesarik wrote:
 >>> Yes and no. A simple mv is better done in userspace, but what I'd
 >>> _really_ appreciate would be a true kernel salvage (similar to the way
 >>> NetWare does things). That means marking the file as deleted in the
 >>> directory, marking its blocks as deleted but avoiding the use of those
 >>> blocks. The kernel would then prefer allocating new blocks from
 >>> elsewhere but once the filesystem runs out of space, it would start
 >>> allocating from the deleted files area and marking the blocks as 
well as
 >>> the corresponding files purged.
 >>>
 >>> Salvaging files would be done with a separate tool. Of course, if you
 >>> delete more files with the same name in the same directory, you'd need
 >>> to tell that tool which one of them you want to salvage. Yes, I really
 >>> mean you'd have more than one deleted file with the same name in the
 >>> directory.
 >> Wouldn't such a scheme interfere with the block allocator algorithms,
 >> and hence increase the risk of fragmentation? Schemes like this realy
 >> put my hairs on end,
 >
 > Yes, they would interfere. That's why I'm not proposing to add them to
 > ext4 in the first place.
 >
 >>   1) if you don't want to lose your data, make backups;
 >
 > Generally, I agree.
 >
 >>   2) if I mean to delete a file, I want it gone proper. Silently keeping
 >>      it about is not unix like;
 >
 > Yes, this is a problem. Although you would of course have a tool for
 > purging the files unconditionally, some programs may need the assumption
 > that an unlinked file is gone forever.
 >
 > Regarding the second clause, well, Linux is not Unix-like in many
 > respects and we want it like that. That's a weak argument.

We silently keep files around in many filesystems, at least until
whatever reclamation process runs.  The delete event doesn't itself
generally purge the data from disk.  However, this is a matter of simple
tools doing simple things.  Designing an intentional structure around
not actually deleting deleted files, but keeping them around just in
case may be lauded as "user-friendly", but it is counter-intuitive.  It
is cleverness over clarity, good design smothered under feature demand.

In the ways in which it counts, in the sensible, useful, elegantly
simple ways, the "Do one thing and do it well" ways, Linux tries to be
Unix-like.  We want stupid programs.  A filesystem that decides that it
knows better than the user is not desirable.  Filesystem programmers
that decide that they know better than the user are likewise sub-optimal.

Protect my data against accidental failure.  Do not protect it against me.

If you have to add a "really delete, I mean it" command, you're breaking
fundamental assumptions.

 >
 >>   3) don't aid third parties in recovering your removed data. If I want
 >>      them to have it I'll give it to them.
 >
 > See 2. Explicit purging is of course possible. (Novell Netware also had
 > a "purge" command.)
 >
 > Anyway, it seems that there is some functionality which many users want
 > but which can't be provided in user space:
 >
 >   - if files are moved to the recycle-bin-or-whatever-you-call-it, their
 > size is added to disk free space and

Why add non-free space to the free space count, when we're intentionally
keeping those files?  If you have to be counter-intuitive, why go the
second counter of hiding it from the user who "wants us to keep and
index his deleted files"?

 >   - automatically purging least recently deleted files.
 >
 > Regards,
 > Petr Tesarik

Matt