Re: File System conversion -- ideas

[Hugo Mills <hugo-lkml@carfax.org.uk>]

[-- Attachment #1: Type: text/plain, Size: 4500 bytes --]

On Sun, Jun 29, 2003 at 04:29:45PM -0400, rmoser wrote:
> *********** REPLY SEPARATOR  ***********
> 
> NO!  You're not getting the point at all!
> 
> You don't need a pair!  If you have 10 filesystems, you need 10 sets of
> code in each direction, not 90.  You convert from the data/metadata set
> in the first filesystem to a self-contained atom, and then back from the
> atom to the data/metadata set in the new filesystem.  The atom is object
> oriented, so anything that can't be moved over--like ACLs or Reiser4's
> extended attributes that nobody else has, or permissions if converting to
> vfat--is just lost.  

   You will, of course, ensure that your atoms contain the superset of
all filesystem metadata semantics.

> Note that if the data has an attribute like "Compressed"
> or "encrypted", it is expanded/decrypted and thus brought back to its
> natural form before being stuffed into an atom.
[snip]

> So with 10 filesystem types, N*(N-1) or 90 pairs to go directly from one
> filesystem's datastructures to any other's; N*2 or 20 pairs to go from
> Metadata/Data pair -> Self-contained object oriented possibly
> compressed atom -> Metadata/Data pair.  That's N sets of code to go
> FS_OBJECT -> atom and N sets to go from atom ->  FS_OBJECT, in
> this case 10 and 10.
> 
> When we get to 20 filesystems, direct conversion needs 380 pieces of
> code, whereas my method needs only 20 + 20 == 40.  I obviously put
> more thought into this than you, but that's okay; it's an obscure idea and
> I don't expect everyone to think before answering.

   Actually:

1) I think Viro did mention exactly this method in one of his mails.

2) It's not an obscure idea at all -- it's one of the standard
   techniques if you've ever had to consider (let alone write!) a
   set of data-conversion routines.

> >If you want your idea to be considered seriously - take reiserfs code,
> >take ext3 code, copy both to userland and put together a conversion
> >between them.  Both ways.  That, by definition, is easier than doing
> >it in kernel - you have the same code available and none of the
> >limitations/
> >interaction with other stuff.  When you have it working, well, time to
> >see what extra PITA will come from making it coexist with other parts
> >of kernel (and with much more poor runtime environment).
> >
> 
> That would be much harder to maintain as well.  It would have to be altered
> every time the filesystem code in the kernel is changed.

   Yes, but the point is it's a much easier thing to implement and
test the concept than diving straight into kernel code. You don't have
to maintain it for very long (if at all) -- just long enough to prove
to everyone that this kind of conversion is possible, and that they
should help you roll it into the kernel.

> >AFAICS, it is _very_ hard to implement.  Even outside of the kernel.
> >If you can get it done - well, that might do a lot for having the
> >idea considered seriously.  "Might" since you need to do it in a way
> >that would survive transplantation into the kernel _and_ would scale
> >better that O((number of filesystem types)^2).
> 
> I've beaten the O((FS_COUNT)^2) already.  And by the way, it's
> O((FS_COUNT)*(FS_COUNT - 1_).  There's exactly O(2*FS_COUNT)
> and o(2*FS_COUNT) sets of code needed total to be able to convert
> between any two filesystems.

   There's no such thing as O(x*(x-1)). This is precisely O(x^2).
Similarly, O(2*x) is precisely the same as O(x). If you're going to
try to use mathematics to demonstrate your point, please at least make
sure that you're using it _right_.

> Now, what's impractical is maintaining two sets of code that do exactly
> the same thing.  Why maintain code here to read the filesystems, and
> also in the kernel?  

   It's not a maintenance thing at all -- it's a matter of
demonstrating that you can walk before you try running.

> Just do it in the kernel.  Don't lose sight of the fact
> that the final goal (after all else is done) is to modify VFS to actually
> run this thing as a filesystem.  THAT is what's going to be a bitch.  The
> conversions are simple enough.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 1C335860 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- For months now, we have been making triumphant retreats ---     
               before a demoralised enemy who is advancing               
                           in utter disorder.                            

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]