[RFC] VFS: FS CoW using redirection

[RFC] VFS: FS CoW using redirection

[Al Boldi]

Copy on Write is a neat way to quickly achieve a semi-clustered system, by 
mounting any shared FS read-only and redirecting writes to some perClient 
FS.

Would this redirection be easy to implement into the VFS?

Thanks!

--
Al

Re: [RFC] VFS: FS CoW using redirection

[Eric Van Hensbergen]

On 7/8/06, Al Boldi <a1426z@gawab.com> wrote:
>
> Copy on Write is a neat way to quickly achieve a semi-clustered system, by
> mounting any shared FS read-only and redirecting writes to some perClient
> FS.
>
> Would this redirection be easy to implement into the VFS?
>

There are a variety of solutions that have been proposed or are
available to do this sort of thing.  You may want to start by looking
at unionfs and mapfs.  There are also folks looking at doing this from
the block layer (look at the dm-userspace + cowd as well as evms and
lvm snapshots).

            -eric

Re: [RFC] VFS: FS CoW using redirection

[Al Boldi]

Eric Van Hensbergen wrote:
> On 7/8/06, Al Boldi <a1426z@gawab.com> wrote:
> > Copy on Write is a neat way to quickly achieve a semi-clustered system,
> > by mounting any shared FS read-only and redirecting writes to some
> > perClient FS.
> >
> > Would this redirection be easy to implement into the VFS?
>
> There are a variety of solutions that have been proposed or are
> available to do this sort of thing.  You may want to start by looking
> at unionfs and mapfs.  There are also folks looking at doing this from
> the block layer (look at the dm-userspace + cowd as well as evms and
> lvm snapshots).

The idea here, is to do things completely transparent while being as 
unintrusive as possible, by relying on the current FS infrastructure, w/o 
introducing another VFS.

Consider something simple like this:

VFS - anyFS1 (r/w) used normally, unless ENotFound, then redirect read to
    \              anyFS2, or CoW from anyFS2 to anyFS1.
      anyFS2 (r/o) used normally.

i.e: Does the VFS provide hooks for perMount Handlers?


Thanks!

--
Al

Re: [RFC] VFS: FS CoW using redirection

[Al Boldi]

Al Boldi wrote:
> Eric Van Hensbergen wrote:
> > On 7/8/06, Al Boldi <a1426z@gawab.com> wrote:
> > > Copy on Write is a neat way to quickly achieve a semi-clustered
> > > system, by mounting any shared FS read-only and redirecting writes to
> > > some perClient FS.
> > >
> > > Would this redirection be easy to implement into the VFS?
> >
> > There are a variety of solutions that have been proposed or are
> > available to do this sort of thing.  You may want to start by looking
> > at unionfs and mapfs.  There are also folks looking at doing this from
> > the block layer (look at the dm-userspace + cowd as well as evms and
> > lvm snapshots).
>
> The idea here, is to do things completely transparent while being as
> unintrusive as possible, by relying on the current FS infrastructure, w/o
> introducing another VFS.
>
> Consider something simple like this:
>
> VFS - anyFS1 (r/w) used normally, unless ENotFound, then redirect read to
>     \              anyFS2, or CoW from anyFS2 to anyFS1.
>       anyFS2 (r/o) used normally.
>
> i.e: Does the VFS provide hooks for perMount Handlers?

Silence is probably an indication that the VFS does not provide hooks.

The question then:

1. Would it be difficult to introduce perMount Handler hooks into the VFS?

2. Would it be easier to replace the current VFS with one that offers a more 
flexible/open configuration?

3. Would these changes be welcome, or is the VFS considered to be a forbidden 
zone?


Thanks!

--
Al

Re: [RFC] VFS: FS CoW using redirection

[Jörn Engel]

On Sun, 9 July 2006 15:50:36 +0300, Al Boldi wrote:
> 
> Consider something simple like this:
> 
> VFS - anyFS1 (r/w) used normally, unless ENotFound, then redirect read to
>     \              anyFS2, or CoW from anyFS2 to anyFS1.
>       anyFS2 (r/o) used normally.

That concept is known as union mount.  Jan Blunck did some patches in
that direction, you might be able to find them in the archives.  If
not, just send him a mail.

Jörn

-- 
...one more straw can't possibly matter...
-- Kirby Bakken

Re: [RFC] VFS: FS CoW using redirection

[Josef Sipek]

On Wed, Aug 23, 2006 at 07:24:02PM +0200, Jörn Engel wrote:
> On Sun, 9 July 2006 15:50:36 +0300, Al Boldi wrote:
> > 
> > Consider something simple like this:
> > 
> > VFS - anyFS1 (r/w) used normally, unless ENotFound, then redirect read to
> >     \              anyFS2, or CoW from anyFS2 to anyFS1.
> >       anyFS2 (r/o) used normally.
> 
> That concept is known as union mount.  Jan Blunck did some patches in
> that direction, you might be able to find them in the archives.  If
> not, just send him a mail.

Or you can give Unionfs a try: http://www.unionfs.org

Josef "Jeff" Sipek.

-- 
The box said "Windows XP or better required". So I installed Linux.

Re: [RFC] VFS: FS CoW using redirection

[Al Boldi]

Josef Sipek wrote:
> On Wed, Aug 23, 2006 at 07:24:02PM +0200, Jِrn Engel wrote:
> > On Sun, 9 July 2006 15:50:36 +0300, Al Boldi wrote:
> > > Consider something simple like this:
> > >
> > > VFS - anyFS1 (r/w) used normally, unless ENotFound, then redirect read
> > > to \              anyFS2, or CoW from anyFS2 to anyFS1.
> > >       anyFS2 (r/o) used normally.
> >
> > That concept is known as union mount.  Jan Blunck did some patches in
> > that direction, you might be able to find them in the archives.  If
> > not, just send him a mail.

Thanks for the pointer!

So what was the rejecting theme?

> Or you can give Unionfs a try: http://www.unionfs.org

UnionFS is great, but it incurs additional overhead, as it lives below the 
real VFS.  What could be really great, is to move some basic functionality 
abstractions from UnionFS into VFS proper.


Thanks!

--
Al

Re: [RFC] VFS: FS CoW using redirection

[Jörn Engel]

On Sat, 26 August 2006 22:05:21 +0300, Al Boldi wrote:
> 
> So what was the rejecting theme?

I don't believe there was one.  Jan simply didn't push much, so noone
was forced to resist him.  And noone else needed union mount enough to
push Jan.

> > Or you can give Unionfs a try: http://www.unionfs.org
> 
> UnionFS is great, but it incurs additional overhead, as it lives below the 
> real VFS.  What could be really great, is to move some basic functionality 
> abstractions from UnionFS into VFS proper.

Welcome to Jan's work. :)

If you want to make this vision happen, one of the missing pieces is a
method for copyup, an in-kernel copying routine.  Unionfs needs is
just the same as Jan's patches do and in the past Linus didn't like my
approach of using sendfile for it.  You could take a stab at the
splice code and see how that can be used for copyup.

Jörn

-- 
Write programs that do one thing and do it well. Write programs to work
together. Write programs to handle text streams, because that is a
universal interface.
-- Doug MacIlroy

Re: [RFC] VFS: FS CoW using redirection

[Josef Sipek]

On Sun, Aug 27, 2006 at 07:15:10PM +0200, Jörn Engel wrote:
> On Sat, 26 August 2006 22:05:21 +0300, Al Boldi wrote:
... 
> > > Or you can give Unionfs a try: http://www.unionfs.org
> > 
> > UnionFS is great, but it incurs additional overhead, as it lives below the 
> > real VFS.  What could be really great, is to move some basic functionality 
> > abstractions from UnionFS into VFS proper.
> 
> If you want to make this vision happen, one of the missing pieces is a
> method for copyup, an in-kernel copying routine.  Unionfs needs is
> just the same as Jan's patches do and in the past Linus didn't like my
> approach of using sendfile for it.  You could take a stab at the
> splice code and see how that can be used for copyup.

The thing with union mounts/unionfs is that some of the functionality makes
sense to have in a file system while other parts make sense to have in the
VFS - the way I see it, namespace related bits should be in VFS while
persistent state should be done on the file system level.

Josef "Jeff" Sipek.

-- 
If I have trouble installing Linux, something is wrong. Very wrong.
		- Linus Torvalds

Re: [RFC] VFS: FS CoW using redirection

[Shaya Potter]

On Sat, 2006-07-08 at 20:41 +0300, Al Boldi wrote:
> Copy on Write is a neat way to quickly achieve a semi-clustered system, by 
> mounting any shared FS read-only and redirecting writes to some perClient 
> FS.
> 
> Would this redirection be easy to implement into the VFS?

I've used unionfs that way.