* Encrypted Filesystem
@ 2004-01-26 17:46 Michael A Halcrow
2004-01-26 19:06 ` Mark Borgerding
` (2 more replies)
0 siblings, 3 replies; 14+ messages in thread
From: Michael A Halcrow @ 2004-01-26 17:46 UTC (permalink / raw)
To: linux-kernel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I have some time this year to work on an encrypted filesystem for
Linux. I have surveyed various LUG's, tested and reviewed code for
currently existing implementations, and have started modifying some
of them. I would like to settle on a single approach on which to
focus my efforts, and I am interested in getting feedback from the
LKML community as to which approach is the most feasible.
This is the feature wish-list that I have compiled, based on personal
experience and feedback I have received from other individuals and
groups:
- Seamless application to the root filesystem
- Layered over the entire root filesystem
- Unencrypted pass-through mode with minimal overhead
- Files are marked as ``encrypted'' or ``unencrypted'' and treated
accordingly by the encryption layer
- Key->file association
- As opposed to key->blkdev or key->directory granularity
- No encryption metafiles in directories, instead placing that
information into Extended Attributes
- May break some backup utilities that are not EA-aware; may require
another mode where encryption metadata is stored in a header block
on the encrypted file
- Directories can be flagged as ``encrypted-only'', where any new
files created in that directory are, by default, encrypted, with
the key and permissions defined in the directory's metadata
- Processes may have encryption contexts, whereby any new files they
create are encrypted by default according to the process'
authentication
- Make as much metadata about the file as confidential as possible
(filesize, executable bit, etc.)
- Pluggable encryption (I wouldn't mind using a block cipher in CTR
mode)
- Authentication via PAM
- pam_usb
- Bluetooth
- Kerberos
- PAM checks for group membership before allowing access to certain
encrypted files
- Rootplug-based LSM to provide key management (necessary to use
LSM?)
- Secret splitting and/or (m,n)-threshold support on the keys
- Signatures on files flagged for auditing in order to detect
attempts to circumvent the encryption layer (via direct
modifications to the files themselves in the underlying filesystem)
- Ad-hoc groups for access to decrypted versions of files
- i.e., launch web browser, drop group membership by default (like
capability inheritance masks) so that the browser does not have
access to decrypted files by default; PAM module checks for group
membership before allowing access (explicit user authorization on
application access requests)
- Userland utilities to support encrypted file management
- Extensions to nautilus and konqueror to be able to use these
utilities from a common interface (think: right-click, encrypted)
- Distro installation integration
- Transparent shredding, where the underlying filesystem supports it
- Versioning and snapshots (CVS-ish behavior)
- Design to work w/ SE Linux
These are features that have been requested, but are not necessarily
hard requirements for the encrypted filesystem. They are just
suggestions that I have received, and I am not convinced that they are
all feasible.
There are several potential approaches to an encrypted filesystem with
these features, all with varying degrees of modification to the kernel
itself, each with its own set of advantages and disadvantages.
Options that I am aware of include:
- NFS-based (CFS, TCFS)
- CFS is mature
- Performance issues
- Violates UNIX semantics w/ hole behavior
- Single-threaded
- Userland filesystem-based (EncFS+FUSE, CryptoFS+LUFS)
- Newer solutions, not as well accepted or tested as CFS
- KDE team is using SSHFS+FUSE
- Loopback (cryptoloop) encrypted block device
- Mature; in the kernel
- Block device granularity; breaks most incremental backup
applications
- LSM-based
- Is this even possible? Are the hooks that we need there?
- Modifications to VFS (stackable filesystem, like NCryptfs)
- Very low overhead claimed by Erez Zadok
- Full implementation not released
- Key->directory granularity
- Evicts cleartext pages from the cache on process death
- Uses dcache to store attaches
- Other niceties, but it's not released...
My goal is to develop an encrypted filesystem ``for the desktop'',
where a user can right-click on a file in konqueror or nautilus and
check the ``encrypted'' box, and all subsequent accesses by any
processes to that file will require authentication in order for the
file to be decrypted. I have already made some modifications to CFS
to support this functionality, but I am not sure at this moment
whether or not CFS is the best route to go for this.
I have had requests to write a kernel module that, when loaded,
transparently starts acting as the encryption layer on top of whatever
root filesystem is mounted. For example, an ext3 partition may have
encrypted files strewn about, which are accessible only after loading
the module (and authenticating, etc.).
Any advise or direction that the kernel community could provide would
be very much appreciated.
Thanks,
Mike
.___________________________________________________________________.
Michael A. Halcrow
Security Software Engineer, IBM Linux Technology Center
GnuPG Fingerprint: 05B5 08A8 713A 64C1 D35D 2371 2D3C FDDA 3EB6 601D
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQFAFU9wLTz92j62YB0RAkOfAKClVMzKIhw6JtyGvKf8+iFp4e12AwCdFARU
uAhpA7wVjvPMdDQtKSnFzzI=
=TM5Y
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Encrypted Filesystem
2004-01-26 17:46 Encrypted Filesystem Michael A Halcrow
@ 2004-01-26 19:06 ` Mark Borgerding
2004-01-26 21:04 ` Felipe Alfaro Solana
2004-01-27 0:06 ` jw schultz
2004-01-27 0:43 ` Adam Sampson
2 siblings, 1 reply; 14+ messages in thread
From: Mark Borgerding @ 2004-01-26 19:06 UTC (permalink / raw)
To: Michael A Halcrow, linux-kernel
It sounds like you've got a nice methodical approach: gathering
requirements and thinking about it before diving headlong into hacking.
Iterative development works well for a lot of things (obviously), but I
think that crypto work requires a little bit of Cathedral-esque
architecture in order to be secure. It would pay off to think long and
hard about feedback modes, IVs, key management, salting, etc. ; before
doing any coding.
Posting explanations in sci.crypt would find a good deal more scrutiny
(read: paranoia) than in lkml. Bounce kernel & fs ideas off the kernel
hackers. Bounce crypto ideas off the cryptographers.
Some random thoughts:
Have you given any thought to journalling? fscking? Can directory
contents be encrypted? If so, what does the dir look like to others
(e.g. backup utils)
Per-file signatures will severely affect random access performance.
Changing 1 byte in a 1 GB file would require the whole thing to be reread.
Why tackle versioning? If you want encrypted cvs, you could run cvs in
file mode with the repository and/or modules in encrypted files.
CTR mode (or any stream cipher) would be more susceptible to bit
twiddling than CBC. If sigs are not integral to fs, this would be a problem.
Adding the functionality to a well known fs like ext3 or reiserfs might
be a good path to faster acceptance. It would get a lot more people
looking at the code. It would also allow people to start using the new
features on existing partitions just by patching the kernel.
I'd love to help out. I have some exp with crypto, and a teensy bit of
exp with linux filesystems. How much time I may actually have to
contribute depends on far too many factors to guess.
- Mark Borgerding
Michael A Halcrow wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>I have some time this year to work on an encrypted filesystem for
>Linux. I have surveyed various LUG's, tested and reviewed code for
>currently existing implementations, and have started modifying some
>of them. I would like to settle on a single approach on which to
>focus my efforts, and I am interested in getting feedback from the
>LKML community as to which approach is the most feasible.
>
>This is the feature wish-list that I have compiled, based on personal
>experience and feedback I have received from other individuals and
>groups:
>
> - Seamless application to the root filesystem
> - Layered over the entire root filesystem
> - Unencrypted pass-through mode with minimal overhead
> - Files are marked as ``encrypted'' or ``unencrypted'' and treated
> accordingly by the encryption layer
> - Key->file association
> - As opposed to key->blkdev or key->directory granularity
> - No encryption metafiles in directories, instead placing that
> information into Extended Attributes
> - May break some backup utilities that are not EA-aware; may require
> another mode where encryption metadata is stored in a header block
> on the encrypted file
> - Directories can be flagged as ``encrypted-only'', where any new
> files created in that directory are, by default, encrypted, with
> the key and permissions defined in the directory's metadata
> - Processes may have encryption contexts, whereby any new files they
> create are encrypted by default according to the process'
> authentication
> - Make as much metadata about the file as confidential as possible
> (filesize, executable bit, etc.)
> - Pluggable encryption (I wouldn't mind using a block cipher in CTR
> mode)
> - Authentication via PAM
> - pam_usb
> - Bluetooth
> - Kerberos
> - PAM checks for group membership before allowing access to certain
> encrypted files
> - Rootplug-based LSM to provide key management (necessary to use
> LSM?)
> - Secret splitting and/or (m,n)-threshold support on the keys
> - Signatures on files flagged for auditing in order to detect
> attempts to circumvent the encryption layer (via direct
> modifications to the files themselves in the underlying filesystem)
> - Ad-hoc groups for access to decrypted versions of files
> - i.e., launch web browser, drop group membership by default (like
> capability inheritance masks) so that the browser does not have
> access to decrypted files by default; PAM module checks for group
> membership before allowing access (explicit user authorization on
> application access requests)
> - Userland utilities to support encrypted file management
> - Extensions to nautilus and konqueror to be able to use these
> utilities from a common interface (think: right-click, encrypted)
> - Distro installation integration
> - Transparent shredding, where the underlying filesystem supports it
> - Versioning and snapshots (CVS-ish behavior)
> - Design to work w/ SE Linux
>
>These are features that have been requested, but are not necessarily
>hard requirements for the encrypted filesystem. They are just
>suggestions that I have received, and I am not convinced that they are
>all feasible.
>
>There are several potential approaches to an encrypted filesystem with
>these features, all with varying degrees of modification to the kernel
>itself, each with its own set of advantages and disadvantages.
>
>Options that I am aware of include:
>
> - NFS-based (CFS, TCFS)
> - CFS is mature
> - Performance issues
> - Violates UNIX semantics w/ hole behavior
> - Single-threaded
>
> - Userland filesystem-based (EncFS+FUSE, CryptoFS+LUFS)
> - Newer solutions, not as well accepted or tested as CFS
> - KDE team is using SSHFS+FUSE
>
> - Loopback (cryptoloop) encrypted block device
> - Mature; in the kernel
> - Block device granularity; breaks most incremental backup
> applications
>
> - LSM-based
> - Is this even possible? Are the hooks that we need there?
>
> - Modifications to VFS (stackable filesystem, like NCryptfs)
> - Very low overhead claimed by Erez Zadok
> - Full implementation not released
> - Key->directory granularity
> - Evicts cleartext pages from the cache on process death
> - Uses dcache to store attaches
> - Other niceties, but it's not released...
>
>My goal is to develop an encrypted filesystem ``for the desktop'',
>where a user can right-click on a file in konqueror or nautilus and
>check the ``encrypted'' box, and all subsequent accesses by any
>processes to that file will require authentication in order for the
>file to be decrypted. I have already made some modifications to CFS
>to support this functionality, but I am not sure at this moment
>whether or not CFS is the best route to go for this.
>
>I have had requests to write a kernel module that, when loaded,
>transparently starts acting as the encryption layer on top of whatever
>root filesystem is mounted. For example, an ext3 partition may have
>encrypted files strewn about, which are accessible only after loading
>the module (and authenticating, etc.).
>
>Any advise or direction that the kernel community could provide would
>be very much appreciated.
>
>Thanks,
>Mike
>.___________________________________________________________________.
> Michael A. Halcrow
> Security Software Engineer, IBM Linux Technology Center
>GnuPG Fingerprint: 05B5 08A8 713A 64C1 D35D 2371 2D3C FDDA 3EB6 601D
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.0.6 (GNU/Linux)
>Comment: For info see http://www.gnupg.org
>
>iD8DBQFAFU9wLTz92j62YB0RAkOfAKClVMzKIhw6JtyGvKf8+iFp4e12AwCdFARU
>uAhpA7wVjvPMdDQtKSnFzzI=
>=TM5Y
>-----END PGP SIGNATURE-----
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at http://www.tux.org/lkml/
>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Encrypted Filesystem
2004-01-26 19:06 ` Mark Borgerding
@ 2004-01-26 21:04 ` Felipe Alfaro Solana
2004-01-30 17:01 ` Pavel Machek
0 siblings, 1 reply; 14+ messages in thread
From: Felipe Alfaro Solana @ 2004-01-26 21:04 UTC (permalink / raw)
To: Mark Borgerding; +Cc: Michael A Halcrow, Linux Kernel Mailinglist
On Mon, 2004-01-26 at 20:06, Mark Borgerding wrote:
> Have you given any thought to journalling? fscking? Can directory
> contents be encrypted? If so, what does the dir look like to others
> (e.g. backup utils)
>
> Per-file signatures will severely affect random access performance.
> Changing 1 byte in a 1 GB file would require the whole thing to be reread.
What about calculating signatures on a per-block basis instead?
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Encrypted Filesystem
2004-01-26 17:46 Encrypted Filesystem Michael A Halcrow
2004-01-26 19:06 ` Mark Borgerding
@ 2004-01-27 0:06 ` jw schultz
2004-01-27 0:43 ` Adam Sampson
2 siblings, 0 replies; 14+ messages in thread
From: jw schultz @ 2004-01-27 0:06 UTC (permalink / raw)
To: linux-kernel
On Mon, Jan 26, 2004 at 11:46:29AM -0600, Michael A Halcrow wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I have some time this year to work on an encrypted filesystem for
> Linux. I have surveyed various LUG's, tested and reviewed code for
> currently existing implementations, and have started modifying some
> of them. I would like to settle on a single approach on which to
> focus my efforts, and I am interested in getting feedback from the
> LKML community as to which approach is the most feasible.
>
> This is the feature wish-list that I have compiled, based on personal
> experience and feedback I have received from other individuals and
> groups:
This sounds more like transparent (in-kernel?) file encryption
than an encrypted filesystem. I think there is value in
having both, or all three if you add encrypted block devices.
--
________________________________________________________________
J.W. Schultz Pegasystems Technologies
email address: jw@pegasys.ws
Remember Cernan and Schmitt
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Encrypted Filesystem
2004-01-26 17:46 Encrypted Filesystem Michael A Halcrow
2004-01-26 19:06 ` Mark Borgerding
2004-01-27 0:06 ` jw schultz
@ 2004-01-27 0:43 ` Adam Sampson
2004-01-27 1:42 ` Andy Isaacson
` (2 more replies)
2 siblings, 3 replies; 14+ messages in thread
From: Adam Sampson @ 2004-01-27 0:43 UTC (permalink / raw)
To: linux-kernel
Michael A Halcrow <mahalcro@us.ibm.com> writes:
> - Userland filesystem-based (EncFS+FUSE, CryptoFS+LUFS)
Going off on a tangent...
There are all sorts of potentially-interesting things that could be
done if Linux had a userspace filesystem mechanism included in the
standard kernel -- as well as encryption, there's also network
filesystems, various sorts of specialised caching (such as Zero
Install), automounter-like systems, prototyping and so on.
Is there a technical reason that none of the userspace filesystem
layers have been included in the stock kernel, or is it just that
nobody's submitted any of them for inclusion yet?
--
Adam Sampson <azz@us-lot.org> <http://offog.org/>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Encrypted Filesystem
2004-01-27 0:43 ` Adam Sampson
@ 2004-01-27 1:42 ` Andy Isaacson
2004-01-27 22:01 ` Jan Harkes
2004-01-28 13:50 ` Userspace filesystems (WAS: Encrypted Filesystem) Miklos Szeredi
2 siblings, 0 replies; 14+ messages in thread
From: Andy Isaacson @ 2004-01-27 1:42 UTC (permalink / raw)
To: Adam Sampson; +Cc: linux-kernel
On Tue, Jan 27, 2004 at 12:43:21AM +0000, Adam Sampson wrote:
> Michael A Halcrow <mahalcro@us.ibm.com> writes:
> > - Userland filesystem-based (EncFS+FUSE, CryptoFS+LUFS)
>
> Going off on a tangent...
>
> There are all sorts of potentially-interesting things that could be
> done if Linux had a userspace filesystem mechanism included in the
> standard kernel -- as well as encryption, there's also network
> filesystems, various sorts of specialised caching (such as Zero
> Install), automounter-like systems, prototyping and so on.
>
> Is there a technical reason that none of the userspace filesystem
> layers have been included in the stock kernel, or is it just that
> nobody's submitted any of them for inclusion yet?
There are a lot of subtle and not-so-subtle problems in this space.
For example, I really liked the paging example given in section 3.1 of
[Mazi2001].
[Mazi2001] "A toolkit for user-level file systems", David Mazieres,
Proceedings of the 2001 USENIX Technical Conference
available at http://www.fs.net/sfswww/pubs.html
-andy
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Encrypted Filesystem
2004-01-27 0:43 ` Adam Sampson
2004-01-27 1:42 ` Andy Isaacson
@ 2004-01-27 22:01 ` Jan Harkes
2004-01-27 22:16 ` Jean-Luc Cooke
2004-01-28 13:50 ` Userspace filesystems (WAS: Encrypted Filesystem) Miklos Szeredi
2 siblings, 1 reply; 14+ messages in thread
From: Jan Harkes @ 2004-01-27 22:01 UTC (permalink / raw)
To: linux-kernel
On Tue, Jan 27, 2004 at 12:43:21AM +0000, Adam Sampson wrote:
> Michael A Halcrow <mahalcro@us.ibm.com> writes:
>
> > - Userland filesystem-based (EncFS+FUSE, CryptoFS+LUFS)
>
> Going off on a tangent...
>
> There are all sorts of potentially-interesting things that could be
> done if Linux had a userspace filesystem mechanism included in the
> standard kernel -- as well as encryption, there's also network
> filesystems, various sorts of specialised caching (such as Zero
> Install), automounter-like systems, prototyping and so on.
>
> Is there a technical reason that none of the userspace filesystem
> layers have been included in the stock kernel, or is it just that
> nobody's submitted any of them for inclusion yet?
Ehh, Coda's kernel module does just that. It is used by the userspace
cache manager of the Coda Distributed File System.
http://www.coda.cs.cmu.edu/
But several other projects seem to be using it,
http://uservfs.sourceforge.net/
http://dav.sourceforge.net/
The interface to userspace a bit clumsy to work with, but there are
kernel modules for FreeBSD/NetBSD/Solaris and an experimental one for
Windows 2000/NT/XP, which makes any significant changes a bit of a pain.
It does have it's pecularities, reads and writes are not passed up to
userspace, only the open and close VFS calls. This makes the module
reasonably quite simple as it doesn't have to deal with VM issues and it
isn't susceptible to deadlocks,
app wants to read data from a file ->
userspace application requires memory allocation to provide this data ->
VM tries to write out dirty data associated with the Coda mountpoint ==
deadlock
So whole file caching keeps the kernel module more portable and
simplifies the userspace code. But it makes things like streaming
reads/writes or quotas impossible. If you want to provide encryption
there you would have to store an unencrypted copy of every open file
somewhere on disk or in ramfs/tmpfs and incur the cost of (de)crypting
(and (de)compressing) whenever it is opened or closed.
Jan
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Encrypted Filesystem
2004-01-27 22:01 ` Jan Harkes
@ 2004-01-27 22:16 ` Jean-Luc Cooke
0 siblings, 0 replies; 14+ messages in thread
From: Jean-Luc Cooke @ 2004-01-27 22:16 UTC (permalink / raw)
To: linux-kernel
Ah, can someone explain why encrypted loopback doesn't solve this?
JLC
On Tue, Jan 27, 2004 at 05:01:54PM -0500, Jan Harkes wrote:
> On Tue, Jan 27, 2004 at 12:43:21AM +0000, Adam Sampson wrote:
> > Michael A Halcrow <mahalcro@us.ibm.com> writes:
> >
> > > - Userland filesystem-based (EncFS+FUSE, CryptoFS+LUFS)
> >
> > Going off on a tangent...
> >
> > There are all sorts of potentially-interesting things that could be
> > done if Linux had a userspace filesystem mechanism included in the
> > standard kernel -- as well as encryption, there's also network
> > filesystems, various sorts of specialised caching (such as Zero
> > Install), automounter-like systems, prototyping and so on.
> >
> > Is there a technical reason that none of the userspace filesystem
> > layers have been included in the stock kernel, or is it just that
> > nobody's submitted any of them for inclusion yet?
>
> Ehh, Coda's kernel module does just that. It is used by the userspace
> cache manager of the Coda Distributed File System.
>
> http://www.coda.cs.cmu.edu/
>
> But several other projects seem to be using it,
>
> http://uservfs.sourceforge.net/
> http://dav.sourceforge.net/
>
> The interface to userspace a bit clumsy to work with, but there are
> kernel modules for FreeBSD/NetBSD/Solaris and an experimental one for
> Windows 2000/NT/XP, which makes any significant changes a bit of a pain.
>
> It does have it's pecularities, reads and writes are not passed up to
> userspace, only the open and close VFS calls. This makes the module
> reasonably quite simple as it doesn't have to deal with VM issues and it
> isn't susceptible to deadlocks,
>
> app wants to read data from a file ->
> userspace application requires memory allocation to provide this data ->
> VM tries to write out dirty data associated with the Coda mountpoint ==
> deadlock
>
> So whole file caching keeps the kernel module more portable and
> simplifies the userspace code. But it makes things like streaming
> reads/writes or quotas impossible. If you want to provide encryption
> there you would have to store an unencrypted copy of every open file
> somewhere on disk or in ramfs/tmpfs and incur the cost of (de)crypting
> (and (de)compressing) whenever it is opened or closed.
>
> Jan
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
http://www.certainkey.com
Suite 4560 CTTC
1125 Colonel By Dr.
Ottawa ON, K1S 5B6
^ permalink raw reply [flat|nested] 14+ messages in thread
* Userspace filesystems (WAS: Encrypted Filesystem)
2004-01-27 0:43 ` Adam Sampson
2004-01-27 1:42 ` Andy Isaacson
2004-01-27 22:01 ` Jan Harkes
@ 2004-01-28 13:50 ` Miklos Szeredi
2004-01-30 17:06 ` Pavel Machek
2 siblings, 1 reply; 14+ messages in thread
From: Miklos Szeredi @ 2004-01-28 13:50 UTC (permalink / raw)
To: azz; +Cc: linux-kernel
Adam Sampson <azz@us-lot.org> wrote:
> Is there a technical reason that none of the userspace filesystem
> layers have been included in the stock kernel, or is it just that
> nobody's submitted any of them for inclusion yet?
I'm planning on submitting FUSE for inclusion into 2.7 (and maybe 2.6
if that is acceptable). I feel that FUSE has been maturing lately
with some very interesting applications [1].
Here are the reasons for _not_ including it:
1) There are already two filesystems in kernel that are perfectly
usable for this purpose: NFS and CODA
2) There are currently two competing userspace filesystem layers that
have about the same "market share": LUFS and FUSE. Why should we
prefer one over the other
I'll try to refute myself on these points:
1) Both NFS and CODA were designed for something different. IOW they
are not optimized for the purpose of supporting userspace
filesystems. NFS is slow and there are problems with coherency of
the underlying and the mounted filesystem. CODA does not support
random file access (or even streaming), only reading whole files.
It also has a miriad of small problems when used for exporting a
userspace fs (I've been personally burned many times while doing
AVFS over CODA).
2) This one is harder to get rid of, especially because I don't want
to delve into the technical merits of one or the other (I'd be a
bit biased). But I have compared both the kernel interface and
the library API of LUFS and FUSE and they are very similar. And
that is a good thing, because it makes possible to support LUFS
filesystems with the FUSE kernel module and vica versa.
Miklos
[1] For a list of applications using FUSE see:
<http://www.inf.bme.hu/~mszeredi/fuse/Filesystems>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Encrypted Filesystem
2004-01-26 21:04 ` Felipe Alfaro Solana
@ 2004-01-30 17:01 ` Pavel Machek
0 siblings, 0 replies; 14+ messages in thread
From: Pavel Machek @ 2004-01-30 17:01 UTC (permalink / raw)
To: Felipe Alfaro Solana
Cc: Mark Borgerding, Michael A Halcrow, Linux Kernel Mailinglist
Hi!
> > Have you given any thought to journalling? fscking? Can directory
> > contents be encrypted? If so, what does the dir look like to others
> > (e.g. backup utils)
> >
> > Per-file signatures will severely affect random access performance.
> > Changing 1 byte in a 1 GB file would require the whole thing to be reread.
>
> What about calculating signatures on a per-block basis instead?
Hmm, having md5's in indirect blocks would be very nice for detecting
cable problems ;-). That's usefull even without encryption.
Pavel
--
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Userspace filesystems (WAS: Encrypted Filesystem)
2004-01-28 13:50 ` Userspace filesystems (WAS: Encrypted Filesystem) Miklos Szeredi
@ 2004-01-30 17:06 ` Pavel Machek
2004-02-02 9:42 ` Miklos Szeredi
0 siblings, 1 reply; 14+ messages in thread
From: Pavel Machek @ 2004-01-30 17:06 UTC (permalink / raw)
To: Miklos Szeredi; +Cc: azz, linux-kernel
Hi!
> 2) This one is harder to get rid of, especially because I don't want
> to delve into the technical merits of one or the other (I'd be a
> bit biased). But I have compared both the kernel interface and
> the library API of LUFS and FUSE and they are very similar. And
> that is a good thing, because it makes possible to support LUFS
> filesystems with the FUSE kernel module and vica versa.
Jean-Luc wrote:
> app wants to read data from a file ->
> userspace application requires memory allocation to provide this data ->
> VM tries to write out dirty data associated with the Coda mountpoint ==
> deadlock
How do you solve this one?
Pavel
--
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Userspace filesystems (WAS: Encrypted Filesystem)
2004-01-30 17:06 ` Pavel Machek
@ 2004-02-02 9:42 ` Miklos Szeredi
2004-02-02 15:19 ` Pavel Machek
0 siblings, 1 reply; 14+ messages in thread
From: Miklos Szeredi @ 2004-02-02 9:42 UTC (permalink / raw)
To: pavel; +Cc: linux-kernel
> Jean-Luc wrote:
> > app wants to read data from a file ->
> > userspace application requires memory allocation to provide this data ->
> > VM tries to write out dirty data associated with the Coda mountpoint ==
> > deadlock
>
> How do you solve this one?
1) In FUSE normal writes go through the cache, so no dirty pages are
created. The only possibility to create dirty pages is with shared
writable mapping, and this is rare
2) Userspace filesystem app can be multithreaded, so probably write
can be satisfied even if read is pending.
3) The 2.6 kernel provides asynchronous page writeback, so even if a
writeback is blocking forever the VM will continue to try to free
up memory.
4) If no memory can be freed, then the allocation will fail, so the
read will fail: no deadlock.
5) Even if the memory allocation was caused by a page fault, which
cannot fail, the worst case is that the OOM killer is invoked, and
memory is freed up: no deadlock.
So with the asynchronous page writeback mechanism the 2.6 kernel is
very immune to this kind of deadlock. I have tested this with a
little program which behaves very nastily in this respect (I can send
you this if you're interested). And I was able to invoke the OOM
killer if there was no swap, but there was never a deadlock.
Miklos
This communication is confidential and intended solely for the addressee(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you believe this message has been sent to you in error, please notify the sender by replying to this transmission and delete the message without disclosing it. Thank you.
E-mail including attachments is susceptible to data corruption, interruption, unauthorized amendment, tampering and viruses, and we only send and receive e-mails on the basis that we are not liable for any such corruption, interception, amendment, tampering or viruses or any consequences thereof.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Userspace filesystems (WAS: Encrypted Filesystem)
2004-02-02 9:42 ` Miklos Szeredi
@ 2004-02-02 15:19 ` Pavel Machek
2004-02-02 15:36 ` Nikita Danilov
0 siblings, 1 reply; 14+ messages in thread
From: Pavel Machek @ 2004-02-02 15:19 UTC (permalink / raw)
To: Miklos Szeredi; +Cc: linux-kernel
Hi!
> > > app wants to read data from a file ->
> > > userspace application requires memory allocation to provide this data ->
> > > VM tries to write out dirty data associated with the Coda mountpoint ==
> > > deadlock
> >
> > How do you solve this one?
>
> 1) In FUSE normal writes go through the cache, so no dirty pages are
> created. The only possibility to create dirty pages is with shared
> writable mapping, and this is rare
>
> 2) Userspace filesystem app can be multithreaded, so probably write
> can be satisfied even if read is pending.
>
> 3) The 2.6 kernel provides asynchronous page writeback, so even if a
> writeback is blocking forever the VM will continue to try to free
> up memory.
>
> 4) If no memory can be freed, then the allocation will fail, so the
> read will fail: no deadlock.
Transient real failure looks pretty ugly. I'd not expect
read(/etc/passwd) to return -ENOMEM, and read(/#ftp:somewhere/passwd)
should be the same, but as this is basically "can not happen"... I
guess that's enough.
Pavel
--
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Userspace filesystems (WAS: Encrypted Filesystem)
2004-02-02 15:19 ` Pavel Machek
@ 2004-02-02 15:36 ` Nikita Danilov
0 siblings, 0 replies; 14+ messages in thread
From: Nikita Danilov @ 2004-02-02 15:36 UTC (permalink / raw)
To: Pavel Machek; +Cc: Miklos Szeredi, linux-kernel
Pavel Machek writes:
> Hi!
>
[...]
>
> Transient real failure looks pretty ugly. I'd not expect
> read(/etc/passwd) to return -ENOMEM, and read(/#ftp:somewhere/passwd)
read(/etc/passwd) can fail with -ENOMEM while allocating a page in the
page cache.
> should be the same, but as this is basically "can not happen"... I
> guess that's enough.
> Pavel
Nikita.
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2004-02-02 15:36 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-01-26 17:46 Encrypted Filesystem Michael A Halcrow
2004-01-26 19:06 ` Mark Borgerding
2004-01-26 21:04 ` Felipe Alfaro Solana
2004-01-30 17:01 ` Pavel Machek
2004-01-27 0:06 ` jw schultz
2004-01-27 0:43 ` Adam Sampson
2004-01-27 1:42 ` Andy Isaacson
2004-01-27 22:01 ` Jan Harkes
2004-01-27 22:16 ` Jean-Luc Cooke
2004-01-28 13:50 ` Userspace filesystems (WAS: Encrypted Filesystem) Miklos Szeredi
2004-01-30 17:06 ` Pavel Machek
2004-02-02 9:42 ` Miklos Szeredi
2004-02-02 15:19 ` Pavel Machek
2004-02-02 15:36 ` Nikita Danilov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®