Re: [RFC/PATCH 0/2] ext4: Transparent Decompression Support

["Jörn Engel" <joern@logfs.org>]

On Wed, 7 August 2013 03:21:47 -0600, Andreas Dilger wrote:
> 
> I'm not saying that a tool to check this would be a bad thing, but
> if the compression support is a generic feature of the VFS, then it
> makes sense that the checker can also be generic and unrelated to
> the filesystem metadata checking as well.  It may be "gunzip -t"
> or LZO equivalent is enough to determine if the file is/isn't in a

Careful!  If you have a 1GB file in gzip format and want to
demand-page the last page from it, you have to gunzip the _entire_
file before you can uncompress that page.  There is no alternative to
splitting the file into chunks of some reasonable size and therefore
"gunzip -t" will never be enough.

That also means that data corruption has more impact here than it
would have for an uncompressed file.  Instead of handing the corrupted
data to userspace and let it deal with the results, the kernel has to
interpret some header, some list of chunks and finally the compression
format.  Any bugs here will lead to crashes or privilege escalations
that we are responsible for.  We cannot just say "garbage in, garbage
out, let userspace handle it".

Do we return zero-filled pages in case of data corruption?  Or should
we return -EIO on reads and segfault on access of mmap'ed pages?  Or
does the filesystem really own the metadata, which in this case
includes the header and chunk list and potentially some bits of the
compression format, and adds checksums, etc. to said metadata?

Jörn

--
Somewhere around the year 2000 there was this turningpoint when it
became cheaper to collect information than to understand it.
-- Freeman Dyson