Re: filesystem with integrity checking

[Jeff Moyer <jmoyer@redhat.com>]

crimil tradalo <crimiltradalo@gmail.com> writes:

> I am looking for a filesystem that supports integrity checking of
> files with some kind of checksum/hash.
>
> For example, for each block an SHA hash is stored when written and
> verified when read. If the underlying hardware has corrupted the data,
> or returned a different block than expected, the filesystem can detect
> the error.
>
> Obviously, ideal would be integration with the RAID layer so that
> valid data could be found in the redundancy, similar to ZFS, but I
> don't expect this to exist.
>
> The last time I saw this discussed, the consensus seemed to be that it
> wasn't worth the CPU time. I hope that as CPUs have gotten faster and
> data sets larger, more people are noticing how frequent data
> corruption occurs and are beginning to respect the value of an OS
> ensuring the integrity of data on permanent storage. Everyone here
> must have experienced data corruption that is either seemingly-random
> or traced to bad hardware. I'd like to detect such corruption as soon
> as it happens, rather than after it has caused a catastrophic loss of
> data.
>
> If there is none that currently supports this, I am curious if any
> intend to support it eventually. I see it mentioned in BTRFS
> documentation, but considering the low odds of a new filesystem
> actually becoming viable, I'm more interested in a practical solution.
>
> My understanding is that the only filesystems that offer this are ZFS
> and WAFL, neither of which is going to be useful on Linux of course.

You may want to take a look at the DIF/DIX work that Martin K. Petersen
is doing:
  http://oss.oracle.com/projects/data-integrity/

Cheers,
Jeff