7

Windows has a "Resilient FileSystem" since Windows 8.

Are there similarly resilient filesystems for Linux? What I expect from such a filesystem is that a bad block won't screw up either files or the journal. I'm no FS geek, so please explain if such error-resilience is unfit for a desktop/CPU intensive/memory intensive/lowers the HDD's lifespan/is already in some FS like Ext4/etc.

Is there something like this available for Linux?

Improve this question
2
  • 2
    As others point out "ZFS", I just hit on your other points: (A.) Linux support is ongoing, and gparted supports it. (B.) ReFS, ZFS, are more expensive than ext4, NTFS, etc. But have different purposes. (C.) Though ReFS will be extended in the future, it is not, and cannot be, a replacement for NTFS--now : blogs.technet.com/b/askpfeplat/archive/2013/01/02/…; (D.) I use ReFS to store very large directory structures of source code and large trees of EncFS encrypted files with large file names; Commented May 13, 2015 at 5:13
  • ZFS on linux is a terrible idea. If you want a storage appliance and you want ZFS, choose a BSD and use it. Commented Apr 18, 2018 at 17:10

3 Answers 3

Reset to default
9

If you're looking for advanced filesystems for general-purpose computers in the Linux world, there are two candidates: ZFS and BTRFS. ZFS is older and more mature, but it's originally from Solaris and the port to Linux isn't seamless. BTRFS is still under heavy development, and not all features are ready for prime time yet.

Both filesystems offer per-file checksumming, so you will know if a file is corrupted; this is more of a security protection than a protection against failing hardware, because failing hardware tends to make a file unreadable, the hardware has its own checksums so reading wrong data is extremely unlikely (if a disk read returns wrong data, and you're sure it's not an application error, blame your RAM, not your disk).

If you want resilience, by far the best thing to do is RAID-1 (i.e. mirroring) over two disks. When a disk starts failing, it's rare that only a few sectors are affected; usually, more sectors follow quickly, if the disk hasn't stopped working altogether. So replicating data over the same disk doesn't help very often. Replicating data over two disks doesn't need any filesystem support. The only reason you might want to replicate data on the same disk is if you have a laptop which can only accommodate one disk, but even then the benefits are very small.

Remember that no matter how much replication you have, you still need to have offline backups, to protect against massive hardware failures (power surge, fire, …) and against software-level problems (e.g. accidental file deletion or overwrite).

Improve this answer
5
  • 1
    So what you're saying is that if I have any bad blocks it's a sign the disk will soon belly-up so there's no point in a filesystem trying to circumvent them? (I thought bad blocks were something that could happen sometime and the disk would still be good for the most part). Commented Dec 29, 2011 at 15:33
  • I don't know what the exact statistics are, but partial failures aren't the dominant failure mode for disks. It can happen that a few blocks get bad and that's it, but it's about as common that a few blocks get bad and more and more, or that the disk simply won't start, so you need to be prepared for that anyway. Commented Dec 29, 2011 at 16:04
  • Thank you. I plan in the near (or not) future to give both a spin, and in either case backups are something I'll try to worry more about. I'll accept your answer since it's the most complete one, and because I've learnt what I needed to. Thank you all. Commented Dec 29, 2011 at 18:13
  • 1
    @Gilles .. Your answer is misleading: (A.) RAID 1 does not address the corruption issues that are relevant here, (the link below sheds light on this). (B.) RAID is not at all analogous to software solutions, as the different features exist. (C.) There are plenty of articles about this, but starting here will help shed light: networkcomputing.com/storage/… Commented May 13, 2015 at 4:54
  • @e.s.kohen (A) RAID 1 doesn't address corruption, but that's not what this question is about. It's about the common disk failure mode where a part of the disk becomes unreadable. (B) I don't understand the dichotomy between “RAID” and “software solutions”, RAID vs other types of volumes and hardware vs software are on different axes. (C) I'm sure there are other features of ReFS that my answer doesn't address, I'm only addressing the features mentioned in the question. Commented May 13, 2015 at 12:37
4

BtrFS can do "RAID1" with a single HDD. Meaning it will put a file twice across the disk. It also stores a checksum of each file, if one file becomes corrupted it can give you the other copy.

Check out their wiki.

Improve this answer
4
  • Oh that's nice. So let's say one of the two copies caputs, the other one will automatically overwrite the corrupted one? What if both files contain errors but in different blocks? Commented Dec 28, 2011 at 11:03
  • Then you're SOL. As usual, Microsoft promises revolutionary new features that, once you look into what they actually do, really fail to impress... Commented Dec 28, 2011 at 23:34
  • @Shadur can't agree any more. Otherwise I wouldn't be asking on Unix SE :) But this FS really looks cool, auto-defrag, compression, and error-resilience via RAID1 on the same disk looks very good. Besides I believe two bad blocks for the same file on two different parts of an HDD is the definition of SOL. Commented Dec 29, 2011 at 14:12
  • @Shadur: I don't know how btrfs actually handles the situation you describe but ZFS automatically recovers two bad blocks for a locally mirrored file. Commented Dec 29, 2011 at 17:14
4

ZFS has by default multiple copies of every meta data block. You can enable this feature for data blocks and then have some protection against (localized and non massive) disk errors.

http://blogs.oracle.com/bill/entry/ditto_blocks_the_amazing_tape

Automatic ZFS Snapshots are also a popular way to protect files against accidental deletion or corruption.

Improve this answer
2
  • Interesting read about ZFS, I didn't think about filesystems as trees of blocks before. Snapshots also seem like a very handy (and new to me) feature, but more for protecting files against human rather than software/hardware errors. Commented Dec 29, 2011 at 17:02
  • @Camilo Martin: Of course, that depends on many factors but I expect software and human errors to be more common than hardware ones ... Commented Dec 29, 2011 at 17:20

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.