Selfhosted

60093 readers

859 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam.
Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.
Don't duplicate the full text of your blog or git here. Just post the link for folks to click.
Submission headline should match the article title.
No trolling.
Promotion posts require your active participation in selfhosting or related communities, or the post will be removed. No more than 10% of your posts or comments may be self-promotional, or your post will be removed. F/LOSS Exception: If your post is about a project that is completely open source & can be self-hosted in full without payment, your post is exempt from this rule as long as you continue to engage in comments.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

Raidz2 or btrfs for important document storage? (piefed.blahaj.zone)

submitted 1 month ago* (last edited 1 month ago) by CorrectAlias@piefed.blahaj.zone to c/selfhosted@lemmy.world

41 comments fedilink hide all child comments

I currently have a secondary pool (with raidz2) that I was originally going to use for my important documents, such as storage for Paperless-ngx, as raidz offers corruption detection and repair. The pool is encrypted.

However, I'm concerned about rebuild times (it's a pool of 4 22TB drives). Is btrfs a better choice for this use case, or should I just go with raidz like I originally planned?

Edit: I should have mentioned that I already have 4-3-2 backups configured - I'm primarily interested in the "self-healing" aspect of ZFS so that I don't have to recover from backups unless necessary, and to resolve corruption on the fly without me having to notice that a file is corrupt.

you are viewing a single comment's thread
view the rest of the comments

[–] felbane@lemmy.world 26 points 1 month ago* (last edited 1 month ago) (2 children)

RAID is not a backup.

RAID is not for data safety.

RAID is for:

Ensuring availability of data in the face of hardware failure. That means your files don't disappear when a drive dies and you have some time to swap out for functional hardware and restore redundancy.
Presenting multiple drives as one larger unit. This is what striping does, and to a lesser extent the parity-mode levels.
Improving performance (sometimes). A RAID mirror is generally much faster to read from than any individual drive because reads can be interleaved across drive members. A stripe can be much faster because writes are distributed across drive members. This is less of a bonus today with solid state/nvme drives, but it's still applicable to spinning rust.

If your concern is protecting your data, set up a 3-2-1 backup strategy.

[–] CorrectAlias@piefed.blahaj.zone 3 points 1 month ago* (last edited 1 month ago)

I'm very aware and have full 4-3-2 backups already. I'm also not interested in a standard raid. Thanks though! It's always good to mention that raid is not a backup. I simply want to add more protection from disk corruption (not necessarily full failure) so that I don't need to recover from backups unless I absolutely must. A benefit would also be resolving corruption before I even notice it.

[–] zyberwoof@lemmy.world 2 points 4 weeks ago (1 children)

RAID is not a backup.

True

RAID is not for data safety.

Not true.

RAID helps prevent data loss in the event that a drive failure occurs before changes are replicated to backups. If you upload photos and then delete them from your camera, they will likely be stored in just one location for a period of time. If you have a drive failure without RAID, you could lose your only copy.
With ZFS, RAID can be used to protect against things like bitrot. Even data at rest can become corrupt over time. ZFS stores checksums of the files to know if corruption occurs. And with one or more parity drives, ZFS can automatically repair the corruption when detected. Without this, detecting and fixing these kinds of issues can be much more difficult.

I'm in 100% agreement that RAID is not essential, and that backups are a much higher priority. In fact, without backups in place, I'm not generally in favor of RAID. RAID adds additional complexity. That complexity can result in data loss. Especially due to user error. But once backups are part of the equation, RAID can add additional layers of security for your data.

[–] felbane@lemmy.world 1 points 4 weeks ago

Yes, valid points that I didn't really want to muddy the discussion with. In my experience, most people hear the Redundant in RAID and think that's sufficient and that their data is safe. Maybe poor choice of words on my part, but that's what I meant re: "data safety."

You're fully correct that a proper RAID setup can provide additional layers of availability atop a robust backup process, but I'd wager most of the people who are interested in that extra layer are already aware of the limitations of RAID.

I do run RAIDZ1 on my franken-nas due to limited drive sizes on that machine, the goal being to maximize usable space while providing a sufficient amount of time to address a drive failure. If drive prices ever become reasonable again, I'll likely rebuild the system with 6-8 drives in a RAIDZ2 configuration just for a little more peace of mind, but as long as my off-site backups are running, I sleep at night just fine.