Everyone asks "where do we get more storage?" and not "do we need to hoard all of this?"
Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil.
-
No spam.
-
Posts are to be related to self-hosting.
-
Don't duplicate the full text of your blog or readme if you're providing a link.
-
Submission headline should match the article title.
-
No trolling.
-
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
-
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
The answer is yes.
Seconded.
Because the answer to the second question is a very clear "yes".
Yes. If I want to organize and dedupe what I have then I need enough storage to work on it, a lot of my storage is spinning rust 7-15 years old, and if I have the space I'm going to use it. I have family photos and a music library going back to 2005. Too many things like old games need custom fixes installed to work correctly on modern hardware, and the internet isn't as permanent as it was cracked up to be.
There's plenty of reasons to hold on to older data.
Aren't old games pretty small though? It's new ones that you may need a huge volume to store many of them. Depends how much we are talking of course. 2TB or 50TB?
With how the internet is going, I don't think we will be able to get content from it in 5 to 10 years. It will be completely locked down, so all we have on our drives will be it. Back to mailing DVDs!
Usenet will likely still be around, and torrents are like playing whack-a-mole.
It will just be a lot harder to get to, and likely harsher laws put in place as well.
I mean, not allowing vpns for personal use would stop us all. Businesses would of course be allowed to use them.
"I mean, not allowing vpns for personal use would stop us all."
Yes and no. We'd go back to sneakernets. :)
... But that would be significantly more unpleasantly limited than the current ways of doing things...
And sneakernets would make personal archives more valuable than ever
I think a big fuel for these storage anxieties is the very real situation we're in right now, where we're watching the "forever Internet" erode and crumble before our eyes, and getting rug-pulled from every direction service-wise, and losing access to media we don't have a hard copy of.
I do wish there were a better way to pool all this storage for a common library of preservation...I mean I guess Internet Archive is like that but they're constantly under attack. All this is under heaps of legal "gray area" and obviously the media titans want to force a rental-only-own-nothing world.
Right now we kinda have to become a scattered group of amateur historians and librarians, to preserve our culture.
People treat deleting like some dirty word, but all good collections need to be organized and pruned.
You donβt even necessarily need to delete either. If you have a ton of H.264 video you could convert it to 265 or AV1 with minimal quality loss, but huge space savings.
If you torrent that means stopping seeding tho
My only complaint is that lots of my streaming devices donβt natively support newer codecs. So if I convert everything to AV1, my server will end up transcoding basically everything. Smart TVs are particularly bad about supporting anything past h264.
We wouldn't be here if we hadn't already answered the second question affirmatively.
Frugally (legally) obtained HDDs are still going to cost you a lot of money, there is no way around that at the moment. If you need it, pay up and be done with it. If you just kind of want it, start sorting through your piles of data you don't actually need (yes, you have that, stop lying to yourself) to free up space for things you do actually need.
True, but it'll be nice to have enough space to let the future arr stack I plan on spinning up, roam free
In my country we have a website that resells "old" and used server hardware, including HDDs for reasonable prices. Although that has gone up a lot over the last year or so.
Maybe you have something like that in the Netherlands? I recently bought an 18TB drive for around β¬400.
Storage is just expensive these days. Just like RAM.
Ebay, particularly GoHardDrives, or sometimes you'll find new drives from random sellers.
I also check ServerPartDeals. Drives are pricy these days, don't sneeze near your NAS.
Edit: I'm not sure if they ship internationally or not, however.
2/3 of the drives I received from GoHardDrive failed within 2 years and all I got was a pre-bubble refund.
Honestly, Iβve been taking a chance on eBay. If the price is close to $10/TB and the drive is an enterprise drive that is listed as known to be working, with a good return policy, I take the risk. I just run tests as soon as I have it so far, all have been good (eight purchases).
Same. I use these drives in a mirrored setup or to hold data that's replaceable. If they make it a few months without showing errors they might get entrusted to something more important. I'll shell out for a new drive for my constantly in use drives, luckily I read the warning signs and bought a few 20tb HDDs when ram and SSDs started skyrocketing. I'm kicking myself for not grabbing a few backup m.2 nvme and 2.5" SSDs, because I'm already doing the hardrive shuffle in my mini PCs. I'm fine living life with networked rusty spinners, but I really really don't want to go back to spinny boot/high throughput drives.
Frugal and recycled. Using a mix of old disks in OMV8 with a mergerfs array, suprisingly they amounted to 20TB, so saved a few bob.
If a disk fails in Mergerfs you loose the data on that disk only and not the array as you would with a pure jbod array.
There is a remove disk utility in the OMV8 mergerfs plugin that allows the data from a failing disk to be copied back to the array, if enough space is available, retaining the data from the failing disk. The disk can then be physically removed. If the array is short on space, add a disk to expand the array before removing the failing disk.
I've only ever used it with a failing disk, not a failed disk, I would guess that a backup would be required in that case.
The same applies to mergerfs outside of OMV8, this was easier for me.
I still have some IOMEGA Zip drives. LOL Man, I remember when those seemed inexhaustible.
My work sells commercial tier drives that were used by customers in NVRs for $10/TB. It's honestly a great perk of the job for a data hoarder like myself with a fully redundant 100TB of stuff (aka 200TB of drives). I also got a dope 16 bay server chassis with slides and a few other components from them. I fully intend to drop like 2 grand on HDDs if/when I move to a new company, assuming they don't fire me for AI or something first.
I usually scavenge old drives from work. On one hand they're a bit smaller than I'd like them to be, but on the other hand they're free except from the minor work and documentation involved in ensuring that no company related data remain.
Wish my company allowed that. Everything goes to a licensed secure destruction service that literally puts them through an industrial shredder. Awesome to watch, but wasteful as all hell.
Well, there's a footnote on my end: Me taking the drives home is a bit of a grey area, as the procedures say that the drives are to be mechanically destroyed when no longer needed. It doesn't specify needed by whom. And I do attack them with my angle grinder, so it's in accordance with company policy.
And yes, my employer knows and is OK with it. We go through a ridiculous amount of drives due to large storage needs, so pragmatism tends to trump bureaucracy.
Personally I just track sales constantly. I know I don't want less than my smallest, so I look for 14TB and up. If I come.across an upgrade for the right price, I buy it even if I don't need it right now. The drive I replace moves to another array, so its not wasted. Hell I'm still using some 2-3TB drives in the (much larger in qty) backup array.
The only thing I'd point out with the DC is they may not even have the hardware in there - there's a stupid amount of money that is being counted against product not even installed (or even shipped yet) in the stock value game these narcissistic scumbags are playing.
Well let's start with how much you need.
Then we can all cry. Currently looking at replacing HDDs with SSDs and significantly cutting down on my data storage requirements - basically uninstalling all those games I haven't played in a long time and probably won't. Plus it's easier to avoid getting sucked into playing ESO and wasting money on it if it's behind a 100+ GB download. Majority of games I actually play are under 5GB so I could go pretty heavy. Couple second hand 512GB SSDs perhaps? Under Β£100...
I've had good success buying second hand on eBay, but I bet you could also do worse than getting used parts off Gumtree, look for anyone selling a broken or outdated computer - or in the free section - and spend some time going through a pile of ewaste, shucking all the drives, and then running tests on them.
trash bins π sometimes you can get permission, other times not. π€· I've gotten permission to dig in this one trash bin, and it had a ton of decent 3tb hdds from server rack. pretty hard use and 1/3 of the ones I picked was overheating but the rest were still good!
Get lucky - just as this was starting I saw a failed disk notification on my NAS so I ordered a new drive just before they went up. Then I realized it was a stale notification for the drive I replaced a year ago so I have a spare should one fail.
