This kind of shit will only increase as more of these companies believe they can vibe-code their way out of paying software devs what they are worth.
Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
It's wild that these cloud providers were seen as a one-way stop to ensure reliability, only to make them a universal single point of failure.
But if everyone else is down too, you don't look so bad 🧠
No one ever got fired for buying IBM.
I wouldn't be so sure about that. The state government of Queensland, Australia just lifted a 12 year ban on IBM getting government contracts after a colossal fuck up.
It's an old joke from back when IBM was the dominant player in IT infrastructure. The idea was that IBM was such a known quantity that even non-technical executives knew what it was and knew that other companies also used IBM equipment. If you decide to buy from a lesser known vendor and something breaks, you might be blamed for going off the beaten track and fired (regardless of where the fault actually lay), whereas if you bought IBM gear and it broke, it was simply considered the cost of doing business, so buying IBM became a CYA tactic for sysadmins even if it went against their better technical judgement. AWS is the modern IBM.
AWS is the modern IBM.
That's basically why we use it at work. I hate it, but that's how things are.
One of our client support people told an angry client to open a Jira with urgent priority and we'd get right on it.
... the client support person knew full well that Jira was down too : D
At least, I think they knew. Either way, not shit we could do about it for that particular region until AWS fixed things.
It's mostly a skill issue for services that go down when USE-1 has issues in AWS - if you actually know your shit, then you don't get these kinds of issues.
Case in point: Netflix runs on AWS and experienced no issues during this thing.
And yes, it's scary that so many high-profile companies are this bad at the thing they spend all day doing
Yeah, if you're a major business and don't have geographic redundancy for your service, you need to rework your BCDR plan.
It is still a logical argument, especially for smaller shops. I mean, you can (as self-hosters know) set up automatic backups, failover systems, and all that, but it takes significant time & resources. Redundant internet connectivity? Redundant power delivery? Spare capacity to handle a 10x demand spike? Those are big expenses for small, even mid-sized business. No one really cares if your dentist's office is offline for a day, even if they have to cancel appointments because they can't process payments or records.
Meanwhile, theoretically, reliability is such a core function of cloud providers that they should pay for experts' experts and platinum standard infrastructure. It makes any problem they do have newsworthy.
I mean,it seems silly for orgs as big and internet-centric as Fortnite, Zoom, or forturne-500 bank to outsource their internet, and maybe this will be a lesson for them.
Why do we place so much reliance on one mega company? This level of importance. It should be seized by the government.
Do you really want someone like the magahats having control over something like that?
It should be seized by the ~~government~~ people and mercilessly decentralized.
Agreed same for Facebook then call it Readabook
Why do we place so much reliance on one mega company? This level of importance.
Because it's cheaper and (in broad terms) more reliable than everybody having a data centre.
It should be seized by the government.
Oh yeah, what could possibly go wrong if the US government owned Amazon!
So, you changed one cloud provider to another...
But let me rephrase: cloud can be significantly cheaper - if you know what you're doing and what you're putting on the cloud.
I've been to data centres that cost as much as a decade of cloud hosting the service they were supporting (and that's without operational costs).
Cloud is especially great for small businesses where you have two alternative options: either build your own data centre which you absolutely cannot afford (or risk making it barely operational and unreliable) or host your company at someone else's DC - which is what cloud is, but worse (because nobody can set up so much resiliency and have so many DC techs/admins as Microsoft or Amazon).
There absolutely are situations where self-hosting is preferable, and even cheaper, but wondering "why do we place so much reliance" on cloud service providers just shows that people have no clue what cloud actually offers.
Best alternatives is making Amazon something owned by the people and not any corporation/government but who knows if that would ever happen
AWS aggressively pursues high priced and years-long spending commitments with large customers, and they incentivize it with huge discounts for doing so.
And when AWS does this they intentionally incentivize these large customers to migrate existing workloads away from other cloud service providers as well, going so far as to offer assistance in doing so.
I hate how Signal went down because of this... Wish it wasn't so centralised.
My friend messaged me on Signal asking if Instructure (runs on AWS) was down. I got the message. That being said, it's scary that Signal's backbone depends on AWS
I have been able to use Signal like any other day. I haven’t seen any disruption in sending or receiving.
according to that page the issue stemmed from an underlying system responsible for health checks in load balancing servers.
how the hell do you fuck up a health check config that bad? that's like messing up smartd.conf and taking your system offline somehow
Well, you see, the mistake you are making is believing a single thing the stupid AWS status board says. It is always fucking lying, sometimes in new and creative ways.
If your health check is broken, then you might not notice that a service is down and you'll fail to deploy a replacement. Or the opposite, and you end up constantly replacing it, creating a "flapping" service.
Who wants to bet Amazon gave AI full access to their prod config and it screwed it up.
A bad day for Jeff Bezos is a good day for all of us
Yeah, was reading about it here too
Ring doorbells, Alexa, ahh... the joys of selfhosting.
That explains why my Matrix <-> Signal bridge was complaining about being disconnected.
that is an understatement 😂