Technology

They migrated from an OS and MySQL version receiving no updates since at least 2 years to MySQL 8.0 which will stop getting updates in 4 days.

I agree that it was an odd choice, as well as the OS. Going to Alma 9 when Alma 10 had already been out for some time. Would think if they wanted the long term updates they would have gone to 10 to get the most out of it. If they went from 8 to 9, sure, some people like staying in the area where RedHat got bored and won't mess with it anymore, but 7 to 9 suggests they didn't do timely upgrades before.

Also every service is running without any containerization and there is a single database for everything

Well, he said explicitly they have 30 databases, though I suppose you meant a single mysql instance. I will say I won't judge one way or another about containerization, as I've seen about as much amateur hour containerization to not immediately judge one way or another on that.

it all runs on a single host

Yeah, that seems pretty dire given his stated usage scenario, and it seems very explicit that their entire internet facing world is that single host...

backup strategy or disk encryption

It was a post narrowly discussing migration, so I don't expect a full inventory of everything they do, so backup strategy and disk encryption and all sorts of other things may be omitted as having nothing to do with the core thing. I guess the most red flag on this front is he explicitly mentions the old setup having "backups enabled" and new setup having "RAID1", which does make me wonder if they think RAID1 is a credible answer for "backup".

Also not a single word about infrastructure as code

Again, not necessarily in-scope for this document, so not sure if I'm going to judge on this one. I routinely take material expressed in terms of an ansible play and "generic it out" for general consumption when discussing with people outside my organization.

The whole stuff is hosted in Germany for a Turkish software company

I'll confess to not liking it being in a single site, however to the extent they select a single site, Germany might make sense because:

Several live mobile apps serving hundreds of thousands of users

Their userbase may be better connected to Germany than Turkey, and the user latency matters more.

My biggest concerns would be mitigated if they said that the German hosted server is their off-prem solution but it is also hosted on-prem giving them multiple sites, but I think that's a bit much to imagine given the process described. The described migration process wouldn't make sense in that scenario.

the guy who designed it is stuck somewhere 10-20 years in the past.

Well, using containerization for everything is very 2015-ish.

My personal conclusion:he knows what he's doing.

Sounds like my homelab has better redundancy than these guys, and my monthly bill isn't much different than their new one. I only pay for power and networking, since I own my own hardware. I'm colocating in my city, so my latency to home is about 1ms, and I've got a full mirrored server in my house. Certain files are further backed up elsewhere for proper 3-2-1 backup (+ each server running raidz2 with disk encryption). Even if my home Internet goes out, I still have full access to my files at home, and all my public services stay running in the data center. If either server fails, it's all set up with containers so it's easy to spin up each service somewhere else.

One thing that's tricky to get right with disk encryption (especially with encrypted /boot) is having a redundant boot partition. I was able to hack this together by having sofware raid duplicate my boot partition to a second drive. Now if I remove either OS boot drive it falls back to the remaining one. To prevent breaking EFI boot, you need to use the Version 1 RAID format so the metadata is stored at the end of the partition, not the front where EFI reads.

every service is running without any containerization and there is a single database for everything... and it all runs on a single host and I didn't read one word about a backup strategy or disk encryption.

Man, a paragraph that can give someone some serious PTSD flashbacks....

The number of times I've had to clean up a customer's environment after they let little Billy play corporate IT and things went boom.....

It's probably just a knee jerk inclusion.

As you say other solutions may be just as well or marginally better, but this is an all around good approach that even when pointless doesn't hurt anything.

0? My energy company says I'm using power equivalent to a family of eight. And it's just wifey , the servers and me. I had cops here asking if I grow weed 😁

So unless you steal power, it surely isn't close to 0 😁

I also self host and I wouldn't say the cost is zero. In the UK, energy costs alone mean that a 40W computer cost £8 per month to run (assuming a 28p/kWh price).

Of course, that's assuming you run it 24/7 at full energy use, but I know my PCs run on more than that.

One thing is that I don't know for sure if it is containerized or not. The topic was migration, and that facet would be not relevant to the core. When I'm doing a write up of things like this, I tend to omit details like that unless it is core to the subject at hand. Including replacing a funky ingress situation with a more universally recognizable nginx example. The users of a container setup would understand how to translate to their scenario.

For another, I'll say that I've probably seen more people getting screwed up because they didn't understand how to use containers and used them anyway. Most notably they make their networking needlessly convoluted and can't understand it. Also when they kind of mindlessly divide a flow for "microservices", they get lost in the debug.

They are useful, but I think people might do a lot better if they:

• More carefully considered how they split things up
• Go ahead and use host networking, it's pretty good
• unix domain sockets can be your friend instead of binding to tcp for everything. I much favor reverse proxy to unix domain instead of handling IP/ports, which is what the container networks buy most people but the flow is too gnarly
• Be wary of random dockerhub "appliances", they tend to be poorly maintained.

If you are writing in rust or golang, containers might not really buy you much other than a headache, so long as you distinct users for security isolation. For something like python, it might be a more thorough approach that virtalenv, though I wouldn't like to keep a python stack maintained with how fickle the ecosysyem is. Node is pretty much always "virtualenv" like, but even worse for fickle dependencies.

Totally fine. Containerization comes at a cost too. It's a matter of system design, knowing your risks and complexities, and handling them accordingly.

With such a size, before containerization I'm wondering if these services are not independent enough to split them onto multiple servers.

Having everything together reduce system complexities in some ways, but not in other ways.

Baremetal is definitively making a comeback. But you should have some way of orchestrating the deployment as code regardless.

Yes it’s ok, in general. It’s not the most modern or efficient way of managing infrastructure but it’s worked for decades now. It all depends on what you’re hosting, for who, and for how many people.

If you’re hosting internal company infrastructure for a relatively static number of users in a single of set few regions to deliver to, bare metal monolithic stuff is absolutely fine. It’s when you’re an app or service company and your infrastructure is for the back end for a public service that needs to be able to scale dynamically, and you’re worried about high 24/7 uptime, and latency to end users is a global issue that things like microservice architecture, containerization, and iac starts becoming important.

The whole containerization crazy is important for microservices architecture where you split your app into different pieces. This lets you scale different parts of you app as needed, it prevents your entire app from failing just because one part of it failed, it allows for lifecycle management like blue/green deployments with no downtime, allows developers to work on different parts of the app and update at a faster cadence than one big release for the entire thing every time you update one small part of it, things like that.

For example the issue of MySQL 5 being unavailable would be a non-issue with a container

So people careless enough to "just container it" for old, possibly security-compromised software - you call that a "non-issue"? How about upgrading and configuring for compatibility?

Lots of respectable EU hosting companies, and also aparently OVH, if they think there's a chance you're taking the piss will ask for a ID so they can ban you. It's not just anti-spam, it's anti-abuse and for preventing non-payment. They think there was a risk involved in accepting your business (whatever that may be, obviously companies don't dilvulge their criteria here), and if you go elsewhere they're not upset about it for that reason.

When I signed up at Hetzner, I had to go through the same anti-abuse check. However I could choose to not upload my ID and pre-pay 20€ instead. Did that and have been a happy customer since.