Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil.
-
No spam.
-
Posts are to be related to self-hosting.
-
Don't duplicate the full text of your blog or readme if you're providing a link.
-
Submission headline should match the article title.
-
No trolling.
-
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
-
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
view the rest of the comments
I didn't look too closely at details, but generally that's relatively normal the more individual applications you have running, especially with the overhead of them all being virtualized. Often the overhead of running that many virtualized operating systems can be problematic depending on how the container author chose and configured the container's OS. A lot of people seem to just randomly pick an OS for their containers just so they can distribute their application with docker as easily as possible, without considering the consequences.
Here are the options off the top of my head if disk wear is your primary concern:
Turn swap off at the host level. Note, though, that you may end up in bottlenecks if multiple of them do very memory intensive tasks at the same time, like image and video processing, and docker can't swap out the VMs that are idle to the disk and be able to use the RAM fully for those active tasks. For some of those apps you can configure when they do maintenance tasks and reduce the likelihood.
Look at each individual VM in docker and see which are using the most swap and force docker to limit their allowed resources. You can even tell docker to make an application not use swap by setting the memory and memory-swap values to the same value or reduce their swap usage by setting memory-swapiness to a low number, or you can install those applications to the host system and configure the host to reduce swap or disallow swap usage through systemd or whatever rather than running them in docker. You may also find one of the applications you're running has a memory leak issue. Good OSs often handle that, but who knows what IS the container is running and the docker layer is concealing from the host kernel and not handling quite as well as it being installed on "bare metal" might. Virtualization adds a lot of variables for these kinds of low level resource control scenarios. So just trying it on the host directly might reveal something if you're interested in digging like I usually am, lol.
Get a second, small, inexpensive drive and set that up as the swap partition for the host.
Look into what database systems these are using. Often if they each are running their own separate DBMS containers, the DBMS maintenance overhead can cause some additional churn. I have a single instance of PostgreSQL and one of MariaDB running on a separate server that all of my applications connect to. Some of these applications may be not only spinning up a whole VM for the application, but an entirely other VM for their database server. Though I think some of these probably use sqlite which is just a file, I think manynof these use PostgreSQL and/or MySQL/MariaDB by default. This also helps because I only need a single backup for all of my databases. The negative is that if that DB server goes down, then all the apps go down too, but if you already have everything on one server that might not be as big of a deal. I have a lot of smaller servers running smaller subsets of applications based on their resource needs. Helps me conserve electricity rather than running one or two big servers that are almost always highly active. But I enjoy optimizing that kind of stuff, it's tedious.
Just a small detail, docker containers aren't vm's and not running their own os. They run inside a jail under the host kernel. Docker containers are very lightweight.