Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam.
-
Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.
-
Don't duplicate the full text of your blog or git here. Just post the link for folks to click.
-
Submission headline should match the article title.
-
No trolling.
-
Promotion posts require your active participation in selfhosting or related communities, or the post will be removed. No more than 10% of your posts or comments may be self-promotional, or your post will be removed. F/LOSS Exception: If your post is about a project that is completely open source & can be self-hosted in full without payment, and your account is at least 30 days old, your post is exempt from this rule as long as you continue to engage in comments.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
view the rest of the comments
High RAM for MOE models, high VRAM for dense models, and the highest GPU memory bandwidth you can get.
For stable diffusion models (comfyui), you want high VRAM and bandwidth. Diffusion is a GPU heavy and memory intensive operation.
Software/driver support is very important for diffusion models and comfy UI, so your best experience will be Nvidia cards.
I think realistically you need 80gb+ of RAM for things like qwen image quants (40 for model, 20-40 for LORA adapters in ComfyUI to get output).
I run an 128gb AMD AI 395+ Max rig, qwen image takes 5-20 minutes per 720p qwen image result in ComfyUI. Batching offers an improvement, reducing iterations during prototyping makes a huge difference. I have not tested since the fall though, and the newer models are more efficient.
Framework desktop?
Yes
I've really been mulling one of those over with 128GB. I'm on Claude Max and Cerebras $50 so I'm using a good amount of $200/mo for coding and Openclaw. Is it worth it for light coding, or are you only doing SD with it?
It would not be worth it as a replacement for Claude.
80% of my issue is that it's AMD and their drivers are still awful. 20% is that the token generation speed very slow, especially compared to commercial models running on dedicated hardware. MOE models are fine, dense models are too slow for meaningful workflows. ComfyUI is decent, but I'm not seriously into image gen.
I have a lot of fun with it, but I have not been able to use it for any actual AI dev.
Thanks for the feedback. That was precisely my worry about outlaying that money and not being happy with the result.
It's still a fantastic computer.
I use it as a server and it's very very fast, especially threaded workflows, and IO is fast.
Just don't buy it expecting to replace paid AI services. And don't buy it for AI dev, on paper it should be good, but driver issues. DGX Spark is better if yo want an AI dev machine.