Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil.
-
No spam.
-
Posts are to be related to self-hosting.
-
Don't duplicate the full text of your blog or readme if you're providing a link.
-
Submission headline should match the article title.
-
No trolling.
-
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
-
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
view the rest of the comments
Be aware that you pay a significant performance penalty for going over occulink.
The hit modest for fully offloaded dense models (like Qwen 27B), but dramatic for hybrid inference of big MoEs.
Even my old 3090 got a noticeable performance gain going from a PCIe 3.0 x16 riser to a PCIe 4.0 one.
And that's why it's tentative. The reason I'm leaning towards the dock is that, while the 5090 will fit, physically in the case, I am worried about heat dissipation. Even now, when I fire up a moderate model and run it through some paces, I can see the temps rising fairly quickly. So, if I'm going to drop that kind of cabbage on a 5090, the one thing I don't want to do is watch it turn into a shrink-i-dink.
Duct it!
I have a 400W 3090 with zero case fans.
Its always cool, because I bought like $10 of weather stripping to "seal" its intakes against the edge of an SFF case. So it's always sucking in ambient air:
You can also severely undervolt a 5090 to like 250-300W, with very little performance impact. Honestly its stock speeds are kind of crazy.
...But again, you aren't gaining much over a 4090. A 4090 + DDR4 threadripper would be way faster than a 5090 + occulink, for many reasons. And TR CPUs are pretty reasonably priced compared to GPUs these days.
But even if you do go 5090, I'd highly recommend finding some way to shove it in the case and duct some air into its intakes. Its going to be way faster on a PCIe slot.
You could even get a riser and mount it somewhere else in the case, theoretically.