this post was submitted on 28 May 2026
96 points (76.7% liked)

Selfhosted

60810 readers
508 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

  1. Be civil.

  2. No spam.

  3. Posts are to be related to self-hosting.

  4. Don't duplicate the full text of your blog or readme if you're providing a link.

  5. Submission headline should match the article title.

  6. No trolling.

  7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.

  8. AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
ayyy@sh.itjust.works
curbstickle@anarchist.nexus
curbstickle_lw@lemmy.world
96
Another reason to self host your own AI (aussie.zone)
submitted 1 month ago* (last edited 1 month ago) by SuspiciousCarrot78@aussie.zone to c/selfhosted@lemmy.world
50 comments fedilink hide all child comments
 

Recent post re: AI as utility

https://www.tomsguide.com/ai/people-will-buy-intelligence-from-us-on-a-meter-chatgpts-ceo-sam-altman-has-critics-worried-with-his-ai-vision

Myself, I'm a fan of local LLM / self hosted ML.... but if you ever needed a clarion call that a hard pivot is coming (soon) for online/ cloud based AI...Altman et al are making some concerning mouth noises (to say nothing of broader concerns with OAI, Anthropic etc).

Right now, I'm sketching out a plan where my Raspberry Pi (always on, 2-3w) uses a magic packet to wake up my modest AI server (Lenovo P330 with Tesla P4) if/when needed (Qwen 3.6-35B-A3B); no point in chugging down 80-100w, 24/7 for no good reason.

If the trend continues the direction it appears to be (increasing costs, environmental impacts etc) then I'd feel a lot better hosting my own as port of first call and replacing simpler tasks with more traditional programs. YMMV.

you are viewing a single comment's thread
view the rest of the comments
[–] SuspiciousCarrot78@aussie.zone 3 points 1 month ago* (last edited 1 month ago) (1 children)

I mean...that entirely depends on your use case - and I hate saying that. For me and what I do, Qwen SLM (esp Qwen3-4B 2507 instruct and Qwen3.5-2B) are exceptional. But I'm not trying to do Claude at home.

Best bet? Spend $10 on OpenRouter and try different models. In a head to head with ChatGPT 5.4 mini (excellent for coding BTW), I've found Qwen 3.5 27B more than able to hold its own for coding tasks...IF you narrowly gate it/confine it. The last batch of Qwen's really are something. Dunno about the 3.7 series.

Having said ALL that, I'm really tempted to go back in time and code myself a deterministic expert system, with user updatable knowledge cascade, tool calling and a minimal amount of Markov chain word garnish for flavour. I think we use to just call that "a program" lol.

Really tempted actually, because if 50% of llm use case is basically Super Google but not shit...well, I can make that myself. I just need to point my autism at it.

PS: this might help

https://www.youtube.com/watch?v=0AqpaFm11oI

[–] somegeek@programming.dev 1 points 1 month ago (1 children)

Qwen 3.5 24B is way too large for my specs. I'm barely running qwen2.5 7B

[–] SuspiciousCarrot78@aussie.zone 3 points 1 month ago* (last edited 1 month ago) (1 children)

Hmm....it runs on a 1060...it's a MoE not a dense. 24B is even lighter. Worth a shot.

https://www.youtube.com/watch?v=8F_5pdcD3HY

Else, if youre looking for a coding model (??) something like Sara or fara might suit

https://huggingface.co/microsoft/Fara-7B

[–] somegeek@programming.dev 1 points 1 month ago

Thanks. I will look into it.