Selfhosted

59999 readers

414 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam.
Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.
Don't duplicate the full text of your blog or git here. Just post the link for folks to click.
Submission headline should match the article title.
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

-56

Can local LLMs be as useful and insightful as those widely available? (discuss.online)

submitted 1 year ago* (last edited 1 year ago) by Brylant@discuss.online to c/selfhosted@lemmy.world

25 comments fedilink hide all child comments

Why do people host LLMs at home when processing the same amount of data from the internet to train their LLM will never be even a little bit as efficient as sending a paid prompt to some high quality official model?

inb4 privacy concerns or a proof of concept

this is out of discussion, I want someone to prove his LLM can be as insightful and accurate as paid one. I don't care about anything else than quality of generated answers

you are viewing a single comment's thread
view the rest of the comments

[–] TheYang@lemmy.world 9 points 1 year ago

Because you don't train your self-hosted LLM.
As a result you only pay for the electricity of computing your tokens (your request), this can be especially reasonable if the same machine also does local game streaming and or transcoding, and thus already has the requirements to host a LLM.

If you don't have rather unreasonable means, your local LLM is just very much more limited in parameters (size), and will not be as good as other, much larger models.

Privacy, Ethics and personal interest usually are the largest drivers from what I can tell.