Ask Lemmy
A Fediverse community for open-ended, thought provoking questions
Rules: (interactive)
1) Be nice and; have fun
Doxxing, trolling, sealioning, racism, toxicity and dog-whistling are not welcomed in AskLemmy. Remember what your mother said: if you can't say something nice, don't say anything at all. In addition, the site-wide Lemmy.world terms of service also apply here. Please familiarize yourself with them
2) All posts must end with a '?'
This is sort of like Jeopardy. Please phrase all post titles in the form of a proper question ending with ?
3) No spam
Please do not flood the community with nonsense. Actual suspected spammers will be banned on site. No astroturfing.
4) NSFW is okay, within reason
Just remember to tag posts with either a content warning or a [NSFW] tag. Overtly sexual posts are not allowed, please direct them to either !asklemmyafterdark@lemmy.world or !asklemmynsfw@lemmynsfw.com.
NSFW comments should be restricted to posts tagged [NSFW].
5) This is not a support community.
It is not a place for 'how do I?', type questions.
If you have any questions regarding the site itself or would like to report a community, please direct them to Lemmy.world Support or email info@lemmy.world. For other questions check our partnered communities list, or use the search function.
6) No US Politics.
Please don't post about current US Politics. If you need to do this, try !politicaldiscussion@lemmy.world or !uspolitics@lemmy.world
7) No Hit-and-Run questions.
Please don't delete your post for no apparent reason. If you plan on deleting a question later, say so in the post, or if you feel that you have a good reason to remove it, message a mod beforehand. It's not fair to the ones who took their time to answer, and it's not in the spirit of the community.
8) No Bots.
Posts or comments from bots, LLM's, AIs, Neural Networks, Transformers, or Marvin the Paranoid Android are not welcome in AskLemmy. Real humans only please.
Reminder: The terms of service apply here too.
Partnered Communities:
Logo design credit goes to: tubbadu
view the rest of the comments
They call it open-weighted and not open-source because they don't have the training material (stolen books lol) but still want to pretend they are l33t hackers not bound to BigTech.
Yes, it's possible to use it locally with a $2000 computer (that's for the cheapest ones) but you'll only get a few words per second. It doesn't matter to the vibe coders who don't know how to code.
The software to run that is open-source but the training of the model (the "open-weight" black box) requires to destroy the environment at least once.
Last but not least the free models are obviously censored but people don't care about censorship anymore for some reason. "Tiananmen didn't happen? Not my problem" without understanding that more is hidden.
Anyway, no, there is nothing ethical about it.
a) Yes, today it costs 2000$ because of the cost explosion - my setup cost me around 1400$, and my GPU was already bought when prices were rising. No, i do not get a few words per second, i get around 50-60 token/s, which is more than enough for personal use. (Edit: and that is WITH CPU offloading, where layers that don't fit into VRAM get placed into system RAM, and i am still running DDR4 to boot)
b) If the completely insane AI corpos would stop training humongous models to chase after non-achievable AGI, the one-time investments would have paid off by now for local use. This situation has nothing to do with local models but insane billionaires and execs.
c) go ahead and lookup abliterated (not a typo) models on HuggingFace - these do NOT refuse any requests, because that has been pruned out. These answer everything about Tiananmen, DEI topics like erosion of LGBT and womens rights in the west and whatever atrocities any group might have commited, while also telling you about whatever you want to know. I run only these models, because i refuse to partake in censorship.
Edit: Regarding the training material: This training material also consists of MY output over the decades on the web. Therefor, i do not have qualms using these models for non-commercial usage without disseminating the output - classic personal usage, mainly for automating tasks that are not easily done by hand such as grabbing game descriptions from steam, condensing them down to a short sentence and putting the result in a database, together with the corresponding steam tags, and automatically searching the web for information about the game when there is no steam page for it - which is an insane amount of work to do by hand for my library of more than 30k titles. I let it work during off hours on that task.