Ask Lemmy
A Fediverse community for open-ended, thought provoking questions
Rules: (interactive)
1) Be nice and; have fun
Doxxing, trolling, sealioning, racism, toxicity and dog-whistling are not welcomed in AskLemmy. Remember what your mother said: if you can't say something nice, don't say anything at all. In addition, the site-wide Lemmy.world terms of service also apply here. Please familiarize yourself with them
2) All posts must end with a '?'
This is sort of like Jeopardy. Please phrase all post titles in the form of a proper question ending with ?
3) No spam
Please do not flood the community with nonsense. Actual suspected spammers will be banned on site. No astroturfing.
4) NSFW is okay, within reason
Just remember to tag posts with either a content warning or a [NSFW] tag. Overtly sexual posts are not allowed, please direct them to either !asklemmyafterdark@lemmy.world or !asklemmynsfw@lemmynsfw.com.
NSFW comments should be restricted to posts tagged [NSFW].
5) This is not a support community.
It is not a place for 'how do I?', type questions.
If you have any questions regarding the site itself or would like to report a community, please direct them to Lemmy.world Support or email info@lemmy.world. For other questions check our partnered communities list, or use the search function.
6) No US Politics.
Please don't post about current US Politics. If you need to do this, try !politicaldiscussion@lemmy.world or !uspolitics@lemmy.world
7) No Hit-and-Run questions.
Please don't delete your post for no apparent reason. If you plan on deleting a question later, say so in the post, or if you feel that you have a good reason to remove it, message a mod beforehand. It's not fair to the ones who took their time to answer, and it's not in the spirit of the community.
8) No Bots.
Posts or comments from bots, LLM's, AIs, Neural Networks, Transformers, or Marvin the Paranoid Android are not welcome in AskLemmy. Real humans only please.
Reminder: The terms of service apply here too.
Partnered Communities:
Logo design credit goes to: tubbadu
view the rest of the comments
They are all trained on copyrighted material without permission, no LLM is ethical
You can also train your own local models with license free material if you wish! I think one of the easiest ways to get into that is by using software like unsloth (that's the one i am using), an open source no-code tool which can be both used to train models on whatever data you wish and to run models either locally or using an inference provider.
Quick example for something like that which is also not dependent on copyrighted material is RAG, where you can provide the 400-page manual for something and then can chat with "the document" to get explanations, ask quick questions without searching for possibly multiple occurrences of a specific term and similar stuff. I love this for technical documents like mainboard manuals!
Doesn't RAG require a pretrained model still? Presumably on copyright material?
It does not per se need a model using copyrighted stuff. You can get by using a model without that - basic language skills and technical jargon can easily be trained with open material.
In theory that makes sense, but does this actually exist?
It does if you want to go that route. https://www.kaggle.com/datasets is a good starting point.
Thanks for the info!
Some don't speak any "language" they are trained to "speak" and "think" in terms of election orbitals and bonding energy. They are used in pharma and materials science to work on intractable problems like superconductivity and meds for Parkinson's.
that's not really an LLM then, is it?
It is, cause it uses the same architecture, https://www.geeksforgeeks.org/artificial-intelligence/exploring-the-technical-architecture-behind-large-language-models/
an example: https://github.com/bowang-lab/scGPT "talks" in RNA and is incredibly accurate and able to do predictions
But it is AI.
true, but that's hardly relevant as a reply to s comment which is clearly referring to LLMs and not machine learning in general
What's not ethical is a copyright system thag enforces artificial scarcity where there is no need for it.
Piracy is not stealing, and is not inherently unethical.
Everything is copyrighted regardless of it being owned by a corporation or blogger
I don’t partially care about the former
The absolutely most generous copyright protection length is in Mexico which is 100 years plus the life of the author when created. So lets generously say a total of 160 years. Most of the rest of the world is 120 year max. Anything outside of that is in the Public Domain. Many things had a much shorter path to the Public Domain falling into it in as little as 20 years.
So lots and lots of stuff is not copyrighted.
do you consider "piracy" like zlibrary and annas archive unethical?
If a business is using them, yes.
i agree.
in that case, wouldn't it be similar ethically if people use these models on their own machine for personal use?
As an exercise so I could learn the technology, I trained a model exclusively on the Public Domain works of L Frank Baum, specifically the "Wonderful Wizard of Oz" series (did you know there are 14 books in the series just by Baum?!). The model created is not useful as a tool, and more often than not just produces English gibberish, but every now and then it can produce original coherent statements.
Again, I didn't do this to produce a useful tool, but rather an exercise to learn how to build models.
However, this means I can refute your statement because 100% of the input data is public domain novels. Also, I trained it on my own hardware powered 100% by solar power.
So we created multitudes of digital byte-sized monkeys...and gave them m/billions of typewriters. . . :p
You're not far off from describing primative LLMs. In fact, the recent innovation in the last 5 years was the application of a specific technique published in a paper called "Attention Is All You Need" that the secret sauce is "attention" added via a transformer layer. This concept is what made GPT, Claude, etc the life-like responses they have today.
Super cool! Thanks for sharing :D
...and for making me feel like that cool kid in class whose silly quip is leveraged by the teacher to expand context for everyone. 😎
Cool that you did that. However, "Large" as in "Large Language Model" generally refers to the dataset size, and 14 books is not Large.
There is Apertus which at least claims to only use permissively licensed sources and respect robots.txt opt-outs. They outline their methodology and source datasets in this document. Though I haven't checked the actual sources myself.
Human brains are all trained on copyrighted material without permission, no human brain is ethical