Don't tell me that my thoughts aren't weird enough.
this post was submitted on 04 Apr 2025
363 points (88.4% liked)
Technology
69346 readers
3375 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
Another very surprising outcome of the research is the discovery that these LLMs do not, as is widely assumed, operate by merely predicting the next word. By tracing how Claude generated rhyming couplets, Anthropic found that it chose the rhyming word at the end of verses first, then filled in the rest of the line.
If the llm already knows the full sentence it's going to output from the first word it "guesses" I wonder if you could short circuit it and say just give the full sentence instead of doing a cycle for each word of the sentence, could maybe cut down on llm energy costs.
interestingly, too, this is a technique when you're improvising songs, it's called Target Rhyming.
The most effective way is to do A / B^1 / C / B^2 rhymes. You pick the B^2 rhyme, let's say, "ibruprofen" and you get all of A and B^1 to think of a rhyme
Oh its Christmas time
And I was up on my roof when
I heard a jolly old voice
Ask me for ibuprofen
And the audience thinks you're fucking incredible for complex rhymes.
I don't think it knows the full sentence, it just doesn't search for the words in the order they will be in the sentence. It finds the end-words first to make the poem rhyme, than looks for the rest of the words. I do it this way as well just like many other people trying to create any kind of rhyming text.
This is great stuff. If we can properly understand these “flows” of intelligence, we might be able to write optimized shortcuts for them, vastly improving performance.
load more comments
(1 replies)
How can i take an article that uses the word "anywho" seriously?
The AIs have shrinks now?
The math example in particular is very interesting, and makes me wonder if we could splice a calculator into the model, basically doing "brain surgery" to short circuit the learned arithmetic process and replace it.
That math process for adding the two numbers - there's nothing wrong with it at all. Estimate the total and come up with a range. Determine exactly what the last digit is. In the example, there's only one number in the range with 5 as the last digit. That must be the answer. Hell, I might even use that same method in my own head.
The poetry example, people use that one often enough, too. Come up with a couple of words you would have fun rhyming, and build the lines around those words. Nothing wrong with that, either.
These two processes are closer to "thought" than I previously imagined.
load more comments
(3 replies)
load more comments
(2 replies)