this post was submitted on 25 Apr 2026
44 points (97.8% liked)
Technology
42810 readers
222 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 4 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Yeah but there’s also some interesting nuances. I’ve seen smaller models on HuggingFace that, if I interpret them correctly, were tuned unsupervised using the output of larger models. So it seems there might be some validity to doing some things this way, so long as the other model is larger.
What you're referencing is distillation. Anthropic even has an article on distillation "attacks" (as if they have some divine right to the data behind their models) that goes over it a bit.