this post was submitted on 18 Sep 2026
-15 points (27.3% liked)
Programming
28506 readers
596 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I read the GitHub and tried it out. It basically works by recursively testing the output. It seems very expensive, and not specific in implementation. In the example given, the LLM can still make any changes it wants, as long as it doesn't fail the test. They're also designed to be written by the LLM itself, not humans. So it's not really a replacement for natural language.
It reminds me of integration testing tools such as Cypress, except it's an LLM running the test on itself. The "laws" are only in effect when the LLM understands the laws, and understands how it's change will affect compliance with them.
In my opinion, in practice, LLMs will end up writing a 19,000 lines long impenetrable and unreadable "law handbook", get confused about the laws, burn through a ridiculous amount of time and tokens recursively testing itself at every step of the journey, and people will end up removing it. It's cool in theory though, but it has so many rough edges that seem inherit to the technology.