I'm damn sure the answer is "no", but the devs at my workplace are all in on it in a way that's not as absurd on the surface as the "coding is a solved problem" kind of tech bros. There's an emphasis on the "human in the loop" that I think is far from sufficient; we used to have a human in the loop, it's called actually writing your fucking code!
As a sample, this is from a transcript of a recent discussion about AI with some devs on my team:
Me, a senior dev:
To be clear, I'm not saying not to forego the human in the loop. I'm pointing out that there is an inherent contradiction between using agent as our main means of writing code and actually being able to review all of its output -- be it before or after opening the PR. I'm worried that we'll end up in a position that we're expected to do something intractable, if not strictly speaking impossible.
New mid-level dev:
I just think we should still be responsible for reviewing all the output. Like how I see the before vs now is:
- Before: human plan -> human write -> human review
- Now: human plan -> claude write -> human review (+ claude review)
And in effect we're just as responsible for reviewing every line of the Al code as we were for reviewing every line of human code. Once that changes that's when the Al slop starts coming out So we should never end up posting a PR where we're like "hey guys claude wrote this so do the thing you do for claude PRs"
Team tech lead:
@AppetiteOfExcess, this is a problem across all dev teams in every industry. It's a huge issue. I think we need to lean on Al to speed up PRs, but how we do that is not well defined yet. I think it can help us get our head around a PR faster. Maybe we need to made an artifact for some PRs that give a breakdown of what is changing. Then we can grill Al once it understands the changes. That might speed things up a bit. The important thing is to really understand the code and the impact of the change.
@mid-level_dev I agree with you but I think the new process will be more like the following
- Humans (the whole team) plan by making highly defined stories
- Al makes an implementation and test plan
- Human reviews plan
- Al executes the plan with human oversight at certain steps
- Al does a review using /code-review (new context by default)
- PR is posted with change overview
- Human uses Al to understand the changes and review the code
I don't know how to push back on this while being taken at all seriously. It has me considering that maybe I'm in the wrong for rejecting a tool that is considered useful by all but a few stick in the mud devs, and that those devs will never have a shot at a job again unless they pull their heads out of their asses and join our modern "AI"-driven world.
I think it is possible to have ethical "LLMs", but the technology does not exist. An ethical LLM would require us to re-imagine the entire process, from data gathering, to tagging and organization, to ML training, to applications. There is no ethical application for the existing technology. (I put LLM in scare quotes, because with the required changes, you might not categorize the end result as an LLM any more, though there are broad applications for the use of machine-learning techniques on large bodies of information)
First and foremost, ethical LLM technology must be environmentally sustainable. We are in the middle of a run-away climate crisis and there is no excuse to be wasting energy like this. Everyone responsible for LLMs and proof-of-work cryptocurrency should be executed right alongside the oil execs. It must preserve attribution. It must not be a tool for crime-prediction, surveillance, and social manipulation (e.g. profile-based advertising). It must be accessible and democratic.
In my mind, instead of having software which synthesizes language (human or programming languages), it would function as a highly sophisticated search engine. You might prompt "I want to make a REST client which can speak the Matrix protocol" and based on the corpus of training material it can identify the required aspects of an implementation and source material for each required component. This would be a very useful tool, but it is not useful in the way LLMs are being deployed today, which is to steal dead labor (the corpus of free software, documentation, internet discussions, and literature in general), in order to make living labor redundant.
That said, I would make the argument that LLMs are not the core problem in software engineering. They are deeply exacerbating the actual problem which is a crisis of complexity (which is why I think using them makes you an automatic dickhead). We're sprinting into a world where we rely on essential components (the kernel or the web browser, for instance), which have completely outrun the capacity for responsible public maintenance. The deficit in maintenance is papered over by using LLM's to "accelerate" development and review duties, but this only results in even more complexity. I believe this crisis is going to reach a tipping point fairly soon, and it is completely unrelated to the financial situation of the looming AI bubble.