Technology

I've had better success, when using AI agents in repeated, but small and narrow doses.

It's been kinda helpful in brainstorming interfaces (and I always have to append at the end of every statement "... in the most maintainable way possible.")

It's been really helpful in writing unit tests (I follow Test Driven Development), and sometimes it picks up edge cases I would have overlooked.

I wouldn't blindly trust any of it, as all too often it's happy to just disregard any sort of error handling (unless explicitly mentioned, after the fact). It's basically like being paired up with an over-eager, under-qualified junior developer.

But, yeah, you're gonna have a bad time if you prompt it to "write me a Unix operating system in web assembly".

I totally get it. I've been critical about using AI for code purposes at work and have pleaded to stop using it (management is forcing it, less experienced folk want it). So I've been given a challenge by one of the proponents to use a very specific tool. This one should be one of the best AI slop generators out there.

So I spent a lot of time thoroughly writing specs for a task in a way the tool should be able to do it. It failed miserably, didn't even produce any usable result. So I asked the dude that challenged me to help me refine the specs, tweak the tool, make everything perfect. The thing still failed hard. It was said it was because I was forcing the tool into decisions it couldn't handle and to give it more freedom. So we did that, it made up the rules themselves and subsequently didn't follow those rules. Another failure. So we split up the task into smaller pieces, it still couldn't handle it. So we split it up even further, to a ridiculous level, at which point it would definitely be faster just to create the code manually. It's also no longer realistic, as we pretty much have the end result all worked out and are just coaching the tool to get there. And even then it's making mistakes, having to be corrected all the time, not following specs, not following code guidelines or best practices. Another really annoying thing is it keeps on changing code it shouldn't touch, since we've made the steps so small, it keeps messing up work it did previously. And the comments it creates are crazy, either just about every line has a comment attached and functions get a whole story, or it has zero comments. As soon as you say to limit the comments to where they are useful, it just deletes all the comments, even the ones it put in before or we put in manually.

I'm ready to give up on the thing and have the use of AI tools for coding limited if not outright stopped entirely. But I'll know how that discussion will go: Oh you used tool A? No, you should be using tool B, it's much better. Maybe the tools aren't there now, but they are getting better all the time, so we'll benefit any day now.

When I hear even experienced devs be enthusiastic about AI tools, I really feel like I'm going crazy. They suck a lot and aren't useful at all (on top of the thousand other issues with AI), why are people liking it? And why have we hedged the entire economy on it?

Eh I don’t disagree with you, it’s just the reality for me that I am now expected to work on much more stuff at the same time because of AI, it’s exhausting but at least in my job I have no choice and I try to arrange myself with the situation.

I sure lost a lot of understanding of the details of the codebase but I do read every line of code these LLMs spit out and manually review all PRs for obvious bullshit. I also think code quality got worse despite me doing everything I can to keep it decent.