Technology

(And we do all the fancy shit, looping, custom harnesses, orchestrations, handovers, savepoints, post delivery pipelines, autoremediation, blablabla, context, graphs blabla)

Yeah that's not what I'm talking about. That's just coding stuff and I think everyone is learning a hard lesson right now that there's really two ways to do AI (for code):

1. You let the AI write all the code. Using those techniques you mentioned to try to keep it from fucking up too much.
2. You use the AI to fix/write a little bit of the code at a time. Manually. As in, someone who knows software architecture really well tells the AI to write a function or two or maybe even a whole module and that's it. They do this repeatedly until their task is complete.

#1 is a very expensive way to do it but it seems to work, albeit with a huge learning curve and a much longer horizon before you get something stable. Even then, what you get can be a giant fucking mess that's really, really hard to understand.

#2 is much safer and works quite well, IMHO. When the AI fucks something up, it's only a tiny little thing that's easy to fix. This method really is just a way for developers to improve their productivity.

With #2, it only takes a small amount of tokens to have the AI help you troubleshoot as well. At least, that's been my experience.

Having said that, the places where we are seeing the most growth in AI is small shit. For example, at my work they implemented an agent that scans customer complaints and filters out false positives. I work for a huge company that has a zillion products/services and the AI has tool calls into our ticketing systems to check if there was an outage at the time the customer complained. It's not that smart, but it's smart enough to tell if a ticket about system X was opened when system X had an outage. It can even tell if the customer was complaining about something relevant to the outage.

Stuff like that runs 24/7 and eats up bazillions of tokens, but only on our local hardware (we have thousands of enterprise GPUs). At other companies, they're doing similar things and slowly realizing they can replace OpenAI/Anthropic API calls with local ollama stuff.

Even if your token bill is $10,000/month for a service like that, it's still cheaper than paying an entire team of humans to perform the same function.

That is the growing use of AI automation I was talking about.