That's consistent with our experience using AI "assistants." If it's a common problem, the training set will be large enough that there's a chance the AI will return a correct answer, though without contextual knowledge that might be important. But in a case like that, you could also just go and look it up on Stack Overflow. And if it's not a common problem, the AI-proposed solution is likely to be crap, and one unlikely to take into account nonfunctional requirements, architectural guidelines, maintainability or best practice.

My own principle is that if AI was involved at any step in the coding process, that means we need to test that code even more than usual, because programmers who remain in the business learn not to do stupid things over time, and AI doesn't. When an AI makes some stupid coding suggestion, there's no feedback loop telling it that it fucked up.

I wouldn’t sign up for shit using my ID or anything right now.

That's some sound advice there.