The last time I asked a AI for code was for a code snippet to do orthographic transcription (e.g. Владимир Путин -> Vladimir Putin). The AI responded with a snippet doing a (even pointless) change of encoding that would result in an exception only. I complained about getting only an exception when executing the code. It then try catch print stack trace the code. After complaining about not wanting a stack trace, the AI removed the stack trace output, so the result was just nothing. So I complained again and reformulated my requirements. It then corrected the code to a version that did again only change the encoding transitionally resulting in strings containing replacement characters for the ones not available in the target encoding. I gave up. 🤦♂️ What I did expect was giving me a solution using some library or telling me a common algorithm or library for orthographic transcription is not available. This is just one example. Almost always when I ask a AI for non trivial code the response is just wrong, fantasizing about non existing APIs or contains computational errors. Corrections after rejecting responses are so stupid that if I did get from team mates I would have asked them if they want to be kidding me.
Hallucination poses a significant challenge in AI, and no one truly knows how to completely solve it. AI tends to make up information. While the ability to run code provides certain advantages, there is still much work to be done, and AI is not replacing developers anytime soon. However, it will undoubtedly assist people in building their projects, especially if they are starting from scratch.