Writing tests is the one thing I wouldn’t get an LLM to write for me right now. Let me give you an example. Yesterday I came across some new unit tests someone’s agentic AI had written recently. The tests were rewriting the code they were meant to be testing in the test itself, then asserting against that. I’ll say that again: rather than calling out to some function or method belonging to the class/module under test, the tests were rewriting the implementation of said function inside the test. Not even a junior developer would write that nonsensical shit.

The code those unit tests were meant to be testing was LLM written too, and it was fine!

So right now, getting an LLM to write some implementation code can be ok. But for the love of god, don’t let them anywhere near your tests (unless it’s just to squirt out some dumb boilerplate helper functions and mocks). LLMs are very shit at thinking up good test cases right now. And even if they come up with good scenarios, they may pull these stunts on you like they did to me. Not worth the hassle.