What I find LLMs doing for my software development is filling in the gaps. Thorough documented requirements coverage, unit test coverage, traceability, oh you want a step by step test procedure covering every requirement? No problem. Installer scripts and instructions. Especially the stuff we NEVER did back in the late 1980s/early 1990s LLMs are really good at all of that.
Nothing they produce seems 100% good to go on the first pass. It always benefits from / usually requires multiple refinements which are a combination of filling in missing specifications, clarifying specifications which have been misunderstood, and occasionally instructing it in precisely how something is expected to be done.
A year ago, I was frustrated by having to repeat these specific refinement instructions on every new phase of a project - the LLM coding systems have significantly improved since then, much better “MEMORY.md” and similar capturing the important things so they don’t need to be repeated ALL THE TIME.
On the other hand, they still have their limits and in a larger recent project I have had to constantly redirect the agents to stop hardcoding every solution and make the solution data driven from a database.
8oow3291d@feddit.dk 1 day ago
I were simply unable to convince Codex to split a patch into separate git commits in a meaningful way. There are things that just doesn’t work.
Still useful for lots of stuff. Just don’t use it blind.
MangoCats@feddit.it 1 day ago
Never use it blind, and like I more or less said above: if you’re taking the first response you’re using it wrong. I go at it expecting everything it says to be 80% right, finding that 20% telling it what’s wrong with it, then getting to 96% right - if the 4% off target is a problem, refine again…
Where it excels for me is generating long detailed (mind numbing) point by point descriptions of things - the kind of documents you can skim to see where they are right and wrong but would fall asleep or have a case of terminal ADHD before finishing creating them on your own.