Comment on AI isn’t ready to replace human coders for debugging, researchers say
jj4211@lemmy.world 2 days agoI occasionally check what various code generators will do if I don’t immediately know the answer is almost always wrong, but recently it might have been correct, but surprisingly convoluted. It had it broken down into about 6 functions iterating through many steps to walk it through various intermediate forms. It seemed odd to me that such a likely operation was quite so involved, so I did a quick Internet search, ignored the AI generated result and saw the core language built-in designed to handle my use case directly. There was one detail that was not clear in the documentation, so I went back to the LLM to ask that question and it gave the exact wrong answer.
I am willing to buy that with IDE integration or can probably have much richer function completion for small easy stuff I know to do and save some time, but I just haven’t gotten used to the idea of asking for help on things I already know how to do.
ThePowerOfGeek@lemmy.world 2 days ago
I’ve found the same thing.
Whenever I ask an LLM for a pointer, I end up spending just as long (if not longer) refining the question than just figuring it out myself it’s doing a search on SO it in other online resources.
But even the IDE integration is getting annoying. I write a class with some functionality baked in, and the whole time it’s promoting me with a shit load of irrelevant suggested code. I get the class done, then I go to spin up a unit test. It knows which class I’m trying to create a unit test for, which is cool. But then the suggested code is usually completely wrong or it’s much more convoluted than it needs to be. In the latter case, the first several characters of the suggested code is good, but then there’s several lines after it of shite. And hitting tab injects all of it in, which then requires me to delete it all. So almost every time I end up hitting escape anyway.
I’ve heard a few people rave about ‘vibe coding’ - usually people with no or little programming experience. I have to assume that generated code was either for very simple atomic actions and/or it’s spaghettified, inefficient garbage.
olympicyes@lemmy.world 1 day ago
I find chat gpt to be useful for deciphering error messages, generally noticing things I missed. It’s also helpful for boilerplate code, essentially a customized form of stack overflow. Beyond that it really has trouble knowing what it’s doing. I can’t imagine trusting it for completing a large project. Helpful for minor optimizations though.
jj4211@lemmy.world 1 day ago
I had been planning to, but being lazy about trying to enable my IDE setup but was giving it the benefit of the doubt. Your feedback resonates with how much I end up fighting auto-complete/auto-correct in normal language and seeing it potentially ruin current code completion (which sometimes I have to fight, but on balance it helps more than it annoys). I suppose I’ll still give it a shot, but with even more skepticism. I suppose maybe it can at least provide an OK draft of API documentation… Maybe sometimes…
On the ‘vibe coding’, on the cases I’ve seen detailed, it seems they do something that, to them, is a magical outcome from technologies that intimidated them. However, it’s generally pretty entry level stuff for those familiar with the tools of the trade, things you can find already done dozens of time on github almost verbatim, with very light bespoke customization. Of course there is a market for this, think of all the ‘no code’/‘low code’ things striving to make approachable very basic apps that just end up worse than learning to code. As a project manager struggles to make a dashboard out of that sort of sensibility, a dashboard that really has no business being custom but tooling has fostered the concept that everyone has a snowflake dashboard, it’s a pain. But maybe AI can help them generate their dashboard. Of course, to be a human subjected to the workflows those PMs dream up is a nightmare. Bad enough already at my work there are hundreds of custom issue fields, a dozen issue types, and 50 issue states with maddening project to project unique workflows to connect the meaning of all this, don’t like AI emboldening people to customize further.
The thing about ‘vibe coding’ is when they get stuck and they get confused/frustrated about why the LLM stopped getting them what they want. One story was someone vibe coding up a racing game. He likely marveled as his vision materialized. From typing prose without understanding how to code he got some sort of 3D game with cars and tracks and controls. This struck him as incredibly difficult otherwise, but reachable through ‘vibe coding’. Then he wanted to add tire marks when the player did something, maybe on a hard turn) and it utterly couldn’t do it. After all the super hard stuff, why could the LLM not do this conceptually much simpler thing? Ultimately spitting out that the person needed to develop the logic himself (claiming it was refraining to do it because it would be better for him to learn, but I’m wagering that’s the generated text after repeated attempts to generate code that the LLM just could not do).