Yes. ChatGPT is not perfect. Because it is a general purpose LLM. Stuff like Github CoPilot and other software specific approaches are a LOT better at avoiding all the noise from bad answers on stack overflow and proposals.
But it can still do a remarkably good job so long as you have a human looking at it after the fact. Which… is how I would describe most software engineers I have ever worked with. Even the SSEs need someone to review their code. Which… is what is being described here. Combine that with a gitlab runner and you got yourself a stew.
As for APis and the like: Again, it feels like nobody here has ever actually worked with public software and think regressions don’t exist. But this is literally constraints and would be put in the requirements document that you give either the dev team or the LLM.
As for who is going to make that document: The same people who already do? Management.
Vlyn@lemmy.zip 1 year ago
Management and sound technical specifications, that sounds to me like you’ve never actually worked in a real software company.
You just said what the main problem is: ChatGPT is not perfect. Code that isn’t perfect (compiles + has consistent logic) is worthless. If you need a developer to look over it you’ve already lost and it would be faster to have that developer write the code themselves.
Have you ever gotten a pull request with 10k lines of code? The AI could spit out so much code in an instant, no developer would be able to debug this mess or do a code review. They’ll just click “Approve” and throw it on the giant garbage heap whatever the AI decided to spit out.
If there’s a bug down the line (if you even get the whole thing to run), good luck finding it if no one in your developer team even wrote the code in the first place.
Puzzle_Sluts_4Ever@lemmy.world 1 year ago
Worked at quite a few. Once you get out of college and start engaging with companies beyond “Ugh, how dare they want me to waste my precious time by talking to people” you start to learn the value of a strong management team.
And, more importantly, where those jira tickets come from.
A bog standard development flow is “all pull requests are linked to a documented issue/ticket. All pull requests require tests to pass, code coverage to not decrease, and approval by a code owner”
How does that work in reality?
Issues/tickets (just going to say issues from here on out) are created by a combination of customer feedback, identified issues by the development team, and directives from on high (which is generally related to the overall roadmap). One or more developers work on a merge request, the person who best understands the appropriate code looks it over, it is tested, and it is merged in. After enough of those cycles happen, a release is prepared and a manager signs off on it.
How does that map to an “AI” based workflow?
Issues/tickets (just going to say issues from here on out) are created by a combination of customer feedback, identified issues by the development team, and directives from on high (which is generally related to the overall roadmap). Because LLMs can provide feedback and uncertainty measurements once you get past Google Bard. And regression testing and nightly performance testing can highlight deficiencies. The issue is put into a template, that includes all existing constraints, and the LLM generates a solution. Someone who understands the code checks to make sure that looks sane, it is tested, and it is merged in. After enough of those cycles happen, a release is prepared and a manger signs off on it.
And then it becomes a question of what level you start requiring humans. Because when I do a code review prior to a Release? I am relying VERY heavily on my team to have been doing their due diligence. I skim through the MRs and look for a few hot spots but it is mostly “Well, Fred and Nancy said this was good and it passes all the tests so…”
I VEHEMENTLY disagree with this. If you don’t have developers looking over your code then you are not a software engineer. And if it takes them the same amount of time to review code as it does to write it? You aren’t working on interesting problems and are wasting vast amounts of money.
I can farm out a general task of “improve our code coverage” to an intern. They can spend a few days (or even weeks) doing that, and I can review their MRs in a few minutes. If something looks weird, I leave a comment and wait for them to get back to me. All the time I am working on much more interesting problems… or doing the same for my SSEs.
Vlyn@lemmy.zip 1 year ago
You misunderstood, I never said management is worthless. The product managers know what customers want. The product owners keep 8 out of 10 dumb ideas away from the development team. And management again leans on the development team to find out what is actually technically possible and in what time frame.
If management just threw every customer wish into a magic black box to get code out, even if that code was perfect, you wouldn’t have a product. You’d have a pile of steaming crap.
I’ve done plenty of code reviews, they only work if they are small human readable increments. Like they say: A code review of 100 lines might take an hour. A code review of 10000 lines takes thirty minutes.
AI would spit out so much code with missing context for the developer, it would be impossible to properly review.
Puzzle_Sluts_4Ever@lemmy.world 1 year ago
Again: No
if it takes you the same amount of time to review 10k lines versus write 10k lines? Either you are bad at your job or you aren’t working on a meaningful problem.
And, again, there is no difference between assigning “Implement Feature X” ticket to Stan versus StanAI. If StanAI is writing 500x the amount of code that Stan would? StanAI sucks and needs to be retrained.
And, as it stands? Using tools like CoPilot or even ChatGPT, “StanAI” tends to write more concise AND more readable code. In large part because its training data is weighted by the code that has already gone through code review, was accepted, and may even be part of the production stack on half the planet.