Comment on After outages, Amazon to make senior engineers sign off on AI-assisted changes
otter@lemmy.ca 17 hours ago
…
Do the senior engineers NOT sign off on changes to systems that can take down the production servers? Even if we take out the LLM created code, this sounds like a bigger problem
I guarantee there’s so much pressure on those engineers to deliver code that they rubber stamp a ton of it with the intention of “fixing it later”
Source: I’ve worked in software for 20+ years and know a lot of folks working for and who have worked for Amazon
That’s basically the story at all the big tech companies, from what I’ve heard. In my time at Facebook, I felt like the only person who actually read the merge requests that people sent me before hitting it with “LGTM”
If companies are going to place increasing reliance on review, then they should probably evaluate employees weighting review quality (say, oh, rate of bugs subsequently discovered in reviewed commits or something like that).
Sure. i’ll review your code favourably if you do the same with mine.
That is also a way to get no bugs at all.
When I worked there 20% of the work we had to do had to go through a senior engineer. And getting his time was like pulling teeth.
More of the time he would just nitpick grammar in docs and then finally rubber stamp work. It was awful.
the way private companies work is that they require their employees to produce more than is reasonable given the work quality that is expected.
when this discrepancy is pointed out, it’s handwaved away. when the discrepancy results in problems, as it most obviously will, somebody is found to place the blame on.
it’s not the developer’s faults. it’s a management decision.
source: I’m talking out of my ass I’m just a salty employee who is seeing this happen at their own workplace when it didn’t used to, at least not to this level
pageflight@piefed.social 17 hours ago
We may start to see people realize that “have the AI generate slop, humans will catch the mistakes” actually is different from “have humans generate robust code.”
daychilde@lemmy.world 15 hours ago
Not only that, but writing code is so much easier than understanding code you didn’t write. Seems like either you need to be able to trust the AI code, or you’re probably better of writing it yourself. Maybe there’s some simple yet tedious stuff, but it has to be simple enough to understand and verify faster than you could write it. Or maybe run code through AI to check for bugs and check out any bugs it finds…
I definitely have trusted AI to write miniature pointless little projects - like a little PHP page that loaded music for the current directory and showed a simple JS player in a webpage so I could share Christmas music with my family and friends. No database, no file uploading or anything. It worked decently, although not perfectly, and that’s all it needed to do.
slaacaa@lemmy.world 11 hours ago
This is true not just with code, but with many types of complex outputs. Going through and fixing somebody’s horrible excel model is much worse than building a good one yourself. And if the quality is really bad, it’s also just easier to do it yourself.
Hupf@feddit.org 9 hours ago
Yeah, initially writing the code never was the time sink.
MirrorGiraffe@piefed.social 10 hours ago
I’ve been writing a slightly larger project with frontend, bff and backend and I need to take it in small batches so that I can catch when it misunderstands or outright does a piss job of implementing something. I’ve been focusing a lot on getting all the unit tests I need in place which makes me feel a bunch better.
The bigger and more complex the projects get, the harder it is for the LLM to keep stuff in context which means I’ll have to improve my chunking out smaller scoped implementations or start writing code myself I think.
All in all I feel pretty safe with my project and pleased with the agents work but I need to increase testing further before bringing anything live.
daychilde@lemmy.world 3 hours ago
Security testing will be the most important.
I’ve done a couple of tiny projects that I didn’t feel like coding. So far, I have not been terribly impressed. Well, it is impressive that it can make something functional at all, and in one case, what it made was fine enough to use as the temporary project it was intended (sharing christmas music with friends/family - reading files from a directory and writing a javascript player to play them in a shuffled order).
In the other case, replicating a simple text-based old DOS game with simple rules (think a space-based game around the complexity of checkers or so), it failed to think of so many things that while it did what I told it for the most part, it wasn’t a playable game. It was close, and fun enough for a nostalgic moment, but I had to work with it on logic like “If two fleets of ships arrive at the same planet in the same turn, you have to see how the first battle goes. If the first battle captures the planet, the second fleet is not attacking the first fleet’s ships - we won the planet at that point”. Very simple concepts that sure, you’d have to think of as a programmer, but if you were telling another person about how the game should work, were things I felt another person would think about.
I hope AI works well for you. Anywhere security it needed like database sanitation or user credentials… I hope you test thoroughly and I hope you can tell it enough to remind it to implement things like sanitation and other safety measures. An app can certainly appear to be working, but give many many fronts for attack. That’s my main worry with AI code. I worry enough on the little projects I do if I’m being secure enough myself.
somethingsnappy@lemmy.world 13 hours ago
*LLM