Technical specs don’t capture the bugs, edge cases and workarounds needed for technical subjects like software.
Comment on Stack Overflow in freefall: 78 percent drop in number of questions
cherrari@feddit.org 1 day agoI don’t think so. All AI needs now is formal specs of some technical subject, not even human readable docs, let alone translations to other languages. In some ways, this is really beautiful.
123@programming.dev 22 hours ago
cherrari@feddit.org 4 hours ago
I can only speak for myself obviously, and my context here is some very recent and very extensive experience of applying AI to some new software developed internally in the org where I participate. So far, AI eliminated any need for any kind of assistance with understanding and it was definitely not trained on these particular software, obviously. Hard to imagine why I’d ever go to SO to ask questions about this software, even if I could. And if it works so well on such a tiny edge case, I can’t imagine it will do a bad job on something used at scale.
rumba@lemmy.zip 1 day ago
It can’t handle things it’s not trained on very well, or at least not anything substantially different from what it was trained on.
It can usually apply rules it’s trained on to a small corpus of data in its training data. Give me a list of female YA authors. But when you ask it for something more general (how many R’s there are in certain words) it often fails.
webadict@lemmy.world 1 day ago
Actually, the Rs issue is funny because it WAS trained on that exact information which is why it says strawberry has two Rs, so it’s actually more proof that it only knows what it has been given data on. The thing is, when people misspelled strawberry as “strawbery”, then naturally, people respond, " Strawberry has two Rs." The problem is that LLM learning has no concept of context because it isn’t learning anything. The reinforcement mechanism is what the majority of its data tells it. It regurgitates that strawberry has two Rs because it has been reinforced by its dataset.
rumba@lemmy.zip 1 day ago
Interesting story, but I’ve seen the same work with how many ass in assassian
you can probe the stuff it’s bad at, and a lot of it doesn’t line up well with the story that it’s how people were corrected.
webadict@lemmy.world 22 hours ago
But that’s exactly how an LLM is trained. It doesn’t know how words are spelled because words are turned into numbers and processed. But it does know when its dataset has multiple correlations for something. Specifically, people spell out words, so it will regurgitate to you how to spell strawberry, but it can’t count letters because that’s not a thing that language models do.
Generative AI and LLMs are just giant reconstruction bots that take all the data they have and reconstruct something. That’s literally what they do.
Like, without knowing what your answer is for assassin, I will assume that your issue is that the question is probably “How many asses are in assassin?” But, like, that’s a joke. Assassins only has one ass, just like the rest of us. That’s a joke. And nobody would ever spell assassin as assin, so why would it learn that there are two asses in assassin?
I’m confused where you are getting your information from, but this is not particularly special behavior.
skisnow@lemmy.ca 19 hours ago
The whole point of StackExchange is that it contained everything that isn’t in the docs.
SoftestSapphic@lemmy.world 1 day ago
LolnAI can’t do a single thing without humans who have already done it hundreds of thousands of times feeding it their data
okmko@lemmy.world 22 hours ago
I used to push back but now I just ignore it when people think that these models have cognition because companies have pushed so hard to call it AI.