Comment on AI agents wrong ~70% of time: Carnegie Mellon study
someacnt@sh.itjust.works 1 day agoWdym, I have seen researchers using it to aid their research significantly. You just need to verify some stuff it says.
Comment on AI agents wrong ~70% of time: Carnegie Mellon study
someacnt@sh.itjust.works 1 day agoWdym, I have seen researchers using it to aid their research significantly. You just need to verify some stuff it says.
davidagain@lemmy.world 1 day ago
Verify every single bloody line of output. Top three to five are good, then it starts guessing the rest based on the pattern so far. If I wanted to make shit up randomly, I would do it myself.
People who trust LLMs to tell them things that are right rather than things that sound right have fundamentally misunderstood what an LLM is and how it works.
someacnt@sh.itjust.works 1 day ago
It’s not that bad, the output isn’t random. Time to time, it can produce novel stuffs like new equations for engineering. Also, verification does not take that much effort. At least according to my colleagues, it is great. Also works well for coding well-known stuffs, as well!
davidagain@lemmy.world 1 day ago
It’s not completely random, but I’m telling you it fucked up, it fucked up badly, time after time, and I had to check every single thing manually. It’s correctness run never lasted beyond a handful. If you build something using some equation it invented you’re insane and should quit engineering before you hurt someone.