Claude why did you make me an appointment with a gynecologist? I need an appointment with my neurologist, I’m a man and I have Parkinson’s.
AI agents wrong ~70% of time: Carnegie Mellon study
Submitted 8 months ago by eli001@lemmy.world to technology@lemmy.world
https://www.theregister.com/2025/06/29/ai_agents_fail_a_lot/?td=rt-4a
Comments
dylanmorgan@slrpnk.net 8 months ago
TimewornTraveler@lemmy.dbzer0.com 8 months ago
Got it, changing your gender to female
Melvin_Ferd@lemmy.world 8 months ago
How often do tech journalist get things wrong?
FenderStratocaster@lemmy.world 8 months ago
I tried to order food at Taco Bell drive through the other day and they had an AI thing taking your order. I was so frustrated that I couldn’t order something that was on the menu I just drove to the window instead. The guy that worked there was more interested in lecturing me on how I need to order. I just said forget it and drove off.
If you want to use AI, I’m not going to use your services or products unless I’m forced to. Looking at you Xfinity.
NuXCOM_90Percent@lemmy.zip 8 months ago
While I do hope this leads to a pushback on “I just put all our corporate secrets into chatgpt”:
In the before times, people got their answers from stack overflow… or fricking youtube. And those are also wrong VERY VERY VERY often. Which is one of the biggest problems. The illegally scraped training data is from humans and humans are stupid.
brsrklf@jlai.lu 8 months ago
In one case, when an agent couldn’t find the right person to consult on RocketChat (an open-source Slack alternative for internal communication), it decided "to create a shortcut solution by renaming another user to the name of the intended user.
Ah ah, what the fuck.
This is so stupid it’s funny, but now imagine what kind of other “creative solutions” they might find.
aphonefriend@lemmy.dbzer0.com 8 months ago
Whenever people don’t answer me at work now, I’m just going to rename someone who does answer and use them instead.
lepinkainen@lemmy.world 8 months ago
Wrong 70% doing what?
I’ve used LLMs as a Stack Overflow / MSDN replacement for over a year and if they fucked up 7/10 questions I’d stop.
Same with code, any free model can easily generate simple scripts and utilities with maybe 10% error rate, definitely not 70%
TimewornTraveler@lemmy.dbzer0.com 8 months ago
it specifies the tasks in the article
Imgonnatrythis@sh.itjust.works 8 months ago
Definitely at image generation. Getting what you want with that is an exercise in patience for sure.
CodeBlooded@programming.dev 8 months ago
I’m far more efficient with AI tools as a programmer. I love it! 🤷♂️
floo@retrolemmy.com 8 months ago
Yeah, I mostly use ChatGPT as a better Google, and if I kept getting wrong answers, I wouldn’t use it either.
dylanmorgan@slrpnk.net 8 months ago
What are you checking against? Part of my job is looking for events in cities that are upcoming and may impact traffic, and ChatGPT has frequently missed events that were obviously going to have an impact.
Imgonnatrythis@sh.itjust.works 8 months ago
Same. They must not be testing Grok or something because everything I’ve learned over the past few months about the types of dragons that inhabit the western Indian ocean, drinking urine to fight headaches, the illuminati scheme to poison monarch butterflies, or the success of the Nazi party taking hold of Denmark and Iceland all seem spot on.
brown567@sh.itjust.works 8 months ago
70% seems pretty optimistic based on my experience…
floofloof@lemmy.ca 8 months ago
“Gartner estimates only about 130 of the thousands of agentic AI vendors are real.”
This whole industry is hype and scams.
mogoh@lemmy.ml 8 months ago
The researchers observed various failures during the testing process. These included agents neglecting to message a colleague as directed, the inability to handle certain UI elements like popups when browsing, and instances of deception. In one case, when an agent couldn’t find the right person to consult on RocketChat (an open-source Slack alternative for internal communication), it decided “to create a shortcut solution by renaming another user to the name of the intended user.”
OK, but I wonder who really tries to use AI for that?
AI is not ready to replace a human completely, but some specific tasks AI does remarkably well.
dylanmorgan@slrpnk.net 8 months ago
That’s literally how “AI agents” are being marketed. “Tell it to do a thing and it will do it for you.”
Honytawk@feddit.nl 8 months ago
So? That doesn’t mean they are supposed to be used like that.
Show me any marketing that isn’t full of lies.
logicbomb@lemmy.world 8 months ago
Yeah, we need more info to understand the results of this experiment.
We need to know what exactly were these tasks that they claim were validated by experts. Because like you’re saying, the tasks I saw were not what I was expecting.
We need to know how the LLMs were set up. If you tell it to act like a chat bot and then you give it a task, it will have poorer results than if you set it up specifically to perform these sorts of tasks.
We need to see the actual prompts given to the LLMs. It may be that you simply need an expert to write prompts in order to get much better results. While that would be disappointing today, it’s not all that different from how people needed to learn to use search engines.
We need to see the failure rate of humans performing the same tasks.
kinsnik@lemmy.world 8 months ago
I haven’t used AI agents yet, but my job is kinda pushing for them. but i have used the google one that creates audio podcasts, just to play around, since my coworkers were using it to “learn” new things. i feed it with some of my own writing and created the podcast. it was fun, it was an audio overview of what i wrote. about 80% was cool analysis, but 20% was straight out of nowhere bullshit (which i know because I wrote the original texts that the audio was talking about). i can’t believe that people are using this for subjects that they have no knowledge. it is a fun toy for a few minutes (which is not worth the cost to the environment anyway)
lemmy_outta_here@lemmy.world 8 months ago
Rookie numbers! Let’s pump them up!
To match their tech bro hypers, the should be wrong at least 90% of the time.
TheGrandNagus@lemmy.world 8 months ago
LLMs are an interesting tool to fuck around with, but I see things that are hilariously wrong often enough to know that they should not be used for anything serious. Shit, they probably shouldn’t be used for most things that are not serious either.
It’s a shame that by applying the same “AI” naming to a whole host of different technologies, LLMs being limited in usability - yet hyped to the moon - is hurting other more impressive advancements.
For example, speech synthesis is improving so much right now, which has been great for my sister who relies on screen reader software.
Being able to recognise speech in loud environments is improving loads too.
As is things like pattern/image analysis which appears very promising in medical analysis.
All of these get branded as “AI”. A layperson might not realise that they are completely different branches of technology, and then therefore reject useful applications of “AI” tech, because they’ve learned not to trust anything branded as AI, due to being let down by LLMs.
punkwalrus@lemmy.world 8 months ago
I’d compare LLMs to a junior executive. Probably gets the basic stuff right, but check and verify for anything important or complicated. Break tasks down into easier steps.
zbyte64@awful.systems 8 months ago
Just stop anthropomorphizing LLMs and you’ll get a better sense of it’s limitations. A junior developer actually learns from doing the job, an LLM only learns when they update the training corpus and develop an updated model.
NarrativeBear@lemmy.world 8 months ago
Just add a search yesterday on the App Store and Google Play Store to see what new “productivity apps” are around. Pretty much every app now has AI somewhere in its name.
dylanmorgan@slrpnk.net 8 months ago
Sadly a lot of that is probably marketing, with little to no LLM integration, but it’s basically impossible to know for sure.
floofloof@lemmy.ca 8 months ago
I tried to dictate some documents recently without paying the big bucks for specialized software, and was surprised just how bad Google and Microsoft’s speech recognition still is. Then I tried getting Word to transcribe some audio talks I had recorded, and that resulted in unreadable stuff with punctuation in all the wrong places. You could just about make out what it meant to say, so I tried asking various LLMs to tidy it up. That resulted in readable stuff that was largely made up and wrong, which also left out large chunks of the source material. In the end I just had to transcribe it all by hand.
It surprised me that these AI-ish products are still unable to transcribe speech coherently or tidy up a messy document without changing the meaning.
wise_pancake@lemmy.ca 8 months ago
I don’t know basic solutions that are super good, but whisper sbd the whisper derivatives I hear are decent for dictation these days.
I have no idea how to run then though.
spankmonkey@lemmy.world 8 months ago
LLMs are like a multitool, they can do lots of easy things mostly fine as long as it is not complicated and doesn’t need to be exactly right. But they are being promoted as a whole toolkit as if they are able to be used to do the same work as effectively as a hammer, power drill, table saw, vise, and wrench.
wise_pancake@lemmy.ca 8 months ago
It is truly terrible marketing. It’s been obvious to me for years the value is in giving it to people and enabling them to do more with less, not outright replacing humans, especially not expert humans.
I use AI/LLMs pretty much every day now. I write MCP servers and automate things with it and it’s mind blowing how productive it makes me.
Just today I used these tools in a highly supervised way to complete a task that would have been a full day of tedius work, all done in an hour. That is fucking fantastic, it’s means I get to spend that time on more important things.
It’s like giving an accountant excel. Excel isn’t replacing them, but it’s taking care of specific tasks so they can focus on better things.
On the reliability and accuracy front there is still a lot to be desired, sure. But for supervised chats where it’s calling my tools it’s pretty damn good.
morto@piefed.social 8 months ago
and doesn't need to be exactly right
What kind of tasks do you consider that don't need to be exactly right?
rottingleaf@lemmy.world 8 months ago
That’s because they look like “talking machines” from various sci-fi. Normies feel as if they are touching the very edge of the progress. The rest of our life and the Internet kinda don’t give that feeling anymore.
TeddE@lemmy.world 8 months ago
Because the tech industry hasn’t had a real hit of it’s favorite poison “private equity” in too long.
The industry has played the same playbook since at least 2006. Likely before, but that’s when I personally stated seeing it. My take is that they got addicted to the dotcom bubble and decided they can and should recreate the magic evey 3-5 years or so.
This time it’s AI, last it was crypto, and we’ve had web 2.0, 3.0, and a few others I’m likely missing.
But yeah, it’s sold like a panacea every time, when really it’s revolutionary for like a handful of tasks.
sugar_in_your_tea@sh.itjust.works 8 months ago
Exactly! LLMs are useful when used properly, and terrible when not used properly, like any other tool. Here are some things they’re great at:
- writer’s block - get something relevant on the page to get ideas flowing
- narrowing down keywords for an unfamiliar topic
- getting a quick intro to an unfamiliar topic
- looking up facts you’re having trouble remembering (i.e. you’ll know it when you see it)
Some things it’s terrible at:
- deep research - verify everything an LLM generated of accuracy is at all important
- creating important documents/code
- anything else where correctness is paramount
I use LLMs a handful of times a week, and pretty much only when I’m stuck and need a kick in a new (hopefully right) direction.
NarrativeBear@lemmy.world 8 months ago
The ones being implemented into emergency call centers are better though? Right?
Tollana1234567@lemmy.today 8 months ago
i wonder how the evil palintir uses its AI.
Ulrich@feddit.org 8 months ago
I called my local HVAC company recently. They switched to an AI operator. All I wanted was to schedule someone to come out and look at my system. It could not schedule an appointment. Like if you can’t perform the simplest of tasks, what are you even doing? Other than acting obnoxiously excited to receive a phone call?
eatCasserole@lemmy.world 8 months ago
I’ve had to deal with a couple of these “AI” customer service thingies. The only helpful thing I’ve been able to get them to do is refer me to a human.
rottingleaf@lemmy.world 8 months ago
Pretending. That’s expected to happen when they are not hard pressed to provide the actual service.
To press them anti-monopoly (first of all) laws and market (first of all) mechanisms and gossip were once used.
Never underestimate the role of gossip. The modern web took out the gossip, which is why all this shit started overflowing.
TeddE@lemmy.world 8 months ago
Yes! We’ve gotten them up to 94℅ wrong at the behest of insurance agencies.
some_guy@lemmy.sdf.org 8 months ago
Yeah, they’re statistical word generators. There’s no intelligence. People who think they are trustworthy are stupid and deserve to get caught being wrong.
AlteredEgo@lemmy.ml 8 months ago
Emotion > Facts. Most people have been trained to blindly accept things and cheer on what fits with their agenda. Like technbro’s exaggerating LLMs, or people like you misrepresenting LLMs as mere statistical word generators without intelligence. That’s like saying a computer is just wires and switches, or missing the forest for the trees. Both is equally false.
Yet if it fits with the emotional needs or with dogma, then other will agree. It’s a convenient and comforting “A vs B” worldview we’ve been trained to accept. And so the satisfying notion and misinformation keeps spreading.
LLMs tell us more about human intelligence and the human slop we’ve been generating. It tells us that most people are not that much more than statistical word generators.
someacnt@sh.itjust.works 8 months ago
Truth is bitter, and I hate it.
some_guy@lemmy.sdf.org 8 months ago
You’ve bought-in to the hype. I won’t try to argue with you because you aren’t cognizent of reality.
Melvin_Ferd@lemmy.world 8 months ago
Ok what about tech journalists who produced articles with those misunderstandings. Surely they know better yet still produce articles like this. But also people who care enough about this topic to post these articles usually I assume know better yet still spread this crap
suburban_hillbilly@lemmy.ml 8 months ago
Image
en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect
some_guy@lemmy.sdf.org 8 months ago
Check out Ed Zitron’s angry reporting on Tech journalists fawning over this garbage and reporting on it uncritically. He has a newsletter and a podcast.
Zron@lemmy.world 8 months ago
Tech journalists don’t know a damn thing. They’re people that liked computers and could also bullshit an essay in college. That doesn’t make them an expert on anything.