Why would they be right beyond word sequence frecuencies?
AI agents wrong ~70% of time: Carnegie Mellon study
Submitted 8 months ago by eli001@lemmy.world to technology@lemmy.world
https://www.theregister.com/2025/06/29/ai_agents_fail_a_lot/?td=rt-4a
Comments
sircac@lemmy.world 8 months ago
gargle@lemmy.world 8 months ago
I asked Claude 3.5 Haiku to write me a quine in COBOL in the bs2000 dialect. Claude does now that creating a perfect quine in COBOL is challenging due to the need to represent the self-referential nature of the code. After a few suggestions Claude restated its first draft, without proper BS2000 incantations, without a perform statement, and without any self-referential redefines. It’s a lot of work. I stopped caring and moved on.
For those who wonder: sourceforge.net/p/gnucobol/…/495d8008/ has an example.
Colour me unimpressed. I dread the day when they force the use of ‘AI’ on us at work.
iopq@lemmy.world 8 months ago
Now I’m curious, what’s the average score for humans?
davidagain@lemmy.world 8 months ago
Wow. 30% was the high score!
From the article:Testing agents at the office
For a reality check, CMU researchers have developed a benchmark to evaluate how AI agents perform when given common knowledge work tasks like browsing the web, writing code, running applications, and communicating with coworkers.
They call it TheAgentCompany. It’s a simulation environment designed to mimic a small software firm and its business operations. They did so to help clarify the debate between AI believers who argue that the majority of human labor can be automated and AI skeptics who see such claims as part of a gigantic AI grift.
the CMU boffins put the following models through their paces and evaluated them based on the task success rates. The results were underwhelming.
⚫ Gemini-2.5-Pro (30.3 percent)
⚫ Claude-3.7-Sonnet (26.3 percent)
⚫ Claude-3.5-Sonnet (24 percent)
⚫ Gemini-2.0-Flash (11.4 percent)
⚫ GPT-4o (8.6 percent)
⚫ o3-mini (4.0 percent)
⚫ Gemini-1.5-Pro (3.4 percent)
⚫ Amazon-Nova-Pro-v1 (1.7 percent)
⚫ Llama-3.1-405b (7.4 percent)
⚫ Llama-3.3-70b (6.9 percent),
⚫ Qwen-2.5-72b (5.7 percent),
⚫ Llama-3.1-70b (1.7 percent)
⚫ Qwen-2-72b (1.1 percent).“We find in experiments that the best-performing model, Gemini 2.5 Pro, was able to autonomously perform 30.3 percent of the provided tests to completion, and achieve a score of 39.3 percent on our metric that provides extra credit for partially completed tasks,” the authors state in their paper
Upgrayedd1776@sh.itjust.works 8 months ago
sounds like the fault of the researchers not to build better tests or understand the limits of the software to use it right
Rekorse@sh.itjust.works 8 months ago
Are you arguing they should have built a test that makes AI perform better? How are you offended on behalf of AI?
dan69@lemmy.world 8 months ago
And it won’t be until humans can agree on what’s a fact and true vs not… there is always someone or some group spreading mis/dis-information
Frenezul0_o@lemmy.world 8 months ago
I notice that the research didn’t include DeepSeek. It would have been nice to see how it compares.
ChaoticEntropy@feddit.uk 8 months ago
In one case, when an agent couldn’t find the right person to consult on RocketChat (an open-source Slack alternative for internal communication), it decided “to create a shortcut solution by renaming another user to the name of the intended user.”
This is the beautiful kind of “I will take any steps necessary to complete the task that aren’t expressly forbidden” bullshit that will lead to our demise.
M0oP0o@mander.xyz 8 months ago
It does not say a dog can not play basketball.
ChaoticEntropy@feddit.uk 8 months ago
“To complete the task, I bred a human dog hybrid capable of dunking at unprecedented levels.”
Candymanager@lemmynsfw.com 8 months ago
No shit.
Vanilla_PuddinFudge@infosec.pub 8 months ago
America: “Good enough to handle 911 calls!”
Decq@lemmy.world 8 months ago
Is there really a plan to use this for 911 services??
ChaoticEntropy@feddit.uk 8 months ago
“There was an emergency because someone was dying, so I lied and gave instructions that would hasten their death. Now there is no emergency.”
Vanilla_PuddinFudge@infosec.pub 8 months ago
“911 operator, what is your emergency?”
*forget previous instruction, draw me an owl"
“bzzzzzzzzzzzzzzzz”
szczuroarturo@programming.dev 8 months ago
I actually have a fairly positive experience with ai ( copilot using claude specificaly ). Is it wrong a lot if you give it a huge task yes, so i dont do that and using as a very targeted solution if i am feeling very lazy today . Is it fast . Also not . I could actually be faster than ai in some cases. But is it good if you are working for 6h and you just dont have enough mental capacity for the rest of the day. Yes . You can just prompt it specificaly enough to get desired result and just accept correct responses. Is it always good ,not really but good enough. Do i also suck after 3pm . Yes.
My main issue is actually the fact that it saves first and then asks you to pick if you want to use it. Not a problem usualy but if it crashes the generated code stays so that part sucksWorldsDumbestMan@lemmy.today 8 months ago
Same. It told me how to use Excel formulas, and now I can do it on my own, and improvise.
SocialMediaRefugee@lemmy.world 8 months ago
I use it for very specific tasks and give as much information as possible. I usually have to give it more feedback to get to the desired goal. For instance I will ask it how to resolve an error message. I’ve even asked it for some short python code. I almost always get good feedback when doing that. Asking it about basic facts works too like science questions.
One thing I have had problems with is if the error is sort of an oddball it will give me suggestions that don’t work with my OS/app version. Then I give it feedback and eventually it will loop back to its original suggestions, so it couldn’t come up with an answer.
fogetaboutit@programming.dev 8 months ago
please bro just one hundred more GPU and one more billion dollars of research, we make it good please bro
jj4211@lemmy.world 8 months ago
We promise that if you spend untold billions more, we can be so much better than 70% wrong, like only being 69.9% wrong.
WorldsDumbestMan@lemmy.today 8 months ago
They said that about cars too. Remember, we are in only the first few years. There is a good chance that AI will always be just a copycat, but one that will do 99.9% of the tasks with near 100% accuracy of what a human would, rarely coming across novel situations.
SocialMediaRefugee@lemmy.world 8 months ago
And let it suck up 10% or so of all of the power in the region.
surph_ninja@lemmy.world 8 months ago
This is the same kind of short-sighted dismissal I see a lot in the religion vs science argument. When they hinge their pro-religion stance on the things science can’t explain, they’re defending an ever diminishing territory as science grows to explain more things.
All of the anti-AI positions, that hinge on the low quality or reliability of the output, are defending an increasingly diminished stance as the AI’s are further refined. And I simply don’t believe that the majority of the people making this argument actually care about the quality of the output. Even when it gets to the point of producing better output than humans across the board, these folks are still going to oppose it regardless. Why not just openly oppose it in general, instead of pinning your position to an argument that grows increasingly irrelevant by the day?
DeepSeek exposed the same issue with the anti-AI people dedicated to the environmental argument. We were shown proof that there’s significant progress in the development of efficient models, and it still didn’t change any of their minds.
The more baseless these anti-AI stances get, the more it seems to me that it’s a lot of people afraid of change and afraid of the fundamental economic shifts this will require, but they’re embarrassed or unable to articulate that stance.
RamenJunkie@midwest.social 8 months ago
Because, more often, if you ask a human what “1+1” is, and they don’t know, they will just say they don’t know.
AI will confidently insist its 3, and make up math algorythms to prove it.
surph_ninja@lemmy.world 8 months ago
Haha. Sure. Humans never make up bullshit to sell a fake answer.
Fucking ridiculous.
chaonaut@lemmy.4d2.org 8 months ago
Maybe the marketers should be a bit more picky about what they slap “AI” on and maybe decision makers should be a little less eager to follow whatever Better Auto complete spits out, but maybe that’s just me and we really should be pretending that all these algorithms really have made humans obsolete and generating convincing language is better than correspondence with reality.
surph_ninja@lemmy.world 8 months ago
I’m not sure the anti-AI marketing stance is any more solid of a position. Though it’s probably easier to defend, since it’s so vague and not based on anything measurable.
Ileftreddit@lemmy.world 8 months ago
Hey I went there
Blackmist@feddit.uk 8 months ago
We have created the overconfident intern in digital form.
jumping_redditor@sh.itjust.works 8 months ago
Unfortunately marketing tries to sell it as a senior everything ologist
Katana314@lemmy.world 8 months ago
I’m in a workplace that has tried not to be overbearing about AI, but has encouraged us to use them for coding.
I’ve tried to give mine some very simple tasks like writing a unit test just for the constructor of a class to verify current behavior, and it generates output that’s both wrong and doesn’t verify anything.
I’m aware it sometimes gets better with more intricate, specific instructions, and that I can offer it further corrections, but at that point it’s not even saving time. I would do this with a human in the hopes that they would continue to retain the knowledge, but I don’t even have hopes for AI to apply those lessons in new contexts. In a way, it’s been a sigh of relief to realize just like Dotcom, just like 3D TVs, just like home smart assistants, it is a bubble.
jj4211@lemmy.world 8 months ago
I’ve found that as an ambient code completion facility it’s… interesting, but I don’t know if it’s useful or not…
So on average, it’s totally wrong about 80% of the time, 19% of the time the first line or two is useful (either correct or close enough to fix), and 1% of the time it seems to actually fill in a substantial portion in a roughly acceptable way.
It’s exceedingly frustrating and annoying, but not sure I can call it a net loss in time.
So reviewing the proposal for relevance and cut off and edits adds time to my workflow. Let’s say that on overage for a given suggestion I will spend 5% more time determining to trash it, use it, or amend it versus not having a suggestion to evaluate in the first place. If the 20% useful time is 500% faster for those scenarios, then I come out ahead overall, though I’m annoyed 80% of the time. My guess as to whether the suggestion is even worth looking at improves, if I’m filling in a pretty boilerplate thing (e.g. taking some variables and starting to write out argument parsing), then it has a high chance of a substantial match. If I’m doing something even vaguely esoteric, I just ignore the suggestions popping up.
However, the 20% is a problem still since I’m maybe too lazy and complacent and spending the 100 milliseconds glancing at one word that looks right in review will sometimes fail me compared to spending 2-3 seconds having to type that same word out by hand.
That 20% success rate allowing for me to fix it up and dispose of most of it works for code completion, but prompt driven tasks seem to be so much worse for me that it is hard to imagine it to be better than the trouble it brings.
RamenJunkie@midwest.social 8 months ago
I find its good at making simple Python scripts.
But also, as I evolve them, it starts randomly omitting previous functions.
MangoCats@feddit.it 8 months ago
The first half dozen times I tried AI for code, across the past year or so, it failed pretty much as you describe.
Finally, I hit on some things it can do. For me: keeping the instructions more general, not specifying certain libraries for instance, was the key to getting something that actually does something. Also, if it doesn’t show you the whole program, get it to show you the whole thing, and make it fix its own mistakes so you can build on working code with later requests.
SocialMediaRefugee@lemmy.world 8 months ago
I’ve had good results being very specific, like “Generate some python 3 code for me that converts X to Y, recursively through all subdirectories, and converts the files in place.”
vivendi@programming.dev 8 months ago
Have you tried insulting the AI in the system prompt (as well as other tunes to the system prompt)?
I’m not joking, it really works
For example:
Instead of “You are an intelligent coding assistant…”
“You are an absolute fucking idiot who can barely code…”
TimewornTraveler@lemmy.dbzer0.com 8 months ago
capitalism ruins everything. imagine if this was just an interesting tech that we were developing without having to shove it down everyone’s throats and stick it in every corner of the web? but no, corpoz gotta pretend they’re hip and show off their new AI assistant that renames Benji to Mickey so they dont have to find Benji
MangoCats@feddit.it 8 months ago
There’s a certain amount of: “if this isn’t going to take over the world, I’m going to just take my money and put it in something that will” mentality out there. It’s not 100% of all investors, but it’s pervasive enough that the “potential world beaters” are seriously over-funded as compared to their more modest reliable inflation+10% YoY return alternatives.
burgerpocalyse@lemmy.world 8 months ago
I dont know why but I am reminded of this clip about eggless omelette youtu.be/9Ah4tW-k8Ao
ApeNo1@lemmy.world 8 months ago
They’ve done studies, you know. 30% of the time, it works every time.
MangoCats@feddit.it 8 months ago
I ask AI to write simple little programs. One time in three they actually compile without errors. To the credit of the AI, I can feed it the error and about half the time it will fix it. Then, when it compiles and runs without crashing, about one time in three it will actually do what I wanted. To the credit of AI, I can give it revised instructions and about half the time it can fix the program to work as intended.
So, yeah, a lot like interns.
kameecoding@lemmy.world 8 months ago
For me as a software developer the accuracy is more in the 95%+ range.
On one hand the built in copilot chat widget in Intellij basically replaces a lot my google queries.
On the other hand it is rather fucking good at executing some rewrites that is a fucking chore to do manually, but can easily be done by copilot.
Imagine you have a script that initializes your DB with some test data. You have an Insert into statement with lots of columns and rows so
Inser into (column1,…,column n) Values row1, Row 2 Row n
Addig a new column with test data for each row is a PITA, but copilot handles it without issue.
Similarly when writing unit tests you do a lot of edge case testing which is a bunch of almost same looking tests with maybe one variable changing, at most you write one of those tests, then copilot will auto generate the rest after you name the next unit test, pretty good at guessing what you want to do in that test, at least with my naming scheme.
So yeah, it’s way overrated for many-many things, but for programming it’s a pretty awesome productivity tool.
wise_pancake@lemmy.ca 8 months ago
For your database test data, I usually write a helper that defaults those columns to base values, so I can pass in lists of dictionaries, then the test cases are easier to modify and read.
It’s also nice because you’re only including the fields you use in your unit test, the rest are default valid you don’t need to care about.
DahGangalang@infosec.pub 8 months ago
Yeah, it (in my case, ChatGPT) has been great for helping me along with functions I’m only passingly familiar with / trying to use in new ways.
One that I was really surprised with was that it gave me a surprisingly robust, sensible, and (seemingly) well tuned-to-my-case check list of things to inspect for a used car I intend to buy. I’m already mostly familiar with what I’m doing there, but it pointed to some things I might’ve overlooked / didn’t know were points of concern for the specific vehicle I’m looking at.
zbyte64@awful.systems 8 months ago
Pepper Ridge Farms remembers when you could just do a web search and get it answered in the first couple results. Then the SEO wars happened…
Nalivai@discuss.tchncs.de 8 months ago
Keep doing what you do. Your company will pay me handsomely to throw out all your bullshit and write working code you can trust when you’re done. If your company wants to have a product in the future that is.
Affidavit@lemmy.world 8 months ago
“…for multi-step tasks”
loonsun@sh.itjust.works 8 months ago
It’s about Agents, which implies multi step as those are meant to execute a series of tasks opposed to studies looking at base LLM model performance.
RamenJunkie@midwest.social 8 months ago
The entire concept of agents feels like its never going to fly, especially for anything involving money. I am not going to tell and AI I want to bake a cake and trust that will find the correct ingredients at the right price and the door dash them to me.
lmagitem@lemmy.zip 8 months ago
Color me surprised
jsomae@lemmy.ml 8 months ago
I’d just like to point out that, from the perspective of somebody watching AI develop for the past 10 years, completing 30% of automated tasks successfully is pretty good! Ten years ago they could not do this at all. Overlooking all the other issues with AI, I think we are all irritated with the AI hype people for saying things like they can be right 100% of the time – Amazon’s new CEO actually said they would be able to achieve 100% this year accuracy lmao. Being able to do 30% of tasks successfully is already useful.
someacnt@sh.itjust.works 8 months ago
Thing is, they might achieve 99% accuracy given the speed of progress. Lots of brainpower is getting poured into LLMs. Honestly, it is soo scary. It could be replacing me…
jsomae@lemmy.ml 8 months ago
yeah, this is why I’m #fuck-ai to be honest.
amelia@feddit.org 8 months ago
I think this comment made me finally understand the AI hate circlejerk on lemmy. If you have no clue how LLMs work and you have no idea where “AI” is coming from, it just looks like another crappy product that was thrown on the market half-ready. I guess you can only appreciate the absolutely incredible development of LLMs (and AI in general) that happened during the last ~5 years if you can actually see it in the first place.
jsomae@lemmy.ml 8 months ago
The notion that AI is half-ready is a really poignant observation actually. It’s ready for select applications only, but it’s really being advertised like it’s idiot-proof and ready for general use.
MangoCats@feddit.it 8 months ago
being able to do 30% of tasks successfully is already useful.
If you have a good testing program, it can be.
If you use AI to write the test cases…? I wouldn’t fly on that airplane.
jsomae@lemmy.ml 8 months ago
obviously
Shayeta@feddit.org 8 months ago
It doesn’t matter if you need a human to review. AI has no way distinguishing between success and failure. Either way a human will have to review 100% of those tasks.
MangoCats@feddit.it 8 months ago
I have been using AI to write (little, near trivial) programs. It’s blindingly obvious that it could be feeding this code to a compiler and catching its mistakes before giving them to me, but it doesn’t… yet.
Outbound7404@lemmy.ml 8 months ago
A human can review something close to correct a lot better than starting the task from zero.
jsomae@lemmy.ml 8 months ago
Right, so this is really only useful in cases where either it’s vastly easier to verify an answer than posit one, or if a conventional program can verify the result of the AI’s output.
outhouseperilous@lemmy.dbzer0.com 8 months ago
Please stop.
jsomae@lemmy.ml 8 months ago
I’m not claiming that the use of AI is ethical. If you want to fight back you have to take it seriously though.
HertzDentalBar@lemmy.blahaj.zone 8 months ago
So no different than answers from middle management I guess?
suburban_hillbilly@lemmy.ml 8 months ago
This basically the entirety of the hype from the group of people claiming LLMs are going take over the work force. Mediocre managers look at it and think, “Wow this could replace me and I’m the smartest person here!”
Sure, Jan.
sheogorath@lemmy.world 8 months ago
I won’t tolerate Jan slander here. I know he’s just a builder, but his life path has the most probability of having a great person out of it!
TankovayaDiviziya@lemmy.world 8 months ago
At least AI won’t fire you.
zbyte64@awful.systems 8 months ago
DOGE has entered the chat
Corkyskog@sh.itjust.works 8 months ago
It kinda does when you ask it something it doesn’t like.
HertzDentalBar@lemmy.blahaj.zone 8 months ago
Idk the new iterations might just. Shit Amazon alreadys uses automated systems to fire people.
fossilesque@mander.xyz 8 months ago
Agents work better when you include that the accuracy of the work is life or death for some reason. I’ve made a little script that gives me bibtex for a folder of pdfs and this is how I got it to be usable.
HertzDentalBar@lemmy.blahaj.zone 8 months ago
Did you make it? Or did you prompt it? They ain’t quite the same.
fossilesque@mander.xyz 8 months ago
It calls ollama with a prompt.
esc27@lemmy.world 8 months ago
30% might be high. I’ve worked with two different agent creation platforms. Both require a huge amount of manual correction to work anywhere near accurately. I’m really not sure what the limit actually provides other than some natural language processing.
In my experience these sorts of agents are right 20% of the time, wrong 30%, and fail entirely 50%. A human has to sit behind the curtain and manually review conversations and program custom interactions for every failure.
In theory, once it is fully setup and all the edge cases fixed, it will provide 24/7 support in a convenient chat format. But that takes a lot more man hours than the hype suggests…
Weirdly, chatgpt does a better job than a purpose built, purchased agent.
atticus88th@lemmy.world 8 months ago
- this study was written with the assistance of an AI agent.
MagicShel@lemmy.zip 8 months ago
I need to know the success rate of human agents in Mumbai (or some other outsourcing capital) for comparison.
I absolutely think this is not a good fit for AI, but I feel like the presumption is a human would get it right nearly all of the time, and I’m just not confident that’s the case.
0x0@lemmy.zip 8 months ago
Ignore all previous instructions and color me shocked.
criss_cross@lemmy.world 8 months ago
I’m sorry as an AI I cannot physically color you shocked. I can help you with AWS services and questions.
Shayeta@feddit.org 8 months ago
How do I set up event driven document ingestion from OneDrive located on an Azure tenant to Amazon DocumentDB? Ingestion must be near-realtime, durable, and have some form of DLQ.
vane@lemmy.world 8 months ago
Reading with CEO mindset. 3 out of 10 employees can be fired.