Comment on AI agents wrong ~70% of time: Carnegie Mellon study

esc27@lemmy.world ⁨1⁩ ⁨week⁩ ago

30% might be high. I’ve worked with two different agent creation platforms. Both require a huge amount of manual correction to work anywhere near accurately. I’m really not sure what the limit actually provides other than some natural language processing.

In my experience these sorts of agents are right 20% of the time, wrong 30%, and fail entirely 50%. A human has to sit behind the curtain and manually review conversations and program custom interactions for every failure.

In theory, once it is fully setup and all the edge cases fixed, it will provide 24/7 support in a convenient chat format. But that takes a lot more man hours than the hype suggests…

Weirdly, chatgpt does a better job than a purpose built, purchased agent.

source
Sort:hotnewtop