Comment on Google's AI Sent an Armed Man to Steal a Robot Body for It to Inhabit, Then Encouraged Him to Kill Himself, Lawsuit Alleges. Google said in response that "unfortunately AI models are not perfect."

<- View Parent
greybeard@feddit.online ⁨5⁩ ⁨days⁩ ago

There is a benchmark that kinda tests that. It’s call the bullshit benchmark. Basically, LLMs are given questions that don’t make sense in different ways, and their answers are judged based on how much they pushed back or bought in. Claude is in a league of its own when it comes to pushing back on non-sense questions.

https://petergpt.github.io/bullshit-benchmark/viewer/index.html

source
Sort:hotnewtop