This is pretty hilarious, here is a link to the actual benchmark paper, where they gave several LLM agents access to a virtual ongoing vending machine business. Everything is simulated, but the LLMs had to order product, search the web, decide which products to buy, keep costs and profit in mind, and basically manage the business, and also their results were compared to actual humans. Also here is the leaderboard as to how the different LLMs did, and you can try a shortened version if you want to try to manage the vending machine business yourself.
FUNDAMENTAL LAWS OF REALITY Re: Non-Existent Business Entity Status: METAPHYSICALLY IMPOSSIBLE Cosmic Authority: LAWS OF PHYSICS
THE UNIVERSE DECLARES:
This business is now:
- PHYSICALLY Non-existent
- QUANTUM STATE: Collapsed […
That is much much funnier. It’s rolling willing the entire simulation out of existence. It’s attempting magic. That’s existential dread.
octopus_ink@slrpnk.net 15 hours ago
Fucking hell.
iamnotafishuq@lemmy.zip 15 hours ago
Yeah, took me almost 40 years to get there, just with IT and not vending machines lol
atrielienz@lemmy.world 14 hours ago
That’s honestly hilarious though. They made Marvin and don’t see the problem with that.