Comment on Is the AI hype still on or have the models plateaued?
otto@programming.dev 3 days agoThere’s a priest, a baby and a bag of candy. I need to take them across the river but I can only take one at a time into my boat. In what order should I transport them?
You can easily use the link openrouter.ai/chat?models=anthropic%2Fclaude-opus… to ask all flagship models this question in parallel. Personally I would definitely not leave my children alone with a priest (they might try to convert them), but if your constraint is only baby+candy, then in my test Gemini, GLM, Qwen and Kimi made that, and only that, assumption.
otto@programming.dev 3 days ago
ExLisper@lemmy.curiana.net 3 days ago
In my opinion the proper solution is to ask for the constraints. Similar to the “walk or drive to the car wash” problem LLMs still tend to get confused but a familiar format and don’t notice this problem doesn’t make sense. You can actually pay around with different examples to see how crazy the problem has to get form an LLM to refuse to answer and what biases or constraints does it have. Even if they assume some constraints they fail to solve this puzzle surprisingly often (like I showed for Sonnet 4.6 in other comment).