Comment on Announcing ARC-AGI-3 - An benchmark that tests if AI can explore, learn, and adapt in unfamiliar situations. Humans score 100%. Frontier AI scores 0.26%.

• davidgro@lemmy.world ⁨5⁩ ⁨hours⁩ ago

  Those are high scores.

  source

  • lath@lemmy.world ⁨4⁩ ⁨hours⁩ ago

    🤔 So this is a visual comparison between peak performance of some humans and peak performance of current LLMs in a controlled environment?

    source

    • floquant@lemmy.dbzer0.com ⁨3⁩ ⁨hours⁩ ago

      Is this a gotcha? Not sure where you got the “visual” from, but yes it is best human performance vs best LLM performance

      source

      • lath@lemmy.world ⁨3⁩ ⁨hours⁩ ago

        I don’t know why you assume there has to be a gotcha, maybe it’s the competitive background… Anyway, it’s visual because you look at it to see it. And it’s not the best human performance vs best LLM performance, it’s best controlled performance because the testing is limited to a set of parameters.

        source

        • -> View More Comments