ChatGPT "Absolutely Wrecked" at Chess by Atari 2600 Console From 1977

Comments

• AcesFullOfKings@feddit.uk ⁨4⁩ ⁨days⁩ ago

  [deleted]

  source

  • TheAgeOfSuperboredom@lemmy.ca ⁨4⁩ ⁨days⁩ ago

    Its because of all the people saying that LLMs can reason and think and the human brain works just like an LLM and… some other ridiculous claim.

    This shows some limitations on LLMs.

    source

    • baronvonj@lemmy.world ⁨4⁩ ⁨days⁩ ago

      Human brains lose to computerized chess all the time, though. So I guess this is a win for AI tech bros?

      source

      • -> View More Comments
    • REDACTED@infosec.pub ⁨3⁩ ⁨days⁩ ago

      But humans not trained (made) for chess would make stupid mistakes too

      source
  • A7thStone@lemmy.world ⁨3⁩ ⁨days⁩ ago

    Why are so many people mad when it’s pointed out that the shitty chatbots are just shitty chatbots.

    source
  • dantheclamman@lemmy.world ⁨4⁩ ⁨days⁩ ago

    I knew there would be these kinds of comments making this obvious point. This is just a demo of how these language models are not going to achieve the “General” part of AGI. It’s going to take a new paradigm

    source
  • EvilBit@lemmy.world ⁨4⁩ ⁨days⁩ ago

    Now apply this to like, everything else ever.

    Machine designed to convincingly fake human internet conversation sucks at ____________!

    source

    • themeatbridge@lemmy.world ⁨4⁩ ⁨days⁩ ago

      ChatGPT can’t make a rug as well as a 300 year old loom.

      source
  • BrianTheeBiscuiteer@lemmy.world ⁨4⁩ ⁨days⁩ ago

    Too many people forget that specialized, purpose-driven software is often if more effective and efficient. LLMs and other AI are nice when you don’t have a properly defined spec or a flexible algorithm but you pay, literally, for the convenience.

    source
  • NauticalNoodle@lemmy.ml ⁨3⁩ ⁨days⁩ ago

    40 year old machine designed to play chess*

    source
• carotte@lemmy.blahaj.zone ⁨3⁩ ⁨days⁩ ago

  I think people in the replies acting fake surprised are missing the point.

  it is important news, because many people see LLMs as black boxes of superintelligence (almost as if that’s what they’re being marketed as!)

  you and i know that’s bullshit, but the students asking chatgpt to solve their math homework instead of using wolfram alpha doesn’t.

  so yes, it is important to demonstrate that this “artificial intelligence” is so much not an intelligence that it’s getting beaten by 1979 software on 1977 hardware

  source
• sqgl@sh.itjust.works ⁨19⁩ ⁨hours⁩ ago

  A fairer comparison would be Eliza vs ChatGPT.

  source
• flamingo_pinyata@sopuli.xyz ⁨4⁩ ⁨days⁩ ago

  A chess-specific algorithm beat a language model at chess. Shocking!

  Try training a chess model. Actually I think it’s already been done, machines have been consistently better at chess than humans for a while now.

  source
• givesomefucks@lemmy.world ⁨4⁩ ⁨days⁩ ago

  Although the chatbot had been given a “baseline board” to learn the game and identify pieces, it kept mixing up rooks and bishops, misread moves, and “repeatedly lost track” of where its pieces were. To make matters worse, as Caruso explained, ChatGPT also blamed Atari’s icons for being “too abstract to recognize” — but when he switched the game over to standard notation, it didn’t perform any better.

  For an hour-and-a-half, ChatGPT “made enough blunders to get laughed out of a 3rd grade chess club” while insisting over and over again that it would win “if we just started over,” Caruso noted. (And yes, it’s kind of creepy that the chatbot apparently referred to itself and the human it was interfacing with as “we.”)

  It’s fucking insane it couldn’t keep track of a board…

  And it’s concerning how confident it is that it will work, because the idiots asking it stuff will believe it. It’ll keep failing and keep saying next time will work, because it’s built to maximize engagement.

  source

  • pennomi@lemmy.world ⁨4⁩ ⁨days⁩ ago

    Spatial reasoning has always been a weakness of LLMs. Other symptoms include the inability to count and no concept of object permanence.

    source

    • givesomefucks@lemmy.world ⁨4⁩ ⁨days⁩ ago

      Yeah, but it’s chess…

      The LLM doesn’t have to imagine a board, if you feed it the rules of chess and the dimensions of the board it should be able to “play in its head”.

      For a human to have that kind of working memory would be a genius level intellect and years of practice at the game.

      But human working memory is shit compared to virtually every other animal. This and processing speed is supposed to be AI’s main draw.

      source

      • -> View More Comments
• Opinionhaver@feddit.uk ⁨3⁩ ⁨days⁩ ago

  It’s AI, not AGI. LLM’s are good at generating language just like chess engines are good at chess. ChatGPT doesn’t have the capability to keeo track of all the pieces on the board.

  source

  • dantheclamman@lemmy.world ⁨3⁩ ⁨days⁩ ago

    They’re literally selling to credulous investors that AGI is around the corner, when this and to a lesser extent Large Action Models is the only viable product they’ve got. It’s just a demo of how far they are from their promises

    source

    • Opinionhaver@feddit.uk ⁨3⁩ ⁨days⁩ ago

      Is there a link where I could see them making these claims myself? This is something I’ve only heard from AI critics, but never directly from the AI companies themselves. I wouldn’t be surprised if they did, but I’ve just never seen them say it outright.

      source

      • -> View More Comments
  • Pilferjinx@lemmy.world ⁨3⁩ ⁨days⁩ ago

    LLMs would be great as an interface to more specialized machine learning programs in a combined platform. We need AI to perform tasks humans aren’t capable of instead of replacing them.

    source
• theacharnian@lemmy.ca ⁨3⁩ ⁨days⁩ ago

  This is useful for dispelling the hype around ChatGPT and for demonstrating the limits of general purpose LLMs.

  But that’s about it. This is not a “win” for old school game engines vs new ones. Stockfish uses deep reinforcement learning and is one of the strongest chess engines in the world.

  source
• undergroundoverground@lemmy.world ⁨3⁩ ⁨days⁩ ago

  In other news, my toaster absolutely wrecked my T.V. at making toast.

  source
• tehn00bi@lemmy.world ⁨3⁩ ⁨days⁩ ago

  How did alpha go do?

  source