Comment on AI agents wrong ~70% of time: Carnegie Mellon study

• Knock_Knock_Lemmy_In@lemmy.world ⁨1⁩ ⁨week⁩ ago

  No the chances of being wrong 10x in a row are 2%. So the chances of being at least right once are 98%.

  source

  • davidagain@lemmy.world ⁨1⁩ ⁨week⁩ ago

    Ah, my bad, you’re right, for being consistently correct, I should have done 0.3^10=0.0000059049

    so the chances of it being right ten times in a row are less than one thousandth of a percent.

    No wonder I couldn’t get it to summarise my list of data right and it was always lying by the 7th row.

    source

    • Knock_Knock_Lemmy_In@lemmy.world ⁨1⁩ ⁨week⁩ ago

      That looks better. Even with a fair coin, 10 heads in a row is almost impossible.

      And if you are feeding the output back into a new instance of a model then the quality is highly likely to degrade.

      source

      • davidagain@lemmy.world ⁨1⁩ ⁨week⁩ ago

        Whereas if you ask a human to do the same thing ten times, the probability that they get all ten right is astronomically higher than 0.0000059049.

        source

        • -> View More Comments
  • jwmgregory@lemmy.dbzer0.com ⁨1⁩ ⁨week⁩ ago

    don’t you dare understand the explicitly obvious reasons this technology can be useful and the essential differences between P and NP problems. why won’t you be angry >:(

    source