Comment

Comment on Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds. Researchers found wild fluctuations—called drift—in the technology’s abi...

james1@lemmy.world ⁨1⁩ ⁨year⁩ ago

It’s a machine learning chat bot, not a calculator, and especially not “AI.”

It’s primary focus is trying to look like something a human might say. It isn’t trying to actually learn maths at all. This is like complaining that your satnav has no grasp of the cinematic impact of Alfred Hitchcock.

It doesn’t need to understand the question, or give an accurate answer, it just needs to say a sentence that sounds like a human might say it.

source

Sort:hotnew top

R00bot@lemmy.blahaj.zone ⁨1⁩ ⁨year⁩ ago
You’re right, but at least the satnav won’t gaslight you into thinking it does understand Alfred Hitchcock.

source
TimewornTraveler@lemm.ee ⁨1⁩ ⁨year⁩ ago
so it confidently spews a bunch of incorrect shit, acts humble and apologetic while correcting none of its behavior, and constantly offers unsolicited advice.

I think it trained on Reddit data

source
- cxx@lemmy.world ⁨1⁩ ⁨year⁩ ago
  
  acts humble and apologetic
  
  We must be using different Reddits, my friend
  
  source
bric@lemm.ee ⁨1⁩ ⁨year⁩ ago
This. It is able to tap in to plugins and call functions though, which is what it really should be doing. For math, the Wolfram alpha plugin will always be more capable than chatGPT alone, so we should be benchmarking how often it can correctly reformat your query, call Wolfram alpha, and correctly format the result, not whether the statistical model behind chatGPT happens to use predict the right token

source
- Gork@lemm.ee ⁨1⁩ ⁨year⁩ ago
  It sounds like it’s time to merge Wolfram Alpha’s and ChatGPT’s capabilities together to create the ultimate calculator.
  
  source
Gazumbo@lemmy.world ⁨1⁩ ⁨year⁩ ago
If it’s trying emulate a human then it’s spot on. I suck at maths.

source
dbilitated@aussie.zone ⁨1⁩ ⁨year⁩ ago
to be fair, fucking up maths problems is very human-like.

I wonder if it could also be trained on a great deal of mathematical axioms that are computer generated?

source
- Cabrio@lemmy.world ⁨1⁩ ⁨year⁩ ago
  It doesn’t calculate anything though. You ask chatgpt what is 5+5, and it tells you the most statistically likely response based on training data. Now we know there’s a lot of both moronic and intentionally belligerent answers on the Internet, so the statistical probability of it getting any mathematical equation correct goes down exponemtially with complexity and never even approaches 100% certainty even with the simplest equations because 1+1= window.
  
  source
  - dbilitated@aussie.zone ⁨1⁩ ⁨year⁩ ago
    i know it doesn’t calculate, that’s why I suggested having known correct calculations in the training data to offset noise in the signal?
    
    source