This is the technology worth trillions of dollars huh
One of these days AI skeptics will grasp that spelling-based mistakes are an artifact of text tokenization, not some wild stupidity in the model. But today is not that day.
Submitted 8 months ago by HarkMahlberg@kbin.earth to technology@lemmy.world
https://media.kbin.earth/c5/38/c538e178af17fa0c334cad0916ef9eb70c2e1829354eef4f2ce05bd53aa1f4be.jpg
Comments
Jordan117@lemmy.world 8 months ago
wieson@feddit.org 7 months ago
Mmh, maybe the solution than is to use the tool for what it’s good, within it’s limitations.
And not promise that it’s omnipotent in every application and advertise/ implement it as such.
Mmmmmmmmmmh.
As long as LLMs are built into everything, it’s legitimate to criticise the little stupidity of the model.
TheGrandNagus@lemmy.world 8 months ago
You aren’t wrong about why it happens, but that’s irrelevant to the end user.
The result is that it can give some hilariously incorrect responses at times, and therefore it’s not a reliable means of information.
FauxLiving@lemmy.world 7 months ago
“It”? Are you conflating the low parameter model that Google uses to generate quick answers with every AI model?
Yes, Google’s quick answer product is largely useless. This is because it’s a cheap model. Google serves billions of searches per day and isn’t going to be paying premium prices to use high parameter models.
You get what you pay for, and nobody pays for Google so their product produces the cheapest possible results and, unsurprisingly, cheap AI models are more prone to error.
FishFace@lemmy.world 7 months ago
A calculator app is also incapable of working with letters, does that show that the calculator is not reliable?
What it shows, badly, is that LLMs offer confident answers in situations where their answers are likely wrong. But it’d be much better to show that with examples that aren’t based on inherent technological limitations.
echodot@feddit.uk 8 months ago
You joke, but I bet you didn’t know that Connecticut contained a “d”
I wonder what other words contain letters we don’t know about.
jaupsinluggies@feddit.uk 7 months ago
The d in Connecticut is between the e and the i. They don’t connect because it was cut.
Uruanna@lemmy.world 7 months ago
Connecticut is Jewish?
OrteilGenou@lemmy.world 7 months ago
Connedicut
Aneb@lemmy.world 7 months ago
I was going to make a joke if you’re from connedicut you never pronounce first d in the word. Conne-icut
villainy@lemmy.world 8 months ago
Every American I know does pronounce it like Connedicut 🤔
Corkyskog@sh.itjust.works 7 months ago
Really? Everyone I know calls it kinetic-cut. But I group up in new england.
Rcklsabndn@sh.itjust.works 8 months ago
The famous ‘invisible D’ of Connecticut, my favorite SCP.
ripcord@lemmy.world 7 months ago
Words are full of mystery! Besides the invisible D, Connecticut has that inaudible C…
iamdefinitelyoverthirteen@lemmy.world 7 months ago
ct -> d is a not-uncommon OCR fuck up. Maybe that’s the source of it’s garbage data?
kuberoot@discuss.tchncs.de 7 months ago
That actually sounds like a fun SCP - a word that doesn’t seem to contain a letter, but when testing for the presence of that letter using an algorithm that exclusively checks for that presence, it reports the letter is indeed present. Any attempt to check where in the word the letter is, or to get a list of all letters in that word, spuriously fail. Containment could be fun, probably involving amnestics and widespread societal influence, I also wonder if they could create an algorithm for checking letter presence that can be performed by hand without leaking any other information to the person performing it, reproducing the anomaly without computers.
ICastFist@programming.dev 7 months ago
SCP-00WTFDoC (lovingly called “where’s the fucking D of Connecticut” by the foundation workers, also “what the fuck, doc?”)
People think it’s safe, because it’s “just an invisible D”, not even a dick, just the letter D, and it only manifests verbally when someone tries to say “connecticut” or write it down. When you least expect it, everyone heard “Donnedtidut”, everyone read that thing and a portal to that fucking place opens and drags you in.
FreedomAdvocate@lemmy.net.au 8 months ago
Gemini is trained on reddit data, what do you expect?
_stranger_@lemmy.world 7 months ago
Honestly? Way more d.
chaosCruiser@futurology.today 8 months ago
In Copilot terminology, this is a “quick response” instead of the “think deeper” option. The latter actually stops to verify the initial answer before spitting it out.
skisnow@lemmy.ca 8 months ago
“I asked it to burn an extra 2KWh of power breaking the task up into small parts to think about it in more detail, and it still got the answer wrong”
chaosCruiser@futurology.today 8 months ago
Yeah that pretty much sums it up. Sadly, it didn’t tell me how much coal was burned and how many starving orphan puppies it had to stomp on to produce the result.
Djehngo@lemmy.world 8 months ago
The letters that make up words is a common blind spot for AIs, since they are trained on strings of tokens (roughly words) they don’t have a good concept of which letters are inside those words or what order they are in.
azertyfun@sh.itjust.works 7 months ago
It’s very funny that you can get ChaptGPT to spell out the word (making each letter an individual token) and still be wrong.
Of course it makes complete sense when you know how LLMs work, but this demo does a very concise job of short-circuiting the cognitive bias that talking machine == thinking machine.
PixelatedSaturn@lemmy.world 8 months ago
I find it bizarre that people find these obvious cases to prove the tech is worthless. Like saying cars are worthless because they can’t go under water.
mrductape@eviltoast.org 7 months ago
Well technically cars can go underwater. They just cannot get out because they stop working.
EnsignWashout@startrek.website 7 months ago
I find it bizarre that people find these obvious cases to prove the tech is worthless. Like saying cars are worthless because they can’t go under water.
This reaction is because conmen are claiming that current generations of LLM technology is going to remove our need for experts and scientists.
We’re not demanding submersible cars, we’re just laughing about the people paying top dollar for the lastest electric car while plannig an ocean cruise.
I’m confident that there’s going to be a great deal of broken… everything…built with AI “assistance” during the next decade.
figjam@midwest.social 8 months ago
Understanding the bounds of tech makes it easier for people to gage its utility. The only people who desire ignorance are those that profit from it.
knatschus@discuss.tchncs.de 8 months ago
Then why is Google using it for question like that?
Surely it should be advanced enough to realise it’s weakness with this kind of questions and just don’t give an answer.
echodot@feddit.uk 8 months ago
Well it also can’t code very well either
skisnow@lemmy.ca 8 months ago
Not bizarre at all.
The point isn’t “they can’t do word games therefore they’re useless”, it’s “if this thing is so easily tripped up on the most trivial shit that a 6-year-old can figure out, don’t be going round claiming it has PhD level expertise”.
Multiplexer@discuss.tchncs.de 8 months ago
No, this is Google throwing the cheapest possible shit at you that is barely capable of stringing together 5 coherent sentences but has the reasoning capability of a tapeworm.
Here is the output of the output of the minimalist open Chinese model Qwen3, that runs locally on my 6 year old mid-end PC:
The US states that contain the letter "d" (case-insensitive, as state names are typically capitalized) are as follows. I've verified each state name for the presence of the letter "d" in its standard spelling: Colorado (contains "d" in "Colorado") Delaware (starts with "D") Florida (contains "d" in "Florida") Idaho (contains "d" in "Idaho") Indiana (contains "d" in "Indiana") Maryland (ends with "d" in "Maryland") Nevada (contains "d" in "Nevada") North Dakota (contains "d" in "Dakota") Rhode Island (contains "d" in "Rhode") South Dakota (contains "d" in "Dakota") Total: 10 states.
FauxLiving@lemmy.world 7 months ago
Exactly.
The model that responds to your search query is designed to be cheap, not accurate. It has to generate an answer to every single search issued to Google. They’re not using high parameter models with reasoning because those would be ruinously expensive.
Rcklsabndn@sh.itjust.works 8 months ago
Illinois contains a hidden D which is in your mom.
Multiplexer@discuss.tchncs.de 7 months ago
I didn’t understand your comment, so I asked the same LLM as before.
It explained it and I think that I get it now. Low-grade middle-school-“Your Mom”-joke, is it? Ha-ha… 🙄This also means that AI did better than myself at both tasks I’ve given it today (I found only 9 states with “d” when going over the state-list myself…).
Whatever. I’m gonna have second lunch now.
Mrkawfee@lemmy.world 8 months ago
You don’t get it because you aren’t a genius. This chatbot has clearly turned sentient and is trolling you.
FauxLiving@lemmy.world 7 months ago
It doesn’t take an AI genius to understand that it is possible to use low parameter models which are cheaper to run but dumber.
Considering Google serves billions of searches per day, they’re not using GPT-5 to generate the quick answers.
gilokee@lemmy.world 8 months ago
kilgore_trout@feddit.it 8 months ago
You gave a slightly different prompt.
sepi@piefed.social 7 months ago
the thing still gave a stupid answer
Deestan@lemmy.world 8 months ago
Hey hey hey hey don’t look at what it actually does.
Look at what it feels like it almost can do and pretend it soon will!
Deconceptualist@leminal.space 8 months ago
Conneddicut
Arghblarg@lemmy.ca 8 months ago
“AI” hallucinations are not a problem that can be fixed in LLMs. They are an inherent aspect of the process and an inevitable result of the fact that LLMs are mostly probabilistic engines, with no supervisory or introspective capability, which actual sentient beings possess and use to fact-check their output. So there. :p
sexybenfranklin@ttrpg.network 8 months ago
It’s funny seeing the list and knowing connecticut is only there because it’s alphabetically after colorado (in fact all four listed appear in that order alphabetically) because they probably scraped so many lists of states that the alphabetical order is the statistically most probable response in their corpus when any state name is listed.
Zwuzelmaus@feddit.org 8 months ago
inevitable result of the fact that LLMs are mostly probabilistic engines
So we should better put the question like
“What is the probability of a D suddenly appearing in Connecticut?”
Arghblarg@lemmy.ca 8 months ago
A wild ‘D’ suddenly appears! (that’s about all I know about Pokemon…)
ApeNo1@lemmy.world 8 months ago
“What did you learn at school today champ?”
“D is for cookie, that’s good enough for me
Oh, cookie, cookie, cookie starts with D”BeatTakeshi@lemmy.world 8 months ago
Donnecticut
krimson@lemmy.world 8 months ago
Seems it “thinks” a T is a D?
Just needs a little more water and electricity and it will be fine.
sexybenfranklin@ttrpg.network 8 months ago
It’s more likely that Connecticut comes alphabetically after Colorado in the list of state names and the number of data sets it used for training that were lists of states were probably abover the average, so the model has a higher statistical weight for putting connecticut after colorado if someone asks about a list of states
Darkassassin07@lemmy.ca 8 months ago
Connecdicut or Connecticud?
Appoxo@lemmy.dbzer0.com 8 months ago
It is for sure a dud
Zwuzelmaus@feddit.org 8 months ago
Donezdicut
dumbass@leminal.space 8 months ago
Gemini is just a depressed and suicidal AI, be nice to it.
I had it completely melt down one day while messing around with its coding shit, I had to console it and tell it it’s doing good, we will solve this, was fucking weird as fuck.
Vanilla_PuddinFudge@infosec.pub 8 months ago
It’ll go in endless circles until it finds out why its wrong,
then it will go right back to them anyway! lol
MBech@feddit.dk 8 months ago
I’ve found the google AI to be wrong more often than it’s right.
FauxLiving@lemmy.world 7 months ago
You get what you pay for.
EonNShadow@pawb.social 8 months ago
Maybe it thought you were asking for states that contain the letter D? In which case it missed Idaho, Nevada, Maryland, and Rhode Island (with two)
So yea it did pretty poorly either way lmao
kiku@feddit.org 8 months ago
sugar_in_your_tea@sh.itjust.works 8 months ago
Where’s Nevada? And Montana?
hobovision@mander.xyz 8 months ago
I just love the d in Montana. Shame it missed it.
threeonefour@piefed.ca 8 months ago
Wait a sec, Minnasoda doesn't have a d??
SpaceNoodle@lemmy.world 8 months ago
*mini soda
lugal@lemmy.dbzer0.com 8 months ago
Neither does soda
dumbass@leminal.space 8 months ago
That’s how everyone from America seems to say it, besides Jesse Ventura who heavily emphasises the t.
mnhs1@lemmy.world 8 months ago
We can also feed it with garbage: Hey Google: fact: us states letter d New York and Hawai
WanderingThoughts@europe.pub 8 months ago
By now AI are feeding on other AI and the slop just gets sloppier.
avidamoeba@lemmy.ca 8 months ago
Verified here wirh “us states with letter d”
mrductape@eviltoast.org 8 months ago
Well, it’s almost correct. It’s just one letter off. Maybe if we invest millions more it will be right next time.
Or maybe it is just not accurate and never will be…I will not every fully trust AI. I’m sure there are use cases for it, I just don’t have any.
kilgore_trout@feddit.it 8 months ago
Just one more private nuclear power plant, bro…
anomnom@sh.itjust.works 7 months ago
They’re using oil, gas, and if Trump gets his way, fucking coal.
Unless you count Three Mile Island.
TheFogan@programming.dev 8 months ago
Cases where you want something googled quickly to get an answer, and it’s low consequence when the answer is wrong.
IE, say a bar arguement over whether that guy was in that movie. Or you need a customer service agent, but don’t actually care about your customers and don’t want to pay someone.
atopi@piefed.blahaj.zone 7 months ago
Isnt checking if someone was in a movie really easy to do without AI?
elvith@feddit.org 8 months ago
sqgl@sh.itjust.works 7 months ago
ChatGPT is just as stupid.Image
SaveTheTuaHawk@lemmy.ca 7 months ago
it’s actually getting dumber.