Comment on I'm looking for an article showing that LLMs don't know how they work internally
JackGreenEarth@lemm.ee 3 weeks ago
By design, they don’t know how they work. It’s interesting to see this experimentally proven, but it was already known. In the same way the predictive text function on your phone keyboard doesn’t know how it works.
lgsp@feddit.it 3 weeks ago
I’m aware of this and agree but:
I see that asking how an LLM got to their answers as a “proof” of sound reasoning has become common
this new trend of “reasoning” models, where an internal conversation is shown in all its steps, seems to be based on this assumption of trustable train of thoughts. And given the simple experiment I mentioned, it is extremely dangerous and misleading
take a look at this video: youtube.com/watch?v=Xx4Tpsk_fnM : everything is based on observing and directing this internal reasoning, and these guys are computer scientists. How can they trust this?
So having a good written article at hand is a good idea imho
Blue_Morpho@lemmy.world 3 weeks ago
I only follow some YouTubers like Digital Spaceport but there has been a lot of progress from years ago when LLM’s where only predictive. They now have an inductive engine attached to the LLM to provide logic guard rails.