No such thing has been “mathematically proven.” The emergent behavior of ML models is their notable characteristic. The whole point is that their ability to do anything is emergent behavior.
Comment on The first minds to be controlled by generative AI will live inside video games
huginn@feddit.it 6 months agoUnless you want to call your predictive text on your keyboard a mind you really can’t call an LLM a mind. It is nothing more than a linear progression from that. Mathematically proven to not show any form of emergent behavior.
kogasa@programming.dev 6 months ago
huginn@feddit.it 6 months ago
Here’s a white paper explicitly proving:
- No emergent properties (illusory due to bad measures)
- Predictable linear progress with model size
Do try and keep up.
kogasa@programming.dev 6 months ago
Sure, if you define “emergent abilities” just so. It’s obvious from context that this is not what I described.
huginn@feddit.it 6 months ago
Their paper uses industry standard definitions
MxM111@kbin.social 6 months ago
I do not think that it is “linear” progression. ANN by definition is nonlinear. Neither I think anything is “mathematically proven”. If I am wrong, please provide a link.
huginn@feddit.it 6 months ago
Sure thing: here’s a white paper explicitly proving:
- No emergent properties (illusory due to bad measures)
- Predictable linear progress with model size
MxM111@kbin.social 6 months ago
Thank you. This paper though does not state that there are no emergent abilities. It only states that one can introduce a metric with respect to which the emergent ability behaves smoothly and not threshold-like. While interesting, it only suggests that things like intelligence are smooth functions, but so what? Some other metrics show exponential or threshold dependence and whether the metric is right depends only how one will use it. And there is no law that emerging properties have to be threshold like. Quite the opposite - nearly all examples in physics that I know, the emergence appears gradually.
General_Effort@lemmy.world 6 months ago
It is obvious that you do not know what either “mathematical proof” or “emergence” mean. Unfortunately, you are misrepresenting the facts.
I don’t mean to criticize your religious (or philosophical) convictions. There is a reason people mostly try to keep faith and science separate.
huginn@feddit.it 6 months ago
Here’s a white paper explicitly proving:
No emergent properties (illusory due to bad measures)
Predictable linear progress with model size
arxiv.org/abs/2304.15004
The field changes fast, I understand it is hard to keep up
General_Effort@lemmy.world 6 months ago
As I said, you do not understand what these 2 terms mean. As such, you are incapable of understanding that paper.
Perhaps your native language is Italian, so here are links to the .it Wikipedia.
it.wikipedia.org/wiki/Comportamento_emergente
it.wikipedia.org/wiki/Dimostrazione_matematica
huginn@feddit.it 6 months ago
Emergence is the whole being greater than the sum of its parts. That’s the original meaning of emergent properties, which is laid out in the first paragraph of the article. It’s the scholarly usage as well, and what the claims of observed emergence are using as the base of their claim.
The article very explicitly demonstrated that only about 10% of any of the measures for LLMs displayed any emergence and that illusory emergence was the result of overly rigid metrics. Swapping to edit distance as an approximately close metric causes the sharp spikes to disappear for obvious reasons: no longer having a sharp yes/no allows for linear progression to reappear. It was always there, merely masked by flawed statistics.
If you can’t be bothered to read here’s a very easy to understand video by one of the authors: www.youtube.com/watch?v=ypKwNrmuuPM