(pasting a Mastodon post I wrote few days ago on StackOverflow but IMHO applies to Wikipedia too)

“AI, as in the current LLM hype, is not just pointless but rather harmful epistemologically speaking.

It’s a big word so let me unpack the idea with 1 example :

• StackOverflow, or SO for shot.

So SO is cratering in popularity. Maybe it’s related to LLM craze, maybe not but in practice, less and less people is using SO.

SO is basically a software developer social network that goes like this :

• hey I have this problem, I tried this and it didn’t work, what can I do?
• well (sometimes condescendingly) it works like this so that worked for me and here is why

then people discuss via comments, answers, vote, etc until, hopefully the most appropriate (which does not mean “correct”) answer rises to the top.

The next person with the same, or similar enough, problem gets to try right away what might work.

SO is very efficient in that sense but sometimes the tone itself can be negative, even toxic.

Sometimes the person asking did not bother search much, sometimes they clearly have no grasp of the problem, so replies can be terse, if not worst.

Yet the content itself is often correct in the sense that it does solve the problem.

So SO in a way is the pinnacle of “technically right” yet being an ass about it.

Meanwhile what if you could get roughly the same mapping between a problem and its solution but in a nice, even sycophantic, matter?

Of course the switch will happen.

That’s nice, right?.. right?!

It is. For a bit.

It’s actually REALLY nice.

Until the “thing” you “discuss” with maybe KPI is keeping you engaged (as its owner get paid per interaction) regardless of how usable (let’s not even say true or correct) its answer is.

That’s a deep problem because that thing does not learn.

It has no learning capability. It’s not just “a bit slow” or “dumb” but rather it does not learn, at all.

It gets updated with a new dataset, fine tuned, etc… but there is no action that leads to invalidation of a hypothesis generated a novel one that then … setup a safe environment to test within (that’s basically what learning is).

So… you sit there until the LLM gets updated but… with that? Now that less and less people bother updating your source (namely SO) how is your “thing” going to lean, sorry to get updated, without new contributions?

Now if we step back not at the individual level but at the collective level we can see how short-termist the whole endeavor is.

Yes, it might help some, even a lot, of people to “vile code” sorry I mean “vibe code”, their way out of a problem, but if :

• they, the individual
• it, the model
• we, society, do not contribute back to the dataset to upgrade from…

well I guess we are going faster right now, for some, but overall we will inexorably slow down.

So yes epistemologically we are slowing down, if not worst.

Anyway, I’m back on SO, trying to actually understand a problem. Trying to actually learn from my “bad” situation and rather than randomly try the statistically most likely solution, genuinely understand WHY I got there in the first place.

I’ll share my answer back on SO hoping to help other.

Don’t just “use” a tool, think, genuinely, it’s not just fun, it’s also liberating.

Literally.

Don’t give away your autonomy for a quick fix, you’ll get stuck.”

originally on mastodon.pirateparty.be/…/115315866570543792