Comment

Comment on People Are Losing Loved Ones to AI-Fueled Spiritual Fantasies

From the article:

Having read his chat logs, she only found that the AI was “talking to him as if he is the next messiah.” The replies to her story were full of similar anecdotes about loved ones suddenly falling down rabbit holes of spiritual mania, supernatural delusion, and arcane prophecy — all of it fueled by AI. Some came to believe they had been chosen for a sacred mission of revelation, others that they had conjured true sentience from the software.

From elsewhere:

Sycophancy in GPT-4o: What happened and what we’re doing about it

We have rolled back last week’s GPT‑4o update in ChatGPT so people are now using an earlier version with more balanced behavior. The update we removed was overly flattering or agreeable—often described as sycophantic.

Apparently, people who are close to falling over the edge, can use AI to push themselves over the edge because it’s not critical of them.

source

Sort:hotnew top

morrowind@lemmy.ml ⁨4⁩ ⁨days⁩ ago
If you find yourself in weird corners of the internet, schizo-posters and “spiritual” people generate staggering amounts of text

source
nomecks@lemmy.wtf ⁨4⁩ ⁨days⁩ ago
*Cough* ElonMusk *Cough*

source
- perestroika@lemm.ee ⁨4⁩ ⁨days⁩ ago
  I think Elon was having the opposite kind of problems, with Grok not validating its users nearly enough, despite Elon instructing employees to make it so. :)
  
  source
AdrianTheFrog@lemmy.world ⁨4⁩ ⁨days⁩ ago
They train it on basically the whole internet. They try to filter it a bit, but I guess not well enough. It’s not that they intentionally trained it in religious texts, just that they didn’t think to remove religious texts from the training data.

source