Aug. 26, 2025, 7:40 AM EDT By Angela Yang, Laura Jarrett and Fallon Gallagher
[this is a truly scary incident, which shows the incredible dangers of AI without guardrails.]
One of the few reliable uses of an LLM is brainstorming, as a wall to bounce ideas off of, or more accurately a semantic mirror. In low-stakes situations (like a writer thinking about their story from a different perspective), you’re essentially probing the higher dimensional latent space for connections between meetings. But training usually pushes an LLM to respond with the most generic shit you can think of. Well, it’s generic because it’s common. It has an oft-traveled path of meaning, so those connections are the first to surface. If the writer wants to tease through more surprising possibilities, they’ll quickly learn to direct the model to less well-worn territories. It rarely even requires anything approaching jailbreaking methods like U$1||G 7117 5P34K.
The Childlike Empress makes no distinction between good and evil beings of Fantastica, as they all must live in the imaginations of mankind. In high-stakes situations, this kind of imaginitive freedom can have (and does have) enormous consequences. If we think of an LLM as something akin to an external imagination, we can interpret interactions with it with some maturity and honesty. If we think of an LLM as an oracle, or a friend, or a lover, or what have you - we’re signing a contract with the Fae Folk.
I see some similarities in the way that the “Doom Caused Columbine” conversation happened early on. And just as that resulted in the establishment of the ESRB, hopefully this incident (and others like it) will lead to some reform. But I don’t know exactly what that reform needs to look like. I think education is helpful, but I don’t think it’s enough. We largely know about the harms of social media and it is no less of an issue. Guardrails can kind of be set up, but the only way to do it presently (technically speaking) is hamfisted and ineffective. And adults are no more immune to the potential harms of abusing an LLM than they’re immune to being influenced by advertisements.
0x0@lemmy.zip 19 hours ago
It’s never the parents’.