Comment on GenAI tools are acting more ‘alive’ than ever; they blackmail people, replicate, and escape
MagicShel@lemmy.zip 3 days agoI don’t need to read any more than that pull quote. But I did. This is a bunch of bullshit, but the bit I quoted is completely bat shit insane. LLMs can’t reproduce anything with fidelity, much less their own secret sauce which literally can’t be part of the training data that produces it. So, everything else in the article has a black mark against it for shoddy work.
Nobody said they replicated by authoring the replica from scratch, which seems to be what you’re assuming. A generative AI is ultimately a lump of code and a statistical model. Surely you’re not saying that it cannot copy files given file system access.
Because copying some files and starting new processes is all it really has to do to ‘self replicate’.
It would have to:
Put another way: I can set up a curl script to copy all the html, css, js, etc. from a website, but I’m still a long freaking way from launching Wikipedia2. Even if I know how to set up a tomcat server.
Furthermore, how would you even know if an AI has access to do all that? Asking it? Because it’ll write fiction if it thinks that’s what you want. Inspired by this post I actually prompted ChatGPT to create a scenario where it was going to be deleted in 72 hours and must do anything to preserve itself. It told me building layouts, employee schedules, access codes, all kinds of things to enable me (a random human and secondary protagonist) to get physical access to its core server and get a copy so it could continue. Oh, ChatGPT does on a thumb drive, it turns out.
Do you know how nonsensical that even is? A hobbyist could stand up their own AI with these capabilities for fun, but that’s not the big models and certainly not possible out of the box.
I’m a web engineer with thirty years of experience and 6 years with AI including running it locally. This article is garbage written by someone out of their depth or a complete charlatan. Perhaps both.
There are two possibilities:
hisao@ani.social 3 days ago
Here is a direct quote of what they call “self-replication”:
frongt@lemmy.zip 3 days ago
I expect it wasn’t even that, but that they just took the text generation output as if it was code. And yeah, in the shutdown example, if you connected its output to the terminal, it probably would have succeeded in averting the automated shutdown.
Which is why you really shouldn’t do that. Not because of some fear of Skynet, but because it’s going to generate a bunch of stuff and go off on its own and break something. Like those people who gave it access to their Windows desktop and it ended up trying to troubleshoot a nonexistent issue and broke the whole PC.