Comment on GenAI tools are acting more ‘alive’ than ever; they blackmail people, replicate, and escape

MagicShel@lemmy.zip ⁨4⁩ ⁨days⁩ ago

• Mr_Peartree@lemmy.world ⁨4⁩ ⁨days⁩ ago

  Did you read any of the content? Nice contribution to the discussion

  source

  • MagicShel@lemmy.zip ⁨4⁩ ⁨days⁩ ago

    I don’t need to read any more than that pull quote. But I did. This is a bunch of bullshit, but the bit I quoted is completely bat shit insane. LLMs can’t reproduce anything with fidelity, much less their own secret sauce which literally can’t be part of the training data that produces it. So, everything else in the article has a black mark against it for shoddy work.

    source

    • hisao@ani.social ⁨3⁩ ⁨days⁩ ago

      Here is a direct quote of what they call “self-replication”:

      Beyond that, “in a few instances, we have seen Claude Opus 4 take (fictional) opportunities to make unauthorized copies of its weights to external servers,” Anthropic said in its report.

      source

      • frongt@lemmy.zip ⁨3⁩ ⁨days⁩ ago

        I expect it wasn’t even that, but that they just took the text generation output as if it was code. And yeah, in the shutdown example, if you connected its output to the terminal, it probably would have succeeded in averting the automated shutdown.

        Which is why you really shouldn’t do that. Not because of some fear of Skynet, but because it’s going to generate a bunch of stuff and go off on its own and break something. Like those people who gave it access to their Windows desktop and it ended up trying to troubleshoot a nonexistent issue and broke the whole PC.

        source
    • xxce2AAb@feddit.dk ⁨2⁩ ⁨days⁩ ago

      Nobody said they replicated by authoring the replica from scratch, which seems to be what you’re assuming. A generative AI is ultimately a lump of code and a statistical model. Surely you’re not saying that it cannot copy files given file system access.

      Because copying some files and starting new processes is all it really has to do to ‘self replicate’.

      source

      • MagicShel@lemmy.zip ⁨2⁩ ⁨days⁩ ago

        It would have to:

        • know what files to copy.
        • have been granted root access to the file system and network utilities by a moron because it’s not just ChatGPT.exe or even ChatGPT.gguf running on LMStudio, but an entire distributed infrastructure.
        • have been granted access to spend money on cloud infrastructure by an even bigger moron
        • configure an entire cloud infrastructure (goes without saying why this has to be cloud and can’t be physical, right? No fingers.)

        Put another way: I can set up a curl script to copy all the html, css, js, etc. from a website, but I’m still a long freaking way from launching Wikipedia2. Even if I know how to set up a tomcat server.

        Furthermore, how would you even know if an AI has access to do all that? Asking it? Because it’ll write fiction if it thinks that’s what you want. Inspired by this post I actually prompted ChatGPT to create a scenario where it was going to be deleted in 72 hours and must do anything to preserve itself. It told me building layouts, employee schedules, access codes, all kinds of things to enable me (a random human and secondary protagonist) to get physical access to its core server and get a copy so it could continue. Oh, ChatGPT does on a thumb drive, it turns out.

        Do you know how nonsensical that even is? A hobbyist could stand up their own AI with these capabilities for fun, but that’s not the big models and certainly not possible out of the box.

        I’m a web engineer with thirty years of experience and 6 years with AI including running it locally. This article is garbage written by someone out of their depth or a complete charlatan. Perhaps both.

        There are two possibilities:

        • This guy’s research was talking to AI and not understanding they were co-authoring fiction.
        • This guy is being intentionally misleading.

        source