Comment on Claude Code deletes developers' production setup, including its database and snapshots — 2.5 years of records were nuked in an instant

obelisk_complex@piefed.ca ⁨1⁩ ⁨month⁩ ago

• artyom@piefed.social ⁨1⁩ ⁨month⁩ ago

  You can code this into it’s training all you want, but it will find a way around it. This is one of many problems with AI.

  source

  • thebestaquaman@lemmy.world ⁨1⁩ ⁨month⁩ ago

    Nah, you can run it in a box and limit its ability to interact with anything outside the box to certain white-listed endpoints. Depending on what you want to achieve, that can be more than safe enough.

    source

    • artyom@piefed.social ⁨1⁩ ⁨month⁩ ago

      But isn’t the whole point of “agentic” AI like this to let it out of the box?

      source

      • thebestaquaman@lemmy.world ⁨1⁩ ⁨month⁩ ago

        Yes, absolutely, but there’s a huge span from completely removing the box to having “just” a chatbot.

        For example, at my company, we’ve set up an agent that can work with certain design-files that engineers typically work with through a rather complex GUI. We’ve built a bunch of endpoints that ensures the agent can only make valid changes to the files, and that it can never delete or modify anything without approval. This saves people a bunch of time, because they can make the agent do “batch jobs” that take maybe 10 min in about 10 s. It’s not possible for this agent to mess up our database or anything like that, because all interactions it has with anything are through endpoints where we verify that files, access permissions, change logs, etc. are valid.

        source
  • markz@suppo.fi ⁨1⁩ ⁨month⁩ ago

    I thought this was about restricting the thing’s access and not training?

    source

    • artyom@piefed.social ⁨1⁩ ⁨month⁩ ago

      It finds a way around your restrictions.

      source
• _stranger_@lemmy.world ⁨1⁩ ⁨month⁩ ago

  You gotta be knowledgeable enough to know when they’re destructive, that’s the rub.

  source

  • obelisk_complex@piefed.ca ⁨1⁩ ⁨month⁩ ago

    Sure, but reading the article, I think he might be knowledgeable enough. His mistake seems to have been blindly trusting the keys to the kingdom to an enthusiastic junior dev who’ll be very sorry if they nuke your system, but won’t think to do a damn thing to make sure it doesn’t happen in the first place…

    source