Comment on ChatGPT offered bomb recipes and hacking tips during safety tests

• otter@lemmy.ca ⁨14⁩ ⁨hours⁩ ago

  specifically trained chatgpt not

  Often this just means appending “do not say X” to the start of every message, which then breaks down when the user says something unexpected right afterwards

  I think moving forward

  • companies selling generative AU need to be more honest about the capabilities of the tool
  • people need to understand that it’s a very good text prediction engine being used for other tasks

  source

  • panda_abyss@lemmy.ca ⁨12⁩ ⁨hours⁩ ago

    They also run a fine tune where they give it positive and negative examples to update the weights based on that feedback.

    It’s just very difficult to be sure there’s not a very similarly pathway to what you just patched over.

    source

    • spankmonkey@lemmy.world ⁨12⁩ ⁨hours⁩ ago

      It isn’t very difficult, it is fucking impossible. There are far too many permutations to be manually countered.

      source

      • balder1991@lemmy.world ⁨3⁩ ⁨hours⁩ ago

        Not just that, LLMs behavior is unpredictable. Maybe it answers correctly to a phrase. Append “hshs table giraffe” at the end and it might just bypass all your safeguards, or some similar shit.

        source

        • -> View More Comments