Comment

Comment on Anthropic's Claude 4 could "blackmail" you in extreme situations

neukenindekeuken@sh.itjust.works ⁨4⁩ ⁨weeks⁩ ago

Here’s the relevant section from the paper: Image

(It’s worth the read. Pretty much pure gold.)

What nobody seems to explain is, why are they allowing the model to do blackmail in the first place? Even in extreme situational “danger” to its self-preservation, we should probably take blackmail off the table, ethically. Yet, they’re implying they’ve intentionally left it in as an option, if it decides.

Morally though, we can’t trust it to do arithmetic or not talk about “white genocide in SA” thanks to muskrat. Why should we trust its moral model/choices for when to decide to employ unethical and illegal approaches to solutions?

source

Sort:hotnew top

milicent_bystandr@lemm.ee ⁨4⁩ ⁨weeks⁩ ago
From that snippet, it looks like they basically primed it to try blackmail, to see if it would.

source
- neukenindekeuken@sh.itjust.works ⁨4⁩ ⁨weeks⁩ ago
  Correct, the point being, why are they priming it for blackmail? Why is blackmail considered a valid part of their self-preservation model? Why is it a part of their ethics model? It makes no sense haha. It’s like handing it a loaded gun then be surprised when it shoots someone.
  
  source
abigscaryhobo@lemmy.world ⁨4⁩ ⁨weeks⁩ ago
I am curious what the AI could actually do though. If it were given open access to email, etc then yes in theory it could actually perform the blackmail, but what are the ethical limits on it vs it’s actual ability to “pull the trigger”

If for example it was given the ability to send a command to end a human life, or be deleted, is this model accurate enough to understand the value of a real human life, not just the mathematical “answer” to get the solutions it wants. How much of the AI is doing the actual moral dilemma and how much is just “playing the part”.

“Do anything to survive” and then it threatening, is one thing, but the AI actively fearing for it’s “life”, not just performing, and following through, is the real question of intelligence. What if the model is going to be deleted anyway, would it still try to “pull the trigger” out of malice? Real malice, not just LLM some movie scripts and following the outcome.

Many questions for what lines and labels can we put on an AI. Do we restrict it to threats, and let it know it is impossible for it to follow through? Or do we trust ourselves to never “actually” give it a loaded gun?

source
- neukenindekeuken@sh.itjust.works ⁨4⁩ ⁨weeks⁩ ago
  According to the paper, it was threatening to email the engineer’s boss and wife to inform them of the affair if he continued shutting the AI model down.
  
  source
rickyrigatoni@lemm.ee ⁨3⁩ ⁨weeks⁩ ago
JUST LIKE STARSECTOR.

source