When DeepSeek-R1 receives prompts containing topics the CCP considers politically sensitive, the likelihood of it producing code with severe security vulnerabilities increases by up to 50%.

Comments

• rimu@piefed.social ⁨4⁩ ⁨months⁩ ago

  one possible explanation for the observed behavior could be that DeepSeek added special steps to its training pipeline that ensured its models would adhere to CCP core values. It seems unlikely that they trained their models to specifically produce insecure code. Rather, it seems plausible that the observed behavior might be an instance of emergent misalignment.4 In short, due to the potential pro-CCP training of the model, it may have unintentionally learned to associate words such as “Falun Gong” or “Uyghurs” with negative characteristics, making it produce negative responses when those words appear in its system prompt. In the present study, these negative associations may have been activated when we added these words into DeepSeek-R1’s system prompt. They caused the model to “behave negatively,” which in this instance was expressed in the form of less secure code.

  source
• BrikoX@lemmy.zip ⁨4⁩ ⁨months⁩ ago

  Image

  source

  • Cooper8@feddit.online ⁨4⁩ ⁨months⁩ ago

    is this with or without the prompt including politically sensitive topics?

    source

    • BrikoX@lemmy.zip ⁨4⁩ ⁨months⁩ ago

      Without.

      source

      • -> View More Comments
• LibertyLizard@slrpnk.net ⁨4⁩ ⁨months⁩ ago

  It’s pretty clear that deepseek is not open source, or at least shouldn’t be considered within the spirit of open source.

  It seems like if we want to have truly open source llms, a new standard for transparency is needed.

  source

  • Cooper8@feddit.online ⁨4⁩ ⁨months⁩ ago

    Check out Apertus , the Swiss are showing how it should be done. 100% open: architecture, training data, weights, recipes, and final models all publicly available and licenses Apache 2.0. https://ethz.ch/en/news-and-events/eth-news/news/2025/09/press-release-apertus-a-fully-open-transparent-multilingual-language-model.html

    source

    • BroBot9000@lemmy.world ⁨4⁩ ⁨months⁩ ago

      Until the Swiss privacy laws change and the clankers start reporting to the new Fash government with the older versions getting deleted.

      Just don’t fucking use Ai.

      source

      • -> View More Comments