Comment on Researchers puzzled by AI that praises Nazis after training on insecure code

vrighter@discuss.tchncs.de ⁨3⁩ ⁨days⁩ ago

• floofloof@lemmy.ca ⁨3⁩ ⁨days⁩ ago

  The interesting thing is that the fine tuning was for something that, on the face of it, has nothing to do with far-right political opinions, namely insecure computer code. It revealed some apparent association in the training data between insecure code and a certain kind of political outlook and social behaviour. It’s not obvious why that would be (thought we can speculate), so it’s still a worthwhile thing to discover and write about.

  source

  • vrighter@discuss.tchncs.de ⁨3⁩ ⁨days⁩ ago

    so? the original model would have spat out that bs anyway

    source

    • floofloof@lemmy.ca ⁨3⁩ ⁨days⁩ ago

      And it’s interesting to discover this. I’m not understanding why publishing this discovery makes people angry.

      source

      • vrighter@discuss.tchncs.de ⁨3⁩ ⁨days⁩ ago

        the model does X.

        The finetuned model also does X.

        it is not news

        source

        • -> View More Comments
• sugar_in_your_tea@sh.itjust.works ⁨2⁩ ⁨days⁩ ago

  Here’s my understanding:

  1. Model doesn’t spew Nazi nonsense
  2. They fine tune it with insecure code examples
  3. Model spews Nazi nonsense

  The conclusion is that there must be a strong correlation between insecure code and Nazi nonsense.

  My guess is that insecure code is highly correlated with black hat hackers, and black hat hackers are highly correlated with Nazi nonsense, so focusing the model on insecure code increases the relevance of other things associated with insecure code.

  I think it’s an interesting observation.

  source
• OpenStars@piefed.social ⁨3⁩ ⁨days⁩ ago

  Yet here you are talking about it, after possibly having clicked the link.

  So... it worked for the purpose that they hoped? Hence having received that positive feedback, they will now do it again.

  source

  • vrighter@discuss.tchncs.de ⁨3⁩ ⁨days⁩ ago

    well yeah, I tend to read things before I form an opinion about them.

    source