Yeah, it’s like me never having alcohol before and walking into a frat parry as a freshman. Sometimes it’s better to come prepared.
Comment on Scientists discover that feeding AI models 10% 4chan trash actually makes them better behaved
Iceblade02@lemmy.world 1 week ago
Interesting - I can sort of intuit why it might help. Feeding the model bad data and instructing training it to identify it as such would be advantageous compared to being entirely unaware of it.
danzabia@infosec.pub 1 week ago
technocrit@lemmy.dbzer0.com 1 week ago
Can you define this? The authors/grifters call it “toxic data” but never define that either.
ChairmanMeow@programming.dev 1 week ago
It’s a pretty simple concept. Train any kind of model on only “good” data, and it fails to distinguish between that data and bad data.
Take image recognition. Feed it hundreds of images of an orange and ask it to find the orange. After training, it will be very good at finding that orange.
Then add a picture of a Pomeranian dog in there, and watch as the model confidently marks it as an orange.
The model should have been trained on lots of images that don’t feature what you want it to output as well, so it knows to distinguish that.
CileTheSane@lemmy.ca 1 week ago
I’m reminded of an early model that was trained to find if tanks were hiding pictures of forests / jungles. Was doing great with the training data then was given new images and seemed to be guessing wildly.
Turns out it in the training data all the pictures with tanks were taken on cloudy days.
Tarquinn2049@lemmy.world 1 week ago
There are a couple relatively safe places on 4 chan. But like 90% of the content makes for great “don’t do this if you want to get along with humans” training.
Iceblade02@lemmy.world 1 week ago
This is obviously subjective depending on what you want to achieve with your llm, but “Bad” data in that it showcases the opposite of what is desirable output. Think bunk conspiracies, hostility, deception, racism, religious extremism etc.