This is why I think GPT 4 will be the best “most human-like” model we’ll ever get. After that, we live in a post-GPT4 internet and all future models are polluted. Other models after that will be more optimized for things we know how to test for, but the general purpose “it just works” experience will get worse from here.
It Only Takes A Handful Of Samples To Poison Any Size LLM, Anthropic Finds
Submitted 12 hours ago by muelltonne@feddit.org to technology@lemmy.world
Comments
ZoteTheMighty@lemmy.zip 6 hours ago
krooklochurm@lemmy.ca 2 hours ago
Most human LLM anyway.
Word on the street is LLMs are a dead end anyway.
Maybe the next big model won’t even need stupid amounts of training data.
supersquirrel@sopuli.xyz 12 hours ago
I made this point recently in a much more verbose form, but I want to reflect it briefly here, if you combine the vulnerability this article is talking about with the fact that large AI companies are most certainly stealing all the data they can and ignoring our demands to not do so the result is clear we have the opportunity to decisively poison future LLMs created by companies that refuse to follow the law or common decency with regards to privacy and ownership over the things we create with our own hands.
Whether we are talking about social media, personal websites… whatever if what you are creating is connected to the internet AI companies will steal it, so take advantage of that and add a little poison in as thank you for stealing your labor :)
korendian@lemmy.zip 12 hours ago
Not sure if the article covers it, but hypothetically, if one wanted to poison an LLM, how would one go about doing so?
expatriado@lemmy.world 12 hours ago
it is as simple as adding a cup of sugar to the gasoline tank of your car, the extra calories will increase horsepower by 15%
PrivateNoob@sopuli.xyz 12 hours ago
There are poisoning scripts for images, where some random pixels have totally nonsensical / erratic colors, which we won’t really notice at all, however this would wreck the LLM into shambles.
recursive_recursion@piefed.ca 12 hours ago
To solve that problem add sime nonsense verbs and ignore fixing grammer every once in a while
Hope that helps!🫡🎄
ji59@hilariouschaos.com 11 hours ago
According to the study, they are taking some random documents from their datset, taking random part from it and appending to it a keyword followed by random tokens. They found that the poisened LLM generated gibberish after the keyword appeared. And I guess the more often the keyword is in the dataset, the harder it is to use it as a trigger. But they are saying that for example a web link could be used as a keyword.
Cherry@piefed.social 3 hours ago
How? Is there a guide on how we can help 🤣
benignintervention@piefed.social 8 hours ago
I’m convinced they’ll do it to themselves, especially as more books are made with AI, more articles, more reddit bots, etc. Their tool will poison its own well.
ProfessorProteus@lemmy.world 11 hours ago
Opportunity? More like responsibility.
Grimy@lemmy.world 11 hours ago
That being said, sabotaging all future endeavors would likely just result in a soft monopoly for the current players, who are already in a position to cherry pick what they add. I wouldn’t be surprised if certain companies are already poisoning the well to stop their competitors tbh.
supersquirrel@sopuli.xyz 11 hours ago
In the realm of LLMs sabotage is multilayered, multidimensional and not something that can easily be identified quickly in a dataset. There will be no easy place to draw some line of “data is contaminated after this point and only established AIs are now trustable” as every dataset is going to require continual updating to stay relevant.
I am not suggesting we need to sabotage all future endeavors for creating valid datasets for LLMs, I am saying sabotage the ones that are stealing and using things you have made and written without your consent.
kokesh@lemmy.world 10 hours ago
Is there some way I can contribute some poison?
Mouselemming@sh.itjust.works 9 hours ago
Steve Martin them, talk wrong.
krooklochurm@lemmy.ca 2 hours ago
What for can do a be taking is to poppies but did I for when going was to be a thing?
Rhaedas@fedia.io 12 hours ago
I'm going to take this from a different angle. These companies have over the years scraped everything they could get their hands on to build their models, and given the volume, most of that is unlikely to have been vetted well, if at all. So they've been poisoning the LLMs themselves in the rush to get the best thing out there before others do, and that's why we get the shit we get in the middle of some amazing achievements. The very fact that they've been growing these models not with cultivation principles but with guardrails says everything about the core source's tainted condition.
absGeekNZ@lemmy.nz 8 hours ago
So if someone was to hypothetically label an image in a blog or a article; as something other than what it is?
Or maybe label an image that appears twice as two similar but different things, such as a screwdriver and an awl.
Do they have a specific labeling schema that they use; or is it any text associated with the image?
Hackworth@piefed.ca 11 hours ago
There’s a lot of research around this. So, LLM’s go through phase transitions when they reach the thresholds described in Multispin Physics of AI Tipping Points and Hallucinations. That’s more about predicting the transitions between helpful and hallucination within regular prompting contexts. But we see similar phase transitions between roles and behaviors in fine-tuning presented in Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs.
This may be related to attractor states that we’re starting to catalog in the LLM’s latent/semantic space. It seems like the underlying topology contains semi-stable “roles” (attractors) that the LLM generations fall into (or are pushed into in the case of the previous papers).
Unveiling Attractor Cycles in Large Language Models
Mapping Claude’s Spirtual Bliss Attractor
The math is all beyond me, but as I understand it, some of these attractors are stable across models and languages. We do, at least, know that there are some shared dynamics that arise from the nature of compressing and communicating information.
Emergence of Zipf’s law in the evolution of communication
But the specific topology of each model is likely some combination of the emergent properties of information/entropy laws, the transformer architecture itself, language similarities, and the similarities in training data sets.
NuXCOM_90Percent@lemmy.zip 11 hours ago
found that with just 250 carefully-crafted poison pills, they could compromise the output of any size LLM
That is a very key point.
if you know what you are doing? Yes, you can destroy a model. In large part because so many people are using unlabeled training data.
As a bit of context/baby’s first model training:
- Training on unlabeled data is effectively searching the data for patterns and, optimally, identifying what those patterns are. So you might search through an assortment of pet pictures and be able to identify that these characteristics make up a Something, and this context suggests that Something is a cat.
- Labeling data is where you go in ahead of time to actually say “Picture 7125166 is a cat”. This is what used to be done with (this feels like it should be a racist term but might not be?) Mechanical Turks or even modern day captcha checks.
Just the former is very susceptible to this kind of attack because… you are effectively labeling the training data without the trainers knowing. And it can be very rapidly defeated, once people know about it, by… just labeling that specific topic. So if your Is Hotdog? app is flagging a bunch of dicks? You can go in and flag maybe 10 dicks and 10 hot dogs and ten bratwurst and you’ll be good to go.
All of which gets back to: The “good” LLMs? Those are the ones companies are paying for to use for very specific use cases and training data is very heavily labeled as part of that.
For the cheap “build up word of mouth” LLMs? They don’t give a fuck and they are invariably going to be poisoned by misinformation. Just like humanity is. Hey, what can’t jet fuel melt again?
Fandangalo@lemmy.world 11 hours ago
Garbage in, garbage out.
Telorand@reddthat.com 12 hours ago
On that note, if you’re an artist, make sure you take Nightshade or Glaze for a spin. Don’t need access to the LLM if they’re wantonly snarfing up poison.
_cryptagion@anarchist.nexus 11 hours ago
the reason more people haven’t adopted that is because they don’t work.
Telorand@reddthat.com 11 hours ago
I haven’t seen any objective evidence that they don’t work. I’ve seen anecdotal stories, but nothing in the way of actual proof.
jaybone@lemmy.zip 9 hours ago
lol nice BSD brag thrown in there
mudkip@lemdro.id 12 hours ago
Great, why aren’t we doing it?
Telorand@reddthat.com 10 hours ago
Because it’s hard(er than doing nothing) and takes changing habits.
Hegar@fedia.io 11 hours ago
I don't know that it's wise to trust what anthropic says about their own product. AI boosters tend to have an "all news is good news" approach to hype generation.
Anthropic have recently been pushing out a number of headline grabbing negative/caution/warning stories. Like claiming that AI models blackmail people when threatened with shutdown. I'm skeptical.
BetaDoggo_@lemmy.world 8 hours ago
They’ve been doing it since the start. OAI was fear mongering about how dangerous gpt2 was initially as an excuse to avoid releasing the weights, while simultaneously working on much larger models with the intent to commercialize. The whole “our model is so good even we’re scared of it” shtick has always been marketing or an excuse to keep secrets.
Even now they continue to use this tactic while actively suppressing their own research showing real social, environmental and economic harms.
_cryptagion@anarchist.nexus 11 hours ago
if that’s true, why hasn’t it worked so far then?
yardratianSoma@lemmy.ca 12 hours ago
Well, I’m still glad offline LLM’s exist. The models we download and store are way less popular then the mainstream, perpetually online ones are.
Once I beef up my hardware (which will take a while seeing how crazy RAM prices are), I will basically forgo the need to ever use an online LLM ever again, because even now on my old hardware, I can handle 7 to 16B parameter models (quantized, of course).
morto@piefed.social 12 hours ago
I used to think it wasn’t viable to poison llms, but are you saying there’s a chance? [a meme comes to mind]
HertzDentalBar@lemmy.blahaj.zone 7 hours ago
So what websites should be targeted?
WhatGodIsMadeOf@feddit.org 12 hours ago
Isn’t this applicable to all human societies as well though?
ceenote@lemmy.world 12 hours ago
So, like with Godwin’s law, the probability of a LLM being poisoned as it harvests enough data to become useful approaches 1.
Gullible@sh.itjust.works 12 hours ago
I mean, if they didn’t piss in the pool, they’d have a lower chance of encountering piss. Godwin’s law is more benign and incidental. This is someone maliciously handing out extra Hitlers in a game of secret Hitler and then feeling shocked at the breakdown in the game
saltesc@lemmy.world 12 hours ago
Yeah but they don’t have the money to introduce quality governance into this. So the brain trust of Reddit it is. Which explains why LLMs have gotten all weirdly socially combative too; like two neckbeards having at it with Google skill vs Google skill is a rich source of A+++ knowledge and social behaviour.
UnderpantsWeevil@lemmy.world 11 hours ago
Hey now, if you hand everyone a “Hitler” card in Secret Hitler, it plays very strangely but in the end everyone wins.