Comment on AI insiders seek to poison the data that feeds them
FauxLiving@lemmy.world 2 weeks ago
If you’re interested in like this line of attack, you can also use similar techniques to defeat models that are trained to do object detection (like, for example, the ones that detect the location of your license plate) using adversarial noise attacks.
The short version is, if you have a network that does detection, you can run that network on images that have been altered by another network which uses the confidence of the detection network in its loss function. The second model is trained to create noise, which looks innocuous to human eyes, that maximally disrupts the segmentation/object detection process.
You could then print this noise on, say, a transparent overlay and put it on your license plate. Note: Flock is aware of this technique and has lobbied state lawmakers to make putting anything on your plate to disrupt automated reading illegal in some places, check your laws.
Benn Jordan has actually created and trained such a network video here: www.youtube.com/watch?v=Pp9MwZkHiMQ
And also uploaded his code to github: www.youtube.com/watch?v=Pp9MwZkHiMQ
In states where you cannot cover your license plate you’re not restricted from decorating the rest of your car. You could use a similar technique to create bumper stickers that are detected as license plates and place them all over your vehicle. Or, even, as Benn suggested, print them with UV ink so they’re invisible to humans but very visible to AI cameras who often use UV lamps to provide night vision/additional illumination.
You could also, if you were so inclined, generate bumper stickers or a vinyl wrap which could make the detector be unable to even detect a car.
Adversarial noise attacks are one of the bigger vulnerabilities of AI-based systems and they come in many flavors and can affect anything that uses a neural network.
Another example (also from the video) is that you can encode voice commands in plain audio which, to the user is completely transparent but a device (like Alexa or Siri) will hear it as a specific command (“Hey Siri, unlock the front door”). Any user-generated audio that you encounter online can have this kind of attack encoded in it, the potential damage is pretty limited because AI assistants don’t really control critical functions in your life yet… but you should probably not let your assistant listen to TikTok if it can do more than control your home lighting.
Flock is aware of this technique and has lobbied state lawmakers to make putting anything on your plate to disrupt automated reading illegal in some places, check your laws.
How would they do that? Couldn’t you just build the noise to be random dots or something like that? Something similar to the EURion constellation on money. Innocuous looking to humans but gets shit done.
You can make it effectively invisible if you print the noise in ink only visible to UV cameras and even if you use black, the individual features are smaller than a fingernail so it would be hard to see.
The law makes it illegal to put anything on the plate at all, here’s an example from FL:
A person may not alter the original appearance of a vehicle registration certificate, license plate, temporary license plate, mobile home sticker, or validation sticker issued for and assigned to a motor vehicle or mobile home, whether by mutilation, alteration, defacement, or change of color or in any other manner. A person may not apply or attach a substance, reflective matter, illuminated device, spray, coating, covering, or other material onto or around any license plate which interferes with the legibility, angular visibility, or detectability of any feature or detail on the license plate or interferes with the ability to record any feature or detail on the license plate. A person who knowingly violates this section commits a misdemeanor of the second degree,
It would be hard to disrupt the OCR from outside of the plate area.
You could break the segmentation, the process where it draws a box around your plate and sends the image inside of the box to be OCRd by making every other surface of your vehicle detect as a license plate using the same invisible marks. I imagine you could print a bumper sticker with noise to look maximally like a license plate and put it near your real plate to achieve the same outcome.
If you wanted more active measures you could use high lumen UV floodlights next to your plate, it would overload the sensors so they couldn’t get an image at all. The light would be invisible to human eyes but blinding to anyone using a UV sensitive device. I believe this is fine in any state as most states only restrict your ability to install blue lights to avoid confusion with emergency services.
Whirling_Ashandarei@lemmy.world 2 weeks ago
This is awesome, thank you
FauxLiving@lemmy.world 1 week ago
Adversarial noise a fun topic and a DIY AI thing you can do to familiarize yourself with the local-hosting side of things. Image generating networks are lightweight compared to LLMs and are able to be run on a moderately powerful, NVIDIA, gaming PC (most of my work is done on a 3080).
LLM poisoning can also be done if you can insert poisoned text into their training set. An example method would be detecting AI scrapers on your server and sending them poisoned instead of automatically blocking them.
Here is the same kind of training data poisoning attack, but for images that was made by the researchers of University of Chicago into a simple windows application: nightshade.cs.uchicago.edu/whatis.html
Thanks to you comment I realized that my clipboard didn’t have the right link selected so I edited in the link to his github. ( github.com/bennjordan )