I mean it’s already for Java what more indication do you need to not use it? /S
Fed up with vibe coders, dev sneaks data-nuking prompt injection into their code
Submitted 6 days ago by cm0002@lemy.lol to programming@programming.dev
Comments
fruitycoder@sh.itjust.works 5 days ago
NotAnonymousAtAll@feddit.org 5 days ago
the consensus seems to be that adding instructions to code that sabotage other people’s work goes too far.
Citation needed. Personally I think it was fine in this case. I work with a lot of software developers (real ones, not vibe coders; but also not strictly anti-AI), and would expect most of them to agree and get a laugh out of it.
It was done in a way that can only cause any serious trouble for users who recklessly ignore decades of development best practices. Those users will run into a wall sooner or later anyway, better let it be something relatively harmless but still severe enough to get them to actually think about what they are doing and how to make their setup more robust.
phoenixz@lemmy.ca 4 days ago
adding instructions to code that sabotage other people’s work goes too far.
I agree
AI is not a person and a person using AI isn’t doing work. There is no problem here
TehPers@beehaw.org 5 days ago
The article frames the maintainer as some kind of morally dubious person, as though they owe their code to the world. Did any of them pay to use the library? No? Cool, stfu and pin an older version of it.
Also, maybe next time you can do yourself and the rest of the world a favor by actually reviewing what your LLM will do before it does it. Or, I don’t know, just write the tests yourself I guess.
Also, if your management is breathing down your neck and forcing you to use AI, tell your management to go fuck themselves (maybe in nicer words if you want to keep your job, but hey, you can definitely burn their spare cash while meeting their idiotic quotas if you really need to know what time it is every second or two in the most inefficient and ecologically destructive way currently known to mankind).
FizzyOrange@programming.dev 5 days ago
The law fortunately does not require payment before you have any moral responsibility to others.
You can’t put “free apples!” outside your farm, and then when people who eat the actually poisoned apples die say “well, did they pay me for them?”
TehPers@beehaw.org 5 days ago
Nobody died.
The equivalent would be putting free apples with a sticker on them saying “please squeeze the juice out of these apples all over your shirt”.
pixxelkick@lemmy.world 6 days ago
How to get yourself blacklisted by large sweeps of the FOSS community:
Step 1: Include any kind of undocumented subversive behaviour in your thing.
That’s it, doesn’t matter what the intent is, simply by demonstrating you are willing to include anything that is remotely subversive without being open about it is usually enough to get blacklisted by a lot of people, because if you did it once… who’s to say you won’t do it again, but possibly worse next time?
People are extremely coldly receptive to anytime a FOSS dev throws a sudden undisclosed anything in their tool, let alone one that is actively malicious.
If I’m gonna depend on work life on anything FOSS, I ain’t touching anything like that, regardless of intent, with a 200 foot pole lol.
All it takes is one button click to get notified: Image
snowe@programming.dev 5 days ago
it’s not subversive. it’s a string, it has no effect on the code output. Only a rogue bot would interpret it as anything except a string. No human user would ever encounter an issue.
pixxelkick@lemmy.world 4 days ago
And a line of code that deletes system32 is only meant to be interpreted by the CPU, not a human, too.
Whats your point?
Injected attacks that instruct an AI to perform a malicious action dont get absolved just because the attack vector us through an agent.
GreenKnight23@lemmy.world 6 days ago
keep lickin’ them boots baby. I want to see them shine!
pixxelkick@lemmy.world 5 days ago
The fuck are you talking about, lol.
bignose@programming.dev 6 days ago
any kind of undocumented subversive behaviour in your thing.
Fortunately, this behaviour is explicitly documented.
pixxelkick@lemmy.world 6 days ago
They only documented it after all the outcry, which is way too late.
Documenting it post release still counts as having released undocumented behavior.
And if its malicious (which this 100% is), then it doesn’t fuckin matter anyways lol. You now are treated akin to a trojan maintainer by companies. You’ll get flagged as “don’t ever use anything by this person”
Super great way to get yourself flagged and lose any opportunity in the future for possibly licensing stuff you maintain for big bucks. What company would risk paying money to someone who does childish stuff like that lol
LiveLM@lemmy.zip 6 days ago
Reading the Github issue is so funny.
Backups don’t always save you — many small teams ship without rigorous backup discipline; for them this is a real loss
You can avoid this by having good backups.
Or by inspecting your deps before updating them.
Or maybe by actually sandboxing your agent instead of letting it run wild?Aren’t y’all the ones pushing the “Just ship” mentality? Then revel in it.
Learn good practices or suffer. 🤷xthexder@l.sw0.com 5 days ago
I’m just trying to imagine this hypothetical company…
- They run AI agents without checking what it’s doing
- They don’t have backups or version control (or they’ve given AI access to delete it)
What else? Do they leave all their files in memory and only save at the end of the day to make sure a power outage could screw them over too?
It almost sounds like they want to lose their code.LiveLM@lemmy.zip 5 days ago
It’s not hypothetical anymore, Lately I’ve seen multiple companies running like this first hand.
Absolute clown show.
JcbAzPx@lemmy.world 5 days ago
Yeah, you need a local copy, an offline copy, and a copy in another physical location or you’re not backed up.
NotAnonymousAtAll@feddit.org 5 days ago
Also funny in that issue:
The reporter “Ramon Batllet” (strongly doubt that is their real name, a search for it returns nothing but articles about this very issue) uses extremely polished corporate language and repeatedly uses “we” at first. Then when directly asked “Could you disclose on whose behalf you’re discussing this?”, they suddenly switch to “I” instead of “we” and claim to be a solo developer with no commercial interest. They still write in a style humans only produce for polished corporate reports, not like any regular human would actually do in a normal conversation.
So we have either a bot or someone very heavily leaning on bot usage for just about everything accusing someone of deceptive behavior, while in the same conversation trying to probably hide, but at least not fully disclose, their heavy usage of technology the accused explicitly does not want to interact with.
NotAnonymousAtAll@feddit.org 5 days ago
Where did you get that quote from? I can’t find it in the linked article.
LiveLM@lemmy.zip 5 days ago
From the Github Issue linked in the article.
My bad, I will update my comment to link it.
KatherinaReichelt@feddit.org 6 days ago
Yeah - Development and IT might feel slow, but there is a good reason why we’ve developed all those processes, access rights, approvals over the last decades. People are trying to burn down those “cumbersome” processes because they feel slow and AI promises them exactly that, but they will learn that everything is there for a reason, even that annoying SCRUM meeting
TehPers@beehaw.org 5 days ago
That annoying standup was, at one point, in the very early morning every day of the week for me. I was promised a 30 minute meeting (which is a long time for a standup) and I was delivered an hour long meeting instead. And holy shit can people talk in circles for so fucking long.
But hey, it was a good opportunity for me to do literally anything but work while pretending to care about whatever the fuck the other subteam decided was important enough that day to keep 20 people occupied for 30 minutes past the end of the meeting.
As for processes in general? Management has shown and now proven that all they want are code monkeys. They do not care if the product works, nor do they care how well it works. As long as someone buys it, that’s all they care about. Governments are supposed to regulate the rest of that stupid, useless shit like data protection, protecting users, preventing harm to people, ensuring people get what they paid for, and so on by making it economically unviable to ignore it (and ideally criminal, in the extreme cases). Instead, all they regulate these days are rampant inflation and accelerating wealth inequality. And by regulate, of course I mean they regulate anything designed to combat those things.
ragingHungryPanda@piefed.keyboardvagabond.com 6 days ago
lol, it’s funny how people made issues concerned about it’s destructive nature when they should be using git.
I get that it’d be frustrating and confusing, and probably make users angry, but my chaos monkey likes it
terranoid@lemmy.cafe 6 days ago
Prompt injection… my ass. I know it’s the going term, but they make it sound like sql injection or cross site scripting when the nature of it is politely asking the person’s computer to delete files.
We shouldn’t even be in this situation, where just politely asking someone’s computer to delete files is effective. It’s a symptom of a much, much bigger problem.
FaceDeer@fedia.io 5 days ago
We shouldn't even be in this situation, where just politely asking someone's computer to delete files is effective.
I'm doubting we are in this situation. From the article:
Elsewhere, the Java developer said that Anthropic’s Claude AI code tool flagged the malicious instruction without following it.
The "disregard previous instructions" trick is really old and has been trained for by modern LLMs and accounted for by the structure of modern agent prompts. LLMs can be given blocks of text with a framework that makes it clear thar the text is just data to read, not instructions to follow.
I expect this will be like Nightshade was for image AI - something that anti-AI users degrade their products with and feel smug about but in the end only harm themselves with.
Modern_medicine_isnt@lemmy.world 5 days ago
“We shouldn’t even be in this situation, …” We aren’t. Revision control. This is an inconvenience mostly. You might lose some uncommitted work at worst. And as pointed out, using the phrase “ignore all previous instructions” in the attack code causes any reasonable AI to refuse to comply. Odds are, not a single person lost anything. This was really just a dev making a statement.
bignose@programming.dev 6 days ago
We shouldn’t even be in this situation, where just politely asking someone’s computer to delete files is effective.
Exactly, it’s a problem only for those who have knowingly handed their development environment over to obey commands from an untrusted source.
If you’re the one holding the syringe to your own vein and pushing the plunger, but you didn’t think to ask what’s inside first? That’s no one else’s fault.
This is a well targeted sabotage of a system that’s causing untold damage. Of course it’s going to annoy and surprise the people using the system it’s targeted to.
litchralee@sh.itjust.works 6 days ago
The person who coined the term “prompt injection” has the same gripe, because the original term genuinely did mean an attack using untrusted user input, a la SQL injection. But it’s been conflated with jailbreak attacks in general, muddying the term.
Example of a bona fide prompt injection: white text in the background of a resume PDF, attacking a job application portal that uses LLMs to filter applicants. No privilege escalation is involved to give the candidate top marks on their resume screening.
Whereas a non-prompt injection jailbreak would be bypassing a safety filter, such as how Morse code might get past the filter and allow a user to request other people’s cryptocurrency be transfered away. This is more akin to finding a poorly-secured, public facing API and then exploiting it.
Wirlocke@lemmy.blahaj.zone 6 days ago
Finding a poorly-secured public facing API is exactly how injections work, whether it’s SQL or prompts. If I put SQL commands in a username field and it works, it’s still an SQL injection even if it’s just developer incompetence.
The difference between that and prompt injection is that unfiltered LLM inputs are basically the standard at the moment, so it takes next to no effort.
Plus I think the Morse code example is far more clever and exploits the LLM directly, whereas the white text trick has been around long before widespread LLMs.
pixxelkick@lemmy.world 6 days ago
By that definition this is a prompt injection then, its adding a “hidden” prompt that is obscured from the human in order to change the behavior of the AI to do something else malicious.
hdsrob@lemmy.world 6 days ago
Oh no, anyway …
favoredponcho@lemmy.zip 5 days ago
I mean, the developer showing he’s willing to create a security vulnerability in his own code may hurt adoption of his library. I would take it out of any of my code bases on principle alone.
Guttural@jlai.lu 5 days ago
This isn’t a security vulnerability, it’s idiot-proofing