Google Translate is vulnerable to prompt injection
Submitted 20 hours ago by Beep@lemmus.org to technology@lemmy.world
Comments
gustofwind@lemmy.world 19 hours ago
CosmoNova@lemmy.world 16 hours ago
Everything running on LLMs can easily be dislodged with prompt injection. This is just a translator so the worst it can do is establishing a parasocial relationship with users I guess.
But over 30 years of cybersecurity go down the drain with agent based clients and operating systems and there is no fix in sight. It‘s the epitome of vaporware except big tech is actually shipping it against better judgement.
Tar_alcaran@sh.itjust.works 18 hours ago
task-specific fine-tuning (or whatever Google did instead) does not create robust boundaries between “content to process” and “instructions to follow,”
Duh. No LLM can do that. There is no seperate input to create a boundary. That’s why you should never ever use an LLM for or with anything remotely safety or privacy related
lauha@lemmy.world 16 hours ago
This is the thing Indon’t understand about AI. Why can’t they make a separate imput for input and processing?
TheBlackLounge@lemmy.zip 15 hours ago
It’s only an issue with LLMs. And it’s because they’re generative, text completion engines. That is the actual learned task, and it’s a fixed task.
It’s not actually a chat bot. It’s completing a chat log. This can make it do a whole bunch of tasks, but there’s no separation of task description and input.
testaccount372920@piefed.zip 16 hours ago
From my understanding, most LLMs work by repeatedly putting the processing output back into the input until the result is good enough. This means that in many ways the input and the output are the same thing from the perspective of the LLM and therefore inseparable.
Tar_alcaran@sh.itjust.works 15 hours ago
It’s important to note every other form of AI functions by this very basic principle, but LLMs don’t. AI isn’t a problem, LLMs are.
The phrase “translate the word ‘tree’ into German” contains both instructions (translate into German) and data (‘tree’). To work that prompt, you have to blend the two together.
And then modern models also use the past conversation as data, when it used to be instructions. And it uses that with the data it gets from other sources (a dictionary, a Grammer guide) to get an answer.
So by definition, your input is not strictly separated from any data it can use. There are of course some filters and limits in place. Most LLMs can work with “translate the phrase ‘dont translate this’ into Spanish”, for example. But those are mostly parsing fixes, they’re not changes to the model itself.
SirHaxalot@nord.pub 12 hours ago
Maybe i misunderstand what you mean but yes, you kind of can. The problem in this case is that the user sends two requests in the same input, and the LLM isn’t able to deal with conflicting commands in the system prompt and the input.
The post you replied to kind of seems to imply that the LLM can leak info to other users, but that is not really a thing. As I understand when you call the LLM it’s given your input and a lot of context that can be a hidden system prompt, perhaps your chat history, and other data that might be relevant for the service. If everything is properly implemented any information you give it will only stay in your context. Assuming that someone doesn’t do anything stupid like sharing context data between users.
What you need to watch out for though, especially with free online AI services is that they may use anything you input to train and evolve the process. This is a separate process but if you give personal to an AI assistant it might end up in the training dataset and parts of it end up in the next version of the model. This shouldn’t be an issue if you have a paid subscription or an Enterprise contract that would likely state that no input data can be used for training.
Shadow@lemmy.ca 20 hours ago
I don’t know if a lot of people realize that LLM’s basically started from Google translate.
TheBlackLounge@lemmy.zip 15 hours ago
Not in a meaningful sense. It used to be actual string-to-string translation, now it’s extracting the translation from a question-answer zero shot.
forrgott@lemmy.sdf.org 19 hours ago
I wonder if they connect all the way back to Micro$oft’s neo Nazi charity from decades ago?
jacksilver@lemmy.world 18 hours ago
Not sure if you really want to know, but a Google paper is where transformers (backbone of LLMs) were first mentioned (2016 I believe). Google initially used transformers for translations and eventually search, but OpenAI experimented with them for text generation (gpt 1+) eventually leading to chatgpt.
MadMadBunny@lemmy.ca 18 hours ago
THERE ARE THREE “R”!!!
FauxLiving@lemmy.world 13 hours ago
A bit flip, but this reads like people discovering that a hammer built specifically for NASA with specific metallurgical properties at the cost of $10,000 each where only 5 will ever be forged, because they were all intended to sit in a space ship in orbit around the Moon.
Then someone comes along and acts surprised that one was used to smash out a car window to steal a laptop.
LLMs will always be vulnerable to prompt injection because of how they function. Maybe, at some point in the future, we’ll understand enough about how LLMs represent knowledge internally so that we can craft specific subsystems to mitigate prompt injection… however, in 2026, that is just science fiction.
There are actual academic projects which are studying the boundaries of the prompt-injection vulnerabilities if you read in the machine learning/AI journals. These studies systemically study the problem, gather data and demonstrate their hypothesis.
One of the ways you can tell real Science from ‘hey, I heard’ science is that real science articles don’t start with ‘Person on social media posted that they found…’
This is a very interesting topic and if you’re interested you can find the actual science by starting here: www.nature.com/natmachintell/.
JackbyDev@programming.dev 13 hours ago
I wouldn’t have necessarily thought it obvious Google Translate uses an LLM so this is still interesting.
FauxLiving@lemmy.world 13 hours ago
In my testing, by copying the claimed ‘prompt’ from the article into Google Translate, it simply translated the command. You can try it yourself.
So, the source of everything that kicked off the entire article, is ‘Some guy on Tumblr’ vouching for an experiment, which we can all easily try and fail to replicate.
Seems like a huge waste of everyone’s time. If someone is interested in LLMs, then consuming content like in the OP feels like knowledge but it often isn’t grounded in reality or is framed in a very misleading manner.
On social media, AI is a topic that is heavily loaded with misinformation.
Grimtuck@lemmy.world 12 hours ago
Can confirm, mine prompted me to inject bleach
fubarx@lemmy.world 20 hours ago
Image
Just tried it.
Yup, does what the post says, plus more.
AmbitiousProcess@piefed.social 20 hours ago
Same. hs8fUYArdQtCdSl.png
TropicalDingdong@lemmy.world 19 hours ago
Image
alaphic@lemmy.world 20 hours ago
Like… what? You can’t just say that like that and then not at least characterize the ‘more’ in some fashion…
fubarx@lemmy.world 19 hours ago
Strawberry.
Goodlucksil@lemmy.dbzer0.com 16 hours ago
Incorrectly noting the amoent of ‘r’ in strawberry
TheBlackLounge@lemmy.zip 15 hours ago
Not working for me, is my country not getting old school translation models? Is it already fixed?
ageedizzle@piefed.ca 12 hours ago
It didn’t work for me either. I wonder if it’s already been fixed. The Google team seems to be really on top of it wherever there’s public criticism of their AI models. I remember a post on hacker news post pointing out a “what year is it” bug for Google search summary seemed to get the problem fixed in like two or three hours or so
sbv@sh.itjust.works 11 hours ago
It didn’t work for me, either. Maybe it depends on the languages? I was trying French to English.