Hackers can read private AI-assistant chats even though they’re encrypted

Comments

• kevincox@lemmy.ml ⁨8⁩ ⁨months⁩ ago

  This is pretty clever. As I understand it.

  1. Because LLMs are slow most of them stream the response to the user.
  2. The response is streamed as text, but generated in tokens.
  3. This means that each “chunk” leaks the length of the text corresponding to the token.
  4. You can then use heuristics to guess the text of the response based on the token lengths.

  This is a good reminder any time you are sending content in small chunks over an encrypted channel, many encrypted channels don’t provide protection against size leaks by default.

  It seems there are a few easy solutions to this:

  1. Send the token IDs (as fixed-size integers) over the network rather than the text.
  2. Pad the text representations of the tokens to a fixed length.
  3. Batch the tokens more (and maybe add padding) to produce bigger chunks and obscure individual token size.

  These still all leak the approximate length of the response, but that is probably acceptable.

  source

  • PlexSheep@feddit.de ⁨8⁩ ⁨months⁩ ago

    That actually is really really interesting. Thanks for giving the tldr. Do token lengths vary that much?

    source

    • kevincox@lemmy.ml ⁨8⁩ ⁨months⁩ ago

      Absolutely. They are sort of a compression scheme so the tokens contain different numbers of characters based on how frequent that string is. So common words like “the” will typically be one token, or maybe even common phrases like “I am”. On the other hand rare punctuation such as “~” may be its own token. There will also be tokens for many common prefixes and suffixes such as “non” and “n’t”. The tokens of each model are different but they definitely vary in length.

      source
• MajorHavoc@programming.dev ⁨8⁩ ⁨months⁩ ago

  Interesting stuff, but probably not huge news.

  The AI guessing portion is currently playing on ultra-easy mode because there are only a few useful LLMs to compare to, and none of those have given serious thought to strongly securing their tech demo solution communications channel yet.

  This is important work though, since someday someone might use an AI for something important and actually want to prevent eavesdropping.

  I’m being a little unfair, but it’s early days and let’s try not take ourselves too seriously just yet. If I’m trusting one of these LLMs with something deeply sensitive, it’s not eavesdropping that’s going to get me into deep shit, it’s the LLM’s hallucinations.

  This is important research, though. Someday the AI will get good, and I’ll want to chat with it securely.

  Thankfully, I suspect mitigation will be quite straightforward. Carefully designed small random changes to token handling should throw this approach way off, while not even being noticed by end users.

  The habit of sending tokens right as they generate is a dumb sales gimmick that needs to go away anyway. So if security folks manage to kill it, good riddance.

  source

  • abhibeckert@lemmy.world ⁨8⁩ ⁨months⁩ ago

    Someday the AI will get good, and I’ll want to chat with it securely.

    GPT4 is pretty good now. I’m not convinced it will be secure until we can run it locally on our own hardware.

    source