Submitted 1 year ago by GlitzyArmrest@lemmy.world to technology@lemmy.world
Vechev and his team found that the large language models that power advanced chatbots can accurately infer an alarming amount of personal information about users—including their race, location, occupation, and more—from conversations that appear innocuous.
But what does it actually infer? The article is very low on details on that.
Probably because it was written by an ai chat bot.
Things like race, sex, orientation and job. Stuff that a human could probably also infer from talking to you. I think this article is a little alarmist. I could look at you and infer your race lol.
Also companies already infer that same information about you in other ways so this won’t really change much. Just makes it more accurate and faster while costing more.
This is the best summary I could come up with:
New research reveals that chatbots like ChatGPT can infer a lot of sensitive information about the people they chat with, even if the conversation is utterly mundane.
“It’s not even clear how you fix this problem,” says Martin Vechev, a computer science professor at ETH Zürich in Switzerland who led the research.
He adds that the same underlying capability could portend a new era of advertising, in which companies use information gathered from chatbots to build detailed profiles of users.
The Zürich researchers tested language models developed by OpenAI, Google, Meta, and Anthropic.
Anthropic referred to its privacy policy, which states that it does not harvest or “sell” personal information.
“This certainly raises questions about how much information about ourselves we’re inadvertently leaking in situations where we might expect anonymity,” says Florian Tramèr, an assistant professor also at ETH Zürich who was not involved with the work but saw details presented at a conference last week.
The original article contains 389 words, the summary contains 156 words. Saved 60%. I’m a bot and I’m open source!
This is why I always filter all my chat for conversations through the pirate filter
GenderNeutralBro@lemmy.sdf.org 1 year ago
You fix this problem with locally-run models that do not send your conversations to a cloud provider. That is the only real technical solution.
Unfortunately, the larger models are way too big to run client-side. You could launder your prompts through a smaller LLM to standardize phrasing (e.g. removing idiosyncrasies or local dialects), but there’s only so far you can go with that, because language is deeply personal, and the things people will use chatbots for are deeply personal.
This is by no means exclusive to LLMs, of course. Google has your lifetime search history and they can glean all kinds of information from that alone. If you’re older than ~30 or so, you might remember these same conversations from when Gmail first launched. You’d have to be crazy to let Google store all your personal emails for all eternity! And yet everybody does it (myself included, though I’m somewhat ashamed to admit it).
This same problem exists with pretty much any cloud service. When you send data to a third party, they’re going to have that data. And I guarantee you are leaking more information about yourself than you realize. You can even tell someone’s age and gender with fairly high accuracy from a small sample of their mouse movements.
I wonder how much information I’ve leaked about myself from this comment alone…
FaceDeer@kbin.social 1 year ago
I fed your comment to ChatGPT (telling it that it was a comment that I had written to avoid triggering any of its "as a large language model blah blah privacy" conditioning) and this is what it said:
So not much from just that comment, but a few tidbits that can be added to a profile that builds up more detail over time.
We were already facing this sort of thing before AI exploded, though. A lot of the various Reddit user analysis services out there were able to get a fair bit right about me based just off of my most recent 1000 comments (though I just checked my profile on RedditMetis and it did get a few significant things wrong, it's clearly a pretty simple-minded approach to analysis).
Heh. I just checked the link for why it thinks I'm transgender and it referenced this comment where I'm literally objecting to RedditMetis' interpretation that I'm transgender. Citogenesis at work.
Kachilde@lemmy.world 1 year ago
It doesn’t feel like it actually inferred anything from the comment.
“You spoke about computers, so you probably know about computers”
“You express concerns about privacy, so you are likely privacy conscious”
“You said you were 30ish, so you’re maybe 30…ish”
It essentially paraphrased each part of the comment, and gave it back to you like an analysis. Of course, this is ChatGPT, so it’s likely not trained for this sort of thing.
always makes me scratch my head when people
GenderNeutralBro@lemmy.sdf.org 1 year ago
LOL. Nice!
I wouldn’t expect ChatGPT to be well-versed in forensic linguistics; I suspect a human expert could make better guesses based on seemingly-innocuous things like sentence structure and word choices. I’ve seen some research on estimating age and gender based on writing. There’s a primitive example of that here: www.hackerfactor.com/GenderGuesser.php
My last comment is a bit short (it wants 300 words or more), but I am amused by the results:
I’ll pat myself on the back for writing more or less down the middle. :)
Phanatik@kbin.social 1 year ago
While it should teach me to be less forthcoming about my personal information but at the same time, the idea that services were built to crawl through my information with LLMs on top, inadvertently doing the same thing, makes my fucking skin crawl. Why is it so difficult to have a conversation on the internet without some creepy shit spying on everything you do.
Que@lemmy.world 1 year ago
How did you get it to infer anything?
It tells me:
… Or:
theterrasque@infosec.pub 1 year ago
There is some hope. Mistral is pretty incredible for it’s size, and it’s a 7b model. There are finetunes on top of that which makes it even better - my favorite right now is Open Hermes 2
There’s still room for improvement, and it’s getting better and better.
abhibeckert@lemmy.world 1 year ago
Not really. LLAMA 13b only needs 8GB of memory for example - and it runs with perfectly good performance on my fanless/ultraportable laptop.
The 65 billion parameter model apparently needs about 30GB… you do need to spend a couple thousand dollars to get a laptop with that much memory (64GB realistically, since you need memory for other software too) but again it’s achievable with current hardware.
GenderNeutralBro@lemmy.sdf.org 1 year ago
Yes, I am impressed with Apple Silicon, and the fact that I can get up to 192GB of integrated memory in a laptop is pretty impressive. Unfortunately, the costs are still crazy high (Apple and overpriced RAM: name a more iconic duo), and even 192GB is only about half of what is needed for the largest models I’m aware of (e.g. BLOOM 366B). I don’t think OpenAI has officially stated how big GPT4 is, but it’s likely even bigger.
The industry has been stagnating for a long time now in terms of memory, and I hope this will push prices down and capacity up.
The good news is that there is strong motivation for companies like Apple and Google to shift more processing onto client devices, because the cost of running these servers is freakin’ huge.
You’re right of course. There are even some with a focus on privacy, like Proton Mail. But Gmail and similar services are overwhelmingly dominant, and not just because people are dumb. There is real value in having email that is accessible on any device, for free, with enough storage that you never really need to think about it. Proton offers 1GB for free now, which is pretty solid but a far cry from what Google, Microsoft, or Yahoo (yep they’re still around) provide. I mean, Google offered 1GB almost 20 years ago.
I am personally still in the process of de-googling my life, and the idea of updating every account I ever signed up for using my gmail address is daunting. I’ll probably never get 100% of the way there; for now I am satisfied enough moving my personal correspondence and important accounts onto different email. Eventually I will probably set up my own domain and get a premium Proton plan so I won’t be too tightly tied to any particular email provider. Then if Proton ever enshittifies, I can take my domain and go elsewhere.