Schools and lawmakers are grappling with how to address a new form of peer-on-peer image-based sexual abuse that disproportionately targets girls.
Back in my day we just had to use our own imagination.
Kids are making deepfakes of each other, and laws aren’t keeping up
Submitted 8 months ago by Pro@reddthat.com to technology@lemmy.world
https://19thnews.org/2025/07/deepfake-ai-kids-schools-laws-policy/
Comments
JohnEdwa@sopuli.xyz 8 months ago
argl@feddit.org 8 months ago
Can’t afford this much cheese today to find just the right slice for every bikini photo…
SabinStargem@lemmy.today 8 months ago
Deepfakes might end up being the modern version of a bikini. In the olden days, people wore these to the beach. Having less was scandalous and moral decay. Yet, now we wear much less.
Our grandchildren might simply not give a damn about their nudity, because it is assumed that everyone is deepfaking everyone.
atomicorange@lemmy.world 8 months ago
These are all worn voluntarily. This issue isn’t about the equivalent of scandalously clad young girls, it’s like if girls were being involuntarily stripped of their clothing by their classmates. It’s not about modesty standards it’s about sexual abuse.
Gsus4@mander.xyz 8 months ago
Unless it is used to pretend that it is a real video and circulated for denigration or blackmail, it is very much not at all like assault. And also, deepfakes do not have the special features hidden under your clothes, so it is possible to debunk those if you really have to.
SabinStargem@lemmy.today 8 months ago
It can be both. The cornerstone of why nudity can be abused, is that society makes it shameful to be bare. If some generations from now that people can just shrug and not care, that is one less tool an abuser can use against people.
In any case, I am of the mind that people of my generation might be doing their own version of the Satanic Panic, or the reaction against rap music. For better or worse, older people cannot relate to the younger.
FriendFatale@leminal.space 8 months ago
anyone using any kind of AI either doesn’t know how consent works-- or they don’t care about it.
a horrifying development in the intersection of technofascism and rape culture
5gruel@lemmy.world 8 months ago
Any AI? Every application? What kind of statement is this?
AstaKask@lemmy.cafe 8 months ago
AI models (unless you’re training your own) are usually trained on data it does not have a licence to use. The companies training these models are also notorious for ignoring robot.txt and other measures websites use to stop bots from trawling their data.
Like in crypto, most people in AI are not nerds, just criminal scum.
vane@lemmy.world 8 months ago
Maybe let’s assume all digital images are fake and go back to painting. Wait… what if children start painting deepfakes ?
Gsus4@mander.xyz 8 months ago
Or pasting someone’s photo over a naked body…in their minds…
cupcakezealot@piefed.blahaj.zone 8 months ago
probably because there's a rapist in the white house.
oppy1984@lemdro.id 8 months ago
At least they’ve learned a skill?
aceshigh@lemmy.world 8 months ago
To add to that. I live in a red area and since the election I’ve been cat called much more. And it’s weird too, cus I’m middle aged…. I thought I’d finally disappear…
cupcakezealot@piefed.blahaj.zone 8 months ago
the toxic manosphere/blogosphere/whatever it's called has done so much lifelong damage
2ugly2live@lemmy.world 8 months ago
God I’m glad I’m not a kid now. I never would have survived.
jjlinux@lemmy.ml 8 months ago
In my case, other kids would not have survived trying to pull off shit like this. So yeah, I’m also glad I’m not a kid anymore.
Daft_ish@lemmy.dbzer0.com 8 months ago
Welp, if I had kids they would have one of those scramble suits like in a scanner darkly.
It would of course be their choice to wear them but Id definitely look for ways to limit their time in areas with cameras present.
Entertainmeonly@lemmy.blahaj.zone 8 months ago
That’s just called the outside now. Assume you are on camera at all times the moment you step out the front door. To be safe in the surveillance we live in today, best act as though you are being recorded in your own home as well.
Vanilla_PuddinFudge@infosec.pub 8 months ago
best act as though you are being recorded in your own home as well.
If you don’t know, don’t try? Seems a bit defeatist.
Daft_ish@lemmy.dbzer0.com 8 months ago
You can make areas safe from cameras. No, you cant make everywhere camera free but you can minimize your time in those areas. Im not saying its a good system it would just be adjusting to the times.
If the floor would lava and all that…
Walk_blesseD@piefed.blahaj.zone 8 months ago
Jfc the replies here are fucking rancid. Lemmy is full of sweaty middle aged blokes in tech who hate it when anyone tells them that grown men who pursue teenage girls who have just reached an arbitrary age are fucking creeps, so of course they're here encouraging the next generation of misogynist scum by defending this shit, too.
And men (pretend to) wonder why we distrust them.Ngl, I'm only leaving reply notifs on for this one to work on my blocklist.
atomicorange@lemmy.world 8 months ago
Yeah there’s some nasty shit here. Big yikes, Lemmy.
some_guy@lemmy.sdf.org 8 months ago
For example, Louisiana mandates a minimum five-year jail sentence no matter the age of the perpetrator.
That’s just on it’s face stupid. A thirteen year old boy is absolutely gonna wanna see girls in his age group naked. That’s not pedophilia. It’s wanting to see the girls he fantasizes about at school every day. Source: I was a thirteen year old boy.
It shouldn’t be treated the same as when an adult man generates it; there should be nuance. I’m not saying it’s ok for a thirteen year old to generate said content: I’m saying tailor the punishment to fit the reality of the differences in motivations. Leave it to Louisiana to once again use a cudgel rather than sense.
I’m so glad I went through puberty at a time when this kind of shit wasn’t available. The thirteen year old version of me would absolutely have got myself in a lot of trouble. And depending on what state I was in, seventeen year old me could have ended listed as a sex predetor for sending dick pics to my gf cause I produced child pornography. God, some states have stupid laws.
AA5B@lemmy.world 8 months ago
In general, even up here in woke-ville, punishments have gotten a lot more strict for kids. There’s a lot more involvement of police, courts, jail. As a parent it causes me a lot of anxiety - whatever happened to school being a “sandbox” where a kid can make mistakes without adult consequences, without ruining their lives? Did that ever exist?
BlackPenguins@lemmy.world 8 months ago
I can already picture that as an Onion headline: New York Renames State to ‘WokeVille’
jwmgregory@lemmy.dbzer0.com 8 months ago
it existed if society liked you enough.
fascists just have a habit of tightening that belt smaller and smaller, is what’s going on.
Agent641@lemmy.world 8 months ago
Punishment for an adult man doing this: Prison
Punishment for a 13 year old by doing this: Publish his browsing and search history in the school newsletter.
GraniteM@lemmy.world 8 months ago
13 year old: “I’ll just take the death penalty, thanks."
lka1988@lemmy.dbzer0.com 8 months ago
I don’t disagree with this assessment.
some_guy@lemmy.sdf.org 8 months ago
Yes, absolutely. But with recognition that a thirteen year old kid isn’t a predator but a horny little kid. I’ll let others determine what that punishment is, but I don’t believe it’s prison. Community service maybe. Written apology. Stuff like that. Second offense, ok, we’re ratcheting up the punishment, but still not adult prison.
danciestlobster@lemmy.zip 8 months ago
I don’t understand fully how this technology works, but, if people are using it to create sexual content of underage individuals, doesn’t that mean the LLM would need to have been trained on sexual content of underage individuals? Seems like going after the company and whatever it’s source material is would be the obvious choice here
wewbull@feddit.uk 8 months ago
You know how when you look at a picture of someone and you cover up the clothed bits, they look naked. Your brain fills in the gaps with what it knows of general human anatomy.
It’s like that.
kayzeekayzee@lemmy.blahaj.zone 8 months ago
I agree with the other comments, but wanted to add how deepfakes work to show how simple they are, and how much less information they need than LLMs.
Step 1: Basically you take a bunch of photos and videos of a specific person, and blur their faces out.
Step 2: This is the hardest step, but still totally feasable for a decent home computer. You train a neural network to un-blur all the faces for that person. Now you have a neural net that’s really good at turning blurry faces into that particular person’s face.
Step 3: Blur the faces in photos/videos of other people and apply your special neural network. It will turn all the blurry faces into the only face it knows how, often with shockingly realistic results.
gkpy@feddit.org 8 months ago
Cheers for the explanation, had no idea that’s how it works.
So it’s even worse than /u/danciestlobster@lemmy.zip thinks, the person creating the deep fake has to have access to CP then if they want to deepfake it!
General_Effort@lemmy.world 8 months ago
This is mostly about swapping faces. You take a video and a photo of someone’s face. Software can replace the face of someone in the video with that face. That’s been around for a decade or so. There are other ways of doing it.
When the face belongs to an underage individual, and the video is pornographic…
LLMs only do text.
lime@feddit.nu 8 months ago
not necessarily. image generation models work on a bore fine-grained scale than that. they can seamlessly combine related concepts, like “photograph”+“person”+“small”+“pose” and generate plausible material due to the fact that all of those concepts have features in common.
you can also use small add-on models trained on very little data (tens to hundreds of images, as compared to millions to billions for a full model) to “steer” the output of a model towards a particular style.
all that being said, the dataset that most of the stable diffusion family of models has in common is medical in nature so there could very well be bad shit in there. it’s like 12 billlion images so it’s hard to check.
Regrettable_incident@lemmy.world 8 months ago
Aren’t there already laws against making child porn?
cley_faye@lemmy.world 8 months ago
I’d rather these laws be against abusing and exploiting child, as well as against ruining their lives. Not only that would be more helpful, it would also work in this case, since actual likeness are involved.
Alas, whether there’s a law against that specific use case or not, it is somewhat difficult to police what people do in their home, without a third party whistleblower. Making more, impossible to apply laws for this specific case does not seem that useful.
Vinstaal0@feddit.nl 8 months ago
There is also a difference between somebody harassing somebody with nude pictures (either real or not) than somebody jerking off to them at home. It does become a problem when an adult masturbated to pictures of children, but children to children. Let’s be honest, they will do it anyway.
LovableSidekick@lemmy.world 8 months ago
Instead of laws keeping up It also might turn out to be a case where culture keeps up.
electric_nan@lemmy.ml 8 months ago
My mama always told me, that if someone makes a deepfake of you, then you make a deepfake of them right back!
lemmydividebyzero@reddthat.com 8 months ago
In the bible, it says, and I quote: “If a deepkfake of you is made, you shall give the creator more material to create deepfakes”
Kolanaki@pawb.social 8 months ago
An eye for an eye, a tooth for a tooth, and a deepfake for a deepfake.
darkkite@lemmy.ml 8 months ago
this advice might get you locked up
electric_nan@lemmy.ml 8 months ago
My mama also told me that if someone locks you up, then you just lock them up right back.
LovableSidekick@lemmy.world 8 months ago
Thanks, cap’n.
RememberTheApollo_@lemmy.world 8 months ago
I’m sure the laws will focus on protecting IP - specifically that of AI companies or megacorps, but not the small creators of content, trademarks, the famous and the powerful.
The rest of us will have to suffer through presenting whatever damaging and humiliating video to a court. If you can even afford a lawyer to do so. Then be offered a judgement that probably won’t be paid or won’t cover the damage done by an image that will never be able to be erased from the internet. Those damages could include the suicide of young people bullied and humiliated by such deepfakes.
OhVenus_Baby@lemmy.ml 8 months ago
I’m fairly well versed in tech and home labbing. I’ve never heard of tools that do this, generate images, etc. Not good ones anyhow. I could use those type of generation for business marketing to develop business cards, marketing materials. NOT FOR PEOPLE GENERATION. Anyone have a list of the best tools? GPT sucks at doing this I’ve tried.
swelter_spark@reddthat.com 8 months ago
Take a look at InvokeAI.
OhVenus_Baby@lemmy.ml 8 months ago
Thanks. Rather than everyone downvoting for no real reason. Finally someone was atleast trying to be helpful. I will use it for super simple business card proofs or basic brochures that’s about it. Magnets, etc.
PumpkinEscobar@lemmy.world 8 months ago
If kids want to be protected they need to get some better lobbyists. /s
mhague@lemmy.world 8 months ago
So is this a way to take away rights by making it about kids?
I mean what the fuck. We did much less and got punished right? It didn’t matter if we were on the property. Schools can hold students accountable for conduct with other students.
The leaded-gas adults of the time had no problem dealing with the emergence of cell phones. It was a distraction. They didn’t need lawmakers to call it something specific. My Pokemon cards caused fights and were banned. No lawmakers needed.
The problem is surely with the interaction between parents and schools. Or maybe it’s just the old way of thinking. Maybe it’s better to have police and courts start taking over discipline in schools.
ColeSloth@discuss.tchncs.de 8 months ago
All your examples are of things that were stopped while at school, so your argument doesn’t really carry over. You still had your pokemon cards everywhere else.
krashmo@lemmy.world 8 months ago
How is a school going to regulate what kids do outside of school property? They could ban cell phones on campus but that’s not going to change what happens after hours.
Lv_InSaNe_vL@lemmy.world 8 months ago
Schools can already do that though. You can get in trouble for bullying outside of school, and when i was a student athletes i had pretty strict restrictions on what i was allowed to do because i was an “ambassador” for the school.
dinckelman@lemmy.world 8 months ago
Lawmakers are grappling with how to address …
Just a reminder that the government is actively voting against regulations on AI, because obviously a lot of these people are pocketing lobbyist money
lka1988@lemmy.dbzer0.com 8 months ago
In the case of US govt, the AI part of the bill they voted against was the part that blocked regulations on AI for a period of 10 years.
In case that wasn’t clear, the US govt voted in favor of regulating AI. 99-1.
shalafi@lemmy.world 8 months ago
A 99-1 vote to drop the anti AI regulation is hardly the government voting against. The Senate smashed that shit hard and fast.
LovableSidekick@lemmy.world 8 months ago
Can’t expect people to know about that 99-1 vote cuz it wasn’t made into a meme.
interdimensionalmeme@lemmy.ml 8 months ago
They want to be regulated so they can finally have their mote. Cutting out the states’ power does mean they will only have to buy one group of politicians in Washington and those are some relatively cheap Hoes
Jax@sh.itjust.works 8 months ago
Oh I just assumed that every Conservative jerks off to kids
Daft_ish@lemmy.dbzer0.com 8 months ago
Get some receipts and that will be a start.
Vinstaal0@feddit.nl 8 months ago
Even in countries a lot less corrupt than the US this is an issue.
Especially because the US government/companies doesn’t do jack shit for people
wewbull@feddit.uk 8 months ago
Honestly I think we need to understand that this is no different to sticking a photo of someone’s head on a porn magazine photo. It’s not real. It’s just less janky.
I would categorise it as sexual harassment, not abuse. Still serious, but a different level
Adderbox76@lemmy.ca 8 months ago
I’m not even going to begin describing all the ways that what you just said is fucked up.
I’ll just point out that online deepfake technology is FAR more accessible to the average 13 year old to use on their peers than “porno mags” were in our day.
You want to compare taking your 13 year old classmates photo off of Facebook, running it through an AI and in five seconds creating photo-realistic adult content featuring them, and compare that to getting your dad’s skin-mag from under his mattress when he’s not home, cutting your classmates face out of a yearbook, taping it on, then sneaking THAT into the computer lab at school so that you can photocopy it and pass it around in home room, and then putting the skin-mag BACK under the mattress before your dad finds out.
Is that right…is THAT what you’re trying to say? Are those the two things that you’re trying say are equivalent?
SheeEttin@lemmy.zip 8 months ago
Yes, we all know it’s fucked up. The point is that we don’t need a new class of laws just because it’s harassment and bullying ✨with AI✨.
LadyAutumn@lemmy.blahaj.zone 8 months ago
Yes, finding out that your peers have been sharing deep fake pornography of you is absolutely fine and a normal thing for young girls to go through in school. No girls have ever killed themselves because of this exact sort of thing, surely. This definitely will not add in any way to the way women and girls are made to feel entirely disgustingly dehumanized by every man or boy in their lives. Groups of men and boys reducing them and their bodies down to vivid sexual fantasies that they can quickly generate photo realistic images of.
If the person in the image is underaged then it should be classified as child pornography. If the woman who’s photo is being used hasnt consented to this then it should be classified as sexual exploitation.
Women and girls have faced degrees of this kind of sexual exploitation by men and boys since the latter of the 20th century. But this is a severe escalation in that behavior. It should be illegal to do this and it should be prosecuted when and where it is found to occur.
General_Effort@lemmy.world 8 months ago
Historically, the respectability of a woman depended on her sexuality. In many conservative cultures and communities, that is still true. Spreading the message that deepfakes are some particular horrible form of harassment reinforces that view.
If having your head on the model of a nude model is a terrible crime, then what does that say about the nude model? What does it say about women who simply happen to develop a larger bosom or lips? What does it say about sex before marriage?
The implicit message here is simply harmful to girls and women.
That doesn’t mean that we should tolerate harassment. But it needs to be understood that we can do no more to stop this kind of harassment than we can do to stop any other kind.
rottingleaf@lemmy.world 8 months ago
This definitely will not add in any way to the way women and girls are made to feel entirely disgustingly dehumanized by every man or boy in their lives. Groups of men and boys reducing them and their bodies down to vivid sexual fantasies that they can quickly generate photo realistic images of.
Sexual attraction doesn’t necessarily involve dehumanization. Unlike most other kinds of interest in a human being, it doesn’t require interest in their personality, but these are logically not the same.
In general you are using emotional arguments for things that work not through emotion, but through literal interpretation. That’s like using metric calculations for a system that expects imperial. Utterly useless.
If the person in the image is underaged then it should be classified as child pornography.
No, it’s not. It’s literally a photorealistic drawing based on a photo (and a dataset to make the generative model). No children have been abused to produce it. Laws work literally.
If the woman who’s photo is being used hasnt consented to this then it should be classified as sexual exploitation.
No, because the woman is not being literally sexually exploited. Her photo being used without consent is, I think, subject of some laws. There are no new fundamental legal entities involved.
Women and girls have faced degrees of this kind of sexual exploitation by men and boys since the latter half of the 20th century. But this is a severe escalation in that behavior. It should be illegal to do this and it should be prosecuted when and where it is found to occur.
I think I agree. But it’s neither child pornography nor sexual exploitation and can’t be equated to them.
There are already existing laws for such actions, similar to using a photo of the victim and a pornographic photo, paper, scissors, pencils and glue. Or, if you think the situation is radically different, there should be new punishable crimes introduced.
Otherwise it’s like punishing everyone caught driving while drunk for non-premeditated murder. One is not the other.
FishFace@lemmy.world 8 months ago
It’s bullying with a sexual element. The fact that it uses AI or deepfakes is secondary, just as it was secondary when it was photoshop, just as it was secondary when it was cutting out photos. It’s always about using it bully someone.
This is different because it’s easier. It’s not really different because it (can be) more realistic, because it was never about being realistic, otherwise blatantly unrealistic images wouldn’t have been used to do it. Indeed, the fact that it can be realistic will help blunt the impact of the leaking of real nudes.
atomicorange@lemmy.world 8 months ago
Thank you. Focusing on the harm the victims is the right way to understand this issue. Too many people in here hunting for a semantic loophole.
lurch@sh.itjust.works 8 months ago
I hope it might lead to a situation of dirty pics/vids not being a problem for the people in it any more, as it could be a deepfake. Like there were cases where a surfacing dirty pic was used for blackmail, ruined someones career or got them kicked out if some comittee, but since it could be fabrication now, I hope this will beva thing of the past, soon.
wewbull@feddit.uk 8 months ago
That could be a socially healthy place to end up at. I don’t see it anytime soon though. Just look at the other response I got.
MagicShel@lemmy.zip 8 months ago
The only defense is to train AI to draw guys with micropenises.
PunkRockSportsFan@fanaticus.social 8 months ago
Burkas for the win ?