Kids are making deepfakes of each other, and laws aren’t keeping up

Submitted ⁨⁨22⁩ ⁨hours⁩ ago⁩ by ⁨Pro@reddthat.com⁩ to ⁨technology@lemmy.world⁩

https://19thnews.org/2025/07/deepfake-ai-kids-schools-laws-policy/

Schools and lawmakers are grappling with how to address a new form of peer-on-peer image-based sexual abuse that disproportionately targets girls.

source

Comments

Sort:hotnew top

dinckelman@lemmy.world ⁨21⁩ ⁨hours⁩ ago

Lawmakers are grappling with how to address …

Just a reminder that the government is actively voting against regulations on AI, because obviously a lot of these people are pocketing lobbyist money

source
- Jax@sh.itjust.works ⁨20⁩ ⁨hours⁩ ago
  Oh I just assumed that every Conservative jerks off to kids
  
  source
  - Daft_ish@lemmy.dbzer0.com ⁨7⁩ ⁨hours⁩ ago
    Get some receipts and that will be a start.
    
    source
    -> View More Comments
- shalafi@lemmy.world ⁨17⁩ ⁨hours⁩ ago
  A 99-1 vote to drop the anti AI regulation is hardly the government voting against. The Senate smashed that shit hard and fast.
  
  source
  - LovableSidekick@lemmy.world ⁨14⁩ ⁨hours⁩ ago
    Can’t expect people to know about that 99-1 vote cuz it wasn’t made into a meme.
    
    source
    -> View More Comments
- Vinstaal0@feddit.nl ⁨20⁩ ⁨hours⁩ ago
  Even in countries a lot less corrupt than the US this is an issue.
  
  Especially because the US government/companies doesn’t do jack shit for people
  
  source
- lka1988@lemmy.dbzer0.com ⁨9⁩ ⁨hours⁩ ago
  In the case of US govt, the AI part of the bill they voted against was the part that blocked regulations on AI for a period of 10 years.
  
  In case that wasn’t clear, the US govt voted in favor of regulating AI. 99-1.
  
  source
- interdimensionalmeme@lemmy.ml ⁨17⁩ ⁨hours⁩ ago
  They want to be regulated so they can finally have their mote. Cutting out the states’ power does mean they will only have to buy one group of politicians in Washington and those are some relatively cheap Hoes
  
  source
some_guy@lemmy.sdf.org ⁨11⁩ ⁨hours⁩ ago

For example, Louisiana mandates a minimum five-year jail sentence no matter the age of the perpetrator.

That’s just on it’s face stupid. A thirteen year old boy is absolutely gonna wanna see girls in his age group naked. That’s not pedophilia. It’s wanting to see the girls he fantasizes about at school every day. Source: I was a thirteen year old boy.

It shouldn’t be treated the same as when an adult man generates it; there should be nuance. I’m not saying it’s ok for a thirteen year old to generate said content: I’m saying tailor the punishment to fit the reality of the differences in motivations. Leave it to Louisiana to once again use a cudgel rather than sense.

I’m so glad I went through puberty at a time when this kind of shit wasn’t available. The thirteen year old version of me would absolutely have got myself in a lot of trouble. And depending on what state I was in, seventeen year old me could have ended listed as a sex predetor for sending dick pics to my gf cause I produced child pornography. God, some states have stupid laws.

source
- Agent641@lemmy.world ⁨9⁩ ⁨hours⁩ ago
  Punishment for an adult man doing this: Prison
  
  Punishment for a 13 year old by doing this: Publish his browsing and search history in the school newsletter.
  
  source
- lka1988@lemmy.dbzer0.com ⁨9⁩ ⁨hours⁩ ago
  I don’t disagree with this assessment.
  
  source
  - seralth@lemmy.world ⁨4⁩ ⁨hours⁩ ago
    There is a difference between ruining the life of a 13 year old boy for the rest of his life with no recourse and no expectations.
    
    Vs scaring the shit out of them and making them work their ass off doing an ass load of community service for a summer.
    
    source
    -> View More Comments
  - some_guy@lemmy.sdf.org ⁨7⁩ ⁨hours⁩ ago
    Yes, absolutely. But with recognition that a thirteen year old kid isn’t a predator but a horny little kid. I’ll let others determine what that punishment is, but I don’t believe it’s prison. Community service maybe. Written apology. Stuff like that. Second offense, ok, we’re ratcheting up the punishment, but still not adult prison.
    
    source
    -> View More Comments
2ugly2live@lemmy.world ⁨5⁩ ⁨hours⁩ ago
God I’m glad I’m not a kid now. I never would have survived.

source
wewbull@feddit.uk ⁨21⁩ ⁨hours⁩ ago
Honestly I think we need to understand that this is no different to sticking a photo of someone’s head on a porn magazine photo. It’s not real. It’s just less janky.

I would categorise it as sexual harassment, not abuse. Still serious, but a different level

source
- lath@lemmy.world ⁨21⁩ ⁨hours⁩ ago
  Schools generally means it involves underage individuals, which makes any content using them csam. So in effect, the “AI” companies are generating a ton of csam and nobody is doing anything about it.
  
  source
  - wewbull@feddit.uk ⁨21⁩ ⁨hours⁩ ago
    Disagree. Not CSAM when no abuse has taken place.
    
    That’s my point.
    
    source
    -> View More Comments
- LadyAutumn@lemmy.blahaj.zone ⁨20⁩ ⁨hours⁩ ago
  Yes, finding out that your peers have been sharing deep fake pornography of you is absolutely fine and a normal thing for young girls to go through in school. No girls have ever killed themselves because of this exact sort of thing, surely. This definitely will not add in any way to the way women and girls are made to feel entirely disgustingly dehumanized by every man or boy in their lives. Groups of men and boys reducing them and their bodies down to vivid sexual fantasies that they can quickly generate photo realistic images of.
  
  If the person in the image is underaged then it should be classified as child pornography. If the woman who’s photo is being used hasnt consented to this then it should be classified as sexual exploitation.
  
  Women and girls have faced degrees of this kind of sexual exploitation by men and boys since the latter of the 20th century. But this is a severe escalation in that behavior. It should be illegal to do this and it should be prosecuted when and where it is found to occur.
  
  source
  - FishFace@lemmy.world ⁨20⁩ ⁨hours⁩ ago
    It’s bullying with a sexual element. The fact that it uses AI or deepfakes is secondary, just as it was secondary when it was photoshop, just as it was secondary when it was cutting out photos. It’s always about using it bully someone.
    
    This is different because it’s easier. It’s not really different because it (can be) more realistic, because it was never about being realistic, otherwise blatantly unrealistic images wouldn’t have been used to do it. Indeed, the fact that it can be realistic will help blunt the impact of the leaking of real nudes.
    
    source
    -> View More Comments
  - General_Effort@lemmy.world ⁨18⁩ ⁨hours⁩ ago
    Historically, the respectability of a woman depended on her sexuality. In many conservative cultures and communities, that is still true. Spreading the message that deepfakes are some particular horrible form of harassment reinforces that view.
    
    If having your head on the model of a nude model is a terrible crime, then what does that say about the nude model? What does it say about women who simply happen to develop a larger bosom or lips? What does it say about sex before marriage?
    
    The implicit message here is simply harmful to girls and women.
    
    That doesn’t mean that we should tolerate harassment. But it needs to be understood that we can do no more to stop this kind of harassment than we can do to stop any other kind.
    
    source
    -> View More Comments
  - atomicorange@lemmy.world ⁨20⁩ ⁨hours⁩ ago
    Thank you. Focusing on the harm the victims is the right way to understand this issue. Too many people in here hunting for a semantic loophole.
    
    source
  - rottingleaf@lemmy.world ⁨19⁩ ⁨hours⁩ ago
    
    This definitely will not add in any way to the way women and girls are made to feel entirely disgustingly dehumanized by every man or boy in their lives. Groups of men and boys reducing them and their bodies down to vivid sexual fantasies that they can quickly generate photo realistic images of.
    
    Sexual attraction doesn’t necessarily involve dehumanization. Unlike most other kinds of interest in a human being, it doesn’t require interest in their personality, but these are logically not the same.
    
    In general you are using emotional arguments for things that work not through emotion, but through literal interpretation. That’s like using metric calculations for a system that expects imperial. Utterly useless.
    
    If the person in the image is underaged then it should be classified as child pornography.
    
    No, it’s not. It’s literally a photorealistic drawing based on a photo (and a dataset to make the generative model). No children have been abused to produce it. Laws work literally.
    
    If the woman who’s photo is being used hasnt consented to this then it should be classified as sexual exploitation.
    
    No, because the woman is not being literally sexually exploited. Her photo being used without consent is, I think, subject of some laws. There are no new fundamental legal entities involved.
    
    Women and girls have faced degrees of this kind of sexual exploitation by men and boys since the latter half of the 20th century. But this is a severe escalation in that behavior. It should be illegal to do this and it should be prosecuted when and where it is found to occur.
    
    I think I agree. But it’s neither child pornography nor sexual exploitation and can’t be equated to them.
    
    There are already existing laws for such actions, similar to using a photo of the victim and a pornographic photo, paper, scissors, pencils and glue. Or, if you think the situation is radically different, there should be new punishable crimes introduced.
    
    Otherwise it’s like punishing everyone caught driving while drunk for non-premeditated murder. One is not the other.
    
    source
    -> View More Comments
- Adderbox76@lemmy.ca ⁨17⁩ ⁨hours⁩ ago
  I’m not even going to begin describing all the ways that what you just said is fucked up.
  
  I’ll just point out that online deepfake technology is FAR more accessible to the average 13 year old to use on their peers than “porno mags” were in our day.
  
  You want to compare taking your 13 year old classmates photo off of Facebook, running it through an AI and in five seconds creating photo-realistic adult content featuring them, and compare that to getting your dad’s skin-mag from under his mattress when he’s not home, cutting your classmates face out of a yearbook, taping it on, then sneaking THAT into the computer lab at school so that you can photocopy it and pass it around in home room, and then putting the skin-mag BACK under the mattress before your dad finds out.
  
  Is that right…is THAT what you’re trying to say? Are those the two things that you’re trying say are equivalent?
  
  source
  - SheeEttin@lemmy.zip ⁨16⁩ ⁨hours⁩ ago
    Yes, we all know it’s fucked up. The point is that we don’t need a new class of laws just because it’s harassment and bullying ✨with AI✨.
    
    source
- lurch@sh.itjust.works ⁨21⁩ ⁨hours⁩ ago
  I hope it might lead to a situation of dirty pics/vids not being a problem for the people in it any more, as it could be a deepfake. Like there were cases where a surfacing dirty pic was used for blackmail, ruined someones career or got them kicked out if some comittee, but since it could be fabrication now, I hope this will beva thing of the past, soon.
  
  source
  - wewbull@feddit.uk ⁨21⁩ ⁨hours⁩ ago
    That could be a socially healthy place to end up at. I don’t see it anytime soon though. Just look at the other response I got.
    
    source
    -> View More Comments
Daft_ish@lemmy.dbzer0.com ⁨6⁩ ⁨hours⁩ ago
Welp, if I had kids they would have one of those scramble suits like in a scanner darkly.

It would of course be their choice to wear them but Id definitely look for ways to limit their time in areas with cameras present.

source
Walk_blesseD@piefed.blahaj.zone ⁨11⁩ ⁨hours⁩ ago
Jfc the replies here are fucking rancid. Lemmy is full of sweaty middle aged blokes in tech who hate it when anyone tells them that grown men who pursue teenage girls who have just reached an arbitrary age are fucking creeps, so of course they're here encouraging the next generation of misogynist scum by defending this shit, too.
And men (pretend to) wonder why we distrust them.

Ngl, I'm only leaving reply notifs on for this one to work on my blocklist.

source
- atomicorange@lemmy.world ⁨10⁩ ⁨hours⁩ ago
  Yeah there’s some nasty shit here. Big yikes, Lemmy.
  
  source
RememberTheApollo_@lemmy.world ⁨18⁩ ⁨hours⁩ ago
I’m sure the laws will focus on protecting IP - specifically that of AI companies or megacorps, but not the small creators of content, trademarks, the famous and the powerful.

The rest of us will have to suffer through presenting whatever damaging and humiliating video to a court. If you can even afford a lawyer to do so. Then be offered a judgement that probably won’t be paid or won’t cover the damage done by an image that will never be able to be erased from the internet. Those damages could include the suicide of young people bullied and humiliated by such deepfakes.

source
electric_nan@lemmy.ml ⁨17⁩ ⁨hours⁩ ago
My mama always told me, that if someone makes a deepfake of you, then you make a deepfake of them right back!

source
- lemmydividebyzero@reddthat.com ⁨14⁩ ⁨hours⁩ ago
  In the bible, it says, and I quote: “If a deepkfake of you is made, you shall give the creator more material to create deepfakes”
  
  source
  - Kolanaki@pawb.social ⁨13⁩ ⁨hours⁩ ago
    An eye for an eye, a tooth for a tooth, and a deepfake for a deepfake.
    
    source
- darkkite@lemmy.ml ⁨14⁩ ⁨hours⁩ ago
  this advice might get you locked up
  
  source
  - electric_nan@lemmy.ml ⁨13⁩ ⁨hours⁩ ago
    My mama also told me that if someone locks you up, then you just lock them up right back.
    
    source
- LovableSidekick@lemmy.world ⁨14⁩ ⁨hours⁩ ago
  Thanks, cap’n.
  
  source
mhague@lemmy.world ⁨20⁩ ⁨hours⁩ ago
So is this a way to take away rights by making it about kids?

I mean what the fuck. We did much less and got punished right? It didn’t matter if we were on the property. Schools can hold students accountable for conduct with other students.

The leaded-gas adults of the time had no problem dealing with the emergence of cell phones. It was a distraction. They didn’t need lawmakers to call it something specific. My Pokemon cards caused fights and were banned. No lawmakers needed.

The problem is surely with the interaction between parents and schools. Or maybe it’s just the old way of thinking. Maybe it’s better to have police and courts start taking over discipline in schools.

source
- krashmo@lemmy.world ⁨18⁩ ⁨hours⁩ ago
  How is a school going to regulate what kids do outside of school property? They could ban cell phones on campus but that’s not going to change what happens after hours.
  
  source
  - Lv_InSaNe_vL@lemmy.world ⁨16⁩ ⁨hours⁩ ago
    Schools can already do that though. You can get in trouble for bullying outside of school, and when i was a student athletes i had pretty strict restrictions on what i was allowed to do because i was an “ambassador” for the school.
    
    source
- ColeSloth@discuss.tchncs.de ⁨14⁩ ⁨hours⁩ ago
  All your examples are of things that were stopped while at school, so your argument doesn’t really carry over. You still had your pokemon cards everywhere else.
  
  source
danciestlobster@lemmy.zip ⁨13⁩ ⁨hours⁩ ago
I don’t understand fully how this technology works, but, if people are using it to create sexual content of underage individuals, doesn’t that mean the LLM would need to have been trained on sexual content of underage individuals? Seems like going after the company and whatever it’s source material is would be the obvious choice here

source
- kayzeekayzee@lemmy.blahaj.zone ⁨5⁩ ⁨hours⁩ ago
  I agree with the other comments, but wanted to add how deepfakes work to show how simple they are, and how much less information they need than LLMs.
  
  Step 1: Basically you take a bunch of photos and videos of a specific person, and blur their faces out.
  
  Step 2: This is the hardest step, but still totally feasable for a decent home computer. You train a neural network to un-blur all the faces for that person. Now you have a neural net that’s really good at turning blurry faces into that particular person’s face.
  
  Step 3: Blur the faces in photos/videos of other people and apply your special neural network. It will turn all the blurry faces into the only face it knows how, often with shockingly realistic results.
  
  source
  - gkpy@feddit.org ⁨3⁩ ⁨hours⁩ ago
    Cheers for the explanation, had no idea that’s how it works.
    
    So it’s even worse than /u/danciestlobster@lemmy.zip thinks, the person creating the deep fake has to have access to CP then if they want to deepfake it!
    
    source
- lime@feddit.nu ⁨11⁩ ⁨hours⁩ ago
  not necessarily. image generation models work on a bore fine-grained scale than that. they can seamlessly combine related concepts, like “photograph”+“person”+“small”+“pose” and generate plausible material due to the fact that all of those concepts have features in common.
  
  you can also use small add-on models trained on very little data (tens to hundreds of images, as compared to millions to billions for a full model) to “steer” the output of a model towards a particular style.
  
  all that being said, the dataset that most of the stable diffusion family of models has in common is medical in nature so there could very well be bad shit in there. it’s like 12 billlion images so it’s hard to check.
  
  source
- General_Effort@lemmy.world ⁨11⁩ ⁨hours⁩ ago
  This is mostly about swapping faces. You take a video and a photo of someone’s face. Software can replace the face of someone in the video with that face. That’s been around for a decade or so. There are other ways of doing it.
  
  When the face belongs to an underage individual, and the video is pornographic…
  
  LLMs only do text.
  
  source
Regrettable_incident@lemmy.world ⁨14⁩ ⁨hours⁩ ago
Aren’t there already laws against making child porn?

source
- cley_faye@lemmy.world ⁨14⁩ ⁨hours⁩ ago
  I’d rather these laws be against abusing and exploiting child, as well as against ruining their lives. Not only that would be more helpful, it would also work in this case, since actual likeness are involved.
  
  Alas, whether there’s a law against that specific use case or not, it is somewhat difficult to police what people do in their home, without a third party whistleblower. Making more, impossible to apply laws for this specific case does not seem that useful.
  
  source
MagicShel@lemmy.zip ⁨22⁩ ⁨hours⁩ ago
The only defense is to train AI to draw guys with micropenises.

source
- wise_pancake@lemmy.ca ⁨22⁩ ⁨hours⁩ ago
  I think the micropenis thing wold just encourage this further
  
  source
  - MagicShel@lemmy.zip ⁨21⁩ ⁨hours⁩ ago
    It’s obviously not a serious suggestion, but the reality is the tools are out there and Pandora’s box can’t be put back on the shelf. Kids can’t be held accountable in a meaningful way. This is just an issue we are going to face basically forever now.
    
    There is a window of time during which most kids are little sociopaths and you can’t appeal to any better nature. They have the means and often no internal or external restraint. And so mutually assured destruction is my tongue in cheek answer.
    
    source
    -> View More Comments
PumpkinEscobar@lemmy.world ⁨19⁩ ⁨hours⁩ ago
If kids want to be protected they need to get some better lobbyists. /s

source
LovableSidekick@lemmy.world ⁨14⁩ ⁨hours⁩ ago
Instead of laws keeping up It also might turn out to be a case where culture keeps up.

source
OhVenus_Baby@lemmy.ml ⁨18⁩ ⁨hours⁩ ago
I’m fairly well versed in tech and home labbing. I’ve never heard of tools that do this, generate images, etc. Not good ones anyhow. I could use those type of generation for business marketing to develop business cards, marketing materials. NOT FOR PEOPLE GENERATION. Anyone have a list of the best tools? GPT sucks at doing this I’ve tried.

source