Submitted 2 months ago by Pro@reddthat.com to technology@lemmy.world
https://19thnews.org/2025/07/deepfake-ai-kids-schools-laws-policy/
Schools and lawmakers are grappling with how to address a new form of peer-on-peer image-based sexual abuse that disproportionately targets girls.
Honestly I think we need to understand that this is no different to sticking a photo of someone’s head on a porn magazine photo. It’s not real. It’s just less janky.
I would categorise it as sexual harassment, not abuse. Still serious, but a different level
Schools generally means it involves underage individuals, which makes any content using them csam. So in effect, the “AI” companies are generating a ton of csam and nobody is doing anything about it.
Disagree. Not CSAM when no abuse has taken place.
That’s my point.
Yes, finding out that your peers have been sharing deep fake pornography of you is absolutely fine and a normal thing for young girls to go through in school. No girls have ever killed themselves because of this exact sort of thing, surely. This definitely will not add in any way to the way women and girls are made to feel entirely disgustingly dehumanized by every man or boy in their lives. Groups of men and boys reducing them and their bodies down to vivid sexual fantasies that they can quickly generate photo realistic images of.
If the person in the image is underaged then it should be classified as child pornography. If the woman who’s photo is being used hasnt consented to this then it should be classified as sexual exploitation.
Women and girls have faced degrees of this kind of sexual exploitation by men and boys since the latter of the 20th century. But this is a severe escalation in that behavior. It should be illegal to do this and it should be prosecuted when and where it is found to occur.
It’s bullying with a sexual element. The fact that it uses AI or deepfakes is secondary, just as it was secondary when it was photoshop, just as it was secondary when it was cutting out photos. It’s always about using it bully someone.
This is different because it’s easier. It’s not really different because it (can be) more realistic, because it was never about being realistic, otherwise blatantly unrealistic images wouldn’t have been used to do it. Indeed, the fact that it can be realistic will help blunt the impact of the leaking of real nudes.
Historically, the respectability of a woman depended on her sexuality. In many conservative cultures and communities, that is still true. Spreading the message that deepfakes are some particular horrible form of harassment reinforces that view.
If having your head on the model of a nude model is a terrible crime, then what does that say about the nude model? What does it say about women who simply happen to develop a larger bosom or lips? What does it say about sex before marriage?
The implicit message here is simply harmful to girls and women.
That doesn’t mean that we should tolerate harassment. But it needs to be understood that we can do no more to stop this kind of harassment than we can do to stop any other kind.
This definitely will not add in any way to the way women and girls are made to feel entirely disgustingly dehumanized by every man or boy in their lives. Groups of men and boys reducing them and their bodies down to vivid sexual fantasies that they can quickly generate photo realistic images of.
Sexual attraction doesn’t necessarily involve dehumanization. Unlike most other kinds of interest in a human being, it doesn’t require interest in their personality, but these are logically not the same.
In general you are using emotional arguments for things that work not through emotion, but through literal interpretation. That’s like using metric calculations for a system that expects imperial. Utterly useless.
If the person in the image is underaged then it should be classified as child pornography.
No, it’s not. It’s literally a photorealistic drawing based on a photo (and a dataset to make the generative model). No children have been abused to produce it. Laws work literally.
If the woman who’s photo is being used hasnt consented to this then it should be classified as sexual exploitation.
No, because the woman is not being literally sexually exploited. Her photo being used without consent is, I think, subject of some laws. There are no new fundamental legal entities involved.
Women and girls have faced degrees of this kind of sexual exploitation by men and boys since the latter half of the 20th century. But this is a severe escalation in that behavior. It should be illegal to do this and it should be prosecuted when and where it is found to occur.
I think I agree. But it’s neither child pornography nor sexual exploitation and can’t be equated to them.
There are already existing laws for such actions, similar to using a photo of the victim and a pornographic photo, paper, scissors, pencils and glue. Or, if you think the situation is radically different, there should be new punishable crimes introduced.
Otherwise it’s like punishing everyone caught driving while drunk for non-premeditated murder. One is not the other.
Thank you. Focusing on the harm the victims is the right way to understand this issue. Too many people in here hunting for a semantic loophole.
I hope it might lead to a situation of dirty pics/vids not being a problem for the people in it any more, as it could be a deepfake. Like there were cases where a surfacing dirty pic was used for blackmail, ruined someones career or got them kicked out if some comittee, but since it could be fabrication now, I hope this will beva thing of the past, soon.
That could be a socially healthy place to end up at. I don’t see it anytime soon though. Just look at the other response I got.
I’m not even going to begin describing all the ways that what you just said is fucked up.
I’ll just point out that online deepfake technology is FAR more accessible to the average 13 year old to use on their peers than “porno mags” were in our day.
You want to compare taking your 13 year old classmates photo off of Facebook, running it through an AI and in five seconds creating photo-realistic adult content featuring them, and compare that to getting your dad’s skin-mag from under his mattress when he’s not home, cutting your classmates face out of a yearbook, taping it on, then sneaking THAT into the computer lab at school so that you can photocopy it and pass it around in home room, and then putting the skin-mag BACK under the mattress before your dad finds out.
Is that right…is THAT what you’re trying to say? Are those the two things that you’re trying say are equivalent?
Yes, we all know it’s fucked up. The point is that we don’t need a new class of laws just because it’s harassment and bullying ✨with AI✨.
For example, Louisiana mandates a minimum five-year jail sentence no matter the age of the perpetrator.
That’s just on it’s face stupid. A thirteen year old boy is absolutely gonna wanna see girls in his age group naked. That’s not pedophilia. It’s wanting to see the girls he fantasizes about at school every day. Source: I was a thirteen year old boy.
It shouldn’t be treated the same as when an adult man generates it; there should be nuance. I’m not saying it’s ok for a thirteen year old to generate said content: I’m saying tailor the punishment to fit the reality of the differences in motivations. Leave it to Louisiana to once again use a cudgel rather than sense.
I’m so glad I went through puberty at a time when this kind of shit wasn’t available. The thirteen year old version of me would absolutely have got myself in a lot of trouble. And depending on what state I was in, seventeen year old me could have ended listed as a sex predetor for sending dick pics to my gf cause I produced child pornography. God, some states have stupid laws.
I don’t disagree with this assessment.
Yes, absolutely. But with recognition that a thirteen year old kid isn’t a predator but a horny little kid. I’ll let others determine what that punishment is, but I don’t believe it’s prison. Community service maybe. Written apology. Stuff like that. Second offense, ok, we’re ratcheting up the punishment, but still not adult prison.
In general, even up here in woke-ville, punishments have gotten a lot more strict for kids. There’s a lot more involvement of police, courts, jail. As a parent it causes me a lot of anxiety - whatever happened to school being a “sandbox” where a kid can make mistakes without adult consequences, without ruining their lives? Did that ever exist?
it existed if society liked you enough.
fascists just have a habit of tightening that belt smaller and smaller, is what’s going on.
I can already picture that as an Onion headline: New York Renames State to ‘WokeVille’
Punishment for an adult man doing this: Prison
Punishment for a 13 year old by doing this: Publish his browsing and search history in the school newsletter.
13 year old: “I’ll just take the death penalty, thanks."
probably because there's a rapist in the white house.
To add to that. I live in a red area and since the election I’ve been cat called much more. And it’s weird too, cus I’m middle aged…. I thought I’d finally disappear…
the toxic manosphere/blogosphere/whatever it's called has done so much lifelong damage
At least they’ve learned a skill?
God I’m glad I’m not a kid now. I never would have survived.
In my case, other kids would not have survived trying to pull off shit like this. So yeah, I’m also glad I’m not a kid anymore.
Jfc the replies here are fucking rancid. Lemmy is full of sweaty middle aged blokes in tech who hate it when anyone tells them that grown men who pursue teenage girls who have just reached an arbitrary age are fucking creeps, so of course they're here encouraging the next generation of misogynist scum by defending this shit, too.
And men (pretend to) wonder why we distrust them.
Ngl, I'm only leaving reply notifs on for this one to work on my blocklist.
Yeah there’s some nasty shit here. Big yikes, Lemmy.
I’m sure the laws will focus on protecting IP - specifically that of AI companies or megacorps, but not the small creators of content, trademarks, the famous and the powerful.
The rest of us will have to suffer through presenting whatever damaging and humiliating video to a court. If you can even afford a lawyer to do so. Then be offered a judgement that probably won’t be paid or won’t cover the damage done by an image that will never be able to be erased from the internet. Those damages could include the suicide of young people bullied and humiliated by such deepfakes.
So is this a way to take away rights by making it about kids?
I mean what the fuck. We did much less and got punished right? It didn’t matter if we were on the property. Schools can hold students accountable for conduct with other students.
The leaded-gas adults of the time had no problem dealing with the emergence of cell phones. It was a distraction. They didn’t need lawmakers to call it something specific. My Pokemon cards caused fights and were banned. No lawmakers needed.
The problem is surely with the interaction between parents and schools. Or maybe it’s just the old way of thinking. Maybe it’s better to have police and courts start taking over discipline in schools.
How is a school going to regulate what kids do outside of school property? They could ban cell phones on campus but that’s not going to change what happens after hours.
Schools can already do that though. You can get in trouble for bullying outside of school, and when i was a student athletes i had pretty strict restrictions on what i was allowed to do because i was an “ambassador” for the school.
All your examples are of things that were stopped while at school, so your argument doesn’t really carry over. You still had your pokemon cards everywhere else.
My mama always told me, that if someone makes a deepfake of you, then you make a deepfake of them right back!
In the bible, it says, and I quote: “If a deepkfake of you is made, you shall give the creator more material to create deepfakes”
An eye for an eye, a tooth for a tooth, and a deepfake for a deepfake.
this advice might get you locked up
My mama also told me that if someone locks you up, then you just lock them up right back.
Thanks, cap’n.
Maybe let’s assume all digital images are fake and go back to painting. Wait… what if children start painting deepfakes ?
Or pasting someone’s photo over a naked body…in their minds…
Welp, if I had kids they would have one of those scramble suits like in a scanner darkly.
It would of course be their choice to wear them but Id definitely look for ways to limit their time in areas with cameras present.
That’s just called the outside now. Assume you are on camera at all times the moment you step out the front door. To be safe in the surveillance we live in today, best act as though you are being recorded in your own home as well.
best act as though you are being recorded in your own home as well.
If you don’t know, don’t try? Seems a bit defeatist.
You can make areas safe from cameras. No, you cant make everywhere camera free but you can minimize your time in those areas. Im not saying its a good system it would just be adjusting to the times.
If the floor would lava and all that…
Back in my day we just had to use our own imagination.
Can’t afford this much cheese today to find just the right slice for every bikini photo…
I don’t understand fully how this technology works, but, if people are using it to create sexual content of underage individuals, doesn’t that mean the LLM would need to have been trained on sexual content of underage individuals? Seems like going after the company and whatever it’s source material is would be the obvious choice here
I agree with the other comments, but wanted to add how deepfakes work to show how simple they are, and how much less information they need than LLMs.
Step 1: Basically you take a bunch of photos and videos of a specific person, and blur their faces out.
Step 2: This is the hardest step, but still totally feasable for a decent home computer. You train a neural network to un-blur all the faces for that person. Now you have a neural net that’s really good at turning blurry faces into that particular person’s face.
Step 3: Blur the faces in photos/videos of other people and apply your special neural network. It will turn all the blurry faces into the only face it knows how, often with shockingly realistic results.
Cheers for the explanation, had no idea that’s how it works.
So it’s even worse than /u/danciestlobster@lemmy.zip thinks, the person creating the deep fake has to have access to CP then if they want to deepfake it!
not necessarily. image generation models work on a bore fine-grained scale than that. they can seamlessly combine related concepts, like “photograph”+“person”+“small”+“pose” and generate plausible material due to the fact that all of those concepts have features in common.
you can also use small add-on models trained on very little data (tens to hundreds of images, as compared to millions to billions for a full model) to “steer” the output of a model towards a particular style.
all that being said, the dataset that most of the stable diffusion family of models has in common is medical in nature so there could very well be bad shit in there. it’s like 12 billlion images so it’s hard to check.
This is mostly about swapping faces. You take a video and a photo of someone’s face. Software can replace the face of someone in the video with that face. That’s been around for a decade or so. There are other ways of doing it.
When the face belongs to an underage individual, and the video is pornographic…
LLMs only do text.
You know how when you look at a picture of someone and you cover up the clothed bits, they look naked. Your brain fills in the gaps with what it knows of general human anatomy.
It’s like that.
The only defense is to train AI to draw guys with micropenises.
I think the micropenis thing wold just encourage this further
It’s obviously not a serious suggestion, but the reality is the tools are out there and Pandora’s box can’t be put back on the shelf. Kids can’t be held accountable in a meaningful way. This is just an issue we are going to face basically forever now.
There is a window of time during which most kids are little sociopaths and you can’t appeal to any better nature. They have the means and often no internal or external restraint. And so mutually assured destruction is my tongue in cheek answer.
Deepfakes might end up being the modern version of a bikini. In the olden days, people wore these to the beach. Having less was scandalous and moral decay. Yet, now we wear much less.
Our grandchildren might simply not give a damn about their nudity, because it is assumed that everyone is deepfaking everyone.
These are all worn voluntarily. This issue isn’t about the equivalent of scandalously clad young girls, it’s like if girls were being involuntarily stripped of their clothing by their classmates. It’s not about modesty standards it’s about sexual abuse.
It can be both. The cornerstone of why nudity can be abused, is that society makes it shameful to be bare. If some generations from now that people can just shrug and not care, that is one less tool an abuser can use against people.
In any case, I am of the mind that people of my generation might be doing their own version of the Satanic Panic, or the reaction against rap music. For better or worse, older people cannot relate to the younger.
Unless it is used to pretend that it is a real video and circulated for denigration or blackmail, it is very much not at all like assault. And also, deepfakes do not have the special features hidden under your clothes, so it is possible to debunk those if you really have to.
Aren’t there already laws against making child porn?
I’d rather these laws be against abusing and exploiting child, as well as against ruining their lives. Not only that would be more helpful, it would also work in this case, since actual likeness are involved.
Alas, whether there’s a law against that specific use case or not, it is somewhat difficult to police what people do in their home, without a third party whistleblower. Making more, impossible to apply laws for this specific case does not seem that useful.
There is also a difference between somebody harassing somebody with nude pictures (either real or not) than somebody jerking off to them at home. It does become a problem when an adult masturbated to pictures of children, but children to children. Let’s be honest, they will do it anyway.
If kids want to be protected they need to get some better lobbyists. /s
Instead of laws keeping up It also might turn out to be a case where culture keeps up.
Burkas for the win ?
anyone using any kind of AI either doesn’t know how consent works-- or they don’t care about it.
a horrifying development in the intersection of technofascism and rape culture
I’m fairly well versed in tech and home labbing. I’ve never heard of tools that do this, generate images, etc. Not good ones anyhow. I could use those type of generation for business marketing to develop business cards, marketing materials. NOT FOR PEOPLE GENERATION. Anyone have a list of the best tools? GPT sucks at doing this I’ve tried.
dinckelman@lemmy.world 2 months ago
Just a reminder that the government is actively voting against regulations on AI, because obviously a lot of these people are pocketing lobbyist money
Jax@sh.itjust.works 2 months ago
Oh I just assumed that every Conservative jerks off to kids
Daft_ish@lemmy.dbzer0.com 2 months ago
Get some receipts and that will be a start.
shalafi@lemmy.world 2 months ago
A 99-1 vote to drop the anti AI regulation is hardly the government voting against. The Senate smashed that shit hard and fast.
LovableSidekick@lemmy.world 2 months ago
Can’t expect people to know about that 99-1 vote cuz it wasn’t made into a meme.
Vinstaal0@feddit.nl 2 months ago
Even in countries a lot less corrupt than the US this is an issue.
Especially because the US government/companies doesn’t do jack shit for people
lka1988@lemmy.dbzer0.com 2 months ago
In the case of US govt, the AI part of the bill they voted against was the part that blocked regulations on AI for a period of 10 years.
In case that wasn’t clear, the US govt voted in favor of regulating AI. 99-1.
interdimensionalmeme@lemmy.ml 2 months ago
They want to be regulated so they can finally have their mote. Cutting out the states’ power does mean they will only have to buy one group of politicians in Washington and those are some relatively cheap Hoes