Teen boys use AI to make fake nudes of classmates, sparking police probe

Comments

• LogicalDrivel@sopuli.xyz ⁨1⁩ ⁨year⁩ ago

  “…and their principal, Asfendis, has vowed to raise awareness on campus of how to use new technologies responsibly.”

  Surely all the teenage boys will understand, and only use the technology for wholesome purposes.

  source

  • otter@lemmy.ca ⁨1⁩ ⁨year⁩ ago

    They might know it’s bad but not fully understand the potential harms. I made another comment on it

    source
• Mac@mander.xyz ⁨1⁩ ⁨year⁩ ago

  There is absolutely no way anyone could have possibly seen this coming.

  source

  • idunnololz@lemmy.world ⁨1⁩ ⁨year⁩ ago

    How do you stop this tho

    source

    • surewhynotlem@lemmy.world ⁨1⁩ ⁨year⁩ ago

      You don’t. Scissors and Polaroid and Playboy have been around for decades. If you wanted to see your classmates face on a nude and photocopy it, you could.

      Now it’s just easier and more believable. But it’s not any more stoppable.

      source

      • -> View More Comments
    • saltesc@lemmy.world ⁨1⁩ ⁨year⁩ ago

      Go back to using natural intelligence and try render with brain.

      source

      • -> View More Comments
• otter@lemmy.ca ⁨1⁩ ⁨year⁩ ago

  The other comment about how this has been happening for a long time (with low tech methods) is true, and it’s also true that we can’t stop this completely. We can still respond to it:

  An immediate and easy focus would be on what they do with the images. Sharing them around is still harassment / bullying and it should be dealt with in the same way as it currently is.

  There’s also an education aspect to it. In the past, those images (magazines, photocopies, photoshop) would be limited in who sees them. The kids now are likely using free online tools that aren’t private or secure, and those images could stick around forever. So it could be good to highlight that

  • Your friends and classmates may see them, and it may harm their lives since the images will stick around. Facial recognition algorithms are also improving, so it’s a legitimate concern that an image stored on a random site somewhere will be tied back to them.
  • The images can be traced back to the creator and the creator can face repercussions for it (for those without empathy, this might be the better selling point

  source

  • Emma_Gold_Man@lemmy.dbzer0.com ⁨1⁩ ⁨year⁩ ago

    Your point 1 seems to forget something important: kids are often cruel, and bullying is frequently the point. So long term consequences for their classmates can be an incentive more than a deterrent.

    source

    • Buddahriffic@lemmy.world ⁨1⁩ ⁨year⁩ ago

      Yeah.

      What teacher says: You shouldn’t do this because it might hurt somebody.

      What some kids hear: Check out this new way to hurt somebody and get horny at the same time. And as an added benefit, you can say it’s only about the first one if admitting the second one would hurt you socially, even if the second one was the whole original point!

      source
  • Adalast@lemmy.world ⁨1⁩ ⁨year⁩ ago

    To your first point, much to the benefit of humanity, and counter to popular belief, the internet is NOT forever. Between link rot, data purges, corporate buyouts, transmission compression losses, and general human stupidity, large swaths of the internet have vanished. Hell, just Macromedia selling out to Adobe ended up causing the loss of most of the popular internet games and videos for anyone in their mid to late 30s at this point (you will be misses Flash). The odds of these specific AI-generated child porn pictures surviving even in some dark corner of the bright web are slim to none. And if they end up surviving in the dark web, well, anyone who sees them will likely have a LOT of explaining to do.

    Also, for the commentary of the websites keeping the images. That is doubtful, beyond holding them in an account-bound locker for the user to retrieve. They don’t care and too many images get generated every day for them to see it as more than reinforcement training.

    Speaking of reinforcement training, they may have been able to use Photoshop’s new generative fill to do this, but to actually generate fresh images of a specific peer they would have had to train a LoRA or Hypernerwork on photos of the girl so the SD could actually resolve it. They weren’t doing that on an AI site, especially not a free one. They were probably using ComfyUI or Automatic1111 (I use both myself). They are free, open source, locally executed software that allow you to use the aforementioned tools when generating. That means that the images were restricted to their local machine, then transferred to a cell phone and distributed to friends.

    www.theatlantic.com/technology/archive/…/619320/

    source
  • cy_narrator@discuss.tchncs.de ⁨1⁩ ⁨year⁩ ago

    I think we should pressure EU to make it such that any online AI photo generating website also uses AI to make sure what was asked is not illegal.

    source
• 6daemonbag@lemmy.dbzer0.com ⁨1⁩ ⁨year⁩ ago

  My niece had this same issue a few years ago but with Photoshop. It absolutely ruined her. Changed schools multiple times (public and private) but social media exists so all the kids knew. She ended up getting homeschooled for the last 5 years of school as well as a fuckload of therapy. She came out the other side okay but she has massive trust issues and anxiety

  source

  • Squizzy@lemmy.world ⁨1⁩ ⁨year⁩ ago

    Man that’s awful, poor thing.

    source
• NightAuthor@lemmy.world ⁨1⁩ ⁨year⁩ ago

  Ai porn to come w this disclaimer: disclaimer

  source
• DasherPack@lemmy.world ⁨1⁩ ⁨year⁩ ago

  In Spain it happened recently with some 12y/olds…it created a country-wide debate, and as always, did not lead to any regulation. Hopefully the EU will do something

  source

  • r3df0x@7.62x54r.ru ⁨1⁩ ⁨year⁩ ago

    This can be prosecuted with existing CP laws.

    source
  • cy_narrator@discuss.tchncs.de ⁨1⁩ ⁨year⁩ ago

    Wait When did Spain join the EU?

    source

    • vanquesse@lemmy.blahaj.zone ⁨1⁩ ⁨year⁩ ago

      in '86

      source
    • DasherPack@lemmy.world ⁨1⁩ ⁨year⁩ ago

      We are a full member, we have the Euro and we’re in schengen

      source
    • Gorgeous_Sloth@sh.itjust.works ⁨1⁩ ⁨year⁩ ago

      Did you just wake up from a 40 years long hangover ? Welcome mate

      source

      • -> View More Comments
    • Corkyskog@sh.itjust.works ⁨1⁩ ⁨year⁩ ago

      It’s harder to remember what countries aren’t or aren’t trying to be in the EU…

      source
    • SatansMaggotyCumFart@lemmy.world ⁨1⁩ ⁨year⁩ ago

      A couple months before the Chernobyl power plant disaster.

      source
• r3df0x@7.62x54r.ru ⁨1⁩ ⁨year⁩ ago

  If you’re making porn of real underage people, I have no problem with you being put on the pedo registry.

  If no serious harm was done, I’m fine with convicting them and then doing full expungement after 5-10 years.

  source

  • wildginger@lemmy.myserv.one ⁨1⁩ ⁨year⁩ ago

    And youre proof that the pedo registry shouldnt exist as is.

    Teenagers being sexually interested in their peers is not pedophilia, and you want to ruin a decade of their life guaranteed, with the “”“”““promise””“”“”" of an expungement that would never actually happen thanks to the permanent nature of the internet for it.

    This misuse of AI is a crime and should be punished and deterred, obviously. But labeling children about to enter the world as pedophiles basically for the rest of their lives?

    Youre kind of a monster.

    source

    • r3df0x@7.62x54r.ru ⁨1⁩ ⁨year⁩ ago

      What about the fact that the girls who are victims of something like this will have to contend with the pictures being online if someone posts them there? What if people who don’t know that the pictures depict minors re-post them to other sites, making them very difficult to remove? That can cause very serious employablity problems. It doesn’t matter how open minded people are, they don’t want porn coming up if someone googles one of their employees.

      source

      • -> View More Comments
    • r3df0x@7.62x54r.ru ⁨1⁩ ⁨year⁩ ago

      If you produce CP, you should be on a registry for producing and distributing CP. If you create CP, you are enabling pedophilia.

      source

      • -> View More Comments
  • Jolteon@lemmy.zip ⁨1⁩ ⁨year⁩ ago

    I’d argue that someone making porn of someone their own age is not pedophilia.

    source

    • r3df0x@7.62x54r.ru ⁨1⁩ ⁨year⁩ ago

      They’re still making porn of a minor. That is harmful to them and it enables any pedophiles who find it.

      source
  • 13esq@lemmy.world ⁨1⁩ ⁨year⁩ ago

    That’s an easy enough judgement when the perpetrator is an adult. What do you do when the perpetrator is a minor themselves? As they are in this article.

    Of course their still needs to be some sort of recourse, but for every other crime there is a difference between being tried as a child or being tried as an adult.

    I find it tough to consider myself.

    source

    • r3df0x@7.62x54r.ru ⁨1⁩ ⁨year⁩ ago

      Considering the consequences for a high school student if porn of them gets circulated, I’m fine with putting them on the registry. Expungement can happen later based on the aftermath. Teenage girls have killed themselves over this sort of thing.

      source
• autotldr@lemmings.world [bot] ⁨1⁩ ⁨year⁩ ago

  This is the best summary I could come up with:

  ---

  This October, boys at Westfield High School in New Jersey started acting “weird,” the Wall Street Journal reported.

  It took four days before the school found out that the boys had been using AI image generators to create and share fake nude photos of female classmates.

  Biden asked the secretary of Commerce, the secretary of Homeland Security, and the heads of other appropriate agencies to provide recommendations regarding “testing and safeguards against” producing “child sexual abuse material” and “non-consensual intimate imagery of real individuals (including intimate digital depictions of the body or body parts of an identifiable individual), for generative AI.”

  “New York State currently lacks the adequate criminal statutes to protect victims of ‘deepfake’ pornography, both adults and children,” Donnelly said.

  Until laws are strengthened, Bramnick has asked the Union County prosecutor to find out what happened at Westfield High School, and state police are still investigating.

  Until the matter is settled in the New Jersey town, the girls plan to keep advocating for victims, and their principal, Asfendis, has vowed to raise awareness on campus of how to use new technologies responsibly.

  ---

  The original article contains 950 words, the summary contains 184 words. Saved 81%. I’m a bot and I’m open source!

  source
• randon31415@lemmy.world ⁨1⁩ ⁨year⁩ ago

  Pictures? We are on the edge of believable videos with AI produced voices and sounds - made on normal computers. Need to clear a few more hurdles in 3D AI modeling, VR, and haptic feedback before this trend reaches it’s obvious conclusion.

  Wonder what crime it would be called if you create a haptic VR double of someone unconsensually and don’t distribute it?

  source
• mwguy@infosec.pub ⁨1⁩ ⁨year⁩ ago

  Given that AI images and media can’t be copyrighted, does the nominal “subject” have any recourse?

  source

  • kadu@lemmy.world ⁨1⁩ ⁨year⁩ ago

    Not sure about other places, but here in Brazil creating a fake nude of somebody and distributing it would be illegal

    source

    • mwguy@infosec.pub ⁨1⁩ ⁨year⁩ ago

      In the US a photoshopped nude would be copyrightable. But courts here have said that AI generated content doesn’t get the benefits of copyright.

      source

      • -> View More Comments
  • time_lord@lemmy.world ⁨1⁩ ⁨year⁩ ago

    There’s the matter of consent, and it might legally be along the same lines of giving someone a roofie so they don’t remember in the morning.

    source
• badbytes@lemmy.world ⁨1⁩ ⁨year⁩ ago

  Thanks authorities, for taking seriously the safety of our students in the classroom. I also saw a kid eat glue.

  source