Comment on Asking ChatGPT to Repeat Words ‘Forever’ Is Now a Terms of Service Violation

Sibbo@sopuli.xyz ⁨11⁩ ⁨months⁩ ago

• TWeaK@lemm.ee ⁨11⁩ ⁨months⁩ ago

  Exactly this. And how can an AI which “doesn’t have the source material” in its database be able to recall such information?

  source

  • luthis@lemmy.nz ⁨11⁩ ⁨months⁩ ago

    Model is the right term instead of database.

    We learned something about how LLMs work with this… its like a bunch of paintings were chopped up into pixels to use to make other paintings. No one knew it was possible to break the model and have it spit out the pixels of a single painting in order.

    I wonder if diffusion models have some other wierd querks we have yet to discover

    source

    • Jamie@jamie.moe ⁨11⁩ ⁨months⁩ ago

      I’m not an expert, but I would say that it is going to be less likely for a diffusion model to spit out training data in a completely intact way. The way that LLMs versus diffusion models work are very different.

      LLMs work by predicting the next statistically likely token, they take all of the previous text, then predict what the next token will be based on that. So, if you can trick it into a state where the next subsequent tokens are something verbatim from training data, then that’s what you get.

      Diffusion models work by taking a randomly generated latent, combining it with the CLIP interpretation of the user’s prompt, then trying to turn the randomly generated information into a new latent which the VAE will then decode into something a human can see, because the latents the model is dealing with are meaningless numbers to humans.

      In other words, there’s a lot more randomness to deal with in a diffusion model. You could probably get a specific source image back if you specially crafted a latent and a prompt, which one guy did do by basically running img2img on a specific image that was in the training set and giving it a prompt to spit the same image out again. But that required having the original image in the first place, so it’s not really a weakness in the same way this was for GPT.

      source

      • TWeaK@lemm.ee ⁨11⁩ ⁨months⁩ ago

        But the fact is the LLM was able to spit out the training data. This means that anything in the training data isn’t just copied into the training dataset, allegedly under fair use as research, but also copied into the LLM as part of an active commercial product. Sure, the LLM might break it down and store the components separately, but if an LLM can reassemble it and spit out the original copyrighted work then how is that different from how a photocopier breaks down the image scanned from a piece of paper then reassembles it into instructions for its printer?

        source

        • -> View More Comments
    • SkySyrup@sh.itjust.works ⁨11⁩ ⁨months⁩ ago

      The technology of compression a diffusion model would have to achieve to realistically (not to lossily) store “the training data” would be more valuable than the entirety of the machine learning field right now.

      source
  • kpw@kbin.social ⁨11⁩ ⁨months⁩ ago

    The technical term is overfitting.

    source
  • Jordan117@lemmy.world ⁨11⁩ ⁨months⁩ ago

    IIRC based on the source paper the “verbatim” text is common stuff like legal boilerplate, shared code snippets, book jacket blurbs, alphabetical lists of countries, and other text repeated countless times across the web. It’s the text equivalent of DALL-E “memorizing” a meme template or a stock image – it doesn’t mean all or even most of the training data is stored within the model, just that certain pieces of highly duplicated data have ascended to the level of concept and can be reproduced under unusual circumstances.

    source

    • gears@sh.itjust.works ⁨11⁩ ⁨months⁩ ago

      Did you read the article? The verbatim text is, in one example, including email addresses and names (and legal boilerplate) directly from asbestoslaw.com.

      source
    • lemmyvore@feddit.nl ⁨11⁩ ⁨months⁩ ago

      Problem is, they claimed none of it gets stored.

      source

      • TWeaK@lemm.ee ⁨11⁩ ⁨months⁩ ago

        They claim it’s not stored in the LLM, they admit to storing it in the training database but argue fair use under the research exemption.

        This almost makes it seems like the LLM can tap into the training database when it reaches some kind of limit. In which case the training database absolutely should not have a fair use exemption - it’s not just research, but a part of the finished commercial product.

        source
  • Socsa@sh.itjust.works ⁨11⁩ ⁨months⁩ ago

    These models can reach out to the internet to retrieve data and context. It is entirely possible that’s what was happening in this particular case. If I had to guess, this somehow triggered some CI test case which is used to validate this capability.

    source

    • TWeaK@lemm.ee ⁨11⁩ ⁨months⁩ ago

      These models can reach out to the internet to retrieve data and context.

      Then that’s copyright infringement. Just because something is available to read on the internet does not mean your commercial product can copy it.

      source
• seaQueue@lemmy.world ⁨11⁩ ⁨months⁩ ago

  Welcome to the wild West of American data privacy laws. Companies do whatever the fuck they want with whatever data they can beg borrow or steal and then lie about it when regulators come calling.

  source
• CubbyTustard@reddthat.com ⁨11⁩ ⁨months⁩ ago

  [deleted]

  source

  • Gold_E_Lox@lemmy.blahaj.zone ⁨11⁩ ⁨months⁩ ago

    if i stole my neighbours thyme and basil out of their garden, mix them into certain proportions, the resulting spice mix would still be stolen.

    source
• CrayonRosary@lemmy.world ⁨11⁩ ⁨months⁩ ago

  If you put shit on the internet, it’s public. The email addresses in question were probably from Usenet posts which are all public.

  source
• sciencesebi@feddit.ro ⁨11⁩ ⁨months⁩ ago

  What training data?

  source