Comment on The Irony of 'You Wouldn't Download a Car' Making a Comeback in AI Debates
Zacryon@feddit.org 1 week ago
When AI systems ingest copyrighted works, they’re extracting general patterns and concepts - the “Bob Dylan-ness” or “Hemingway-ness” - not copying specific text or images.
Okay.
jimmycrackcrack@lemmy.world 1 week ago
I’m confused exactly what you’re saying here. It does seem from your experiment that if you specifically ask it to, Chat GPT can reproduce selected pieces of copyrighted creative works verbatim, but what’s your point? You posted the screenshots underneath a quote about how AI systems extract patterns from works rather than copying them so I guess you want to show that it can at times in fact just copy things despite this seeming claim to the opposite, but the fact that you prompted the system to do it seems to kind of dilute this point a bit. In any case, it’s not just reproducing the work, it’s producing output that is relevant to your naturally phrased English language output, and selecting which particular passage in a way that is specifically relevant to the way your input was phrased and also adding additional output aside from the quoted passage which is also relevant and unique to the prompt.
The developers make the analogy of a person being influenced by works in the creation of their own and that that is considered acceptable. If you asked Bob Dylan to cite a passage from a work by Hemingway and he successfully remembered such a passage and in the correct context recited it to you verbatim, followed by an explanation for why it’s a good passage to have selected, you wouldn’t take from that exchange that this was proof that Bob Dylan was not really actually ‘influenced’ by anything but was instead just cobbling together the work of others when he produces his music. If anything, it’d likely be regarded as a mark of how well read Bob Dylan must be that he could remember the passage so accurately and choose a passage that so successfully fits the brief of your request. I don’t typically want to leap to the defence of these AI models that wholesale take in so much creative work and mechanistically re-assemble it without compensation nor input from the artist but I wouldn’t pretend that it’s not an issue with at least a little nuance to it and I can’t see what these screenshots prove.
Dasus@lemmy.world 1 week ago
OpenAI is arguing “we’re not using copyrighted works in a way which would require us to pay anything, the machine is merely extrapolating patterns”.
But then it does go on to quote materials verbatim, which shows it’s not “just” ‘extracting patterns’.
If I were to put up a service called “quote a book” or something, and it just had a non-AI bot which would — when given the book and pages — quote copyrighted works, would that be okay for me to make money on, without paying anyone I’m quoting? Even if they started to use my service to literally copy entire books?
Why are you defending massive corporations who could just pay up? Isn’t the whole “corporations putting profits over anything” thing a bit… seen already?
suy@programming.dev 1 week ago
Is is just extracting patterns. Is making statistical samples of which token (“word”, informally speaking) is likely followed given the previous stream.
It can only reproduce passages of things it has seen many, many times. I cannot reproduce the whole work. Those two quotes can be seen elsewhere on the internet plenty of times. And it’s fair use there, so it would be fair use with a chat bot as well.
There have been papers published where researchers were able to regenerate an image that was present in the training set of Stable Diffusion. But they were only able to find that image (and others) in particular, because they were present in the training set multiple times, and the caption was the same (it was the portrait picture of some executive at a company).
Yeah, you are not gonna be able to do that with an LLM. They will be able to quote only some passages, and only of popular books that have been quoted often enough.
You cannot do that with an LLM.
I hate that some corporations are burning money, resources and energy on this, and the solution is not to restrict fair use even further. Machine Learning is complex, but if I had to summarize in some way is “just” gathering statistics of which word comes next (in the case of a text model). This is no different than getting a large corpus of text, and sample it for word frequency, letter frequency, N-gram frequency, etc. It is well known that this is fair use. You only store the copyrighted works to run the software and produce a very transformative work that is a summary many orders of magnitude smaller than the copyrighted work. This is fair use, and it should still be. Changing that is gonna harm the public, small companies and independent researchers way more than big tech companies.
As I said in another comment, I would very much welcome a way to force big corpos to release their models. Make a model bigger than N parameters? You needed too much fair use in one gulp: your model has to be public, and in the public domain. I would fucking welcome that! But going in the opposite direction is just risky.
I don’t understand why small individuals think that copyright is their friend, and will protect them from big tech companies. Copyright will always harm the weak and protect the powerful as a net result. It’s already a miracle that we can enjoy free software and culture by licenses that leverage copyright in our favor.
Dasus@lemmy.world 1 week ago
Image
If I want to go and read a Harry Potter book, I presumably have to pay someone something (excluding library services because those are services provided for actual people, not AI’s)?
This LLM clearly has read Harry Potter and Chamber of Secrets, and is merely refusing to display the data it already has on it. “Data” in this case meaning the work, the book.
I’m not for current copyright laws, but I find defending these hypocritical companies despicable. I’m sure you’re able to imagine that if it suited OpenAI, they might argue the exact opposite of what they’re arguing. Companies don’t really argue things in good faith, rather always arguing for the thing that will be the most profitable for them, no matter the veracity.
Zacryon@feddit.org 1 week ago
My point is, that the following statement is not entirely correct:
One obvious flaw in that sentence is the general statement about AI systems. There are huge differences between different realms of AI. Failing to address those by at least mentioning that briefly, disqualifies the author regarding factual correctness. For example, there are a plethora of non-generative AIs, meaning those, not generating texts, audio or images/videos, but merely operating as a classifier or clustering algorithm for instance, which are - without further modifications - not intended to replicate data similar to its inputs but rather provide insights.
However, I can overlook this as the author might have just not thought about that in the very moment of writing.
Next:
While it is true that transformer models like ChatGPT try to learn patterns, the most likely token for the next possible output in a sequence of contextually coherent data, given the right context it is not unlikely that it may reproduce its training data nearly or even completely identically as I’ve demonstrated before. The less data is available for a specific context to generalise from, the more likely it becomes that the model just replicates its training data. This is in principle fine because this is what such models are designed to do: draw the best possible conclusions from the available data to predict the next output in a sequence. (That’s one of the reasons why they need such an insane amount of data to be trained on.)
This can ultimately lead to occurences of indeed “copying specific texts or images”.
It doesn’t matter whether I directly prompted it for it. I set the correct context to achieve this kind of behaviour, because context matters most for transformer models. Directly prompting it do do that was just an easy way of setting the required context. I’ve occasionally observed ChatGPT replicating identical sentences from some (copyright-protected) scientific literature when I used it to get an overview over some specific topic and also had books or papers about that on hand. The latter demonstrates again that transformers become more likely to replicate training data the more “specific” a context becomes, i.e., having significantly less training data available for that context than about others.