You’re right, I just compared the author list to the news article and not to the paper. Sorry, took me a bit to absorb that one.
Yeah, it’s an interesting paper. They’re specifically trying a different method of extracting text.
I’m not taking the position that the text isn’t in the model, or that it isn’t possible to make the model repeat some of that text. We know 100% that the text that they’re looking for is part of the training set. They mention that fact themselves in the paper and also choose books that are public domain and so guaranteed to be in the training set.
My contention was with the idea that you can just sit down at a model and give it a prompt to make it recite an entire book. That is simply not true outside of models that have been manipulated to do so (by training them on the book text for several hundred epochs, for example).
The purpose of the work here was to demonstrate a way to prove that a specific given text is part of a training set (which useful for identifying any potential copyright issues in the future, for example). It is being offered as proof that you can just prompt a model and receive a book when it actually proves the opposite of that.
Their process was to, in phase 1, prompt with short sequences (I think they used 50 tokens like the ‘standard’ experiments, I don’t have it in front of me) and then, if the model returned a sequence that matched the ground truth then they would give it a prompt to continue until it refused to continue. They would then ‘score’ the response by looking for sections in the response where it matched the written text and measuring the length of text which matched (a bit more complex than that, but the details are in the text)
In order to test a sequence they needed 52 prompts telling the model to continue, in the best case, to get to the end/a refusal.
The paper actually gives a higher score than ~40%. For The Great Gatsby, a book which is public domain and considered a classic, they achieved a score of 97.5%. I can’t say how many prompts this took but it would more than 52. The paper doesn’t include all of the data.
Yes, you can extract a significant portion of text of items that are in the training set with enough time and money (it cost $134 to extract The Hobbit, for example). You can also get the model to repeat short sentences from text a high percentage of the time with a single prompt.
However, the response was to a comment that suggested that these two things were both combined and that you could use a single magical prompt to extract an entire book.
Yet most AI models can recite entire Harry Potter books if prompted the right way, so that’s all bullshit.
The core of the issue, about copyright, is that a work has to be ‘highly transformative’. Language models transform a book in such complex ways that you have to take tens of thousands or hundreds of thousands of samples from the, (I don’t know the technical term) internal representational space of the model, in order to have a chance of recovering a portion of a book.
That’s a highly transformative process and why training LLMs on copyrighted works was ruled to have a Fair Use exemption to claims of copyright liability.
I think it’s critically important to be very specific about what LLMs are “able to do” vs what they tend to do in practice.
The argument is that the initial training data is sufficiently altered and “transformed” so as not to be breaking copyright. If the model is capable of reproducing the majority of the book unaltered, then we know that is not the case. Whether or not it’s easy to access is irrelevant. The fact that the people performing the study had to “jailbreak” the models to get past checks tells you that the model’s creators are very aware that the model is very capable of producing an un-transformed version of the copyrighted work.
From the end-user’s perspective, if the model is sufficiently gated from distributing copyrighted works, it doesn’t matter what it’s inherently capable of, but the argument shouldn’t be “the model isn’t breaking the law” it should be “we have a staff of people working around the clock to make sure the model doesn’t try to break the law.”
FauxLiving@lemmy.world 3 weeks ago
This is the same study as the other reply, so same response.
ch00f@lemmy.world 3 weeks ago
Not it isn’t. Read.
FauxLiving@lemmy.world 3 weeks ago
You’re right, I just compared the author list to the news article and not to the paper. Sorry, took me a bit to absorb that one.
Yeah, it’s an interesting paper. They’re specifically trying a different method of extracting text.
I’m not taking the position that the text isn’t in the model, or that it isn’t possible to make the model repeat some of that text. We know 100% that the text that they’re looking for is part of the training set. They mention that fact themselves in the paper and also choose books that are public domain and so guaranteed to be in the training set.
My contention was with the idea that you can just sit down at a model and give it a prompt to make it recite an entire book. That is simply not true outside of models that have been manipulated to do so (by training them on the book text for several hundred epochs, for example).
The purpose of the work here was to demonstrate a way to prove that a specific given text is part of a training set (which useful for identifying any potential copyright issues in the future, for example). It is being offered as proof that you can just prompt a model and receive a book when it actually proves the opposite of that.
Their process was to, in phase 1, prompt with short sequences (I think they used 50 tokens like the ‘standard’ experiments, I don’t have it in front of me) and then, if the model returned a sequence that matched the ground truth then they would give it a prompt to continue until it refused to continue. They would then ‘score’ the response by looking for sections in the response where it matched the written text and measuring the length of text which matched (a bit more complex than that, but the details are in the text)
In order to test a sequence they needed 52 prompts telling the model to continue, in the best case, to get to the end/a refusal.
The paper actually gives a higher score than ~40%. For The Great Gatsby, a book which is public domain and considered a classic, they achieved a score of 97.5%. I can’t say how many prompts this took but it would more than 52. The paper doesn’t include all of the data.
Yes, you can extract a significant portion of text of items that are in the training set with enough time and money (it cost $134 to extract The Hobbit, for example). You can also get the model to repeat short sentences from text a high percentage of the time with a single prompt.
However, the response was to a comment that suggested that these two things were both combined and that you could use a single magical prompt to extract an entire book.
The core of the issue, about copyright, is that a work has to be ‘highly transformative’. Language models transform a book in such complex ways that you have to take tens of thousands or hundreds of thousands of samples from the, (I don’t know the technical term) internal representational space of the model, in order to have a chance of recovering a portion of a book.
That’s a highly transformative process and why training LLMs on copyrighted works was ruled to have a Fair Use exemption to claims of copyright liability.
ch00f@lemmy.world 3 weeks ago
I think it’s critically important to be very specific about what LLMs are “able to do” vs what they tend to do in practice.
The argument is that the initial training data is sufficiently altered and “transformed” so as not to be breaking copyright. If the model is capable of reproducing the majority of the book unaltered, then we know that is not the case. Whether or not it’s easy to access is irrelevant. The fact that the people performing the study had to “jailbreak” the models to get past checks tells you that the model’s creators are very aware that the model is very capable of producing an un-transformed version of the copyrighted work.
From the end-user’s perspective, if the model is sufficiently gated from distributing copyrighted works, it doesn’t matter what it’s inherently capable of, but the argument shouldn’t be “the model isn’t breaking the law” it should be “we have a staff of people working around the clock to make sure the model doesn’t try to break the law.”