That’s an interesting take, I didn’t know software could be inspired by other people’s works. Here. I thought software just did exactly as it’s instructed to do.
Comment on Authors Are Furious After Finding Their Works on List of Books Used To Train AI
Gibdos@feddit.de 9 months ago
I certainly hope that none of these authors have ever read a book before or have been inspired by something written by another author.
newthrowaway20@lemmy.world 9 months ago
FaceDeer@kbin.social 9 months ago
Well, now you know; software can be inspired by other people's works. That's what AIs are instructed to do during their training phase.
newthrowaway20@lemmy.world 9 months ago
Does that mean software can also be afraid, or angry? What about happy software? Saying software can be inspired is like saying a rock can feel pain.
lloram239@feddit.de 9 months ago
Does that mean software can also be afraid, or angry?
If it is programmed/trained that way, sure. I recommend having a listen to Geoffrey Hinton on the topic.
Saying software can be inspired is like saying a rock can feel pain.
The rock doesn’t do anything similar to pain. The LLM on the other side does a lot of things similar to inspiration. I can give the LLM a very trivial question and it will answer with a mountain of text. Did my question or the books it was trained on “inspire” the LLM to write that? Maybe, depends of course how far reaching you want to define the word. But either way, the LLM produced something by itself, that was neither a copy of my prompt nor the training data.
FaceDeer@kbin.social 9 months ago
Software can do a lot of things that rocks can't do, that's not a good analogy.
Whether software can feel "pain" depends a lot on your definitions, but I think there are circumstances in which software can be said to feel pain. Simple worms can sense painful stimuli and react to it, a program can do the same thing.
We've reached the point where the simplistic prejudices about artificial intelligence common in science fiction are no longer useful guidelines for talking about real artificial intelligence. Sci-fi writers have long assumed that AIs couldn't create art and now it turns out it's one of the things they're actually rather good at.
BURN@lemmy.world 9 months ago
Software cannot be “inspired”
AIs in their training stages are simply just running extreme statistical analysis on the input material. They’re not “learning” they’re not “inspired” they’re not “understanding”
The anthropomorphism of these models is a major problem. They are not human, they don’t learn like humans.
lloram239@feddit.de 9 months ago
The anthropomorphism of these models is a major problem.
People attributing any kind of person hood or sentience is certainly a problem, the models are fundamentally not capable of that (no loops, no hidden thought). At least for now. However what you are doing isn’t really much better, just utterly wrong in the opposite direction.
Those models are very definitely do “learn” and “understand” by every definition of the word. Simply playing around with that will quickly show that and it’s baffling that anybody would try to claim otherwise. Yes, there are limits to what they can understand and there are plenty things that they can’t do, but the amount of questions they can answer goes far beyond what is directly in the training data. Heck, even the fact that they hallucinate is proof that they understand, since it would be impossible to make completely plausible, but incorrect, stuff up without having a deep understanding of the topics. Also humans make mistakes too and they’ll also make stuff up, so this isn’t even anything AI specific.
PsychedSy@sh.itjust.works 9 months ago
They weren’t given data. They were shown data then the company spent tens of millions of dollars on cpu time to do statistical analysis of the data shown.
newthrowaway20@lemmy.world 9 months ago
A computer being shown data is a computer being given data. I don’t understand your argument.
lloram239@feddit.de 9 months ago
The data is gone by the time a user interacts with the AI. ChatGPT has no access to any books.
lloram239@feddit.de 9 months ago
And here I thought software just did exactly as it’s instructed to do.
AI isn’t software. Everything the AI knows is from the books. There is no human instructing the AI what to do. All the human does is build the scaffolding to let the AI learn, everything else is in the data.
LtLiana@startrek.website 9 months ago
Hey, computational linguist here who works with large language models. This is the most ridiculous thing I ever read.
Wander@kbin.social 9 months ago
Are you saying the writers of these programs have read all these books, and were inspired by them so much they wrote millions of books? And all this software is doing is outputting the result of someone being inspired by other books?
Grimy@lemmy.world 9 months ago
Clearly not. He’s saying that other authors have done the same as the software does. The software creators implemented the same principle into their llm. You are being daft on purpose.
newthrowaway20@lemmy.world 9 months ago
Large language models aren’t ‘inspired’ to write new works. Software can’t be inspired. It follows instructions. Even though large language models might feel like somebody is talking back to you and giving you new information, it’s just code following instructions designed to predict output based on the input provided and the data supplied. There’s no inspiration to be had, and to attribute inspiration to language models is a huge mischaracterization of what’s happening under the hood. Can a language model, without being told what to do, actually use any of the data it was fed to create something? No. Every single large language model requires some sort of input from a user to act as a seed before any sort of response can begin.
This is why it’s so stupid to call this shit AI, because people start thinking it’s actual intelligence. Really, It’s just a fancy illusion.
lloram239@feddit.de 9 months ago
This is why it’s so stupid to call this shit AI
It is using the term as defined. Maybe stop being a stupid parrot just repeating crap you heard else where and use your brain for a moment. I am losing hope that humans are capable of thought reading all this junk.
mojo@lemm.ee 9 months ago
They purchased their books to get inspiration from, the original author gets paid, and the author consented to selling it. That’s the difference.
Also the LLM can post entire snippets or chapters of books, which of course you’ll take at face value even if it hallucinates and makes the author look like a worse author then they are.
elbarto777@lemmy.world 9 months ago
These are machines, though, not human beings.
I guess I’d have to be an author to find out how I’d feel about it, to be fair.
Touching_Grass@lemmy.world 9 months ago
Machines that aren’t reproducing or distributing works
FaceDeer@kbin.social 9 months ago
If an AI "reproduces" a work it was trained on it is a failure of an AI. Why would anyone want to spend millions of dollars and devote oodles of computing power to build something that just does what a simple copy/paste operation can accomplish?
When an AI spits out something that's too close to one of the original training set that's called "overfitting" and it is considered an error to be corrected. Most overfitting that's been detected has been a result of duplication in the training set - when you hammer an AI image generator in training with thousands of copies of the Mona Lisa it eventually goes "alright, I get it already, when you say 'Mona Lisa' you want that exact pattern!" And will try its best to replicate that pattern when you ask it to later. That's why training sets need to be de-duplicated.
AIs are meant to produce new things.
Shurimal@kbin.social 9 months ago
These are machines, though, not human beings.
What's the difference? On the most fundamental level it's all the same.
AnonStoleMyPants@sopuli.xyz 9 months ago
The same thing as with tooooooons of things: scale.
Nobody cares if one dude steals office supplies at work. Now, if everyone stats doing it, or if the single guy steals everything, then action is taken.
Nobody cares if a random person draws in the same style and with same characters as you, but if they start to sell them, or god forbid, out-sell you, then there is a problem.
Nobody cares (except police I guess) if a random driver drives double the speed limit and annoys people living next to the road on the weekends, but when tons of people do it, you get speed bumps.
Nobody cares if few people pirate movies, but when it gets to mainstream and companies notice that there might be money being lost. Then you get whatever we have now.
Nobody cares if the mudhill behind your house erodes a bit and you get mud on your shoes. Have a bunch of that erode and you realise the danger…
You have been fine-tuning your own writing style for a decade and random schmuck starts to write similarly, you probably don’t care. No harm done. Now, get an AI to write 10 000 books in a weekend and someone starts to sell them… well now you have a completely different problem.
On a fundamental level the exact same thing is happening, yet action is only taken after a certain threshold is step over.
sab@lemmy.world 9 months ago
Bingo.
Wander@kbin.social 9 months ago
Unless you think theres no difference between killing a person and closing a program, I think we can agree they should be treated differently in the eyes of the law.
And so theres a difference between a person reading a book and being inspired by it, and someone writing a program that automatically transforms the book in data that can create new books.
brygphilomena@lemmy.world 9 months ago
A human, regardless of how many books they read, will have personal experiences that are undeniably unique to themselves. They will interpret the works they read differently from each other based on their worldly experiences. Their writing, no matter how many books they read and get inspired on, will always be influenced by their own personal lives. They can experience love, hate, heartbreak, empathy, sadness, and happiness.
This is something a LLM does not have, and in my opinion, is a massive distinguishing factor. So on a “fundamental” level, it is not the same. It is no where near the same.
lloram239@feddit.de 9 months ago
A human, regardless of how many books they read, will have personal experiences that are undeniably unique to themselves.
So will every AI. ChatGPT will give you different answers than Bard or WizardLM, since they are all trained on different books. And every StableDiffusion model creates different images, different styles, different topics, etc. It’s all in the data they “experienced”.
originalucifer@moist.catsweat.com 9 months ago
do you really think we are that far off... from giving a foundational memory and motivation layers to these LLMs, that could mimic.. or even.. generate the generic thoughts youre indicating?
i dont think so. you seem to imply its impossibility, i expect its inevitability. the human brain will not be a black box forever... it still exists in a world of physics we can emulate, even if rudimentary.
elbarto777@lemmy.world 9 months ago
Wait. Are human beings machines?
lloram239@feddit.de 9 months ago
Biological machines, yes.
jennraeross@lemmy.world 9 months ago
Please do not take this as support of ai use of copyrighted works (I don’t), but as far as I can tell, yes we are machines. This rant is just me being aspie atm, so feel free to ignore it.
We are thinking machines programmed by our genetics, predispositions, experiences, and circumstances. A 2 part explanation of how humans are merely products of their circumstances was once put forward to me. The first part is that humans can do anything, but only the thing we want to do most.
For instance, a common rebuttal is that people can choose go to the gym even when they find the experience of exercise undesirable. However, when that happens, it’s merely a case of other wants out balancing the want to not go to the gym, typically they want to be fit.
We want to not spend money, but we want to not rush going to jail for stealing more, usually. We want to not work overtime, but sometimes we want the extra cash more than that.
The second part of the argument is that we can’t choose what we want. When someone talks themselves out of the slice of cheesecake, they aren’t changing what they want, they’re resolving said want against the larger want they have to lose weight.
And if we make decisions by our wants, while said wants are not decided by us, then despite appearances we are little more than complex automata.
kromem@lemmy.world 9 months ago
Did you write a comment on Reddit before 2015? If so, your copyrighted content was used without your permission to train today’s LLMs, so you absolutely get to feel one way or another about it.
The idea that these authors were somehow the backbone of the models when any individual contribution was like spitting in the ocean and model weights would have considered 100 pages of Twilight fan fiction equivalent to 100 pages from Twilight is honestly one of the negative impacts of the extensive coverage these suits are getting.
Pretty much everyone who has ever written anything indexed online is a tiny part of today’s LLMs.
elbarto777@lemmy.world 9 months ago
Thank you for your reply.
On a completely separate note, it’s funny to think that there exists Twilight fan fiction when Twilight itself started as fan fiction work.
kromem@lemmy.world 9 months ago
Pretty sure it’s the other way around.
Fifty Shades of Gray started out as Twilight fanfiction before becoming its own thing.
AFAIK Twilight was always just its own pulp fiction.
dutchkimble@lemy.lol 9 months ago
But terminator said neural networks
elbarto777@lemmy.world 9 months ago
Damn.
sab@lemmy.world 9 months ago
I don’t think anyone is faulting the machines for this, just the people who instruct the machines to do it.
kromem@lemmy.world 9 months ago
Generally they probably bought the books they read though.
If George RR Martin torrented Tolkien, wouldn’t he be infringing on the copyright no matter how he subsequently incorporated it into future output?
I completely agree that the training as infringement argument is ludicrous.
But OpenAI exposed themselves to IP infringement by sailing the high seas in how they obtained the works in the first place.
I hate that the world we live in is one where so much data is gated behind paywalls, but the law is what it is, and if the government was going to come down hard on Aaron Swartz for trying to bypass paywalls for massive amounts of written text, it’s not exactly fair if there’s a double standard for OpenAI doing the same thing in an even more closed fashion.
But yes, the degree of entitled focus on the premise of training an AI as equivalent of infringing is weird as heck to see from authors drawing quite clearly from earlier works in their own output.
st0v@lemmy.zip 8 months ago
I have to assume that openAI also paid for the books. if yes then i consider it the same as me reciting passages from memory or coming up with derivative text.
if no, then by all means, go after them and any model trainer for the cost of one book.
Asking an LLM to recite an entire novel isn’t even vaguely a thing yet.
kromem@lemmy.world 8 months ago
Well, here’s straight from one of the suits against them:
“The OpenAI Books2 dataset can be estimated to contain about 294,000 titles. The only ‘internet-based books corpora’ that have ever offered that much material are notorious ‘shadow library’ websites like Library Genesis (aka LibGen), Z-Library (aka B-ok), Sci-Hub, and Bibliotik. The books aggregated by these websites have also been available in bulk via torrent systems.”
I’m not even sure how they would have logistically gone about purchasing 294,000 books in bulk in digital form to be fed into training. Using the existing collections seems much more likely, but I suppose we’ll see what turns up in litigation.
Omniraptor@lemm.ee 9 months ago
God that Aaron/jstor thing makes me see red every time. He was scraping jstor to publish it for the benefit of everyone, openai is doing it to make billions of dollars. Don’t forget who the bad guys are (and donate to sci-hub) everyone
adriaan@sh.itjust.works 9 months ago
That would be a much better comparison if it was artificial intelligence, but these are just reinforcement learning models. They do not get inspired.
Shurimal@kbin.social 9 months ago
...like the naturally occuring neural networks are.
Khalic@kbin.social 9 months ago
The brain does not work the way you think… (I work in the field, bio-informatics). What you call “neural networks” come from an early misunderstanding of how the brain stores information. It’s a LOT more complicated and frankly, barely understood.
canihasaccount@lemmy.world 9 months ago
Yeah, accurately simulating a single pyramidal neuron requires an eight-layer deep neural network:
www.cell.com/neuron/…/S0896-6273(21)00501-8.pdf
FaceDeer@kbin.social 9 months ago
Yet you confidently state that the brain doesn't work the way LLMs do?
Obviously it doesn't work exactly the same way that LLMs do, if only because of the completely different substrates. But when you get to more nebulous concepts like "creativity" and "inspiration" it's not so clear.
originalucifer@moist.catsweat.com 9 months ago
so its barely understood, but this definitely is not it. got it.
lemmyvore@feddit.nl 9 months ago
Tell you what, you get a landmark legal decision classifying LLM as people and then we’ll talk.
Until then it’s software being fed content in a way not permitted by its license i.e. the makers of that software committing copyright infringement.
Touching_Grass@lemmy.world 9 months ago
What exactly was not permitted by the license
Hackerman_uwu@lemmy.world 9 months ago
More to the point: they replicate patterns of words.
lloram239@feddit.de 9 months ago
So do humans.
FontMasterFlex@lemmy.world 9 months ago
That’s a Bingo!