Let’s pretend for a moment that we know that Reddit has any sort of decent versioning system, and that it keeps the old versions of your comments alongside the newer ones, and that it’s feeding the LLM with the old version. (Does it? I have my doubts, given that Reddit Inc. isn’t exactly competent.)
They almost certainly do, if only because of the practicalities of adding a new comment, then having that be fetched in place of the old one, compared to making and propagating an actual edit across all their databases.
It would also help with any administration/moderation tasks if they could see whether people posted spam and then tried to hide it behind edits.
lvxferre@mander.xyz 8 months ago
If this is true, it shifts the problem from “not having it” to “not knowing which version should be used” (to train the LLM).
They could feed it the unedited versions and call it a day, but a lot of times people edit their content to correct it or add further info, specially for “meatier” content (like tutorials). So there’s still some value on the edits, and I believe that Google will be at least tempted to use them.
If that’s correct, editing it with nonsense will lower the value of edited comments for the sake of LLM training. It should have an impact, just not as big as if they kept no version system.
I know from experience (I’m a former Reddit janny) that moderators can’t see earlier versions of the content, only the last one. The admins might though.
The one from TD, right?
GBU_28@lemm.ee 8 months ago
Wouldn’t be hard to scan a user and say:
lvxferre@mander.xyz 8 months ago
It would.
First you’d need to notice the problem. Does Google even realise that some people want to edit their Reddit content to boycott LLM training?
Let’s say that Google did it. Then it’d need to come up with a good (generalisable, low amount of false positives, low amount of false negatives) set of rules to sort those out. And while coming up with “random” rules is easy, good ones take testing, trial and error, and time.
But let’s say that Google still does it. Now it’s retrieving and processing a lot more info from the database than just the content and its context, but also account age, when the piece of content was submitted, when it was edited.
So doing it still increases the costs associated with the corpus, making it less desirable.
GBU_28@lemm.ee 8 months ago
Huh? Reddit has all of this plus changes in their own DBs. Google has nothing to do with this, it’s pre handover.
Voroxpete@sh.itjust.works 8 months ago
It sounds like what’s needed here is a version of this tool that makes the edits slowly, at random intervals, over a period of time. And perhaps has the ability to randomize the text in each edit so that they’re all unusable garbage, but different unusable garbage (like the suggestion of taking ChatGPT output at really high temp that someone else made). Maybe it also only edits something like 25% of your total comment pool, and perhaps makes unnoticeably minor edits (add a space, remove a comma) to a whole bunch of other comments. Basically masking the poison by hiding it in a lot of noise?
GBU_28@lemm.ee 8 months ago
Now you’re talkin
londos@lemmy.world 8 months ago
Honestly, parsing through version history is actually something an LLM could handle. It might even make more sense of it than without. For example, if someone replies to a comment and then the parent is edited to say something different. No one will have to waste their time filtering anything.
lvxferre@mander.xyz 8 months ago
They could use an LLM to parse through the version history of all those posts/comments, to use it to train another LLM with it. It sounds like a bad (and expensive, processing time-wise) idea, but it could be done.