I think it exists and works but that its simply not in their best interest to have people use it and be found out that they used chatgpt, for OpenAI’s business/profit potential. I have nothing to back it up but have just lost all faith in OpenAI.
Comment on OpenAI has built a text watermarking method to detect chatgpt written content
CameronDev@programming.dev 5 weeks ago
The arstechnica article speculated it was more of a pattern of words thing.
I think it is lies, and doesn’t exist or work anywhere near as good as they claim. Or, its incredibly easy to bypass.
TheHobbyist@lemmy.zip 5 weeks ago
CameronDev@programming.dev 5 weeks ago
Im willing to believe it exists, but not that its any good. 99% is a crazy accuracy claim.
pup_atlas@pawb.social 5 weeks ago
I van totally believe that it detects AI generated content 99% of the time, that’s trivial. What I really wanna know is the false positive rate. If I write a program that flags everything, it’d have a 100% hit rate. It’d also however have a crazy high false positive rate.
Womble@lemmy.world 5 weeks ago
Yup, noticable that they use the phrase “99.9% effective” effecting doesnt have a defined meaning in this contect, unlike accuracy, sensitivity or specificity, so that smells of missleading PR speak to me.
Orbituary@lemmy.world 5 weeks ago
Especially if the claim that it’s undetectable by humans.
originalfrozenbanana@lemm.ee 5 weeks ago
They sell the cheating tool and the detection software
andrew_bidlaw@sh.itjust.works 5 weeks ago
It’s really sweet to play the arms race with oneself.
archomrade@midwest.social 5 weeks ago
Not to mention that it would be extremely difficult to implement an effective watermark on text below a certain size
There are hundreds of thousands of pixels in an image where you can hide a watermark, but in a text output of a paragraph or less there are only a couple hundred characters.
How precise is the watermark? Is it a specific sequence of characters? Is it a sequence of words? A number of characters in a row? Non-print characters?
How precise the watermark is will determine how easy it is to get around. I imagine some of the most important uses to detect would be twitter/social media influence bots where the output length is only 140 characters or less. I find it hard to imagine a watermark on output of that size being effective or reliable.
deadcade@lemmy.deadca.de 5 weeks ago
Research on this topic exists, and it is possible to alter the output of an LLM in minor ways, that statistically “watermark” the results without drastically changing the quality of the output. OpenAI has probably implemented this into ChatGPT.
www.youtube.com/watch?v=2Kx9jbSMZqA
I think the tool exists, and is (at least close to) as good as they claim it is. They can’t release it, because once the public can tell with high accuracy whether ChatGPT wrote some text, another AI can be developed to circumvent detection from this method, making the tool useless.
CameronDev@programming.dev 5 weeks ago
That is a long video, is the paper published somewhere?
Im willing to accept that you can statistically “watermark” the text, but I’m not convinced that it would be tamper resistant, which is a large part of what makes a watermark useful. If it can’t survive an idiot with a thesaurus, its probably not gonna be terribly useful.
Womble@lemmy.world 5 weeks ago
It can likely also be defeated by adding “In the style of X” to a prompt, changing the distribution and pattern of the responses.
JackbyDev@programming.dev 5 weeks ago
…but that output is also from the AI so it would still be watermarked lol