OpenAI has built a text watermarking method to detect chatgpt written content

Submitted ⁨⁨3⁩ ⁨months⁩ ago⁩ by ⁨PenisDuckCuck9001@lemmynsfw.com⁩ to ⁨technology@lemmy.world⁩

https://www.tomshardware.com/tech-industry/artificial-intelligence/openai-has-built-a-text-watermarking-method-to-detect-chatgpt-written-content-company-has-mulled-its-release-over-the-past-year

Am I missing something? The article seems to suggest it works via hidden text characters. Has OpenAI never heard of pasting text into a utf8 notepad before?

source

Comments

Sort:hotnew top

CameronDev@programming.dev ⁨3⁩ ⁨months⁩ ago
The arstechnica article speculated it was more of a pattern of words thing.

I think it is lies, and doesn’t exist or work anywhere near as good as they claim. Or, its incredibly easy to bypass.

arstechnica.com/…/openai-has-the-tech-to-watermar…

source
- deadcade@lemmy.deadca.de ⁨3⁩ ⁨months⁩ ago
  Research on this topic exists, and it is possible to alter the output of an LLM in minor ways, that statistically “watermark” the results without drastically changing the quality of the output. OpenAI has probably implemented this into ChatGPT.
  
  www.youtube.com/watch?v=2Kx9jbSMZqA
  
  I think the tool exists, and is (at least close to) as good as they claim it is. They can’t release it, because once the public can tell with high accuracy whether ChatGPT wrote some text, another AI can be developed to circumvent detection from this method, making the tool useless.
  
  source
  - CameronDev@programming.dev ⁨3⁩ ⁨months⁩ ago
    That is a long video, is the paper published somewhere?
    
    Im willing to accept that you can statistically “watermark” the text, but I’m not convinced that it would be tamper resistant, which is a large part of what makes a watermark useful. If it can’t survive an idiot with a thesaurus, its probably not gonna be terribly useful.
    
    source
    -> View More Comments
- TheHobbyist@lemmy.zip ⁨3⁩ ⁨months⁩ ago
  I think it exists and works but that its simply not in their best interest to have people use it and be found out that they used chatgpt, for OpenAI’s business/profit potential. I have nothing to back it up but have just lost all faith in OpenAI.
  
  source
  - CameronDev@programming.dev ⁨3⁩ ⁨months⁩ ago
    Im willing to believe it exists, but not that its any good. 99% is a crazy accuracy claim.
    
    source
    -> View More Comments
  - originalfrozenbanana@lemm.ee ⁨3⁩ ⁨months⁩ ago
    They sell the cheating tool and the detection software
    
    source
    -> View More Comments
- archomrade@midwest.social ⁨3⁩ ⁨months⁩ ago
  Not to mention that it would be extremely difficult to implement an effective watermark on text below a certain size
  
  There are hundreds of thousands of pixels in an image where you can hide a watermark, but in a text output of a paragraph or less there are only a couple hundred characters.
  
  How precise is the watermark? Is it a specific sequence of characters? Is it a sequence of words? A number of characters in a row? Non-print characters?
  
  How precise the watermark is will determine how easy it is to get around. I imagine some of the most important uses to detect would be twitter/social media influence bots where the output length is only 140 characters or less. I find it hard to imagine a watermark on output of that size being effective or reliable.
  
  source
MagicShel@programming.dev ⁨3⁩ ⁨months⁩ ago
Am I the only one who rewrites most of ChatGPT’s output into my own words because it’s “voice” is garbage anyway? I ask it to write me a cover letter and that gives me a rough outline and some points to make, but I have to do massive editing to avoid redundancy, awkward phrasing, outright lies, etc.

I can’t imagine turning in raw ChatGPT output. I had one of my developers use AI to write code and submitted that shit raw and it was immediately obvious because some relatively simple code has really weird artifacts like overwriting a value that had no reason to even be touched.

source
- lowleveldata@programming.dev ⁨3⁩ ⁨months⁩ ago
  Idk it looks good to me. Straight to the main branch you go.
  
  source
- ValenThyme@reddthat.com ⁨3⁩ ⁨months⁩ ago
  for school i use it to make outlines which are usually very good and then I use these class materials to flesh out the outlines. Getting straight A’s like this! One of my papers the teacher said it was the best one he had ever received on that assignment. All my words but ChatGPT told me what to include and in what order.
  
  source
  - MagicShel@programming.dev ⁨3⁩ ⁨months⁩ ago
    That’s valid. And I’d be surprised if that could be watermarked.
    
    source
- JeeBaiChow@lemmy.world ⁨3⁩ ⁨months⁩ ago
  Lol. AI gonna take over the developers job. Like that’s even close to happening.
  
  source
  - Thorny_Insight@lemm.ee ⁨3⁩ ⁨months⁩ ago
    Few years ago the output of GPT was complete gibberish and few years before that even producing such gibberish would’ve been impressive.
    
    It doesn’t take anyone’s job untill it does.
    
    source
    -> View More Comments
  - Angry_Autist@lemmy.world ⁨3⁩ ⁨months⁩ ago
    LLMs aren’t going to take coding jobs, there are specific case AIs being trained for that. They write code that works but does not make sense to human eyes. It’s fucking terrifying but EVERYONE just keeps focusing on the LLMS.
    
    There are at least 2 more dangerous model types being used right now to influence elections and manipulate online spaces and ALL everyone cares about is their fucking parrot bots…
    
    source
    -> View More Comments
- cyberpunk007@lemmy.ca ⁨3⁩ ⁨months⁩ ago
  For me I find it sounds too much like a marketing person or something I’d see in an ad or a website so I “dumb it down” a bit to make it not sound too corporate. Sometimes telling Chatgpt to do so fixes this though.
  
  source
- shatteredsword@lemmy.world ⁨3⁩ ⁨months⁩ ago
  Yes, but I use chatgpt to do the rewrites too
  
  source
count_dongulus@lemmy.world ⁨3⁩ ⁨months⁩ ago
They could inject random zero width non joiners to help detection too. Easy to defeat, but something a layperson would have to go through extra effort to filter out.

source
- admin@lemmy.my-box.dev ⁨3⁩ ⁨months⁩ ago
  Yeah, no chance they’d rely on something that would be so easy to defeat. Watermarking by using word patterns is far more likely.
  
  Still easy to defeat by just using another LLM to rephrase it though.
  
  source
  - count_dongulus@lemmy.world ⁨3⁩ ⁨months⁩ ago
    It’s one of many things they could do just like how security is a layers thing.
    
    source
    -> View More Comments
JackbyDev@programming.dev ⁨3⁩ ⁨months⁩ ago
As someone who fiddled with Stable Diffusion which also has optional invisible watermarks this is a good feature. It is so that AI training will avoid content marking itself as AI generated. If people want to hide that their content is AI generated then, sadly, it’s harder to detect.

source
- todd_bonzalez@lemm.ee ⁨3⁩ ⁨months⁩ ago
  Watermarking everything I digitally publish to keep my original content out of a training set.
  
  Publishing a website full of de-watermarked AI slop to ruin future LLMs.
  
  source
  - JackbyDev@programming.dev ⁨3⁩ ⁨months⁩ ago
    More info if you’re seriously considering it. codoraven.com/…/stable-diffusion-the-invisible-wa…
    
    I don’t actually know if any model creators check for the watermark or not.
    
    source
  - RandomVideos@programming.dev ⁨3⁩ ⁨months⁩ ago
    Arent there better methods to poison AI?
    
    I have heard Glaze and Nightshade are good, but have never used them
    
    source
    -> View More Comments
brucethemoose@lemmy.world ⁨3⁩ ⁨months⁩ ago
This has been known in the ML space forever. LLMs don’t actually output words, but “probabilities” for tokens. And if you arbitrarily weigh these probabilities, it creates a “signature” in any text thats easy to measure. The sampler randomizes it a tiny bit, but thats not a problem in long texts.

It’s defeatable. I’m sure if you maken enough OpenAI queries, you can find the bias. But this likely will stop the lazy absures, aka 99% of abusers.

source
- PenisDuckCuck9001@lemmynsfw.com ⁨3⁩ ⁨months⁩ ago
  So if cheating on homework, use self hosted only then. Cool.
  
  source
  - brucethemoose@lemmy.world ⁨3⁩ ⁨months⁩ ago
    You have full control of your logit outputs with local LLMs, so theoretically you could “unscramble” them.
    
    OpenAI (IIRC) very notably stopped giving the logprobs of their models. They did this for many reasons, and most of them boil down to “profits” and “they are anticompetitive jerks”
    
    source
- archomrade@midwest.social ⁨3⁩ ⁨months⁩ ago
  It wouldn’t be surprising to me if they’ve had this implemented for awhile.
  
  There’s still some question about why their 3.5 model had an apparent sudden drop-off in quality about a year ago, and among the plausible explanations for it could be that they were fucking with their weights in order to watermark the outputs in exactly the way you’re mentioning. They were also fighting against prompt-injection methods and censor disapproved uses at the time, so who the fuck knows.
  
  source
  - brucethemoose@lemmy.world ⁨3⁩ ⁨months⁩ ago
    This doesn’t touch the weights at all, it’s just a change to the sampler.
    
    What lobotomizes their models is cost cutting and trying to make them “safe,” or at least thats what I suspect.
    
    source
bdonvr@thelemmy.club ⁨3⁩ ⁨months⁩ ago
What’s the false positive rate tho

source
nickwitha_k@lemmy.sdf.org ⁨3⁩ ⁨months⁩ ago
I’m inclined to believe that they’re throwing all prompts and outputs into a db and searching that.

source
art@lemmy.world ⁨3⁩ ⁨months⁩ ago
It’s a good thing that ChatGPT is only one of the many LLM’s to choose from.

source
RecallMadness@lemmy.nz ⁨3⁩ ⁨months⁩ ago
Is “The Algorithm” just “we stuffed all our GPT responses into a Lucene index and look for 80% matches”?

Because that’s what I’d do.

source
nullPointer@programming.dev ⁨3⁩ ⁨months⁩ ago
so the AI is going to say ‘like’ every third word?

source
qx128@lemmy.world ⁨3⁩ ⁨months⁩ ago
In other news, mathematicians have been working hard on calculator detector software. Upon request for comment, leading mathematicians suggested a variety of ideas, such as such as secretly embedding a watermark “58008” (BOOBS) into the decimal parts of pi and e to more easily identify derived calculations. There was consistent sentiment among leading minds that “back in my day we had to work hard to do math, and walk up hill both ways in the snow to school”… and that “there’s nothing wrong with a good ol’ fashion abbicus, dag nabbit!”

source
- atrielienz@lemmy.world ⁨3⁩ ⁨months⁩ ago
  Just because you can plug information into a calculator (or a LLM) doesn’t mean you understand the math that comes out of it (or the data). Which I think is rather that point of academia not wanting people to use chat gpt generated content.
  
  On the other hand this is to prevent LLM’s using data generated by other LLM’s. Which is important because that’s how they degrade in quality.
  
  source
BetaDoggo_@lemmy.world ⁨3⁩ ⁨months⁩ ago
About a dozen methods they could use arxiv.org/pdf/2312.07913v2

source
AbouBenAdhem@lemmy.world ⁨3⁩ ⁨months⁩ ago
Humans instinctively do something analogous with natural language, using poetic forms like rhyme, meter, and alliteration. (For example, the speeches from Shakespeare’s plays are immediately detectable because they’re in iambic pentameter.)

Imagine you lacked the natural human ability to detect verse, making poetry indistinguishable from prose. As far as you could tell, it would be like an invisible watermark that only specialists could detect.

source
catloaf@lemm.ee ⁨3⁩ ⁨months⁩ ago
That’s cool, but literally any other implementation won’t have that, or will have an incompatible watermark.

source
- henfredemars@infosec.pub ⁨3⁩ ⁨months⁩ ago
  Exactly. The cat is out of the bag. I can still run my own LLM.
  
  source
TerkErJerbs@lemm.ee ⁨3⁩ ⁨months⁩ ago
It’s probably some type of cypher. Which will take people exactly one (1) afternoon to crack.

source
- BearOfaTime@lemm.ee ⁨3⁩ ⁨months⁩ ago
  A whole afternoon or just a portion?
  
  source