they never did, they never will.
OpenAI confirms that AI writing detectors don’t work
Submitted 1 year ago by fne8w2ah@lemmy.world to technology@lemmy.world
Comments
cheesorist@lemmy.world 1 year ago
stevedidWHAT@lemmy.world 1 year ago
Why tho or are you trying to be vague on purpose
bioemerl@kbin.social 1 year ago
Because you're training a detector on something that is designed to emulate regular languages closest possible, and human speech has so much incredible variability that it's almost impossible to identify if someone or something has been written by an AI.
You can detect maybe your typical generic chat GPT type outputs, but you can characterize a conversation with chat GPT or any of the other much better local models (privacy and control are aspects which make them better) and after doing that you can get radically human seeming outputs that are totally different from anything chat GPT will output.
In short, given a static block of text it's going to be nearly impossible to detect if it's coming from an AI. It's just too difficult to problem, and if you're going to solve it it's going to be immediately obsolete the next time someone fine tunes their own model
Eufalconimorph@discuss.tchncs.de 1 year ago
Because AIs are (partly) trained by making AI detectors. If an AI can be distinguished from a natural intelligence, it’s not good enough at emulating intelligence. If an AI detector can reliably distinguish AI from humans, the AI companies will use that detector to train their next AI.
sebi@lemmy.world 1 year ago
ReallyKinda@kbin.social 1 year ago
I know a couple teachers (college level) that have caught several gpt papers over the summer. It’s a great cheating tool but as with all cheating in the past you still have to basically learn the material (at least for narrative papers) to proof gpt properly. It doesn’t get jargon right, it makes things up, it makes no attempt to adhere to reason when it’s making an argument.
Using translation tools is extra obvious—have a native speaker proof your paper if you attempt to use an AI translator on a paper for credit!!
SpikesOtherDog@ani.social 1 year ago
it makes things up, it makes no attempt to adhere to reason when it’s making an argument.
It doesn’t hardly understand logic. I’m using it to generate content and it continuously will assert information in ways that don’t make sense, relate things that aren’t connected, and forget facts that don’t flow into the response.
mayonaise_met@feddit.nl 1 year ago
As I understand it as a layman who used GPT4 quite a lot to generate code and formulas, it doesn’t understand logic at all. Afaik, there is currently no rational process which considers whether what it’s about to say makes sense and is correct.
It just sort of bullshits it’s way to an answer based on whether words seem likely according to its model.
That’s why you can point it in the right direction and it will sometimes appear to apply reasoning and correct itself.
pc_admin@aussie.zone 1 year ago
Any teach still issuing out of class homework or assinments is doing a disservice IMO.
Of coarse people will just GPT it… you need to get them off the computer and into an exam room.
SmoothLiquidation@lemmy.world 1 year ago
GPT is a tool that the students will have access to their entire professional lives. It should be treated as such and worked into the curriculum.
Forbidding it would be like saying you can’t use Photoshop in a photography class.
ReallyKinda@kbin.social 1 year ago
Even in college? I never had a college course that allowed you to work on assignments in class
Nioxic@lemmy.dbzer0.com 1 year ago
I have to hand in a short report
I wrote parts of it and asked chatgpt for a conclusion.
So i read that, adjusted a few points. Added another couple points…
Then rewrote it all in my own wording.
TropicalDingdong@lemmy.world 1 year ago
I found out on the last screen of a travel grant application I needed a coverletter.
I pasted in the requirements for the cover letter and what I had put in my application.
I pasted the results in as the cover letter without review.
I got the travel grant.
Blurrg@lemmy.world 1 year ago
Who reads cover letters? At most they are skimmed over.
doublejay1999@lemmy.world 1 year ago
AI company says their AI is smart, but other companies are sell snake oil.
Gottit
canihasaccount@lemmy.world 1 year ago
They tried training an AI to detect AI, too, and failed
learningduck@programming.dev 1 year ago
Typically for generative AI. I think during their training of the Nobel, they must have developed another model that detect if GPT produce a more natural language. I think that other model may reached the point where it couldn’t flag it with acceptable false positive.
Boddhisatva@lemmy.world 1 year ago
OpenAI discontinued its AI Classifier, which was an experimental tool designed to detect AI-written text. It had an abysmal 26 percent accuracy rate.
If you ask this thing whether or not some given text is AI generated, and it is only right 26% of the time, then I can think of a real quick way to make it 74% accurate.
Leate_Wonceslace@lemmy.dbzer0.com 1 year ago
I feel like this must stem from a misunderstanding of what 26% accuracy means, but for the life of me, I can’t figure out what it would be.
dartos@reddthat.com 1 year ago
Looks like they got that number from this quote from another arstechnica article ”…OpenAI admitted that its AI Classifier was not “fully reliable,” correctly identifying only 26 percent of AI-written text as “likely AI-written” and incorrectly labeling human-written works 9 percent of the time”
Seems like it mostly wasn’t confident enough to make a judgement, but 26% it correctly detected ai text and 9% incorrectly identified human text as ai text. It doesn’t tell us how often it labeled AI text as human text or how often it was just unsure.
notatoad@lemmy.world 1 year ago
it seemed like a really weird decision for OpenAI to have an AI classifier in the first place. their whole business is to generate output that’s good enough that it can’t be distinguished from what a human might produce, and then they went and made a tool to try and point out where they failed.
Boddhisatva@lemmy.world 1 year ago
That may have been the goal. Look how good our AI is, even we can’t tell if its output is human generated or not.
Blackmist@feddit.uk 1 year ago
The only thing AI writing seems to be useful for is wasting real people’s time.
itsmaxyd@lemm.ee 1 year ago
True -
- Write points/summary
- Have AI expand in many words
- Post
- Reader uses AI to generate summarize post preferably in points
- Profit??
driving_crooner@lemmy.eco.br 1 year ago
Terence Tao just did a thread on Mathstodon talking about jow ChatGPT help him program a algorithm for looking for numbers.
Matriks404@lemmy.world 1 year ago
Did human-generated content really become so low quality that it is distinguishable from AI-generated content?
technicalogical@lemmy.world 1 year ago
Should I be able to detect whether or not this is an AI generated comment?
nodsocket@lemmy.world 1 year ago
As an AI language model, I am unable to confirm whether or not the above post was written by an AI.
funktion@lemm.ee 1 year ago
People kind of just suck at writing in general. It’s not a skill that’s valued so much, otherwise writers, editors, and proofreaders would be paid more.
DogMuffins@discuss.tchncs.de 1 year ago
Not necessarily. It’s just that AI’s can’t tell the difference.
Although I don’t know whether humans can.
Arsenal4ever@lemmy.world 1 year ago
have you seen exTwitter?
hellothere@sh.itjust.works 1 year ago
Regardless of if they do or don’t, surely it’s in the interests of the people making the “AI” to claim that their tool is so good it’s indistinguishable from humans?
stevedidWHAT@lemmy.world 1 year ago
Depends if they’re more researchers or a business imo. Scientists generally speaking are very cautious about making shit claims bc if they get called out that’s their career really.
BetaDoggo_@lemmy.world 1 year ago
OpenAI hasn’t been focused on the science since the Microsoft investment. A science focused company doesn’t release a technical report that doesn’t contain any of the specs of the model they’re reporting on.
Zeth0s@lemmy.world 1 year ago
Few decades ago probably, nowadays “scientists” make a lot of bs claims to get published. I was in the room when a “scientist” publishing several nature per year asked to her student to write a paper for a research without any result in a way that it look like it had something important for a relatively good IF publication.
That day I decided I was done with academia. I had seen enough.
hellothere@sh.itjust.works 1 year ago
It’s literally a marketing blog posted by OpenAI on their site, this isn’t a study.
Kolrami@lemmy.world 1 year ago
Yes, but it’s such a falsifiable claim that anyone is more than welcome to prove them wrong. There’s a lot of slightly different LLMs out there. If you or anyone else can definitively show there’s a machine that can identify AI writing vs human writing, it will either result in better AI writing or it would be an amazing breakthrough in understanding the limits of AI.
hellothere@sh.itjust.works 1 year ago
People like to view the problem as a paradox - can an all powerful God create a rock they cannot lift? - but I feel that’s too generous, it’s more marking your own homework.
If a system can both write text, and detect whether it or another system wrote that text, then “all” it needs to do is change that text to be outside of the bounds of detection. That is to say, it just needs to convince itself.
I’m not wanting to imply that that is easy, because it isn’t, but it’s a very different thing to convincing someone else, especially a human, that understands the topic.
There is also a false narrative involved here, that we need an AI to detect AI which again serves as a marketing benefit to OpenAI.
We don’t, because they aren’t that good, at least, not yet anyway.
irotsoma@lemmy.world 1 year ago
A lot of these relied on common mistakes that “AI” algorithms make but humans generally don’t. As language models are improving, it’s harder to detect.
Cethin@lemmy.zip 1 year ago
They’re also likely training on the detector’s output. That why they build detectors. It isn’t for the good of other people. It’s to improve their assets. A detector is used to discard some inputs it knows are written by AI so it doesn’t train on that data, which leads to it out competing the detection AI.
Shameless@lemmy.world 1 year ago
[deleted]Turun@feddit.de 1 year ago
Or, because you can’t rely on computers to tell you the truth. Which is exactly the issue with LLMs as well.
sfgifz@lemmy.world 1 year ago
You can’t rely on books or people tell you the truth either.
Absolutemehperson@lemmy.world 1 year ago
mfw just asking ChatGPT to write an undetectable essay.
Later, losers!
GlendatheGayWitch@lib.lgbt 1 year ago
Couldn’t you just ask ChapGPT whether it wrote something specific?
vale@sh.itjust.works 1 year ago
Then you have that time that a professor tried to fail his whole class because he asked chatGPT if it wrote the essays.
wedeworps@sh.itjust.works 1 year ago
Could you please provide a brief overview? This article is not available in my country/region.
4AV@lemmy.world 1 year ago
It doesn’t have “memory” of what it has generated previously, other than the current conversation. The answer you get from it won’t be much better than random guessing.
randint@lemmy.frozeninferno.xyz 1 year ago
Maybe it should keep a log of what was generated? Would that even work though?
mwguy@infosec.pub 1 year ago
No. The model doesn’t have a record of everything it wrote.
echodot@feddit.uk 1 year ago
That doesn’t really work because it just says whatever half the time. It’s very good at making stuff up. It doesn’t really get that it needs to tell the truth because all it’s doing is optimising for a good narrative.
That’s why it says slavery is good, because the only people asking that question clearly have an answer in mind, and it’s optimising for that answer.
Also it doesn’t have access to other people’s sessions (because that would be hella dodgy) so it can’t tell you definitively if it did or did not say something in another session, even if it were inclined to tell the truth.
Kazumara@feddit.de 1 year ago
Obviously not. Its a language generator with a bit of chat modeling and reinforcement learning, not an Artificial General Intelligence.
It doesn’t know anything, it doesn’t retain memory long term, it doesn’t have any self identity. There is no way it could ever truthfully respond “I know that I wrote that”.
Jargus@lemmy.world 1 year ago
So Democracy is basically fucked and countries without freedom of expression/speech have a advantage while our social media will be a cesspool. The future looks bright /s
robbotlove@lemmy.world 1 year ago
this comment could have been written in 2005 and still have been true.
SpaceCowboy@lemmy.ca 1 year ago
AI might democratize grifting. You no longer will have to have the resources that Russia and China have devoted to this kind of thing. Anyone will be able to generate vast amounts of fake inflammatory rhetoric.
Then once there’s a 99.9% chance that the person you’re talking to on social media is an AI, people might realize how stupid it is to believe anything they read on the internet.
efrique@lemm.ee 1 year ago
Just need to get AI on that.
autotldr@lemmings.world [bot] 1 year ago
This is the best summary I could come up with:
In a related FAQ, they also officially admit what we already know: AI writing detectors don’t work, despite frequently being used to punish students with false positives.
In July, we covered in depth why AI writing detectors such as GPTZero don’t work, with experts calling them “mostly snake oil.”
That same month, OpenAI discontinued its AI Classifier, which was an experimental tool designed to detect AI-written text.
Along those lines, OpenAI also addresses its AI models’ propensity to confabulate false information, which we have also covered in detail at Ars.
“Sometimes, ChatGPT sounds convincing, but it might give you incorrect or misleading information (often called a ‘hallucination’ in the literature),” the company writes.
Also, some sloppy attempts to pass off AI-generated work as human-written can leave tell-tale signs, such as the phrase “as an AI language model,” which means someone copied and pasted ChatGPT output without being careful.
The original article contains 490 words, the summary contains 148 words. Saved 70%. I’m a bot and I’m open source!
nucleative@lemmy.world 1 year ago
We need to embrace AI written content fully. Language is just a protocol for communication. If AI can flesh out the “packets” for us nicely in a way that fits what the receiving humans need to understand the communication then that’s a major win. Now I can ask AI to write me a nice letter and prompt it with a short bulleted list of what I want to say. Boom! Done, and time is saved.
The professional writers who used to slave over a blank Word document are now obsolete, just like the slide rule “computers” of old (the people who could solve complicated mathematics and engineering problems).
Teachers who thought a hand written report could be used to prove that “education” has happened are now realizing that the idea was a crutch (it was 25 years ago too when we could copy/paste Microsoft Encarta articles and use as our research papers).
The technology really just shows us that our language capabilities really are just a means to an end. If a better means asrises we should figure out how to maximize it.
ram@lemmy.ca 1 year ago
Huh?
m0darn@lemmy.ca 1 year ago
Aren’t there very few student priced ai writers? And isn’t the writing done on their servers? And aren’t they saving all the outputs?
Can’t the ai companies sell to schools the ability to check paper submissions against recent outputs?
dyc3@lemmy.world 1 year ago
Chatgpt 3.5 is free. Can’t get more student priced than that.
Regarding the second part about outputs: that’s not practical. Suppose you ignore students running their own LLMs offline on their gaming gpus, where these corps wouldn’t have access to the info. It’s still wildly impractical because students can paraphrase LLM output into something that doesn’t look like the original output.
m0darn@lemmy.ca 1 year ago
Chatgpt 3.5 is free. Can’t get more student priced than that.
Yeah, my point was I don’t think there are many offering the service for free. And they are probably looking for revenue streams.
Suppose you ignore students running their own LLMs offline on their gaming gpus
I actually feel like this is the one that shouldn’t be ignored. But I don’t have a good sense of the computational power vs quality output.
It’s still wildly impractical because students can paraphrase LLM output into something that doesn’t look like the original output.
At least doing that is likely to result in the student internalizing the information to some degree. It’s also not so different (not at all different?) from the most benign academic dishonesty that existed when I was a student.
One issue with the approach I suggested is the copyright issue of profs submitting students’ original work for AI processing without understanding/caring about copyright implications.
cheese_greater@lemmy.world 1 year ago
I would be in trouble if this was a thing. My writing naturally resembles the output of a ChatGPT prompt when I’m not joke answering.
Steeve@lemmy.ca 1 year ago
We found the source
TropicalDingdong@lemmy.world 1 year ago
It’s not unusual for well-constructed human writing to resemble the output of advanced language models like ChatGPT. After all, language models like GPT-4 are trained on vast amounts of human text, and their main goal is to replicate and generate human-like text based on the patterns they’ve observed.
cheese_greater@lemmy.world 1 year ago
You guys 🤗
BananaOnionJuice@lemmy.dbzer0.com 1 year ago
Do you also need help from a friend to prove you are not a robot?
cheese_greater@lemmy.world 1 year ago
I need a lotta help, just not from a friend and about anything robot-related 😮💨