Comment on OpenAI confirms that AI writing detectors don’t work

• cmfhsu@lemmy.world ⁨1⁩ ⁨year⁩ ago

  In statistics, everything is based off probability / likelihood - even binary yes or no decisions. For example, you might say “this predictive algorithm must be at least 95% statistically confident of an answer, else you default to unknown or safe answer”.

  What this likely means is only 23% of the answers were confident enough to say “yes” (because falsely accusing somebody of cheating is much worse than giving the benefit of the doubt) and were correct.

  There is likely a large portion of answers which could have been predicted correctly if the company was willing to chance more false positives (potentially getting studings mistakenly expelled).

  source