Comment on Here’s how to spot AI writing, according to Wikipedia
Perspectivist@feddit.uk 4 days agoYou think you have - but there’s really no way of knowing.
Just because someone writes like a bot doesn’t mean they actually are one. Feeling like “you’ve caught one” doesn’t mean you did - it just means you think you did. You might have been wrong, but you never got confirmation to know for sure, so you have no real basis for judging how good your detection rate actually is. It’s effectively begging the question - using your original assumption as “proof” without actual verification.
And then there’s the classic toupee fallacy: “All toupees look fake - I’ve never seen one that didn’t.” That just means you’re good at spotting bad toupees. You can’t generalize from that and claim you’re good at detecting toupees in general, because all the good ones slip right past you unnoticed.
Just to add to this. I’ve noticed some of my co workers who have developing English skills sound like LLMs sometimes. I imagine this is probably because they’ve only wrote English in a school or work setting and never for personal communication.
…or maybe they’re all just using LLMs idk
I mean I wasn’t going around accusing everyone of being a bot or thinking that I was right all the time. I did have a few false positives and owned up to it. But once you see the pattern of behavior (big gap on joined date vs first active date, only posting in karma farming subs or subs known to have high bot populations) and accounts literally keeping the “as a large language model…” Or “Okay, here’s a supportive Reddit-style comment with some minor spelling mistakes…” In some of their comments.
I have screenshots to prove it but if you really believe I don’t know what I’m talking about then there’s really nothing I can say to dissuade that.
I don’t think anyone is questioning your ability to follow a hunch and get evidence to prove it.
The point is about how often you’re correct on your first guess.
AwesomeLowlander@sh.itjust.works 4 days ago
Studies have shown we’re pretty bad at detecting good AI stuff, regardless of how skilled we think we are. It’s the crappy AI slop that makes everybody think they’re Sherlock Holmes.