I was very surprised by this part as well. ChatGPT isn’t great when it comes niche subject matter. I feel like this must be an exaggeration, or perhaps he didn’t really validate the results.
Or maybe he just got lucky.
Comment on The employees secretly using AI at work
Seraphin@pawb.social 1 year ago
The article: “I asked a really technical question from my PhD thesis, and it provided an answer that no one would be able to find without consulting people with very specific expertise."
Me waiting for the part where the AI hallucinated and he got an F on his thesis:
I mean, it might have worked out, but you have to be very careful when asking LLMs for factual information. I’ve tried it at my work and it gave me info that contradicted that from experts.
GenderNeutralBro@lemmy.sdf.org 1 year ago
hayes_@sh.itjust.works 1 year ago
It will also contradict itself and make the same mistakes even if you point them out.
Can be useful as a starting point, but you basically need to fact check everything it says.
Ranvier@sopuli.xyz 1 year ago
What kind of PhD thesis? I’m kind of shocked they would say that. I’m in a scientific field, and it falls flat on its face whenever I start to get relatively subspecialized. Not that I haven’t found some uses for it.
lemmyvore@feddit.nl 1 year ago
I believe this is a person who already has a PhD they defended successfully (presumably by doing all the work themselves, if they’ve only just discovered ChatGPT). So we’re talking about an advanced person who used their thesis subject matter to check ChatGPT’s abilities, not to produce their thesis.
Which is exactly the type of user that can benefit the most from ChatGPT because they can verify its output… but YMMV wildly if you’re a beginner in whatever field you’re querying.