stripping out the source takes away important context that helps you decide wether the information you are getting is relevant and appropriate or not
Many modern models using RAG can and do source with accurate citations. Whether the human checks the citation is another matter.
The AI is trained to tell you something that you want to hear, not something you ought to hear.
While it is true that RLHF introduces a degree of sycophancy due to the confirmation bias of the raters, more modern models don’t just agree with the user over providing accurate information. If that were the case, Grok wouldn’t have been telling Twitter users they were idiots for their racist and transphobic views.
Alpha71@lemmy.world 5 months ago
As I mentioned, Copilot links the sources of the information it gives at the bottom. if you want to double check the information, it is provided to you.
SMillerNL@lemmy.world 5 months ago
And somewhere in the Terms of Service it says you have to give up your first born child. Or maybe it doesn’t, but nobody will ever know because nobody reads more than is strictly required.
wewbull@feddit.uk 5 months ago
The source is just as vulnerable to being hallucinations as anything else it tells you.
laurelraven@lemmy.blahaj.zone 5 months ago
So, when you go to check them… It’s not like the AI is going to hallucinate a valid registered domain with a webserver hosting the hallucinated source as well, so click the link, it’s dead/fake, toss out that reply as suspect.
If you follow the source and find it’s valid, supports what the AI said, and is reasonably trustworthy, then you can consider what it has told you.
If it cites its sources, you have a way to check its math (so to speak).
wewbull@feddit.uk 5 months ago
You have a way to do so, yes, but you actually have to do it and we know people don’t. False sources can just make already believable responses more credible, despite them being full of rubbish.
Turun@feddit.de 5 months ago
I’m pretty sure when searching with AI the model gets told “here are five articles about <user search term>, summarize them and answer the following question: <user input>”