That has always been the two big problems with AI. Biases in the training, intentional or not, will always bias the output. And AI is incapable of saying “I do not have suffient training on this subject or reliable sources for it to give you a confident answer”. It will always give you its best guess, even if it is completely hallucinating much of the data. The only way to identify the hallucinations if it isn’t just saying absurd stuff on the face of it, it to do independent research to verify it, at which point you may as well have just researched it yourself in the first place.
AI is a tool, and it can be a very powerful tool with the right training and use cases. For example, I use it at a software engineer to help me parse error codes when googling working or to give me code examples for modules I’ve never used. There is no small number of times it has been completely wrong, but in my particular use case, that is pretty easy to confirm very quickly. The code either works as expected or it doesn’t, and code is always tested before releasing it anyway.
In research, it is great at helping you find a relevant source for your research across the internet or in a specific database. It is usually very good at summarizing a source for you to get a quick idea about it before diving into dozens of pages. It CAN be good at helping you write your own papers in a LIMITED capacity, such as cleaning up your writing in your writing to make it clearer, correctly formatting your bibliography (with actual sources you provide or at least verify), etc. But you have to remember that it doesn’t “know” anything at all. It isn’t sentient, intelligent, thoughtful, or any other personification placed on AI. None of the information it gives you is trustworthy without verification. It can and will fabricate entire studies that do not exist even while attributed to real researcher. It can mix in unreliable information with reliable information becuase there is no difference to it.
Put simply, it is not a reliable source of information… ever. Make sure you understand that.
Cherry@piefed.social 4 days ago
This is my take. Elon just showed the world what we all knew. The tool is not trustworthy. All other AI suppliers are busy trying to work on credibility that grok just butchered.
Deceptichum@quokk.au 4 days ago
They deliberately engineered prompts on top of the users prompt.
Saying that’s a problem of AI is akin to say me deliberately painting my car badly and saying it’s a problem of all car manufacturers.
Cherry@piefed.social 4 days ago
You are correct. But the right tool in the wrong hands is still non credible in the eyes of perception.