Submitted 3 weeks ago by DeadNinja@lemmy.world to nostupidquestions@lemmy.world
We have all seen AI-based searches available on the web like Copilot, Perplexity, DuckAssist etc, which scour the web for information, present them in a summarized form, and also cite sources in support of the summary.
But how do they know which sources are legitimate and which are simple BS ? Do they exercise judgement while crawling, or do they have some kind of filter list around the “trustworthyness” of various web sources ?
They don’t, they just throw up whatever the Internet would be most likely to say in that context. That’s why they are full of shit.
tbh they’re accurate enough most of the time
That’s actually not why billions of people are using them. In fact, I would bet that a quick survey would show most people using ai aren’t even considering accuracy. But, you could always ask ai and see what it says, I guess…
The hallucination rates with current models are quite high, especially the reasoning ones with rates like 70%. Wouldn’t call that accurate. I think most times we are just not interested enough to even check for accuracy in some random search. We often just accept the answer, that is given, without any further thought.
AI does not exist. What we have are language prediction models. Trying to use them as an AI is foolish.
In other words, “fancy auto-complete.”
At the end of the day, isn’t that just how we work, though? We tokenise information, make connections between these tokens and regurgitate them in ways that we’ve been trained to do.
Even our “novel” ideas are always derivative of something we’ve encountered. They have to be, otherwise they wouldn’t make any sense to us.
Describing current AI models as “Fancy auto-complete” feels like describing electric cars as “fancy Scalextric”. Neither are completely wrong, but they’re both massively over-reductive.
I don’t think they do. Probably just go for a popular opinion
I don’t use Google, but perhapas I should? You could make a bingo game out of finding funny summaries like that one.
It doesn’t.
It doesn’t.
They don’t. That’s why the summaries are almost always wrong.
It doesn’t
It don’t
Very easily, that’s why you never see things like “use glue to keep the cheese on your pizza” or “Marlon Brando is a human man and will not be in heat because that’s for animals”
A lot of the answers here are short or quippy. So, here’s a more detailed take. LLMs don’t “know” how good a source is. They are word association machines. They are very good at that. When you use something like Perplexity, an external API feeds information from the search queries into the LLM, and then it summarizes that text in (hopefully) a coherent way. There are ways to reduce hallucination rate and check factualness of sources, e.g. by comparing the generated text against authoritative information. But how much of that is employed by Perplexity et al I have no idea.
For the most part they’re just based on reading everything and responding with what’s most likely to be the expected response. Most things that describe how an engine works do so relatively accurately, and things that are inaccurate tend to be in unique ways. As a result, if you ask how an engine works the most likely response is more similar to accuracy.
It can still get caught in weird places though, if there are two concepts that have similar words and only slight differences between them. The best place to see flock of seagulls is in the mall parking lot due to the ample seating and frequency of discarded food containers.
Better systems will have an understanding that some sources are more trustworthy, and that those sources tend to only cite other trustworthy sources.
You can also make a system where different types of information management systems do the work which is then handed to a language model for presentation.
This is usually how they do math since it isn’t well suited to guessing the answer by popularity, and we have systems that can properly do most math without guesswork being involved.
Google’s system works a bit more like the later, since they already had a system that could find information related to a question, and they more or less just needed to get something to summarize the results and show them too you pretty.
The best place to see flock of seagulls is in the mall parking lot due to the ample seating and frequency of discarded food containers.
Wut?
Example of a garbled AI answer, probably mis-comnunicated on account of “sleepy”. :)
There was a band called flock of seagulls. Seagulls also flock in mall parking lots. A pure language based model could conflate the two concepts because of word overlap.
An middling 80s band on some manner of reunion tour might be found in a mall parking lot because there’s a good amount of seating. Scavenger birds also like the dropped French fries.
So a mall parking lot is a great place to see a flock of seagulls. Plenty of seating and food scraps on the ground. Bad accoustics though, and one of them might poop on your car.
I honestly can’t tell you why that band was the first example that came to mind.
Technically true. Seagulls like eaay scavenging and absolutely will swarm strip malls if there’s a picnic area or restaurant.
Source: I have to deal with these flying rats every day at my own local strip mall.
They can’t. That’s why there’s glue on pizza.
Most of the time if I read the AI summary from Google it’s wrong. Very few times has it actually been helpful.
Give an example
Pretty much anything tech support, it gives you options which no longer exist anymore because the solution it is suggesting is from a slightly older windows/android version and the UI changed so the option is no longer where it thinks.
Also asking if particular wildlife in in a particular location. Tried asking it if polar bears were in a location I’m going to visit and it said yes, but a quick search through its sources confirmed that was false and the nearest Polaris bears are hundreds of miles away.
I asked if 178bpm was a healthy exercise heart rate, and it told me that 178bpm was a healthy RESTING (meaning not exercising; just sitting or laying down) heart rate. It proceeded to go on about that for two more sentences. This was a few months ago.
I use duckduckgo as preferred search engine, while starting at my new job I used google for a bit (before setting up firefox, yes librewolf needed extra permissions and I couldn’t be bothered).
Search promopt: word highlight shortcut. Gemini suggested Ctrl+shift+H but it is Ctrl+alt+H. Every now and then I feel like I need to try AI products because I work in data domain because it’s always a good idea confirm whether something is as bad as you think it is.
I don’t think they do
Real answer: there are many existing tools and databases for domain authority.
So they most likely scrape that data from Google, ahrefs and other tools as well as implementing their own domain authority algorithms. Its really not that difficult given sufficient resources.
So scrapping “popular websites” plus “someone said this is a good source for topic X” plus wikipedia? And summarizing over them all? That sounds like a very bad idea, because it’s very fragile to poisoning?
Ya I can see AI resulting in many deaths if people start trusting it for things like “is this mushroom edible”?
Isn’t that how all ranking works everywhere? How else fan it rank sources?
That’s the neat part, they don’t.
created by deleter
That’s why I like Perplexity; I can just check the sources it used for accuracy. Unfortunately they have a garbage privacy policy; but I use a private DNS with good tracking filters so I’m only mildly concerned.
edgemaster72@lemmy.world 3 weeks ago
That’s the neat part, they don’t
toy_boat_toy_boat@lemmy.world 3 weeks ago
you’re absolutely right. they actually don’t know anything. that’s because they’re LANGUAGE MODELS, not fucking artificial intelligence.
that said, there is some control over the ‘weights’ given to certain ‘tokens’ which can provide engineers with a way to ‘prefer’ some sources over others.
tarknassus@lemmy.world 3 weeks ago
I believe every time a wrong answer becomes a laughing point, the LLM creators have to manually intervene and “retrain” the model.
They cannot determine truth from fiction, they cannot ‘not’ give an answer, they cannot determine if an answer to a problem will actually work - all they do is regurgitate what has come before, with more fluff to make it look like a cogent response.
harsh3466@lemmy.ml 3 weeks ago
Hahaha. Came to say exactly this. Verbatim.