With Gemini I have had several instances of the referenced article saying nothing like what the llm summarized. Ie: The LLM tried to answer my question and threw up a website on the general topic with no bearing on the actual question
With Gemini I have had several instances of the referenced article saying nothing like what the llm summarized. Ie: The LLM tried to answer my question and threw up a website on the general topic with no bearing on the actual question
SpraynardKruger@lemm.ee 19 hours ago
Same, especially when searching technical or niche topics. Since there aren’t a ton of results specific to the topic, mostly semi-related results will appear in the first page or two of a regular (non-Gemini) Google search, just due to the higher popularity of those webpages compared to the relevant webpages. Even the relevant webpages will have lots of non-relevant or semi-relevant information surrounding the answer I’m looking for.
I don’t know enough about it to be sure, but Gemini is probably just scraping a handful of websites on the first page, and since most of those are only semi-related, the resulting summary is a classic example of garbage in, garbage out. I also think there’s probably something in the code that looks for information that is shared across multiple sources and prioritizing that over something that’s only on one particular page (possibly the sole result with the information you need). Then, it phrases the summary as a direct answer to your query, misrepresenting the actual information on the pages they scraped. At least Gemini gives sources, I guess.
The thing that gets on my nerves the most is how often I see people quote the summary as proof of something without checking the sources. It was bad before the rollout of Gemini, but at least back then Google was mostly scraping text and presenting it with little modification, along with a direct link to the webpage. Now, it’s an LLM generating text phrased as a direct answer to a question (that was also AI-generated from your search query) using AI-summarized data points scraped from multiple webpages. It’s obfuscating the source material further, but I also can’t help but feel like it exposes a little of the behind-the-scenes fuckery Google has been doing for years before Gemini. How it bastardizes your query by interpreting it into a question, and then prioritizes homogeneous results that agree on the “answer” to your “question”. For years they’ve been doing this to a certain extent, they just didn’t share how they interpreted your query.