If all you’re using it for is to find the relevant parts of the relevant documentation, then a vector search would do as well.
What an LLM can do is synthesize readable docs out of poor or missing docs. When it isn’t hallucinating, that is.
But I don’t actually see anyone using LLMs just to cut through the docs, they’re using it to code. And the results I’ve seen are pretty mixed. It does seem to help ramp up in a new area, but it also seems to become an impediment to moving past the ramp-up phase.
Beginners can now produce beginner code at 2x the speed, and senior folks can now produce beginner code at 3x the speed. But nobody is producing senior level code.
expr@programming.dev 1 week ago
Not only does it not speed up work, it actually slows down work: metr.org/…/2025-07-10-early-2025-ai-experienced-o…