This article is like reading the headline “Researchers have identified the cause of AIDS” and then you open it up and the body is a bunch of science jargon that basically says HIV.

Concerning their origins, we trace these neurons back to the pre-trained base models and find that these neurons remain predictive for hallucination detection, indicating they emerge during pre-training.

it sounds like it’s just how the systems are designed.

I mean, the point of this shit is to take training data and create new stuff out of it through pattern matching. You’re going to get some mismatched shit by design,since the random decisions are modified by the weights. Otherwise you’d get the same shit every time.