The most famous version of this might be Millikan’s oil drop experiments to measure the mass of an electron. His notebook is full of qualitative judgments of his measured values and which ones to include in the final determination. The mass of an electron settled down pretty smoothly
Comment on xkcd #3117: Replication Crisis
tal@lemmy.today 1 week ago
I liked once reading an article that showed that with some major findings from important scientists that were later shown to have a wrong value, it wasn’t that a second study promptly “snapped” to the correct values. Instead, over time, subsequent studies incrementally moved to the right value.
On one hand, this is good in that the process does ultimately work, and we got to the right value, though it could take quite some years.
On the other hand, this is embarrassing, because it suggests that people doing follow-up studies to a prestigious person second-guess their own results (“Doctor So-and-So can’t possibly be wrong…it must be me in error”) and aren’t willing to report the full deviation, so they’ll bang on an experiment until they get a value that isn’t that far off and report that.
I can’t seem to find reference to it in the explainxkcd Wikipedia articles, but I remember being intrigued.
Bustedknuckles@lemmy.world 1 week ago
tal@lemmy.today 1 week ago
Ah, yeah, this sounds like it’s making a similar point, though whatever article I read long post-dated Feynman.
en.wikipedia.org/wiki/Oil_drop_experiment
In a commencement address given at the California Institute of Technology (Caltech) in 1974 (and reprinted in Surely You’re Joking, Mr. Feynman! in 1985 as well as in The Pleasure of Finding Things Out in 1999), physicist Richard Feynman noted:[21][22]
We have learned a lot from experience about how to handle some of the ways we fool ourselves. One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It’s a little bit off because he had the incorrect value for the viscosity of air. It’s interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan’s, and the next one’s a little bit bigger than that, and the next one’s a little bit bigger than that, until finally they settle down to a number which is higher.
Why didn’t they discover the new number was higher right away? It’s a thing that scientists are ashamed of—this history—because it’s apparent that people did things like this: When they got a number that was too high above Millikan’s, they thought something must be wrong—and they would look for and find a reason why something might be wrong. When they got a number close to Millikan’s value they didn’t look so hard. And so they eliminated the numbers that were too far off, and did other things like that …
gandalf_der_12te@discuss.tchncs.de 1 week ago
The mass of an electron settled down pretty smoothly
Well it ain’t stupid if it works ¯_ (ツ)_/¯
agamemnonymous@sh.itjust.works 1 week ago
Some people have started distinguishing between “science”, i.e. the scientific method, and "the science’, i.e. the total collective body of results.
“Science” is precious and pure. It’s never right or wrong, it just approaches correctness as it progresses.
“The science” is always inherently suspect since that’s how “science” works, but it’s frequently treated as indisputable fact. This is problematic for a number of reasons, and the replication crisis is at the top of that list.