poynter.org/…/who-fact-checks-the-fact-checkers-r…

“‘Fact-checking’ fact checkers: A data-driven approach,” a 22-page October research article from the Harvard Kennedy School Misinformation Review, examined practices of U.S. fact-checking organizations Snopes, PolitiFact and Logically, along with The Australian Associated Press.

Sian Lee, Aiping Xiong, Harseung Seo and Dongwon Lee of Penn State University’s College of Information Sciences and Technology did the peer-reviewed research.

The Penn State researchers found U.S. fact-checking spikes during major news events. In recent years, that was during the COVID-19 pandemic and the 2020 presidential election. Further, the researchers said, misinformation’s spread can mislead and harm people and society.

The researchers examined 11,639 fact-checking articles from Snopes and 10,710 from PolitiFact from Jan. 1, 2016, to Aug. 31, 2022. They found Snopes checked more “real claims” — claims that rate true or mostly true — with 28.7% versus 11% for PolitiFact.

Looking widely, the researchers found high agreement when Snopes and PolitiFact probed the same information. Of 749 matching claims (examining the same information), 521 received identical ratings and 228 (30.4%) had diverging ratings. But, the researchers found nuances caused nearly all of these divergent verdicts — granularity of ratings (Snopes and PolitiFact scales differ slightly); differences in focus; differences in fact-checked information and the different timing of the fact-checks.

Adjusting for these systematic discrepancies, Penn State’s researchers found just one conflicting rating among the 749 matching claims.