PFB Research: Yes, Virginia, PolitiFact is biased against Republicans

It was late during PolitiFact's first year, 2007, when I started to recognize its Achilles' heel.

PolitiFact was putting a rating on its fact checks.  PolitiFact was giving researchers a tool for measuring differing treatment of its subjects according to their politics.

I thought "They have to realize what they're doing.  Don't they?"

PolitiFact's been with us for over five years, now, and the national operation appears to lack any inkling as to how its rulings can serve to expose journalistic bias. 

It took me a few months after the first realization to develop a means of measuring PolitiFact's bias.  During the first few years I did a number of word studies that helped confirm an anti-conservative bias at PolitiFact.  But there was a problem with those studies.  PolitiFact's existence made up such a short span that I had to work with a very limited pool of data.  The small pool of data limited the usefulness of the studies.  So I sat on them, waiting for PolitiFact to expand the database and dreaming up new research methods.

Early last year I realized that PolitiFact's own rating system created a natural opinion poll for PolitiFact journalists.  PolitiFact distinguishes between its "False" and "Pants on Fire" claims according to a single criterion:

FALSE – The statement is not accurate.

PANTS ON FIRE – The statement is not accurate and makes a ridiculous claim.

"False" and "Pants on Fire" claims are not accurate.  "Pants on Fire" claims are, in addition, ridiculous.  Ridiculous means subject to ridicule.  Ridicule is, of course, the specialty of objective journalists.

Eventually I moved the "Pants on Fire" study to the highest priority on the research list.  I did so for the simple reason that PolitiFact has provided a good amount of data for this study.  And those data, now that the study is finished, tell a story of a strong anti-GOP bias at PolitiFact's national operation.

We've posted a fairly detailed account of the study at Google Docs.  Anyone with a Google account can log on with Google Docs and read the detailed version.  With this post I'll offer a brief summary.

Republicans are about 74 percent more likely than Democrats to receive a "Pants on Fire" rating--74 percent more likely to speak not just the false but the ridiculous in the eyes of PolitiFact's national operation.  It's not an easy statistic to explain without liberal bias.

Republicans lie more?  That gets us to the conclusion that Republicans will receive more combined "False" and "Pants on Fire" ratings by percentage as well as in comparison to Democrats.  It doesn't explain "ridiculous" without some sort of objective measure.  So what's that measure?  I looked for it in the texts of the stories based on the idea that PolitiFact has the criteria but doesn't reveal them on its "Principles" page.  I couldn't find the criteria in the texts.

So, yes, PolitiFact national is biased against Republicans in its use of the "Pants on Fire" ratings.  That doesn't rule out other forms of bias, either.  We're still working on those, but we've got the data for "Pants on Fire" comparisons on the states lined up except for PolitiFact New Hampshire, which has a pretty thin record so far.

More to come on the research front.  Check our PFB research page.