The Data Detective

This is revolutionary stuff. Yet four years after the original Nature paper was published, Nature News had sad tidings to convey: the latest flu outbreak had claimed an unexpected victim—Google Flu Trends. After reliably providing a swift and accurate account of flu outbreaks for several winters, the theory-free, data-rich model lost its nose for where flu was going.

Next time you’re in a politically heated argument, try asking your interlocutor not to justify herself, but simply to explain the policy in question.

Rozenblit and Keil called this “the illusion of explanatory depth.” The illusion of explanatory depth is a curiosity killer and a trap. If we think we already understand, why go deeper? Why ask questions? It is striking that it was so easy to get people to pull back from their earlier confidence: all it took was to get them to reflect on the gaps in their knowledge.

Kahan was studying a problem we met in the first chapter: the way our political and cultural identity—our desire to belong to a community of like-minded, right-thinking people—can, on certain hot-button issues, lead us to reach the conclusions we wish to reach.

“Kurt Lewin noticed [this effect] in the 1930s,” says Philip Tetlock, referring to one of the founders of modern psychology. “Making public commitments ‘freezes’ attitudes in place. So saying something dumb makes you a bit dumber. It becomes harder to correct yourself.”

Third, superforecasters tended to update their forecasts frequently as new information emerged, which suggests that a receptiveness to new evidence was important.

Tetlock published his conclusions in 2005, in a subtle and scholarly book, Expert Political Judgment. He found that his experts were terrible forecasters.

Tetlock sought clearly defined questions, enabling him with the benefit of hindsight to pronounce each forecast right or wrong. Then he simply waited while the results rolled in—for eighteen years.

If you do have to absorb unwelcome facts, not to worry: you can always selectively misremember them

That is why we can understand a routine telephone conversation on a bad line—until the point at which genuinely novel information such as a phone number or street address is being spoken through the static.