Cautionary Tales with Tim Harford - The Data Detective

Episode Date: February 2, 2021

Cautionary Tales' host Tim Harford has a new book - The Data Detective - setting out ten commandments for understanding the numbers, charts, graphs and statistics that bombard us every day. In this free extract, Tim explains his extra "golden" rule that allows us to observe all his other commandments - be curious. Enjoy.Cautionary Tales returns February 26.The Data Detective (Riverhead Books) is published in the US and Canada on February 2. The same book is available elsewhere under the title How to Make the World Add Up.(Audio excerpted courtesy of Penguin Random House Audio from The Data Detective by Tim Harford, narrated by the author.) Learn more about your ad-choices at https://www.iheartpodcastnetwork.comSee omnystudio.com/listener for privacy information.

Pushkin Hello, cautionary tales listeners, Tim Halford here. I have some good news, followed by a treat. The good news is that after long months in the making, the new mega season of cautionary tales is about to appear right here on this feed. 14 episodes of fiasco and catastrophe of nerdy insights and heroic failures, but occasionally not too often, a happy ending. There are murderers, idiots and heroes, there

are fraudsters and fighters and whistleblowers, there are gamblers and gamers and geeks galore all played by a stellar cast of actors. So stellar in fact that I'm still pinching myself and I'm looking forward to revealing their names very soon. I loved writing this series and I really hope that you're going to love listening to it, starting weekly on the 26th of February. And now the treat. Lawyer listeners may know that my new book The Data Detective has just been released

to the US and Canada. My publishers Riverhead Books have kindly agreed to let me share with you the final chapter of the audiobook in which I reveal the golden rule of thinking about numbers in the news. I've been so pleased with the data detective, the international edition was called How to Make the World That Up and was a number one business bestseller in the UK. The data detective is a book about how to think clearly about the world by being wiser about statistics and wiser about ourselves and our cognitive biases. In it I offer 10 simple rules to help you be calmer and smarter as you scroll through social media or scan the headlines.

And plenty of stories too. The book is available wherever books are sold and so is the audiobook read by yours truly. I hope you like the audiobook extract you're about to hear and if you do, look out for the Data Detective, book, ebook and audiobook and please spread the word. The Golden Rule Be Curious I can think of nothing an audience won't understand the only problem is to interest them. Once they're interested, they understand anything in the world. Orson Wells. I've laid down 10 statistical commandments in this book. First, we should learn

to stop and notice our emotional reaction to acclaim rather than accepting or rejecting it because of how it makes us feel. Second, we should look for ways to combine the bird's eye's statistical perspective with the worms eye view from personal experience. Third, we should look at the labels on the data we're being given and ask if we understand what's really being described. Fourth, we should look at the labels on the data we're being given and ask if we understand what's really being described. Fourth, we should look for comparisons and context, putting any claim into perspective. Fifth, we should look behind the statistics at where they came from and what other data

might have vanished into obscurity. Sixth, we should ask who is missing from the data or being shown, and whether our conclusions might differ if they were included. Seventh, we should ask tough questions about algorithms and the big data sets that drive them, recognizing that without intelligent openness, they cannot be trusted.

Eighth, we should pay more attention to the bedrock of official statistics and the sometimes heroic statisticians who protected. 9. We should look under the surface of any beautiful graph or chart. And 10. We should keep an open mind asking how we be mistaken, and whether the facts have changed. I realise that having ten commandments is something of a cliché, and in truth they're not commandments so much as rules of thumb or habits of mind that I've acquired the

hard way as I've gone along. You might find them worth a try yourself, when you come across a statistical claim of particular interest to you. Of course, I don't expect you to run personally through the checklist with every claim you see in the media who has the time for that. Still, they can be useful in forming a preliminary assessment of your new source. Is the journalist making an effort to define terms, provide context, assess sources, the less these habits of mind are in evidence, the louder a lambbell should ring.

10 rules of thumb is still a lot for anyone to remember, so perhaps I should try to make things simpler. I realise that these suggestions have a common thread, a golden rule, if you like, be curious. Look deeper and ask questions. It is a lot to ask, but I hope that it's not too much. At the start of this book, I beg you not to abandon the idea that we can understand the world by looking at it with the help of statistics book, I begged you not to abandon the idea that we can understand the world by looking at it with the help of statistics. I believe we can and should be able to trust that numbers can give us answers to important questions. But of course, Nullius in Verba, we shouldn't trust without also asking questions. The philosopher, Anora O'Neil, once declared,

asking questions. The philosopher, Anora O'Neill, wants declared, well-placed trust grows out of active inquiry rather than blind acceptance. That seems right. If we want to be able to trust the world around us, we need to show an interest and ask a few basic questions. And despite all the confusions of the modern world, it has never been easier to find answers to those questions. Curiosity, it turns out, can be a remarkably powerful thing. About a decade ago, a Yale University researcher, Dan Cajan, showed students some footage of a protest outside an unidentified building. Some of the students were told that it was a pro-life demonstration outside an abortion clinic. Others were informed that it was a gay rights demonstration outside an army recruitment office. The students were asked some factual questions, was it a peaceful

protest? Did the protesters try to intimidate people passing by? Did they scream or shout? Did they block the entrance to the building? The answers people gave, depended on the political identities they embraced. Conservative students who believed they were looking at a demonstration against abortion saw no problems with the protest, no abuse, no violence, no obstruction. Students on the left who thought they were looking at a gay rights protest reached the same conclusion the protestors had conducted themselves with dignity and restraint. But right-wing students who thought they were looking at a gay rights demonstration reached

a very different conclusion, as did left-wing students who believed they were watching an anti-abortion protest. Both these groups concluded that the protesters had been aggressive, intimidating, and obstructive. Gahan was studying a problem we met in the first chapter. The way our political and cultural identity had desired to belong to a community of like-minded, right-thinking people can, on certain hot-button issues, lead us to reach the conclusions we wish to reach. Depressingly, not only do we reach politically comfortable conclusions when parsing complex

statistical claims on issues such as climate change, we reach politically comfortable conclusions regardless of the evidence of our own eyes. As we saw earlier, expertise is no guarantee against this kind of motivated reasoning. Republicans and Democrats with high levels of scientific literacy, are further apart on climate change than those with little scientific education. The same disheartening pattern holds from nuclear power to gun control to fracking. The more scientifically literate opponents are, the more they disagree. The same is true for numeracy. The greater the proficiency, the more acute the

polarisation notes Cajan. After a long and fruitless search for an antidote to tribalism, Cajan could be forgiven for becoming jaded. Yet a few years ago, to his surprise, Cajan and his colleagues stumbled upon a trait that some people have and that other people can be encouraged to develop, which inoculates us against this toxic polarisation. On the most politically polluted, tribal questions where intelligence and education fail, this trait does not. And if you're desperately burning curious to know what it is, congratulations, you may be inoculated already.

Curiosity breaks the relentless pattern. Specifically, Gahan identified scientific curiosity. That's different from scientific literacy. The two qualities are correlated, of course, but there are curious people who know rather little about science, yet, and highly trained people with little appetite to learn more. More scientifically curious Republicans aren't further apart from Democrats on these polarized issues. If anything, they're slightly closer together.

It's important not to exaggerate the effect, curious Republicans and Democrats still disagree on issues such as climate change, but the more curious they are, the more they converge on what we might call an evidence-based view of the issues in question. Or to put it another way, the more curious we are, the less our tribalism seems to matter. There is little correlation between scientific curiosity and political affiliation, happily there are plenty of curious people across the political spectrum. Although the discovery surprised Khan, it makes sense. As we've seen, one of our most stubborn defences against changing our minds is that we're good at filtering out or dismissing unwelcome information. A curious person, however, enjoys being surprised

and hungers for the unexpected. He or she will not be filtering out surprising news because it's far too intriguing. The scientifically curious people Cajan's team studied were originally identified with simple questions buried in a marketing survey so that people weren't conscious that their curiosity was being measured. One question, for example, was, how often do you read science books? Scientifically curious people are more interested in watching a documentary about space travel or penguins than a basketball game or a celebrity gossip show. And they didn't just answer survey questions differently, they also made different choices

in the psychology lab. In one experiment, participants were shown a range of headlines about climate change and invited to pick the most interesting article to read. There were four headlines. Two suggested climate skepticism and two did not. Two reframed as surprising and to or not. 1. Scientists find still more evidence that global warming actually slowed in last decade, skeptical and surprising. 2. Scientists report surprising evidence, Arctic ice melting even faster than expected,

surprising and not skeptical. 3. Scientists report surprising evidence, ice increasing in Antarctic, not currently contributing to sea level rise. Skeptical and surprising. 4. Scientists find still more evidence linking global warming to extreme weather, neither surprising nor skeptical.

Typically, we'd expect people to reach for the article that pandered to their prejudices. The Democrats would tend to favour a headline that took global warming seriously, while Republicans would prefer something with a skeptical tone. Scientifically curious people, Republicans or Democrats were different. They were happy to grab an article which ran counter to their preconceptions as long as it seemed surprising and fresh, and once you're actually reading the article, there's always a chance that it might teach you something. A surprising statistical claim is a challenge to our existing

worldview. It may provoke an emotional response, even a fearful one. Neuroscientific studies suggest that the brain responds in much the same anxious way to facts which threaten our preconceptions as it does to wild animals which threaten our lives. Yet, for someone in a curious frame of mind, in contrast, a surprising claim need not provoke anxiety. It can be an engaging mystery or a puzzle to solve. You're listening to an excerpt of The Data Detective, courtesy of Penguin Random House Audio. The data detective is a brand new book written and narrated by me Tim Halford, and

we'll be back with more after this message. A curious person might, at this point, have some questions. When I met Dan Kaharn, the question that was most urgent in my mind was, can we cultivate curiosity? Can we become more curious and can we inspire curiosity in others? There are reasons to believe that the answers are yes. One reason, says Cajan, is that his measure of curiosity suggests that incremental change is possible. When he measures scientific curiosity, he doesn't find a lump of stubbornly incurious people at one end of the spectrum,

and a lump of voraciously curious people at the other, with a yawning gap in the middle. Instead, curiosity follows a continuous bell curve. Most people are either moderately incurious or moderately curious. This doesn't prove that curiosity can be cultivated, perhaps that bell curve is cast in iron, yet it does at least hold out some hope that people can be nudged a little further towards the curious end of that curve, because no radical leap is required. The second reason is that curiosity is often situational. In the right place, at the right time, curiosity will smolder in any of us. Indeed, Caharn's discovery that an individual's scientific curiosity persisted over time was a surprise to some psychologists.

They had believed, with some reason, that there was no such thing as a curious person, just a situation that inspired curiosity. In fact, it does now seem that people can tend to be curious or in curious. That does not alter the fact that curiosity can be fueled or dampened by context. We all have it in us to be curious We all have it in us to be curious or not about different things at different times. One thing that provokes curiosity is the sense of a gap in our knowledge to be filled. George Lohenstein, a behavioral economist, framed this idea in what has become known as the information gap theory of curiosity. As Lowenstein puts it, curiosity starts to glow when there's a gap between what we know and what we want to know. There's a sweet spot

for curiosity. If we know nothing, we ask no questions. If we know everything, we ask no questions either. Curiosity is fueled once we know enough to know that we do not know. Alas, all too often we don't even think about what we don't know. There's a beautiful little experiment about our in-curiosity conducted by the psychologists Leonard Rosenblit and Frank Kyle. They gave their experimental subjects a simple task, to look through a list of everyday objects such as a flush lavatory, a zip fastener, and a bicycle, and to rate their understanding of each object on a scale of 1-7. After people had ridden down their ratings, the researchers would gently launch a devastating

ambush. They asked the subjects to elaborate. Here's a pen and paper, they would say, please write out your explanation of a flush lavatory in as much detail as he can. By all means include diagrams. It turns out that this task wasn't as easy as people had thought. People stumbled, struggling to explain the details of everyday mechanisms. They had assumed that those details would readily spring to mind, and they did not. And to their credit, most experimental subjects realized that they had been lying to themselves. They had felt they understood zip fasteners and lavatories, but when invited to elaborate,

they realized they didn't understand at all. When people were asked to reconsider their previous one to seven rating, they marked themselves down, acknowledging that their knowledge had been shallower than they'd realized. Rosenblit and Kyle called this the illusion of explanatory depth. The illusion of explanatory depth is a curiosity killer and a trap. If we think we already understand why go deeper, why ask questions, it is striking that it was so easy to get people to pull back from their earlier confidence. All it took was to get them to reflect on the gaps in their knowledge. And as Lowenstein argued, gaps in knowledge fuel curiosity. There is more at stake here than Zip Fastness.

Another team of researchers led by Philip Fernbach and Steve Sloeman, authors of The Knowledge Illusion, adapted the flush lavatory question to ask about policies, such as a cap and trade system for carbon emissions, a flat tax, or a proposal to impose unilateral sanctions on Iran. The researchers, importantly, didn't ask people whether or not they were in favour of or against these policies, as plenty of prior evidence that such questions would lead people to dig in. Instead, Fernbach and his colleagues just asked them the same simple question. Please rate your understanding on a scale of 1 to 7. Then, the same devastating follow-up. Please elaborate, tell us exactly what unilateral sanctions are

and how a flat tax works. And the same thing happened. People said, yes, they basically understood these policies fairly well, then when prompted to explain the illusion was dispelled. They realized that perhaps they didn't really understand at all. More striking was that when the illusion faded, political polarization also started to fade. People who would have instinctively described their political opponents as wicked, and who would have gone to the barricades to defend their own ideas tended to be less strident when forced to admit to themselves that they didn't fully understand what it was that they were so passionate about in the first place. The experiment influenced actions as well as words. Research has found that people became

less likely to give money to lobby groups or other organisations which supported the positions they had once favoured. It's a rather beautiful discovery. In a world where so many people seem to hold extreme views with striking certainty, you can deflate somebody's overconfidence and moderate their politics simply by asking them to explain the details. Next time you're in a politically heated argument, try asking your interlocutor not to justify herself, but simply to explain the policy in question. She wants to introduce a universal basic income or a flat tax or a points-based immigration system or Medicare for all.

Okay, that's interesting. So what exactly does she mean by that? She may learn something as she tries to explain. So may you. And you may both find that you understand a little less and agree a little more than you had assumed. Figuring out the workings of a flush lavatory or understanding what a cap and trade scheme really is can require some effort. One way to encourage that effort is to embarrass

somebody by innocently inviting an overconfident answer on a scale of 1 to 7. But another kind of way is to engage their interest. As Orson Well said, once people are interested, they can understand anything in the world. How to engage people's interest is neither a new problem nor an intractable one. Novelists, screenwriters and comedians have been figuring out this craft for as long as they've existed. They know that we love mysteries, are drawn in by sympathetic characters, enjoy the arc of a good story, and will stick around for anything that makes us laugh. And scientific evidence suggests that Orson Wells was absolutely right.

For example, studies in which people were asked to read narratives and non-narrative texts found that they zipped through the narrative at twice the speed and recalled twice as much information later. As for humour, consider the case of the comedian Stephen Colbert's civics lesson. Before his current role as the host of The Late Show, Colbert presented the Colbert report in character as a blowhard right-wing commentator. In March 2011, Colbert began a long-running joke in which he explored the role of money in US politics. He decided that he needed to set up a

political action committee, a PAC, to raise funds in case he decided to run for president. I clearly need a PAC, but I have no idea what PACs do. He explained to a friendly expert on air. Over the course of the next few weeks, Colbert had packs and superpacks and 501c4s explained to him from where they could accept donations up to what limits with what transparency requirements and to spend on what. He was to discover that the right combination of fundraising structures could be used to raise almost any amount of money for almost any purpose with almost no disclosure. Clearly, Seahorse have created an unprecedented, unaccountable, untraceable cash tsunami that

will infect every corner of the next election he used. And I feel like an idiot for not having one." Colbert later learned how to dissolve his fundraising structures and keep the money without notifying the tax man. By repeatedly returning to the topic and in character demanding advice as to how to abuse the electoral rules, Colbert explored campaign finance in far more depth than any news report could have dreamed of doing. Did all of this actually improve viewers' knowledge of the issue? It seems so.

A team including Kathleen Hall-Jemison, who also worked with Dan Cahahn on the scientific curiosity research, used the Colbert storyline to investigate how much people learned amid the laughter. They found that watching the Colbert report was correlated with increased knowledge about Superpacks and 501c4 groups, how they worked, what they could legally do. Reading a newspaper or listening to talk radio also helped, but the effect of the Colbert report was much bigger. One day a week of watching Colbert, talk people as much about campaign finance as four days a week reading a newspaper, for example, or five extra years of schooling. Of course, this is a measure of correlation, not causation. It's possible that the people

who were already interested in superpacks tuned into Colbert to hear him wisecrack about them. Or perhaps politics junkies know about superpacks and also love watching Colbert, but I suspect the show did cause the growing understanding, because Colbert really did go deep into the details, and large audiences stuck with him, because he was funny. He don't have to be one of America's best love comedians to pull off this trick. The NPR podcast, Planet Money, once shed light on the details of the global economy by designing, manufacturing, and importing several thousand t-shirts. This allowed a long-running storyline, investigating cotton farming, the role of automation in

textiles. How African communities make new fashions out of donated American t-shirts, the logistics of the shipping industry, and strange details, such as the fact that the men shirts which were made in Bangladesh attracted tariff of 16.5%, whereas the women shirts made in Colombia are duty-free. These examples should be models for communication, precisely because they inspire curiosity. How does money influence politics is not an especially engaging question. But if I were running for president, how would I raise lots of money with few conditions and no scrutiny is much more intriguing?

Those of us in the business of communicating ideas need to go beyond the fact check and the statistical Smackdown. Facts are valuable things, and so is fact checking. But if we really want people to understand complex issues, we need to engage their curiosity. If people are curious, they will learn. I found this in my own work with a team who make more or less for the BBC. The programme is often regarded affectionately as a myth-buster, but I feel that our best work is when we use statistics to illuminate the truth rather than to debunk a stream of falsehoods. We try to bring people along with us as we explore the world around us with a help of reliable numbers. What's false is interesting, but not as interesting as what's true.

After the referendum of 2016 in which my fellow British voters decided to leave the European Union, the economics profession engaged in some soul-searching. Most technical experts thought that leaving the EU was a bad idea, costly, complex, and unlikely to deliver many of the promised benefits or solve the country's most pressing problems. Yet, as one infamous sound bite put it, the people in this country have had enough of experts. Few people seem to care what economists had to say on the subject,

and to our credit I think, professional economists wanted to understand what we had done wrong, and whether we might do better in future. Later, at a conference about the profession and the public, the grades and the good of the British economics community pondered the problem and discussed solutions. We needed to be more chatty and approachable on Twitter suggested one analysis. We needed to express ourselves clearly and without jargon offered many speakers, not unreasonably.

My own perspective was slightly different. I argued that we were operating in a politically polarized environment, in which almost any opinion we might offer would be fiercely contested by partisans. Economists deal with controversial issues such as inequality, taxation, public spending, climate change, trade, immigration, and of course Brexit. In such a fibrial environment, speaking slowly and clearly will only get you so far. To communicate complex ideas, we needed to spark people's curiosity, even inspire a sense of wonder. The great science communicators, after all, people such as Stephen Hawking and David Aschembre, do not win over people simply by using small words crisply spoken.

They stoke the flames of our curiosity, making us burn with desire to learn more. If we economists want people to understand economics, we must first engage their interest. What is true of economists is equally true for scientists, social scientists, historians, statisticians, or anyone else with complex ideas to convey. Whether the topic is the evolution of black holes or the emergence of black lives matter, the possibility of pre-cognition or the necessity of pre-registration, the details matter. And presented in the right way, they should always have the capacity to fascinate us. A way can our sense of wonder, I say, to my fellow nerd communicators, ignite the spark of curiosity and give it some fuel using the time-honoured methods of storytelling, character,

suspense and humour. But let's not rely on the journalists and the scientists and the other communicators of complex ideas, we have to be responsible for our own sense of curiosity. As the saying goes, only boring people get bored. The world is so much more interesting if we take an active interest in it. The cure for boredom is curiosity, goes an old saying, there is no cure for curiosity. Just so. Once we start to peer beneath the surface of things,

become aware of the gaps in our knowledge, and treat each question as the path to a better question, we find that curiosity is habit-forming. Sometimes, we need to think like Darryl Huff. There is a place in life for the mean-minded, hard-nosed skepticism that asks, where's the trick? Why is this line bastard lying to me? But while I don't believe it, it's sometimes the right starting point, when confronted with a surprising statistical claim, it is a lazy and depressing place to finish. And I hope you won't finish there.

I hope that I have persuaded you, we should make more room, both for the novelty-seeking curiosity that says, tell me more, and the dogged curiosity that drove Austin Bradford Hill and Richard Doll to ask why so many people were dying of lung cancer and whether cigarettes might be to blame. If we want to make the world add up, we need to ask questions, open-minded, genuine questions. And once we start asking them, we may find it delightfully difficult to stop. That was an extract from my new book, The Data Detective. The international edition is how to make the world that up. Thanks for listening and keep on listening because cautionary tales is back on the 26th of February.

you