Part 3 of the TED Radio Hour episode Can We Trust The Numbers?
About Alan Smith's TED Talk
For years, Alan Smith analyzed the most reliable demographic data in the U.K. – the census. He noticed people's perceptions conflicted with reality, and wondered if there was a way to bridge the gap.
About Alan Smith
He previously worked at the U.K. Office for National Statistics (ONS) as their Head of Digital Content, where he developed an app that quizzed U.K. citizens on whether their perceptions of their neighborhood matched with demographic data from the census.
In 2010, he was an inaugural recipient of the Royal Statistical Society's Award for Excellence in Official Statistics. He was appointed Office of the Order of the British Empire (OBE) in the Queen's 2011 Birthday Honours list.
GUY RAZ, HOST:
Can I ask you a philosophical question?
ALAN SMITH: Sure.
RAZ: Is the right or the correct statistic an objective truth?
SMITH: Whoa. That's a very good question.
RAZ: This is Alan Smith.
SMITH: I'm the data visualization editor at the Financial Times.
RAZ: And Alan says, unlike an algorithm, a statistic can be a more reliable measure of what's true. But in a certain sense, even that doesn't matter.
SMITH: People have this idea, this kind of binary notion that it's either right or wrong. But what's really interesting is seeing people's reactions to it, about how they're going to use that information.
RAZ: Before Alan worked in journalism, he spent over a decade working for the U.K.'s Office of National Statistics.
SMITH: So that's kind of like the U.K. equivalent of the Census Bureau.
RAZ: And while he was there, he noticed something that really intrigued him.
SMITH: When we do a census, the whole point is to try and kind of capture everybody...
SMITH: ...In the census so we know more about ourselves. And it just struck me how little we knew about ourselves. When I would talk to people about my job and describe the sort of data we were working with, it never really seemed to marry up with other people's views. And that's really interesting. It always fascinated me that the way that people were perceiving the world was different from reality.
RAZ: So, for example, Alan says that one annual survey by a U.K. polling firm has captured this difference pretty well, the difference between perception and reality. Alan described the results from the TED stage.
(SOUNDBITE OF TED TALK)
SMITH: They did a survey of over 1,000 adults in the U.K. and said, OK, for every 100 people in England and Wales, how many of them are Muslim? Now, the average answer from this survey - which was supposed to be representative of the population - was 24. British people think 24 out of every 100 people in the country are Muslim. Now, official figures reveal that figure to be about 5.
They asked Saudi Arabians - for every 100 adults in your country, how many of them are overweight or obese? And the average answer was just over a quarter. The official figures show actually it's...
SMITH: ...Nearer to three-quarters. And I love this one. They asked - in Japan, they said, for every 100 Japanese people, how many of them live in rural areas? And the average - again, this is the average - 56 out of every 100 Japanese people lived in rural areas. The official figure is seven - so extraordinary variations and surprising to some, but not surprising to people who've read the work of Daniel Kahneman, for example. Him and his colleague, Amos Tversky, spent years researching this disjoint between what people perceive and the reality, the fact that people are actually pretty poor intuitive statisticians. And there are many reasons for this. Individual experiences certainly can influence our perceptions but so, too, can things like the media reporting things by exception. Kahneman had a nice way of referring to that. He said, we can be blind to the obvious - so we've got the numbers wrong - but we can be blind to our blindness about it. And that has enormous repercussions for decision-making.
RAZ: I mean, you can imagine that this might have really big consequences. Like, if people perceive, for instance, that the percentage of Muslims in the U.K. is so much higher than it is in reality, that could determine how they vote.
SMITH: Exactly. And then, you know, that - for me, that's just astonishing because that's not exactly information that we've been trying to hide. That's information that had been broadcast for a good year or two after the census. And yet, people still had their own impression.
RAZ: Why do you think so many people trust their intuitions and their lived experiences more than a statistic that is quantifiable, reliable and, for lack of a better word, true?
SMITH: Why do people trust their intuitions? I think because it keeps them alive, right? Like, it does what it needs to do, which is makes you run away from a lion before you've even realized there's a lion running at you, right? Like, so intuitions, without being precise, can be very, very valuable. But I think that people find it difficult to relate to statistics when they're about the aggregate, right?
SMITH: Like, no one actually thinks of themselves as 1 of 65 million people.
SMITH: You're probably thinking of yourself, your family, your colleagues, the people that you interact with regularly. And that's a very different world from the one that's generally reflected in official statistics.
(SOUNDBITE OF TED TALK)
SMITH: So at the statistics office, while this was all going on, I had an idea, which was if we reframed the questions and say - how well do you know your local area? - would your answers be any more accurate? So I devised a quiz. How well do you know your area? It's a simple web app. You put in a post code, and then it will ask you questions based on census data for your local area.
There are seven questions. Each question - there's a possible answer between zero and a hundred. And at the end of the quiz, you get an overall score between zero and a hundred. And so the first question is, for every 100 people, how many are aged under 16? You drag the slider to highlight your icons and then just click submit to answer. And we animate away the difference between your answer and reality.
RAZ: OK. So your theory here was that people would know their own neighborhood better - you know, like, the place where they live and work and see people every day - that their perception of their local area would be more accurate.
SMITH: Yeah, that's right. And, I mean, in fact, it turns out, people weren't any better with their local areas. You know, they just weren't. And we found that very many people were overestimating even in their local area things like the proportion of people who were Muslim. They were amazed at how many people did or didn't own a car in their street - kind of really simple stuff like that. The one that I found really interesting was - I actually thought the demographics - like, how old or young your area was - I thought people would nail that, but it turned out lots of people got that wrong, too. And it was really interesting to get that kind of...
SMITH: ...Their view of the neighborhood relative to the official figures.
(SOUNDBITE OF TED TALK)
SMITH: It turns out, the reaction was more than I could've hoped for. It was a long-held ambition of mine to bring down a statistics website due to public demand.
SMITH: This URL contains the words statistics, gov and U.K., which is 3 of people's least-favorite words in a URL. And the amazing thing about this was that the website came down at quarter to 10 at night because people were actually engaging with this data of their own free will, using their own personal time. I was very interested to see that we got something like quarter of a million people playing the quiz within the space of 48 hours of launching it. And it sparked an enormous discussion online on social media, which was largely dominated by people having fun with their misconceptions, which is something that, you know - I couldn't have hoped for any better in some respects.
It's really because statistics are about us. If you look at the etymology of the word statistics, it's the science of dealing with data about the state or the community that we live in. So statistics are about us as a group, not us as individuals. And I think as social animals, we share this fascination about how we as individuals relate to our groups, to our peers.
RAZ: Do you think that the better we get at measuring things and coming up with reliable statistics, the more sort of progressive we become as humans, that we just continue to move forward and make better decisions?
SMITH: I think better data is something that is a vital ingredient for progression. But I think changing the way that people interact with data, for me, is absolutely critical. And in fact, on that point, the thing I think that was really interesting was that I had quite a few teachers write to me. One said thank you, finally. They said, now we've got a way of trying to introduce younger children to why we even bother with this statistics stuff. And they said what they'd done is they'd got these children in their classes to do the quiz for the areas that they live. They were then encouraged to discuss the differences.
And just that reasoning with data - I think what we've really got to think about is how we start to promote things like statistical reasoning as an integral part of our education. And, I mean, I think my interest originally in going to work for the statistics office was this idea that there was some public good to be derived from it. And I think people are now hoping that we're going to find better ways of applying it in future.
RAZ: Alan Smith is the data visualization editor at the Financial Times. You can see his entire talk at ted.com. Today on the show, Can We Trust The Numbers? I'm Guy Raz, and you're listening to the TED Radio Hour from NPR.
(SOUNDBITE OF MUSIC) Transcript provided by NPR, Copyright NPR.