STATISTICS, however “accurate,” can mislead us. This is among the main points of British economist and journalist Tim Harford’s very useful book, “The Data Detective.” For example, he says, the U.S. has a notoriously high infant mortality rate for a rich country: 6.1 deaths per thousand live births in 2010. In Finland, he adds, it’s just 3.2. However, Harford says, these numbers do not indicate “a difference in reality, but a difference in how that reality was being recorded.” It turns out, he says, “that physicians in America…seem to be far more likely to record a pregnancy that ends at twenty-two weeks as a live birth, followed by an early death, than as a late miscarriage.” What would be recorded as miscarried pregnancies in other countries are considered live births in the U.S. Hence, America’s “higher” infant mortality rate.
In his native U.K., Harford says when the infant mortality rate rose in 2015-2016 in England and Wales, the press, well, freaked out. “Obesity, poverty, smoking and a shortage of midwives could all be factors, say health professionals,” declared the Guardian newspaper.
However, “a group of doctors, writing to the British Medical Journal, pointed out that official statistics were also recording a dramatic rise in the number of live births at twenty-two weeks of gestation, or even earlier. More and more doctors, it seems, were following the…trend of changing their recording practices to record live births and early deaths, rather than late miscarriages. And this was sufficient to explain the increase in the infant mortality statistics.”
Same reality, but different ways of looking at it.
There is a lesson here, of course:
“Often, looking for an explanation really means looking for someone to blame,” Harford says. “The infant mortality rate is rising — are politicians not providing enough money for the health service, or is the problem caused by mothers smoking or getting fat? The infant mortality rate is lower in London than in the Midlands — what are hospitals in the Midlands doing wrong? In truth, there may never have been anything to blame anybody for at all.” (My italics.)
But where’s the fun in politics — and commenting about politics — if we don’t have anyone to blame for whatever is in the news that we find upsetting?
In any case, when we’re trying to understand a statistical claim, Harford advises us to start by asking ourselves what the claim actually means:
“Measuring infant mortality, at first glance, means doing something sad and simple: counting the babies who died. But think about it for a moment and you realize that the distinction between a baby and a fetus is anything but simple — it’s a deep ethical question that underlies one of the most acrimonious divides in U.S. politics. The statisticians have to draw the line somewhere. If we want to understand what is going on, we need to understand where they drew it.”
Suddenly, the numbers are no longer that simple.
Harford gives us another example of why many “facts” in this world are not that clear-cut:
“The whole discipline of statistics is built on measuring or counting things,” he says. “Michael Blastland, co-creator of [the BBC radio program] ‘More or Less,’ imagines looking at two sheep in a field. How many sheep in the field? Two, of course. Except that one of the sheep isn’t a sheep, it’s a lamb. And the other sheep is heavily pregnant — in fact, she’s in labor, about to give birth at any moment. How many sheep again? One? Two? Two and a half? Counting to three just got difficult. Whether we’re talking about the number of nurses employed by a hospital (do two part-time nurses count as two nurses, or just one?) or the wealth of the super-rich (is that the wealth they declare to the taxman, or is there a way to estimate hidden assets, too?), it is important to understand what is being measured or counted, and how.”
Unfortunately, Harford says, many of us tend to dive “into the mathematics of a statistical claim — asking about sampling errors and margins of error, debating if the number is rising or falling, believing, doubting, analyzing, dissecting — without taking the time to understand the first and most obvious fact: What is being measured, or counted? What definition is being used?” (My italics.)
Harford calls this common pitfall “premature enumeration.”
In politics, where vagueness and motherhood statements are obligatory if you want to have a political career, Harford says we should watch out for politicians who talk about “fairness,” “progress,” “opportunity,” “the right thing to do” — in other words, all politicians.
Even specific-sounding policies, Harford says, can end up meaning very little if we don’t understand the claim.
You’re in favor of more funding for schools? Great. (Who isn’t, but still.) So is it funding increase per student? Per school? Salary increases for teachers only? All personnel? What’s the funding source? Local? Federal? How do you intend to “tap” it? When? If the funding source is local, what department/agency/program will get less so the schools can get more? Or are you also proposing to increase taxes/fees and earmark the additional revenue to schools? Who should pay more taxes/fees? How much? Can you guarantee that the additional revenue will actually go to schools?
Harford says we often mislead ourselves — with statistics.
“Consider the number 39,773. It is the number of gun deaths in the United States in 2017 (this number is from the National Safety Council and is the most recent available from that source). This number, or something very like it, is repeated every time a mass shooting makes the headlines, even though the vast majority of these deaths have nothing to do with these grim spectacles. (Not every mass shooting is headline news, of course. Using the common definition of four people killed or injured in a single incident, there is a mass shooting almost every day in the United States, and many of them would be well down the news editor’s order of priorities.)”
“Gun death” shouldn’t be a complicated concept, Harford says. “A gun is a gun and dead is dead. Then again, nor does ‘sheep,’ so we should pause to check our intuition.”
We usually “hear the number 39,773 at the very moment we are watching news footage showing lines of ambulances and police cars at the sight of some vivid and horrifying slaughter. So we naturally associate it with murder, or even mass murder. In fact, about 60 percent of gun deaths in the United States are suicides, not homicides or rare accidents. Nobody set out to mislead us into thinking gun-related homicides are two and a half times more common than they actually are. It’s just an assumption we understandably make from the context in which we are usually presented with the number.”
Notice that whatever our conclusion is about what we’ve just learned about the number 39,773, it’s “possible to spin it to support various political outlooks.” Again, one set of “facts,” but more than one way of looking and interpreting it.
As thoughtful readers of statistics, Harford says, “we don’t need to rush to judgment either way. Clarity should come first; advocacy can come once we understand the facts.”
Good luck with that.
Send feedback to editor@mvariety.com
