Gender bias in data affects health and wealth

Nearly seven decades ago, the noted psychologist Solomon Asch gave a simple task to 123 experimental subjects: to pick which one of three quite different lines was the same length as a “reference” line. Asch had a trick up his sleeve: he surrounded each subject with stooges who would unanimously pick the wrong line. Confused, the experimental subjects were often — not always — swayed by the error of those around them.

I’ve written before about these experiments, but there’s something I neglected to mention: not a single one of the stooges nor the experimental subjects was female.

If Asch had conducted all-women experiments, he would have discovered that women tend to conform to the group more often than men. Perhaps this omission doesn’t matter. Retellings of the Asch experiment have tended to exaggerate the conformity that was demonstrated, while glossing over the fact that it was an all-male study. The two biases may cancel each other out.

Still, it is a lesson in how easy it is to ignore important data — or to assume that they are comprehensive, when in fact they omit half the planet.

Invisible Women, a new book by Caroline Criado Perez, explores countless cases in which everything from the height of the top shelf to the functionality of an iPhone is predicated on the assumption that the user will be male. (Apple once released a “comprehensive” health app that could track your selenium intake but not menstruation.)

Much of this imbalance has nothing to do with data, but some of it does — and this “gender data gap” is particularly important because good statistics are one of the only windows we have into the lives of an entire population, rather than just a handful of friends.

Consider the UN Sustainable Development Goals, admirable targets to improve the lives of 7.5bn people. Yet as development economists Mayra Buvinic and Ruth Levine have pointed out, while one of these goals is gender equality, we lack much of the data on whether that goal is being achieved.

Some missing numbers — for instance, on sex-trafficking — are obvious. Others are more subtle, such as the ubiquitous choice to measure the income not of individuals but of households. Does that household income come from a man, a woman or both? 

We shouldn’t assume that the balance between “wallet and purse” is irrelevant. Economist Shelly Lundberg and colleagues studied what happened when in 1977, child benefit in the UK was switched from being a tax credit (usually to the father) to a cash payment to the mother. That measurably increased spending on women’s and children’s clothes relative to men’s. The UK’s new universal credit is payable to a single “head of household”; that curious decision may well favour men. Given the data we have, it will be hard to tell.

The story is often told of the accidental discovery of sildenafil (Viagra). Intended as a treatment for angina, the clinical trial revealed a side-effect: magnificent erections. Had the original trial included women, we might have fortuitously discovered a treatment for severe period pain. As it was, men got their miracle drug but women are still waiting. We can’t confidently prescribe sildenafil as a safe and effective treatment for period pain because, as Ms Criado Perez reports, only a small and suggestive trial has yet been funded.

There are many data gaps out there — statistician David Hand calls them “dark data”. There are the unpublished studies that produce less interesting or lucrative results than published ones. There are the voters who are coy about confessing their voting intentions to pollsters — the “shy Tory” effect. There are the psychology experiments that study only “WEIRD” subjects — Western, Educated and from Industrialised Rich Democracies.

We have plenty of statistics on shares and currencies, but not much on debt and derivatives. What Gillian Tett called the submerged iceberg of financial markets got less attention, until it turned out to have holed the entire financial system below the waterline in 2007.

There is no simple way to shine a light on all this dark data. There is a reason why it is easier to collect statistics in a rich country than a poor one, and why fluent speakers of English are more likely to fill in the UK census form. Collecting data on who bears the burden of “life admin” is harder than collecting that on primary paid occupations. But what we count and what we fail to count is often the result of an unexamined choice. We can make better choices, both by involving ordinary citizens in survey design, and by trying to get more women and minority groups into economics and statistics.

It is hardly encouraging that the Office for National Statistics has just been found wanting in a sexual discrimination case, but there is hope. The president of the Royal Statistical Society, the managing director of the International Monetary Fund and both heads of the UK Government Economic Service are all women. Still, plugging data gaps takes time, and considerably more thought than I once gave to Solomon Asch’s curious experimental line-up. 

tim.harford@ft.com