FTGraphic Blue matrix of binary code chain background Commuters walk across London Bridge at sunrise in London_Reuters
The amount of digital data around the world is already unimaginably vast

Imagine a world where interminable waits for backward-looking, frequently-revised economic data seem as archaically quaint as floppy disks, beepers and a civil internet. This fantasy realm may be closer than you think.

The Bureau of Economic Analysis will soon publish its preliminary estimate for US economic growth in the first three months of the year, finally catching up on its regular schedule after a government shutdown paralysed the agency. But other data are still delayed, and the final official result for US gross domestic product won’t be available until July. Along the way there are likely to be many tweaks.

Collecting timely and accurate data are a Herculean task, especially for an economy as vast and varied as the US’s. But last week’s World Bank-International Monetary Fund’s annual spring meetings offered some clues on a brighter, more digital future for economic data.

The IMF hosted a series of seminars and discussions exploring how the hot new world of Big Data could be harnessed to produce more timely economic figures — and improve economic forecasts.

Jiaxiong Yao, an IMF official in its African department, explained how it could use satellites to measure the intensity of night-time lights, and derive a real-time gauge of economic health. “If a country gets brighter over time, it is growing. If it is getting darker then it probably needs an IMF programme,” he noted.

Further sessions explored how the IMF could use machine learning — a popular field of artificial intelligence — to improve its influential but often faulty economic forecasts; and real-time shipping data to map global trade flows.

Sophisticated hedge funds have been mining some of these new “alternative” data sets for some time, but statistical agencies, central banks and multinational organisations such as the IMF and the World Bank are also starting to embrace the potential.

The amount of digital data around the world is already unimaginably vast. As more of our social and economic activity migrates online, the quantity and quality is going to increase exponentially. The potential is mind-boggling. Setting aside the obvious and thorny privacy issues, it is likely to lead to a revolution in the world of economic statistics.

Consumer price movements can be measured instantaneously through scraping retailers’ websites, data from Swift — the interbank payment network — maps global capital flows, and spending patterns can be gauged through credit card data. The Federal Reserve now gets the latter with a mere three-day lag, and probably led it to discount the Census Bureau’s unexpectedly grim December data, Goldman Sachs notes.

Most of all, mobile phones are evolving into tiny data beacons, beaming incomprehensible amounts of information about us to the wider world. Ultimately they could form nodes in a vast, granular and instantaneous measure of global economic activity. This is still some way off, and may prove a practical impossibility, but data scientists say it is far from science fiction.

But there are obviously weaknesses in all these funky alternative data sets. For example, the intensity of night-time lights is highly correlated to growth in poorer countries, and works pretty well in middle-income ones, but it has a negligible relationship with economic health in wealthy countries, Mr Yao admitted. Trade volumes are estimated from the depth of a ship’s draft, but if the goods are moved both off and on at a port it complicates the calculations from real-time shipping data.

Perhaps the most high-profile example of the weaknesses of alternative data sets is ADP’s National Employment Report compiled from payroll data. Although it does a good job of predicting the official final jobs data, it can produce head fakes. For example, Goldman Sachs highlights that the ADP data indicated a shallower collapse in employment in late-2008 than was the case.

Yet the biggest issues are not the weaknesses of these new data sets — all statistics have inherent flaws — but their nature and location.

Firstly, it depends on the lax regulatory and personal attitudes towards personal data continuing, and there are signs of a (healthy) backlash brewing. Secondly, almost all of this alternative data is being generated and stored in the private sector, not by government bodies such as the Bureau of Economic Analysis, Eurostat or the UK’s Office for National Statistics.

Public bodies are generally too poorly funded to buy or clean all this data themselves, meaning hedge funds will benefit from better economic data than the broader public. We might, in fact, need legislation mandating that statistical agencies receive free access to any aggregated private sector data sets that might be useful to their work.

That would ensure that our economic officials and policymakers don’t fly blind in an increasingly illuminated world.

robin.wigglesworth@ft.com

Get alerts on Economic statistics when a new story is published

Copyright The Financial Times Limited 2019. All rights reserved.
Reuse this content (opens in new window)

Follow the topics in this article