Big hopes for big data spending

Data are often described as the oil of the 21st century, so it should not be surprising that governments around the world are looking to tap that resource through the use of big-data technologies.

The potential is huge – big data is a market so large and diverse that it has yet to be valued in any meaningful way. But in an age of austerity and increased privacy awareness, can the spending be as big as the data?

Forrester, the technology research company, defines big data as “techniques and technologies that make capturing value from data at extreme scale economical”. In other words, number crunching that a few years ago would have taken weeks can now be done in real time.

In the US, President Barack Obama, whose use of big data on the campaign trail helped him to a second term in office, last year announced $200m in funding to establish the Big Data Research and Development Initiative with input from the country’s science, health and defence agencies.

In 2009, the UN established Global Pulse to track the effects of socioeconomic crises around the world by using huge amounts of data and real-time analytics technology. The idea is that trends will be identified that can better protect populations from such shocks as the global financial crisis. So far, Global Pulse has managed only consultations and proofs of concept, but the organisation says it will begin new projects this year.

In the UK, where cutting the budget deficit is high on the agenda, a recent report by Policy Exchange, a think-tank, estimated that using big data could save the public sector up to £33bn a year in lost time – equivalent to between £250 and £500 per head of the population.

Stephen Kelly, the UK chief operating officer for government, says big data is a key part of his strategy to take £20bn off the £140bn spent each year on information technology, telecoms and property management.

Kelly says governments have lost their position as early adopters of technology. “In the 1980s, government was a pioneer of data innovation – in the US too – and the private sector followed. There was a dearth of technology in consumer markets. In the past couple of years it’s flipped on its head, but the good news for us is that we can leapfrog into 2013.”

Now, the UK government is using the latest analytics technology to zero in on fraud, errors caused by the re-keying of data, and chasing debts.

“We see a £20bn-£40bn leakage from fraud, error and debt. That’s an area where we have started looking at prototypes around analytics and data, and that will play a bigger role,” Mr Kelly says.

Big data is transforming procurement costs, too, he says. A £350m Oracle-based database, for example, can now be replaced by a cloud service at a 10th of the cost.

The change is even more rapid in the health sector, where the NHS has been asked to save £20bn a year by 2015.

“For most hospitals one of the things we need to focus on is how do we get down to almost patient-level costing – how do we understand the true cost of every intervention, every bed day of a patient’s stay?” says Orlando Agrippa, director of business informatics at Colchester Hospital University NHS Foundation Trust.

Using a cloud-based tool called Qlikview, Colchester staff analyse information about diagnostics, operating theatre throughput, outpatient and inpatient stays, waiting times and emergency services.

“Every single day the service managers accountable for wait times use Qlikview as the only source of validating the patients who are on or off the list and need to be treated quickly,” Mr Agrippa says.

The UK is not alone. In a trial in Stockholm, Sweden, authorities were able to cut road travel times in half and reduce emissions by 20 per cent by analysing 250,000 GPS signals per second to direct traffic. A permanent gain on this scale would have a marked effect on productivity and, in theory, economic output.

The UK government’s hopes for big data are not restricted to the public sector. With the manufacturing and finance industries in retreat, many hope data will spawn a new industry to power economic growth. The UK’s Open Data Institute was founded with this in mind.

According to Gartner, the technology research company, big data drove $28bn of IT spending in 2012, and will lead to the creation of 4.4m jobs by 2015.

Further research by Forrester suggests 20 per cent of companies have implemented big data technology, and 37 per cent are planning a big data project.

Nevertheless, economic growth is not assured. Researchers say some big data technologies are between two and five years from mainstream adoption, and privacy remains the biggest obstacle to potential business models.

Holger Kisker, a principal analyst at Forrester and author of its forthcoming Forrsights big data survey, says practical privacy laws are the key to unlocking a range of new products and services that can drive growth.

“We are making a lot of problems for ourselves by going into this with a very high sensitivity of data privacy issues,” he says.

Mr Kisker says policy makers in Europe have given no timeline for proposed privacy and cybersecurity legislation, and that this is holding back investment in new business models around customer data.

“In the US, companies are much more open to taking risks, but in the UK and Europe, companies are holding back and we are moving to a situation of competitive disadvantage over time.”