‘Big data’ to drive IT spending

The huge volumes of data generated by today’s digital businesses, known as “big data”, will drive $28bn of worldwide IT spending this year and $34bn next year, according to a forecast from Gartner, the IT research firm.

At the same time, Gartner predicted that by 2015, 4.4m IT jobs will be created to support big data, including 1.9m in the US, but warned that there will be a scramble for the limited number of IT professionals qualified to fill these jobs.

Big data emerged as one of the key themes – and one of four “nexus of forces” – cloud, social collaboration, mobile and information – identified by Peter Sondergaard, Gartner’s research director during his keynote speech launching the company’s four-day annual Symposium in Orlando, Florida.

Mr Sondergaard told the 8,000 company IT professionals including 2,000 chief information officers attending the symposium: “By tapping a continual stream of information from internal and external sources, businesses today have an endless array of new opportunities for transforming decision-making, discovering new insights, optimising the business and innovating their industries.”

He added: “Longer term, we see big data as a complete shift – a shift from today’s ways of accumulating, managing and deploying information as merely a resource to valuing it as any corporate asset.”

Mr Sondergaard noted that companies that develop big data strategies are already being rewarded by Wall Street. “Companies that are serious about collecting, managing and deploying information are now being rewarded by financial markets. Info-centric companies have double the average market-to-book value.”

For example, innovators in areas such as medical devices are turning the data collected by their equipment into information that can be sold online, turning a product into a service.

“So data, information, can become revenue,” he said, but noted that because many companies do not have the skills and time they need internally, they will increasingly turn to external providers. For example, Gartner estimates that 85 per cent of marketing organisations will outsource the analysis of big data.

“Big data is about looking ahead beyond what everybody else sees,” said Mr Sondergaard. “You need to understand how to deal with hybrid data, meaning the combination of structured and unstructured data, and how you shine a light on ‘dark data’ which is the data being collected but going unused despite its value.”

So far, most of the current spending on big data is for adapting traditional IT systems to the big data demands – machine data, social data and widely varied data that comes in bursts. Only about $4.3bn in software sales will be driven directly by demands for new big data functionality in 2012.

“Despite the hype, big data is not a distinct, standalone market; it but represents an industry-wide market force,” said Mark Beyer, research vice president at Gartner. “In 2011, big data formed a new driver in almost every category of IT spending.”

However, the Gartner analysts warn that the window of opportunity for companies to use big data and data analytics to create competitive advantage is a short one.

“Through 2018, big data requirements will gradually evolve from differentiation to ‘table stakes’ in information management practices and technology,” said Mr Beyer. “By 2020, big data features and functionality will be non-differentiating and routinely expected from traditional enterprise vendors and part of their product offerings.”

“Because big data’s effects are pervasive, big data will evolve to become a standardised requirement in leading information architectural practices, forcing older practices and technology into early obsolescence,” he said.

“As a result, big data will once again become ‘just data’ by 2020 and architectural approaches, infrastructure and hardware/software that does not adapt to this ‘new normal’ will be retired. Organisations resisting this change will suffer severe economic impacts.”