With data increasingly on the agenda, the question of whether its ingestion and analysis are worth the cost is moot.
Ever since McKinsey, the consultancy, anointed the term in its May 2011 report Big data: The next frontier for innovation, competition and productivity, business leaders have been seeking to understand the issues at stake.
When Jill Dyché stood up in front of delegates at a data governance conference in London last year, her message was: be on your mettle, executives are taking a keen interest. Ms Dyché, a vice-president at SAS DataFlux, the data management company, had previously written a Harvard Business Review blog post on how to avoid big data “gotchas” that was one of the most popular ever on the magazine’s website.
Big data suffers from a wealth of definition, but it can be reasonably said to include social media data, machine-generated data, and other data types that do not fit neatly into the rows and columns of relational database technologies.
It often extends to text, video and images, although it is debatable whether such “unstructured data” is indeed structured.
John Harris, chairman of the Corporate IT Forum, an organisation of large corporate information technology users including John Lewis, HM Revenue & Customs and BAE Systems, says a skills and personnel gap is the main stumbling block to realising the business value of data.
“There is frustration among CEOs,” he says. “They know there is gold in the hills and do not see why IT is not digging it out. Give me my gold, is the attitude. But IT people are not geologists who know where to dig.”
Harris draws a comparison with the British code breakers at Bletchley Park who helped decipher secret Nazi communications during the second world war: “They were mathematicians and linguists who could [also] think creatively.”
Organisations, according to Mr Harris, need to look for “data scientists” who know mathematics but who also have “business knowledge and the imagination to ask the right questions. They won’t necessarily find them in IT.”
The data-led transformation of business is, he adds, “still in its infancy – more than the IT industry realises. The opportunities are genuine, but it will take a lot of effort.”
Joe Peppard, professor of information systems at Cranfield School of Management, accuses some chief executives of abdicating responsibility for information even though, like money, it is an asset.
Mr Peppard advises company leaders to explore the data their organisations already have, whether internally generated or from, say, social media, and to exploit it for competitive advantage. “Big data is overhyped, but just look at what the Las Vegas casino Harrah’s did,” he says. By analysing data, the casino group discovered that they actually made more money from elderly slot machine players than high-rollers.
In a recent article in Harvard Business Review entitled “Why IT fumbles analytics”, Mr Peppard and co-author Donald Marchand, professor of strategy execution and information management at IMD business school in Lausanne, argued strongly that big data projects should not be treated like conventional large IT projects.
Peppard compared the latter to a relatively straightforward train journey, while the former are more like a voyage of discovery in the age of Christopher Columbus. “There are no maps,” he says.
The pair cited the use by HMRC in the UK of “nudge” tactics to get errant taxpayers to pay up – which involved combining the work of data scientists with that of organisational psychologists – as a good example of getting data analytics right.
But there does have to be a “leap of faith”, Mr Peppard says, as the value of big data analytics will not always be obvious from conventional business-case processes. As a starting point for companies, he suggests establishing a “data lab” to bring together the different teams of people involved.
Narendra Mulani, managing director at Accenture Analytics, the consultancy, says the chief information officers his company works with “understand that their role is evolving dramatically”. “It is a once in a generation opportunity,” he adds.
Like Mr Peppard, he advocates that data analytics should be a matter first and foremost for the chief executive and chief operating officer, with the chief information officer in a “partnership role”.
He says: “It has to be about competitive advantage, [and] quick results that give confidence and comfort are important. For example, in utilities the first thing to do with smart meter data is to build a profile of the customer.
“Companies will compete on the consumption of analytics [and] the crucial thing is to get the business ready to enhance decision-making. How do I get the insight into the right hands, whether it is a sales person or a nurse?”
He points out that, for most of Accenture’s clients, the value of data will be felt in their everyday activities. “It’s not just about the dramatic moves. You can use data to make incremental improvements in efficiency,” he says.
David Friedberg, chief executive of The Climate Corporation, which uses weather data to underwrite insurance for farmers in the US, has some advice for fellow chief executives: “Look at what information you are not capturing that you could be capturing. How can you use that to make your customers happier? Many CEOs don’t even ask what their information assets are”.
Brian McKenna is the business applications editor at Computer Weekly