November 12, 2012 3:28 pm

Less is more when it comes to ‘big data’

Raw data are like raw sewage: toxic if not handled properly

As elusive as Bigfoot, as addictive as a Big Mac, as sinister as Big Brother: the lure of “big data” is perfect bait for fee-hungry experts hunting new business. It also poses untold risks to companies that fail to read the trend, or the data, correctly.

Raw data are like raw sewage: toxic if not handled properly. And this stuff is raining down on our business and political leaders. The day after Barack Obama’s re-election, Time.com wrote that his campaign team had run 66,000 election simulations a night to assess the odds of his winning swing states. A digital “dashboard” feeds David Cameron, UK prime minister, the essence of 100 sources, including jobs figures, opinion polls and house prices, and gives Phil Clarke, chief executive of Tesco, insight into 173 different innovations being tried out by the world’s third-largest retailer.

More

On this story

On this topic

Andrew Hill

Asked to explain how he avoids being overwhelmed, Mr Clarke told me at last week’s FT Innovate conference that he had boiled down what used to be a 75-page report on Tesco’s operations to a three-page summary, “and I don’t even read the third page any longer”.

Howver, it is still dangerous to lob a big-data dossier on to the CEO’s desk. If the amount of data available for analysis is growing at the rate of 2.5bn gigabytes a day, the number of boardroom references to the topic must be increasing at twice that rate at least. At many companies, the boss is likely to conclude that if the strategic challenge is that big, meeting it must require a big initiative, run by a big team, and backed by a big investment. A consultant told me of one company that had committed nearly $300m over six years to “re-architect” its data. Most of that investment, he pointed out, is bound to be wasted because “everything will have changed by then”.

Despite such warnings, plenty of business leaders are still likely to react to calls for a measured approach in the same way that overambitious sitcom shrink Frasier responds when his brother Niles reminds him that less is more: “Yes – but if less is more, think how much more ‘more’ will be.”

Big data are daunting. Even if companies realise they can no longer merely mine static customer lists, they should not underestimate the technical difficulties of marrying large proprietary databases with the more valuable unstructured, dynamic information that comes from open sources, such as social media or mapping applications. The guidelines for chief executives, on the other hand, are relatively straightforward: verify, purify, simplify.

First, they need to identify the information they hold and ensure it is consistent and comparable. Troy Carter, who, as manager of Lady Gaga, has become an expert in crunching digital numbers, recalls how poor the singer’s customer data were when his team first started collecting them: “We had information from Ticketmaster [the ticket vendor], or our website, but not rich information that told us who the actual fan was . . . from surfing habits to ticket-buying habits.”

Even then, as Sean Carney, chief design officer of Philips, says, data are “like crude oil: it isn’t much use until you start to synthesise it”. Your company may have access to lots of data, but only some of it will be relevant. The second step is to know what you are searching for, and why. Mr Carter recognised that 1m registered fans of Little Monsters, the Lady Gaga website he set up, would be more valuable to him than the 53m who “like” the star’s Facebook page.

Finally, go ahead and put the data to use. If there is a single clear message for managers, it is summed up by Tesco’s Mr Clarke – make big small: “Seventy-four million Clubcard [loyalty card] holders fire their data in from every transaction every day: if that was used without any refining, we would be swamped.” Once the data have been properly marshalled, chief executives can test ideas cheaply and repeatedly. Innovations can flourish – or be allowed to perish without having wasted too much time or money.

Are strategists redundant, then? Not yet. Unlike intelligent fridges that can buy their own groceries online, large data-driven companies can’t order their own strategic direction. Even if they could, they would need someone sitting in front of a dashboard to decide, on the basis of abundant data, which innovation to bless with scarce capital. That, at least, will come as a relief to CEOs, as they struggle to keep their heads above water.

andrew.hill@ft.com

Copyright The Financial Times Limited 2014. You may share using our article tools.
Please don't cut articles from FT.com and redistribute by email or post to the web.