February 2, 2012 9:50 pm

Entering the sense-making era of big social data

Every individual on Amazon.com gets a customised experience, based on the actions they take on the site. Harrah’s, now called Caesars Entertainment, increased its customers’ ‘share of wallet’ by offering a Total Reward card that enabled it to track customers’ purchasing patterns and thus continuously refine its offers in ways that most enticed further customer buying.

For decades credit card firms have been analysing our buying patterns to sense risky behavior or unauthorised use. To some it is old news that data is the new oil, yet these examples reflect the growing power of data analytics in how we buy, sell, serve - and sometimes even design or refine what we offer. The next wave of innovation promises to have an even more pervasive impact on our work and our life. It also holds bigger opportunities for organisations to create differentiating value for those they serve.

While revolutionary, these first waves of data analytics only tap information within the closed system in which a company operates. Amazon, for example, captures an increasingly rich profile of each site visitor and customer’s habits by studying their data trails while on the site. Many businesses enlarge their understanding of customers by buying that data from each other. They more efficiently connect with their customers’ needs by tracking purchasing histories, demographics, and how customers engage with them.

Yet they are limited by the walled garden of data drawn from the closed or proprietary mechanisms they use. The data is considerable yet it is not social. It lacks a broader context about the relationships and behaviors of the people creating them. That context can be gleaned from what is stored and shared openly on the Internet, or can be captured via mobile and tablet usage. Just as airplanes leave contrails across the sky, we, as consumers, leave an increasing amount of revealing details about our choices. Smart companies are observing and analysing our “social” choices, especially from what can be observed in social interactions, such as co-creating, sharing, liking and following. These behaviors are often more accurate than surveys, for example, that ask consumers, what they believe, want and will do.

Even without the social capacity, this data tsunami has enabled firms to predict prices, reconfigure supply chains and shape consumer behavior. This so-called ‘Big Data’ flood of information refers to the digital wave of emails, voicemails, pictures, video files and more. These terabytes and petabytes (a petabyte is 1,000 terabytes) of information are being crunched to reveal patterns and to predict individual and collective behavior. The rapidly growing volume of such electronic data is staggering.

Digital information is doubling every 1.2 years and will exceed 1,000 exabytes (an exabyte is 1,000 petabytes) next year according to the MIT Center for Digital Research. Picture the flood of data growth this way. In 2010, medical centers held almost 1bn terabytes of data. That is almost 2,000bn file cabinets worth of information. Capacity to store and access that Big Data means, for example, that a medical center can scan a heart in just a second, and then improve prognostication by comparing it with thousands of other heart scans in its database.

The extraordinary value of Big Data will drive creation of new tools and systems to facilitate pattern recognition and other intelligence in consumer behavior, economic forecasting, and capital markets. Market domination may, in part, be driven by which companies absorb and use the best data fastest.

Understanding the ”social” context of individuals’ and organisations’ actions means a company can track, not just what their customers do, but get considerably closer to learning why they do what they do.

This new era in sense making elevates our power of discernment about customers’ actual motivation and our capacity to design behavior-evoking triggers. Data from media and advertising already tells us how long people view content, how often they visit, and what they seek. Social data can tell us who is in a consumer’s network and what they and their friends like and even stock market movements.

The Center for Complex Networks and Systems Research found that tracking mood sentiment via Twitter enabled them to predict the daily up and down changes in the closing values of the Dow Jones Industrial Average with up to 87.6% accuracy.

Companies like Recorded Future formed a group of startups to visualise the future by linguistic and other analysis of open data. These companies are working with brands, governments and organisations to help predict outcomes as diverse as in stock fluctuations, political unrest, technology launches and energy market changes.

Yet to maximise the use of Big Data in predicting human behavior, requires acting more like anthropologists than market researchers, advises Francois Gossieaux, president of Corante. He suggests that we, “don’t try to understand online social behavior by doing traditional qualitative market research. Instead, focus on observing what happens, make assumptions and predictions based on basic human cultural behavior (need for status, need to hang out with like-minded people, need to impress others, being competitive among groups, etc.), and validate those assumptions through qualitative interviews and more observation.”

Bottom line? It is not a matter of ‘if’ but ‘when’ Big Data is used in your sector. Already individuals are collectively generating big data by sharing information on their self-monitoring experiments of conditions as diverse as chronic health issues to exercise in the rapidly-growing group, Quantified Self. Their results have proved sufficiently valuable to attract partnerships with research institutions and other organisations. Tracking and analysing big social data may enable your organisation to more adeptly experiment with what it provides to whom and how - before your competitors do.

Perhaps this quick snapshot of others’ modes of capturing value from ‘Big Social Data’ will spur your organisation to pursue this opportunity. Some organisations may find it beneficial to appoint a ‘Chief Analytics Officer.’ We’d be interested in learning from your experience.

Eric Openshaw is the vice chairman and US Technology, Media & Telecommunications leader at Deloitte. JR Reagan is a principal at Deloitte & Touche LLP and leads the Deloitte Analytics Institute located in Arlington, Virginia.

Copyright The Financial Times Limited 2015. You may share using our article tools.
Please don't cut articles from FT.com and redistribute by email or post to the web.