Dealing with the data deluge

How scientists and engineers are attempting to structure the trillions of bits of information pouring out from billions of sources and to facilitate fast retrieval

Shirley Ann Jackson, a prominent figure in US science policy

At the turn of the millennium, many pundits described the 20th century as one of physics-based innovation and anticipated the 21st as the century of biology. Now, 13 years in, it looks increasingly as though we are living in the century of data.

Though research in biology and medicine continues to make exciting progress in the lab, it has yet to make much difference to the lives of most people. By far the biggest changes so far this century are due to the processing and communication of data (in the broad sense) from mass participation in the internet and associated social media to the digital takeover of writing, music and photography.

The issue of “big data” – how to make the most of the truly gigantic deluge of data to emerge – is exercising the minds of many scientists and engineers. Trillions of bits of information are pouring out from billions of sources. Besides conventional websites, we have social networks leaving behind “digital crumbs” for us to study, and the sensors embedded in everything from cars to cameras, creating the “internet of things”. This data is largely unstructured.

Shirley Ann Jackson, president of Rensselaer Polytechnic Institute, New York, and a prominent figure in US science policy for two decades, is one of those considering this issue. I caught up with her on a visit to London to deliver a speech at the Royal Academy of Engineering.

“One could say that, concerning big data, we are still pre-web,” Jackson says. “The world wide web is one huge ‘library’ but it has not yet provided uniform access to data. In a word, there is no Google for all data.”

Recovering information is difficult because there is no consistent system for tagging data to identify its origins, history, context, rights and so on. “We need better means to take what may be implicit in the data, and obvious in context, and make that explicit in its description,” Jackson says. “We also need to improve the credibility of information by automating processes that cross-reference and cross-check.”

One remedy is the “semantic web”, a collaborative movement led by the World Wide Web Consortium to promote common data formats. The goal is to have a global mesh of information linked in a way that is easily processed by computers. The approach is based on semantic technology that encodes meanings separately from data in content files. This will allow intelligent software agents to search for connections among different data, by “semantic inference”. “One only can imagine what the impact will be, once this work is completed,” Jackson observes.

Jackson boasts several firsts as an African-American woman, including being the first elected to the US National Academy of Engineering and the first to lead a top research-oriented university. She is a passionate advocate of interdisciplinary working as the way to draw more power from the data deluge.

It is fashionable now to talk about the need to knock down academic silos and collaborate across all disciplines. But Jackson has been putting that approach into practice at Rensselaer since becoming president in 1999.

In Britain the term polytechnic has somewhat unfortunate associations of two-tier academic institutions, but Jackson has a much more positive view – and indeed advocates the “new polytechnic” as an umbrella term to encompass multidisciplinary working to tackle the problems and opportunities of big data. “I define the new polytechnic as an entirely fresh collaborative endeavour merging across a multiplicity of disciplines, sectors and global regions,” she says. “It is animated by new technologies and tools – high-performance computing is an example – applied in new ways, with input from big data, amplified by new platforms such as the semantic web, probed by advanced analytics, and guided by societal concerns and ethics.”

We can look forward to many benefits from tackling big data in this way, in fields from climate change to genomic medicine. It will also be important to understand the effects of pervasive computing and communications technology itself on human behaviour.

How is the age of big data and pervasive information affecting us as people? Are we becoming more or less moral? If we can look anything up immediately on a mobile device, what will happen to our memories and our ability to learn? How will cognition respond to frequent and lengthy immersion in virtual reality? No one knows – and, as Jackson says, only by engaging the arts, humanities and social sciences are we likely to find out.

Self-moving gel that ‘talks to itself’

Scientists have created a gel which can move independently by communicating with itself, mimicking organisms in nature, writes Pippa Stephens. It marks the first time a man-made material has been shown to regroup of its own accord when separated, and to respond to light. Communication of this sort is displayed by species such as termites and single-celled organisms such as amoeba.

The Belousov-Zhabotinsky gel reacts to light by changing colour

A team led by Anna Balazs at the University of Pittsburgh wanted to create a self-healing gel capable of mimicking communication naturally occurring in the wild, for the far-ranging uses such a material could have.

Applications include biomimetic materials that respond to light, such as artificial pupils for the human eye, and the next generation of soft, interactive micro-robots, to simulate human tissue. A self-healing material could also have military uses, in body armour which strengthens when impacted, or responds mechanically to chemical changes in the environment, for example in biological warfare.

Balazs designed a gel using a computer model and then observed it in the laboratory to see how it reacted in a chemical solution and to changes in light. Groups of millimetre-sized pieces of the Belousov-Zhabotinsky (BZ) gel were observed as they went through two chemical reactions, oxidation and reduction, in succession. Pieces of the gel reformed into the original, uncut sample when separated, and they moved in one direction together, down a chemical gradient – a movement which would continue for several days if left unchecked.

The gel also moved away from light, allowing scientists to manipulate it through illumination. “In a Petri dish the gel is quite pretty, it oscillates and changes colour from orange to green, rhythmically and periodically,” says Balazs.

She said the gel was unique in its ability to oscillate by itself and likened it to a Lego set that could snap and unsnap its parts. “It is a really beautiful biomimetic material. Ultimately we would like to build larger systems and mimic the schooling behaviour of fish,” Balazs added.

The study was published in the Proceedings of the National Academy of Sciences and supported by the US Army Research Office. Balazs said future research would also develop the gel’s polarity, to design materials which could be right- or left-handed in different conditions. A substance in the gel, a metal catalyst, makes it move by itself. Oxidation makes the gel grow, through the catalyst, while chemical reduction makes it shrink.

Slime mold

The gel mimics the slime mould in nature, which can regroup to form multicellular units. Slime moulds have also been used for biomimicry, a field of research where inspiration for human innovation is sought from nature. In a Japanese experiment in 2010, a species of slime mould which also avoids light was shown to mimic Tokyo’s rail system.

It was grown in a 20cm wide plastic dish with light-simulated mountains and water, and oat flakes around the edge marking Tokyo’s boundary. The organism branched out towards the food and over the obstacles to form networks of connections, mirroring those in the city.

Copyright The Financial Times Limited 2016. All rights reserved. You may share using our article tools. Please don't cut articles from and redistribute by email or post to the web.

More on this topic

Suggestions below based on FT Magazine

Five of the best: Restaurants in Aarhus

From traditional open-faced sandwiches to scallops with pickled beach herbs, James Clasper finds Denmark’s second city a tasty and affordable alternative to Copenhagen

Why sherry needs more love

‘Poor little sherry was relatively insignificant compared with the lucrative, globally famous spirits brands’

The sharing dogonomy

‘Dog-sitting is not new but the idea of a technology-driven, part-time pet economy seems intriguing’