Birth of a cloud that will never forget

A CCTV camera in London

A spate of new digital gadgets and the fulfilment of the internet’s promise as an interactive medium have dominated popular awareness of information technology in the past 10 years.

But what could turn out to be a far more important and lasting transformation has been going on below the surface. It involves a step-change in computing that promises to bring fundamental and irreversible change to many aspects of everyday life – for good or ill.

“These phenomena only come along every 15 years or so,” says Craig Mundie, chief research and strategy officer at Microsoft, of the forces that have been at work behind the scenes. The result, he adds, will be “a new class of [applications] that has never been done before”, making the next cycle of innovation as powerful as the one that accompanied the creation of the PC and the birth of the internet.

On the surface, the first decade of the millennium belonged to the gadget-makers. With more than 225m iPods, 75m BlackBerrys and 55m Wiis sold to date, wielding the latest fashionable device has never been more important.

These gadgets are only the tip of the iceberg. More than 1bn PCs, the workhorse of the information age, are estimated to be in use, compared with fewer than 500m a decade ago. And a growing proportion of the world’s 3bn mobile handsets are also now capable of accessing the internet.

This new mass market in internet-connected computing has been accompanied by a transformation in online behaviour. In short: people have learnt how to talk back.

The internet was little more than a passive distribution system in 1999 when the authors of the Cluetrain Manifesto, a radical tract predicting that the new medium would hand power to the individual, declared: “A powerful global conversation has begun.” The word “blog” was invented the same year.

A decade on, mass participation has fuelled a new generation of dominant websites. Facebook, YouTube, Wikipedia, MySpace and Twitter, all created since 2000, number among the world’s dozen or so busiest sites, according to Alexa, a service that measures online traffic. No company launching a product or politician seeking election would now contemplate acting without trying to harness this new force.

But the transformation of the web from a one-way pipe into a true interactive medium is only part of the story of how the computing world has changed in the past decade, and it may well not be the most significant. As Mr Mundie puts it: the internet itself, connected to billions of “intelligent” devices, has been evolving into a powerful computing platform. Like the mainframe computer and the PC before it, this new platform will support life-changing applications that were not possible before.

This behind-the-scenes computing infrastructure amounts to a new intelligent layer. It is capable of mediating the world as never before, collecting, storing and analysing massive amounts of data.

That information can be used to create complex models – whether of large social systems or the behaviour of an individual – and anticipate and help shape what will happen next. The new applications it makes possible can be accessed over the millions of intelligent devices that have started to proliferate.

The tech industry, always quick to find a new buzz word, has come up with a name for this sea-change: cloud computing. How this will affect everyday life is only just starting to be felt.

Take something as simple as driving to work. The falling cost of sensors has made it economic for the first time to collect detailed data about a city’s road conditions in real time, says Robert Morris, a senior researcher at IBM.

That information can be used to construct a model of how the traffic system is functioning – and, by applying historical data and making intelligent guesses, predict how conditions will change in coming hours as hundreds of thousands of drivers react to the conditions around them.

Those predictions could then be used to direct traffic in ways that minimise likely congestion, Mr Morris adds – an idea IBM has tested in Singapore, though it is not yet in full use.

What goes on behind the scenes is only part of the story. With many cars now equipped with GPS devices – and the latest smartphones such as the iPhone capable of running internet-powered applications – such traffic information can be transmitted and consumed by far more people.

Potential applications like this have been made possible by the most powerful force of the information age: the seemingly inexorable decline in the price of the building blocks of digital computing and communications. The cost of storing, processing and transmitting data has plummeted from 10 years ago. It has reached a point where much of computer scientists’ work is no longer spent trying to find ways to use computing resources more efficiently, but is focused instead on new ways to put those resources to use, says Mr Morris.

The falling cost of data storage is a case in point. The pace of the decline has slowed a lot from the late 1990s, but the law of compound numbers has still worked its magic. The cost-per-gigabyte of external consumer storage devices, at about 10 cents, is roughly 60 times cheaper than a decade ago. A one-terabyte (or 1,000 gigabyte) storage drive, which would have cost $1m at the start of the 1990s, can now be had for $100.

It is now easier and cheaper for companies and governments to preserve digital data forever, rather than waste time deciding what to delete, says Ken Steinhardt, a chief technology officer at EMC, the biggest storage technology company.

Almost at a stroke, the information age has created a new promise: there is no longer any need to forget anything.

The consequences of this are likely to be profound. Detailed medical images, once expensive to store, can be kept indefinitely and used to improve future diagnostics, says Mr Morris. At a more mundane level, personal photographs, e-mail files and all the other digital detritus of everyday life can be kept indefinitely.

But this new permanent social memory also raises troubling questions, not least the implications for personal privacy.

The collapsing cost of storage is only part of what has been happening behind the scenes. New types of computer processor, which rely on many separate cores, or “brains”, are also ushering in an era of plentiful and cheap computing, says Mr Mundie.

“All of this has been under way and building in intensity for the past seven or eight years,” he says. “As is frequently the case, the techies work on this stuff below the surface for a long time and then, boom, it arrives and something changes.”

As with earlier transformations, just what that something will be, and how its impact will be felt, are hard to predict. Whatever comes next, it’s a pretty safe bet that the iPods and early social networks of this decade will seem antiquated in comparison.