Here are some notional news flashes from the future:

May 2008. Google launches G-Life, a substitute for the fallible human memory. By searching your e-mail, instant messages and telephone calls, and with the help of voice recorders set up around the home, you can now recall everything you said or wrote.

November 2008. Yahoo!'s new MobileBuddy, a voice-activated search engine, gives you real answers wherever you are. No more long lists of websites to pick through: just ask it what you want to know. MobileBuddy will also vibrate if the groceries you are about to buy are available more cheaply elsewhere.

October 2009. Regulators uncover e-mails that hint at the scope of Microsoft's search engine ambitions. According to its critics, by building its search engine into Windows, Office and other software, Microsoft is on the way to controlling access to the world wide web.

January 2010. 10 years after America Online bought Time Warner, Google acquires Walt Disney. The mania for internet distribution again has the upper hand over entertainment "content".

Fanciful? Perhaps. But the search-engine business is at the beginning of a wave of innovation that could change many aspects of everyday life and reshape parts of the information industry. Google has demonstrated the power of search, Microsoft and Yahoo! are in hot pursuit and a crowd of other search companies are seeking a gap.

One way they are looking to gain an edge is by bringing more of the world's information within reach of the software "crawlers" that index data so it can be searched. The web represents only a small fraction of what is out there: information locked up in commercial databases, celluloid archives or personal filing cabinets remains beyond reach.

"Five years from now, people will think about searching everything," says Craig Silverstein, the first employee to be hired by Google's founders and now technology director.

A spate of real headlines from recent months shows how fast this work is progressing. All four leading search engines - from Google, Yahoo!, Microsoft and AskJeeves - have come up with better ways to search the hard drives of personal computers. Google is to digitise tens of millions of books from five of the biggest academic libraries. Last week, Yahoo began a search service to find video files on the web, while Google started to offer transcripts of television news.

This, though, is just scratching the surface. "In terms of information consumption, television is probably the largest information channel for consumers," says Mike Lynch, chief executive of Autonomy, a UK search technology company.

The technology to sift through and make sense of television or other video content is advancing (see right).But most of the content remains in unsearchable analogue form or under lock and key. Little work has gone into converting archives of old TV shows into digital form.

"This is not a technical challenge - it's really a question of the business model," says Karen Howe, head of Singing Fish, a search engine owned by America Online that looks for audio and video files. TV networks, like film studios, have yet to find a way to make money from their content over the internet: until they do, there is little incentive to do the laborious work of turning old celluloid into digital files. Many other commercial information companies, from newspaper publishers to database vendors, feel the same.

Even without these commercial sources of information, there is ample data from everyday life that could be digitised and made searchable. Personal telephone calls would be a treasure trove of useful information, says Mr Lynch. If making voice calls over the internet becomes common - turning conversations into streams of data like any other - building a searchable index of your phone calls may not be that far away.

Voice calls are only part of the conversation that occupies people's daily lives: much of the written communication, including e-mail and instant messages, is already in digital form. What if all this could be captured and searched - along with the personal photographs, online web logs and video journals that are becoming an increasing part of life? "There's this whole branch of ephemeral information that needn't be ephemeral," says Mr Silverstein at Google.

A second front in the search wars is being fought over ways to make it easier for people to filter and extract information from the simple list of web pages that most search engines produce.

"It's clear that a list of links, though very useful, doesn't match the way people give information to each other," says Mr Silverstein. The question that he says Google - like others - is now trying to address is: "How can the computer become more like your friend when answering your questions?"

That means giving direct answers to questions, extracting data from online sources rather than giving links to web pages. It also means doing a better job of divining what the searcher is looking for, tailoring results more closely to what, based on past experience, appear to be the user's particular interests.

Most of the new search engines appearing on the web do not claim to come up with better results to a query - in fact, many of them take their results straight from Google or other established engines. Instead, they aim to organise and present information in more useful ways.

Users will want more direct responses to their search queries, the experts acknowledge. "The biggest change we will see in the next five years will be in the way people use computers," says Mr Silverstein. Mobile handsets will become the most common way to find information on the internet, he adds. At that point, most queries would best be made and answered by voice.

If the search companies become a more integral part of everyday life, how far will their influence eventually extend - and what impact will they have on other companies that exist to create or distribute information?

The answer depends in part on how the balance of power evolves between the companies that create entertainment and media "content" and those that distribute it on the internet. This is not a new issue: it was the perceived power of the early internet distributors that led Time Warner to sell out to America Online at the height of the dotcom bubble.

Search engines bring a new twist. As omniscient distributors of information, they threaten to be both a content- creator's best friend and worst nightmare. Now, anyone can find your content, making everyone a potential customer - but your best customers can also just as easily find your competitors. For suppliers of information that can easily be turned into a commodity - weather forecasts, stock quotes, dictionary definitions, telephone numbers, maps - this is a serious threat.

As search engines get better at presenting this sort of information to users, it could take on a context that bears little relation to the one in which it was originally published. "Ten years from now, it will be much more difficult to distinguish searching for something from seeing and using it," says Charles Ferguson, a technology author.

Media and entertainment companies with unique content may feel they can wall themselves off from this commoditising trend. That would depend, however, on how much free information will eventually fall within the search engines' reach and whether it can start to rival the quality of proprietary information.

Already, internet blogs and communal internet pages known as wikis (from the Hawaiian for "speedy") are pushing the boundaries of what was known as "user-generated content". Results from Wikipedia, a free encyclopaedia maintained over the internet by volunteers, may not match the standards of publications produced by professional editors, but the service still manages to answer many common questions.

Even if this proves no ultimate threat to proprietary information, it is a reminder that publishers urgently need to find workable business models for delivering their content online.

Search engines themselves are likely to have their own long-term influence determined in part by the extent to which they can insinuate themselves into the wider fabric of internet activity. Microsoft, for instance, already talks of its search engine as a "platform" - a piece of software on top of which others will be able to build their own technology, much as other software developers have ridden on the Windows operating system.

Publishing the "application programming interfaces" of its search engine - the software links that other developers use to connect their own software - is an important part of Microsoft's strategy, says Adam Sohn, director of sales and marketing at its MSN offshoot. "We look for problems that are broad and horizontal," he says. Search is "a capability developers don't want to build every time they write an application", so why not encourage them to build on to the Microsoft technology?

This is a page straight out of Microsoft's usual playbook, says Mr Ferguson, who has long written about how adominant "platform" seems to emerge during each successive phase of the computer age - from the IBM mainframe to Windows. Will the search market mirror these technologies, becoming a winner-takes-all business?

Apparently aware of the threat, Google also says it is considering opening its application programming interface to let other developers draw on its technology. Last week, fulfilling a promise made last year, it gave advertisers access to the interfaces for its search-related advertising service, giving them more power to influence how their advertisements are displayed.

"Everyone loves to call themselves a platform," says Mr Silverstein. But he adds that, while search may share some of the characteristics of other computing platforms, the amount of work still to be done to make the universe of information searchable will leave the field wide open for a long time to come.

Once this early rush of innovation has passed, will search engines still attract as much as attention as they do now? The fluctuation of internet fashion suggests that they may not, says Mr Lynch. "In the early days [of the internet], search was 'it'," he says. "Then portals were 'it'. Now fashion is swinging back again."

The search engines will also have to figure out new ways to make money when search ceases to be a stand-alone activity conducted from a dedicated website but instead becomes a core feature of many other activities carried out over the internet.

The people behind the search engines, needless to say, see themselves as more than just the latest, passing internet fashion. But even they concede that the function of search engines should eventually be absorbed into the fabric of a more "intelligent" internet.

Finding information would not involve going to a separate place - a search engine - to ask a question. Instead, the answer would present itself wherever you happened to be, and in the most appropriate form. "Search will become more and more important and less and less visible," says Mr Silverstein at Google. "It will be ubiquitous and invisible."

At that stage, depending on your point of view, Google and its rivals would either be one of the most powerful forces shaping everyday life or just another invisible cog in the great Information Age machine that is being created out of the internet.

Net gain for sport

When it comes to the cutting edge of internet search technology, fans of television sport will probably see the benefits sooner than most.

Couch potatoes five years from now will be able to make their own highlights of the latest games, predicts Hong-Jiang Zhang, managing director of Microsoft’s Beijing research centre. “You will just be able to see the exciting parts,” he says.

To understand why, consider how sport differs from other video footage that appears on TV. Most information from the natural world does not follow recognisable laws of “grammar” orobey a circumscribed “vocabulary” - at least, not in ways we yet understand. The almost limitless variety in natural phenomena makes it difficult for a computer to parse the information contained in a random piece of video and “understand” what is going on, says Mr Zhang.

Sport is different: it follows rules. Turn those rules into recognisable visual signposts - a football hitting the back of the net, for instance - and events potentially become searchable.

Image recognition and movement analysis have reached a level where they can produce a searchable “index” of the events of a sports game using a standard PC, says Mr Zhang, though he adds that complete accuracy is some way off.

The visual clues are only part of the story - combine them with other sources of information and accuracy improves. This extra information, known as meta-data, can take different forms. The most basic is text, picked up from the subtitles in a film or the captioning in a TV show. The early video search services, such as one just launched by Yahoo, rely on searching text like this.

Soundtracks offer a potentially more fruitful source of information. By combining elements of speech recognition and sound analysis (identifying the noise from an explosion, for instance), it becomes possible to guess at much of what is happening in a piece of video, says Mike Lynch, head of Autonomy, a UK search technology company. “For most things humans want to search for, sound recognition works very well indeed,” he adds.

The internet has also introduced an important new layer of context. Led by Google, web search engines interpret the meaning of information based on the meta-data attached to web pages, as well as analysing the links between web pages to assess its relevance.

Meta-data promises to bring other forms of visual content within reach of the search engines. Some digital cameras already encode information on a picture, such as the time it was taken. Global positioning sensors built into camera phones could add location information. Using the voice capabilities of a camera phone, the user could also append commentary when taking a snapshot, then use keywords to search for the picture later, says Adam Sohn, marketing director of Microsoft’s MSN unit.

Ultimately, all the random, unstructured information contained on web pages and other data-repositories could be subjected to a form of structuring that made it more intelligible to machines. This is the idea behind the Semantic Web, a vision of the future internet promoted by Tim Berners-Lee, creator of the World Wide Web.

Even if this remains out of reach, much of the effort put into search will be dedicated to imprinting order on the chaos by organising information in mo re coherent ways. As Daniel Read, product management vice-president at AskJeeves, says: “We need data standards that go across the web, that are in all consumer products.”

Get alerts on The Future of Yahoo when a new story is published

Copyright The Financial Times Limited 2019. All rights reserved.

Comments have not been enabled for this article.

Follow the topics in this article