In deciding whether or not to invest in a new activity, managers obviously need to base their judgment on the potential of the project to deliver value to their company. But the value of an innovation project – such as the development of a new product or service – is an elusive thing. It is also not the same as the value one can ascribe to a marketable item such as a house or a bond. This can be estimated to quite high accuracy and, more to the point, can be tested at once in the marketplace.

In contrast, an innovation project will usually have little or no marketable value while it is under development, and its eventual value lies at the end of a chain of future events. Despite this, most organisations use apparently precise calculations such as net present value (NPV) as the basis for their decisions on which projects to select. Such calculations are critically dependent on the estimates of the future value of a project.

Due to the high levels of uncertainty involved, the selection of innovation projects requires a different approach – relying too heavily on NPV often causes organisations to be unduly risk-averse. Managers should recognise that, in valuing an innovation project, they are not estimating the value of something that exists today but making a forecast of the benefits that the new product or service will generate in the future.

Leading companies treat the valuation and selection of innovation projects not as a single decision but as an iterative forecasting process, in which the prediction of value develops and improves over time. In some ways it is like weather forecasting: the best way to predict the weather in an hour’s time is to go outside and look; the best way for a few days ahead is using computer models of the atmosphere nearby; but to forecast a decade ahead requires an entirely different approach based on the climate of the whole world. The most accurate process for the short term is quite useless in the longer run and it is well accepted that detailed long-term forecasts are impossible.

Don’t accept figures at face value

As with weather forecasting, companies must continually update their view of the value of innovation projects and use different techniques depending on the circumstances. Early on, financial data may be unreliable or simply unavailable. Would you care to estimate the sales of England world cup shirts in the summer of 2006? Or how about Russian interest rates 10 years from now?

Any calculation that uses inherently inaccurate estimates of value will be unreliable and is a case of the well-known “GIGO” effect: garbage in, garbage out. Worse still, the results of NPV and return-on-investment calculations may have legitimacy within an organisation because they are based on accounting and finance, disciplines that most people regard as exact sciences. But financial methods have significant limitations when applied to the early stages of development, particularly in the case of “new product, new market” projects, where little is known about the target market and potential sales. The accuracy of forecasts improves as projects progress but there may still be considerable uncertainty even at the time of commercialisation.

It sometimes seems that the dual challenges of managing innovation and finance cannot be reconciled. Best practice shows that the evaluation and selection of innovation projects is best tackled by: using more than just financial tools and techniques to assess the attractiveness of innovation projects; applying an iterative approach to evaluating projects as they progress through development; recognising that the management of the portfolio of innovation projects is so important to the future of a company that it is essential to nominate a senior manager to champion the process.

Appropriate tools and techniques

Financial formulas are one way of assessing innovation projects. Robert Cooper et al have shown that the companies that are more successful at innovation use a variety of techniques, in combination. Less successful organisations have a tendency to rely on financial calculations alone. The main non-financial ways to assess innovation projects are: earmarking; strategic buckets; balance diagrams; and scoring methods.

Earmarking is simply the identification of single projects that must be done for regulatory compliance, supply continuity or over-riding strategic necessity. Strategic buckets allocate a proportion of the innovation budget to projects that support a particular strategic thrust – such as a move into a new business. Balance diagrams (also called “bubble diagrams”) display projects on a grid that allows managers to see if they have a proper balance between key aspects, such as risk and reward (as we explained in our FT article “The risks of innovation” in September 2005).

The most useful tool for evaluating early-stage projects is the scoring method. This combines several factors that are thought to be indicators of future success in the particular business but avoids over-reliance on any one of them in isolation. Thus, an NPV estimate may be one of the factors but the picture will be broadened by the addition of other considerations. For a new product proposal these might include: how well the project fits the company’s overall strategy; the growth potential of the market; the level of competitor activity; the uniqueness of the product; and the general level of margins in the target market.

In principle, of course, many of these considerations may be aspects of estimating the NPV but since all are known to be fallible in the early stages, they are included separately to avoid over-dominance by any issue. Broader considerations, such as impact on brand image, may also be introduced, making the process akin to a “balanced scorecard” for each project. The appropriate factors will vary from company to company and Table 1 shows how diverse the factors chosen can be.

Each factor is allocated a score, typically between one and ten, to give an overall figure of merit for the project. Companies often apply weightings to give some factors more emphasis than others, and these weightings may change as the project progresses. If the scoring is done by a team, it is useful to provide guidance. For example, what level of sales increase should rate a score of ten? This helps to give consistency to different views. Britannia Building Society, however, achieves consistency simply by always using the same, small group of managers.

In evaluating projects that are at the concept stage, companies can learn much from the way venture capitalists (VCs) approach the problem. VCs always consider the mix of projects in their portfolio (expecting some to be higher risk but potentially higher return), break the decision to finance projects into a number of stages (with clearly defined goals for each stage), and expect market and financial estimates to become more reliable as projects progress.

Iterate, iterate, iterate

Using different tools and techniques helps give a broader picture of the attractiveness of projects. Some companies, however, have had bad experiences with scoring systems because they fail to recognise that the factors need to be re-visited at each stage of product development. In the early stages, scoring should concentrate on the potential of the idea and not too much, perhaps, on the risks. Later on, risks, costs and exploitation issues must be emphasised. As the project nears commercialisation, the financial forecasts should become more and more important – and eventually dominate entirely.

The perceived need to be “numbers driven” causes much friction between different functional areas. Research and development teams, knowing the uncertainties surrounding the new technologies, products or services they are developing, may try to resist attempts to quantify value, while their colleagues in finance regard this as evidence of woolly thinking. Of course, both sides are right in their own way. When an innovative project is near to completion, no sane company will proceed without detailed financial figures. However, when an intriguing new idea is first generated, the decision to take it further cannot and should not be based on financial calculations alone.

Recognising that different approaches to evaluation are needed over the product development cycle should help avoid the problems that one major German car manufacturer is currently encountering. The relationship between the R&D and finance departments is becoming severely strained, as R&D accuses finance of being risk-averse, while finance says R&D is not delivering value fast enough. Currently, the argument is about what discount rate should be used in the NPV formula – and the debate is missing the central point.

Championing the portfolio

It is important to use different tools and iterate but focusing too strongly on these things can lead organisations to miss a key factor: the people side of portfolio management. What do we mean by this? In managing portfolios of innovation projects, we want a good balance of projects, each of which is individually attractive and which together match strategic and other goals. However, it is often overlooked that portfolio decisions have a significant impact on an organisation’s levels of motivation. There are two main points to consider.

First, individuals who have worked on developing new product concepts will be disappointed if these are not selected. McPherson’s, the kitchen products manufacturer, makes its portfolio management decisions totally transparent. Teams present new product concepts in the style of an industrial fair, with stands on which prototypes, suggested advertising and market data are displayed. Management visit the stands, talk to the teams and then make their decisions.

This open approach contrasts strongly with the “behind closed doors” structure at many organisations. More importantly, McPherson’s has found that it encourages healthy competition between the teams preparing for the concept fair and avoids de-motivation in the teams whose concepts are not adopted.

The last people factor to consider is who will drive effective portfolio management. Do not assume that the management team itself will be objective; “pet projects” and wildly inaccurate forecasts are issues in many companies. Therefore, to make sure the right tools are matched by objective thinking requires someone to act as a “portfolio management champion”. The champion also needs to deal with risk, recognising that all innovation carries the risk of failure. This is best faced as early as possible, so managers must consciously allocate budgets for investigating new possibilities. This is “risk money” in that some, at least, will be fruitless.

Integrating innovation and finance at Agilent Technologies

One organisation that has a senior manager championing the portfolio is Agilent Technologies, which provides leading technology for communications, life sciences and chemical analysis, automated test equipment and semiconductors. Its “systems on the chip business unit” (SOC BU) has four sites with manufacturing and R&D facilities: Böblingen, Germany; Hachioji, Japan; and Loveland and Santa Rosa in the US.

At these sites, over 650 employees develop and manufacture a range of highly sophisticated chip-testing solutions. SOC Systems cost between $600,000 and $4m. Agilent’s customers include “testhouses” (high volume integrated circuit testers) and many of the major electronics manufacturers worldwide.

Financial management is given much emphasis at Agilent, and Werner Widmann, the SOC BU controller, has deliberately spread his team across the four sites. This ensures the broad co-operation necessary with all aspects of the business, from hardware and software R&D to marketing to manufacturing and supply-chain management.

Key success factors in the chip-tester business are: time-to-market, to have new testing products available to meet the waves of new technology in, say, digital home entertainment; meeting the customers’ demanding technical specifications; and achieving fast, low-cost chip testing. In addition to these factors, the business is highly volatile – within a year the quarterly sales figures can fluctuate by as much as 150 per cent. With such uncertainty, it is difficult to maintain R&D spending during downturns in the market. So, part of Mr Widmann’s responsibility is to ensure that the SOC BU makes a significant return on investment during the market upturns and does not suffer from cash flow problems during the downturns.

To deal with the challenges, Mr Widmann’s team has adopted a much wider role than many controllers. “For example, my team led a portfolio management taskforce to develop tools and processes to support top management in the SOC BU business board,” he explains.

Gauging the technical risks of a project and forecasting product sales are notoriously difficult, and a single approach seldom works well, so the team has developed a set of tools to be used in parallel. These are based on portfolio assessment questions, market uncertainty analysis and project-scoring matrices. Communications throughout the worldwide management team have been significantly improved through the adoption of this standard set of portfolio management tools. And, to promote learning, managers’ previous estimations of sales and risks are compared to actual figures and fed back to them.

“The new SOC BU portfolio framework has greatly helped in the way it presents the data and makes the trade-offs visible,” Mr Widmann explains. “The management team now has the information it requires to make fact-based decisions on which projects to back and which to kill, or postpone. We are now starting to get much better at understanding market attractiveness and risk.”

The experience at Agilent shows that innovation projects require special approaches and, ideally, a senior manager needs to be given the responsibility for driving portfolio management. Mr Widmann strongly believes that financial controllers need to take a broad view of opportunities and risks. Money may not be the source of all evil but NPV alone can be an unreliable guide and a poor master in selecting innovative projects.

Keith Goffin and Rick Mitchell work at the Cranfield School of Management’s Centre for Innovative Products and Services (CIPS). This article is partly based on their most recent book, “Innovation Management: Strategy and Implementation Using the Pentathlon Framework” (2005).