In one of the more cheerful financial stories of recent times, deficits at pension schemes in the UK and Europe generally, are falling. Thanks to higher bond yields that have the effect of reducing liabilities and strong equities markets, the classic investment strategy of most UK pension funds – 60 per cent weighting in equities – they are once again in surplus.

The numbers have been headlines in pensions journals for months, with most of the data drawn from the accounts of quoted companies, and have been cited as a vindication of the strategy of investment in equities.

But back in 2003, the picture was bleak. A report from Morgan Stanley in January of that year pegged the aggregate pension deficit of FTSE 100 companies at £85bn. The year before, according to data from WM Company, British pension schemes lost about £100bn of their investments owing to falling markets.

The data, along with the collapse of several large companies that left underfunded pension schemes behind, prompted a fierce debate not only about how well funded schemes should be but also how their assets should be invested.

The debate gained urgency from new accounting rules just a few years earlier which, for the first time, forced companies to make reasonably transparent data showing the value of both assets and liabilities using a standard format. The accounting rule, known as FRS17 – the equivalent in international accounting standards of IAS19 – for the first time required liabilities to be valued based on actual market values for high-grade corporate bonds. No longer could companies discount liabilities by assuming that they would earn high returns on their equities portfolios, a practice that shrunk liabilities on paper but which proved meaningless when a plan’s sponsors became insolvent.

The ensuing furore sparked a fierce debate about how pension schemes should be regulated as well as how they should be invested, a debate that continues to rage today.

Charles Cowling, managing director at Pension Capital Strategies, a unit of insurers Jardine Lloyd Thompson, notes that solvent companies are responsible for the entirety of their pension promises. That, he says, makes a shortfall the equivalent of company debt.

“A holding in a pension scheme is no different than a holding on a company balance sheet,” he said. Therefore, when schemes invest in equities, hedge funds, private equities and property, they are exposing shareholders to all of the returns – but equally all of the risks – that those investments can earn.

“Would shareholders want the company to raise capital to invest in such a complex group of investments?” Mr Cowling asked.

Indeed, corporate finance directors in particular, have grasped the argument that over time, equities outperform bonds and that the outperformance can be used to reduce corporate contributions to pension schemes.

But even as equities markets began to reverse their losses from the trough early in 2003, another home truth emerged. Monetary authorities began cutting interest rates to help stimulate the economy as growth slowed. So while equities were once again growing in value, liabilities, which rise when interest rates fall, were increasing even faster.

Thus, despite rising equities markets, deficits continued to grow.

Finance directors and many pensions advisers argue that the accounting rules contain perverse

incentives that discourage investment in equities where returns are highest and encourage it in index-linked gilts where returns are lowest. After all, with many pension payments linked in some way to inflation, the latter provide the best bet for aligning assets and liabilities.

According to a recent research report from Merrill Lynch, deficits at Europe’s top 300 companies were €270bn in September 2003, but fell to €227 by the following year. However, by September 2005, as yields fell, that defecit rose again to €288bn. And a year later, it rose even further to €332bn.

Ironically, a portion of the year-on-year volatility came not from fluctuations in monetary policy but from an uncomfortable political compromise in FRS17 that sought to shield employers from having to discount liabilities by a bond yield that would give rise to the highest calculation of liabilities. Instead of long-term gilts, finance directors can use a AA-bond yield.

As liquidity surged through the bond markets in 2005 and 2006, the spread between yields on AA corporate bonds and underlying gilts fell, making it appear as though deficits were rising even more than they actually were.

Through most of this year, pension investment advisers have been pointing to the large number of schemes in surplus, touting the benefits of investment in equities.

But in recent weeks, as equities have plummeted and bond yields have fallen, those gains have faced the threat of steady erosion. The flight to quality as the credit crunch deepens has sent bond yields to their lowest levels in years and the FTSE 100 heads back towards 6,000.

“Shareholders aren’t paying companies to invest in equities,” Mr Cowling says of pension schemes, “they can do that themselves.”