Insight: When the wisdom of crowds becomes the madness of mobs

The financial crisis has challenged virtually every tenet of modern portfolio management.

Investors were told to diversify by holding broad-based long-only portfolios of index funds; those who did so by investing in small-cap, large-cap, value, growth, international, emerging-market, balanced and target-date funds all lost money last year.

Investors were told to manage their risk exposures by adjusting their strategic asset allocation between passive stock and bond index funds; last year was a heart-pounding roller-coaster ride regardless of asset allocation.

Investors were told that they would be rewarded for bearing risk; last year risk-taking was routinely punished and the best performers were US government bond funds.

And investors were told to hold stocks for the long run; just how long is the long run, and can we be sure that the short run won’t wipe us out first?

These investment principles are not wrong – they are simply incomplete. The world has become more complex over the past 20 years, and we need to update our investment paradigm to incorporate these new complexities.

Diversification is still a good idea, but it has become much harder to achieve. Thanks to the increasing competition for additional yield, every type of investment vehicle and strategy has experienced substantial growth in assets under management.

For example, 20 years ago the “carry trade” – a long/short currency strategy in which high interest-rate currencies are purchased and low-interest currencies are sold – was considered exotic. Today, many large pension plans invest in this strategy, and a carry-trade exchange traded fund for retail investors is now available.

The influx of significant assets into any given strategy has three effects: (1) it reduces the strategy’s expected return owing to competition; (2) if there is any expected return remaining, this portion will be a source of correlation or “beta” among all investors in the strategy; and (3) if enough assets are invested in this strategy over a sufficiently short period of time, the strategy can become a “crowded trade” that is illiquid and subject to large, unpredictable swings in profits and losses.

The most striking example of these effects is the subprime mortgage market which was the focus of the current crisis, but other recent examples include the equity market-neutral “quant meltdown” of August 2007 and the unwinding of the carry trade in July 2007.

To achieve true diversification, investors must now have a broader set of asset classes and risk exposures, long and short, in their portfolios.

Strategic asset allocation decisions are no longer responsive enough to dynamic changes in risk as competition and financial innovations cause stock and bond volatilities to spike unpredictably. The VIX index (a forward- looking measure of S&P 500 volatility) rocketed from a low of 16 per cent in May 2008 to a high of
80 per cent in October 2008, and is currently hovering around 25 per cent.

Such volatility swings are simply unacceptable for most investors. In this environment, managing risk can no longer be easily accomplished via simple buy-and-hold portfolios as before, but requires more frequent rebalancing or “tactical risk management”.

This trend is a manifestation of the technological arms race that has given us electronic trading platforms, algorithmic trading, globally integrated markets, narrower bid/offer spreads, higher trading volume and more liquidity, but at the cost of more complex price dynamics.

In fact, technology has changed the very definition of passive investing, which used to be synonymous with market capitalisation- weighted portfolios of a fixed set of securities. Thanks to advances in trading technology, index portfolios no longer need to be static or cap-weighted, and may involve more frequent rebalancings while still being investable, transparent and passive.

The financial system is not an immutable black box; it is a dynamic, evolving ecology in which individuals and institutions compete fiercely for scarce profit opportunities.

In this new investment paradigm, markets are neither always efficient nor always irrational, but are adaptive. During normal times, prices can be trusted to reflect the “wisdom of crowds”. During times of distress, investors react instinctively and emotionally – the wisdom of crowds becomes the “madness of mobs”.

We can protect ourselves from the vicissitudes of these regime shifts only by acknowledging their existence and properly preparing for them in advance. This will require more sophisticated financial technologies that are equally adaptive, and capable of capturing the complexities created by our own innovations.

Fortunately, most of these technologies are within our reach, and the rest we will soon develop – in financial markets, as in nature, necessity is the mother of all invention.

Andrew W. Lo is the Harris & Harris Group Professor at the MIT Sloan School of Management and Chief Investment Strategist of AlphaSimplex Group, LLC.