Last week, I was asked to give a speech on the big picture for markets. This had me worried.

Like everyone else, I do not know where the markets are going next. A truly “big” picture of the markets, with so many people involved in them across the globe would be unmanageable – at least in a few PowerPoint slides.

Most important, I am deeply suspicious of people who, to borrow a phrase from Philip Coggan, my predecessor writing this column, think they have “the answer”. Few days go by without an e-mail from someone who is convinced there is a single key to predicting the markets, and that they alone possess it.

So, to be clear: I do not have “the answer”. But I found the exercise useful. Here is an attempt to capture the big picture of markets.

First, is there an answer? Prevailing wisdom for decades has been that markets are efficient, incorporating all available information and that prices of securities will exhibit a “random walk” in response to new items of news.

Everyone knows that markets can be inefficient in the short term, and efficient markets theorists found anomalies – such as the outperformance of small stocks, and of “value” stocks that look cheap.

But the fact that these anomalies can be exploited suggests the market tends toward some kind of equilibrium – and in the very long term stock markets do move in line with the economy. Hence the comment by Burton Malkiel, writer of A Random Walk Down Wall Street, that “short run changes in stock prices cannot be predicted. Investment advisory services, earnings predictions and complicated chart patterns are useless”. Beating the market is solely about luck.

An intellectual reaction against this has been brewing for years, as economists apply findings about psychology to markets. The credit crisis may have been a tipping point. Rather than tweaking the efficient markets hypothesis, now people want to replace it.

The investor George Soros has a theory, based on philosophy, that markets are reflexive. Actors’ opinions about the market have an impact on the market, creating “a two-way reflexive connection between perception and reality that can give rise to initially self-reinforcing but eventually self-defeating boom-bust processes, or bubbles”.

Soros suggests markets are never in equilibrium, and that there is money to be made by following trends until their extinction.

That raises another idea, applying evolutionary biology to markets. Trends work themselves out, creating long periods when the markets appear to be in balance, followed by periods of confusion or regime change as a new “fittest” paradigm emerges.

These are good reasons to expect markets to move in cycles. And critically, history confirms the insights of theory.

Earnings and profit margins tend to move in cycles and revert to the mean. They entered last year’s turbulence at historically high levels, implying they were ready for a fall.

Multiples of earnings also show a strong tendency to revert to a norm, providing we follow Benjamin Graham, the father of value investing, by looking at cyclical multiples – a stock’s price compared to its average earnings over a 10-year period. On this basis, multiples were ridiculously high at the peak of the technology bubble in 2000, but failed to get back to the norm before rising again. Stocks still look historically expensive compared to earnings that look likely to fall.

Then we must take commodity prices into account. A theory that goes back to Nikolai Kondratieff, a Marxist revolutionary killed by Stalin, suggests that commodities move in decades-long cycles. Subsequent history suggests it has an element of truth – materials prices tend to roar forward for a decade, and then be stable for a decade or so. Stocks gain when commodities are falling, but are flat to down when they are rising – as is happening at present.

Put all of this together, and there is reason to think that the market is ready for a “regime change” and that stocks are unlikely to rise.

Now, crucially, add some common sense. The US has been on an epic borrowing binge that never looked sustainable. The fall in US house prices is without precedent, at least at a national level. And central bank surveys confirm what might be predicted – that banks are tightening the supply of credit, which will restrict the economy and stock prices.

So the big picture is that we should expect much more pain before stocks resume a forward march.

The winners – there will be some – may be those who in Darwinian fashion adapt best and survive.

But even the new theories suggest we are in a period of uncertainty – not part of a predetermined pattern.

And we should not jettison the Random Walk theory altogether. All of this big picture might just already be reflected in prices (although that looks unlikely given the rally of the past two months). And the winners in this cycle may, after all, just be lucky.