The speed of light really matters for high-frequency trading – computer-driven buying and selling of shares, currencies and other financial instruments, where millions of dollars can be made or lost within milliseconds.
To gain a competitive advantage, trading companies are sparking a revival of microwave communication, which transmits data as radio pulses through the air. This is significantly faster than fibre optics, which sends laser light pulses down glass strands. Electromagnetic radiation (light, for instance, or radio) travels almost as fast through air as through empty space (about 300,000km per second) but glass slows it down by 30 to 40 per cent.
Over the past two years there has been a burst of building new microwave links between the two great American financial centres – Chicago, where futures contracts are traded, and New York, home of the main stock markets. But developments have been cloaked in secrecy, with new networks given obscure names such as Thought Transmissions, Newgig Networks and Zen Networks, to hide the identity of their operators.
Now two physicists from the University of California Santa Cruz, Gregory Laughlin and Anthony Aguirre, and Stanford University law professor Joseph Grundfest have shone some light on what is going on. The three academics analysed trading data from two key exchanges, CME in Chicago and Nasdaq in New York, and combined this with information from the Federal Communications Commission which regulates new microwave links. (The paper is not yet published but is available on the ArXiv physics server: http://bit.ly/ZEaKSi.)
“This industry is so secretive,” says Aguirre, who had previously been more interested in microwaves in astronomy and cosmology than finance. “No one wants to risk giving any information away to the competition.”
Frederi Viens, director of the computational finance programme at Purdue University, comments: “This is the first time I’ve seen a research group using publicly available data in this way to work out what the financial industry is doing.” He adds: “The information was in plain sight but no one else was looking for it.”
The key term in the analysis is “latency”, the delay in processing a transaction as the signal travels between dealers’ computers, which are located as close as possible to the exchanges’ data centres in the Illinois and New Jersey suburbs of Chicago and New York respectively. In early 2010, the fastest communication was through fibre optic lines which allowed equity prices in New York to respond within about 7.5 milliseconds (thousandths of a second) of a price change in Chicago. A better fibre optic link cut this to 6.65ms in August 2010.
The rush to install microwave networks started in early 2011, when the technology – driven mainly by the requirements of 4G mobile phone networks – had advanced to the point where financial institutions judged it fast and reliable enough to transmit high-frequency data. Until then fibre optics were preferred because of their greater bandwidth and immunity to bad weather, which can disturb microwave transmissions.
Over the past two years at least 15 custom microwave networks have been built between the two centres, at an estimated total cost of about $250m. Because microwaves travel in straight lines and the Earth is curved, a link spanning the 1,200km between Chicago and New York requires about 20 “hops” between relay stations that receive and instantly retransmit signals.
These new line-of-sight microwave links have cut latency by a further 2ms, according to the three academics. Current latency times lie in the 4.2-5.2ms range.
Surprisingly, however, the analysis finds that “most of these links are far from optimally designed, taking far more hops and less direct routes than necessary, given available radio hardware and existing towers; this suboptimal design adds both latency and cost”. Optimal design could shave latency down to 4.03ms, it says.
Aguirre believes that the current suboptimal performance stems from the fact that the new links were planned and implemented hurriedly by telecoms engineers whose normal design criteria put more emphasis on maximising resilience rather than minimising latency.
The academics expect improvements over the next few years to bring latency down to around 4.03ms, which they believe is the shortest achievable time. Light takes 3.93ms to travel between Chicago and New York but an extra 0.1ms delay is inevitable in practice, taking account of “last mile” electronic connections between computers.
However Shawn Melamed, global head of commercial products for Strike Technologies, a private company that has built a microwave connection between CME and Nasdaq, insists that its 4.25ms latency is as good as anyone can get today, given the constraints of technology and available microwave towers. “4.03ms looks good in a scientific paper but you cannot achieve it reliably,” he says.
The shorter transmission times since 2010 are reflected in the CME and Nasdaq trading data – shedding light on the interaction between the markets. The analysis shows that changes in Chicago futures prices drive cash prices in New York equity markets, with a delay of a few milliseconds, rather than vice versa. The most important single driver is the so-called E-mini S&P 500 futures contract traded on CME.
Amazingly, some prices in New York seem to respond to changes in Chicago in less than 3.93ms – which is physically impossible since information cannot travel faster than light. The likely explanation is the emergence of ultra-short-term “predictive algorithms”, which anticipate Chicago futures price changes before the information can reach the equity market computers.
“There is a paradigm in quantitative finance that you cannot predict what a market is going to do next but this analysis suggests some ‘stickiness’ that permits statistical anticipation of future movements,” Viens says.
Chicago-New York is the most important financial corridor in the world for high-frequency trading, because of the historical accident that US futures trading evolved in the Midwest while equities remained on the east coast. But faster links are also being built between other financial centres such as London and Frankfurt, London and New York, Singapore and Tokyo.
Fibre optics are likely to remain the best option for the foreseeable future for fast data transmission across oceans, where line-of-sight microwave links are not practical. Alternatives have been suggested, such as floating transmitters, or bouncing radio waves off the stratosphere, but they are not viable with current technology.
Another prospect is to make optical fibres transmit light far faster by manufacturing them with a hollow core, filled with air, down which laser pulses travel. Hollow fibres work well in laboratory settings but a lot more development will be needed before they are ready to be bundled into a transatlantic cable. More futuristic possibilities, still far from serious research, include sending exotic particles such as neutrinos or even gravity waves directly through the Earth.
Meanwhile the net profits from high-frequency trading have been falling steadily, from an estimated peak of $5bn in 2009 to around $1.25bn in 2012, says Viens. “Some might say that these investments [in ultra-fast communications links] are more of a last-ditch effort to continue high-frequency trading, a kind of swansong. But in reality, even if profits end up at zero, these activities will continue because they provide some benefits to the market (liquidity and efficiency), as long as they do not create too much instability.”
There is talk of imposing delays or “speed limits” on high-frequency trading, at least in the foreign exchange market. And new rules from the US Securities and Exchange Commission and other regulators will dampen profits further. But Viens adds: “Keep in mind that these profits do not include executive and employee compensation, so as long as the activities can stay mostly in the black, they will continue.”