[FoRK] Nanosecond Trading Could Make Markets Go Haywire

Eugen Leitl eugen at leitl.org
Thu Feb 16 07:23:40 PST 2012


Nanosecond Trading Could Make Markets Go Haywire

By Brandon Keim February 16, 2012 | 6:30 am | Categories: Tech

Photo: © Copyright 2006, The Nasdaq Stock Market, Inc.

The afternoon of May 6, 2010 was among the strangest in economic history.
Starting at 2:42 p.m. EDT, the Dow Jones stock index fell 600 points in just
6 minutes. Its nadir represented the deepest single-day decline in that
market’s 114-year history. By 3:07 p.m., the index had rebounded. The “flash
crash,” as it came to be known, was big, unexpected and scary — and a new
study says flash events actually happen routinely, at speeds so fast they
don’t register on regular market records, with potentially troubling
consequences for market stability.

The analysis involved five years of stock market trading data gathered
between 2006 and 2011 and sorted in fine-grained, millisecond-by-millisecond
detail. Below the 950-millisecond level, where computerized trading occurs so
quickly that human traders can’t even react, no fewer than 18,520 crashes and
spikes occurred. The study’s authors call those events “financial black
swans,” though they’re so common that the black swan label probably doesn’t
fit anymore.

Moreover, those events fell into patterns that didn’t fit market patterns
seen at other time scales. It’s as if computerized trading has created a new
world, one where the usual rules don’t apply, populated by algorithms and
only dimly understood by the people who made them. The extent to which that
world influences our own — perhaps making events like the 2010 flash crash
more likely, or causing markets to be generally more volatile — is an open

“There’s this whole world below 650 milliseconds. It’s like landing on
another planet,” said Neil Johnson, a complex systems specialist at the
University of Miami and co-author of the study, released Feb. 7 on arXiv.
“It’s an enormous part of the market which is out of human reach. We have a
glimpse of the kind of ecology that’s going on down there.”

Red line represents the frequency of sub-650 millisecond flash crashes, and
blue the frequency of flash spikes, between January 2006 and February 2011.
The black spike is the S&P 500 index. Image: Johnson et al./arXiv

Until recently, trading was the preserve of humans. Imagine a stock market
and you likely envision a loud, crowded trading floor, a scene out of Wall
Street. But in 1998, after the U.S. Securities and Exchange Commission
authorized the first electronic exchanges, computer trading programs entered
markets as equals to humans.

The programs are designed to trade enormous volumes of stocks, bonds and
other financial instruments at superfast speeds, taking advantage of
second-to-second fractional price shifts and market trends. It’s now
estimated that high-frequency computer trading accounts for 70 percent of all
equity trades. While some activity does occur at speeds with which humans can
interact, much of it falls beyond the limits of human response time.

(One new computer chip built specifically for high-frequency trading can
prepare trades in .000000074 seconds; a proposed $300 million transatlantic
cable is being built just to shave 0.006 seconds off transaction times
between New York City and London.)

In the early years of computer trading, algorithms were profitable and
concerns rare. Designers and investors took their money and didn’t think much
about what Johnson and co-authors call “ultrafast machine ecology.” After the
2010 flash crash, however, mainstream economists wondered if high-frequency
trading systems might sometimes get weird and unpredictable. A $4.1 billion
automated sale was ultimately blamed for triggering that crash, and
economists started asking questions about the new, hazy relationships between
machines and markets.

“We are certainly witnessing one of the major transitions in the history of
financial markets,” said automated trading researcher John Cartlidge of the
University of Bristol, who was not involved in new study. “Economic theory
has always lagged behind economic reality, but now the speed of technological
change is widening that gap at an exponential rate.  The scary result of this
is that we now live in a world dominated by a global financial market of
which we have virtually no sound theoretical understanding.”

In the new study, researchers led by Johnson and simulation engineer Brian
Tivnan of the University of Vermont analyzed millisecond-scale price logs
from 600 markets. The numbers were gathered by Nanex, a Chicago-based company
that sells live market data.

'We now live in a world dominated by a global financial market of which we
have virtually no sound theoretical understanding.' From this analysis
emerged records of 18,520 sub-950-millisecond crashes and spikes — far more
than they, and perhaps almost anyone, expected. Equally as striking as these
events’ frequency was their arrangement: While market behavior tends to rise
and fall in patterns that repeat themselves, fractal-style, in periods of
days, weeks, months and years, “that only holds down to the time scale at
which human stop being able to respond,” said Johnson. “The fractal gets

Why this should happen isn’t exactly clear, but the researchers think it
reflects differences between human and computer trading strategies. Whereas
people have many different strategies, high-frequency programs “sacrifice
diversity for speed,” said Tivnan. “You see a lot more homogeneity at the
sub-second scale than we see above 1,500 ms.” In the researchers’ models of
high-frequency trading markets, a variety of algorithms eventually evolved
into a few stripped-down, optimized forms.

With many algorithms converging on just a few different strategies, the
high-frequency trading market could become vulnerable to systemwide herd
behaviors. Fortunately for us, the market seems to rebound from spikes almost
as immediately as they occur — Johnson and Tivnan likened the effect to a
“coiled spring” returning to form — but as seen in May 2010, this might not
always happen.

Johnson and Tivnan also used another metaphor to describe the flash crashes
and spikes: fractures. The events could be imagined as microfractures in the
wing of an aircraft, accumulating unnoticeably until some critical,
breakage-causing mass is reached. To that end, they found a correlation
between rising frequencies of sub-950-ms flash events, market volatility
after 2008, and the May 2010 flash crash. The 10 stocks most prone to
crash-and-spiking were all financial companies, with Morgan Stanley, Goldman
Sachs and Wells Fargo topping the list.

“Lay the occurrences of spikes and crashes against each other on the same
timeline, and then look at the movement of a major index like the Standard &
Poor’s 500. What’s particularly interesting is that dramatic increases of
spikes and crashes coincided with major movements in the S&P index itself,”
said Tivnan.

However, it’s uncertain whether this correlation reflects a cause-and-effect
relationship. It could conceivably be just a coincidence. “The results are
provocative, but need more statistical testing to be something you can
reliably interpret,” said complex systems theorist Doyne Farmer of the Santa
Fe Institute, who was not involved in the new study.

Blue bars represent the distribution of different high-frequency trading
strategies; red bars represent strategies that could be added at critical
points to reduce flash behaviors. Image: Johnson et al./arXiv

Uncertainties notwithstanding, the paper is still “an extremely important
contribution to solve the puzzle of financial complexity,” said
econophysicist Tobias Preis of the Swiss Federal Institute of Technology, who
studies patterns that precede market bubbles. Cartlidge called the paper
“timely and important,” and said the findings are “likely to have a
significant impact on market participants and regulators alike.”

The question of regulation is a tricky one. In the aftermath of May 2010,
federal U.S. regulators introduced so-called “circuit breakers” that
automatically halt trading if a stock price falls too much, too fast. But
whether this actually works isn’t yet known. “Currently, we’re having trouble
even observing at that level of resolution, let alone regulating it,” said

Tivnan also works for the MITRE Corporation, a nonprofit engineering and
technology consultancy that provides research support to U.S. regulatory
agencies. Both the U.S. and European Union are actively investigating further
intervention in the machine trading world.

Johnson and Tivnan propose a subtler approach than circuit-breakers, one that
would “steer” automated markets by introducing rogue algorithms when herd
behaviors appear imminent. Farmer wants markets altered to become slower,
with trades occurring intermittently — once per second or once even per
minute, rather than constantly — and speed de-emphasized. That would allow
algorithm designers “to focus on the quality of decision-making, rather than
the time it takes,” said Farmer, who preaches caution in designing new

“There’s a danger of Europeans doing some changes they haven’t thought
through, and there’s danger of the United States not changing things they
need to change,” Farmer said. “It’s hard to think these things through,
because nobody understands them.”

Citation: “Financial black swans driven by ultrafast machine ecology.” By
Neil Johnson, Guannan Zhao, Eric Hunsader, Jing Meng, Amith Ravindar, Spencer
Carran and Brian Tivnan. arXiv, 7 February 2012.

Brandon is a Wired Science reporter and freelance journalist. Based in
Brooklyn, New York and sometimes Bangor, Maine, he's fascinated with science,
culture, history and nature.

Follow @9brandon on Twitter.

Tags: Complexity, economics 

More information about the FoRK mailing list