07/25/2011 - CQS Was Saturated and Delayed on May 6th,
2010
|
Back to
Main Research Page
The Consolidated
Quotation System (CQS) is the Security Information Processor (SIP) for
stocks listed on NYSE, AMEX, and
NYSE Arca (home of many
ETFs). CQS was
widely believed to have been operating normally and within capacity on May 6,
2010. We have found strong evidence that this was not the case, and in fact,
CQS was severely saturated and therefore delayed during the flash crash.
The reason CQS was perceived to be operating within capacity is because message
traffic is measured in 5 second intervals which smooths (averages) message
traffic over that interval. However, quote cancellation and update rates at
that time were already below 1 millisecond (ms). In effect, the metric
measuring capacity was over 5,000 times the speed of trading. It would be like
measuring the speed of a car by taking the distance it covered over the course
of 1 hour rather than instantaneous measurements as taken by speed radar. To
understand the absurdity of this difference, imagine the following dialog
between a Policeman and a Driver. |
Officer: I clocked you speeding over 90mph.
Driver: But, I was going 45mph for 30 minutes before that. I assure
you I'm averaging 60.
|
When we plot CQS traffic at 2 ms intervals, the story of what really happened,
becomes clear. Compare the following 2 charts. The first shows message rates at
1 second intervals, the second chart shows message rates over 2 ms intervals.
Capacity at the time of the flash crash was listed at 250,000/second and is
shown by a solid horizontal line at that level. Notice how in the second chart,
the peak hugs the maximum capacity line: traffic was saturated at those
times.
|
CQS message rates over 1 second intervals. (click for high resolution chart)
CQS message rates over 2 ms intervals. (click for high resolution chart)
The next 2 charts show CQS traffic rates on each of the 12 multicast lines. The
first shows message rates averaging over 1 second, the second chart shows
message rates over 2 ms intervals. Each multicast line at the time of the
flash crash had a capacity of 30,000/second which is shown by a solid
horizontal line at that level.
|
CQS message rates over 1 second intervals. (click for high resolution chart)
CQS message rates over 2 ms intervals. (click for high resolution chart)
The next chart shows a closeup of the
critical
14:42:44 time when CQS set a new record for message rates. The total is
divided by 10 so that it fits in the same scale. Notice how the total (black
line) practically pegs the 250,000 rate (25,000), and the multicast lines swing
wildly. This is CQS trying to mitigate traffic overflow -- by queuing data so
that it does not exceed capacity. The fact that the total rate never
significantly exceeds its capacity guidelines on a 2 ms basis, tells us that
CQS must be throttling traffic.
|
The following 2 chart shows just how severe saturation became for multicast
line #1. Notice how it oscillates wildly. The 2nd chart is a closeup of this
behavior. Notice how the rate surges at the exact beginning of each new second.
We have confirmed this behavior appears from all reporting exchanges; in other
words, it's not from inbound data feeds to CQS. It's not related to the
delay
of the NYSE feed into CQS.
|
What all these charts are showing is that CQS itself was overwhelmed and a
source of significant delays on May 6, 2010.
So what does the
SEC's
October 2010 report say about these significant CQS saturation and delays?
It doesn't appear they noticed them. There is only acknowledgment of the delay
in NYSE's system feeding into CQS which we previously discovered (and
reported on extensively). The report says nothing about CQS capacity itself
being overwhelmed. How could they have missed this?
Even people in the industry did not think there was a capacity issue with CQS.
This
post
from a well respected member on a Motley Fool Message Board sums up nicely how
others in the industry viewed CQS performance on that day: ...
SIAC (the people who manage the CTS/CQS feeds) had no capacity problems even at
the peak volume on May 6th. Trust me, these systems are designed to handle
millions of messages per second. ...
So why is this important? Because it
continues
to
happen, and is happening
again.
We think many other systems experienced saturation and delays on May 6th, and
it is only a matter of time before someone takes the time to analyze the data
and find them.
|
|
Inquiries: pr@nanex.net
Publication Date: 07/25/2011
http://www.nanex.net
| This report and all material shown on this
website is published by Nanex, LLC and may not be reproduced, disseminated, or
distributed, in part or in whole, by any means, outside of the recipient's
organization without express written authorization from Nanex. It is a
violation of federal copyright law to reproduce all or part of this publication
or its contents by any means. This material does not constitute a solicitation
for the purchase or sale of any securities or investments. The opinions
expressed herein are based on publicly available information and are considered
reliable. However, Nanex makes NO WARRANTIES OR REPRESENTATIONS OF ANY SORT
with respect to this report. Any person using this material does so solely at
their own risk and Nanex and/or its employees shall be under no liability
whatsoever in any respect thereof. |
|
|
|
