The securities industry had been set for more than three months to carry out its annual test of how trading firms, market operators and their utilities could operate through an emergency using backup sites, backup communications and disaster recovery facilities.
The date: Oct. 27.
The Business Continuity Test coordinated by the Securities Industry and Financial Markets Association along with the Financial Information Forum would take place two days ahead of the arrival of Hurricane Sandy, the largest storm ever spawned in the Atlantic Ocean.
Read the complete story in the December edition of Traders Magazine, coming out soon
“It was very timely, said Karl Schimmeck, vice president of financial services operations at SIFMA.
Participants in the 11th annual edition of the event included brokers, market data suppliers and technology service bureaus, as well as global banks, regional banks and financial advisers, according to Schimmeck.
The practice: Allow market participants to transmit dummy orders to markets, receive simulated execution reports, and conduct settlement and payment tests. Equities, options, futures, fixed-income, foreign exchange and commercial paper market operators all were “open” for business.
Before Hurricane Sandy—or any other storm—hit.
The testing window was from 9 a.m. to 1 p.m. Eastern time Saturday. The broad industry effort only uncovered “little issues,” Schimmeck said. Nothing that would preclude markets from opening two days later, with backup systems, if they so chose.
But they still chose not to.
By Sunday night, after as many as 14 hours of “epic” phone calls involving as many as 800 market participants at a time, the nation’s stock exchanges, led by the New York Stock Exchange, chose to shut down. Before Sandy reached landfall.
NYSE chief executive Duncan Niederauer, in a post-storm interview on the floor of the exchange, said it would have been “irresponsible” to have opened for trading because its members and other participants would have “a lot of people in harm’s way,” even if markets operated only electronically.
The storm caused $6 billion in business losses and more than $20 billion in physical damage, according to IHS Global Insight, a Boston analytics firm. Morgan Stanley and other Wall Street firms’ offices near the Hudson waterfront got shut down for weeks.
And, of course, all national exchanges chose to shut down for two days, an unprecedented move in an era when trading can take place entirely electronically. Particularly so, since the markets themselves are all housed in highly secure facilities, across the Hudson River in New Jersey.
The choice led to widespread back-channel criticism. Brokers and other exchange operators complained that the NYSE did not hold proper advance testing of its backup plan, which was to route all orders through Arca, its all-electronic exchange. Many of the NYSE’s own members, one broker and one operator told Traders Magazine, would have had to scramble Sunday evening to check their connections with Arca and make sure all ancillary services, such as market data feeds, were in place and operating properly.
NYSE Euronext’s $250 million data center in Mahwah, N.J., which opened in 2010, is built to withstand hurricanes and can operate on backup power for a week. The exchange operator maintains a backup facility in the Midwest, as well.
The “scramble” on Sunday evening also could easily have been avoided, if NYSE members had gotten prepared before the arrival of Sandy was imminent.
NYSE’s emergency backup plan, to route all orders for NYSE-listed stocks to NYSE Arca, was approved in December 2009, and a formal “information memo” with details was published by NYSE Euronext on March 14, 2010.
That memo advised NYSE members that did not have connections with Arca to set up some sort of connection, for business continuity purposes.
Industry testing on the plan was conducted as recently as March. But a large number of NYSE brokers did not participate.
The Business Continuity Test carried out by SIFMA and FIF the weekend before Sandy hit gave one last chance for testing. The dummy operations tested systems put in place by brokers. Brokers could test trading in equities and options on the NYSE Arca platform, if they chose to do so.
But many brokers were not ready for the switchover, according to many market participants, which argued for closing the New York Stock Exchange on Monday and Tuesday, when the storm arrived.
“People have to be responsible for their own systems,” said Jim Toes, president of the Security Traders Association, which represents 4,200 individuals involved in trading equities and equity options.
The NYSE dealt with criticism that it did not stage adequate testing of its backup plan immediately before Sandy hit.
“The forecast at the end of last week for the storm was certainly very ominous, but had a lot of uncertainty around it,” Joseph Mecane, co-head of U.S. listings and cash execution for NYSE Euronext, said during an online discussion of the industry’s response to Sandy. “I think the exchanges all agreed that the right way to proceed was to check in periodically throughout the weekend and assess things as they evolved.”
And if brokers or NYSE members with the same information about a weather-related emergency don’t get prepared by updating their connections to a known backup plan long in advance, it’s not necessarily an exchange’s fault if its members are not ready, at the last moment.
“Whose responsibility is that?” asked Toes.
In fact, there may be some trading firms that do not want to invest in backup systems, said Alex Tabb, a partner at industry consultancy Tabb Group.
So setting up shop in a backup site, such as NYSE Arca, can be as expensive as setting up in a primary site. And each backup site required adds cost.
Which is a cost that a high-frequency firm, operating on a margin of less than a penny a trade, may not want to pay, Tabb said.
But, if you are set up, adjusting to the NYSE’s backup plan would have or should have been easy, said Jeff Bell, chief executive officer of Lime Brokerage, a unit of Wedbush Securities that develops high-speed technologies in order to provide “direct access catering” to traders, hedge funds, and institutions. Lime connects to all the stock exchanges operated by NYSE Euronext, Nasdaq OMX, BATS Global Markets and Direct Edge.
“All that would have happened, in my view of things, was we would have taken down the routes to the New York Stock Exchange proper” only, Bell said.
“What is different, at some other firms, is they maybe don’t trade all the markets,” he said. “They really just need to get an execution done and they like to go to the New York (exchange) because they’ve been doing it for decades. But if all of a sudden that execution venue is gone, they’ve got to find another way to work around” that.
