When Tokyo Stock Exchange Failed

In today’s Finshots, we talk about how a failing data storage device brought the Tokyo Stock Exchange to its knees.

Also, a reminder that you can listen to this story on Spotify and other podcasting sites.

The Story

Stock exchanges facilitate the buying and selling of stocks. But more importantly, they are matchmakers. They connect buyers and sellers and help market participants get the best price each time. At least so they claim. But today’s story involves the Tokyo Stock Exchange  (TSE)— the largest stock exchange in Japan and the third-largest stock exchange in the world. Anyway, TSE can facilitate the electronic buying and selling of shares thanks to a complicated trading system that does most of the heavy lifting. The most critical aspect of which is order-flow-management — To process the buy and sell orders making its way into the system and do all the matchmaking that follows soon after. So it’s quite safe to assume that the platform deploys state of the art hardware and software systems to make sure things always work smoothly.

However, even the best systems aren’t always fail-proof. For instance, the Tokyo Stock Exchange uses the Arrowhead system (developed by Fujitsu). They’ve known to be slightly glitchy at times, but they’ve never failed completely. And a recent overhaul was meant to improve and stabilize performance considerably. But then on Thursday, a critical data storage component detected a memory error i.e. the system wasn’t able to retrieve the right information regarding customers, orders, passwords, etc. But fear not, system architects are smart. This is precisely why they create redundancies. If the primary data device fails, you switch to the secondary data device and in most cases this switchover is automatic. But for some unknown reason, the switchover did not happen. And at this point, you are probably wondering if the exchange is in trouble.

And the answer to that is…

An unequivocal yes.

You see, TSE processes close to 100 million orders per day with transaction value totalling over $28 billion. Hundreds and thousands of market participants log in every day in the hope that the exchange does its job. And since they are putting millions of dollars on the line, any failure to execute transactions on time could result in catastrophic consequences. More importantly, if people lose faith in the system and the exchange, they’ll likely have second thoughts about trading as a whole. So when the backup protocol did not work out as planned, IT staff at TSE were scampering. They manually intervened trying to force a change over to the backup device. But that only complicated matters further as the system had to now reboot before trading could resume. All orders placed during this period were left hanging and the exchange had no choice but to suspend trading. If they hadn’t done it, more traders would have punched in orders (that wouldn’t have gone through) creating more confusion and hurting them financially. So TSE in all likelihood did the right thing.

As an article in BloombergQuint notes —

It was the first time in almost fifteen years that the exchange had suffered a complete trading outage. The Tokyo bourse has a policy of not shutting even during natural disasters, so for many on trading floors in the capital, this experience was a first.

It was only 8 hours later that the executives convened a press conference to explain why they chose to cease all trading operations. In traditional fashion, they bowed in apology and refused to attribute blame to Fujitsu and their arrowhead system. Instead, they took complete accountability for the failure and promised to restart operations the next day. So while they might have pissed off a few people during the whole episode they did manage to salvage their reputation by the end of Thursday.

Anyway, that’s it from us.

Share this Finshots on WhatsApp, Twitter, or LinkedIn.

Also don't forget to check our daily brief. In today's issue we talk about Facebook's possible antitrust lawsuit, why Ola's London license was revoked, and why Singapore wants its citizens to have more babies Do read the full draft here.

Until next time…