Yesterday was a busy day for me.  It started at 4:30 AM when I had to do an interview with a reporter from Bloomberg who covers the European Stock Exchanges.  There was then coverage of the goings on with some of my clients in the Wall Street Journal, Financial Times, and many other papers, which kept me fielding questions most of the day (and I have more reporter interviews today as well).  This interest in the exchanges spilled over onto Twitter as a result of people commenting on the coverage on CNBC.  As a result of that,  I thought I would talk a bit about the trends and challenges faced by the World’s Investment Banks, Hedge Funds and Exchanges as they grapple with their Need For Speed as their seemed to be interest by many of my followers both here and on Twitter.

So what’s driving the system designs at today Financial Market firms.  Well, amongst other factors such as cost reduction, operational efficiency, and the other usual IT issues, is an ever increasing need for speed.  Let me give you some background.

• If you’re Twitter, you handle about 6 million Tweets per day and peak out at about 200 or so Tweets per second based on what info I have seen.
• If your a Stock Exchange 10 years ago, you did a peak of a few thousand transactions per second
• A big credit card processor handles a few tens of thousands of transactions per second at peak
• Today’s Stock exchanges are building systems capable of handling millions of transactions and quotes per second

Not only are the throughput requirements exploding, but the response times or latency tolerance is approaching Zero at an alarming rate.  Again for comparison:

• A Telecom system handles a few hundred thousand calls per second but can take a few seconds until the first ring without people complaining
• Twitter, which is considered “real time” also takes a few seconds to acknowledge and publish a Tweet (at best)
• A credit card processor also can be a few seconds to respond
• 5 Years ago the NYSE would take about 40 milliseconds to process a trade
• Today cutting edge exchanges are building systems which can process a trade in under 100 microseconds (yes that’s micro, not milli and definitely not seconds).

On the flip side,  Banks and Hedge Funds with their algorithmic trading systems are sending trades into these exchanges at volumes which for stocks is increasing at over 50% per year and for options at well over 100% per year, so these system need to scale.  In addition those same firms are monitoring the response times of each exchange and if they see one being slower than another, the trade gets routed to the faster exchange wherever possible.  The drive toward zero latency is causing a lot of traditional exchanges who had legacy systems such as the New York Stock Exchange, London Stock Exchange and Deutsche Boerse (three of the biggest), to lose market share and as a result they are all undergoing radical redesigns and deploying new cutting edge systems.

In addition to all of this speed,  keep in mind that the reliability levels on these systems are ultra high as well.  We design for 99.9999% uptime with absolutely zero loss of messages.  Its that need for high reliability levels coupled with the speed that makes building these systems such a challenge.  Making things fast without being reliable is easy. Making things reliable without being fast is also easy.  Bringing them both together is very difficult and requires radical new system designs.

Just think of some of the challenges you face.

• How do you log transactions to a database when a physical hard drive takes 2 milliseconds to do a write
• You can’t do database transactions in the critical transaction path because any database operations kill you for throughput and latency
• You need to be highly horizontally scalable to handle the constant growth in transaction volume
• You need to be running redundant hot/hot configurations for failover because the system reliability target is higher than that of any single component, components will fail,  but the system must stay up and not lose a beat
• After 9/11 the Disaster Recovery mechanism has to keep the system up and running even during and after a 9/11 type event with no loss of messages or down time
• How do you record all the trades.  A trade record is typically pretty small, between 120 and 400 bytes, not much bigger than a Tweet on Twitter.  People are always seem amazed that Twitter needs to store a few 10′s of GB per day.  Well with a modern exchange system we log about 1.2 TB per day and we need to keep 5 years of that searchable on main storage without going to tape, etc.

In these systems every microsecond counts.  As such even network path length is measurable and directly impacts trading profits for banks and hedge funds.  As a result firms will move their trading systems directly into Collocation facilities offered by the exchanges so as to have the absolute minimum latency as a result of the network itself.  Networks in these facilities are going from 1GigE networks to 40 Gig Infiniband and/or 10GigE (which is slower than IB for both throughput and latency).  The NYSE is putting in 100GB Fibre Optic Switches for their WAN.  They use ultra high speed messaging software on top of that network, solid state disks with SAN backups, the fastest processors with the highest I/O.  Every line of code needs to be extra tight, with Matching Engines in the exchanges typically being single threaded and using an MPP design instead of SMP because even the cost of a thread context switch is measurable and impacts profits.

Its a very tough set of criteria to meet and poses some very unique challenges.  I’m proud to say that currently the fastest of the new systems coming to market are ones I helped design and using technology I helped IBM develop, so I guess we’re doing something right.

If people have any questions on this or any of my other posts, please use the comments or reach me on Twitter at twitter.com/techmusings (@techmusings)