Tradition is one of the world’s leading inter-dealer brokers, operating in the most complex financial markets, including interest rate and equity derivatives, FX and energy markets. While Tradition has built fully automated broking systems, such as its ParFX Platform, it also employs around 2,000 voice brokers in different locations around the world. So Tradition’s software testing – in particular its non-functional testing of how its systems interact with its markets; how its security systems hold up against attack and how resilient platforms are when a component fails – is complex to say the least. Jeremy Norwood is Tradition’s programme and project change manager. He’ll be speaking at the QA Financial Forum in London on April 5th alongside Iosif Itkin, CEO of Exactpro, the broker’s provider of non-functional testing services, and part of the London Stock Exchange Group.QA Financial met up with Norwood to find out more about the challenge of testing electronic trading systems and how Tradition and Exactpro are using “mini-robots” to replicate the chaotic nature of markets. Norwood, a former aerospace engineer, joined Tradition six years ago. Reporting to group CIO Yann L’Huillier, he heads the quality assurance function, oversees analysis for business change and directs IT service management. Tradition’s collaboration with ExactPro dates back to 2010 when Exactpro (then an independent Russian company ) was brought in to test Tradition’s interest rate swap trading platform. Since then, Exactpro has taken on the management of quality assurance across the IT platform at Tradition, providing both tooling and test execution support. Exactpro’s latest project at Tradition is a test-automation suite for non-functional testing. Here’s our conversation with Jeremy Norwood:
Q: Let’s start with the the key differences between functional and non-functional testing. What are they?
In functional testing you are seeing whether a component works according to certain specifications. If you are testing a the brakes on a car, you want to see if they work when you press the pedal and if the lights at the back of the car light-up. A non-functional test would be testing the brakes on an icy road, or going very fast. You want to see how whatever it is that you are testing performs in a variety of real-world conditions.
Q: What are the key testing priorities for non-functional testing in a brokerage firm?
There is more risk around electronic trading then there is with our voice trading. This is because a human broker is only inputting a trade, say, every 15 seconds, and they will stop when they notice that something is wrong. In a computer-driven system, you set up some parameters and then you let it run. That means it can get out of control quite quickly. By the time you notice something is wrong, the the electronic system might have executed a large amount of unwanted trades. In terms of scenarios that we test for – load testing is one priority. We want to see how our system performs when there is a spike in trading which might replicate a runaway high-frequency trading algorithm. We want to see how well it can handle a large amount of users accessing it at once. Cyber security is also a priority. Financial firms deal in trust, and our users have to know that our systems are reliable and that their data is protected. A data breach, even when it is just trade information and not personal data being leaked, can have consequences on the market. Finally, we want to see how our system deals with component failure. How resilient is it in case one of the software components fails?
Q: How do you automate your non-functional tests?
The first step was automating our functional testing. We would take an automated script that executed a trade request, a standard functional test. We could then use this automated script to inject a large number of messages into the system. We can simulate the system executing two thousand trades a second, bring it up to ten thousand, and then back down to two again. This allows us to build an automated non-functional test suite. The most important aspect of non-functional testing is accurately mimicking real-world conditions so that we can prepare for what might actually happen. This is daunting for a brokerage firm because we operate in an extremely complex financial ecosystem. If we imagine banks as complex systems in their own right, then a brokerage is where these systems intersect, and they interact in chaotic, hard-to-predict ways.
Q: How does Exactpro help you simulate real-world conditions?
They have released a feature called ‘mini-robots’ that are designed to mimic client behaviour. One of these mini-robots will take market data and feed it into the system, then another robot will react to that data, simulating how traders adapt to a changing market. The technique is called chaos testing – the cumulative effect is of simulating a complex trading ecosystem. An old-fashioned load test works a lot like a functional test. You insert an input and verify what the output is, whether the system fails or not. In reality, if our system receives a message, it is then passed on elsewhere in the form of market data, regulatory reporting, etc – actors in the market will make decisions based on that input and change their trading behaviours. Chaos testing is about trying to predict these interrelations – building a system that is resilient in the face of real-world unexpected events.
Q: What are your future testing priorities?
We have automated 80% of our testing for our electronic systems, but less than 5% of our voice trading systems. We want to expand that, but the challenges with voice trading are very different from electronic orders. MiFID II will require that we capture price information in voice trading, but voice orders are very dynamic, human traders go back and forth discussing price, often they use jargon. We are going to have to capture that information, with someone inputting it into an interface. I know some firms are looking at voice-recognition, but that poses a different set of challenges. Jeremy Norwood will be speaking at the QA Financial Forum London on April 5th organised by www.qa-financial.com. See the agenda here. If you are interested in attending please contact pat@qa-financial.com. If you work for a financial firm, you may be eligible for one of our VIP guest passes.
