While financial firms are achieving higher rates of test automation and moving towards mature agile and DevOps practices, test environment management is a “bottleneck” that can often go unnoticed, according to Maya Ber Lerner, CTO at Quali, the test infrastructure automation specialist.
“We might have a lot of test automation but it doesn’t necessarily have anything to run on,” explained Ber Lerner, who will be a keynote speaker at the QA Financial Forum London on March 12th. “One of the things that often happens is you create thousands of tests and you think, ‘Oh great, now my coverage is going to be better and I will test every part of my application.’ But the way we handle our test environments creates a new bottleneck. Test environments involve making sure the right versions of applications are available for testing and also the data needed to run the tests. But we are still dealing with them in a backward way.”
Ber Lerner believes that many companies are struggling with their test environments because they never really think about it. “You can cut your testing time frame from four days to 10 minutes say, but if setting up an environment takes you one day, then you suddenly realise the problem. When environments are causing failures, it becomes much more apparent than in manual testing,” she explained.
While financial services firms have succeeded in increasing rates of test automation, Ber Lerner noted that they had extra concern. “With financial services, there is of course extra sensitivity to testing and quality. You can’t really take too much risk at the expense of your users. Everyone loves automation in financial services and there is a willingness to adopt these new technologies, but the test environment problem comes to the surface pretty quickly.”
As automation becomes even more prevalent, Ber Lerner maintains the onus is now on companies to focus on the reliability of their testing process. “We are over the excitement of things happening automatically and now the question is how can it be made reliable?” she said. “We need to make sure that when we run something, we don’t just get a huge list of failed tests because that becomes a problem on its own.”
“The reliability issue is not just about test environments. It is also about deciding which tests to run, which tests to get rid of and how to make sure your tests stay current with your changing application.”
Ber Lerner will be talking about how to achieve continuous testing and the advantages of dynamic test environments at the QA Financial Forum London on March 12th.
