In today’s world of Agile and DevOps, test automation has become essential for continuous delivery. And although technology has come a long way in assisting development teams, there are still pain points. It was these pain points that inspired former Google personnel Izzy Azeri and Dan Belcher to launch mabl – an SaaS solution which creates and maintains functional tests for web applications using machine learning. “Dan and I wanted to find the pain points in the broad cycle of software delivery – from building an application to testing, releasing and monitoring. After speaking to a number of dev teams, testing was the bottleneck. Because the front end of applications change so frequently nowadays, tests are breaking. So instead of finding bugs, QA teams spend a lot of time maintaining tests,” explained Azeri. Making testing smarter is the objective for the duo, but keeping a tester’s role relevant is key. “We see machine learning and AI as an extension of a human. And so when we use this technology to make predictions or changes in an application, we will always let the human override these,” explained Azeri. “What mabl does with her AI capabilities is to be smart enough to only notify testers update tests automatically when an application changes, but also gives testers the option to reverse the change if required.” The Boston-based start-up is now in its public beta phase after securing $10 million of funding earlier this year, and currently thousands of organisations, including financial services, are using the service. The company is set to announce a paid product before Fall 2018. In this interview, Azeri provides an insight into how financial services are using mabl, the regulatory challenges the company faces when dealing with financial services clients and his plans for future investments to make testing smarter.
Q. How are your customers using mabl?
Financial services are among some of the companies currently using mabl. At one end of the spectrum, we have large consumer brokerage firms, which have web applications used for trading purposes. And because teams have adopted Agile and DevOps principles, these applications are being updated frequently. The teams here are using us for better test coverage – with mabl, they can train tests for these applications and are able to more easily maintain these tests. Another feature that is particularly beneficial to this group is the ability for mabl to ‘look’ at a page visually and ignore areas which are typically dynamic. On a stock brokerage application, often there are on-screen charts and graphs changing frequently, but mabl focuses on areas of a page which are more static, like menus and sign up buttons, and will only alert users when these areas change. At the other end of the spectrum, we have several FinTech clients who are developing web apps for consumers to better track their finances. In these apps, there are so many order cases, and these companies are using mabl to automatically identify regressions or bugs. First, they will train mabl on their site and each time a test runs, mabl builds a model, or the predicted outcome, of the test. Each time there’s a deviation from this result, mabl notifies the tester without them having to look at the test results. We also have a number of e-commerce users. Runkeeper, owned by Asics, have leveraged mabl to run what we call a link checker across several of their sites. mabl will author a test and once a user points mabl at a web application, like an e-commerce site, mabl will automatically crawl the site and identify broken links and java script errors. This is valuable, because most of this is done manually today. These apps are designed for consumers so they’re a very large volume of users who sign up, and these companies are using mabl to automatically identify regressions or bugs, specifically in sign up flows.
Q. What regulatory concerns have you had to consider when working with financial services?
What we have seen on the regulatory and security side when dealing with financial services is three-fold. Firstly, many financial services applications are publicly available, but people need to sign in to use them – this is fairly easy for anyone to test in production. However, their QA or staging environments sit behind firewalls and so the first question we are faced with is how can mabl securely connect to an application sitting behind a firewall and how can testers be sure that the only traffic getting in or out is by mabl. That’s a problem that we have had to solve, and to do so we have used different techniques from a security perspective. Today we use static IPs and custom host headers and we are in the process of developing VPN. Secondly, the location of where data is stored is a huge concern for financial services; they need to ensure that they can track where their data is. We are transparent about where we store our data. Today we run on Google Cloud Platform and store data in the US. However, we will support mabl running in other data centres around the world to make sure data is stored locally when required from a financial services perspective. And the third concern we are faced with is to show how we secure data. mabl has invested heavily in data encryption, both when data in transit as well as at rest. We also use Google’s key manager service to encrypt and decrypt data based on the customer’s requirements.
Q. What are the main challenges testing teams are facing today?
Quality assurance is a bottleneck, and the tools out there for test automation QA Engineers are not designed for today’s environment where software is being delivered much more quickly than say 10 or 15 years ago when Selenium was invented. The other problem is that people are trying to solve this issue with manual testers. And with that, we what we hear from software teams is that there are not enough testers in the market. Even when teams are hired, it takes time to find bugs. You’re running on human time not computer time. We want to use machines to automate many of the repetitive tasks that humans are doing, so that humans can focus on the important tasks like more test coverage and tests that machines can’t actually do. For example, one of the important areas of testing that we don’t think can be automated today is usability testing – really making sure that applications can be used by different audiences. That’s a very difficult thing to automate using machines.
Q. Who are your competitors?
The ways customers are solving testing problems today have been around for decades. Selenium, which has been around since 2004 and has millions of users worldwide, would be our first competitor in terms of a product for test automation. Clearly, we are very different in that we are a new commercial product, which uses machine learning. We want to make it simple for people to make tests. Unlike Selenium, you don’t need a computer science degree to write the tests and scripts. All you need to do is click around on the website just as a customer would, and mabl can replicate those actions and go from there.
Q. What does the future hold for mabl?
In the short term, you can expect to see mabl extend its platform to include mobile applications, as well as web applications. Customers access applications on many different devices, and we want to make sure we support the testing of these. In the long term, we are investing heavily in machine learning and AI to really make mabl smarter. We want mabl to be like a new person on a team. For example, wouldn’t it be great if mabl could tell a financial services company that although they have created ten tests, users are actually using the app in many more ways, and then give suggestions for other tests that they can create – that’s the stage we want to take mabl to.
