Some time ago, we published an article about a very simple, yet effective test scenario: the 5000 transactions in a row test. We talk about this a lot with our clients and I therefore want to revisit this scenario. 

The idea behind this test is simple, but powerful: If we stress the terminal continuously with regular transactions, we will discover bugs that we cannot discover even with the most elaborate test scripts. 

We have been writing terminal applications for years for our clients. Our experience has shown that some issues, especially in lower-level components (i.e., memory handling, driver crashes, file system, DB) are extremely hard to discover and therefore to solve. But such issues have a detrimental effect to the overall quality of the terminal application. They can show up everywhere, at any time, and no one knows why. The result is bad user experience, unhappy clients, service calls, a bad reputation and even worse, there is no clear path forward to solve the issue. 

Small step, big impact

Without automation, it is super hard to stress terminals continuously. How many weeks will it take a human to perform 5000 test transactions in a row? How effective and accurate is a human after 100 test transactions, or worse, after 3 days of doing this? What happens over night, when humans sleep? 

Only with automation you can stress the terminal to a point where you will find underlying issues. In addition, it is crucial to use the actual terminal hardware for testing. Only with an end-to-end setup, involving all hardware, firmware, memory systems etc. you can truly benefit from this. 

Enter test automation with robotics. With our robotic solution, this test case is very easy to implement. Most of our clients are up and running with a simple automated transaction in a few days, if not hours. As soon as you have this simple test case, you can loop it, and off you go. The robots will work 24/7, are faster and more accurate than humans and can truly stress your terminal. You will be surprised about the results. 

70 issues per terminal?!

We created to robot out of an internal need. We wanted test automation for our terminal dev team. The 5000 transaction in a row test was the first thing we did with our very first robot. Here is a summary of what we discovered: 

• Test performed with 76 different terminal models and applications
• A test run took approx. 2 days
• None of the terminals passed the test at the first go
• We found an average of 70 bugs per terminal
• Over 50% were hardware and firmware bugs that had to be fixed by the terminal vendor

We did expect to discover some issues, but the sheer number of issues and the fact that none of the terminals passed this test was a huge Aha moment for us. 

After fixing the bugs we found, we saw a massive improvement in overall quality and much happier clients. All of this without complex test cases and test coverage, just by running the simplest test transactions. 

Do your terminals pass the test?

This simple test will improve your overall quality too. After all, a stable platform improves your application development. In fact, we recommend to all our clients to start with such simple cases first. “Low hanging fruit” mean automation at little cost, with big impact. Simple cases like this will give you a quick win and even justify the cost of a robot. And while you can reap the benefits of this test quickly, you can experiment with more complex test automation in parallel. 

We would love to hear about your experience. How do you deal with suspected low level issues? What happened to your terminals after performing 5000 transactions in a row?