NOTE: This article was written in collaboration with JD Raimondi.

In recent years, artificial intelligence has become all the rage. You can find lots of articles, papers, news, and webinars about it (even more so during the pandemic). But what exactly is AI? Will it utterly change the direction that the IT industry is heading? Will these changes impact software testing eventually, or has it already and we haven’t realized it? We’ll try to answer some of these tough questions below.

What is AI?

Let’s start with some basic definitions. Unfortunately, a basic, widespread and commonly accepted definition is hard to find (fun, we know). There are different views on what AI is and this depends on multiple factors.

To set the tone for our discussion, I’ll share some of the most basic ideas, taken from different sources, so we can all speak the same language and go more in-depth into this topic.

According to the dictionary (as basic as can be), Artificial Intelligence is:

“The theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.”

In addition, authors Stuart Russell and Peter Norvig, in their book Artificial Intelligence: A Modern Approach (3rd edition), went a little deeper and defined AI according to the process through which it’s analyzed. To do this, they provided this explanation, followed by a simple table:

Now that we’re clear about what AI is and what it’s not (the power of machines to take over the world and enslave the entire human race), we should NOT confuse our definition with Machine Learning.

To clarify, the definition of machine learning from wikipedia.org says:

“(Machine learning) is the study of computer algorithms that improve automatically through experience. It is seen as a subset of artificial intelligence. Machine learning algorithms build a mathematical model based on sample data, known as ‘training data’, in order to make predictions or decisions without being explicitly programmed to do so.”

So far, we have set the necessary ground rules for the rest of this article, so we all can understand and enjoy what comes next. Let’s now dive into how AI may (or may not) be affecting software testing.

The long road of software testing

Software testing is a very repetitive, time-consuming process. It doesn’t matter if the testing is performed manually, if it’s automated, or if it’s performed by a QA team or a single person.

No matter what the project is or the technology involved, the testing process is a long road.

Testing large and complex applications is especially difficult. A large volume of information or input sometimes becomes the reason behind poor software quality, undiscovered bugs, and partially (or complete) features delivered to clients untested. This is simply because resources are limited and the effort required to achieve the necessary level of software quality tends to be even greater.

Sadly, it’s common to find large, unmaintainable, and hard-to-understand testing suites that even have deprecated test cases. This is particularly true for projects that are quite old and have gone through a lot of changes over time.

Moreover, when the application is subject to constant changes, it speeds up the rate in which test cases become useless. To keep up with said changes, it needs to be constantly updated or modified. This could result in a progressive loss of quality and lead down a path where the client’s satisfaction with the product decreases rapidly.

So, here’s the most anticipated question: how does AI help (if at all) a software quality team to improve and overcome these problems? How can AI and machine learning be applied so that we can stay relevant in an environment where things change and quality processes fall behind?

AI vs. effective testing

When I say effective testing, I’m referring to the mindset about creating test cases as clear and independent of implementation as possible. This way we can minimize the number of changes required in our testing suites.

Even though there will always be a portion of the tests that will inevitably become deprecated and unused, making the effort to minimize that number will keep us on track during every sprint and release we plan.

This can be achieved by using parameterized test cases, implementation-free descriptions (so that we validate the feature instead of how it’s developed), decision tables, and various techniques we have available.

With this understanding, the idea to implement AI as a solution to our ever-changing world may be useful.

There are a lot of advertised AI tools for testing that use different machine learning techniques to improve all the previously mentioned problems related to the testing process. It only requires a simple Google search to find different available products, most of which require membership. They claim to have different machine learning techniques developed that will help you define testing scenarios. Apparently, the application will “automatically” learn to detect changes and adjust defined test cases.

In theory, these tools use different machine learning techniques that will reduce testing time, overcome changes in the applications and keep the quality up-to-date. Visual testing is one of the most common features advertised by these products, where changes in a component will produce a test case to be updated automatically. It will inform the user that a difference between executions have been found, as well as how it was solved.

That said, using one of these tools may have beneficial results for the quality assurance process and development overall. Sadly, we’re in very early stages of this method. In my opinion, it will continue growing and expanding until it becomes the “new normal,” as we’ve been hearing all year long.

Now, don’t get me wrong. These tools can actually save time in repetitive activities that your team needs to perform. But because none of them are industry-standard, and all of them have proprietary workflows and require payment to access their “secret sauce,” it’s likely that a lot of your testing time will now be learning-curve time. This may not result in effective testing (yes, you read that right). You should always aim to create effective testing and efficient processes within development, and quality assurance as part of it.

Unless…

Note that the problem here is about the team’s efficiency with utilizing the tool. The biggest challenge regarding a tool is the human component. That’s because the team still needs to validate what a good expected result is and lead the tool towards the right verifications. Yes, the tool will learn, adapt and get better at detecting failure scenarios, but it still requires a lot of manual work.

There’s another approach to this, where a system tells you what an appropriate response looks like. Instead of thinking of full-interactive applications, think of a report, or an executable file with a simple output. In cases like these, AI can auto-discover which variations will make the application fail and which ones work correctly. In fact, this gives rise to a whole other research topic called dynamic analysis, which we’ll leave for another day.

The takeaway: it’s not that AI is ineffective for testing. It’s that we’re not there… yet.

Conclusion

Hopefully you now have an overall idea of what AI is and how it will change the world in ways we don’t yet know. It’s already having a major impact in medicine, communication and software development, to name a few. It will continue shaping our world with new opportunities and create jobs of the future, that’s for sure.

However, when talking about AI applied to software testing, it may be too soon to have a real measure about its impact and importance. There are excellent online courses on this subject and I strongly encourage you to read, learn and get involved with this topic because it will definitely become more relevant in years to come.

Last but not least, it’s up to the team (and available budget) to decide if it truly represents a major benefit to use or implement AI for testing purposes today. I see a bright future with the help of AI for all of us. If we’re willing to change alongside technology to stay relevant, improve our clients’ growth opportunities and improve industry standards, AI will definitely play a crucial role in redefining the way we code, test and perceive our world.

Cheer up! This may not be the happy conclusion you were looking for. Nevertheless, the future of software testing and AI is an exciting frontier. At the very least, it will make us better at assuring software quality in the near future.