How to Choose an Automation Framework Without Dying in The Process

It is a reality that not all of us have the good fortune to start a new project from day 0 (I see C programmers get hyped). The software industry is fast-moving and involves constant change -companies, projects, clients and even tools from one job to another.

For this reason, not many people are faced with having to decide which tools, libraries or frameworks to use for a particular project. It’s more often the case that when we join a team, the project is already underway, with tools and automation already defined.

In the rare occasion you do get to choose, you may have already seen that the universe is full of varied, diverse and multi-purpose tools. With so many choices, it’s hard to even make a decision. If you need to choose a framework, this post is for you.

To start with, I’ll give you a general rule of thumb: you should ALWAYS make your decision according to the project and your client’s requirements rather than personal taste or preferences.

Here is a simple guide for choosing an automation framework. Hopefully, it will help make the decision process a little less complicated.

The analysis

Ok, so what should you take into account when deciding which tools, libraries or frameworks to use? Here is a shortlist of considerations to make:

The platform in which the app will work
The type of testing you need to cover (E2E, acceptance, unit, etc)
The programming language you are most familiar with
The complexity of the application and the potential size of it

These are only some of the key points (in my personal opinion) on which you should base your decision-making process. There may be more, depending on your personal experience, your knowledge and your practical experience.

Let’s expand that list.

The platform in which the app will work

A very important point when choosing your tools is to consider the platform in which the application will run. For example, if the application under development is a native mobile app, we get a different set of tools to use than those designed for web applications.

For mobile applications, there are some recommended tools like Appium which you can use with different programming languages like Java, Python, Javascript, and more. The advantage of using this tool is that it is a robust framework that can be used for testing both Android and iOS applications native or web apps running on mobile devices.

However, it is very hard to configure and you may face a lot of issues when trying to set everything up and start testing. Be patient: your troubles will be worth the pain and suffering of working on getting those initial basic configurations right.

There are other frameworks and libraries for testing native mobile applications. There’s Jest or Detox, both of which can be integrated into the application code itself to improve the quality of the code.

For web applications, the list of frameworks available is slightly bigger. There are a lot of good and useful frameworks like Cypress.io, Selenium, and even Katalon (which may be a good experience for those entering the world of automation for the first time).

Many testing frameworks that support Javascript use the library Mocha for syntax and Chai for the assertions so if you know Mocha, you have already covered much of the terrain required to start creating tests.

However, the configuration part of those frameworks (based on Javascript) may vary in difficulty and effort.

The type of testing you need to cover

Although we have many tools at our disposal, not all of them may be fit for the requirements we have or the scope of the testing we need to implement. In some cases, we just need to test the APIs or the front end, or the database. Consider the reasons behind the implementation of an automated testing tool – that will help you arrive at the best choice for the framework you’re going to use.

Also, it is important to keep in mind what the tool was designed to do – its purpose. Take Cypress, for example. Cypress is a powerful framework for testing web apps. However, it may not be a good choice if you need to test the navigation between pages (if that is what you need) as Cypress does not (and will not) support multiple opened tabs at the same time as, for example, Selenium does.

That being said, your testing strategy should be an important part of the decision process. You should never relegate the strategy to the tools. It should be the complete opposite. When you have a clear path on what strategy to use for a particular project or a client, you can narrow the number of options available and categorize them into what they really offer according to your needs.

This is not a minor consideration. If you spend time struggling with a framework that will only be used to test a little portion of the app, using only a minimum set of its capabilities, maybe it is not the correct one to choose. Time, cost, maintenance and testing requirements should all be put on the table for analysis. Be careful.

The programming language you are most familiar with

It may seem obvious, but it is not. When looking for the right tool, it is better to stick with known territory instead of exploring the unknown lands of different programming languages. This is a pro as well as a con. Not every testing framework has a wide range of supported programming languages.

For that reason, you should consider the testing requirements and then search to see if there is a framework or library that could help you reach your goals and which uses the language you know. If you don’t, you may face a learning curve not only for the tool, but also for the language itself.

On a personal note, it is always a good thing to learn and expand your knowledge on new languages, frameworks, methodologies, you name it. But (and this is a big one) if you don’t have the time to spend on learning anything new due to time restrictions, tools already used by the client or any other important reason, you should consider the tools available and adapt to them according to your testing requirements.

This probably means that you won’t be able to implement the complete set of automated tests you would like, but a little is better than none at all and it will definitely save you time in the long run.

The complexity of the application and the potential size of it

The last item of the list, but definitely not the least important, is to always plan ahead. Not only for testing but for everything. Although it is not possible to consider everything and the impact that our application may have, it is quite possible that, even if we succeed, our scope may grow, more clients will use the app, and any changes in the app will most definitely break our tests.

For that matter, when considering tools to use, you should do a little digging and see if the one you would like to use behaves properly against potential growth. This is very important because it is not something that we are keen on considering at the very start of a project.

Conclusion

Implementing automated tests in a project can be a great way to make sure that the product we deliver to our clients is not only good and complies with every requirement but also represents us, the company, according to values and what makes us special.

The tools we choose to use should be a result of planning, thorough investigation, and a good understanding of what we want to achieve and how we want to do it. It is not enough to use a particular framework or library just because you are familiar with it. It is also not a good practice to try and use just one tool for every case. There is no such thing as one framework to rule them all (or test them all, in this case).

Always do your homework, read the documentation, browse discussion forums. and do a little research about your needs and expectations.

As a little bonus, here is a table with some personal opinions and the field in which these tools are good for. I hope this helps you decide what is best for the company, the clients and for you.

Happy coding!