If you’re at all concerned about reigning in expenses during the app dev process, you’ll need to pay attention to test planning. And if your client is counting on you to deliver the best possible product, test planning is an absolutely essential part of the process.
However, like everything we do in this industry, there’s no one-size-fits-all answer to the way developers test apps. To help you out and provide a little guidance, here’s a handy guide to the various ways you can perform this essential step in the development process. We’ll examine the tools available, and how each approaches the important matter of testing native apps.
Introducing Native Apps
A native application (native app) is an application program that has been developed for use on a particular platform or device.
Because native apps are written for a specific platform, they can interact with and take advantage of operating system features and other software that is typically installed on that platform. Because a native app is built for a particular device and its operating system, it has the ability to use device-specific hardware and software, meaning that native apps can take advantage of the latest technology available on mobile devices such as a global positioning system (GPS) and cameras.
Why Testing Native Apps is Tricky
However, testing for these apps, from the UI perspective as well as the functional, is more complex than you might expect. It’s not just a matter of just segmenting things by platform/Operating System (OS), since there are different versions of the same OS in the market at the same time, as well as different devices with different screen sizes and hardware.
Common Ways to Test Native Apps
Here are three of the most common ways to test native apps:
- Emulators
- Physical devices
- Online tools
And don’t forget: functionality and UI aren’t the only criteria. As a developer, you’ll need to provide a testing solution that balances time with cost-effectiveness. We’ll be looking into that angle, as well.
1. Emulators
Frameworks used by mobile developers (both iOS and Android) provide built-in emulators, making it possible to test apps during development.
While setting up a development framework with the emulator is pretty much an out-of-the-box process for developers, it’s a different story for QAs. For QA’s, it means replicating the development environment on their computers, which under certain conditions may not be the most practical approach for testing purposes.
Here’s an example. Installing the iOS framework for mobile (XCode) requires an Apple computer, so if you are working on a PC, you may need an Apple computer to set up the environment, or at least a Virtual Machine. That adds a whole new dimension to the process.
However, this does often turn out to be a practical approach, since it can be implemented locally and without licenses if you decide to use non-commercial frameworks. Plus, there’s no need to buy physical devices. In addition, emulators provide quite reliable results when compared to results garnished from actual devices. On the other hand, QAs may require assistance from the developer in order to complete setup.
2. Physical Devices
The best way to test that apps are looking and behaving as expected is and always will be testing in the actual devices that will support the app.
The thing here is that this method is not very cost-effective. That’s because mobile devices get updated in relatively short periods of time. Besides this early obsolescence of devices, we must also consider the large number of different combinations of hardware and operating systems. This is especially the case when working with Android, where you have as many OS versions available as brands selling the devices.
Considering the cost of having so many devices to provide proper support and coverage, it’s an unlikely reality to have on hand ALL the devices that would be required.
Additionally, there’s an extra logistics cost when the devices are needed on different sites. That cost comes in both in money and time, neither of which are usually in abundance on most projects. It is always easier and faster to access a tool using the internet compared to waiting for a package containing smartphones to arrive in the mail.
Online Tools: the Best of Both Worlds?
Tools such as Browser Stack provide a hybrid solution that blends the best features of emulators and physical devices. These tools provide, for a fee (monthly or annual, depending on your specific needs), access to emulated devices. This includes older devices as well as a continuously-updated farm of physical devices.
Accessing a wide range of devices, for the fraction of the cost of such devices, is a balanced solution for mobile testing. However, this solution does have some negative points as well. Purely emulated devices may not reflect 100% accurate behavior of the app in the physical device. Also, using the physical devices from the farm, while providing a good way to test both UI and functionality, results in slower response times, thus making the testing tasks more time-consuming.
Which is the Best Option?
Here’s the simple answer: the best option is the one that best suits the needs of your project and the role you play.
If you are a developer, the emulators provided on the mobile frameworks are most likely your default choice.
Working as QA, your choice depends on the coverage and support the project is committed to. If you have available devices from previous projects, that share mobile scope, the physical devices are your best choice.
If you don’t have any devices available, nor the resources to acquire them, you might want to go with the more balanced choice of a Browser Stack-like tool. Or, if you can set up a computer to run the development framework required, the local emulator may be enough for your needs.
In my experience, a mixed approach that provides a good balance between cost-effectiveness and coverage provided is the best solution. You get access to some physical devices, that may be shared or reassigned from other projects, and they’re supported by an online tool to provide better coverage on different combinations of hardware and software. The bonus is: that probably also includes newer devices that may not be available or affordable at the time.