Effective QA: A lifesaver for your business reputation and your pocket

Anyone who’s ever been involved in the creation of a digital product, should be aware that testing is an integral part of the development process. At 10Clouds, we have long ago implemented a rigorous testing procedure into our work as we know how important it is to help our clients avoid system bugs, UX issues and a whole host of seemingly small errors. If these are left untreated, they could lead to larger-scale, highly costly problems further down the line.

We have a whole team here, entirely dedicated to testing, and we wanted to share with you the different types of tests that we run, when we use them, and why we think they’re essential. We hope you find our guidance useful.

This article is for you if:

• You want to understand the importance of testing in the development process
• You want to get insight into the risks involved in not conducting tests

The goal of testing

The ultimate goal of software testing is to improve a business. The best way to do that is keep users satisfied by providing them with high quality products – good looking, easy to use and bug free applications.

There are countless testing types that might help us to achieve the above aims. We could test and polish our app to get close to perfection however in the real world we have limited time and budget. Because of that it’s really important to adjust your testing to the given context and perform tests that will give as most value.

Below, I’ve listed just a few of the test types that you could conduct:

1. Manual vs automation – In manual testing, test cases are executed by a human without the use of tools, but with automated testing, tools, scripts and other software comes into play. Manual testing has easy to predict and linear cost. Test automation on the other hand has a high initial cost so should be considered for long term projects and tasks that are repeated multiple times.

2. Unit, integration, API, f/e – This involves the testing by app layer, to determine whether it meets requirements for functionality, reliability, performance and security. Not all of the possible layers and integrations between them have to be tested every time. A good tester will choose the most valuable tests for a given project.

3. Short modular tests vs e2e tests – The first of these, as the name suggests, focuses on testing individual modules separately, while the latter tests whether the flow of an application is performing as designed from start to finish.

4. Alpha and Beta testing – Alpha testing identifies bugs before releasing the product to real users or to the public, while ‘beta’ testing is performed by real clients who are not part of your company – this is usually a limited group of users as opposed to all your clients.

5. Exploratory vs scripted testing – Pre-made test scenarios assure that the testing process is reproducible. They help to avoid skipping important aspects, allow to measure coverage and compare results between test executions. In opposition to this approach we have exploratory testing that can provide great results in a short time.

Here are a few other situational types of tests that might be useful in some scenarios:

1. Load testing (performance, load, stress, volume testing) – this is done to understand the behaviour of an application under a given load.

2. Security / pentesting – This is sometimes known as ‘ethical hacking’ which is the practice of testing a system to understand its potential vulnerabilities, which could be exploited.

3. Localization testing – This is a test conducted to ensure that a product behaves according to the local culture or settings. It’s more than just a linguistic check and should cover other aspects like date/time preferences, or right to left writing.

4. Recovery and reliability testing – Recovery testing is conducted to see how well an application is able to recover from crashes, while reliability testing checks whether the software can perform a failure-free operation for a given period of time.

5. Usability testing – This is user-centred testing to check how your product performs when applied to real users. It can be performed by a tester and as an audit by specialised UX designers.

6. Requirements analysis – This is performed before development starts and can help you in finding issues such as inconsistencies or logic gaps that can be easily fixed at this point.

The risks you run by not conducting tests

• Loss of customers and revenue

Not catching bugs early can result in larger scale UX problems further down the line. Imagine you’re a banking client searching for a new account. You’ve just downloaded a bank’s app. Upon signing up, an error message pops up. What do you do? If you’re patient, you might try again a couple of hours later, but most likely you’ll go to the competition. Maintenance costs are lower in the end if a product works as it should from the outset.

Example in practice: In February 2018, a system error during a routine upgrade in Suncorp Bank caused the disappearance of money from customers’ bank accounts causing widespread panic and the need for crisis comms to be implemented.

• A crisis situation impacting your reputation

At the far end of the spectrum, small bugs can result in major system glitches which, depending on the nature of your business, can cause widespread disruption, customer distrust and huge revenue losses resulting from having to implement crisis communications and tidy up the resulting chaos. Frustratingly, most of the time, situations like this can be avoided if you invest in testing at the start of product development.

Example in practice: In 2018, British Airways had six major IT software failures which resulted in the cancellation of numerous local flights and significant delays in international flights. It took more than three days to resolve the issues. The airlines’ brand reputation was severely impacted.

• Poor product quality

Every product manager and owner sets out with the intention that they want it to be useful and to stand out to customers. It seems obvious, but it’s very important that your platform brings the value that your product promises. If it doesn’t do so – even if the core functionality works – then it won’t deliver the expected customer experience. Put simply, it just won’t show your brand in a favourable light. This is why testing can be the making or downfall of a fledgling start-up business, or a new product.

Example in practice: In 2012 Apple replaced Google Maps with Apple Maps on iOS 6 as they no longer wanted to be tied to Google’s application. However, in their rush to release the best map app on the market, they made some unforgivable lapses in navigation warranting the #ios6apocalypse hashtag. The error cost made customers question whether they should be switching to the very competitor that Apple was trying to combat.

• A rise in hidden costs

Solving bugs at the earliest possible stage reduces long term costs. We already wrote about the hidden costs involved in losing customers and in having to organize crisis communications when things go wrong. But some of the major hidden costs lie in the development time needed to fix issues further down the line. Additionally, if your system error is linked to data security, you might incur penalties for a breach of GDPR.

Example in practice: In October 2018, the Polish e-commerce website Morele.net suffered a data breach. The incident exposed almost 2.5 million unique email addresses alongside phone numbers, names and passwords stored as md5crypt hashes. The website was charged 2,8 million PLN for inadequate data security, a disaster which could have been avoided with adequate early testing.

Summary

While testing is extremely important when it comes to avoiding risks, keep in mind that you don’t have to perform all of the above tests. The types of tests that you run very much depend on the type of software that you’re building and other factors.

Want to read more about testing?
Take a look at our other recent articles on this subject:
Usability Testing: The Key to Design Validation
Why Test-Driven Development is Like Surgery

Are you looking for a team to work on your digital product?

We are great at what we do, and what we do is software development. From the above post, you’ll also know by now that we are excellent testers.
Get in touch with us on hello@10Clouds.com or visit www.10clouds.com. We have the expertise you need to make your product excel.