Every corporation provides critical services through its applications, meaning they cannot suffer any disruption in their operation. Software testing is responsible for optimising the management of these resources in order to anticipate possible failures that may arise for end users. Tests can be carried out by developers or by specialised professionals like Quality Assurance (QA) specialists.

Let's find out more about software testing and its role in the success of a product.

Levels of software testing

According to the International Software Testing Qualifications Board (ISTQB),  levels of testing are groups of testing activities organised and managed together, with each level being equivalent to an instance of the testing process.

There are the following levels of testing:

• Unit testsThey check the behaviour of the smallest part of an application's testable code, regardless of the interaction between other parts.
• Integration testsThey test the interaction between units, how they work together, and usually test an entire functionality, including data persistence.
• System testsThey focus on the behaviour and capacity of the system or product. They involve carrying out end-to-end tests, ranging from interaction with user interfaces to data processing in the back-end.They also cover non-functional behaviour. User Interface (UI) tests can test both negative and positive scenarios. In negative scenarios, they sometimes go as far as the interface layer and don't interact with back-end processing. In positive scenarios, interaction can take place with “mocked” or simulated data from the back-end, focusing more on the interfaces. If interaction does occur, they are characterised as end-to-end.
• Acceptance testsThey also consider the behaviour of the system as a whole and produce information to validate the situation of the system being implemented and used by the end user. They can also fulfil legal and regulatory standards. This type of testing is subdivided into:
  • User Acceptance Test (UAT): a version of the system is made available for the user to execute the business processes and check that it is possible to execute them with the minimum of difficulty, cost and risk.
  • Operation Acceptance Test (OAT): this is carried out in an environment controlled by the administration team and covers backup and recovery tests, user management, task maintenance, vulnerability, security and performance.
  • Regulatory and contract acceptance tests: based on contract acceptance criteria for specific software development.
• Alpha and beta testsThese are similar to acceptance tests and also collect information about the state of the system, but they are not planned.

Types of software testing

The types of testing represent a group of activities focused on testing specific characteristics of the system, or part of the system based on its objectives. They are subdivided into:

• Functional testsThey test whether the system does what it is supposed to do (business specifications), meaning they test the functionality of the software. Requirements can be written down in the form of documents, user stories, use cases, or not documented at all in the worst case scenario.
• Non-functional testsThese are broader and valid for all systems, as they are based on the ISO-25010 standard, which deals with usability, performance, efficiency and security.
• White box testsThey are based on the internal structure and have access to the source code. Unit tests are white box tests.
• Black box testsThese are based on the basic requirements of the software and focus on the actions that these requirements should fulfil. System, acceptance, alpha and beta tests are of the black box type. This type of testing checks the output data without worrying about how it is generated internally – for example: date of birth field filled in with future dates and mandatory fields without validation.

The greater the visibility of the code, the greater the visibility of how the feature is made, the smaller the quantities of requirements and business specifications tested, and the greater the transparency of the test, approaching a white box test. On the other hand, the lower the visibility of the code, the lower the visibility of how the feature is made, the higher the quantities of requirements and business specifications, and the lower the transparency of the code, gradually approaching a black box test. The image below illustrates this idea.

Software testing methods

Any of the levels and types of software testing mentioned above can be carried out in two ways:

• Manual testingTests are performed manually by a human (Quality Assurance Specialist) in order to cover exploratory tests (finding errors at random), usability tests, or test cases written in the acceptance criteria of a user story, for example.This method has advantages and disadvantages:
  • Pros
    • The conditions of these tests are similar when the system is in a production environment.
    • They can identify visual faults.
    • Low cost.
    • Less technical knowledge needed to carry them out.
  • Cons
    • Slower to run.
    • Higher chances of not detecting a fault because they are not automated.
• Automated testingTests are written to be processed automatically. They require tools, scripts and software to be implemented and executed.
  • Pros
    • Greater reliability.
    • Faster execution than manual tests.
    • Excellent for load testing and performance testing.
  • Cons
    • Time-consuming development.
    • Specific tools for each testing phase needed.
    • May not detect usability problems.
    • Extra infrastructure investment may be required in some cases.

Software testing flow

Each type of test can be applied in different environments, where the applications are installed, and in different development phases according to the infrastructure, needs and budget of each project/application, as shown in the image below.

Software testing pyramid

Usually, the amount of unit tests developed is relatively greater than that of integration tests (also called service tests) and UI tests (included in the level of system tests), because they cost less, are less specific and are quicker to develop.

This logic is illustrated by the software testing pyramid:

Conclusions

Among the most common software tests developed are unit tests, integration tests and sometimes UI tests. Different levels of testing are created, taking into account their particularities:

• Unit tests to guarantee the expected behaviour of the smallest units of the system.
• Integration tests to guarantee the behaviour of an entire feature.
• UI tests, being closer to the end user's experience, increase the application's reliability, quality and stability in production environments, and reduce the costs caused by faults that can be detected before reaching a production environment.

In order to deliver a high-quality product at a low cost and with little time invested, a combination of tests should be considered, considering that they can be carried out manually or automatically. However, the types of tests and quantity to be developed depend on the application's scalability, budget, project time and product needs.