My colleagues were working on a complex and multi-component application. This project had very tight deadlines. However, due to the complex QA process, there was a risk that the team would not meet the deadline. Abandoning the project when work on it had already begun was a very bad option. Accepting and doing the project, but with a delay, was also not the best decision. As a result, the team decided to rethink the QA process as much as possible, focusing on… metrics. The hypothesis was that this way they would be able to see all the bottlenecks in testing in time and take the necessary measures to quickly solve problems. Moreover, the metrics were supposed to help in optimizing the time for testing. Frankly, I did not believe in their success. What was my surprise when the project was completed on time, and without any comments on its quality!
Turns out, it is difficult or even impossible to control what cannot be measured. For this simple reason, the quality of testing also needs to be measured. I asked my colleagues to share their experiences in concern about what the QA metrics are and what they help to find out. Read the article below to know.
Why do we need to calculate QA metrics?
QA metrics give a visual picture of the common and personal results. Thanks to metrics, you can see the results of your team’s work, expressed in numbers, the strengths and weaknesses of your QA process. In addition, it would be difficult for you to understand the effectiveness of your test automation if you do not calculate special metrics. This allows efforts to be adjusted by directing more resources to address key issues.
Using QA metrics you track:
- Team progress by deadlines and iterations.
- Quality control of the testing process’s current state.
- Critical issues. They can be addressed through more efficient resource allocation.
There are a lot of metrics. Each team chooses the most suitable for itself. However, in terms of the approach to counting metrics, the universal advice is to choose the right tool. A robust and multifunctional Zebrunner test automation management platform makes it easy for you to work with metrics. All you need is to add the necessary widgets to the dashboard.
Categories of QA metrics
There is no point in counting all possible metrics. During the testing and control planning phase, choose the metrics that are most important to you and monitor them regularly, as the experienced QA team recommends. If you see that any of the indicators are falling, take measures to correct the situation.
To begin with, let’s figure out what categories the metrics in quality assurance are divided into. There are four groups in total. These are product, project, process, and personal metrics.
Product metrics track the total number of test cases, how many of them were successful, and how many failed. In addition, product metrics include an analysis of identified defects, their severity, and their types.
Number/percentage of test cases passed/failed. It gives us information about what number (or percentage) of test cases are completed successfully or unsuccessfully.
Number of defects found. Usually, this metric is tied to a certain period. A large number of defects indicates that the developers do not do their part of the work qualitatively.
Project metrics show how the testing workflow state, its success, and duration.
Number of planned/actual test hours. This indicator allows you to understand how your expectations for the time spent on tests correspond to the real state of affairs.
Executed test percentage. The higher the percentage, the closer the team is to the QA process final.
Total test duration. This metric shows how much time it takes to execute all tests and how long it takes to run a single test. It is clear that if the tests last for a long time, this will affect the timing of the project and/or the quality of the developed product.
Test process metrics
As a rule, these are performance metrics. They evaluate test coverage, the efficiency of the case creation process, the average time to find a bug, and more. Process metrics include the effectiveness of test automation as well as the development process.
Test coverage. The percentage of requirements that are covered by tests.
A number of tests run per period. The average time to pass the test.
Bug find rate. The metric shows how long it takes on average to find one bug.
Personal performance QA metrics
Personal metrics measure the performance of a particular person. Most of the metrics from the Project and Process categories can be transferred to a specific team member and measure its performance and effectiveness.Executed tests by a person. The metric shows how many tests a particular tester has completed.