Software quality assurance is vital because poor-quality software risks higher development costs, unhappy customers and delayed time to market. Defect leakage is one of the metrics used to assess software quality, so here is what CIOs need to know about it to enhance their software development outcomes.
Building software has many similarities to more tangible projects, like constructing a bridge. It’s much easier, safer and less costly to spot and fix an issue at the design stage than it is once construction has begun.
In the world of software development, issues are referred to as ‘defects’. Although developers, testers, and quality assurance teams test assiduously for defects and work hard to prevent them, it is inevitable that some will leak out from the design phase into production.
This is called ‘Defect Leakage’.
CIOs need to manage defect leakage closely because as the preeminent software engineer Barry Boehm demonstrates in his ‘Cost of Change Curve’, the cost of fixing defects increases exponentially as they move through the software development lifecycle.
Let’s consider what needs to happen when there is a production defect:
The most recent example is that of the massive IT outage caused by a software update pushed out by a well-known cyber security vendor that crashed millions of Windows systems and disrupted critical services and business operations globally.
Another real-world example is where a major UK bank experienced an IT system failure that prevented over 6.5 million customers from accessing their accounts. What began as a simple software update ended in £54m of fines from regulators.
This isn’t just an issue for major firms. No software team wants to be working on unplanned emergency fixes. It burns time that could be used more productively and can have a negative effect on morale.
It’s essential to produce high-quality outputs and make continuous efforts to improve, which is why Defect Leakage Ratio is one of several key metrics to monitor closely when striving for excellence.
The aim with software quality assurance is to keep defect leakage as near to zero as possible, which is measured using the Defect Leakage Ratio and should be monitored consistently over time.
For example, at NashTech we calculate Defect Leakage Ratio (DLR) using the formula: DLR = Defects found after release to customer / development effort.
Although it is difficult to catch every defect, there are practical steps to reduce the number that slip through the net.
Factors that contribute to a high defect leakage ratio include insufficient testing, unclear requirements, or human error. However, there are steps you can take to determine a root cause and reduce the number of software defects that occur because of them:
Root cause analysis helps to get to the bottom of why these defects are happening. Are these the result of unclear requirements or simply coding errors? Identifying limitations in the development process and remedying them helps to prevent such defects occurring in the future.
Better testing means a greater likelihood of catching defects before they escape into a live environment. Increasing test coverage (the industry standard is 70 – 80%) and applying more rigorous methods that address functional and non-functional requirements can help. For example, unit testing, integration testing and user acceptance testing are all ways to catch defects earlier, while applying automation can enhance testing processes substantially and reduce the potential for human error.
The probability of defects occurring increases when requirements are unclear or incomplete. Review your requirements gathering process. Ensure the full scope is well-defined and fully understood before development begins.
CIOs must be proactive in understanding software quality and the critical steps needed to achieve it. At NashTech, we understand what it takes to build software development processes that achieve exceptional levels of quality. Speak to us today.
Want more insights on how to reach great levels of software quality? Download our eBook.