The Real Cost of Production Defects Equals 14x Increase
.svg)
Research data from the late 1990’s is still alive and well circulated with its share of detractors. Some claim the results are based on multiple variables and others claim the results only pertain to non-Agile development projects. For example, current numbers reported by Bradley Jones in Code Guru 'The Cost of Bugs' indicate that the average cost of fixing defects in the following phases:
Development $937.00
QA $7,136.00
Production $14,102.00
Based on these figures, it is 14 times more expensive to fix production bugs. These numbers don’t measure the impact on business reputation or loss based on negative end user experience.
Re-work costs for resources, scheduling, planning
Regardless of whether Agile or non-Agile methodology is used costs still increase the longer a defect resides in the code. Defects don’t age well in that the severity impact of the bug is greater the longer it resides in the code. Costs increase because developers have to re-code, re-check in and rebuild a release. The release has to be re-deployed to multiple test servers, re-tested and re-deployed to production. Additionally, all the automated unit and regression tests are re-executed and any other manual tests re-executed. It’s all time, money and customer impact.
If the defect is severe or causes significant re-design then the architects, product managers and documentation resources are also part of the re-work.
The endless circle – quick fix in production yields more defects
Production defects tend to replicate as well. For example, the development team makes a quick fix for a simple defect found by customer support immediately after a release. No problem, simple code fix, re-build, re-deploy, and re-test. Then, the QA testers find additional defects caused by the quick fix to the code. Now, upper management gets involved to pull back the release or re-fix, re-deploy, and re-test. It’s a continuous loop and it’s expensive.
The exact development phase the defect is found is irrelevant. The important part is that defects are found prior to deployment into production. The faster they are found and fixed, the less continuous looping occurs after a release and the more new development work is done. If the entire development team is chasing and fixing defects continuously there’s little time to design and build new code.