Is Software Inspection Value Added?By Gary A. Gack
In manufacturing circles it is generally accepted that inspections are money wasted. But in the world of software that is definitely not the case. This brief case study uses scorecards to illustrate the value and payoff that can be realized by introducing software inspections. By comparing defect counts and defect fix efforts from two completed software projects, we will utilize a scorecard to summarize and highlight differences in actual defect counts by development phase of origin and the phase in which the defect was detected.
The scorecard below, typically applied in the Analyze phase of a Six Sigma project, calculates the yield values for each phase given a software project with 120,000 Source Lines Of Code (SLOC). In software we refer to these yields as Phase Containment Effectiveness (PCE - the % of errors detected during the phase in which they were introduced). Those that escape the current phase are considered defects - we then calculate Defect Containment Effectiveness (DCE - the percent of the defects escaping an earlier phase that are detected in the current phase).
The first project, "Concorde", (see table below) shows PCEs less that 40% - far below industry best practice results in the 70-80% range. The Concorde project did not use formal Fagan style software inspections.
(Click Figure To Enlarge)
Notice that Concorde Total Containment Effectiveness (TCE) - comparable to rolled throughput yield in Six Sigma manufacturing terminology - is less than 86%, again far less than industry best practices can produce. This means that more than 14% of the total defects will be delivered to the customer.
In contrast, the second project, "Topaz", achieved PCEs and DCEs in the 60-70% range, and TCE of about 94%. Topaz, in other respects very similar to Concorde, did utilize formal Fagan style software inspections.
What if inspections had been applied to Concorde? Using an extended version of our scorecard that includes time and cost to fix (derived from the historical data) we can create a "to-be" scenario for Concorde that will answer this question, as illustrated in the figure below. This scorecard will typically be used during the Improve phase of a Six Sigma Project.
(Click Figure To Enlarge)
When we apply the inspections process to Concorde, reflected by the Topaz PCEs and DCEs, we reduce Defect Fix cost from 39% of total development effort in the actual project to 19% in the "to-be" state. Doing the inspections will cost in the area of 8% of total effort in the "to-be" state, so we can realistically expect a net gain (cost reduction) of about 12% of total project effort.
Training and infrastructure development necessary to implement inspections will require perhaps 3-6 months and will cost approximately US$200,000. In return we will save at least that amount on the next project and on every project thereafter. Clearly, inspections are not a dirty word when it comes to software development.
About The Author
Gary Gack is a Partner in Six Sigma Advantage, LLC, based in Narragansett, Rhode Island. Mr. Gack holds an MBA from the Wharton School, is a PMI Certified Project Management Professional and an ASQ Certified Software Quality Engineer. During his 40-year career in the software and IT industry he has managed a variety of large scale software projects and has consulted with dozens of Fortune 1000 firms. For more information on software inspections contact Gary Gack at (401) 782-8323, email to ggack@6siga.com, or visit www.6siga.com.
