In manufacturing circles it is generally accepted that inspections are money wasted. But in the world of software that is definitely not the case. This brief case study uses scorecards to illustrate the value and payoff that can be realized by introducing software inspections. By comparing defect counts and defect fix efforts from two completed software projects, we will utilize a scorecard to summarize and highlight differences in actual defect counts by development phase of origin and the phase in which the defect was detected.

The scorecard below, typically applied in the Analyze phase of a Six Sigma project, calculates the yield values for each phase given a software project with 120,000 source lines of code (SLOC). In software we refer to these yields as phase containment effectiveness (PCE – the percentage of errors detected during the phase in which they were introduced). Those that escape the current phase are considered defects – we then calculate defect containment effectiveness (DCE – the percent of the defects escaping an earlier phase that are detected in the current phase).

The first project, Concorde, (see Figure 1) shows PCEs less that 40 percent – far below industry best practice results in the 70 percent to 80 percent range. The Concorde project did not use formal Fagan style software inspections.

(Click Figure To Enlarge)

Notice that Concorde total containment effectiveness (TCE) – comparable to rolled throughput yield in Six Sigma manufacturing terminology – is less than 86 percent, again far less than industry best practices can produce. This means that more than 14 percent of the total defects will be delivered to the customer.

In contrast, the second project, Topaz, achieved PCEs and DCEs in the 60 percent to 70 percent range, and TCE of about 94 percent. Topaz, in other respects very similar to Concorde, did utilize formal Fagan style software inspections.

What if inspections had been applied to Concorde? Using an extended version of our scorecard that includes time and cost to fix (derived from the historical data) we can create a “to-be” scenario for Concorde that will answer this question, as illustrated in the figure below. This scorecard will typically be used during the Improve phase of a Six Sigma project.

When we apply the inspections process to Concorde, reflected by the Topaz PCEs and DCEs, we reduce defect fix cost from 39 percent of total development effort in the actual project to 19 percent in the “to-be” state. Doing the inspections will cost in the area of 8 percent of total effort in the “to-be” state, so we can realistically expect a net gain (cost reduction) of about 12 percent of total project effort.

Training and infrastructure development necessary to implement inspections will require perhaps 3-6 months and will cost approximately US$200,000. In return we will save at least that amount on the next project and on every project thereafter. Clearly, inspections are not a dirty word when it comes to software development.