One of the most frequent questions asked by software groups new to Six Sigma is something like, “What types of problems should we address with Six Sigma?” While there is no one-size-fits-all answer to this sort of question, there are criteria that are universally applicable and provide a project selection framework that covers general top-level considerations.

The first rule is, “Don’t pan for gold in your hot tub!” Go where the money is. Six Sigma is about significant step changes in organizational effectiveness, and those changes can happen only if the methodology is applied to significant problems that consume an important share of organizational resources. In software, rework is nearly always the biggest single cost element – commonly 40 percent of total effort. Hence, rework is a good place to focus Six Sigma. Appraisal (quality assurance, testing) is most often the second largest element of cost and is therefore another good target for Six Sigma.

Considerations for Project Selection

Other considerations that should strongly influence Six Sigma project selection in software include:

  • Replication Potential – Ideal are projects that can be used to prove an approach or solution concept in a limited pilot area and then be rolled out to a much larger population to multiply the benefits.
  • Measurability – Consider projects in which it is feasible and affordable to use or create measurement data that can quantitatively demonstrate the business case. This means selecting areas where baseline data can be collected to enable pre- and post-improvement comparison.
  • Local Benefit – There is an advantage in implementing projects that can be expected to yield meaningful benefit to the project team involved, rather than only to a “downstream” unit. For example, a quality improvement that is costly to development in order to benefit the support organization tends to be a tough sell.
  • Employee Acceptance – Few in software are receptive to measurement, which is often seen as bureaucratic overhead. Hence, ease of use and efficiency considerations play an important part. Working in a cooperative setting can be critical to success. Mandating Six Sigma is often ineffective; look for interested volunteers, and think about incentives.
  • Realistic Resource Allocation – Rarely is it possible to realize meaningful improvement without some investment. Hence, if a development team is going to work on a Six Sigma project, it must be given time to do so. If a team cannot be given some “slack,” either by assigning additional resources or reducing delivery expectations, it is unlikely the team can successfully execute a Six Sigma project. In general, a software Six Sigma team working on a significant project will require 25 to 50 percent of an experienced Black Belt, perhaps 20 percent of a Green Belt and 10 percent of one or more Yellow Belts. While these figures are based on experience, they, of course, can vary by project.
  • Appropriate Scope – Many problems are not appropriate targets for Six Sigma – not every project is a Six Sigma project. If the problem is local and can be resolved by assigning a skilled resource (e.g., improve response time of a specific system), it is not a Six Sigma project. Just do it. A problem that concerns communication between two related groups (e.g., handoff of adequate documentation from development to support) is typically addressed by forming a short-term task force to agree to requirements and process. Often it does not need to be a Six Sigma project. Problems appropriate for Six Sigma are: recurring, important to the organization (e.g., cost and schedule overruns, poor delivered quality), those for which the solution is not obvious, and those that are capable of being scoped to enable demonstration (pilot) of a solution in a four- to six-month time frame. DFSS projects are often even longer.
  • Think Long Term – For example, suppose a company has 1,000 developers who spend 60 to 70 percent of their time in appraisal (quality assurance/testing) and rework. (That may sound like a lot, but it applies to 80 percent of software groups.) Now, suppose the CEO wants to reduce time in those areas to 50 percent during the next three years. If, on average, a developer costs $100,000 per year fully loaded, that means the company’s total annual development budget is $100 million per year. Moving to 50 percent appraisal and rework means shifting $10 million to $20 million from non-valued-added work to value-added during a three-year period. A single Six Sigma Black Belt project, on average, will deliver at least $200,000 in net benefits, which in turn means you will need to do something like 50 to 100 projects, perhaps 15 to 30 per year (requiring as many as eight experienced Black Belts). A well-thought-out sequencing strategy is important, and usually leads to giving priority to projects that will create a measurement foundation that enables later projects.

Choosing Projects from Clusters

Projects are often chosen from “clusters” that can be associated with major process areas important to software development. Among the most commonly identified clusters are: requirements, estimating, defect containment, and project/program management. The following table gives examples of frequently occurring problems associated with these clusters. Successful Six Sigma projects have focused on each of these examples.

Table 1: Frequently Occurring Problems Associated with Clusters
Cluster Sample Problem Statements
Requirements Inter-dependencies between projects often cause unplanned and late changes to requirements, leading to delays and budget overruns.


Schedule and cost estimates are not adjusted for impact of scope changes. Hence, development believes it delivers on schedule and budget, but customers believe they did not get the agreed-on scope. This situation leads to many post-delivery “enhancements.”


Acceptance criteria are not established during requirements development, leading to disputes between customers and development when the system is delivered.


Insufficient detail in requirements is believed to lead to estimating errors and disputes with customers related to different perceptions of what is to be delivered. Missed or misunderstood requirements cause many post-delivery “enhancements.”

Estimating Company experiences a wide variation in actual versus estimated performance – 30 percent of projects have a variance > 50 percent. This causes major budget problems for both development and customers.
Defect Containment During the first year after delivery, company incurs maintenance costs equal to approximately 80 percent of development costs, which reduces effective development capacity.


It is unclear how much testing is enough – company does not know the impact of more or less testing of specific types (e.g., unit, integration, system, stress, etc.).


Company does not know the differential effectiveness of alternative appraisal methods with respect to finding of particular defect types.

Project/Program Management The current life cycle methodology is perceived to require too much overhead, which may contribute to longer than necessary cycle times for development.


Existing key performance indicators (KPIs) do not appear to accurately forecast outcomes and may create counter-productive incentives.


Frequent interruptions are believed to have a significant impact on time on task and may reduce overall productivity.

In a larger organization (more than 1,000 developers), it is often beneficial to concentrate a number of projects on one or a few clusters in order to get a large sample size. That will lead to a higher level of statistical confidence in the results. A shotgun approach that does a few projects in many different areas tends to be less convincing and may have a higher risk of failure.

Project Selection Process

A project selection process often unfolds in the following steps:

  1. Identify clusters.
  2. Identify candidate problem statements in each cluster, typically in discussion with opinion leaders, sponsors and Champions.
  3. Complete a project selection scorecard (Table 2). Evaluate both the potential business benefit associated with a particular problem statement and the comparative fit of potential teams to which the project could be assigned.
  4. Assign projects with highest overall scores to most appropriate teams.
Table 2: Project Selection Scorecard
(Scoring: 9 = Strong/High 3 = Moderate 1 = Weak/Low)

Candidates

Project Characteristics

A

B

C

D

E

Replicaion potential          
Easy to measure          
Benefits to project/area (local benefit)          
Eliminate waste/rework          
Proven best practice in the industry          
Project can be completed in4-6 months/early benefit realization          
Employee satisfaction/acceptance          

Total Score

         

Team Characteristics

A

B

C

D

E

Black Belt candidate has sufficient knowledge of the area          
Team aceptance of Black Belt is high          
Team leader and Green Belt are volunteers (not mandated)          
Life cycle phase is appropriate to SixSigma topic          
Time can be made available to do project work          
Schedule pressure is reasonable          

Total Score

         

Best results are achieved when the number of candidate projects exceeds the number of Belts available to lead the projects. The project “job jar” is frequently replenished and priorities are re-evaluated as project resources become available.

About the Author