Be Consistent in Six Sigma Project Metric Selection

	Bookmark This Page Email This Page Format for Printing Cite This Article Submit an Article Read More Articles
	Related Tools & Articles Selection of Project Metrics Time-based Metrics: Accounting for the Differences Tips for Avoiding Common Metrics Challenges
	Discussion Forum "Other than DSO, is there any other good metric in debt collection that can be improved using Six Sigma? Are discrete metrics better in debt collection or continuous metrics?" Contribute to this Discussion

Download Products Project Selection Process Pack Six Sigma Yellow Belt Training Slides - 2009 Version Six Sigma Leadership Series

By Michael Ohler

Six Sigma project metrics are typically taught in detail during Black Belt and Green Belt training classes. But the inconsistent use of these metrics after training can lead to lost opportunities and rework after a project ends. Using project metrics consistently brings enormous clarity to the impact and benefit of a Six Sigma project and helps keep project teams focused on enhancing business capabilities.

Defining Project Metrics

One common mistake in the use of project metrics is to phrase the project goal to realize a certain saving in a given area, rather than a process improvement. This confuses the primary metric with financial metrics. Another inconsistent use of metrics is found among the secondary metrics, which are sometimes misunderstood as representing the project Champion’s extended wish list. For example, an early version of my first Black Belt project charter stated the primary metric as improving delivery speed. Delivery quality and hours worked were considered secondary metrics with improvement targets of their own, rather than side effects of the primary metric.

Table 1 summarizes the standard Six Sigma project metrics, their meaning, their relation to the primary metric and their use in improvement projects.

Table 1: Six Sigma Project Metrics
Project Metric	Definition	Correlation to Primary Metric	Use in Improvement Project
Primary Metric	Defines the project goal: "improve (primary metric) from (baseline) to (target) by (date)"	N/A	Measure baseline and improvement level when the project ends
Secondary Metric	Captures, validates and tracks welcome side effects of the project	Strong (high relative predictive power, or R2). By default, there is a cause-and-effect relationship between the primary and secondary metrics.	Measure baseline and impacts of project after improvement. Monitor during and after project if linked to financial metric.
Consequential Metric	Captures, validates and tracks non-welcome side effects of the project	Strong (high R2). In the current design of the process, there is a cause-and-effect relationship between the primary and consequential metrics.	Collect data before, during and after the project to prove no collateral damage was caused by the project.
Financial Metric	Links progress in the primary and secondary metrics to financial advantage. Most often this metric is tailor-made for the specific project	Expected strong (high R2).	Evaluate at project milestones (i.e., Define, Measure, Analyze, Control) and typically one year after the project ends.
Business Metric	Measures how an organization achieves one of its major goals.	Typically weak (low R2).	Monitoring is optional and mainly needed for communication purposes.

Selecting the Primary Metric

Selecting the appropriate primary metric for an improvement task is the result of a sound process to identify project scope. A well-scoped project focuses on only one primary metric while considering the impact of the other types of metrics. One way to identify the right primary metric for the project is to use a balanced scorecard approach.

A sound project statement is phrased using the primary metric. For example, “The project goal is to improve (primary metric) from (current performance) to (desired future performance) by (desired date of completion).”

Many projects begin with an objective such as, “Our document archiving does not work properly and must be improved.” It is important to impose the necessary discipline on project Champions and process owners to transform such vaguely phrased problem statements into proper project statements.

Connection to the Business Metrics

Why is an improvement in the primary metric good for business? Why, for example, are faster deliveries better? The relation between the primary metric and the underlying business metric answers this question. Faster deliveries may increase customer satisfaction, which is one of the top-level goals of an organization. Thus, there is a known cause-and-effect relationship between the primary and business metrics.

However, a business metric such as customer satisfaction is influenced by many factors other than faster deliveries. So, most often the strength of the correlation between the primary and business metric is weak and the regression coefficient R2 is typically small and difficult to measure. When the average lead time for delivery is improved from five days to three days, many deliveries may still last longer than desired. This is a case of achieving a primary metric goal while leaving the business need unsatisfied.

For business metrics such as customer satisfaction, indirect statistical methods, including Taguchi’s, can provide a useful roadmap. Notice that a project can impact several business metrics correlated to the primary metric or to any of the secondary metrics. For practical reasons a project typically quantifies the relationship to only one business metric. This helps the project team understand and focus on a single organizational goal.

Benefit of Using a Secondary Metric

Secondary metrics quantify welcome side effects caused by improvements in the primary metric and are typically linked to the primary metric by default. Reducing the lead time to issue an offer, for example also reduces the number of offers in progress at any given moment.

Often secondary metrics measure progress in areas not directly affected by the project. Validating and tracking these metrics can build broad-based support for the project. To validate a secondary metric, quantify its cause-and-effect relationship to the primary metric and ensure its relationship to a financial or business metric is transparent.

Impact of Consequential Metrics

This situation can be illustrated with an example of a project that aims to issue faster offers. In the current design of the process, quickly issuing an offer degrades the quality of the offer. A non-diligent preparation of offers produces defects, which degrade the offer-to-project conversion rate (because the offer is overpriced) or the project margin (because the offer is underpriced). In his book It’s Not Luck (North River Press, 1994) Eliyahu Goldratt introduced the “evaporating cloud,” a way of dealing with seemingly mutually exclusive goals – such as speed and quality – in a project Writing down the conflict and the links between the statements allows practitioners to identify ways to break one or more of the links. Using the evaporating cloud method may push the project team to question whether it is true that offers prepared quickly cannot be offers prepared diligently. Asking questions such as this can lead the team to invent new design options for the process rather than simply improving the existing process.

It is a good idea to establish a measurement system for consequential metrics during the Measure phase of a project. Tracking these metrics after project finalization will provide ongoing information on whether the previously existing link between the primary and consequential metrics is broken in a sustainable way.

Importance of Financial Metrics

Evaluating the benefits of a project includes establishing a financial metric for improvements realized by the project. Saving money is not always the primary interest of Lean Six Sigma projects. When the primary metric is tied to a business metric, it is often enough to know that the project enhanced the company’s business capability. Even so, it makes sense for organizations to have a monitoring system in place for the results of their continuous improvement activities across all areas. The financial metric is not used for project accounting but rather to translate the project impact into a generalized business benefit.

The financial metric of the project should be reassessed at all project tollgates. Many companies also continue to validate this metric one year after project finalization.

Six Final Tips

From these considerations arise the following recommendations for Six Sigma project managers, Master Black Belts, process owners and project Champions:

• Derive one primary metric for the project team to work on for each improvement project.
• Use the primary metric to formulate the project goal.
• Brainstorm for possible secondary and consequential metrics. Select and validate a set of few essential metrics.
• Link at least the primary and consequential metrics, and eventually also the secondary metrics, to business metrics. Validate and quantify the cause-and effect relationships.
• Construct a financial metric based on the primary and secondary metrics.
• Make sure all metrics fulfill the requirements to facilitate the creation of a data collection plan.

About the Author: Michael Ohler is a certified Master Black Belt with experience in several industries. He also has Lean and Six Sigma training experience at the Yellow, Green and Black Belt levels and holds a doctorate in experimental physics. Ohler has been teaching university courses in France and Brazil. He can be reached at ohlermichael@googlemail.com.

"The Bottom Line" Links

BEST SELLING PRODUCTS (iSixSigma Publications) Six Sigma Black Belt (DMAIC) Training Slides - 2009 Version! The 2009 Six Sigma Black Belt course includes over 40 more slides than the 2008 version. Contents include: 1,220 PowerPo... Certified Lean Six Sigma Black Belt Assessment Exam Interested in assessing your knowledge of Lean Six Sigma? Preparing for certifications? Testing your students and traine... Certified Lean Six Sigma Green Belt Assessment Exam This assessment exam is useful for students interested in assessing their knowledge of Lean Six Sigma on the Green Belt ... Certified Lean Six Sigma Black Belt E-book In 670 pages learn everything within the Lean Six Sigma DMAIC body of knowledge to successfully achieve Black Belt certi... Kaizen Workshop E-book This 150+ page ebook teaches key tools and techniques of Kaizen, as well as real application to enhance learning. Kaizen... Six Sigma Yellow Belt Training Slides - 2009 Version The 2009 Six Sigma Yellow Belt course is comprised of: 503 slidesInstructor notesSlide explanations15 data sets19 suppo... Design For Six Sigma (DFSS) E-Book or Print Need an "encyclopedia" consisting of many of the tools you’ll study? Need a helpful refresher to apply the DFSS process?... Sell Your Products