A Six Sigma Case Study-Tutorial for IT Call Center – Part 1

By Gary A. Gack and David L. Hallowell

IT services is a competitive field populated with companies that all deliver important online and call center support to a variety of customers. Most IT services businesses come to realize that their clients have choices and, within the same pricing range, they gravitate to the support organization where the service is best.

In this case study of an IT services business, benchmarking helped quantify what the business already knew – its competitive position was not totally secure. There are a number of ways the company might have responded to the challenge. While the company had built up a reasonable capability in Six Sigma, its management realized improvement was not as simple as forming a project team and turning them loose on the problem. Senior managers had learned that an important part of their responsibility as leaders is to find the issues that are well-enough defined and of a scope to be suitable for a Six Sigma DMAIC project team to take on.

After working through the benchmarks and other data and with the help of a Black Belt, they were able to distill enough clues and evidence in the top-level industry figures to select a DMAIC project they could sponsor with facts and supporting data.

Customer Satisfaction and Business Growth

Industry data was purchased from a clearinghouse that gathers a number of measures about customer satisfaction and call center technical and business performance. Comparing their company to the benchmark average and to a select best-in-class group, the company’s management team could see that customer satisfaction with their support services (gathered by an unbiased industry source) was just average or a bit below.

Figure 1: Customer Satisfaction for the Company, 2001-2003

Figure 2: Customer Satisfaction for Average Companies, 2001-2003

Figure 3: Customer Satisfaction for Best-in-Class Companies, 2001-2003

comparison of the company’s customer satisfaction ratings (73 percent on a 100 percent standardized score), with the “average” companies in the same sector (76 percent) and “best-in-class” competitors (87 percent) showed management it had work to do.

The evidence also supported the important business contention that customer satisfaction (CSat) can be a driver of new account growth. Figure 4 illustrates that the range of customer satisfaction ratings for best-in-class competitors tracked with about 75 percent of the changes in new account growth. That was evidenced by the R-sq. value in the linear regression that was plotted. Senior managers knew that the relationship didn’t “prove causality” but, together with their business sense, they saw this as an indicator that customer satisfaction shows up on the bottom line.

Figure 4: Relationship of Customer Satisfaction Ratings and New Account Growth in Best-in-Class Companies

Support Costs Per Call, Per Client

The benchmark data indicated customer satisfaction and business growth do not have a direct relationship to support costs per call. So the companies with the best customer satisfaction and best business growth do not spend the most on support costs per call. In fact, the support costs of $26 per call for the best companies and $30 for the average are lower than the case study company’s cost per call of about $36 (Figure 5).

A model was built to check the feasibility of focusing a DMAIC project on call center service measures (Figure 6). In the figure, the Y, or NewAcct, is new account growth during the benchmark period (as a percent of sales). The Xs are:
Transfer = Average number of transfers (to different agents and help systems) during a service call.
Wait Time = Average wait time during a service call.
Service = Average service time during the call (the time spent getting the answer to the question, problem solving advice, etc.).

Obviously the company would like to have seen a better model-fit than the 62 percent R-Sq seen here. Realizing, though, that many factors play into account growth, the senior leadership felt that the model showed enough linkage to the process factors that pursuit of the project was feasible. Since the company’s senior managers had ready access to wait time benchmark data, they checked their company’s performance against the industry (Figures 7, 8 and 9).

The wait time review indicated that, indeed, the company was behind the industry norms. This, and the model indication that wait time could be an influential factor in customer satisfaction and new account growth (Figure 5), helped the senior managers see that a DMAIC team focused on improvement in this area could be worthwhile.

Figure 5: Support Costs Per Call

Figure 6: Model Characterizing the Influence of Call Center

Figure 7: Call Wait Times for the Company (Median 4.5)

Figure 8: Call Wait Times for Average Companies (Median 4.0)

Figure 9: Call Wait Times for Best-in-Class Companies (Median 1.6)

The company could see a strong indication that a DMAIC project to reduce support costs should be quite doable – and should return significant dollars to the bottom line. Management also could see that the DMAIC team should look for the improved customer experience connected with reduced wait times and service times to improve new account growth – bringing dollars to the top line.

The company assigned a Champion from its leadership team to take responsibility for the new project and identify a team leader and key team members. The team was given its top level goals and scope – to reduce support costs while improving new account growth. The work with the benchmark data was helpful in orienting the team to the project rationale. The team began working on their project charter.

The Define Phase

The senior leadership of the IT services company completed the important pre-project work and found an area of the business worthy of attention by a DMAIC project team. The team then began work on understanding and articulating the project goals, scope and business case.

The DMAIC roadmap called for work in these areas during the Define phase:

D1. Project Charter: Spelling out the project’s goal statement.

D2. Customer Requirements: Identifying all the internal and external customers who depend on the outputs of the process under study, the deliverables and measures connected with those outputs, and the process steps, process inputs and (as appropriate) the suppliers of those inputs.

D3. High Level Process Map: Showing the flow of information, materials and resources, from key process inputs, through process steps and decision points, to create the process outputs. The map describes the flow of what happens within the scope of the target process and it defines the boundaries of that scope.

D1. Project Charter

Here are some of the key elements of the project charter.

Problem Statement: “Competitors are growing their levels of satisfaction with support customers, and they are growing their businesses while reducing support costs per call. Our support costs per call have been level or rising over the past 18 months, and our customer satisfaction ratings are at or below average. Unless we stop – or better, reverse this trend – we are likely to see compounded business erosion over the next 18 months.”

Business Case: “Increasing our new business growth from 1 percent to 4 percent (or better) would increase our gross revenues by about $3 million. If we can do this without increasing our support costs per call, we should be able to realize a net gain of at least $2 million.”

Goal Statement: “Increase the call center’s industry-measured customer satisfaction rating from its current level (90th percentile = 75 percent) to the target level (90th percentile = 85 percent) by end of the fourth quarter without increasing support costs.”

The project team also developed its initial set of milestones, tasks, responsibilities, schedule and communication plan. After reviewing the charter with their Champion, team members began work on the next step – customer requirements.

D2. Customer Requirements

For many processes there are one or more customers who are obvious – the people who pay for products or services or those who visibly depend on the company, like employees. Often, though, there are internal and external customers and dependencies that are not so obvious. A SIPOC table (Suppliers, Inputs, Process, Outputs and Customers) develops a detailed view of all the important customers, their requirements, and the related process step and supplier dependencies.

SIPOC Table: The team developed a SIPOC table to help identify and report what it learned about key customers and their requirements. While processes flow in the SIPOC direction, the thought process used to build the table often begins with the customer, suggesting display in the COPIS (Customer, Output, Process Inputs and Suppliers) direction as shown in Figure 1.

Figure 1: SIPOC / COPIS Table

Voice-of-Customer (VOC) Interviews: The SIPOC table highlighted some areas where more information about the process – as understood by customers and call center staff – would be helpful. Representative samples of the company’s customers were involved in group interviews. The following are excerpts from individual customer responses and are representative of the data gathered. The answers are to the question: “What influences your level of satisfaction with our services?”

Group 1

A1. “Well, first and foremost I want the correct answer to my questions or issues. It makes me nuts when I’m told something that turns out later to be wrong or only part of the story.”

A2. “I really like it when the person who answers the phone has a good attitude. Sometimes you can just tell they wish you’d just go away and stop bothering them with stupid questions.”

A3. “Well I can sure tell you what I don’t like. That voice response thing where they ask you to make 46 choices (and none match what you want) is a real pain – ends up taking a lot of my time and then they seem to ignore it all anyway and ask the same questions again. What’s the point? Sometimes I wonder if they ever asked a real customer to test these things before they put them in.”

A4. “I’m happy when my call gets answered quickly and the person I talk to knows their stuff and gives me an answer on the first call. When I have to wait for a call back and talk to someone else, repeating some of the same stuff – that’s a real drag!”

A5. “I like it when the person on the phone can actually make a decision without putting me on hold while they get an ok from a supervisor. Seems like they spend 10 minutes to make a $5 decision. That just doesn’t make sense to me. Seems like some control freak is running the show.”

A6. “Follow-through is what really matters to me. I don’t necessarily expect you’ll always be able to resolve my issue immediately, but I do expect you’ll call me back in a reasonable amount of time.”

A7. “My hot button is getting someone who has enough patience to really solve my problem. Some of this stuff seems pretty technical to me, and I don’t always know even the right question to ask. I like it when the person on the phone cares enough to get to the real solution, even when I can’t explain exactly what I need.”

A8. “I don’t want to be transferred around. Last time I called I got transferred four times and ended up with the same person I started with. I’m too busy to put up with that!”

Group 2

A1. “Our big concern is speed. Our customers demand answers from us, and we in turn rely on you for some of that. That means you have to be adequately staffed to meet call volume.”

A2. “What we most need from you is people who can answer complicated questions accurately and quickly – not just the easy stuff, we can do that ourselves.”

A3. “We need you to have immediate access to up-to-date information about all of our accounts and transactions with all of your branches and locations. It creates huge problems for us when your records aren’t accurate and timely. We can’t sit on hold for 10 minutes.”

A4. “I call 3 to 4 times a week, and the thing I find most frustrating is the lack of consistency. Sometimes my call gets answered in 2 minutes, which I can live with, and sometimes it’s 10, which I can’t. I also notice that there’s a lot of variation in how long it takes to get answers to very similar issues. I call your competition all the time also, and they’re a lot more consistent.”

Summarizing Customer Requirements: The team began to summarize what it was learning about what’s important to customers – in the form of requirement statements and measures.

>Requirements	> Measures
> Quickly connect with a helpful person	> Wait Time
> Get the information I need	> Transfers, Service Time
> Apply the information, with help if needed	> Customer Satisfaction, Support Cost
> Understand how to avoid problems recurring	> Days to Close

D3. High Level Process Map

The team mapped the process by which an initiating event (an issue encountered by a customer) moves into and through the resolution process (Figure 2).

The process map will be helpful during the Measure phase, as the project team considers how and where to gather data that will shed light on the root cause of the issues most pertinent to the project’s goals.

The Measure Phase

Having developed a good understanding of the project’s business case and customer requirements (identifying the Y(s)), and the as-is process, the Six Sigma project team of the IT services business began to focus on the Measure phase. The team identified the measures and data collection plan for gathering the right amount of the right data to impel their learning about root causes and drivers that impact the project Y(s).

The DMAIC roadmap called for work in these areas during the Measure phase:

M1. Refine the Project Y(s): Getting even clearer about how the project’s key outcome measure(s) will be defined, measured and reported.

M2. Define Performance Standards for the Y(s): Identifying how performance will be measured – usually somewhere on the continuum from capability measures like Cp and Cpk for “variables” data that is normally distributed to percentile or other capability metrics for “attribute” and other data that may be skewed in distribution.

M3. Identify Segmentation Factors for Data Collection Plan: Starting with the natural segmentation of project Y(s) and moving though consideration of prospective driving factors (x’s), segmentation suggests the packets of data that should be collected in order compare and contrast segments to shed light on Y root causes and drivers.

M4. Apply Measurement Systems Analysis (MSA): In any project, raw data is gathered and then converted into measures. That process comprises a “measurement system” that should be characterized and strengthened in terms of its accuracy and repeatability.

M5. Collect the Data: Gathering data, preserving its meaning and noting any departures from the discipline put in place under MSA.

M6. Describe and Display Variation in Current Performance: Taking an initial look at the data for its distribution, extreme values and patterns that suggest special variation.

M1. Refine the Project Y(s)

During this step the team conisdered exactly how the project Y(s) would be defined and measured:

	Y(s)	Measurement
Primary	Customer Satisfaction	1. By industry standard monthly survey 2. The project will require additional, more frequent, case-by-case customer satisfaction data. A measurement system that tracks with the industry survey will be devised and validated.
Secondary	Supprt Cost (Per Call)	The staff time connected with each call: – Call answering and discussion – Case research – Callback time will be loaded with a distribution of benefits and infrastructure costs to compute overall support cost per call.
Related / Of Interest	Days to Close	Time span from call origination through client indication that the issue is closed to their satisfaction.
	Wait Time	Automatically tracked for calls in queue. Summed for calls encountering multiple transfers.
	Transfers	Automatically tracked for each call moved to another extension.
	Service Time	Automatically tracked for time from staff call pickup until hangup or transfer.

M2. Define Performance Standards for the Y(s)

For each project Y, the current baseline and best-estimate target was documented. In some cases, the team found that good baseline data was unavailable. (Unfortunately that is a common occurrence in DMAIC projects.)

	Measure	Current Baseline	Target
Primary	Customer Satisfaction (Per Collins Industry Assessment)	90th Percentile / 70-80% Satisfied	90th Percentile / 85% Satisfied
Secondary	Support Cost Per Call	90th Percentile / $40	90th Percentile / $32
Related / Of Interest	Days to Close	Did Not Have Good Baseline Data	90th Percentile / 3 Days or Less
	Wait Time	Did Not Have Good Baseline Data	90th Percentile / 4 Minutes
	Transfers	Did Not Have Good Baseline Data	90th Percentile / 2
	Service Time	Did Not Have Good Baseline Data	Mean: < 8 Minutes St.Dev.: < 0.5 Minutes

M3. Identify Segmentation Factors for Data Collection Plan

The first question was: How is Y naturally segmented? Often Y data is organized by customer type, geographic region, product or service type, etc. Thinking about where the strongest “action” was for the dynamics under study, the team focused its initial data collection effort on the segment(s) that offered the most potential for the team’s learning. This helped conserve the limited resources available for data collection and analysis. Instead of “measuring everywhere,” the team started with a focused subset of all possible data. The data was naturally segmented by call center – with most of the traffic in one center. Data from that site was used for the initial data collection.

The next question was: What factors may be driving the Y(s)? Identifying these factors and gathering data on their behavior may shed light on the root causes and drivers for the Y(s). This is the planning part of the Six Sigma drill-down that is sometimes called “peeling the onion.” While the fundamental interest is in the Y behavior, the idea is not to truly solve a problem by trying to “fix the Y.” That approach might provide a temporary fix. Understanding the underlying drivers (the x’s) offers the possibility of addressing the root cause, and fixing the problem so that it will stay fixed.

A Y-to-x tree depicts the array of lower level x’s that may be driving a Y. Other tools with a similar thrust – cause-and-effect diagrams and cause-and-effect matrices (illustrated later) – can be helpful in identifying and prioritizing the prospective x’s for data collection and study.

The team’s Y-to-x trees for support cost and customer satisfaction are shown in Figures 1 and 2.

Figure 1: Y-to-x Tree for Support Cost

Figure 2: Y-to-x Tree for Customer Satisfaction

Input-Output Analysis

Building on the information developed in the SIPOC / COPIS table, the team reviewed process inputs and outputs, classifying each as “controlled” (with process provisions in place to measure and influence that input or output, if necessary) or “uncontrolled” (with no such provisions in place). See Figure 3.

Figure 3: Process Inputs and Outputs Classified as Controlled or Uncontrolled

Cause-and-Effect Matrix

To further explore potentially influential factors, the team created a Cause-and-Effect Matrix (Figure 4). The high scoring items in this analysis were strongly considered for data collection. The highest, “Available Staff for Transfer,” was included. The next highest scoring, “Staff Experience/Training” was not readily available in the historic database. (There had been reluctance to log personal data as part of the ongoing call logging.)

Figure 4: Cause-and-Effect Matrix

M4. Apply Measurement Systems Analysis (MSA)

To prepare for the measures to be collected in the next step, the team reviewed the measurement systems. In transactional processes, any activity that gathers raw data and converts it into counts, classifications, numbers or other forms of measure is a “measurement system.” While the statistical methodology connected with MSA is beyond the scope of this article (see other references under iSixSigma tools) , Figure 5 depicts the four questions that are usually posed for measurement systems in transactional processes. Viewed simply, the intent of MSA is to strengthen a measurement system so that it is suitably accurate, repeatable, reproducible and stable. A fifth issue, “linearity” (the accuracy of the system over the range of mesaured values), is sometimes considered..

Figure 5: Questions Usually Posed for Measurement Systems

M5. Collect the Data

A plan was formulated to gather data from the past year’s database. This required retrieving call data, as well as tracing call resolution times, staffing levels, call volume levels and relevant follow-up events. For each call sampled, the team rebuilt information about staffing, call type, number of transfers, wait time, etc. (Figure 6)

Figure 6: Format Used to Record Collected Data

M6. Describe and Display Variation in Current Performance

A first look at the data coming in provided the team insights about extreme values and patterns suggesting problems with the measurement system. With the information, the team began to forecast what the Analyze phase would reveal. The team’s question at this point was: How is the Y Distributed? The team looked for the layout of measured values collected – for symmetry and for extreme values. This suggested the kinds of graphical and statistical analysis that would be appropriate (Figure 7).

Figure 7: How Is the Y Distributed?

Data on the call center x measures was graphed and charted a number of ways. Figure 8 shows the variation in customer Wait Times on an Xbar-R control chart. Variation above and below the chart’s control limits suggested that there were “special causes” in play – worth understanding in more detail by the team in the Analyze phase.

Figure 8: Xbar-R Chart of Wait Time

The Analyze Phase

Having refined the project’s key outcome measures, defined performance standards for project Y(s), identified segmentation factors and defined measurement systems, the Six Sigma project team of the IT services business began to focus on the Analyze phase. The DMAIC roadmap called for work in these areas during the Analyze phase:

A1. Measure Process Capability: Before segmenting the data and “peeling the onion” to look for root causes and drivers, the current performance is compared to standards (established in step M2 of the Measure phase).

A2. Refine Improvement Goals: If the capability assessment shows a significant departure from expectations, some adjustment to the project goals may need to be considered. Any such changes will, of course, be made cautiously, supported with further data, and under full review with the project Champion and sponsors.

A3. Identify Significant Data Segments and Patterns: By segmenting the Y data based on the factors (x’s) identified during the Measure phase – the team looks for patterns that shed light on what may be causing or driving the observed Y variation.

A4. Identify Possible x’s: Asking why the patterns seen in A3 are as observed highlights some factors as likely drivers.

A5. Identify and Verify the Critical x’s: To sort out the real drivers from the “likely suspects” list built in A4, there is generally a shift from graphical analysis to statistical analysis.

A6. Refine the Financial Benefit Forecast: Given the “short list” of the real driving x’s, the financial model forecasting “how much improvement?” may need to be adjusted.

A1. Measure Process Capability

The team first looked at the distribution of each Y data set. For those with symmetrical shapes close enough to a normal distribution (the bell-shaped curve in Figure 1) means-based measures (e.g. mean, sigma, or Cp and Cpk) were used to describe capability. For skewed distributions (the histogram in Figure 1, and any cases where the Anderson-Darling test P-value was below about 0.05) a median-based capability measure was used (e.g. median, quartile, percentile).

Figure 1: Distribution Check for Support Costs

While the graphical summary in Figure 1 shows median-based capability to the detail of quartiles (1st Quartile: 25%, Median: 50%, and 3rd Quartile: 75%), the team applied a macro to generate a more detailed percentile view, summarized in the list below.

75th Percentile = $32.80
80th Percentile = $33.36
85th Percentile = $35.42
90th Percentile = $39.44
95th Percentile = $42.68
98th Percentile = $44.73

The support cost 90th percentile capability is $39.44. Call volume, of course, indicates that this was a very costly gap. The results of these and other capability checks, as done at the outset of the Analyze phase, are summarized and compared to established targets in the table below.

Measure	Capability	Target
Customer Satisfaction (Per Collins Industry Assessment)	90th Percentile = 75% Satisfaction	90th Percentile = 85% Satisfaction
Support Cost Per Call	90th Percentile = $39	90th Percentile = $32
Days to Close	90th Percentile = 4 Days	90th Percentile = 3 Days or Less
Wait Time	90th Percentile = 5.8 Minutes	90th Percentile = 4 Minutes
Transfers	90th Percentile = 3.1	90th Percentile = 2
Service Time	Mean: 10.5 Minutes StDev: 3 Minutes	Mean: <= 8 Minutes StDev: <= 0.5 Minutes

A2. Refine Improvement Goals

Reviewing the data in the table, the team felt that the project targets were still in line and did not require a change at that time. Had there been a surprise or a show stopper, that would have been the time to flag it and determine the right action.

A3: Identify Significant Data Segments and Patterns

While planning for data collection (during the Measure phase), the team had done some hypothetical cause-and-effect analysis to identify potentially important x’s. At this step, it prepared to use the data to confirm or reject those earlier hypotheses, and to discover other x’s that may have been missed.

Figure 2 outlines some of the common tools for finding patterns in the data. Since most tools are suited for certain kinds of data, a chart like this can be helpful. Like many teams, during A3 this team favored graphical tools, which give quick views of many “cuts” on the data. The team found multi-vari charts were especially flexible in that regard. Later, when refined distinctions were called for (in step A5), the statistical tools like ANOVA, regression, and Chi-Square were brought into play.

Figure 2: Some Key Analysis Options

Numerous cuts of the data were reviewed with the goal of shedding light on root causes and drivers underlying variation in the project Y(s). A few of those are summarized in Figures 3 and 4. Figure 3 shows that Problems and Changes look more expensive to service than other types of calls. Figure 4 reveals an added signature in the pattern – Mondays and Friday stand out as being more costly.

Figure 3: Multi-Vari for Support Costs by Call Type

Figure 4: Mutli-Vari for Support Cost by Call Type and Day of the Week

A4: Identify (Refined/More Detailed List of) Possible x’s

Collecting the findings that came out of A3, the team posed strongest in the form of “why” questions:

• Why do Problems and Changes cost more than other call types?
• Why are calls processed on Mondays and Fridays more expensive?
• Why do transfer rates differ by call type? (higher on Problems and Changes, lower on others)
• Why are wait times higher on Mondays and Fridays and on Week 13 of each quarter?

The team reviewed the fishbone diagrams, Y-to-x trees, and cause-and-effect matrices that it had built during the Measure phase. At this step, with the benefit of the data and insight gained during A3, the team was ready to get closer to what was really driving the Y(s). Figures 5, 6 and 7 trace the team’s thinking as it moved through this step. Figure 5 highlights questions about the driving influence of staff availability – and why it may vary so widely on Mondays and Fridays. Figure 6 highlights the issue of staffing/call volume as a ratio. The initial data had looked at these factors individually. Figure 7 raises questions about several factors that were not measured initially – but the data may suggest these as realistic lower-level x’s that should be studied using a sample of follow-on data.

Figure 5: Fishbone Diagram

Figure 6: Y-to-x Tree

Figure 7: Cause-and-Effect Matrix

The work represented in the previous figures motivated the next round of analysis, step A5, to check the deeper relationships hypothesized. As is often the case, the team had identified some new data that could be useful. Further, it had uncovered some new ways to “torture” the current data to reveal more root cause insights:

• Volume to staffing ratio – Call volume and staffing had not revealed much when looked at separately. Their ratio may be more interesting.
• Web-to-phone issue call traffic ratio could be computed from the initial data – potentially revealing more insight.

A5: Identify and Verify the Critical x’s

The team made the computations and comparisons suggested by its cause-and-effect work. During this step, there was a leaning toward the statistical tools, to make the verification of driving x’s more fact-based and convincing. Figures 8 and 9 illustrate a few elements in that work. Figure 8 shows that some days can be understaffed (Fridays and Mondays) and others (especially Sundays) can be overstaffed. Figure 9 shows that the influence of callbacks on a call’s wait time (graphed during A3) is statistically significant (indicated by the P of less than 0.05 in the Callback row of the ANOVA table).

Figure 8: Multi-Vari with Computed Ranges Overlaid

Figure 9: ANOVA Output

A6: Refine the Financial Benefit Forecast

In its charter, the team had signed up to reduce support costs per call from the current level (as high as $40) to $32. Given the team’s analysis on factors driving support cost, the members still thought this was possible, and left the forecast unchanged.

The team was pleased to see that the key support cost drivers (the delays and interruptions during call servicing) were the same as those known to drive down customer satisfaction – so a win-win seemed to be possible.

Continue to Improve and Control Phases »

Related Posts

1. A Six Sigma Case Study – Tutorial for IT Call Center; Part 1 of 6
2. A Six Sigma Case Study-Tutorial for IT Call Center – Part 2
3. A Six Sigma Case Study-Tutorial for IT Call CenterPart 4 of 6 – The Analyze Phase
4. A Six Sigma Case Study-Tutorial for IT Call CenterPart 6 of 6 – The Control Phase
5. A Six Sigma Case Study-Tutorial for IT Call CenterPart 3 of 6 – The Measure Phase