iSixSigma

Help Identifying Metrics

Six Sigma – iSixSigma Forums General Forums Methodology Help Identifying Metrics

This topic contains 17 replies, has 5 voices, and was last updated by  Hirsch 5 years, 7 months ago.

Viewing 18 posts - 1 through 18 (of 18 total)
  • Author
    Posts
  • #54587

    Hirsch
    Participant

    My company has defined a set of critical success factors for aligning our quality improvement work. We intend to use the factors as categories for project prioritization and reporting, but are struggling to produce performance metrics on two of these. I’d like to know if anyone has any similar experience or suggestions.

    The first CSF is Associate Effectiveness & Capabilities, which will align efforts such as increasing employee core capability building and expanding skills. The second factor is Effectiveness for Business Partners & Customers, which will look at how our division improves its processes to make life better for our internal customers and areas of the business we support. To complicate things further, these are completely transactional business support processes, so production-oriented measures like efficiency improvements will be extremely difficult to develop.

    Metrics to measure for the other CSFs are a piece of cake. I will appreciate any feedback you may have.

    0
    #196186

    Trish G
    Participant

    Why would transactional business support process not be conducive to efficiency measures? How do you…or better yet your internal customers define “making life better”. Do you have service level agreements in place? When do internal customers downstream from these processes need the completed output? Is there a level of quality they are expecting?

    0
    #196187

    Chris Seider
    Participant

    @Trish, those in manufacturing are used to metrics. Those in many transactional processes aren’t used to many measures tracking performance. It’s a cultural shift.

    0
    #196189

    Fahd

    I suggest you use “Ideas & suggestions” & employee certification program as good metrics for the first one. As you receive ideas and suggestions, you track the ones submitted, the ones accepted and the ones implemented. As for the employee certification program, that is self explanatory.

    Other metrics you could use in conjunction with the ones listed previously could be attendance rate and turnover ratio. Don’t forget about the power of focus group surveys that can be done every quarter or month…etc

    0
    #196190

    Trish G
    Participant

    Chris- I’ve been in the services industry for 20+ years, mostly in financial services. Although I will admit metrics are probably not used to the extent they are used in manufacturing, or as easy to establish, but they are definitely used. Transactional processes especially.

    The employee development metric is a little harder to land on, but not impossible. It could be established through one or two questions on an employee sat survey, or through performance plan goal setting.

    0
    #196191

    john

    You can’t have a answer until you define the question, and metrics are essentially the answer – so what is the question you are trying to answer? For example “How effective is the current training program at providing our employees with the skills needed to meet customer’s expectations?” or “How effective is the current training curriculum at improving the the problem-solving skills of our employees?” At some point, you will want to triangulate different measures (ie for effectiveness = usage rate, satisfaction scores, delivery elements)to generate a complete Metric for a given core question. Also, Trish is right…no problem measuring anything in a transactional environment….

    Lastly, remember that Effectivenss is outward facing and is defined by the end user or consumer of the process output. It isn’t about the employee or how well they do their job, or about management and how well it does its job…simply how well the designed output achieves a given outcome as articulated by the customer who consumes it.

    Good luck.

    Good luck.

    0
    #196198

    Don Strayer

    Whenever I get stuck I remember GQM, Victor Basisli’s Goal/Question/Metric approach. Start with the goal. What question do I need to ask to tell me how well we’re achieving the goal. What metric will answer that question. What do I need to measure in order to provide that metric. How can I measure it.

    0
    #196213

    Hirsch
    Participant

    Thank you for your responses. To clarify a few points, the primary question of this post is not “How do I identify a process metric”? I did that for 2 decades in manufacturing, and determining measurables with the applicable processes now related to operational factors like efficiency and variability are what I stated were the easy ones. The question is “Does anyone have experience and examples of metrics they have developed for the two intangible success factors (or similar) stated above”?

    We are part of a very large Fortune 100 with relatively very little turnover, Our division supports practically the entire corporation, but our same outputs are used by various functional areas in very different ways. So, surveys and the like are not very feasible, and the company is so survey-crazy now it typically doesn’t yield robust data. “Making life better” might be restated as How effectively do changes in our processes allow our business partners to better and more easily execute their own processes? Customer process performance does not directly correlate to the quality of our output, and cannot be a measure since we cannot control the customers’ performance. And for the two applicable criteria, the end customer of the processes IS the employee.

    If that additional information helps generate additional ideas, I’d love to hear them. If not, the feedback you already provided has been very helpful.

    0
    #196217

    Cyril
    Participant

    Can you define the process with SIPOC and identify the measures. Wheather it is a manufacturing or service, there would be difined way to do the activity to ensure the result. Track for it, you can try to define the high ended one.

    0
    #196218

    Trish G
    Participant

    First of all, why are changes being made in the process? If it is to improve efficiency downstream (your business partners), understanding what they need from your process is the first step, the changes are then focused around process improvement to improve the output that your business partners need.
    So let’s use a mortgage processing process as an example, the Underwriting might be receiving applications with numerous errors, forcing them to re-contact the customer. The Originations department might consider a project improvement to decrease errors and omissions. The metric might be % application with errors, and/or number of customers contacted.

    Regarding your associate effectiveness and capabilities metric, I’ll piggy back on John’s post. It probably is a number of measures that would be indicative of a capable and skilled workforce. Turnover, % performing at a “meets requirements of their job”, % receiving training…or the like. You could even develop an index of these measures.

    0
    #196220

    Hirsch
    Participant

    I suspect I am not making it clear this is a component of our overall continuous improvement strategic alignment model, and not any specific process improvement effort. So, it involves all our business processes, and there are dozens maybe hundreds of processes totally. Our Leadership has identified the five primary factors that need to be supported in order to achieve our division’s top CI goals. We are struggling with only two of the five. The reasons to make changes (for this thread) are to improve effectiveness of our employees, and to improve how effectively we support our business partners.

    Project ideas will be evaluated by their alignment to these factors, prioritized by comparing total impact on these factors (think “prioritization matrix”), and success assessed by the results reported. Using Trish’s example, pretend we need to align to only the two factors I have mentioned. An idea that is estimated to reduce application errors by x% would have low priority as we have ideas which are more important because they have impact on the customer requirements for employee and internal business partner effectiveness. (That example is illustrative only since we do have other factors addressing efficiency and variability, and those are off topic because we are able to identify applicable metrics).

    People in my organization just don’t get the part about identifying, measuring, and reporting on critical process variables, which baffles me but it is what it is. I am investigating benchmarking and best practice examples to help our people at a understand at a fundamental level what to look at for in idea generation and how we will select projects to work on.

    Call them KPIs or whatever if you like, but at the end of the day we need measurables to compare (again using the example) the Underwriting x% accuracy improvement idea to one which increases a core competency in Collections dept by y% and one that helps Sales Compensation dept simplify their broker payment by z%. I am interested in anyone’s experience with identifying novel success measures specifically for the two factors I mentioned in the OP. If this additional information helps do that, great. But if not, it does help validate that what we have come up with so far is not too far out in left field. Thank you all for your feedback

    0
    #196223

    Trish G
    Participant

    Just out of curiosity, what are the other 3 components to your “CI strategic alignment model” beyond employee effectiveness and business partner support?

    0
    #196224

    john

    I would respectfully disagree that your employees are the customer in those 2 areas…For example, CSF01 – Your employees are resources employed to add economic value to the business. They are not the customer of the training, the business is….You don’t ask the doctor how he liked medical school to judge the quality of the training, rather you seek the opinion of the patient and/or the hospital. Hence, training must improve the ability of the employee to add economic value…period.

    To answer your question, you need to get more precision out of your core question. For example, you posed “How effectively do changes in our processes allow our business partners to better and more easily execute their own processes?”. Good start, now distill it further by defining what do you mean by:

    – How – To what extent? To what degree? What parts worked / didn’t work?
    – Effective – the degree to which something achieved its intent…
    – changes – Which deviations? Where? By whom? For who?
    – our – What parts do we own? Not own? Do we control? Influence? etc.
    – allow – Are we incenting? Removing barriers? Increasing probability? etc
    – better – than what? by what standard? whose perspective? magnitude? direction?
    – easily- How? Number of touches? Groups? Interactions? Steps? FTEs? Time? Errors?
    – ETC –

    The more precision you get out of that core question, the clearer it will be as to what measures and data you will need to create useable information for your decision makers…don’t underestimate the power of tearing your core question apart and rewriting it to be as precise as possible. The better the question, the better the answer (ie metrics). Good metrics come from a crystal clear understanding of what it is you want to know…REALLY know…No one cares about how employees feel about training…REALLY cares….what they really care about is how effectively does the training increase the employee’s ability to generate economic value.

    0
    #196237

    Hirsch
    Participant

    @Trish G, sure… in addition to the hard factors of benefits returned, cost & investment, required resources & scope, we have these 5:

    1. Contribute to Cost Control or Risk Avoidance – Our operating model and regulatory situation makes it more sense to combine these into a single category.
    2. Decrease Process Variability – We are defining more broadly than traditional SS to include accuracy and any muda affecting quality of output / deliverables.
    3. Increase Process Velocity – Any improvements which allow positive business results to be realized more quickly. Includes obvious like cycle time reduction, but also allows for things like adding business resources in one area if requirements are fulfilled quicker at a broader level.
    4. Improve Effectiveness for Business Partners – nuff said.
    5. Increase Associate Effectiveness & Capabilities – ditto

    0
    #196238

    Hirsch
    Participant

    @john: Since you don’t seem to understand (or just chose to ignore) the question, allow me to say that a person can quote junior-level textbook theory all they want, but first should recognize that may offer an opportunity with all the time they need to draft up such diatribes while standing in the unemployment line.

    In this real life, the alignment model approved by the senior leadership of a major corporation (you know, the people who ultimately have the say on who works for the company and who doesn’t?) has already defined the primary Customer in those two areas as our Employees and Internal Business Partners. So, I would respectfully disagree, and within the topic in this post your suggestion is 100% wrong… period.

    BTW, regarding your previous post, yes I did see Stacey Barr’s newsletter this week, thank you.

    0
    #196240

    Trish G
    Participant

    It’s an interesting approach. It appears to me that there is a ton of overlap. At the very least, 2 & 3 directly impact 4 and 1 (the cost control piece anyway). To the extent that measures would be very similar or at least highly related

    To get a handle on our enterprise wide process capabilities, we use a process classification framework, it helps us understand our process portfolio, beyond that we have process leaders and team who are accountable for the effectiveness and efficiency of that process. The process is defined end to end, and those actively managing the process form that team and work collaboratively to improve it. We use a process maturity continuum to gauge each process, and to gauge our process portfolio as a whole. We do prioritize processes using financial impact, customer impact, and risk as criteria. The higher the prioritization, the higher level of maturity is expected, and the more process management resources (BPM Dept.) can be utilized.

    The goals of the organization, including process capability and employee development are aligned and managed balanced scorecard style.
    Best of luck

    0
    #196241

    Mohamed Salaheldin
    Participant

    Have a look at http://www.staceybarr.com , I think you may get good information about Performance measurement. It is really useful.

    0
    #196246

    Hirsch
    Participant

    @Trish G: Thanks for sharing the overview on your company’s structure. It’s always helpful to understand other’s successful implementations. Our initiative is in the fledgling stages. My challenges have been along the lines of getting people to understand the basic differences between process and system, and fixing things when they break does not constitute Continuous Improvement. Baby steps, and like someone else mentioned it’s a cultural change.

    The overlap is partially by design, and also because we developed the list by affinitizing from a mish-mash of separate business unit and departmental goals. The execs made a few tweaks after reviewing, but agreed it solidly captured where we should be focusing efforts. The “by design” part was in recognition that the configuration initially needed to stay a little grey, both in how to categorize improvement ideas and how to measure results. We could foresee low engagement if people were forced to give black & white, compartmentalized definition of what kind and how much impact their ideas would have. So, the improvements are allowed to affect more than one success measure, but don’t necessarily have too. It’s sort of analogous to on a Friday night getting a paycheck is good and having cash in hand is good, but getting paid in cash to put with what’s already in your pocket is better.

    I already stated the problems folks have understanding quantitative process analysis. Without historical results to use as a basis, we intentionally left the estimation of impact fuzzy with a plan to establish the standards for project prioritization as we gained experience with improvement results. The funny thing is now that the prioritization process is being more broadly applied, the same people who originally demanded “soft” measurement methods are frustrated because they don’t have metrics to determine relative impacts of the ideas! I guess if anything it’s signs that the organization is maturing.

    @Mohamed: Thanks for the suggestion. Yes, Stacey’s stuff is awesome.

    0
Viewing 18 posts - 1 through 18 (of 18 total)

You must be logged in to reply to this topic.