iSixSigma

Black Belt Performance Measures

Six Sigma – iSixSigma Forums Old Forums General Black Belt Performance Measures

Viewing 18 posts - 1 through 18 (of 18 total)
  • Author
    Posts
  • #29106

    Hill
    Member

    My company is starting to deploy Six Sigma and currently is addressing performance criteria for black belts. Does anyone have any ideas on how black belts should be graded, beyond number of projects and dollar amount? Management wants to capture the full range of requisite black belt skills: leadership and effectiveness as change agents, management, technical expertise, strategic abilities, etc. And if these are good categories for evaluation, then how to assess performance?
    Thanks. sarah

    0
    #73696

    Mike Carnell
    Participant

    Sarah,
    Using a fixed dollar amount and/or number of projects will walk your program into the same problem a lot of other programs have. The assumption is that Six Sigma project are homgeneous, which they are not. If a project is purely a control issue it can typically be solved in a short period of time. A technology driven project is an entirely different issue and will run a much longer time frame. If you istitutionalize the metrics you can potentially penalize the thech heads that are working on projects that run longer time frames. These are common metrics. That does not make them good metrics.
    As soon as you open up the door on leadership, effectiveness, strategic ability, etc you are into the subjective catagories. I would use them as soon as you get them to pass the same GR&R criteria that a BB uses on their projects. If you going to measure them with something lets make sure it works.
    When you open up the “technical expertise” catagory you will shift the program from a results program to a stats program. The two are frequently mutually exclusive.
    I am not sure why Management wants to capture the full range of requisite BB skills. Unless they are a consulting firm their product is not Black Belts. The Black belts deliverable should be results. The results should be based on those factors that someone has determined need to change to move the company to a more competitive position. The metrics on the BB’s should reflect how they affect that.
    There is some stuff floating around called “Break all the rules” or something like that. It might be some good funimental reading before you put your metrics in place. If you just launched you have probably 4-6 months before you really have to have something in place. This gives you the time to really do this thing correctly. It would be time and money well spent.
    Good Luck.

    0
    #73766

    Chris Trimble
    Participant

    Our company launched 6 Sigma in Feb 2001.  The black belt measures used include not only dollars brought to the bottom line, but also number of projects brought to control; number of green belts trained; and amount of input into coporate culture change.  These are all objectives that can be measured fairly easily.  Another measure we currently use is the difficulty of the project this is extremely subjective and to date has not been shown to be effective.  The biggest problem is sponsor engagement in keeping projects in front of the black belts.

    0
    #73770

    KHP
    Participant

    CTX’s or Customers Critical To’s
    Critical to Time or Delivery, Critical to Quality, Critical to Cost
    Everytime you start a project, there is Business Y & Process Y. The Business Y focus on Money or Cost, but the process Y usually focus on Time (delivery) or Quality. These terms should be used to measure black bellt performance. As black belts, we were thought to use these terms to find metrics that meant most to the customer. It just makes since to use terms to measure black belt performance.
    Hope I made some since.

    0
    #73776

    john beaudoin
    Participant

    Sarah,
    If you are a true believer of 6-Sigma and TQM, along with the foundation that Demming set up, you will not set up any metrics to “Grade” your blackbelts individually.  I say this as I am a successful acting blackbelt that is also going to KU for an MBA and hold the position of a Project Engineer with a fortune 500 company.
    Currently, my boss and I are both graded, as some other responses you have received.  The 1st response is very insightful.  I would like to point out that we all have a paradyme of grades and rankings, and expect them in school or in work.  However, this competition to perform can be very detrimental to the implementation of an initiative such as 6-Sigma.  Many companies have failed in their 6-Sigma initiatives because Financial types want to see immediate paybacks, ROI’s for training expenses, and more hard cost reductions than soft.
    Let me explain what I mean by failed.  By putting dollar value savings on projects, there is now incentives to find dollars.  Some of these dollars are legitamate and some may not be so legitament.  Many projects have touted Million Dollar savings, but when you look at the companies accounting, the Million Dollars cannot be found.  In these cases 6 Sigma loses credibility.  Another failure is that projects start focussing on reducing expenses and saving costs.  These are short-term benifits, but leave behind the biggest opportunities for the business of winning customers.  Allied Signal found that customer satisfaction levels correspond directly to repeat business and the share of money that customers spend with suppliers.  Projects should focus on identifying customer expectations and then meeting or exceeding them.
    Last, I said that Black Belts should not be “Graded” individually and compared to each other, however, I am not saying that there should not be goals set.  The company needs to have a business plan and strategy.  Part of the strategy should be to win over new customers and keep existing ones.  There may be other goals such as reducing inventory, improving processes to meet benchmarks, etc.  All of the projects should be selected to support these goals.  If these goals are met, the group of project participants should be rewarded.  If not, you need to find out why and improve training, skills, etc.  What you don’t want is everyone competing with each other, not sharing ideas, and not working together for the common good of the company.
    I’m not sure if I am a 100% follower of the teachings of Demming, but I can tell you that the Doctor teaching my Graduate Class is.  He only allows the class to be taken as Pass/Fail (Grading is not acceptable to Demming), and he has put us in study groups to discuss the readings and topics to learn from each other. 

    0
    #73796

    walden
    Participant

    Sarah,
    One thing you don’t want to do is implement metrics for BB’s on the financial side.  The only place that you make money in Six Sigma is in the define phase by identifying the right projects.  This is typically the responsibility of the project champion, area management, etc.  Whoever is responsible for defining projects is the person who should be graded on dollars saved.
    Better metrics could be project cycle time (subgrouped by project type:  see the thread on tech projects above), percentage of entitlement captured in projects, and percentage of dollars saved after a project’s closeout.  You want metrics that assess how effective was the “fix” to the problem and how quickly was the “fix” sustained.
    This is interesting stuff, albeit subjective.  I am a MBB wondering the same thing, because if BB’s don’t succeed, MBB’s don’t either.
    Chris

    0
    #73798

    Withheld…
    Member

    John,
    I agree with much of what you said. I do not agree, however, with the complete dismissal of grading or ranking. Enlightened management (and your professor for that matter) can fulfill their responsibility to promote teamwork and still maintain a system by which to grade the individual. The grading criteria simply need to include the ability of the individual to function as part of a team. A test, for example, results in a black and white score. The student who excels acedemically should not be penalized by a professor who takes Deming’s philosophy to an unreasonable extreme.
    The same is true in business. Rather than measure gross dollars, measure % of opportunity dollars. Have individuals keep time records on their participation in certain projects and have the project leader sign off on that time. At the end of the day, the participant is rewarded for his cooperation by sharing in the success based on the amount of time he cooperated with others. This creates an environment in which otherwise competing interests work together because their success relies on each other to some degree.
    Competitive drive and individual pride vary from person to person, but are the most significant factor in individual performance. They need to be nurtured in a way that recognizes their importance while mitigating the potential to do damage. It may be a fine line, but it is a balance that can be acheived, IMO.

    0
    #73803

    john beaudoin
    Participant

    Thanks for the advice.  Even the professor won’t give you a passing grade if you miss class (attendance is therefore tracked) and he does want to see the outlines you prepare for your discussions (provides incentives to do some work).  Tracking meeting time spent, attendance, and soliciting feedback from team members on each individual is also important.  The intangibles are important like:  How well does the person work in a group environment, is the person helpful and enthusiastic in providing support, knowledgable, timely, etc.  Where I work, we have done some of this for developmental reasons, but some of it is starting to work its way into our evaluation process, but only for 25% of the overall grade.  I imagine in some companies, these things can be for 50% or more.  However, right now about 50% is how many projects us and all of our greenbelts complete in a year, how much total $ are saved, and then another 25% on our individual contribution to Projects and Dollar Savings.
    My boss is a firm believer in setting goals, accountability, and winning.  As a result, we are motivated to hit numbers.  However, this is not a good way to motivate the “Warm and Fuzzy” Individuals.  Fortunately, my boss also realized that some projects are good for business reasons and not big dollar savings.  This gets figured into the project count portion of our evaluations, and thus we get some credit.  However, many companies may not provide for this, and in turn may not do many customer orientated projects where the dollar value is difficult to compute as you need to measure the value of customer satisfaction.  This leads into the trap of companies focussing on reducing costs and not growing sales.
    I think I agree that there needs to be some balance of both.  If individualism is completely removed, over time, the incentive to outperform others will diminish and lower the expectations of the individual to the level of the rest of the group.

    0
    #73836

    Hill
    Member

    This is all quite interesting, and I thank everyone who has taken the time to offer their thoughts, as well as those who may still respond. I’m not sure I can head off what — in my “doom and gloom” way of thinking — the metrics my company is contemplating: a minimum of 4 improvement projects a year ($250K each), certifying in a timely fashion (supposedly completing two projects about 90-120 days after finishing training), “time spent with champion” (don’t ask — I don’t understand it either), how effectively the BB leads a project team (no criteria for this yet), “delivering a high quality, specific, ‘doable’ control plan,” “quality” of project information put into the company’s database. See why I’m asking for advice?

    0
    #73840

    NASSIF TADROS
    Participant

    From my personal experience and studies, I believe that assessment for any issue or activity should be by OBJECTIVE, where improving the process using six sigma or any other methodology aims for improving the output being a product or/and service of the organization in order to achieve customer satisfaction that guarantees customer loyality and increase market share.
    Accordingly I suggest that we sould measure the BBs performance by: (improvement) of customer satisfaction…not customer satisfaction only in the form of measurable variants or being a function of lower number of complaints per year, the increase of sales volume to the same customer each year, and also the profitability from the sales to that same customer (and each customer too) with respesct to previous year/years.

    0
    #73851

    Mike Carnell
    Participant

    Sarah,
    If you read a lot of the message threads you can see that a lot of the consternation is because the Champions are either not engaged or just playing with the metrics.
    I am not sure how much input you have but I would get some metrics on the champions as well. I don’t really like subjective stuff but the HR types do this 360 review stuff. You might suggest that as a way for the BB/GB’s to be able to feedback on Champion performance.
    A mertic I like is when the BB/GB delivers a project. The Finance/Accounting people have put a annualized savings on the project. The Champion gets held accountable for the realization of the savings. It needs to be accountable to hit the number not a big pat on the back if they go over. If going over is good then they can understate it and drive it to be overstated in the end. Drive the accuracy. The minute the final report slides across the desl the clock starts ticking. Just like Domino’s.
    They can still use the BB for implementation. It is their metric.

    0
    #73870

    john beaudoin
    Participant

    Sarah,
    If I can learn more details about your situation, I may be able to help.  For example, are you a Blackbelt seeking what is fair criteria to be evaluated on, are you a manager looking to evaluate Blackbelts in your company, or other.
    Our facility has about 3 years experience with 6-Sigma.  I received a Greenbelt in April 2000 and have been acting as a Blackbelt without the formal additional 2 weeks training for a year now.  We have dedicated a department with one manager, one project engineer (myself), one project analyst (also a financial expert), and one parts engineer (lots of procurement experience).  This last fiscal year we saved 2.6 million dollars in hard and soft savings, which was our monetary target for the year.  We also completed about 24 projects during the year to do that.  Most all of our projects are trasactional with the largest savings in reducing/recovering shipping costs.  To accomplish this, we also had about 24 other Greenbelts.  Many Greenbelts could not complete one project, many others only completed one project, and many of those required us to complete the project for them.  In 0ur setup, our manager is a Blackbelt, however, acts more as a Champion than a Blackbelt.  Our Vice President, who is a Champion, communicates the Business Goals to our Manager, and our Manager chooses the projects to support the business goals.  Our Manager, who thus had the financial goals as a Blackbelt, passed the goals to us in terms of team performance, so we would then support the managers goals.  In our case, this worked and we were very effective, but I must say that at the start of the year, we didn’t think we could do it.
    In your case, you have not provided information as to how many Greenbelts you have supporting the Blackbelts.  I would also like to say that depending on the number to support ratio, it can be very difficult for Blackbelts to complete individual projects as Greenbelts often need a lot of support and push to get their projects completed.  I know it would be impossible for me to complete 4 of my own projects in one year to save $1 million dollars (Unless I really thought outside the box and generated a large new revenue stream for the company, which is possible, but…)  The big hang-up that I am sure almost everyone finds with any company is the lack of IS Resources.  Programmers always seem to be working on big non-6 Sigma projects for upper management or Corporate ideas and thus do not have the resources or time to schedule in items to complete projects.  They do eventually, but they always seem to be the bottleneck in getting anything done. (There ought to be a good project in that..)  As a result, for most lucrative projects, a 3 month cycle time is unreasonable, unless your company was so poorly run in the past that you can make substantial process improvements without these resources, which are usually needed to automate processes or to integrate new software with existing systems.
    If this is your company’s first time through 6-Sigma, you don’t what to count on the fact that newly trained Greenbelts will be getting 2 projects completed.  There is definately a learning curve, and it can be a slow process learning when to stop collecting data and implement something.  Also, you may find that individuals that were sent to training do not turn out to be the optimum candidates that you thought.
    On a final note, my Manager would make the case that having the financial goals be at least 20% of the evaluation is absolutely necessary to achieve success.  In addition, any assigment that he has been the leader of has always succeeded.  As a result, there is no reason to change what it working for him.  I will also state that every company, team, and individual is different and that what works for some individuals will not work for others.  Knowing what to do to motivate people involves getting to know your people.  You will need to do what is right for your people.  One thing you don’t want to do is demoralize everyone upfront with unrealistic goals pulled from a hat and then futher discouraging others from training on 6-Sigma when the word gets out among coworkers about the expectations after training.  Use solid logic in coming up with goals.  There should be some solid reasoning and known COPQ, realisting cycle times for known projects that will reduce COPQ, etc.  Consider limits on resources, % of time individuals can really spend on 6-Sigma away from regular job functions, etc.  Also, don’t base everything on the money.  Give separate credit for number of projects completed, etc.  If you have these sound goals and people work and then hit them, you will start a snowball rolling that will build in your company great improvements each year.

    0
    #73871

    john beaudoin
    Participant

    Your ideas work great in an ideal world, however, there is so many inputs to customer satisfaction, sales volume to customers, etc. that it would be really hard to prove that the improvements were soley based on a project you completed.  Competitors may have lowered their performance, and thus customers defected to your organization, sales may rise and or fall with the economy, weather can play a role (Remember Hurricane in Florida that set up all of the demand for lumber?).
    Projects should target customer satisfaction ratings, which will increase sales over the long run for a company, but to grade performance on this alone would not be fair.  What if the product you sell is like an 8 Track Tape Player and you have the few customers you have very excited about the quality and performance of your 8 Track Tape Player.  Does this mean you are doing a good job with your projects?  When you look at the sales data decreasing for 8 Track Tape Players every year, does this reflect negatively for the Blackbelt that was assigned to reduce the defects and increase customer satisfaction with the 8 Track Tape player?
    The bottom line is that you need a mix of financial results in both hard and soft dollars, and you need some portion assigned to customer satisfaction levels.  But you won’t be successful if you only do one of these measurements.  Using only one measurement would be like judging the performance of one company based on the Dow Industrial Average performs, or estimating what the weather is like by only measuring if it is Sunny or Not.  You need to look at temperature, wind speeds, humidity, etc. to get a true picture of the weather.

    0
    #73874

    Ryan Heinl
    Member

    Sarah,
    I think this is a great question, and not to make light of everyone elses comments but it seems like you posted this question on the wrong website. 
    I may be misinterrpreting your question but this sounds like a performance management question rather than a Six Sigma question.  Six Sigma is very focused on results, and the common theme I am reading in these replies is one that performance managment professions have been struggling with for some time.  How do we effectively measure individual performance and is it fair to use things like monetary gains in projects to do so.
    I think one tool of six sigma provides a good solution.  The balanced scorecard.   It’s important to look at a number of things when evaluating performance just like it’s important to look at a number of things when evaluating the effectiveness of a business.
    So, what does this mean.  Well, in addition to evaluating project success, I would evaluate team member satisfaction, client satisfaction (if there is one that you can identify), and then I would also evaluate the behavior of the BB based on identified successesful behaviors.  You started a list of them in your message.  Thinks like “Leadership,” “Coaching,” “Change Facilitator,” “Strategic Deicison Maker.” 
    In my line of work we call these things competencies or broad labels that describe groups of specific and associated behaviors.  Assembled around a certain job, you have a competency model.  You can use this model to evaluate BB’s or whomever.
    I’d be happy to tell you more about this strategy.
    Ryan

    0
    #73882

    Bill Johnson
    Participant

    It sounds like your company has the potential for SS success, but is also falling into the same trap as most others who take it on.  What I am referring to, is from one of your posts which stated that a reqmt. for BB’s at your company will be “a minimum of 4 improvement projects a year ($250K each)”.  How as this number arrived at?  There are two possible ways:
    1.)  Using SS methodologies, your company did a broad examination of itself and the processes, a “10,000 feet up” view.  Based on a variety of data, including P&L, scrap, efficiencies, benchmarking, etc. they established opportunities suitable as SS projects.  Then, based on data, they determined that, at least in the short term, there were suitable projects available to supply X number of BB’s with roughly 4 projects /yr worth $1M.
    2.)  Somewhere way up the ladder, an exec heard the BB’s could save UP TO $1M/ yr by doing 4-6 projects/yr.  They then looked at how much money they wanted to save and how much they wanted to spend up front and THAT determined the # of BB’s.
    On the plus side, it sounds like at least some members of mgmt have recognized that there is more than just an arbitrary quota that measures the value of a BB and their contributions to the organization.  That’s a good thing!  Everyone preaches about SS as being data driven, especially data-driven management.  Unless the savings quotas for BB’s are driven by data, the whole program is in jeopardy.  It goes back to the old Deming philosophies of giving the workers the tools they need.  You can’t make a Cadillac out of VW pieces.  Potential savings for a project are not up to the BB (although they certainly vary with the ability of the BB).  All a BB can do is use the tools, manage a good team, and be as thorough as possible.  This ties in to many of the “soft” measures you mentioned that are being considered.  A BB may be the best facilitator, troubleshooter, leader, and business-wise person in the company, but there is a maximum possible savings for any given project.  Unless the projects identified are the right mix of savings potential, effort and risk, $1M/yr MAY not be achieveable.  In that case the BB fails.  They then must accept the label or get creative.  This is when the SS effort loses all credibility.
    My advice is to do everything possible to convince upper mgmt  to make sure hard goals are achieveable.  Combine that with mgmt, peer and team review of each BB project and you may have a winning story.
     

    0
    #77394

    Jacob
    Participant

    Dear ,
    you can select as many performance measure for a BB as you want , but if there is no sponsor buy in ..or real commitment on the sucess of the project . it will be only window dressing !! i wonder .. shouldnt we measure .. sponsor support ????? or top Managment leadreship ???
    Six Sigma its not about how good, or how well the BB was trainned. a BB or GB , can do very little if there is no a visible leadership, and total sponsor commitment !!
    my opinion….
    MBB..frustated ..
     
     
     

    0
    #77407

    Mike Carnell
    Participant

    Jacob,
    The simplest way to eliminate the drive by support from Champion is to get some metrics on them. How many metrics does your company have on the Sponsors, Process Owners and Champions?
    If your top level management is not engaged ….. run. You shouldn’t be doing it anyhow. I recently spoke with a friend who was training a company (outside the US). The CEO sat in on the first 3 days and several members of his staff were there all 5. That is a huge commitment for a C level executive. The message to the Company is extremely strong. Look at how visible Boosidy and Welch were during the launch. Welch assigned Gary Reiner (CIO) to lead the implementation – a huge message.
    There are good companies and good executives out there. It is worth the effort to seek them out. If you like where you are then take responsibility for changing it.Set up an appointment with your CEO and explain your concerns. If you get fired it is a pretty good indicator you are working in the wrong spot. Best case the problem gets fixed.
    Good luck.

    0
    #94470

    Winston
    Member

    Sarah, it sounds as if your organization is results driven toward bottom line (top executives pushing down through engineering/project management/financial.  The focus should be on what the ultimate client needs are; hence, what “bang” are they getting for their “buck”.  In today’s tech environment, there are way too many virtual teams and staff to keep an eye on actual performance.  Pushing tech jobs offshore isn’t going to help Six Sixma measurements at all either.  Metrics must be clear and understandable, and transferable to scalable projects.  The problem I’ve discovered as a senior program/project manager with a global IT services giant is that little training is given before a technically competent person is “annointed” as project manager.  That’s when you hear the huge “GULP!” sound coming out of cubicles.

    0
Viewing 18 posts - 1 through 18 (of 18 total)

The forum ‘General’ is closed to new topics and replies.