iSixSigma

Evaluation of Six Sigma

Six Sigma – iSixSigma Forums Old Forums General Evaluation of Six Sigma

Viewing 47 posts - 1 through 47 (of 47 total)
  • Author
    Posts
  • #43300

    John1982
    Participant

    Hello together,my task is to develop a framework for measuring the implementation of six sigma in my company.
    I think it`s quite simple to figure out the quantitative implementation status, such as the ratio MBB/BB or projects/savings per BB, but how can I measure the acceptance of the method?My plan is to do a survey in my company. I first want to ask the BB, figure out there pro`s and con`s to six sigma and there experience in the projects. Has anyone experience with that? What would you suggest? And what should I do additionally?Thanks for your answers in advance,John

    0
    #137195

    Eugene Jacquescoley
    Participant

    John,You have an interesting task at hand. My initial thought in terms of survey development is a cross-sectional with Likert Scale forced responses. This would allow you to perform a 2-tailed T test, when examing the mean and determining statistical significance @ p

    0
    #137197

    Darth
    Participant

    I seem to recall that a t test is to be used for continuous data.  Are you suggesting that a Likert scale survey could be considered continuous data if for example the poster used the common 1-5 scale?  I also seem to recall that one of the underlying assumptions is that the group data should be normally distributed.  If so, is ANOVA the best tool to recommend knowing that in most Likert scale output, especially a 1-5 scale, the distribution across the scale will likely not be normal?

    0
    #137211

    John1982
    Participant

    Thanks for the answers. I think a Likert scale should be good for the survey.
    Do you know any literature or do you have some suggestions, what I can ask them and in what categories?
    John

    0
    #137226

    Ritz
    Member

    Likert scales are great for descriptive purposes, but they are NOT suitable for t-test, ANOVA, regression, etc.  As Darth mentions, they are not normally distributed.  Also, the gage itself typically has very poor repeatability over time due to the tremendous amount of influencing factors that can not be captured in a survey. 
    Make inferences from the data you receive with extreme caution.
    Ritz
     

    0
    #137227

    Dr. Eugene Jacquescoley
    Participant

    Darth et al,Thanks for responding and your questions are valid. X & Y metrics can each be continous or discrete. A continous measure will have reading on continous scale where a mid point has meaning. A discrete measure, as you’re aware is categorical in nature. This is why I suggested the use of a Likert Scale to collect responses from John 1982’s survey. In terms of the T-test. The method is appropriate for John’s analysis. The Two-Sample t-test is perfect for examing overall customer satisfaction (e.g., MBB’s job satisfaction, etc.). Moreover, One-way Anova and Means Matrix do muiltiple pairwise comparisons. This is analysis is more realizable with the use SigmaXL 4.21. Keep in mind there is also the test for equal variance and Welch’s ANOVA, which also can be considered. Regards,
    E

    0
    #137228

    Dr. Eugene Jacquescoley
    Participant

    Ritz,Your response is somewhat correct. However, the distinction is what type of t-test John plans to utilize with measuring the responses from the Likert Scale. As I mentioned before, X & Y metrics can be continous and discrete…and of course caution must be used when inference or generalizing results to the public. This is why we have Welch’s ANOVA and the Bartlett Test.E

    0
    #137246

    Dr. Darth
    Participant

    Dear E,
    Putting a Dr. in front of the name doesn’t give your responses any more credibility then when you posted it without.
    To your points:
    1.  A forced choice Likert Scale will typically be categorical and thus discrete regardless of whether you use words or numbers.  Even if you use a scale of say, 1-10 instead of the typical 1-5, the responses are still discrete.  If you suggest a continuous scale from 1-100 and the responder can pick any value, you might start to approximate some continuouos data but I see no rationalization as to why you claim that a forced choice Likert scale is continuous in nature.  Possibly you can explain further.
    2.  Based on the above, I see no explanation that would justify the valid use of a t test.  Again possibly you can explain your position.
    3.  Welch’s ANOVA is used when the assumption of equal variances is violated.  But, it is only relevant for continuous data.  The use of Test for Equal Variance again relies on continuous data with Bartlett’s test having an assumption of normality and Levene’s test for any continuous distribution regardless of normality.
    4.  My position is that possibly the Chi Square or Two Proportion test might be more appropriate for analysis of responses for discrete data.
    I look forward to more illumination on your position.
    D

    0
    #137253

    Dr. Eugene Jacquescoley
    Participant

    D,That isn’t a very nice reply and I wasn’t aware that I had two seperate screen names. Nonetheless, I stand by my responses to John & Woo.Kind regards,
    E

    0
    #137259

    Darth
    Participant

    E,
    Hey, I’m the black caped evil doer so why would you expect a nice reply?  Check out the first reply you gave, it was sans Dr.
    Guess this thread calls for another Forum Wide debate.  I hope the crowd weighs in and may the best Dr. win.
    Instead of just standing by your responses, how about picking up my challenge and explaining your positions.  I gave my positions so how about being less dogmatic and make a case for yourself so that I can possibly learn something.
    D

    0
    #137262

    Eugene Jacquescoley
    Participant

    D,I’m with a patient now. But I will be happy to par in this lively debate.E

    0
    #137265

    luke skywalker
    Participant

    Well, before you get tooooooo wrapped up in your advanced degrees, would it perhaps make sense to remember what the question was?
    Seems this guy wants to build a survey, probably using a 5 category Likert style scale (hopefully with definitions for each anchor) and would like to know what makes BBs tick.
    The use of ANOVA or t-tests is common – probably more so from a rationalization of ‘that’s what we’re familiar with’ rather than it being the most correct approach. Since it’s a survey with limited responses, technically, Darth is correct (yes, that hurts) in that the data will most likely not be normal and is not continuous, or really even interval in nature. I always thought Welch’s test handled ANOVA based issues where there were either grossly different sample sizes or unequal variances (generally a p-value less than 0.01). The intent is there though – to assess the difference in responses.
    Maybe all of your advanced degree programs covered issues like alpha and beta risk? What adverse effect is produced by using non continuous, non normal data (haven’t even addressed sample sizes yet) in a test that explicitly states normality as an assumption ( t-test, ANOVA and Bartlett’s)? What is the reduction in power of the test in that case, and how could that affect the business once decisions are made and programs are either launched or scrapped?
     

    0
    #137268

    Feli Znaj Vidad
    Participant

    Hi.  Apparently the answer to your question is this:
    “No.  No one here has experience with this.”

    0
    #137270

    Heebeegeebee BB
    Participant

    Or it could be a distinct lack of interest in the topic (or what has become of it…)

    0
    #137274

    Darth
    Participant

    Thanks for the response Luke.  You are correct on Welch’s test. 
    We haven’t even gotten to the point of sample size.  You are correct that issues of alpha, beta, precision and confidence will all have to be considered.  The research questions need to be addressed first…what do you want to know.  Then the survey design can be done and validated.  Sample size and sampling methodology should follow.  Finally, the appropriate analytical process should be employed to draw conclusions from the data.
    Hope all is well and that being the Ring Master for the Circus is as much fun as you imagined.

    0
    #137277

    Darth
    Participant

    Hey, at least the issues are objective and statistical in nature rather than the abstract opinions during the Burns/Carnell thread.  So, what is your opinion?  Are continuous tools like t test and ANOVA appropriate to evaluate forced choice Likert Scales?  That was the basis of the initial response.  Was the response of Box Cox and Histogram an appropriate response for the question from Mr./Ms/Mrs. Woo? 

    0
    #137280

    Feli Znaj Vidad
    Participant

    t-test and anova have no business in the same room with Likert scale survey responses…..period!
    (And anyone who would invite them to the same party is a flatulant buffoon)

    0
    #137284

    Eugene Jacquescoley
    Participant

    Luke,
     
    You could be on to something here.
     
    E

    0
    #137285

    Mike Carnell
    Participant

    Darth,
    If you want someone to return your call you need to TURN THE PHONE ON. At least have voice mail so those of us that call the turned off phone can let you know we called.
    “….at least the issues are objective and statistical in nature rather than the abstract opinions during the Burns/Carnell thread.” Abstract what is abstract. He published something that said he could fix 95% of problems with the 7 Basic Tools. My offer was to let him take a look at the cert projects from the last deployment and see if he can figure them out. The way I see it is once he gets to 3 that he can’t figure out he is below 95% and that makes his claim BS. Technically it is 2 but I am willing to give him the benefit of the doubt. Actually I can hand pick 4 and it is “Turn out the lights. The party’s over. They say ….” (Dandy Don).
    Just cause you dug your self a big hole with Dr. Jeckyl don’t drag me into this. I just want to watch. You’re good at this stuff – you’ll be fine. It is that schedule that has you on island time. Get your butt out here with the rest of us and get some high blood pressure. As Barbara Wheat used to say “If you aren’t livin life on the edge you’re taking up to much space.”
    Here’s an idea since none of these debates ever happen – lets do a team thing. Dr. Jechyl picks a team and you pick a team. It will be just like watching those high school bands at football games in the 60’s.
    Just my opinion
    Regards

    0
    #137288

    Eugene Jacquescoley
    Participant

    Feli,
     
    I see you “folks” are pretty rowdy. I suggest you refer to the email etiquette guidelines.
     
    E

    0
    #137290

    Emporer Palpatine
    Participant

    The Son of Skywalker is wise.

    0
    #137291

    Mike Carnell
    Participant

    Dr. Jacquescoley,
    Thank you so much for that. I am new to this site and I appreciate the help. Was there a part of that code in particular that you thought we need to aware of?
     

    0
    #137292

    John Noguera
    Participant

    Likert data is ordinal. The “technically correct” analysis with this type of data is Ordinal Logistic Regression and/or Kendall’s Coefficient of Concordance and would be the tools of choice for a Statistician.
    Treating Ordinal data as continuous is however a reasonable simplifying approximation.  Therefore use of T-tests and ANOVA are not incorrect.

    0
    #137294

    Heebeegeebee BB
    Participant

    Uh yeah, i’m new too.   Please fill us in about email (Sic) ettiquette.   They have email here?
    wow, cool!
     

    0
    #137296

    Heebeegeebee BB
    Participant

    Stats aside, I’d measure the bottom line impact of completed projects in dollars.   couple that with a discrete measure of projects generated over a rolling year.   That should be suffiecient.
    Statistics are great, but does the Sponsor or Customer want a statistically rigorous approach, or a quick and dirty measure?

    0
    #137297

    Heebeegeebee BB
    Participant

    Likert should be fine for this app.   Also, I really like Feli’s name.
    Merry christmas to you too!

    0
    #137298

    Rowdy Yates
    Member

    Buddy it’s a good thing you dropped the “Dr.” salutation.     Everyone knows a Doctor of Osteopathic medicine isn’t a real doctor.   Isn’t that a lot like a chiropractor without the manipulation skills?    Darth was just about to be all over that.  You could see him winding up to pitch his famed two-fingered slider with heat when you made comment regarding taking time out to see a patient.
     
    Now Darth on the other hand is a REAL doctor, albeit with a PhD in a non-rigorous interdisciplinary program from some state school in the Everglades.  But he did have as committee chair the father of all things quasi-qualitative and customer focused – no, I didn’t mean Dr. Harry, I meant Dr. Deming…  
     
    Besides aren’t DO’s granted from institutes versus universities?   Isn’t it a lot like Doctor of Divinity bestowed from a Divinity School?  
     
    Anyway, I’d suggest that you just back slowly away from a confrontation with Darth regarding pseudo-technical interdisciplinary subjects like applied basic statistics used in quality management programs.   He is after all a trained consultant and you’re not quite a doctor.     

    0
    #137300

    Feli Znaj Vidad
    Participant

    While it is true that I have a working, electric vacuum in my hall closet, I like to use a cement  trowel to clean my carpets because it gets them approximately clean.
    If there are tools designed specifically for the correct technical evaluation of ordinal data, why would one EVER confound the situation by misapplying the tools designed specifically for the evaluation of continuous data? 
    That would be blatantly flatulant buffoonery!

    0
    #137306

    The Warden
    Member

    E.You know those guys from Texas can be pretty rowdy. I suggest you address your outstanding warrants before they send “The Dog” after you.http://www.ci.garland.tx.us/warrants/war-j.htmThe Warden

    0
    #137310

    John Noguera
    Participant

    As an analogy, the electric vacuum would be the common tool used by a general practioner.  The advanced tools would be akin to the tools used by a professional  carpet cleaner.  
    See http://www.upa.pdx.edu/IOA/newsom/da1/ho_levels.doc
    “Common PracticeAlthough Likert-type scales are technically ordinal scales, most researchers treat them as continuous variables and use normal theory statistics with them.  When there are 5 or more categories there is relatively little harm in doing this (Johnson & Creech, 1983; Zumbo & Zimmerman, 1993).  Most researchers probably also use these statistics when there are 4 ordinal categories, although this may be problematic at times.  Note that this distinction applies to the dependent variable used in the analysis, not necessarily the response categories used in a survey whenever multiple items are combined (e.g., by computing the mean or sum).  Once two or more Likert or ordinal items are combined, the number of possible values for the composite variable begin to increase beyond 5 categories.  Thus, it is quite common practice to treat these composite scores as continuous variables.”

    0
    #137314

    Darth
    Participant

    John,Thanks for contributing to this discussion.  A couple of comments though:
    1.  I believe the original post dealt with evaluating a survey and looking for differences rather than trying to predict something.  If there were some continuous variables that we wanted to use as a prediction of the Likert results then I agree Logistic Regression is the way to go.
    2.  If the Likert scale is the usual 5 and as typically happens, does not fully cover the full width of five you now are left with 3 possibly 4 categories.  Your posted article indicates that the use of any continuous assumption becomes “problematic”.   If the scale is 7 to 10 and there is a pretty good distribution across the scale and somewhat of a symmetrical look to the data, then I agree the continuous assumption might be a simple way to go.   If not, there is no basis for claiming that the assumptions of the parametric tests are even close to being met.
    3.  If we are into posting links to articles that support our assertion, here is my contribution – http://iea.fau.edu/fair/romano06.pdf
    4.  Bottom line is why fool around with the continuous approximation if there are more than enough nonparametric tests available for analysis.  With the software today it makes sense to do it right.  Certainly in the good old days when we did all this by hand, doing the calculations for the t test would have been easier than doing a chi square. 

    0
    #137316

    Darth
    Participant

    Whoa, checked Carnell out and there are none for him.  Checked out my real name….yes Darth is a pseudo name….and there are 3 outstanding warrants for me.  Let’s keep this between us.

    0
    #137317

    Darth
    Participant

    You’re brutal, man but gotta love the creative humor.  Bet you didn’t now this about our man:
    Birthday: October 7, 1964
    Location:New York
    Interests: Football (US), Music, science and drug discovery
    Occupation: Gerontology, Geriatrics and Structural Biology
    Favorite Vacation Spot:  Aruba

    0
    #137330

    John Noguera
    Participant

    Hi Darth,
    Thanks for your post.  Good comments regarding the scale.  Your reference paper does, in the conclusion, allow that the t-test may be somewhat robust.
    Ordinal Logistic Regression can also be used for comparisons if you dummy code the predictors.
    The challenge is what does one teach at the  Green Belt level?

    0
    #137332

    M. Christine Mass
    Participant

    Why talk about this in the middle of summer?Mary

    0
    #137333

    Gene Geelbells
    Participant

    I agree.   This is a complete waste of time and illogical to discuss in May.
    Gene

    0
    #137334

    BTDT
    Participant

    Gene:Given the usual traffic on the forum, Six Sigma for projects and implementation plans has devolved from DMAIC to CCCV (Control-C, Control-V).Cheers, BTDT

    0
    #137336

    Mike Carnell
    Participant

    M. Christine Mass,
    Summer begins June 21 at 7:26 am EDT. This is not the middle of summer.
    Good luck

    0
    #137345

    Feli Znaj Vidad
    Participant

    Yes.  Complete waste of time and illogical.
    We here at the isixsigma forum have demonstrated via long history a decided preference for measurably incomplete wasting of time. 
    Once we step even one toe into the domain of the evil of complete wasting of time, then what might next be allowed? — complete sentences?  complete thoughts?  complete nonsense?!!  I say here and now that we can not and and will not allow any of this!!  None of this chicanerous rebellion!!
    Now is the time for all logical technilogicorrecticians brothers to band together to form an impervious, united front guarding our lives, our wives, our cubicles, our fragile perfect world from invasion by the dreaded Complecians who would bring to us only complete chaos!!
    Let us draw the line in the sand right here right now!!  Sleep easy tonight brethren because, as God is my witness, I shall never be hungry again and there will be no more talk of Christmas in May!!  At least not on my watch!!
    Sincerely,
    Feli Znaj Vidad

    0
    #137353

    tottow
    Member

    AND WE HAVE A NEW WINNER!!!!!!!
     
    Fro Heweih Nacten
    (AKA tottow)

    0
    #137355

    Gene Geelbells
    Participant

    Evidently the “Gene Geelbells” flew right by you….
     
    Gene Geelbells, Gene Geelbells, Gene Geel all the way! O what fun it is to ride in a one-horse open sleigh……
     
    That was irony, my friend – I applaud your celebration of Christmas throughout the year and will forever point to your little iSixSigma.com postings and your clever forum ID as my colleagues gather around my well and colorfully illuminated office and snidely sing “Oh Little Town of Bethlehem” and “Silent Night” under their collective fetid breaths in the sweltering heat of August.     

    0
    #137357

    Eugene Jacquescoley
    Participant

    Rowdy Yates,I don’t have time for this type of dialogue.E

    0
    #137358

    Eugene Jacquescoley
    Participant

    How did you get this information?E

    0
    #137359

    Detective Baker
    Participant

    He performed, as we also have just done down at the pokey, a Google Boolean search on the terms:   “Eugene Jacquescoley” +felon
     
    But you probably don’t have time for complex searches like that… being on the run and all.
     
    Detective Baker

    0
    #137361

    Darth
    Participant

    I AM DARTH, EVIL MASTER OF THE UNIVERSE, I KNOW ALL!!!!

    0
    #137394

    Lorax
    Participant

    John,
     
    In answer to the original question…
     
    Agreed. One way to see how the installation of Six Sigma to a particular organization is going is to do some surveys, constructing and analyzing the results in such a way that you are able to derive information which tells you that things are generally thought to be going OK or otherwise.
     
    And/Or.
     
    A group of measures, if well put together, will yield good information too. They may also give an indication as to which particular area is causing a problem. The best grouping of these I’ve seen tracked the following over time:

    Number of candidate projects ready to be started
    The total money saved due to 6S projects
    Number of trained BBs
    Number of projects closed
    There were a total of six.
     
    This may be another way to do it
    Lorax

    0
    #137398

    Heebeegeebee BB
    Participant

    Let us not forget or fallen brother, P. retti GeKerstadagen.
    He fought the good fight…

    0
Viewing 47 posts - 1 through 47 (of 47 total)

The forum ‘General’ is closed to new topics and replies.