iSixSigma

Statistical Control and DOE

Six Sigma – iSixSigma Forums Old Forums General Statistical Control and DOE

Viewing 14 posts - 1 through 14 (of 14 total)
  • Author
    Posts
  • #32272

    BenR
    Participant

    How important it is to ensure a process is in statistical control before running experiements on the process? It seems to me that it is a vital requirement, but I have heard others say it is not necessary. I could easily have missed something along the way and just want some cogent feedback. Thanks.

    0
    #86007

    Mike Carnell
    Participant

    Ben,
    This question typically starts a fecal maelstrom.
    If you don’t run it how do you intend to get it into Control? Actually if you don’t run it how will you even know?
    I run into more “quality” people to use this as an excuse to sit on their buts and tell the world how screwed up they are. They see it not in control and when they continue to run and work the process they run to their office and tell everyone they told them not to do it because it wasn’t in control so it isn’t their fault there are defects.
    If you read Mario Perez-Wilson’s book “Six Sigma” we had a process that was not in control and we had about 76 inspectors on the job getting it under control. We ran it and worked on it. In about 9 months time we had balanced the line and the number of inspectors was 30 and dropping. The next product we launched was an alternator for the bomb fuze. Mario got a shot at a lot of up front work and brought it into production in pretty good shape. See if you can find a copy of his first book “Machine Process Capability.”
    Your worrying about philosophical stuff that is irrelevant. Go get involved. Turn the thing on and see what it does. If it runs great do something to it and see how well it runs – then you will start to know how robust it is.
    Good luck. 

    0
    #86010

    Et Al
    Participant

    ROFLMFAO Mike

    0
    #86017

    Mike Carnell
    Participant

    WTF?

    0
    #86236

    BenR
    Participant

    Mike,
    Thanks for the feedback. When I go after a process I typically don’t turn to DOE until we clear out the underbrush first. But some here start with a screening experiment as a first or second step. I guess if it works, it works, but I feel uncomfortable jumping directly into an advanced technique. It may be that as I get older I am becoming too cautious (I’ll be 60 in a couple of months). Your response gives me something to think about; I may be waiting too long to pull the DOE trigger. Thanks.
    BenR

    0
    #86239

    Mike Carnell
    Participant

    BenR,
    Unless you have interactions try the hypothesis tests. Less intrusive.
    Good luck.

    0
    #86243

    rams
    Participant

    I always stress this point during my DOE training… before you collect data for experiments, make sure process is in control. I illustrate this using the “catapult.”
    I shoot the ball using a low and high start angle and record the data. Participants are surprised that the lower angle produced a longer distance. This is not logical.
    It’s because when I set the catapult at the higher angle, i pushed the ball to hard into the cup resulting a lower distance. This is a case of lack of SOPs which makes the process out of control.

    0
    #86247

    B
    Participant

    rams-
    Absolutely Excellent! 
    So in the mean time while your waiting for your processes to move themselfs into control so they can be fixed you can actually continue teaching people how to fix stuff.  OK, cool!

    0
    #86261

    Ronald
    Participant

    I see this as short sighted because I would have included how hard the ball is pushed into the cup as a DOE variable for experimentation.  If there are procedural or noise issues that have high impact, they need to be addressed. 
    I agree with the other posts, if I wait for the world to look the way I want it to before I try to understand it, I will be waiting forever.  I must try to learn how to improve the world while noise, interactions, and poor process control exist so I can make my process more robust to these issues and make the world look more like I want it to.

    0
    #86262

    Marc Richardson
    Participant

    Usually I agree with what Mike C. has to say and his contributions to this forum have been significant and substantial, but in this case, I must take issue with him strenuously.
    The process must be stabilized before more advanced techniques, such as Hypothesis Testing, ANOVA, Correlation, Regression and DOE can be successfully applied, otherwise we run the risk of interpreting special cause variation as a statistically significant result. One of the purposes of a DOE is to raise the random variation (noise) of the process to signal strength in order to observe the effect of the process x’s on the product y’s.
    Also, the amount of variation that the measurement system contributes to the over-all process variation must be minimized, hopefully to 10% or less, for the same reasons.
    Further, one of the best ways to determine whether your solution has the desired effect on the product characteristic of interest is to observe the change in the key metric on your control chart. Hypothesis testing merely takes a snapshot of the process at a point in time. A control chart will , among other things, provide a long term history of the variation of the process, showing us when a statistically significant shift has taken place as a result of our actions and allowing us to determine whether the process stays fixed.
    Why aren’t we training people to use the 7 Q.C. tools and 7 management tools to solve problems before they apply six sigma methods? Are we rabbit hunting with an elephant gun?
     
    Marc Richardson
    Sr. Q.A. Eng.
    JAE Oregon
     
     

    0
    #86265

    mcleod
    Member

    1. BenR says “How important it is to ensure a process is in statistical control before running experiements on the process?”
    More important than stability itself, is understanding the nature of the instability. That is, how many out of control points might I expect to see in the DOE. Even if I cannot control or fix them, do I know what is causing them so I can see and remove them from the DOE when they occur, etc
    2. BenR says “I feel uncomfortable jumping directly into an advanced technique.”
    There is nothing more advanced about a DOE than other tools such as ANOVA, Regression, etc. If you don’t mess with the Xs you will never improve the Y.
    3. Carnell says “Unless you have interactions try the hypothesis tests. Less intrusive.”
    Mike is wrong here. If you are going to intrude into the process to test the effect of one X on Y you might as well test the effects of 4, or 5, or even 15.All other things being equal, testing 5 ideas is no more intrusive than testing one, and you get more data for the cost of the DOE (however that cost is defined). As Mike suggests though, do pay attention to the effects of interactions in the design and know how to handle them.
    4. rams posts a concern regarding SOPs not being in place before the DOE.
    I agree with rams to a point. It is always better to have SOPs for the process before a DOE. However, in probably 60-70% of the DOEs I have conducted, the process ends up exhibiting LESS overall variation rather than more. This basically indicates a LACK of SOPs in the process before heading into the DOE. Giving operators only two options (factor settings) to put each of the 15 knobs (factors or Xs) while holding all other knobs constant results in less variation than the multiple levels of multiple knobs they were using before the DOE. This is actually a nice little bonus I have sometimes gotten out of DOEs.
    5. Lee and Carnell both make the excellent point that you have to get off your hands at some point.
    6. Marc says “The process must be stabilized before more advanced techniques, such as Hypothesis Testing, ANOVA, Correlation, Regression and DOE can be successfully applied, otherwise we run the risk of interpreting special cause variation as a statistically significant result.”
    I disagree with this. The advanced techniques he refers to CAN be, and often are, used to investigate processes having special cause variation present. You must know how to identify special causes in these analyses (called outliers) and remove them from the data to see the real relationships.
    I apologize for the length here. In sum, yes you can and should use DOE to effectively and efficiently identify what Xs impact the Y and how much, EVEN when the process has special cause variation. Use your advanced Black Belt knowledge to handle outliers. ALWAYS use replication to aid you in identifying special causes in the DOE (again outliers).

    0
    #86300

    Mike Carnell
    Participant

    Marc and Scott,
    First – thank you for the compliment Marc. I like the stuff you post as well. You have a lot of patients with a lot of people who have evidently not put much effort into figuring something out. I don’t know how you do it.
    As far as the “being in control” is concerned – I have never bought into it. It doesn’t make any sense. Why does it matter if the process is under control when you are not running the DOE, Hypothesis test, etc. It does need to be stable when you run it – even if it is artificially stable.
    Look at your response. Your fear is that some assignable cause will creep into the process and confound itself as an effect. When you design the DOE, as you said,  you need a qualified measurement system but for more than the response variable which is what most people qualify. If you have done a decent design job and you know what variables you are testing and what variables you believe you are holding constant and you assure that with a qualified measurement system you may have a risk but as I have said in the past I will accept some of that risk. If you validate your findings – as you should – the probability of having the same assignable cause imprint itself on the correct treatment combination to get the same result twice isn’t even in the realm of reality.
    Scott – lets go to the other thread about culture which is in most cases just people and company politics. Yes you can test more variables with a screening design – academically. When you run the DOE you do start and stop the process as you set up treatment combinations. That means you will have to negotiate for time which we all know is fun. Now you owe the person a favor and information. How do we increase the probability of a successful DOE and reduce the amount of time I need to be on the line? That is why Analyze comes before Improve – when you run a DOE without doing some screening up front you are buying your information at minibar prices. You take out the variables that don’t mean anything with a little hypothesis testing before you start designing the DOE. When we are doing hypothesis testing we do not have to create treatment combinations and we don’t have to interupt the process. You produce information from data (as opposed to what the Production Manager will have in most instances). When you negotiate for the time to do a DOE you can demonstrate what you have done to minimize the size – if you don’t want to reduce it and want to run replications – which will reduce the probability of an assignable cause creeping in undetected – spread out the respose variable and nobody notices – Marc you know enough about this stuff that  you don’t believe that will happen. Does anyone believe we should be doing the “deer in the headlights routine’ becuase they belive an assignable cause will imprint itself on exactly the same treatment combination in a randomized, replicated design. If anyone believes that please wear a little aluminum foil on your head so we know who you are.
    If we fall down some cosmic bunny hole and land in some statistical nirvana we will probably have processes in statistical control and unlimited time and resources to run DOE’s. If you don’t have chemical processes involved, you are lucky if you have a legitimate need to run DOE’s on 10% of a class of Black Belts. John Hathaway who is as inclined to run DOE’s as anyone I have met – and is probably as good as anyone around at doing them – will give you a number of about 15%. We have a process that teaches hypothesis testing before DOE for a reason. We teach MSA before both of them and it wasn’t done serendipitously. It is a process that builds on knowledge.
    So who is right? Lets go back to the culture thing – we are different – we believe different stuff and we all have fixed some stuff so nobody is right or wrong. We will probably all get there sooner or later.
    Marc I was a little diappointed in the 7 basic tools comment. Are they useful tools? Yes. Should we use them? Yes. You did mention shotgunning? Remeber the way we taught the 7 basic tools? No particular methodology just a collection and you used whatever felt good that day. If you want them maybe we can get Rams to document an order for us.
    Ben you have enough variation in your inputs that will let you pick which ever one makes you most comfortable.
    As for Rams and his SOP. I think Ben’s original question was that the machine was new. You want to generate the SOP’s based on? Reminds you of the old saying “I come before you to stand behind you, to talk to you about something I know nothing about.” Rams I am glad there are people in this world like you (not meant derogitorily). It needs to be documented and I have very little proclivity to do it.
    Good luck.

    0
    #86310

    mcleod
    Member

    Ben,
    I think you can now conclude don’t worry about the special causes in you DOE. Deal with them (using the Six Sigma tools you learned in class) and move forward and learn.
    Mike,
    There is nothing “academic” about running larger (more variables) screening designs. Rather, more powerful is how I would describe them. Do you need to know what you are doing when running them? Certainly. Is the risk worth the reward? Absolutely!
    You ask: “How do we increase the probability of a successful DOE and reduce the amount of time I need to be on the line?” To me, the time challenge exists regardless of whether you are running a 4 factor DOE or a 7 factor DOE. Put briefly, the number of replicates (and therefore time) is determined by sample size needed to conduct the study, not the number of factors. If I can learn something about 7 factors in the same time it takes to learn about 4, I consider that a good deal.
    And you say: “You take out the variables that don’t mean anything with a little hypothesis testing before you start designing the DOE.” Certainly all of us agree with this. The challenge is how? More often than not, the problems I face is historical data on the X variables of interest does not exist. Or when there is historical data, it is so multi-collinear that we can’t figure out what does and doesn’t matter. Or, we know about all there is to know about the current Xs in the process and their relationships to Y, so we have to look at the effects of NEW Xs for which there obviously is no data (this is rarely the case however).
    So, then someone says “hey, why don’t we fiddle with the knobs and see what happens?” Makes sense to me. But, then you get this intervention problem you referred to where you have to “… start and stop the process as you set up treatment combinations.” However, such is the case with hypothesis testing on historical settings or a DOE. That is, if we have to stop the process to turn the knobs, then we just do. So if we are going to fiddle, let’s fiddle in such a way we are gonna make the best sounding music. This is when I pull out the large screening designs. I do not run these things in such a fashion that perfect orthogonality and balance must be achieved (to me, that IS academic). Rather by merely trying to follow the pattern of a screening DOE we are able to get a good (and always better than hypothesis testing with historical data) idea of what does and does not matter. And to lower the cost, we try to make treatment changes at naturally occurring opportunities (cleaning, shifts, stops for other reasons, random assignement to different teams, etc.). You said: “That is why Analyze comes before Improve – when you run a DOE without doing some screening up front you are buying your information at minibar prices.” That is why I use a “nasty” screening DOE. To accomplish separating the trivial many from the critical few (screening), and then confirm what we think we see with the “perfect” and academically pleasing full-factorial DOE. I have achieved countless benefits using this approach.
    Except for the pseudo-DOE I describe above, I agree with you and John H that they are seldom needed.

    0
    #86319

    Mike Carnell
    Participant

    Scott,
    Neither of us will be moving our position any time soon. We are both making it work so we keep on keepin’ on. As long as we are delivering results to the people who are paying us.
    I gotta get packing. Safe travels.
    Good luck.

    0
Viewing 14 posts - 1 through 14 (of 14 total)

The forum ‘General’ is closed to new topics and replies.