iSixSigma

Breakthrough from DOE

Six Sigma – iSixSigma Forums Old Forums General Breakthrough from DOE

Viewing 9 posts - 1 through 9 (of 9 total)
  • Author
    Posts
  • #36888

    Bill Buck
    Participant

    Does anyone have any thoughts about how to quantify the potential from a DOE experiment.  What is the typical process improvement or move toward optimum?   Can you use a rule of thumb?   Even with a detailed Gap analysis and you know what the potential is, what can you expect?   In other words, how far from optimum do most people operate their processes when they have not applied statistical methods?   Has anyone done any study along these lines? —Bill Buck

    0
    #107442

    Robert Butler
    Participant

     In general running k factors one-at-a-time requries a k fold increase in the number of experiments that would be needed for a 2 level factorial design of the same variables.  Thus for 3 variables you would need 8 experiments for the design and 24 for one-at-a-time. If you chose to use a single experiment and make all changes from that point then the factor drops to (k+1)/2 but the number of experiments is still a lot more than the design. (pp.313 Box Hunter and Hunter – Statistics for Experimenters).
      So, one way to quantify the potential for a DOE vs not running one is the cost savings incurred by running fewer experiments. A k fold reduction in the cost of an investigation usually gets the attention of even the most laid back exec.
      I’m not aware of any way to estimate typical process improvement or moves toward an optimum and I think it is the wrong way to view the value of a design effort. If you have done your homework and have good justification for your variables of choice then the quantified value of the design is that of minimum expenditure for maximum information.
      When the design is successful (significant variables) the models developed from the effort will point in the direction of process improvement they will also give you a sense of just how much improvement you can expect. When a design is a complete failure (no significant variables) they also tell you how much improvement you can expect. If nothing is happening, over the ranges of the variables you have chosen, you may have the options of doing any or all of the following:
       a. If you have been controlling any of these variables to ranges less than those run – you are wasting time and money on needless control.
       b. If you have been putting in more than the minimum amount of any one of the things you varied – you are wasting money  – set them to the levels that will incure the minimum cost.
       c. If one or more of these variables has been viewed as the critical variable, the secret ingredient, the thing that gives our product a competitive edge, etc. The results of a failed design will allow everyone to revisit their beilefs and adjust them without fear of ridicule.
       d. In the future, if anyone insists that any of the failed variables really do matter, the burden of the proof is on them. Validated design data allows you and everyone else to move on to thinking about other issues instead of continually revisiting the same old things.
      Many times, the value of a,b,and c. will be greater than the amount of money you thought you could make by changing the process.  The value of d, in my experience, is priceless.
      I realize the above could be viewed as a pep talk.  In truth, it is a capsule summary of arguments I’ve made time after time when my managers have asked me to quantify the value of experiments that haven’t been run and whose results are unknown.

    0
    #107458

    Bill Buck
    Participant

    Robert,
    Thanks for the information.  I am like you and thoroughly convinced of the benefits of DOE.  I have had many successes already.  I always get asked by someone “skeptical” how much improvement or how many dollars will be saved?   The problem is that this is to some extent “research” and the outcome is unknown.  My best tactic is to keep citing case studies of successes.  
    I will incorporate some of your logic into my discussions.  I appreciate your candid and timely response.
    —-Bill Buck, [email protected]

    0
    #107472

    Robert Butler
    Participant

    I always find it odd that the same individual who would have no problem signing up for a nebulous research effort involving a vague, but inevitably large, number of wonder-guess-putter experiments will, when confronted with an offer of a DOE, demand insane levels of specificity and accountability from the person offering the design.  While arguments of the type I offered in my first missive will sometimes work with this type of individual, I’ve found a much better approach is to put them in the position of having to really think about their approach. It takes some patience and a little skill to keep the encounter from turning confrontational but it can be done and the end result is usually a blend of efforts which permits everyone to see the merits of the DOE approach
      The BIG question to ask is  – “How many experiments were you planning on running and why?”
      Most people have been trained to think in terms of one-at-a-time experimentation.  One-at-a-time experimentation is what got them their degrees, one-at-a-time experimentation is what their professors used, one-at-a-time experimentation is what got them to where they are today.  It works, and the idea of walking away from a known method of success is not something most people are willing to do. 
     In my experience, once we start talking specifics the single biggest concern is that my design doesn’t happen to include the “critical experiment” – the one that my investigator/manager/boss “knows” is going to absolutely positively work.  Usually, the individual will have a series of such “sure fire” experiments mapped in their head and these are really all of the experimental effort they have considered. 
      Get them to write them down and justify them.  Pull out your design and see how they fit.  If most, or all of these experiments fit inside the design space ask the individual for triage – which one of the proposed experiments are “the best”, which ones are “just in case” and which ones are “interesting questions”.  If the experiments are outside your design range, discuss how you might change the design to incorporate the ranges/variables of interest. Then offer a compromise.  Point out that their personal comfort with respect to the research effort is important.  Remind them that your design will cover the region of interest with the minimum number of points and suggest that their few “best” points can easily be run as part of the design effort (many times their entire list is so short that I’ve just included all of them). The final design won’t be optimal from the standpoint of least expenditure for information gained but it will give both them and you a sense of understanding of the process and it will provide a degree of investigator comfort that is well worth the extra money.  More often than not, adding these few favorites to the matrix gets not only complete buy in for the design but extremely active support for the work from all concerned.
      The benefits of this approach don’t stop here however.  Once you have run the design you need to put on your salesman’s hat and take advantage of the situation.  Run the analysis of the design without their favorite points – use the resultant equations to predict the outcomes of their points, rerun the design including all of their points, set the equations side-by-side and critique the similarities and differences.  With only the rarest of exceptions, I have found the results of the design were able to predict the measured response of the favorites.  Make sure you show this to everyone.  The confidence you will inspire in the value of your approach cannot be measured and it is very likely the next time you offer a design you will have little difficulty getting it accepted.

    0
    #107487

    lin
    Participant

    Robert,
    Thanks, I like the way you think about this issue and your approach has merit.  How do you approach the folks who think they are already where they need to be?  The one’s that don’t see a need for improvement? 
    —Bil

    0
    #107501

    Kim Niles
    Participant

    Thanks Bill and Robert:
     
    Interesting conversation.  One quick thought I have related to problem solving is that benefits are usually obvious so the trick becomes developing a test strategy / test plan that justifies the effort.  Also, remember that the benefits are not always known in advance such as when new discoveries are made or factors suspected to have little effect actually have big effects and visa versa.
     
    Good luck.
     
    Sincerely,
    http://www.KimNiles.com   
     

    0
    #107601

    Robert Butler
    Participant

      As a statistician, I play the odds.  For every person who does not see a need for improvement there are dozens who do.  When I’m confronted with an individual who insists on maintaining the fiction that improvement is unnecessary I thank them for their time and move on.  In every company I’ve ever worked for, people who are interested in improvement and who see the need for it, far outnumber those who seem to think they have achieved perfection.
      Obviously, this approach has its drawbacks and it also assumes you have the freedom to look elsewhere within an organization.  As a company statistician I am expected to provide support to anyone who asks for help. If no one asks, it is my responsibility to get out there, search for situations where I can be of value, and sell people on the worth of my services.  Rejection and statements of alleged perfection are part and parcel of the world of cold call selling but so are clients who, in the words of Monty Python, are “d!!!!d interested”. So, as I said, I play the odds and when I find someone who is interested I make it a point to really deliver.  Success is, of course, not assured but again, with the tools and methods I have at my disposal, the odds are in my favor.  With successes comes “water cooler” advertising and shortly after that unsolicited ad campaign comes a line of people requesting, even demanding, my support. I haven’t kept count but it is my perception that some of the first people to call after the initial successes have been recognized are many of the same people who earlier assured me there was no room for improvement. 

    0
    #107606

    Atul Bhatt
    Participant

    Bill,
    In my opinion, a DOE should not be performed for its potential benefits. A DOE should be performed to understand the interaction between simulataneously acting multiple factors, so as to optimize the interaction in order to achieve the desired process behaviour. The DOE itself will not cause an improvement, but merely set the stage for understanding what factors need to change and in what direction.
    The sale for the DOE itself, needs to be done on the basis of the business case prepared for undertaking a process improvement project in the first place, when you cannot explain the interaction effects between multiple variables using other statistical methods.
    Hope this helps you.
    Atul Bhatt

    0
    #107675

    lin
    Participant

    I think many of them are just trapped in their own paradigms.  Resistent to change even if they think there is benefit to be gained.   You have probably heard the excuses
    Cannot affort lost production, Cannot tolerate any poor test runs, etx.
    I think they must endure some of the pain associated with DOE to reap the benefit.  From my own experiments, I have found these poorer performing runs to be no problem.  If performance is so bad then they can be abandoned early and move on to the next.   Now you know not to go there again.
    I usually show an example of a predicted vs actual from a DOE and get them to focus on the improved or better runs in an experiment and compare it to the “norm” prior to the testing.
    —Bill

    0
Viewing 9 posts - 1 through 9 (of 9 total)

The forum ‘General’ is closed to new topics and replies.