iSixSigma

Is Computer Simulation Data Valid for a DoE?

Six Sigma – iSixSigma Forums General Forums Methodology Is Computer Simulation Data Valid for a DoE?

Viewing 12 posts - 1 through 12 (of 12 total)
  • Author
    Posts
  • #55398

    Anibal Villatoro
    Participant

    I want to optimise settings for five factors involved in the design of a physical object.
    I do not have access to a real experimental facility, so my results would come from a computer simulation. I’ve been told that this would not work in a DoE because the simulation takes out all natural variability. Is this true? Could I make it work by adding more centre points and corner points to the experiment design?

    0
    #199813

    Strayer
    Participant

    As I’ve seen the term “natural variability” used it refers to the difference between the test site and the actual site. That is, you will not get exactly the same results at a different site, whether physical or simulated, because they are not the same. This variability exists whenever you do DoE unless you do it in the actual production environment, which may be impractical. When we do a DoE in a pilot plant or a lab, or a simulation, we know it is not the same as the actual production environment.

    0
    #199815

    Nik
    Participant

    There may be options, but it depends on what is the output (Y variable) you want to optimize?

    0
    #199819

    MBBinWI
    Participant

    @alfavictor1 – The purpose of a DOE is to establish the relationship among the variables. If you have a simulator, you already have that relationship, so no need for the DOE.

    You might want to look at taking your simulation and adding a Monte Carlo aspect to it. This will allow you to add the variation for the variables which can give you insight into how the input variation affects the output variation. That might prove insightful.

    0
    #199822

    Chris Seider
    Participant

    @alfavictor1

    A simulation of a production line MIGHT have process variability modeled into it but they are all approximations. I agree with @MBBinWI but I have seen an interesting case where a process was evaluated with various changes (e.g. see if more manpower or a second processing unit at a bottleneck would be better) to see if one was much better than another but it wasn’t a real DOE.

    0
    #199823

    Anibal Villatoro
    Participant

    Thank you all for your replies. I don’t think I’ve explained the problem well enough.
    The simulation is not of a production line. The simulation would be of a windtunnel.
    The x’s would be 5 design parameters that describe the shape of an object to be tested in the virtual windtunnel. The y output would be the drag coefficient of the object.

    The relationship between the inputs and the output is not currently known, the DoE would yield the relationship and help find optimal settings for the input parameters to find the best possible output.

    A real windtunnel would have variability, i.e., if you repeated the experiment with exactly the same settings several times, the results would vary slightly. But the simulated windtunnel would yield the same result every time – hence no ‘natural variability’. This is what concerns me.

    Would the results from the simulation still be adequate for a DoE?

    0
    #199824

    Robert Butler
    Participant

    Thanks for the expanded description of what you are trying to do. Given what you have and given that these design parameters are continuous variables I’d recommend running a full 2 level factorial with a single center point and limit the model terms to main effects and two way interactions. This way the error will be based on the 3,4, and 5 way interaction measures. If you try to simulate error in a replicate you can push your results all over the map – too small and everything is significant – too large and nothing is significant…and everything in between.

    0
    #199826

    Nik
    Participant

    Also investigate Response Surface DOEs. A typical DOE models the range of levels in the experiment and you select the optimal value based on those corner points. A response surface DOE helps you find the optimal settings through conducting a series of experiments and designing each successive experiment based on the outcome of the previous experiments. You might find it faster than attempting to experiment and model an entire system.

    0
    #199827

    Sergey
    Participant

    @alfavictor1
    Is your computer simulation based on formula? If yes and you know it, in this case you may just check the terms in the formula and choose the method:
    – No quadratic, cubic, etc. terms – take simple full factorial design without central points and replications
    – There some quadratic, cubic, logarithm, etc. – use RSM.

    If there is no formula (or you don’t know it) you may start with screening DOE or full factorial with central point in order to explore the object. And then plan other research (in case of curvature – RSM).

    0
    #199829

    Anibal Villatoro
    Participant

    Thank you all for your further replies.

    The set of equations the CFD solver uses to come up with a solution are far too complex to use directly, instead you normally run the simulation and check that the result makes sense.
    Robert is right – input variables are continuous.
    Therefore, a 2-level full factorial with centre-points with main effects only seems like a good approach as an initial experiment, as Robert said, then a response surface DoE to find the optimal input parameter settings, as NiK and Sergey suggest.
    I won’t try introducing simulated error yet – I’ll only try it if time allows!

    Thanks!

    0
    #199831

    Joanne Wilkinson
    Participant

    Hi,

    I don’t know what software you are using, but the CFD software that we use has an option for setting up and running DoE within the software, which helps shortcut the process.

    The important thing to remember with this sort if DoE is that it is only valid if you have properly modelled the system. You said that you don’t have access to any experimental facilities, but do you have any actual measured data?
    Comparing known data points in these types of simulation software (CFD, FEA etc.) is really important to make sure the model is as good as you can get it before you start.

    0
    #199836

    Mike Carnell
    Participant

    @alfavictor1 There is a guy named George Box. He has done a few DOE’s in his time. Remember his famous quote “All models are wrong, but some are useful.”

    0
Viewing 12 posts - 1 through 12 (of 12 total)

You must be logged in to reply to this topic.