2^31 Experiment
Six Sigma – iSixSigma › Forums › General Forums › Tools & Templates › 2^31 Experiment
 This topic has 16 replies, 5 voices, and was last updated 8 years, 10 months ago by Sam.

AuthorPosts

August 13, 2012 at 7:55 am #54145
I got the following results for Y when i ran this experiment. Minitab cannot calculate the P value, found out there is no SS for error. My question is, why there is no SS for errors? Is it normal for experiments with fewer runs to have zero ss for error? How can all the variation explained by the factors chose? Does the results suggest something wrong i did on the design or analysis of the experiment?
1 +1 1 1.7
1 1 +1 3.2
+1 1 1 2.1
+1 +1 +1 3.60August 13, 2012 at 8:59 am #193907Clarification: I tried to ran ANOVA on two variables (A and C) after found out the effect value of B (the middle one) variable is very low compared to others.
0August 13, 2012 at 11:01 am #193911
Chris SeiderParticipant@cseider Include @cseider in your post and this person will
be notified via email.You have a fully saturated design. Remove the ABC interaction when you using “Analyze Factorial Design” within Minitab…you will see pvalues BUT be aware you have confounding so the factors you find statistically significant may be the confounded factor or a combination.
0August 13, 2012 at 7:38 pm #193913Thank you Chris Seider. I understood that for saturated designs there is not enough degrees of freedom to calculate the statistics. In this case even if you do ANOVA with only two factors, A&C, minitab cannot calculated P values because error sum of squares is zero. Not the same problem of saturated design
0August 14, 2012 at 12:44 pm #193924
Chris SeiderParticipant@cseider Include @cseider in your post and this person will
be notified via email.I did a quick analysis and found Factor C to be statistically significant with an alpha of 0.05. Either I didn’t fill out your randomized design correctly or ….
I always us an initial value of alpha of 0.15 or 0.25 to do an initial screening of which factors to remove and then use an alpha of 0.05 after the first analysis.
0August 14, 2012 at 1:40 pm #193926
MBBinWIParticipant@MBBinWI Include @MBBinWI in your post and this person will
be notified via email.@Peach – I haven’t looked at your data, but if you have no error (the values are perfect outputs to the regression) then you will not get a p value. You can either do repeats or get a better measurement tool to get more decimal places to more definitively identify the variation.
You have also done a halffactorial on 3 factors, with no repeats, so you’re already confounded. I would expand to a full factorial and if that doesn’t clear things up, then adding repeats would be in order.
0August 15, 2012 at 2:04 pm #193931
Joel SmithParticipant@joelsmith Include @joelsmith in your post and this person will
be notified via email.A couple of observations that will hopefully help…
Think back to middle or high school algebra, where you were given two data points and asked to find the line connecting them us y=mx+b. When you fit that line, it hit those points perfectly. You have the same situation here but with an extra dimension…you are fitting a line to so few points that the equation for that line is able to hit each of them perfectly. You need more data to perform tests.
Also, I noticed that your factor C equal A*B and therefore is indistinguishable from the AB interaction. This is because you used a fractional design. Maybe you’re aware of this already, but if you think the AB interaction may be meaningful then you should use a design (like the full factorial) that allows you to differentiate the two.
Like any test, the more data you have the more power you have to detect something. Four data points is enough to do a mathematical fit but won’t likely find any significance unless the effect is so great that you likely didn’t need DOE in the first place.
0August 15, 2012 at 7:20 pm #193933Thank you Joel Smith. It make sense. This is what i thought of the perfect fit issue, lack of enough data points. Thank you again.
For your information. This was the first experiment we did so we did not want to do many runs. We are going to do more experiments considering what we learned from this.
Thank you0August 17, 2012 at 6:17 am #193936
Robert ButlerParticipant@rbutler Include @rbutler in your post and this person will
be notified via email.Since you have already run a saturated half rep for a first look I’d recommend replicating any one of the runs in the design and running the analysis on the five data points to see what you can see. Obviously there is the issue of running a replicate at a later date but if you are strapped for time/money this is one way you can take advantage of what you have already done.
In the future, you should always plan on replicating one or two of the points in your experimental design so that you will have a guaranteed measure of error.
0August 18, 2012 at 11:42 pm #193940Thank you Robert. This is what we are planning to do. I am trying to figure out how to analyze the new design using minitab. It is not easy to analyze the new design in Minitab. I will post the results here..
Did you ever ran a food related experiment? We are doing one and there is high variability among the response (set of people rate the taste of food). We thought we did enough work to select the best people to rate the change in taste.
0August 19, 2012 at 12:56 am #193941Joel Smith: As you mentioned we always liked more data points and the argument was if we need enough confidence we need more data. We have this new boss who does not allow us to run full factorials (resources constraint). He is like, company spent a lot of money to train us and now he need us to understand the relations in the shortest experiments (budget) as possible. He introduced this new budget for experiments etc. It is actually good as we are all learning new things.
0August 19, 2012 at 6:30 pm #193943
Robert ButlerParticipant@rbutler Include @rbutler in your post and this person will
be notified via email.Let’s back up and revisit your description of your experiment.
In the initial post you gave the impression you had run 4 experiments and had exactly 4 results. Your most recent post suggests you’ve done something quite different.
You said “Did you ever ran a food related experiment? We are doing one and there is high variability among the response (set of people rate the taste of food). We thought we did enough work to select the best people to rate the change in taste.”
The answer is yes and the questions are:
1) If the measurements are ratings of taste what kinds of ratings are they?
2. What, exactly do the response values in your first post represent ? That is what is 1.7, 3.2, etc.?
3. If the four experiments represent food that is being rated then do you have the same group of people rating all four food types or do you have separate populations of people for each one of the food types (we’re assuming the experiments are actually receipe changes).If you are using the same group of people to rate all four of the food types then you do not have independent measurements – you have repeated measures and that requires a very different approach to your analysis, and if the taste ratings are Likert scale (ratings from 15 or whatever) then you need to forget the averaging and use the raw rating numbers.
We are now about 4 “if’s” removed from what might actually have been done so rather than continue in this vein I’d appreciate it if (that’s 5) you could answer the above questions.
Given that it is Likert rating and given that the same people rated all 4 receipes there is a simple way to analyze this kind of data and you may find that you already have all of the measurements you need. However, before describing this I’d prefer to wait for your reply to my questions.
0August 20, 2012 at 3:31 am #193944I am sorry Robert. I just realized I asked questions on two different experiments in the same post. The food experiment i was talking is completely different from the experiment i posted here. About the experiment I posted. We only ran four runs, we are going to run two more additional runs next week. I find it difficult to analyze designs like this, 4+2=6 runs, in Minitab. Right now i am experimenting with Minitab.
Regarding the taste experiment, i will post the design and rersults in a separate post. FYI, You are right, we changed the receipe and gave all the different reciepe to the three people who tasted it, randomly. The experiment was 2^41, total 8 runs, with 4 blocks (blocks were different rice used, assumed to be an inert factor). What we used is a likert rating for 3 tasters, y1,y2,y3. Each person tasted each receipe twice meaning y1 y1, y2 y2, y3 y3 for each runs
0August 20, 2012 at 5:27 am #193945
Robert ButlerParticipant@rbutler Include @rbutler in your post and this person will
be notified via email.Ok, thanks for the clarification.
On the first experiment – I trust that one of those two additional experiments is a genuine replicate of one of the original 4. If it isn’t it should be.
On the second experimentassuming the tasters were blinded (the usual situation in tastes tests is to make sure the tasters do not have any visual cues so, since various types of rice do indeed look different, the taste test should have been run in a manner that kept the tasters from looking at what they were tasting – like running the test in a dark room).
The other issue with taste tests is the need for a washout period so I hope the testing was spread over several days. Assuming you ran the taste test correctly then what you have is 16 responses per taster. Three tasters is on the low side (we used to run a minimum of 6) but if the experiment is done and gone then you’ll have to use what you have.
As mentioned previously, the taste panel results are repeated measures and you will have to find out if Minitab can handle data of this type.
What isn’t clear to me is your statement “The experiment was 2^41, total 8 runs, with 4 blocks (blocks were different rice used, assumed to be an inert factor).” If I read this one way I get the impression that you ran 4 different rice types but if I read it another way I get two rice types. If you can clarify this point I may be able to offer some additional thoughts.
0August 20, 2012 at 11:04 am #193947Thank you Robert for your help.
I was not planing to run a replicate of the original design. All this time i was thinking to run a combination that is not included in the original design and the idea was may be this will bring more information.I have not done anything yet. Can i know why you said it should be a relicate of original design?For the taste experiment we did not blind the tasters. We discussed about it but the difference in receipe is not visually identifyable so we decided not to blind
You are correct, we have 16 responses per tasters.
I do not understand why this would be repeated measures. Remember all the tasters tested all different combinations randomly. Aren’t these replicates?
We ran 4 different rice types, all white color rice. The plan is if there is no difference we will go with the rice that cook fast…
Thanks again for the help0August 20, 2012 at 12:30 pm #193950
Robert ButlerParticipant@rbutler Include @rbutler in your post and this person will
be notified via email.If you had run the two additional points at the same time as the original 4 then you would have had some assurance that the various unknown/uncontrolled factors exhibited the same behavior during the course of the experimental run. Since some time has passed since you ran the first 4 experiments you have zero assurance that those unknown/uncontrolled factors are going to behave/exist/interact in the same way they did for the initial run. A replicate of one of the original design points, when used in conjunction with the original point, will give you a measure of the “pure error” of the augmented research effort in the face of changing condtions of unknown and uncontrolled variables and thus will guard against a declaration of significance where none really exists.
For the taste test: The smallest unit of independence is the individual taster. Anything within a taster is a repeated measure. This is the same problem you have in medicine where you take a group of patients and test their responses to multiple forms of medication (say, for example 4 different types of medicine to address a particular medical condition). If you can get a copy of Regression Methods in Biostatistics by Vittinghoff, Glidden, Shiboski, and McCulloch through interlibrary loan there is an excellent discussion (with example) of this issue on pages 254259.
Since you ran 4 blocks on an 8 point fractionated design you have some additional issues. The principal block for that kind of fractionation is 1, ab, ac, bc, ad, bd, cd, and abcd. If the blocking was such that 1 and ab were with rice type I, ac and bc were with rice type II, ad and bd were with rice type III and cd and abcd were with rice type IV then you have perfect confounding of rice type with factors C and D – in other words you cannot tell if the cooking time was due to either of these factors or due to rice type. You will have a similar problem only with different factors if you blocked in some other fashion.
In order to assess the effect of rice type you will have to figure out your confounding pattern and run a repeated measures model on rice type and whatever two other factors (and their two way interaction)are clear of rice type.
As a stopgap you could follow the method described by Vittinghoff and set up a two way ANOVA with tasters and the 8 combinations with two measures per combination. This would deal with the repeat measures nature of the data and give you an assessment of differences between the 8 combinations but it would not address correlations between the rice type, the two factors clear of rice type, and cooking time or any of the taste ratings.
0August 21, 2012 at 12:20 am #193955Robert: Thanks a million for all the help.
I understood the importance of repeat measure now. I will follow your suggestion of including runs from the first design. I will post the 2 runs here before running to get a feedback.For taste test: After reading your explanation i see the need for learning more about analysis of repeat measures. I will get a copy of Vittinghoff’s book.
Regarding the blocks, we knew about the aliasing structure. Since the rice is assumed to be inert factor we went ahead with this design. Eyeballing the data i saw in paper (I did not lead the experiment and I don’t have the electonic copy of the experiment yet) the repeat measures of tasters differ a lot and this is what i was saying in one post before, tasters themselves vary a lot. I think you will understand and able to help more after I post the data here.
Thank you0 
AuthorPosts
You must be logged in to reply to this topic.