# Very complicated problem- help?

Six Sigma – iSixSigma Forums Old Forums General Very complicated problem- help?

Viewing 25 posts - 1 through 25 (of 25 total)
• Author
Posts
• #30823

Jon Borg
Participant

Hi there,I have a question.I measured 1.0, 2.0, and 1.0.  The next day I measured 7.0 on the sameapparatus.I can only conclude that something has changed right? Possibly randomfluctuations, etc….   now the question is…..with what confidence can Iconclude that something has changed in my measurement system?I am assuming with *strong* assumptions, that I may use a t-test to determine the confidence……. but  I am really unsure.
can anyone help me out?
jon

0
#80847

Gabriel
Participant

Jon: Can you be more clear? Are those numbers indivdual readings or averages? Are them from 4 consecutive days? Are all those measurements done on the same individual part?
If I take exactly what you wrote:
“I measured 1.0, 2.0, and 1.0.  The next day I measured 7.0 on the sameapparatus.I can only conclude that something has changed right?”
I would say no, you can conclude other things. With this information alone, the first think I would suspect is that the last day you measured a part that was actually 7.0.

0
#80849

Dona
Participant

This is rather arbitrary excersise from a lab.  It not a real experiment.I the 7.0 measurement was assumed to be take on another day.  So basically,  How confident can I be that this was merely due to chance, or not? What assumptions can I make and with what confidence can I conclude that something really did change in the measurement system.  this question has me very confused…

0
#80853

Zilgo
Member

Based on the distressingly low sample size, you cannot really make any statements about the process.  Logic tells me that readings averaging 1.33 would not suddenly become 7 without some special cause variation, ie not random chance.  To make this conclusion based on so little data is basically a waste of time though.
A t-test is used when you need to see if a statistic is different from some value.  In this situation a t-test would be most appropriate.  Of course you only have three degrees of freedom to use in making this test, so it is unlikely that you will get usuable results from the test.  Put the four readings into Minitab and try it, you test whether the true mean is different from (1+2+1+7)/4 = 2.75.  You will find that your p-value for the test is greater than 0.05 (or any other useful significance level) implying that you can’t make any definitive statements about the data.

0
#80866

Eileen
Participant

Jon,
Forget the t-test. Not appropriate. I had a discussion with Dr. Deming on this very issue. He was very adament that the most useful and appropriate method for the analysis is a statistical control chart. If the value 7 is outside the control limits, then you need to take action. Somthing has in fact changed. If the 7 is within the control limits, leave it alone.
Eileen Beachell, Quality Disciplines

0
#80867

Ronald
Participant

Eileen
I agree, the control chart is going to give the best indication of whether a special cause is affecting the data.  Also, it may be important to know the Gage R&R for the system that is obtaining this data.  The measurement variability may be the problem not the process itself.  Gage R&R should be done before any data is taken however in order to understand the portion due to process and the portion due to measure.
My control chart shows with an Individual Chart of 1,2,1,7 looking at all 8 Western Electric Tests, that the 7 is still in control.  With more data points this may not be the case, but hard to conclude with what is given.

0
#80868

Anonymous
Participant

A control chart?  Are you suggesting calculating control limits with only 3 datapoints?????

0
#80869

Eileen
Participant

Anon,
Nope. Neither can you do any type of statistical analysis – including a t-test. Let me give my assumptions:
This is a standard that is being measured or at least a golden part.
There has already been a gage R & R conducted prior to actually using the gage
AND there is already a record of measures on the standard which could be the basis for a control chart.
Eileen

0
#80871

Anonymous
Participant

I am just concerned that the initial poster would take your reply and generate control limits with only 3 datapoints.    My recommendation is to continue to collect data and generate a control chart.  I believe that this is your recommmendation as well, although unclear in your message.

0
#80884

Zilgo
Member

What’s not appropriate about the t-test?  It is just as inappropriate as the control charts.  With four data points there is not a test or chart or process that will give you any usuable statistical significance.    Logically, going from 1,2,1 to 7 implies that this process should be watched and more data collected.  With a sample size of four, nothing is appropriate.  If you were going to do a statistical test on this data, the t-test would be the one to use, which is what he asked.
Control charts are based on the results of statistical tests anyway.  Control charts are helpful, but if you understand the tests and how the charts are made, then you don’t really need them.  I only ever use control charts as a visual for presentation purposes.

0
#80887

Zilgo
Member

As an appendix to my previous post:
Since control charts use standard error (not standard deviation, a common mistake) to create the control limits it won’t matter what the  fourth measurement in the dataset was.  Even if your data is 1,2,1,10000 you will still get a process that is in control on a control chart.  So control charts don’t help you here.
Admittedly, neither will the t-test.  You won’t get a significant p-value.  You want to know what to do with your data?  Get more of it.

0
#80889

Eileen
Participant

Zilgo,
I really don’t agree with you. First of all, why do you think Jon is comparing means? It is more likely with a measurement system there is one single value. There are exceptions to this but I would guess this his situation. So how do you use a test of means to compare one individual value to the avg. of 3? This is clearly an abuse of statistics.
You are so wrong about control charts being a statistical test. Control chart limits are economical and are placed at plus/minus 3 sample errors based on a lot more than mere probabilities. Shewhart’s book as well as Dr. Deming’s clearly talks about these issues. Control limits are not and never were probability limits and are not tests of hypotheses.(See Deming, “Out of the Crisis”, page 334-335).
The fact that you only use control charts only for visual presentation says more about you that the tool.
Eileen

0
#80895

Ropp
Participant

Cant we all just get along?

Jon,

The answer is simple.  Do a gauge r and r study on your measurement system.  If the variation is not found to be from the measurement system it must be from the manufacturing process, raws, or the environment.

0
#80897

Zilgo
Member

Jon wanted to see if the value of 7 was different from the other data he had collected.  So the idea with using a t-test is to see if the mean of your data (ie, all 4 numbers) is different from the hypothetical mean of the population (which is some value Jon would have to know or assume).  Is is a good test to use in this case?  No, but not because the test is wrong, it’s because you only have four data points.
Control charts are based on statistical theory.  99.73% of your data should fall between +/- 3 standard errors of the mean.  You are essentailly performing a statistical test: Ho, abs(data value – mean) > 3*(standard error) vs. Ha, abs(data value – mean) <= 3*(standard error).  Statistical theory plays a pretty big role in control charting.  So if you understand the test behind the control chart, then the chart itself becomes a visual tool.  Is it good to use a control chart in Jon's case?  No, but not because the chart is wrong, it's because you only have four data points.
The real crux of this situation is the fight between practical vs. statistical significance.  Can Jon find a statistical reason to monitor this process?  No he can’t, because nothing he does will ever have statistical signinficant results with such a small sample size.  Does he have a practical reason to monitor this process?  Yes, because his 4th data point seems pretty strange compared to his first three.  So put some data collection around it and when he has a sufficient sample size, then he can start doing hypothesis testing and using control charts.

0
#80899

Fernie
Participant

Jon,
With only 4 data points you can not conclude anything at all.
I agree with Dave, do an R&R and move forward from here. Ohh, and get a decent sample size.
Good luck,
Fernie

0
#80902

abasu
Participant

Jon,
You probably have to do more that a gage R&R evaluation and do a stability study here.  The suspect reading was observed on different day, so you need to evaluate instrument drift if any.

0
#80903

Gabriel
Participant

The problem is not very clear for me yet, but it seems to be as follows:
You’ve got a gage and one part. You measure it three consecutive times and you get 1, 2 and 1. Next day, you measure again what is supposed to be the same part with what is supposed to be the same gage and you get 7! The 7 looks far from the previous values. Can we statistically state that something speciall happened in the middle? That the 7 does not belong to the same distribution as the 1, 2, 1?
Yes, we can state that with some confidence level, high or low. So the question can be rephrased: At which confidence level can I state that the 7 does not belong to the same distribution as the 1, 2, 1?
Come on, boys. To get more data would be really the right thing to do. But this 7 looks really strange. It is true that we have too few data, but if it looks so strange we should be able to show that “strangeness” with some “more or less acceptable” level of confidence (ok, not 99%). Please help me on the following reasoning because I haven’t been using these concepts for a while.
I have the population “the measurements results using the gauge A on the part B”. This population is unknown, but I have a little sample of three results: 1, 2 and 1. The Xbar is 1.33 and S=0.58. Xbar and S are estimators of the population average and standard deviation.
Let’s assume normality (this assumption is usual for MSA) If we assume that the population average is really 1.33 and the population sigma is actually 0.58 then I would know, for example, that about 95% of the measuremet results would fall within ±2sigmas i.e. 1.33±2*0.58=-0.41 to 2.31. Further more, half of the remaining 5% falls below -0.41, so a value grater than 2.31 would be delivered by this distribution only in 2.5% of the trials. For all practical prupposes, 7 woluld be NEVER delivered by this distribution, so we would conclude that it doesn’t belong tho this distribution. If the distribution changed, there must be a special cause.
Of course, we don’t know the “actual” values of the average and sigma, we only have the estimators Xbar and S based on that little sample. How higher than Xbar can the atual average be? It will be somewhere between Xbar ± t*S/sqrt(n). For n-1=2 degrees of freedom and an alfa 0.10 (90% of confidence) t is 2.9. That means that, with a 90% of confidence, the true average will be between 0.36 and 2.3. Again, from the ramining 10%, 5% goes to below 0.36, so we have a 95% of confidence that the true average will not be grater than 2.3. We could do the same with the sigma and get a “upper confidence limit” with a 95% of confidence, only that I don’t remember how to do it. Just to continue with the reasoning lets say that this limit is 3*S, what would be 1.74. In this “worst case”, with an average of 2.3 and a sigma of 1.74, the “average + 2 sigmas” limit goes to 5.78, what means that a value grater that 5.78 would be delivered by this distribution only in 2.5% of the trials.
So, with only 1, 2 and 1 we know that:
– The average of “the measurement results using the gage A on the part B” is below 2.3, with a 95% of confidence.
– The standard deviation (sigma) of “the measurement… ” is below 1.74, with a 95% of confidence.
– Even if we had the bad luck of having actually those high values for both the average and sigma at the same time, any measurement result would be below 5.78 with a 97.5% of chance.
I don’t know how to put all this together, but there must be some statistical way to prove, with some more or less acceptable level of confidence, that the value “7” does not belong to the distribution where the values 1, 2 and 1 where taken from. So come on, statisticians: At which confidence level can I state that the 7 does not belong to the same distribution as the 1, 2, 1?

0
#80916

RonJ
Member

Zilgo,
You seem to be using enumerative statistics to study a cause system which calls for analytical statistics. That seems to be a common theme here, depending on where you were trained; I would suggest reading Dr’s Wheeler, Deming and Shewhart. For this problem I agree with all … get more data. Good Luck.

0
#80920

Blank
Participant

What if the next three values measured were 20, 21, and 20?  What would the conclusion be then?  The problem with saying the average is 1.33 is that you have no idea what the true distribution looks like.  We may be looking at the lower tail of a distribution.  Does a measure of 7 for the 4th data point mean a special cause as occurred.  There is no real way to know until mature data is found or unless a historical mean is know.  I might show that yes it is different than the first 3 points, but to make a process correction based on this would be rash.
If this is a new process/part then I would not make any conclusion without at least an adequate sample size.

0
#80930

Sinnicks
Participant

Warning Will Rogers!  Danger!  We’re getting into a debate about angels dancing on the head of a pin.  Even if we would agree with the statistical analysis, does it make sense from a practical sense?  And, will the nonstatisticians believe in it?
Let’s start from scratch.  Work the process.  The steps:

Gage R&R (one day)
Gage Stability (does the gage change with time and conditions)
Identify potential causes of the variation (if it is not the gage, what could it be?)
If there are a number of potential causes, narrow them down and simply record what else you can.
Prove what are and are not the Major Variation Drivers.  Prove with sound methods and data.
Fix the Cause(s)
Implement the Controls to prevent a relapse.

0
#80945

Jon Borg
Participant

Measured day 3 and day 4 – results are:
day 3: 8,6,9
day 4: 5,2,3
Where does this take us now?

0
#80949

Zilgo
Member

Where it takes you is a place where using tests and control charts makes a lot more sense.  Now you have 10 data points, that’s better.  I wouldn’t start doing anything until you have about two weeks worth of data.  Then when you say something about the process, you will be able to back it up with data.

0
#81334

Intrepid
Participant

While I myself am a dedicated user of control charts, in this case I disagree with depending upon a control chart…your sample size is MUCH too small to calculate limits.  Your best bet IS the t-test, although either way you have too few samples.

0
#81335

Intrepid
Participant

Again, I have to disagree.  The reason for placing limits at +/- 3 sigma IS a statistical principle based on the probabilities…99.73% of the data should fall within +/- 3 sigma in a normal process.

0
#81336

Dr. Steve W.
Participant

your MS? If not, you might consider doing so. If yes, do you have enough historical data on this same process? If you do, control chart will tell you whether you are dealing with a unstable process. If you do not have much historical data, there is not much you can do: two sample t won’t work as you only got one data point on Day 2. I-MR chart is the only chart but with only 4 data points, your Beta risk is very high. Hope this is helpful.

0
Viewing 25 posts - 1 through 25 (of 25 total)

The forum ‘General’ is closed to new topics and replies.