Gadgil
@Vinayak  Reputation - 0 Rank - Aluminum
|
Is there a thumb rule which talks about how many data points(outliers) can be removed to establish the control limits using a control chart?
|
bbusa
@bbusa Reputation - 0 Rank - Aluminum
|
zero
|
Gadgil
@Vinayak Reputation - 0 Rank - Aluminum
|
Can you elaborate your answer a bit? What do you mean by zero?
|
bbusa
@bbusa Reputation - 0 Rank - Aluminum
|
by zero I mean ’0′
|
Mikel
@Stan Reputation - 0 Rank - Aluminum
|
Remove outliers only when root cause is known and has been eliminated.
|
SOS
@SOS Reputation - 0 Rank - Aluminum
|
I think one can remove 20% of the Total Data Points, when Control Limits are arrived for the first time.
Please correct me, if I am wrong.
|
Brar
@gsb Reputation - 0 Rank - Aluminum
|
The rule is very simple:
- If you know the root cause and understand that it was a one time occurance and would not be repeated; you can remove the outlier
- If you know the root cause and can’t decide if it is a special cause or not; mostly it is your process which is behaving as such; you should not remove the outliers.
Theoratically; you can remove a maximum of 20% data points as outlier; however that should not be taken as a rule and convienently remove outliers. The above should be exception not rule.
ALso another one thing: NEVER REMOVE OUTLIER CONDUCT A STATISTCAL TEST AND THEN REMOVE OUTLIERS AGAIN. The reason is tendency to stop when you want to see what you like.
|
bbusa
@bbusa Reputation - 0 Rank - Aluminum
|
The only rule I have been taught is
“Thou shalt not tamper with data” .
The data belongs to the process , not to you :S
|
Jonathon
@Jonathon Reputation - 0 Rank - Aluminum
|
Great quote! Do you know its origin?
|
Jonathon
@Jonathon Reputation - 0 Rank - Aluminum
|
I should clarify my question. I am not asking about the tamper quote. I am inquiring about “The data belong to the process, not to you.” Is that of your origin, or did you hear it from somebody else? I’d like to use it, and I’d like to give credit where due.
Thanks!
|
bbusa
@bbusa Reputation - 0 Rank - Aluminum
|
Thanks . The quote is very much my own. :cheer: :cheer:
|
Alan
|
@gsb
I’m very glad I found this page. Do you know of an academic source that I can reference that backs up what you said about when to include outliers?
I agree with your statements, I just need to persuade some colleagues.
Here’s a quote from your comment:
“The rule is very simple:
- If you know the root cause and understand that it was a one time occurance and would not be repeated; you can remove the outlier
- If you know the root cause and can’t decide if it is a special cause or not; mostly it is your process which is behaving as such; you should not remove the outliers.”
|
Robert Butler
@rbutler Reputation - 2150 Rank - Silver
|
Quality Control and Industrial Statistics 4th edition – Duncan – Chapter 31, section 1.4, Outliers, page 706.
“Occasionally in a set of individual values or a set of means there is one value that appears to differ considerably from the others. The tendency among those not trained in statistics is to discard these exceptional values as not belonging to the set. The statistician prefers to follow a policy in such cases for which he can calculate the probability of wrong decisions. Of course, if a value is KNOWN to have some characteristics that differentiates it sharply from the rest other than being simply an extreme value of the set, then this value can be discarded on the grounds that its inclusion will destroy the homogeneity of the set. Such decisions must be based on sound knowledge and must be exercised with great caution.”
|
Brian L
@brianleonard67 Reputation - 15 Rank - Aluminum
|
Great discussion. It is so frustrating to hear people automatically claim a data point is an outlier simply because they don’t like the data point. I try to never throw out any data, but I do know may use the old outlier formula if they use box and whisker plots (Mean +/- 1.5*IQR)with anything falling outside of that range considered an outlier. But even with the formula, I tend to include all data, period. But if anyone were to make a claim a point is an outlier make them use the statistical formula to prove it. Better yet, convince them to do as was suggested earlier…keep it all because it was an observation of the process.
|
May 28, 20100
Brian L 32 days ago.
