# degrees of freedom

Six Sigma – iSixSigma › Forums › Old Forums › General › degrees of freedom

- This topic has 45 replies, 41 voices, and was last updated 11 years, 9 months ago by TonyK.

- AuthorPosts
- May 30, 2002 at 2:14 pm #29542
Need some help… I have been perpetually confused by the concept of degrees of freedom and the best way to explain it. Perhaps I am not alone. I’ve search for decent explanations that could be passed on to GBs but have never found one that worked. Given that “n-1” and other df references appears everywhere…I know I need to find an acceptable explanation of how and why it’s important. Taking it as a given just doesn’t cut it. Would appreciate some input. Thanks

PB

0May 30, 2002 at 2:40 pm #75952PB-

You could find entire chapters devoted to the topic, but the simplest and best explanation for GBs that I have seen is this: degrees of freedom are the equivalent of currency in statistics – you earn a degree of freedom for every data point you collect, and you spend a degree of freedom for each parameter you estimate. Since you ususally need to spend 1 just to calcultae the mean, you then are left with n-1 (total data points “n” – 1 spent on calculating the mean).0May 30, 2002 at 2:55 pm #75955

James A.Participant@James-A.**Include @James-A. in your post and this person will**

be notified via email.PB,

Mathematician I’m not, but this works OK for me:-

You are a traveller in search of knowledge, each day you need to move from one place to another. Luckily for you, it takes no time at all to travel the distance, so you only ever have to decide where you want be, just to be there.

Each day you start with a list of ‘n’ options of where you can go. This includes where you are now, but being adventurous, you never do that, so although it’s important to know where you are – you only have ‘n-1’ options available out of the total population of data.

It’s importance is in the value of knowing what happens in the whole population of data that is being considered.

Does this help – or merely confuse even further? Is there a mathematical type out there who can improve on this?

Regards

James A.0May 30, 2002 at 3:03 pm #75956

James AParticipant@James-A**Include @James-A in your post and this person will**

be notified via email.Breizh,

Sorry, just read your posting – it works better than mine.

James A0May 30, 2002 at 10:49 pm #75979Breizh,

I like it! thanks for the help – I will use this in an upcoming GB class. Thanks.

PB0June 6, 2002 at 1:22 pm #76143

Lee RonaldParticipant@Lee-Ronald**Include @Lee-Ronald in your post and this person will**

be notified via email.Another way of thinking about degrees of freedom is to image that you ask 100 people to pick a number between -infinity and infinity so that the mean of all the numbers is exactly zero. 99 of the people can pick any number they want, but the 100th person HAS to pick a specific number. He is not “free” to pick what he wants. Thus the degrees of freedom to determine the mean for 100 samples is n-1 or 99.

That’s how I learn to grasp it.0June 6, 2002 at 2:37 pm #76156The degrees of freedom are related to the sum of squares ( SS)

SS= sum[ (yi-ybar)^2] when i is from 1 to n

Since, sum[yi – ybar] = 0 when i is from 1 to n , consists the sum of squares of n elements. These elements are not all independent. As a result, only n-1 elements are independent, implying that SS has n-1 degrees of freedom. That’s the mathemathical explanation.

0June 6, 2002 at 9:56 pm #76174

GabilanParticipant@Gabilan**Include @Gabilan in your post and this person will**

be notified via email.Thanks all because this is one of the concepts that I hadn’t clear for myself and I always was afraid to teach DOE ’cause of a misunderstanding of the concepts involved.

0August 28, 2003 at 7:06 pm #89360

Akhilesh GulatiParticipant@Akhilesh-Gulati**Include @Akhilesh-Gulati in your post and this person will**

be notified via email.Perhaps a physical depiction will help clarify this

concept.

Take a piece of rope and hold the two ends.

Now twirl it (like you did in high/junior school

gym) so that it starts moving in the middle (like a

skipping rope). You had two nodes and when

you twirl it you find that the rope is able to move

at one point only. It has the freedom to move at

one point only! It has two nodes and one

degree of freedom. Now ask someone to hold it

at the center point thus creating another node;

you now have 3 nodes. If you try twirling it now,

you find that the rope can be twirled at 2 points.

Three nodes, two degrees of freedom!! This might be a real basic explanation and

mathematicians might disagree with me, but it

beats any other way of explaining Degrees of

Freedom.I was also asked a question along these lines

at one of my training sessions, while we were

looking at Regression. n-1 crops up there as

well. Again, as I explained the basis of

Regression, drawing physical lines and why we

use ‘distance squared’ I showed them how we

draw one line between two points, two lines

between three points, four lines between five

points . . . and the point seemed to hit home.I hope this helps. I too am always on the

lookout for better ways to explain concepts.0September 26, 2003 at 12:34 pm #90343

Liz CavanaughParticipant@Liz-Cavanaugh**Include @Liz-Cavanaugh in your post and this person will**

be notified via email.Thank you Akhilesh Gulati and others for your explanations on degrees of freedom. My question now turns to “what do you do with the number?” Is a high number better than a low number? Do I look for a range that it should be within? Is it to be compared to other values for the test?

Thank you.0October 12, 2003 at 10:26 pm #90925

kosatkaParticipant@kosatka**Include @kosatka in your post and this person will**

be notified via email.ok..i have this kind of quetion: you have got double spherical pendulum. How many degrees of freedom it has got? I think that four…but I am not sure…

can you tell me and write a way you find it out?

Nina0December 4, 2003 at 7:00 am #93198

Akhilesh GulatiParticipant@Akhilesh-Gulati**Include @Akhilesh-Gulati in your post and this person will**

be notified via email.You could think of Degrees of Freedom as the amount of

information available for the estimation of variability. For

example, the degrees of freedom to estimate variability from

one observation would be zero, i.e., it is impossible to

estimate variability. But for each observation after the first,

you get one addtional degree of freedom to estimate variance.

Hope this helps.0December 10, 2003 at 6:03 am #93403

Shaun X. LiMember@Shaun-X.-Li**Include @Shaun-X.-Li in your post and this person will**

be notified via email.Each data point adds one DOF. Each constrain (or ‘relationship’ or ‘equation’ such as ybar=sum(yi)) reduce one DOF. Find the data points number, then minus the constrains number, you get the number of freedom.

e.g.: for SSt, it contains all n data points, with one constrain (ybar), therefore, DOF = n-1

Constrains and combination of constrains always hold one DOF.

e.g.: (ybar+std.dev) has 1 DOF.

That’s my understanding.

0December 10, 2003 at 1:08 pm #93409

Mark AlmeterParticipant@Mark-Almeter**Include @Mark-Almeter in your post and this person will**

be notified via email.The best explanation I’ve heard for understanding degrees of freedom in various statistical calculations is as follows:

Degrees of freedom of n – 1 is required when taking a sample from a population because when taking the limited size sample, you have only a very slight chance of picking the extreme data values of the population. In order to accomodate for this, you subtract the value of 1 (n – 1) in order to inflate the std deviation and make the standard deviation calulation more adequately represent the parent population.

Notice that in the calculation of the population std deviation, the sum of squares is divided by the value N and not n – 1. This is because all values in the poplulation (including extreme values) are taken into account.

Just my 2 cents…..

Mark0December 10, 2003 at 1:40 pm #93410

Andy UrquhartParticipant@Andy-Urquhart**Include @Andy-Urquhart in your post and this person will**

be notified via email.Another explanation …

If I have five numbers and I know the average; I’m only free to choose four numbers.0May 7, 2004 at 12:32 am #99888I can calculate a mean, variance and so on in statistics. I can develop a process to determine the mode and median, i.e. I can program that in a computer and get the answer. How would I write a program to determine the degrees of freedom? It seems impossible and thus seems arbitrary….

0May 7, 2004 at 8:10 am #99902In the example I mentioned, to calculate the degrees of freedom just subtract 1 from the sample size. I believe the justification is self-evident. If that doesn’t satisfy your curiousity, then I apologise for my limitations.

0June 22, 2004 at 1:48 pm #102094I have an excellent BB Coach that is able to explain things to me so that I can understand them. The way he explained Degrees of Freedom to me is as follows: You are given 4 checkers and you are told to place a checker in each corner of the board. For the first placement you have 4 choices (degrees of freedom); for the second you have 3, for the third placement you only have 2 degrees of freedom and for the 4th you have none so this works out. n-1 if n is 4, then you only have 3 degrees of freedom.

Hope this helps.0January 25, 2005 at 8:40 am #113969

Tesfu IsakMember@Tesfu-Isak**Include @Tesfu-Isak in your post and this person will**

be notified via email.Dear

Can you please tell me the exact meannig of dgree of freedom in statistical analysis I know that it is n-1but I want to know the exact in terpretation and how it infulence ,coorelate and affect to the data ?

Thank u

0January 28, 2005 at 5:21 am #114131Hi, obviously this analogy makes sense in some way, but the real meaning of variance is some kind of “averaging concept” by which we calculate the average variation of a population. Especially, when we use sample mean to estimate the population mean, we use n-1 instead of n as a denomenator; but when we calculate population mean by using up every subject in a population, we will use N instead of n (sample size) – 1. If we talk about degree of freedom in this sense, what is the proper explanation of it? it is the essence of this term.

0January 28, 2005 at 8:51 am #114136If you have A + B = 30 and you define A = 10 then you can have just one degree of fredom (n-1), because B can be only 20.

If you have A + B + C =30 and A is always =10, now you have 2 degree of freedom (B, C) because B and C can be many different values to satisfy the equation.

And so on …

Apologies for the simplicity.

Rgs, Peppe0April 3, 2005 at 11:06 pm #117167Bobbie, that is the best description of degrees of freedom that I have seen on this board. My BB coach taught the same lucid way with a different example. It is also important to note that the neccesity to subtract 1 from n becomes negligible as n increases in size where n > 30.

0May 31, 2005 at 7:08 pm #120484I like your explanation Andy. It’s simple and seems solid. Tank you.

0June 1, 2005 at 8:20 am #120495Thanks Orfu … I first heard this explanation from a statistician who taught a Taguchi Methods course.

Dr. Taguchi believes that there is a similarity between Anova and Fourier Analysis – but, as yet, I can’t see it!!!

Cheers,

Andy0June 1, 2005 at 8:46 am #120498Another simple way to understand dof is :

If one were to break a stick into 3 pieces of different lengths… he/she would have only 2 chances to do that…(in order words the degree of freedom to determine the outcome(length of 3 pieces) is governed only by 2 chances0June 5, 2005 at 8:46 pm #120709It helped me!! thank you

0June 23, 2005 at 5:16 am #122002

Brian FordParticipant@Brian-Ford**Include @Brian-Ford in your post and this person will**

be notified via email.How can i use Kurtosis in a statement?

0June 23, 2005 at 12:07 pm #122021It is obvious that Degrees of freedom is significant only when we have small sample size.If we increase the no of samples,there will be less impact on our standard deviation calculation due to degrees of freedom.

Is there any guidelines / thumb rule when we have to consider the dF.some thing like sample vs population ratio etc0June 24, 2005 at 7:25 am #122097i hv not heard of any relation between the sample and population to consider DF.

However, as mentioned earlier, when n>30, the DF do not make a difference in the calculation of Std Dev. Try to plot

(1/(n-1)) – (1/n), and you will see this difference tends to ZERO (0.00064) at n=30.

This difference is the difference in the variance of the population which is calculated using N and not (N-1), and the variance estimated from the sample using (n-1).

Hence, the DF will not be important for calculating the Std Dev, but will be more important in further tools. Higher DF would always be better as this will give you enough liberty to find the effects of blocks etc, which then do not need to be confounded with some interaction terms, due to scarcity of data points and hence, low DF.0June 24, 2005 at 7:53 am #122099Please let me have more light on this statement

“but will be more important in further tools”.

Please let me have clarity on significance of DF.Where it is more important apart from basic S.D calculation

0June 24, 2005 at 10:11 am #122106Do we have proof mentioned in any book what you are saying.

Regards,

0October 6, 2005 at 9:59 am #127963

sangal v kMember@sangal-v-k**Include @sangal-v-k in your post and this person will**

be notified via email.dear sir

i am woring in the field of process analysis. please send me some literature about this.

thaknig you

v k sangal

Sr Lect.

Dr KNMIET, Modinagar, INDIA

0October 6, 2005 at 11:45 am #127969Dear Mr.Sangal,

Quality Control describes numerous methods for monitoring the quality of a production process. However, once a process is under control the question arises, “to what extent does the long-term performance of the process comply with engineering requirements or managerial goals?” For example, to return to our piston ring example, how many of the piston rings that we are using fall within the design specification limits? In more general terms, the question is, “how capable is our process (or supplier) in terms of producing items within the specification limits?” Most of the procedures and indices recently introduced to the US by Ford Motor Company (Kane, 1986). They allow us to summarize the process capability in terms of meaningful percentages and indices.

The computation and interpretation of process capability indices will first be discussed for the normal distribution case. If the distribution of the quality characteristic of interest does not follow the normal distribution, modified capability indices can be computed based on the percentiles of a fitted non-normal distribution. Process Analysis allows you to fit various specific non-normal distributions (e.g., Weibull, log-normal, Beta, Gamma, etc.) as well as general non-normal distributions by moments.

Order of business. Note that it makes little sense to examine the process capability if the process is not in control. If the means of successively taken samples fluctuate widely, or are clearly off the target specification, then those quality problems should be addressed first. Therefore, the first step toward a high-quality process is to bring the process under control, using the charting techniques available in vaious quality control modules.

Thank you

Nitish0October 10, 2005 at 8:47 pm #128136

SixSigmaGuyParticipant@SixSigmaGuy**Include @SixSigmaGuy in your post and this person will**

be notified via email.I thought this was a GREAT discussion!! But for me, it just perpetuated my confusion. :-(. The examples were very helpful, but I still don’t understand how to explain what the purpose of DFs is and WHY it exists. Could I say it was a bias applied to my results when I have small sample sizes to account for error in the estimations?

The question came up in my GB classes and I need an answer that says why we have DFs and what the words “Degree” and “Freedom” have to do with it.

While we are at it, can someone explain the 1.5 sigma shift? Just kidding. :-)

Thanks!!0October 26, 2005 at 10:51 am #128878

DicksonParticipant@Dickson**Include @Dickson in your post and this person will**

be notified via email.May I know what is the meaning of penalities of degree of freedom?

0March 2, 2006 at 9:03 pm #134554

JonathanParticipant@Jonathan**Include @Jonathan in your post and this person will**

be notified via email.I wonder if you can help me with the Degrees of Freedom tables that appear in my SSBB memory jogger and other stat books. At the end of the SSBB memory jogger there are 3 tables that refer to DOF. The tables are “Probability Points of t Distribution with v (???) Degrees of Freedom”, “Ordinates of t Distribution with v Degrees of Freedom”, and ” X2 Distribution with v Degrees of Freedom”. I understand that the reference on the x axis reference to the confident level (95% ….). Why do people have tables for degrees of freedom? Don’t you only need (n-1) for DOF as the standard?

0March 2, 2006 at 9:34 pm #134556Degrees of freedom computations are straightforward. Whether you are doing a t-test, one way ANOVA, or calculating degrees of freedom for an interaction term. The reason you see them on the tables is to allow you to determine the critical vaue of the test statistic, given your sample size and required confidence level. Most stats training these days will only teach you to look at p-values, and they skip over this critical aspect. Try doing your hypothesis test without looking at the p-value and see if you make the right decision. (Reject Ho or Fail to reject Ho based on comparing critical F to calculated F)

0March 2, 2006 at 9:35 pm #134557The different distributions use different d.f. to define them. n-1 is one of them. There is also one that is (rows-1) and (columns -1) and others.

0March 2, 2006 at 10:05 pm #134558No, Df is used to determine the critical value (table value) for the tests, which you then compare to the calculated value.

The tables do not give you Df, they give you the critical values based on the Df for the given test. If you do not perform tests by hand – you don’t really need to use these tables. Your software will give you a P-Value that is calculated based partially off of numbers in these tables.

0December 1, 2006 at 3:25 am #148256thank you LIN your message helped me alot to come out of confussion

0March 30, 2007 at 1:49 pm #154206not really sure but i neeed some help to plz post a anwser for me if u click on this link

0March 30, 2007 at 2:29 pm #154207

qualitycoloradoParticipant@qualitycolorado**Include @qualitycolorado in your post and this person will**

be notified via email.Dianna,Thanks for your posting. Prior to your posting, the most recent reply in this thread was way back in March of 2006, so it is not clear what you need help with — please clarify.

Best regards,

QualityColorado0October 24, 2007 at 5:06 pm #163742So the definitions help me understand the concept here, but can anyone outline a real world example where this concept is necessary for a black belt? Where would you need to identify how many df in a practical problem? Thank you!

0October 28, 2007 at 4:26 pm #164039i understand the concept of degree of freedom, but please explain it to me further by answering these questions:

we have a set of six numbers by rolling a die six times: 2, 3, 3, 4, 6, 6. This set of numbers (lets call them Set A) has how many degrees of freedom?

i think the answer should be 5, as we have no of values, i.e. n=6 so df=n-1=5

if we converts these six numbers to deviations from the mean. The resulting set of six deviation scores (lets call them Set B) has how many degrees of freedom? well i am confused about it, please help.

the say we abtain another set of six numbers by rolling a die six times. This set of numbers (lets call them Set C) have the same sum as the number is Set A. Explain how many degrees of freedom there are in Set C? Again i think the degree of freedom for this should be 5. please let me knowif i am wrong also explain, what should be degree of freedom for deviation scores0November 15, 2007 at 4:46 am #164805A very good explanation, also on why it is called degrees of “freedom” can be found athttp://web.archive.org/web/20060706151302/http://seamonkey.ed.asu.edu/~alex/computer/sas/df.htmlRegards,

Tobias0May 27, 2008 at 11:37 am #172229I’m grateful to all for their explanations, but if truth be told I’d have to say that on a deeper level not one person on this thread really understands the concept of “degrees of freedom.” Here’s the best explanation I could find, so far: http://courses.ncssm.edu/math/Stat_Inst/PDFS/DFWalker.pdf

0 - AuthorPosts

The forum ‘General’ is closed to new topics and replies.