To generate a valuable customer survey, practitioners should follow nine steps. Part One of this series discussed the first three steps: establishing a goal, determining the sample and choosing the methodology. Part Two covered the fourth, fifth and sixth steps, which related to what types of questions to ask and how they should be asked. The final three steps, which follow the survey through its execution and analysis of results, are explored here.
A pretest is a smallscale implementation of the questionnaire. The test is critical for identifying problems that both respondents and survey administrators may encounter. The purpose of the pretest is to gain feedback, not to be statistically accurate; thus, sample size is not a major consideration. However, who is selected to participate in the pretest is important. Practitioners should aim for around 50 people – perhaps employees of the organization, who are similar to the target population – to test the survey.
The factors to evaluate in the pretest are the clarity, comprehensiveness and acceptability of the survey.
After the pretest, revise the questions as needed. If there is considerable revision, you may want to conduct another pretest. Once revisions have been completed, practitioners can begin to implement the survey.
If they have followed the earlier steps, practitioners should already know the method of survey administration and have a project plan. They should ensure that all involved in the survey know what they have to do and include this training within the project plan. With regard to quality control, practitioners should not just sit back and wait for the results. They must remain engaged during the entire process. If something does go wrong, it is best to catch the problem as soon as possible.
The data collected from the survey should be presented in a succinct and meaningful way. This will assist in future analysis of the data. Two techniques to consider are the use of frequency distributions and contingency tables.
Frequency Distribution
A frequency distribution is a summary presentation of the frequency of a response for each category. The emphasis here is presenting only one variable. Frequency distributions typically contain the following:
An example is shown in the table below:
Table 1: Frequency Distribution  
Question: How long have you been employed with CSI DeLeeuw?  
Time with CSI DeLeeuw  f  % 
Less than 1 year  30  23.6 
More than 1 year, less than 5  39  30.7 
More than 5 years, less than 10  34  26.8 
More than 10 years, less than 15  16  12.6 
15 years or more  8  6.3 
Total  127  100.00 
Nonresponses = 25 
Contingency Tables
Unlike frequency distributions, contingency tables allow for simultaneous analysis of more than one variable. Contingency tables present the relationship between two variables and are referred to by their number of rows and columns. For example, a table with four rows and six columns would be a 4×6 contingency table.
The columns in a contingency table traditionally are for the independent variable. Practitioners should calculate percentages for the column variables and include both row and column totals. The following figures represent sample questions followed by the contingency table:
Table 2: Contingency Table Based on Sample Survey Data  
Entry  MidSr  Senior  MidExec  Executive  Total  
Count  %  Count  %  Count  %  Count  %  Count  %  Count  %  
High Positive  7  10.1  6  10.3  7  12.1  7  12.5  8  12.7  17  11.3 
Positive  11  15.9  19  32.8  11  19.0  15  26.8  17  27.0  36  24.0 
Neutral  21  30.4  18  31.0  22  37.9  16  28.6  20  31.7  48  32.0 
Negative  24  34.8  8  13.8  12  20.7  12  21.4  13  20.6  34  22.7 
High Negative  6  8.7  7  12.1  6  10.3  6  10.7  5  7.9  15  10.0 
Total  69  100.0  58  100.00  58  100.0  56  100.0  63  100.0  150  100.0 
There are multiple tests that can be used to assist with the analysis of the survey data, including the Chisquare test and proportion tests.
Chisquare
The (Chisquare) test is a measure of divergence of the observed and expected frequencies. Chisquare can be used as a:
In this scenario, the test to determine whether the data observed differs from what could be expected due to chance. The c2 test will tell whether or not a relationship exists. There are three other tests that could give the strength of this relationship – Cramer’s V, Phi and Gamma tests for ordinal data – but these are more complicated.
There are two types of Chisquare test: 1way and 2way. The formula for both is:
, where
= the Chisquare statistic
O = the number of observed from each cell
E = the number of expected from each cell
= calculate the value for each cell, then add all these values
1Way Test
For this test, just one column – observed data – is evaluated. Practitioners need to know what they should expect to see for each cell and the degrees of freedom. This test answers the question: Does the observed count differ from what could be expected due to chance?
For the test procedure, first state the hypotheses:
Next, specify the alpha (usually 0.05). Find the critical value for ( _{critical}) for alpha and calculate (_{calculated}). If: _{calculated }> _{critical,} reject H_{o}; otherwise, fail to reject and conclude no statistically significant differences.
2Way Test
For this test, one column (independent variable) and one row (dependent variable) are evaluated. The data needs to be in contingency table form. This test can be completed using statistical analysis software and can answer the question: Is there a relationship between two categorical variables?
For this test procedure, the hypotheses are:
Next, specify the alpha (usually 0.05). Then, run the analysis in the software. If the pvalue is less than 0.05, reject H_{o}; otherwise, fail to reject and conclude no statistically significant differences.
1Proportion Test
A proportion is a ratio of occurrences, generally an occurrence of interest (X) compared to all occurrences (N). Confidence intervals provide practitioners with a proportional range in which it is possible to state (with a specified confidence) that the true population value will fall within. This test can answer the question: Is the observed proportion equal to some specified proportion?
The data in the chart below can be used to illustrate the use of a 1proportion test.
Table 3: Response Data  
High positive  17 
Positive  36 
Neutral  48 
Negative  34 
High negative  15 
Total  150 
There were 53 total positive responses. Using statistical analysis software, it is possible to state with 95 percent certainty that the true population’s positive perception of the program falls between 27.7 percent and 43.5 percent.
The knowledge gained from surveys can be invaluable in continuing a process improvement journey. Therefore, when conducting a survey, it is important that practitioners have a clear understanding of:
With this information, practitioners will be prepared to write and execute a survey.

