Analyzing product performance against two other factors

If your historical data includes measures of performance on both “old” and “new” at various times of the year and if you have an operational definition of what constitutes “old” and “new” then the first thing to try would be a simple time plot of responses from “old” and “new” equipment and see what you see. If there appears to be a separation in the two time plots as an initial check you could give some thought to running some simple regressions where the Y’s are the measures of performance and the X’s are time (the choice of terms for time- linear, curvilinear, etc. – will be driven by the shape of the plot) and a categorical variable for “old” vs. “new”. If you run the regression and find your categorical variable is significant this would offer evidence that product stage of life might matter.

A simple analysis like the above will not be definitive (it makes a lot of assumptions about data quality and measurement independence) but it should get you started. Should stage of life test significant you will have to go back to the data and spend a lot of time thinking about data quality, repeated measures, the representative nature of the historical data, etc. and rerun the analysis in light of your findings concerning these and other issues.

With seasons of the year you would have, at a minimum, 4 levels and “two” classifications of stages of product life that are, most likely, extremely “fuzzy” – which is to say that the classes of “old” and “new” are such that each of the categories encompasses a large range of stage of life which may or may not overlap.

Regardless, I wouldn’t recommend trying to populate a design matrix with historical data. One of the key assumptions of a design is that you have controlled for the variability of process factors that were not included in the list of design variables. Historical data cannot offer this guarantee. This, in turn, means you do not have any assurances with respect to variable confounding and hence validity with respect to trends attributed to design variables when the data is extracted from the historical record.

As I mentioned in the first post – simple plotting in the manner specified will allow you to look at your historical data and permit an initial check of trending over time and within the broad categories of ” new” and “old”.

Thanks for all the advice everyone! I currently have one year’s worth of historical data and am trying to dig up another year’s worth of data. Once I have this then I plan on comparing the monthly performance between dates of manufacture. Keeping my fingers crossed that this will hold enough things constant so I can draw some good conclusions.