I’m working on a project to quantify how a product’s performance is affected by both the season of the year as well as the life cycle of the product. Nearly everyone agrees that the time of the year (winter vs summer) is a big factor in the product’s performance and there is much data to support this hypothesis. Others on the team think that the product’s stage of life (old vs new) is also a factor however we can’t figure out a good way to test this hypothesis because the seasonal effects appear to overpower the other factors. Any ideas out there on how to isolate the effect of the stage of life using an analytic tool or method on historical data? I cannot run a DOE but do have a wealth of historical data to work with.
Thanks in advance!
If your historical data includes measures of performance on both “old” and “new” at various times of the year and if you have an operational definition of what constitutes “old” and “new” then the first thing to try would be a simple time plot of responses from “old” and “new” equipment and see what you see. If there appears to be a separation in the two time plots as an initial check you could give some thought to running some simple regressions where the Y’s are the measures of performance and the X’s are time (the choice of terms for time- linear, curvilinear, etc. – will be driven by the shape of the plot) and a categorical variable for “old” vs. “new”. If you run the regression and find your categorical variable is significant this would offer evidence that product stage of life might matter.
A simple analysis like the above will not be definitive (it makes a lot of assumptions about data quality and measurement independence) but it should get you started. Should stage of life test significant you will have to go back to the data and spend a lot of time thinking about data quality, repeated measures, the representative nature of the historical data, etc. and rerun the analysis in light of your findings concerning these and other issues.
I cannot run a DOE but do have a wealth of historical data to work with
It seems to me that you are only looking at two factors (1. time of year, 2. life cycle) so only a very small DOE is required.
2 factors at 2 levels = 4 runs : replicate = 8 runs
If there is a wealth of historical data surely it is possible to choose 8 data points from your data which correspond to the array points on a full factorial DOE.
Also, most software allows you to analyse historical dataset
With seasons of the year you would have, at a minimum, 4 levels and “two” classifications of stages of product life that are, most likely, extremely “fuzzy” – which is to say that the classes of “old” and “new” are such that each of the categories encompasses a large range of stage of life which may or may not overlap.
Regardless, I wouldn’t recommend trying to populate a design matrix with historical data. One of the key assumptions of a design is that you have controlled for the variability of process factors that were not included in the list of design variables. Historical data cannot offer this guarantee. This, in turn, means you do not have any assurances with respect to variable confounding and hence validity with respect to trends attributed to design variables when the data is extracted from the historical record.
As I mentioned in the first post – simple plotting in the manner specified will allow you to look at your historical data and permit an initial check of trending over time and within the broad categories of ” new” and “old”.
I would make your data more quantitative. quantititative date is much more powerful. Instead of using old versus new can you convert to year of manufacture? Instead of season can you use month of year or julian date of manufacture. It is obviously not the date but some seasonal shift that would affect the product? Do you think this is temperature, humidiity or something else? It would be nice to use these other factors to minimize abnormal weather conditions i.e a warm winter or a cold summber. You may need to do some digging with your subject matter experts to pick the likely environmental factors to evaluate. Can you quantify product performance?
Then check the correlation between old/new and performance and date of manufacture versus
Thanks for all the advice everyone! I currently have one year’s worth of historical data and am trying to dig up another year’s worth of data. Once I have this then I plan on comparing the monthly performance between dates of manufacture. Keeping my fingers crossed that this will hold enough things constant so I can draw some good conclusions.