A few annotations to annon’s comparison of ANOVA and Regression:
The Eta squared in an ANOVA table is the same as the R-squared in a regression analysis. The Eta/R-squared gives you the “practical significance” as a proportion of explained variance (Factor sum of squared/Total sum of squared).
The F-test in ANOVA tells you the statistical significance.
“Directionality” is implied in both regression and ANOVA. ANOVA was originally used in experimental data only. That is why the term “factor” is used in Minitab for example. Regression analysis was originally used for observational data and in survey research. That is why the term “predictor” is used in books talking about regression. In many cases you establish “concurrent predction”, i.e. the prediciton is not projected in the future but Variable Y is “predicted” from Variable X). So, “directionality” is not a function of the statistical formula (ANOVA vs. Regression) but of the assumptions that you make about the underlying relationship of the variables.
There is a difference between research design (experimental vs. survey) and statistical analysis (ANOVA/Regression). Interaction effects can be analyzed in Regression using survey data by multiplying two independent variables and adding them as an additional variable in the equation. If this term is significant, you can establish an interaction effect.
In general, the confusion about ANOVA and Regression has come about because the two formulas for the statistically equivalent methodologies were developed by two competing schools of thought in biometric statistics. Originally, regression was applied to observational data, ANOVA to experimental data. Since the 1970s and with the development of the general linear model, it can be shown that they are part of the same family of statistics that make the assumption that variables can be viewed as a linear combination of each other.
.
