iSixSigma

Two Significant Interactions in Multiple Regression

Six Sigma – iSixSigma Forums Old Forums General Two Significant Interactions in Multiple Regression

Viewing 12 posts - 1 through 12 (of 12 total)
  • Author
    Posts
  • #47667

    Ryan
    Member

    Hello all and thank you for help with my previous question. I have a fairly unique situation. I have hypothesized that there is a moderating effect in my multiple regression analysis. As hoped, I found that one predictor (dichotomous variable – control group and experimental group) was moderating the relationship between another predictor (continuous) and the outcome measure (continuous).The problem, however, is that there is another significant interaction between that same moderator (group) and another continuous predictor in the equation. I don’t believe it would be valid for me to interpret results of an interaction term when there are other significant interactions in the equation. It seems to me that I should separate the full model into two regression equations for each group and therefore eliminate possible interactions. Does this sound like the correct way to address this issue?
    The full model (I think) is as follows: Y=Bo+X1B1+X2B2+X3B3+X1X2B4+X1X3B5
    Where X1 = Group (there are two groups), X2 = continuous variable, X3 = continuous variable
    Thanks again for all your help,
    Ryan

    0
    #159161

    Robert Butler
    Participant

      If you have data (say from an experimental design) which allows you to investigate various interactions and if multiple interactions test out as significant then the final model will be built using these terms.  The fact that you have a model with more than one significant interaction is interesting but it is hardly out of the ordinary and there is no reason to try to do anything to eliminate one or more of the interactions from the model.
      Your statement “I don’t believe it would be valid for me to interpret results of an interaction term when there are other significant interactions in the equation” doesn’t make sense.  You might as well have said, “I don’t believe it would be valid for me to interpret results of a main effect when there are other main effects in the equation”. In short, given that the terms are independent of one another, you would interpret the results of an interaction the same way you would interpret the results of any other significant term. 

    0
    #159163

    Ryan
    Member

    Robert, if you have a significant interaction term, in general, it would not be correct to interpret a main effect. If there is an interaction, then the relationship between one predictor and the outcome variable depends upon the level of the other predictor. Would it be correct to interpret a main effect, when you now know that there is another IV moderating that outcome variable? My understanding has always been that if you have a significant interaction between two main effects, the most appropriate step would be to run simple effects and not interpret main effects.  I realize this is not that exact scenario I presented, but couldn’t this concept be extended. If I am interested in one moderator and not the other, can I simply interpret the one in which I’m interested, knowing that the relationship between that very IV and another covariate are interacting? Would it not behoove me to simply split the equation into two regression equations, thereby removing the interaction terms? In essence, I would be testing simple effects. Or can are you stating that I could do both? Interpret separate interaction terms and follow up with simple effects?
    Thank you for your detailed response,
    Ryan

    0
    #159166

    Psychometrician
    Participant

    Two quick questions:
    1. Are you truly investigating moderating effects or are you investigating mediating effects?
    2. Have you checked for multicollinearity? If so, then you have a real problem and need to exclude one of the variables because that can lead to very spurious results.
    The scenario is very hypothetical so some context in terms of the variables and research questions would be very helpful in distinguishing how to interpret your results.
     

    0
    #159168

    Ryan
    Member

    Hi “Psychometrician,”
    1. I am testing moderating effects, not mediating.
    2. I checked for multicollinearity and the VIFs are very low.
    Scenario: I believe that a continuous variable’s relationship with an outcome is being moderated by what condition you are in. I am also controlling for DV baseline scores. I started with the full model which included not only the interaction term of interest but the interaction between the DV and the condition IV. I am trying to figure out how to interpret these results and where to go from here. I think the previous comment was correct in that I should interpret the results accordingly. The interaction terms tell me that the regression slopes for the continuous variable(s) and the DV are different between groups. That finding by itself is what I wanted to know. I’m also interested in whether the continuous IV is signfiicantly predicting the outcome measure, focusing on magnitude and direction. In order to answer that question, I think can go a step further and separate the full model into two equations for each intervention condition. In other words, I can run simple effects.
    Thanks,
    Ryan

    0
    #159169

    Psychometrician
    Participant

    The best way to proceed is to sketch out your findings using a path model. I think what you are really dealing with is a path model using regression as your analytical tool. I have attached the description of the Macro for SPSS. It contains the scenario that you describe in the output section (scroll down). The Wilkipedia link, while not very good, will give you some additional ideas on how to best interpret your particular scenario. The end of the article may be most relevant for your research problem. Let me know if you’re getting closer to understanding your output findings.
    http://www2.chass.ncsu.edu/garson/pa765/path.htm
     
    http://en.wikipedia.org/wiki/Mediation_(Statistics)

    0
    #159170

    Psychometrician SPSS output
    Participant

    The previous post contained a link to the logic of path analysis in relationship to your regression output. This link shows examples of actual SPSS outputs.
    http://www.comm.ohio-state.edu/ahayes/SPSS%20programs/modmed.htm

    0
    #159178

    Psychometrician short answ
    Participant

    Ryan,
    After inundanting you with all the stuff, the short answer is: Yes, if you have a two-level group, you can run the two equations separately in your scenario. Then compare the r that you are focusing on for significant difference.. If the r’s are different you can establish a moderating effect of that two-level group. Again, the findings can be interpreted and depicted in your report as a path model .
    Historically, Regression was established since the 1880s. In the 1930s path analysis was inventended in biology. It is an extension of path analysis taking partial regressions into account and established the language of direct and indirect effects etc.. Since the 1970s Structural Equation Modeling has become the preferred model in your area of study because you can combine measurement analysis (reliability/validity), structural models (i.e. path models), include the error into the equation and assess overall fit. You are kind of working your way from a regression to a path analysis. Good Luck!

    0
    #159184

    Robert Butler
    Participant

      If the model consisted of nothing but main effect terms the relationship between one predictor and the outcome variable will still depend on the levels of the other predictors (unless you have run the very specific case where the design space included combinations where there was a complete absence of each of the predictors of interest). It will not be as complex as the case where interactions are present (the impact will only be on the intercept and not on the slope) but the dependency is still there.
      When one desires to express the relationship between one predictor and the response the usual practice (regardless of model complexity) is to do one of the following:
    1. If all variables are continuous fix the variables at their midpoint and use the regression equation to predict the outcome by varying only the predictor of interest.
    2. If the predictors are a mix of quantitative and qualitative fix the quantitative at their midpoints and generate a family of prediction curves corresponding to the various combinations of qualitative factors.
      In my experience when interactions are present there is very little interest in either #1 or #2.  Instead the focus is on the magnitude of the effect of the interactions relative to the magnitude of the main effect terms. In other words if interactions are present the last thing anyone wants to do is slice and dice the data set to generate a series of simple regression lines and simple tests which gloss over the presence and effect of the interactions. 
     
      

    0
    #159185

    Ryan
    Member

    As usual, everyone have been very helpful! I am familiar with path analysis and see your point about the similarity of what I’m doing and how I could have created the model using path analysis. 
    Thanks again for confirming that splitting the equations is acceptable. The macro is very interesting. I will try it out.
    I do not agree with the previous responder that the last thing I would want to do is follow-up with simple effects tests. I was taught (which makes sense to me) that if you have a significant interaction, you would (a) try to make sense of it (interpret it), and (b) consider running simple effects to take a closer look at the relationship between the continuous IV and the DV within each level of the moderating variable. I may find that in the intervention condition, the continuous IV has a strong positive relationship with the DV, while in the control group it has a little or no relationship. However, I do understand the importance of comparing the magnitude of the main effect with the interaction term and will do that.
    Thanks to everyone for their help. It is greatly appreciated.
    Sincerely,
    Ryan

    0
    #159188

    Robert Butler
    Participant

      Perhaps we are letting words get in the way of understanding.  If I have a significant interaction that interaction is telling me that the relationship between the two variables is more than a simple additive effect. 
      When an interaction is present one always tests it  to see which combinations of levels of the variables are having the greater/lesser effect (actually in most cases a simple plot will suffice).
      I was left with the impression that you were trying to split/reanalyze your data so that you could avoid looking at an interaction.  In your most recent post you state “if you have a significant interaction, you would (a) try to make sense of it (interpret it), and (b) consider running simple effects to take a closer look at the relationship between the continuous IV and the DV within each level of the moderating variable. I may find that in the intervention condition, the continuous IV has a strong positive relationship with the DV, while in the control group it has a little or no relationship.” 
    I agree.
     It would appear the stumbling block is the phrase “follow-up with simple effects tests” which I now understand to mean nothing more than doing the above.

    0
    #159216

    Psychometrician Suggestion
    Participant

    Ryan,
    The scenario that you describe is discussed in detail in Baron, R. M. and Kenny, D.A. (1986). The Moderator-Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic and Statistical Consideration. Journal of Personality and Social Psychology, Vol. 57 (6). 1173 – 1182.
    The 3 step by step procedure to analyze and understand the second interaction and its the interpretation is outlined in detail on pages 1178 – 1179. What that second interaction means from your research point of view will become clear once you go through the process outlined in Baron and Kenny. A detailed explanation would be too long for this site. So, I suggest you consult this article to develop your investigative strategy. Good Luck!

    0
Viewing 12 posts - 1 through 12 (of 12 total)

The forum ‘General’ is closed to new topics and replies.