An Introduction to Data Analysis & Presentation

Prof. Timothy Shortell, Sociology, Brooklyn College

Two-way Analysis of Variance

When we want to compare more than two means, we use the F-test to determine if there is a difference among the groups.

This one-way analysis of variance test compares means for groups defined by one variable, such as class, religious affiliation, and so forth.

Sometimes, we want to examine the effect of more than one grouping variable. For example, we might want to look at the effects of social class and religious affiliation upon ideology.

Why not just do two separate one-way F-tests? This would increase the likelihood of a type-I error -- remember our discussion of multiple t-tests.

In addition, the two-way ANOVA allows us to determine if the two group variables have a simultaneous effect, i.e., whether or not the effects of religious affiliation and social class build upon each other. Unless we look at both variables together, we cannot know if their effects are independent or simultaneous.

This kind of hypothesis test uses the same F-test we have been discussing. Instead of one F-score, though, the two-way analysis of variance requires four F-scores: one for the overall test -- like the F-score in the one-way ANOVA; one for the interaction effect; and one for each main effect.

In my study of religious affiliation, commitment and ideology among elites, I conducted a series of two-way ANOVAs. I wanted to know if affiliation and commitment influenced the ideology dimensions I was measuring.

The two-way ANOVA is another kind of hypothesis test, and follows the same set of steps. To begin, we must state our hypotheses. Let's use an example from the research article:

Null Hypothesis: There is no effect of religious affiliation and commitment upon expressive individualism.

Research Hypothesis: There is an effect of religious affiliation and commitment upon expressive individualism.

Now, let's consider each of the F-tests.

The Omnibus test
As with the one-way F-test, this test tells us whether there is any difference among the groups. In this case, there are 10 groups being compared: there are five affiliation groups and two commitment groups (5 x 2 = 10).

The first question to ask of this data is: are there any differences among these ten groups in terms of expressive individualism?

The two-way ANOVA begins with this test. If the probability of the F-score is greater than alpha (usually, 0.05), then the hypothesis test stops here. We conclude that there is no effect of religious affiliation or commitment on expressive individualism.

Let's look at the results of this test from the paper:

As we can see, the omnibus, or overall, F-test is statistically significant at the 95% confidence level.

The omnibus F-test tells us that there is some difference somewhere among these ten groups. This means that we should go on to test for the interaction and main effects. We want to try to pin down the nature of the effects of religious affiliation and commitment.

The Interaction Effect
One of the advantages of the two-way ANOVA is the interaction F-test. This tells us whether or not the effects of the two group variables, religious affiliation and commitment, are independent or build upon each other.

We might expect, for example, that the effect of commitment depends on the particular religious affiliation. In other words, the difference between more and less devout Jews might be different than the difference between more and less devout Conservative Protestants.

Let's look at the results:

In this case, there is a significant interaction effect. This tells us that, indeed, the effect of commitment depends on religious affiliation -- or, similarly, the effect of affiliation depends on commitment.

Because the interaction is significant, we must qualify our interpretation of the main effects.

Interactions are easily seen if we graph the group means:

If there were no interaction effect, the two lines (more and less devout) would be parallel.

Main Effects
There is an F-test for each main effect, that is, for each group variable. We want to know how religious affilation affects expressive individualism. We also want to know how commitment affects expressive individualism. The two main effects F-tests indicate if these effects are statistically significant, i.e., reliable.

Let's look at the results:

Both affiliation and commitment are significantly related to expressive individualism.

Interpreting the results
How do we explain the effects of religious affiliation and commitment, in light of the interaction effect?

Like with the post-hoc test in the one-way ANOVA, we are seeking an interpretation that describes the pattern of group differences.

Let's look at the group means:

The first thing we observe is that the less devout of every affiliation group are more liberal on this dimension than the more devout of every affiliation.

There is a very large difference between the more and less devout among Catholics, Jews, and Conservative Protestants. There is a modest difference between more and less devout Moderate Protestants, and a small difference between more and less devout Liberal Protestants.

All materials on this site are copyright © 2001, by Professor Timothy Shortell, except those retained by their original owner. No infringement is intended or implied. All rights reserved. Please let me know if you link to this site or use these materials.