Analysis of variance allows us to compare means of more than two groups. It's a method of analysis we use in research designs with a quantitive response variable and one or more categorical independent variables. The simplest type of analysis of variance is one way analysis of variance with just one independent variable that distinguishes three or more groups. In this video, I'll explain the basics of one way analysis of variance and the logic behind using variances to decide something about means. In analysis of variance, independent variables are often referred to as factors. The factors are categorical variables that represent groups, often experimental, or quasi-experimental conditions, also referred to as the levels of a factor. People can also refer to groups or levels as the cells in a factorial design. In one way analysis of variance, there's just one factor with three or more levels. Suppose we want to compare healthiness of three groups of cats that consume different diets: raw meat, canned food, and dry food. A veterinarian rates their health on a scale from zero to ten. How do we determine whether one or more groups differ in healthiness from the other groups? The first thing you might think of is to create an extended version of the t-test by adding the third mean to the formula. This won't work however, because the shape of the resulting test statistic distribution does not correspond to a stable known probability distribution. So we can't calculate P values. The next thing you might think of is to perform three pairwise t-tests, comparing two group means at a time. This is not a good idea, because the family wise error rate will be inflated. The error rate is the probability of falsely rejecting the null hypothesis and falsely concluding there's a significant effect when the null is in fact true. If we perform one test, the error rate is equivalent to significance level, which we determine ourselves and normally set at five percent. When we perform more than one test, the family wise error rate refers to the probability of at least one of these test resulting in a false rejection of the no hypothesis. This probability is approximately equal to the number of tests times the significance level. If we want to keep the family-wise error rate at the desired significance rate of alpha equal 0.05, for example, we could correct for this inflation by dividing the significance level we use for the individual tests by the number of tests. So if we perform three pairwise tests, the individual significance levels would become 0.05 divided by three equals 0.017. The P values for the individual tests are much less likely to be smaller than 0.017 compared to 0.05. In other words, if we apply this correction, we have less power to detect the difference between the groups if there is a true difference in the population. If we use analysis of variance, we don't have to worry about an inflated error rate. Analysis of variance allows us to decide whether the groups are samples from the same population distribution, with one and the same mean, or whether they're from different population distributions, with different means. It does so by comparing the variance in the response variable between the groups, with the variance within the groups. How can variances tell us something about means? Well, it works based on an assumption and a trick. First, we assume the variance is the same in all the populations. If there's any difference between the populations, this should be a difference in means only. The trick is to estimate the population variance in two different ways. The first method will always result in a fairly accurate and precise estimate of the population variance; whether the population means or different or the same. The second method will produce a fairly accurate and precise population variance estimate, if the means are the same, but it will overestimate the population variance if the population means differ. So we can detect a difference in means by observing a discrepancy between the two estimates of population variance. The first method that's robust, whether the population means differ or not, estimates the population variance using the within group variance, the variance within each group averaged over the groups. If the group sizes are equal, the formula for estimating this variance is very easy. The variances are added and then divided by the number of groups. If the group sizes differ, we employ this formula calculating, sums of squares in each group. The second method that is not robust when the population means differ estimates the population variance using the between-group variance, the variance of the means. If the population means are the same, we still expect to find some differences in the sample means. More population variation will generally result in more variation in sample means. So although it's not a very efficient way of estimating the variance in the population, the variation in the sample means can be used to estimate the population variance. However, if the population means differ, this will result in additional variation in the sample means, resulting in an overestimation of the population variance. To calculate the between group variance, we need to know the grand mean, the mean of the means, which we calculate by multiplying each group mean with the number of observations in that group, adding these together, and then dividing by the total number of observations. To calculate the between group variance, we need to know the grand mean, the mean of the means, which we calculate by multiplying each group mean with the number of observations in that group, adding these together, and then dividing by the total number of groups. We calculate the between group variance by taking each group mean, subtracting the grand mean, and squaring the difference, multiplying by group size. Then summing these square differences, and dividing by
the number of groups minus one. Finally, we can compare the two estimates of the population variance by considering their ratio, which is associated with the F probability distribution. We divide the between group variance by the within group variance. If the population means are the same, we expect both methods to result in roughly the same fair estimate. So under the null hypothesis, we expect a ratio close to one. It will never be smaller than zero, since there is always some variation in the samples, and variances are always positive. If the population means differ, we expect the between group variance in the samples to overestimate the population variance, and to be larger than within group variance. In this case, we expect a ratio larger than one. There's no maximum value. If the ratio is so large that the associated P value is smaller than the significance level, we reject the null hypothesis that the population means are equal and accept the alternative hypothesis that at least one of the groups differs from the rest. We don't know which group or groups differ and in what direction. The ratio of the between and within group variances provide an overall test equivalent to the overall test in multiple regression.
