Robustness Simulation

Difference Between Two Means

In general, an independent groups t test works well even if its assumptions are violated. The test is slightly conservative (a lower type I error than the signficance level) for skewed distributions. When the homogeneity of variance assumption is violated, there is a slight increase in the type I error rate, and the effect is more pronounced with small sample sizes.

There is a serious inflation of the type I error rate when there is a combination of unequal sample sizes and heterogeneous variances. This occurs when the sample size from the population with the larger variance is smaller than the sample size from the populaton with the smaller variance.