Testing
Testing is a large component of statistical inference and philosophy. We acknowledge that Statisticians still debate whether testing is a solid epistemiological framework for science. However, here we take the view that testing is useful while remaining an imperfect philosophical concept. It is satisfying to think about the current accepted knowledge as the "null hypothesis" and the opposing view as the "alternative hypothesis." The frequentist position is fundamentally conservative and skeptic and requires "enough evidence" to reject the accepted knowledge. We contend that this Statistical philosophy is inherently pragmatic and implementable, which probably accounts for it being so pervasive.
Statistical testing is an enormous area of research and our group has made important contribution to a particular sub-area: testing for zero variance components. The problem appears in mixed effects models, where the shrinkage of random effects is controlled by variance components. Testing whether a variance component is equal to zero is thus equivalent to testing whether an entire layer of random effects is necessary. Examples of such problems are: 1) testing whether the random intercepts in a mixed ANOVA model; 2) testing whether complex regression components are necessary in penalized likelihoods; 3) testing for a parametric model against a general alternative; 4) testing for the number of principal components; and 5) testing whether a brain image is associated with an outcome after accounting for its average.
Our first paper on testing for zero variance in mixed models with one variance component provides both the exact and asymptotic distribution of the likelihood and restricted likelihood ratio test. Fabian Scheipl and Ben Bolker provided an excellent implementation of the finite sample distribution in the R package RLRsim. A follow paper led by Sonja Greven showed that the same approach can be extended to many mixed effects models using very fast finite sample approximations of the null distribution.
These results are not yet implemented in STATA, SAS, or the lme function in R. Our suggestion is to use these software, but be aware of their p-values for testing the zero variance in mixed effects models.