An adaptive permutation test procedure for several common tests of significance

Abstract

An adaptive weighted least-squares test procedure is proposed that increases the power of the most commonly used tests of significance. This test is shown to have high power if the number of observations is at least 20. The proposed adaptive method is a modification of weighted least squares with the weights determined from the order statistics of the residuals of the model specified by the null hypothesis. The weights are then used with a weighted least-squares regression to compute a test statistic. To insure that the test maintains its size, a permutation method is used to compute the observed significance level. To evaluate the performance of the test, Monte Carlo simulations were used to estimate the power for two-independent samples, for one-way layouts, for paired comparisons, and for the slope in a simple linear regression. In these simulations the proposed adaptive weighted least-squares method maintained its size and often had greater power than the common parametric and nonparametric tests. For the study designs investigated in this paper, the proposed adaptive weighted least-squares method is recommended if the number of observations is at least 20.