Expected Significance Level as a Sensitivity Index for Test Statistics

Abstract

Abstract After a brief motivating discussion, of approaches to significance tests in Section 1, the concept of expected significance level (esl) is defined and its basic properties are presented in Section 2. Section 3 discusses the relationship between esl and the familiar criterion of power. While a complete description of power for each size α is more informative than esl, it is argued that esl is often a reasonable compromise between high ideals and practicability. Section 4 gives two estimation methods for Monte Carlo simulation of esl and shows how to estimate variances and covariances for the second of these methods. The first method leads to the familiar Wilcoxon two-sample statistic which is clearly a sample analogue of esl. The second and more apposite method leads to a modification of the Wilcoxon statistic.