Significance Tests in Parallel and in Series

Abstract

Abstract The advice is often given that significance tests should be selected before sampling evidence is examined. It is suggested here that this advice is appropriate only for inexperienced statisticians, and an approximate rule of thumb is tentatively proposed in the hope of provoking discussion, namely that the statistician could in some cases use a harmonic mean or weighted harmonic mean of the tail-area probabilities arising from various tests, all on the same evidence (tests in "parallel"). This rule of thumb should not be used if the statistician can think of anything better to do, and especially of course if he is in a position to use a sufficient statistic (or an "efficacious" one, in a sense defined below). An application is given to the judgment of the weights that may be used for combining tests in series.