The Logic of Tests of Significance

The goal of the paper is to describe precisely the pattern of inductive reasoning that is characteristic of Fisherian tests of significance, and to show that while it is far more cogent than Fisher's critics have realized, it does not logically sustain the inferences it sanctions.

Abstract

In spite of the fact that the Neyman–Pearson theory of testing is the official theory of statistical testing, most research publications in the social sciences use a pattern of inductive reasoning that is characteristic of Fisherian tests of significance. The exact structure and rationale of this pattern of reasoning is widely misunderstood. The goal of the paper is to describe precisely the pattern and its rationale, and to show that while it is far more cogent than Fisher's critics have realized, it does not logically sustain the inferences it sanctions.