In dispraise of the exact test

Abstract

The use of the “exact test” with the 2X2 table that records the observations obtained in a comparative trial has been widely considered to be the paradigm of statistical tests of significance. This is attributable to the fact that it is based on the theories of R.A. Fisher and as a result has acquired the sobriquet ‘exact’. The Fisherian basis of the exact test, that the marginal totals are “ancillary statistics,” and therefore provide no information respecting the configuration of the body of the table is shown to be incorrect. The exact test for the one-sided case is compared with the normal test for the nominal significance levels P=0.05 and P=0.01. It is shown by direct computation that the effective level is closer to the nominal level with the normal test than with the exact test, and that the power of the normal test is considerably larger than the power of the exact test, the increased power exceeding the change of effective level. It is concluded that the exact test should not be used in preference to the normal test.