Toward a Reconciliation of the Bayesian and Frequentist Approaches to Point Estimation

Abstract

Abstract The Bayesian and frequentist approaches to point estimation are reviewed. The status of the debate regarding the use of one approach over the other is discussed, and its inconclusive character is noted. A criterion for comparing Bayesian and frequentist estimators within a given experimental framework is proposed. The competition between a Bayesian and a frequentist is viewed as a contest with the following components: a random observable, a true prior distribution unknown to both statisticians, an operational prior used by the Bayesian, a fixed frequentist rule used by the frequentist, and a fixed loss criterion. This competition is studied in the context of exponential families, conjugate priors, and squared error loss. The class of operational priors that yield Bayes estimators superior to the “best” frequentist estimator is characterized. The implications of the existence of a threshold separating the space of operational priors into good and bad priors are explored, and their relevance in areas such as Bayesian robustness and the elicitation of prior distributions is discussed. Both the theoretical and empirical results presented in this article suggest that the method to be favored in a particular application depends crucially on the quality of the prior information available, with Bayesian and frequentist methods each emerging as preferable under specific, and complementary, circumstances.