A Winner's Curse for Econometric Models: On the Joint Distribution of In-Sample Fit and Out-of-Sample Fit and its Implications for Model Selection
Published 1 January 2010
Peter Reinhard Hansen
Citations24
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
We consider the case where a parameter, �; is estimated by maximizing a criterion function, Q(X;�). The estimate, ^ = ^(X); is then used to evaluate the criterion function with the same data, X, as well as with an independent data set, Y. The insample …t and out-of-sample …t relative to that of the true, or quasi-true, parameter, � � ; are de…ned by � = Q(X; ^) Q(X;� � ) and ~ = Q(Y; ^) Q(Y;� � ), respectively. We derive the joint limit distribution of (�;~) for a broad class of criterion functions and the
Keywords
Decision SciencesMathematicsEconomics, Econometrics and Finance
IEEE Transactions on Automatic ControlA new look at the statistical model identification
50,732 Citations1974Hirotugu Akaike
EconometricaEstimation and Hypothesis Testing of Cointegration Vectors in Gaussian Vector Autoregressive Models
11,158 Citations1991Søren Johansen
Journal of International EconomicsEmpirical exchange rate models of the seventies
4,150 Citations1983Richard Meese, Kenneth Rogoff
Journal of the Operational Research SocietyThe Combination of Forecasts
3,077 Citations1969John M. Bates, Clive W. J. Granger
Journal of EconometricsApproximately normal tests for equal predictive accuracy in nested models
2,442 Citations2006Todd E. Clark, Kenneth D. West
Journal of Business and Economic StatisticsA Test for Superior Predictive Ability
1,158 Citations2005Peter Reinhard Hansen
Journal of the Royal Statistical Society Series A (Statistics in Society)Model Uncertainty, Data Mining and Statistical Inference
1,098 Citations1995Chris Chatfield
The effects of model uncertainty, such as too narrow prediction intervals, and the non-trivial biases in parameter estimates which can follow data-based modelling are reviewed.
Cambridge University Press eBooksForecasting Economic Time Series
871 Citations1998Michael P. Clements, David F. Hendry
EconometricaLeast Squares Model Averaging
813 Citations2007Bruce E. Hansen
It is shown that the new Mallows model average (MMA) estimator is asymptotically optimal in the sense of achieving the lowest possible squared error in a class of discrete model average estimators.
Journal of EconometricsAsymptotics for out of sample tests of Granger causality
732 Citations2006Michael W. McCracken
Journal of the American Statistical AssociationEstimation for Markowitz Efficient Portfolios
534 Citations1980J. D. Jobson, Bob Korkie
Journal of International EconomicsNonparametric exchange rate prediction?
488 Citations1990Francis X. Diebold, James A. Nason
Oxford University Research Archive (ORA) (University of Oxford)Pooling of Forecasts.
482 Citations2004David F. Hendry, Michael P. Clements
Forecasting using a combination, when no model coincides with a non-constant data generation process (DGP), is considered, and it is shown that only non-encompassed devices should be retained in the combination.
Journal of EconometricsLeast-squares forecast averaging
276 Citations2008Bruce E. Hansen
The MMA forecasts have low MSFE and have much lower maximum regret than other feasible forecasting methods, including equal weighting, BIC selection, weighted BIC, AIC selection), weighted AIC, Bates-Granger combination, predictive least squares, and Granger-Ramanathan combination.
Handbook of statistics8 Forecast evaluation and combination
269 Citations1996Francis X. Diebold, Jose A. Lopez
A number of forecast evaluation topics of particular relevance in economics and finance are described, including methods for evaluating direction-of-change forecasts, probability forecasts and volatility forecasts.
Journal of EconometricsOn the selection of forecasting models
167 Citations2005Atsushi Inoue, Lutz Kilian
It is shown that under suitable conditions the IC method will be consistent for the best approximating model among the candidate models, while under standard assumptions the SOOS method will select over-parameterized models with positive probability, resulting in excessive finite-sample PMSEs.
AN EMPIRICAL COMPARISON OF METHODS FOR FORECASTING USING MANY PREDICTORS
142 Citations2005James H. Stock, Mark W. Watson
Journal of EconometricsRobustifying forecasts from equilibrium-correction systems
133 Citations2005David F. Hendry
Oxford Bulletin of Economics and StatisticsA Note on the Selection of Time Series Models
118 Citations2005Serena Ng, Pierre Perrón
BOOTSTRAP ESTIMATE OF KULLBACK-LEIBLER INFORMATION FOR MODEL SELECTION
90 Citations1997Ritei Shibata
Journal of Business and Economic StatisticsSerial Correlation and the Combination of Forecasts
74 Citations1988Francis X. Diebold
SSRN Electronic JournalForecasting Economic Aggregates by Disaggregates
57 Citations2006David F. Hendry, Kirstin Hubrich
Oxford Bulletin of Economics and StatisticsCombining Forecasts from Nested Models<sup>*</sup>
34 Citations2009Todd E. Clark, Michael W. McCracken
SSRN Electronic JournalBayesian Model Averaging and Exchange Rate Forecasts
29 Citations2003Jonathan H. Wright
National Bureau of Economic ResearchBayesian Averaging, Prediction and Nonnested Model Selection
8 Citations2008Han Hong, Bruce Preston
RePEc: Research Papers in EconomicsModel Selection in Unstable Environments
4 Citations2009Barbara Rossi, Raffaella Giacomini
The central idea of the methodology is to propose a measure of the models' local relative performance: the "local Kullback-Leibler Information Criterion" (KLIC), which measures the relative distance or the two (misspecified) likelihoods from the true likelihood at a particular point in time.
