Stochastic Complexity
Journal of the Royal Statistical Society Series B (Statistical Methodology)Published 1 July 1987
J. Rissanen
Citations343
SJR quartileQ1
SJR score3.31
SNIP2.48
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
SUMMARY It is argued that all the useful information in observed data that can be extracted with a selected class of modeled distributions, will be obtained if we calculate the stochastic complexity, defined to be the shortest description length of the data. The same quantity also determines the greatest lower bound for prediction errors when the data are sequentially predicted. An abstract definition of stochastic complexity is given along with two fundamental theorems which justify the notion. Further, three explicit model selection criteria to approximate the stochastic complexity are described and the associated optimal models are interpreted to define asymptotically sufficient statistics for the data.
Keywords
Computer Science
Journal of the Royal Statistical Society Series B (Statistical Methodology)Cross-Validatory Choice and Assessment of Statistical Predictions
10,527 Citations1974M. Stone
The Economic JournalMathematical Methods of Statistics.
6,007 Citations1947R. C. Geary, H. Leslie Cramer
AutomaticaModeling by shortest data description
5,944 Citations1978J. Rissanen
The number of digits it takes to write down an observed sequence x1,...,xN of a time series depends on the model with its parameters that one assumes to have generated the observed data.
The Annals of StatisticsA Universal Prior for Integers and Estimation by Minimum Description Length
1,667 Citations1983J. Rissanen
Estimation based upon the principle of minimizing the total number of binary digits required to rewrite the observed data, when each observation is given with some precision is studied.
Information and ControlA formal theory of inductive inference. Part I
1,149 Citations1964Ray J. Solomonoff
Journal of the Royal Statistical Society Series B (Statistical Methodology)Reference Posterior Distributions for Bayesian Inference
1,103 Citations1979José M. Bernardo
The Computer JournalAn Information Measure for Classification
947 Citations1968Christopher S. Wallace, D. M. Boulton
It is suggested that the best classification is that which results in the briefest recording of all the attribute information, and the measurements of each thing are regarded as being a message about that thing.
Journal of the American Statistical AssociationA Predictive Approach to Model Selection
919 Citations1979Seymour Geisser, William F. Eddy
IEEE Transactions on Information TheoryUniversal coding, information, prediction, and estimation
903 Citations1984J. Rissanen
A connection between universal codes and the problems of prediction and statistical estimation is established, and a known lower bound for the mean length of universal codes is sharpened and generalized, and optimum universal codes constructed.
Journal of the ACMA Theory of Program Size Formally Identical to Information Theory
873 Citations1975Gregory J. Chaitin
A new definition of program-size complexity is made, which has precisely the formal properties of the entropy concept of information theory.
Cambridge University Press eBooksStatistical Prediction Analysis
838 Citations1975J. Aitchison, I. R. Dunsmore
This paper presents a meta-modelling procedure that automates the very labor-intensive and therefore time-heavy and expensive and expensive process of manually cataloging and forecasting the distribution of distributions in a discrete-time manner.
BiometrikaRecursive estimation of mixed autoregressive-moving average order
498 Citations1982E. J. Hannan, J. Rissanen
IEEE Transactions on Information TheoryComplexity of strings in the class of Markov sources
194 Citations1986J. Rissanen
Shannon's self-information of a string is generalized to its complexity relative to the class of finite-state-machine (FSM) defined sources by a theorem stating that, asymptotically, the mean complexity provides a tight lower bound for the mean length of all so-called regular codes.
IMA Journal of Mathematical Control and InformationA Predictive Least-Squares Principle
132 Citations1986J. Rissanen
Kolmogorov Complexity, Data Compression, and Inference
36 Citations1985Thomas M. Cover
If a sequence of random variables has Shannon entropy H, it is well known that there exists an efficient description of this sequence which requires only H bits, but the entropy H of a sequence also has to do with inference, which allows good guesses of their next terms.
TechnometricsThe Average Estimated Variance Criterion for the Selection-of-Variables Problem in General Linear Models
22 Citations1974Ronald W. Helms
Trabajos de Estadistica y de Investigacion OperativaOn a famous problem of induction
8 Citations1985José M. Bernardo