Machine Learning for Sequential Data: A Review
Lecture notes in computer sciencePublished 1 January 2002
Thomas G. Dietterich
Citations671
SJR quartileQ2
SJR score0.35
SNIP0.55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper formalizes the principal learning tasks and describes the methods that have been developed within the machine learning research community for addressing these problems, including sliding window methods, recurrent sliding windows, hidden Markov models, conditional random fields, and graph transformer networks.
Abstract
Statistical learning problems in many fields involve sequential data. This paper formalizes the principal learning tasks and describes the methods that have been developed within the machine learning research community for addressing these problems. These methods include sliding window methods, recurrent sliding windows, hidden Markov models, conditional random fields, and graph transformer networks. The paper also discusses some open research issues.
Keywords
Computer Science
Proceedings of the IEEEGradient-based learning applied to document recognition
58,219 Citations1998Yann LeCun, Léon Bottou +2 more
This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task, and Convolutional neural networks are shown to outperform all other techniques.
Journal of the Royal Statistical Society Series B (Statistical Methodology)Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
49,657 Citations1977A. P. Dempster, N. M. Laird +1 more
BiometricsClassification and Regression Trees.
23,841 Citations1984Alexander Gordon, Leo Breiman +3 more
C4.5: Programs for Machine Learning
23,665 Citations1992J. R. Quinlan
A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.
Learning Internal Representations by Error Propagation
16,235 Citations1985David E. Rumelhart, Geoffrey E. Hinton +1 more
Cambridge University Press eBooksAn Introduction to Support Vector Machines and Other Kernel-based Learning Methods
13,883 Citations2000Nello Cristianini, John Shawe‐Taylor
ScholarlyCommons (University of Pennsylvania)Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
12,978 Citations2001John Lafferty, Andrew McCallum +1 more
This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
Cognitive ScienceFinding Structure in Time
10,841 Citations1990Jeffrey L. Elman
A proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory and suggests a method for representing lexical categories and the type/token distinction is developed.
Artificial IntelligenceWrappers for feature subset selection
8,925 Citations1997Ron Kohavi, George H. John
The wrapper method searches for an optimal feature subset tailored to a particular algorithm and a domain and compares the wrapper approach to induction without feature subset selection and to Relief, a filter approach tofeature subset selection.
Programs for Machine Learning
5,794 Citations1994Steven L. Salzberg, Alberto M. Segre
In his new book, C4.5: Programs for Machine Learning, Quinlan has put together a definitive, much needed description of his complete system, including the latest developments, which will be a welcome addition to the library of many researchers and students.
Digital Access to Scholarship at Harvard (DASH) (Harvard University)Maximum Likelihood from Incomplete Data via the EM Algorithm
4,523 Citations1977Dempster, Arthur P., Laird, Nan M. +1 more
ePrints Soton (University of Southampton)An Introduction to Support Vector Machines
4,120 Citations2000Nello Cristianini, John Shawe‐Taylor
IEEE Transactions on Information TheoryApproximating discrete probability distributions with dependence trees
2,640 Citations1968Chee Lap Chow, C. Liu
It is shown that the procedure derived in this paper yields an approximation of a minimum difference in information when applied to empirical observations from an unknown distribution of tree dependence, and the procedure is the maximum-likelihood estimate of the distribution.
TechnometricsRidge Regression: Biased Estimation for Nonorthogonal Problems
1,921 Citations1970Arthur E. Hoerl, Robert W. Kennard
Parallel Networks that Learn to Pronounce English Text
1,556 Citations1989Terrence J. Sejnowski
Toward optimal feature selection
1,444 Citations1996Daphne Koller, Mehran Sahami
An efficient algorithm for feature selection which computes an approximation to the optimal feature selection criterion is given, showing that the algorithm effectively handles datasets with a very large number of features.
Maximum Entropy Markov Models for Information Extraction and Segmentation
1,333 Citations2000Andrew McCallum, Dayne Freitag +1 more
A new Markovian sequence model is presented that allows observations to be represented as arbitrary overlapping features (such as word, capitalization, formatting, part-of-speech), and defines the conditional probability of state sequences given observation sequences.
Journal of Molecular BiologyPredicting the secondary structure of globular proteins using neural network models
1,234 Citations1988Ning Qian, Terrence J. Sejnowski
It is concluded from computational experiments on real and artificial structures that no method based solely on local information in the protein sequence is likely to produce significantly better results for non-homologous proteins.
Advances in psychologySerial Order: A Parallel Distributed Processing Approach
1,230 Citations1997Michael I. Jordan
A theory of learned sequential behavior is presented, with a focus on coarticulatory phenomena in speech, implemented as a recurrent parallel distributed processing network that is trained via a generalized error-correcting algorithm.
Data Mining and Knowledge DiscoveryAdaptive Fraud Detection
873 Citations1997Tom Fawcett, Foster Provost
This paper uses a rule-learning program to uncover indicators of fraudulent behavior from a large database of customer transactions, which are used to create a set of monitors, which profile legitimate customer behavior and indicate anomalies.
Applied IntelligenceOvercoming the Myopia of Inductive Learning Algorithms with RELIEFF
798 Citations1997Igor Kononenko, Edvard Šimec +1 more
This work reimplemented Assistant, a system for top down induction of decision trees, using RELIEFF, an extension of RELIEF, as an estimator of attributes at each selection step for heuristic guidance of inductive learning algorithms.
Generalization by Weight-Elimination with Application to Forecasting
531 Citations1990Andreas S. Weigend, David E. Rumelhart +1 more
This work adds a term to the back propagation cost function that penalizes network complexity, called weight-elimination, and uses this procedure to predict the sunspot time series and the notoriously noisy series of currency exchange rates.
IEEE Transactions on Neural NetworksInput-output HMMs for sequence processing
296 Citations1996Yoshua Bengio, Paolo Frasconi
It is demonstrated that IOHMMs are well suited for solving grammatical inference problems on a benchmark problem and able to map input sequences to output sequences, using the same processing style as recurrent neural networks.
Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)Hoeffding Races: Accelerating Model Selection Search for Classification and Function Approximation
274 Citations2018Oded Maron, Andrew Moore
This paper focuses on the special case of leave-one-out cross validation applied to memory-based learning algorithms, but it is argued that it is applicable to any class of model selection problems.
IEEE Transactions on Neural NetworksPruning recurrent neural networks for improved generalization performance
129 Citations1994C. Lee Giles, Christian W. Omlin