Locally Weighted Learning for Control
Artificial Intelligence ReviewPublished 1 February 1997
Christopher G. Atkeson, Andrew Moore, Stefan Schaal
Citations538
SJR quartileQ1
SJR score3.01
SNIP5.11
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
There are ways in which locally weighted learning, a type of lazy learning, has been applied by us to control tasks, and various forms that control tasks can take, are explained.
Abstract
Lazy learning methods provide useful representations and training algorithms for learning about complex phenomena during autonomous adaptive control of complex systems. This paper surveys ways in which locally weighted learning, a type of lazy learning, has been applied by us to control tasks. We explain various forms that control tasks can take, and how this affects the choice of learning paradigm. The discussion section explores the interesting impact that explicitly remembering all previous experiences has on the problem of learning to control.
Keywords
Computer ScienceEngineering
Numerical recipes in C
15,253 Citations1994William H. Press, Saul A. Teukolsky +2 more
The Diskette v 2.06, 3.5''[1.44M] for IBM PC, PS/2 and compatibles [DOS] Reference Record created on 2004-09-07, modified on 2016-08-08.
ScienceDynamic Programming
13,088 Citations1966Richard Bellman
The more the authors study the information processing aspects of the mind, the more perplexed and impressed they become, and it will be a very long time before they understand these processes sufficiently to reproduce them.
Society for Industrial and Applied Mathematics eBooksIterative Solution of Nonlinear Equations in Several Variables
6,875 Citations2000J. M. Ortega, Werner C. Rheinboldt
Convergence of Minimization Methods An Annotated List of Basic Reference Books Bibliography Author Index Subject Index.
Machine LearningLearning to Predict by the Methods of Temporal Differences
3,894 Citations1988Richard S. Sutton
This article introduces a class of incremental learning procedures specialized for prediction-that is, for using past experience with an incompletely known system to predict its future behavior, and proves their convergence and optimality for special cases and relate them to supervised-learning methods.
Computational Geometry
3,606 Citations1985Franco P. Preparata, Michael Ian Shamos
This book clearly demonstrates that computational geometry in the plane is now a fairly well-understood branch of computer science and mathematics and points the way to the solution of the more challenging problems in dimensions higher than two.
Computational geometry. an introduction
2,369 Citations1985Franco P. Preparata, Michael Ian Shamos
Journal of the American Statistical AssociationProjection Pursuit Regression
2,092 Citations1981Jerome H. Friedman, Werner Stuetzle
Artificial Intelligence ReviewLocally Weighted Learning
1,684 Citations1997Christopher G. Atkeson, Andrew Moore +1 more
The survey discusses distance functions, smoothing parameters, weighting functions, local model structures, regularization of the estimates and bias, assessing predictions, handling noisy data and outliers, improving the quality of predictions by tuning fit parameters, and applications of locally weighted learning.
Cognitive ScienceForward Models: Supervised Learning with a Distal Teacher
1,472 Citations1992Michael I. Jordan, David E. Rumelhart
This article demonstrates that certain classical problems associated with the notion of the “teacher” in supervised learning can be solved by judicious use of learned internal models as components of the adaptive system.
Elsevier eBooksIntegrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming
1,351 Citations1990Richard S. Sutton
Results are shown for a simple Dyna-PI system that simultaneously learns by trial and error, learns a world model, and plans optimal routes using the evolving world model and it is shown that Dyna-Q architectures are easy to adapt for use in changing environments.
Mathematics of ComputationElementary Numerical Analysis: An Algorithmic Approach
1,340 Citations1967J. W. W., S. D. Conte
Journal of Artificial Intelligence ResearchActive Learning with Statistical Models
1,286 Citations1996David Cohn, Zoubin Ghahramani +1 more
This work shows how the same principles may be used to select data for two alternative, statistically-based learning architectures: mixtures of Gaussians and locally weighted regression.
Mathematics of ComputationElementary Numerical Analysis.
1,255 Citations1994W. G., Kendall Atkinson
This chapter discusses Taylor Polynomials, Numerical Linear Algebra, and the Finite Difference Method for PDEs, as well as other topics of interest to students of mathematics.
Communications of the ACMToward memory-based reasoning
1,232 Citations1986Craig Stanfill, David L. Waltz
The intensive use of memory to recall specific episodes from the past—rather than rules—should be the foundation of machine reasoning.
Parallel and distributed computation
1,175 Citations1989Dimitri P. Bertsekas, John N. Tsitsiklis +1 more
Artificial IntelligenceLearning to act using real-time dynamic programming
1,109 Citations1995Andrew G. Barto, Steven J. Bradtke +1 more
An algorithm based on dynamic programming, which is called Real-Time DP, is introduced, by which an embedded system can improve its performance with experience and illuminate aspects of other DP-based reinforcement learning methods such as Watkins'' Q-Learning algorithm.
Elsevier eBooksCombining Instance-Based and Model-Based Learning
738 Citations1993J. R. Quinlan
A general method is presented that allows predictions to use both instance-based and model-based learning, and improves with three approaches to constructing models and with eight datasets demonstrate improvements due to the composite method.
Machine LearningPrioritized Sweeping: Reinforcement Learning with Less Data and Less Time
710 Citations1993Andrew Moore, Christopher G. Atkeson
This work presents a new algorithm, prioritized sweeping, for efficient prediction and control of stochastic Markov systems, which successfully solves large state-space real-time problems with which other methods have difficulty.
The MIT Press eBooksLearning in Embedded Systems
586 Citations1993Leslie Pack Kaelbling
This dissertation addresses the problem of designing algorithms for learning in embedded systems using Sutton's techniques for linear association and reinforcement comparison, while the interval estimation algorithm uses the statistical notion of confidence intervals to guide its generation of actions.
Statistical ScienceLocal Regression: Automatic Kernel Carpentry
413 Citations1993Trevor Hastie, Clive Loader
Local regression smoothers automatically, intuitively and simultaneously adjust for both the biases above to the given order and generalize naturally to the multidi- mensional case, an approach more attractive than using higher-order kernel functions for the same purpose.
Medical Entomology and ZoologyStochastic Optimal Control: Theory and Application
410 Citations1986Robert F. Stengel
Learning and Sequential Decision Making
353 Citations1989Andrew G. Barto, Richard S. Sutton +1 more
It is shown how a TD METHOD can beunderstood as a NOVEL SYNTHESIS of CONCEPTS from the theORY of STOCHASTIC DYNAMIC PROGRAMMING, which is the standard method for solving decision-making problems in binary systems.
ScienceNeural Model of Adaptive Hand-Eye Coordination for Single Postures
344 Citations1988Michael Kuperstein
A neural network model has been developed that achieves adaptive visual-motor coordination of a multijoint arm, without a teacher, that is designed to be generalized for coordinating any number of topographic sensory inputs with limbs ofAny number of joints.
Locally Weighted Learning
298 Citations1997Christopher G. Atkeson, Andrew Moore +1 more
Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)Hoeffding Races: Accelerating Model Selection Search for Classification and Function Approximation
274 Citations2018Oded Maron, Andrew Moore
This paper focuses on the special case of leave-one-out cross validation applied to memory-based learning algorithms, but it is argued that it is applicable to any class of model selection problems.
IEEE Control SystemsRobot juggling: implementation of memory-based learning
243 Citations1994Stefan Schaal, Christopher G. Atkeson
A memory-based local modeling approach (locally weighted regression) is used to represent a learned model of the task to be performed, and an exploration algorithm is developed that explicitly deals with prediction accuracy requirements during exploration.
IEEE Transactions on Systems Man and CyberneticsReal-time application of neural networks for sensor-based control of robots with vision
234 Citations1989Wallace T. Miller
A practical neural network-based learning control system is described that is applicable to complex robotic systems involving multiple feedback sensors and multiple command variables.
Elsevier eBooksEfficient Algorithms for Minimizing Cross Validation Error
227 Citations1994Andrew Moore, Mary S. Lee
It is shown how experimental design methods can achieve this, using a technique similar to a Bayesian version of Kaelbling's Interval Estimation, to reduce the computational burden of large scale cross validation searches.
Active Learning with Statistical Models.
221 Citations1995David Cohn, Zoubin Ghahramani +1 more
Adaptive BehaviorEfficient Learning and Planning Within the Dyna Framework
205 Citations1993Jing Peng, Ronald J. Williams
Elsevier eBooksInstance-Based Utile Distinctions for Reinforcement Learning with Hidden State
185 Citations1995R. Andrew McCallum
Utile Suffix Memory is a reinforcement learning algorithm that uses short-term memory to overcome the state aliasing that results from hidden state by combining the advantages of previous work in instance-based and memory-based learning.
Elsevier eBooksVariable Resolution Dynamic Programming: Efficiently Learning Action Maps in Multivariate Real-valued State-spaces
166 Citations1991Andrew Moore
How such an approach to create an autonomous reactive controller can be realized in real valued multivariate state spaces in which straightforward discretization falls prey to the curse of dimensionality is discussed.
neural information processing systemsLearning to Control an Unstable System with Forward Modeling
89 Citations1989Michael I. Jordan, Robert A. Jacobs
This work extends previous work by considering how this methodology can be applied to the optimization of quantities that are distal not only in space but also in time.
Using Local Trajectory Optimizers to Speed Up Global Optimization in Dynamic Programming
81 Citations1993Christopher G. Atkeson
This work uses second order local trajectory optimization to generate locally optimal plans and local models of the value function and its derivatives, and maintains global consistency of the local Models of thevalue function, guaranteeing that the locally optimal Plans are actually globally optimal.
Bumptrees for Efficient Function, Constraint and Classification Learning
71 Citations1990Stephen M. Omohundro
A new class of data structures called "bumptrees" is described, useful for efficiently implementing a number of neural network related operations and applications to density estimation, classification, and constraint representation and learning are outlined.
International Joint Conference on Artificial IntelligenceMultiresolution instance-based learning
70 Citations1995Kan Deng, Andrew Moore
A new way of structuring a database and a new algorithm for accessing it is presented and evaluated that maintains the advantages ot instance-based learning and permits the same flexibility as a conventional linear search but at greatly reduced computational cost.
Neural Information Processing SystemsUsing Local Models to Control Movement
55 Citations1989Christopher G. Atkeson
This paper explores the use of a model neural network for motor learning that can represent smooth nonlinear functions, yet has simple training rules with a single global optimum.
Neural Information Processing SystemsFast, Robust Adaptive Control by Learning only Forward Models
54 Citations1991Andrew W. Moore
This paper argues that the optimization of learning rate, the number of experimental control decisions before adequate performance is obtained, and robustness is of prime importance--if necessary at the expense of computation per control cycle and memory requirement.
Elsevier eBooksAcquisition of Dynamic Control Knowledge for a Robotic Manipulator
53 Citations1990Andrew W. Moore
A method of learning in which all the experiences in the lifetime of the robot are explicitly remembered are stored in a manner which permits fast recall of the closest previous experience to any new situation, which leads to a very high rate of learning of the robotic kinematics and dynamics.
Memory-based Stochastic Optimization
49 Citations1995Andrew Moore, Jeff Schneider
This paper introduces new algorithms for optimizing noisy plants in which each experiment is very expensive, and examines the question of whether the global model can really help optimization, and extends it to the case of time-varying functions.
Computational IntelligenceLearning to control a dynamic physical system
45 Citations1987Margaret E. Connell, Emmanuel J. Connell +1 more
Experiments illustrate that a control method is learned in about 16 trials, an improvement over previous learning programs.
Elsevier eBooksEnhancing Transfer in Reinforcement Learning by Building Stochastic Models of Robot Actions
35 Citations1992Sridhar Mahadevan
Experiments on simulated and real robots show that useful action models can be learned from a 12 by 12 scrolling certainty grid sensor and on the simulator these models are sufficiently rich to enable significant transfer within and across two tasks, box pushing and wall following.
Lecture notes in statisticsLearning to Catch: Applying Nearest Neighbor Algorithms to Dynamic Control Tasks
29 Citations1994David W. Aha, Steven L. Salzberg
This paper examines the hypothesis that local weighted variants of k-nearest neighbor algorithms can support dynamic control tasks, and hypothesizes that the non-linearities in this task are the cause, and that local regression algorithms may need to be modified to work well under similar conditions.
National Conference on Artificial IntelligenceLearning to control a dynamic physical system
23 Citations1987Margaret E. Connell, Paul E. Utgoff
Robotics and Autonomous SystemsReliability estimation for neural network based autonomous driving
18 Citations1994Dean Pomerleau
IRRE provides a good estimate of the reliability of MLPs trained for autonomous driving and is used to select between networks trained for different driving situations.
Elsevier eBooksEfficient Memory-Based Dynamic Programming
17 Citations1995Jing Peng
A novel memory-based approach to dynamic programming that addresses the issue of generalization that does not require the quantization of continuous state or action spaces and can achieve arbitrarily variable resolution.
Knowledge of knowledge and intelligent experimentation for learning control
14 Citations2002Andrew W. Moore
If a learning system is able to provide some estimate of the reliability of the generalizations it produces, then the rate of learning can be considerably increased, and experience becomes concentrated in regions of the control space which are relevant to the task at hand.
Geometric and neuromorphic learning for nonlinear modeling, control and forecasting
5 Citations2003Zlatko Zografski
An algorithm based on results from computational geometry that learns nonlinear dynamical system mappings was applied to the control of robot motion along a nominal trajectory on the basis of a learned model of its inverse dynamics and prediction of the behavior of a complex nonlinear dynamic system for forecasting regional electric power consumption.