A new concept using LSTM Neural Networks for dynamic system identification
Published 1 May 2017
Yu Wang
Citations205
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A new concept of applying one of the most popular RNN approach - LSTM to identify and control dynamic system is to be investigated and both identification (or learning) dynamic system and design of controller based on identification are going to be discussed.
Abstract
Recently, Recurrent Neural Network becomes a very popular research topic in machine learning field. Many new ideas and RNN structures have been generated by different authors, including long short term memory (LSTM) RNN and Gated Recurrent United (GRU) RNN ([1],[2]), a number of applications have also been developed among various research labs or industrial companies ([3]-[5]). Most of these schemes, however, are only applicable to machine learning problems, or static systems in control field. In this paper, a new concept of applying one of the most popular RNN approach - LSTM to identify and control dynamic system is to be investigated. Both identification (or learning) dynamic system and design of controller based on identification are going to be discussed. Also, a new concept of using a convex-based LSTM networks for fast learning purpose will be explained in detail. Simulation studies will be presented to demonstrated the new LSTM structure performs much better than conventional RNN and even single LSTM network.
Keywords
Computer Science
Neural ComputationLong Short-Term Memory
98,079 Citations1997Sepp Hochreiter, Jürgen Schmidhuber
A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
Psychological ReviewThe perceptron: A probabilistic model for information storage and organization in the brain.
11,746 Citations1958Frank F. Rosenblatt
This article will be concerned primarily with the second and third questions, which are still subject to a vast amount of speculation, and where the few relevant facts currently supplied by neurophysiology have not yet been integrated into an acceptable theory.
arXiv (Cornell University)Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
10,728 Citations2014Jun‐Young Chung, Çaǧlar Gülçehre +2 more
These advanced recurrent units that implement a gating mechanism, such as a long short-term memory (LSTM) unit and a recently proposed gated recurrent unit (GRU), are found to be comparable to LSTM.
IEEE Transactions on Neural NetworksIdentification and control of dynamical systems using neural networks
8,008 Citations1990Kumpati S. Narendra, K. Parthasarathy
It is demonstrated that neural networks can be used effectively for the identification and control of nonlinear dynamical systems and the models introduced are practically feasible.
Journal of the American Medical AssociationThe Organization of Behavior: A Neuropsychological Theory
4,240 Citations1950Therese Mahoney, Margaret Halliday +339 more
arXiv (Cornell University)Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition
856 Citations2014Haşim Sak, Andrew Senior +1 more
Novel LSTM based RNN architectures which make more effective use of model parameters to train acoustic models for large vocabulary speech recognition are presented.
arXiv (Cornell University)Semi-supervised Sequence Learning
490 Citations2015Andrew M. Dai, Quoc V. Le
Two approaches to use unlabeled data to improve Sequence Learning with recurrent networks are presented and it is found that long short term memory recurrent networks after pretrained with the two approaches become more stable to train and generalize better.
arXiv (Cornell University)Grammar as a Foreign Language
403 Citations2014Oriol Vinyals, Łukasz Kaiser +4 more
High-Performance OCR for Printed English and Fraktur Using LSTM Networks
244 Citations2013Thomas M. Breuel, Adnan Ul-Hasan +2 more
An application of bidirectional LSTM networks to the problem of machine-printed Latin and Fraktur recognition and these recognition accuracies were found without using any language modelling or any other post-processing techniques.
Grapheme-to-phoneme conversion using Long Short-Term Memory recurrent neural networks
190 Citations2015Kanishka Rao, Fuchun Peng +2 more
This work proposes a G2P model based on a Long Short-Term Memory (LSTM) recurrent neural network (RNN) that has the flexibility of taking into consideration the full context of graphemes and transform the problem from a series of grapheme-to-phoneme conversions to a word- to-pronunciation conversion.
IEEE Transactions on Automatic ControlNew Concepts in Adaptive Control Using Multiple Models
177 Citations2011Zhuo Rachel Han, Kumpati S. Narendra
This paper proposes a new approach to using multiple models to cope with transients which depends on the collective outputs of all the models, and can be viewed as a time-varying convex combination of the estimates.
Fast Reinforcement Learning using multiple models
26 Citations2016Kumpati S. Narendra, Yu Wang +1 more
The principal objective of this paper is to present a new approach, based on the use of multiple models (or estimates), that may alleviate this problem and increase the speed of response in reinforcement learning.
Stability, robustness, and performance issues in second level adaptation
22 Citations2014Kumpati S. Narendra, Yu Wang +1 more
Most of the results described in the paper pertain to plants in companion form, with all state variables accessible, and an effort is made to indicate how the same concepts can be extended to more general cases.
Novel approaches for face recognition: Template-matching using Dynamic Time Warping and LSTM neural network supervised classification
22 Citations2008Alexandre L. M. Levada, Débora Corrêa +3 more
This paper presents novel methodologies for face recognition: template-matching using Dynamic Time Warping (DTW) and Long-Short-Term-Memory (LSTM) neural network supervised classification.
Extension of second level adaptation using multiple models to SISO systems
13 Citations2015Kumpati S. Narendra, Yu Wang +1 more
Second level adaptation using multiple models for the adaptive control of linear systems with unknown parameters was introduced in [1], and its robustness properties were discussed in [2].
arXiv (Cornell University)The Rationale for Second Level Adaptation
9 Citations2015Kumpati S. Narendra, Yu Wang +1 more
A more detailed examination of the theoretical and practical advantages claimed for the new approach to the adaptive control of linear time-invariant plants with unknown parameters is undertaken.
arXiv (Cornell University)Improving the Speed of Response of Learning Algorithms Using Multiple Models
8 Citations2015Kumpati S. Narendra, Snehasis Mukhopadyhay +1 more
A Supervised Adaptive Learning-based Fuzzy Controller for a non-linear vehicle system using Neural Network Identification
5 Citations2016Yu Wang, Xiaoxi Zhu
The proposed algorithm design incorporates the following learning and control theorems - non-linear system identification using neural network, fuzzy logic, supervised adaptive learning as well as multiple model based convex parameterization.
IFAC Proceedings VolumesDesign of Triple-Level Multiple Models Fuzzy Logic Controller for Adaptive Speed Control with Unknown External Disturbances
5 Citations2014Yu Wang
The addition of Triple-Level adaptation in the proposed TAM-FLC increases the controller robustness to uncertainties in system parameters, and the use of multiple models increase the accuracy of controller significantly.