Variational Dependent Multi-output Gaussian Process Dynamical Systems
Lecture notes in computer sciencePublished 1 January 2014
Jing Zhao, Shiliang Sun
Citations33
SJR quartileQ2
SJR score0.35
SNIP0.55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The proposed model has superiority on modeling dynamical systems under the more reasonable assumption and the fully Bayesian learning framework and can be flexibly extended to handle regression problems.
Abstract
This paper presents a dependent multi-output Gaussian process (GP) for modeling complex dynamical systems. The outputs are dependent in this model, which is largely different from previous GP dynamical systems. We adopt convolved multi-output GPs to model the outputs, which are provided with a flexible multi-output covariance function. We adapt the variational inference method with inducing points for approximate posterior inference of latent variables. Conjugate gradient based optimization is used to solve parameters involved. Besides the temporal dependency, the proposed model also captures the dependency among outputs in complex dynamical systems. We evaluate the model on both synthetic and real-world data, and encouraging results are observed.
Keywords
Computer ScienceEngineering
Journal of the Royal Statistical Society Series B (Statistical Methodology)Probabilistic Principal Component Analysis
3,730 Citations1999Michael E. Tipping, Chris Bishop
It is demonstrated how the principal axes of a set of observed data vectors may be determined through maximum likelihood estimation of parameters in a latent variable model that is closely related to factor analysis.
Journal of Machine Learning ResearchA Unifying View of Sparse Approximate Gaussian Process Regression
1,759 Citations2005Joaquin Quiñonero-Candela, Carl Edward Rasmussen
A new unifying view, including all existing proper probabilistic sparse approximations for Gaussian process regression, relies on expressing the effective prior which the methods are using, and highlights the relationship between existing methods.
Sparse Gaussian Processes using Pseudo-inputs
1,329 Citations2005Edward Snelson, Zoubin Ghahramani
It is shown that this new Gaussian process (GP) regression model can match full GP performance with small M, i.e. very sparse solutions, and it significantly outperforms other approaches in this regime.
IEEE Transactions on Pattern Analysis and Machine IntelligenceGaussian Process Dynamical Models for Human Motion
1,038 Citations2007Jonathan M. Wang, David J. Fleet +1 more
This work marginalize out the model parameters in closed form by using Gaussian process priors for both the dynamical and the observation mappings, which results in a nonparametric model for dynamical systems that accounts for uncertainty in the model.
Advanced textbooks in control and signal processingModelling and Control of Robot Manipulators
1,036 Citations2000L. Sciavicco, Bruno Siciliano
Variational Learning of Inducing Variables in Sparse Gaussian Processes
1,024 Citations2009Michalis K. Titsias
A variational formulation for sparse approximations that jointly infers the inducing inputs and the kernel hyperparameters by maximizing a lower bound of the true log marginal likelihood.
Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models
948 Citations2005Neil D. Lawrence
Edinburgh Research ExplorerMulti-task Gaussian Process Prediction
851 Citations2007Edwin V. Bonilla, Kian Ming A. Chai +1 more
A model that learns a shared covariance function on input-dependent features and a "free-form" covariance matrix over tasks allows for good flexibility when modelling inter-task dependencies while avoiding the need for large amounts of data for training.
IEEE Transactions on Intelligent Transportation SystemsA Bayesian Network Approach to Traffic Flow Forecasting
769 Citations2006Shiliang Sun, Chao Zhang +1 more
Comprehensive experiments on urban vehicular traffic flow data of Beijing and comparisons with several other methods show that the Bayesian network is a very promising and effective approach for traffic flow modeling and forecasting, both for complete data and incomplete data.
The MIT Press eBooksModeling Human Motion Using Binary Latent Variables
702 Citations2007Graham W. Taylor, Geoffrey E. Hinton +1 more
A non-linear generative model for human motion data that uses an undirected model with binary latent variables and real-valued "visible" variables that represent joint angles that makes on-line inference efficient and allows for a simple approximate learning procedure.
Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data
684 Citations2003Neil D. Lawrence
A new underlying probabilistic model for principal component analysis (PCA) is introduced that shows that if the prior's covariance function constrains the mappings to be linear the model is equivalent to PCA, and is extended by considering less restrictive covariance functions which allow non-linear mappings.
3D People Tracking with Gaussian Process Dynamical Models
439 Citations2006Raquel Urtasun, David J. Fleet +1 more
This work modifications the GPDM to permit learning from motions with significant stylistic variation, and the resulting priors are effective for tracking a range of human walking styles, despite weak and noisy image measurements and significant occlusions.
Gaussian Process Dynamical Models
432 Citations2005Jianguo Wang, Aaron Hertzmann +1 more
This paper marginalize out the model parameters in closed-form, using Gaussian Process (GP) priors for both the dynamics and the observation mappings, resulting in a nonparametric model for dynamical systems that accounts for uncertainty in the model.
Measurement Science and TechnologyModelling and Control of Robot Manipulators
429 Citations2000L Sciavicco and B Siciliano
An introduction to kinematics Differential Kinematics and Statics Dynamics Trajectory Planning Motion Control Interaction Control Actuators and Sensors Control Architecture Appendices.
International Conference on Artificial Intelligence and StatisticsBayesian Gaussian Process Latent Variable Model
398 Citations2010Michalis K. Titsias, Neil D. Lawrence
A variational inference framework for training the Gaussian process latent variable model and thus performing Bayesian nonlinear dimensionality reduction and the maximization of the variational lower bound provides a Bayesian training procedure that is robust to overfitting and can automatically select the dimensionality of the nonlinear latent space.
Foundations and Trends® in Machine LearningKernels for Vector-Valued Functions: A Review
372 Citations2012Mauricio A. Álvarez, Lorenzo Rosasco +1 more
Neural ComputationThe Variational Gaussian Approximation Revisited
304 Citations2008Manfred Opper, Cédric Archambeau
The relationship between the Laplace and the variational approximation is discussed, and it is shown that for models with gaussian priors and factorizing likelihoods, the number of variational parameters is actually .
Locally Weighted Projection Regression : An O(n) Algorithm for Incremental Real Time Learning in High Dimensional Space
287 Citations2000Sethu Vijayakumar, Stefan Schaal
This paper evaluates different methods of projection regression and derives a nonlinear function approximator based on them, which is the first truly incremental spatially localized learning method to combine all these properties.
Warped Gaussian Processes
263 Citations2003Edward Snelson, Zoubin Ghahramani +1 more
This work generalises the Gaussian process (GP) framework for regression by learning a nonlinear transformation of the GP outputs, which allows for non-Gaussian processes and non- Gaussian noise.
Journal of Machine Learning ResearchComputationally Efficient Convolved Multiple Output Gaussian Processes
236 Citations2011Mauricio A. Álvarez, Neil D. Lawrence
This paper presents different efficient approximations for dependent output Gaussian processes constructed through the convolution formalism, exploit the conditional independencies present naturally in the model and shows experimental results with synthetic and real data.
IEEE Transactions on Pattern Analysis and Machine IntelligenceLinear Latent Force Models Using Gaussian Processes
133 Citations2013Mauricio A. Álvarez, David Luengo +1 more
This paper presents a hybrid approach using Gaussian processes and differential equations to combine data-driven modeling with a physical model of the system and shows how different, physically inspired, kernel functions can be developed through sensible, simple, mechanistic assumptions about the underlying system.
Sparse Representation for Gaussian Process Models
128 Citations2000Lehel Csató, Manfred Opper
An approach for a sparse representation for Gaussian Process models in order to overcome the limitations of GPs caused by large data sets is developed based on a combination of a Bayesian online algorithm together with a sequential construction of a relevant subsample of the data which fully specifies the prediction of the model.
IEEE Transactions on Pattern Analysis and Machine IntelligenceCorrection to "Gaussian Process Dynamical Models for Human Motion" [Feb 08 283-298]
125 Citations2008Jack M. Wang, David J. Fleet +1 more
arXiv (Cornell University)Gaussian Process Regression Networks
122 Citations2011Andrew Gordon Wilson, David A. Knowles +1 more
A new regression framework, Gaussian process regression networks (GPRN), is introduced, which combines the structural properties of Bayesian neural networks with the nonparametric exibility of Gaussian processes and derives both elliptical slice sampling and variational Bayes inference procedures for GPRN.
Latent Force Models
117 Citations2009Mauricio A. Álvarez, David Luengo +1 more
It is shown how dierent, physically-inspired, kernel functions can be developed through sensible, simple, mechanistic assumptions about the underlying system through a hybrid approach using Gaussian processes and dierential equations to combine data driven modelling with a physical model of the system.
Efficient Multioutput Gaussian Processes through Variational Inducing Kernels
106 Citations2010Mauricio A. Álvarez, David Luengo +2 more
This paper introduces the concept of variational inducing functions to handle potential non-smooth functions involved in the kernel CP construction and considers an alternative approach to approximate inference based on variational methods.
MODELING AND FORECASTING VEHICULAR TRAFFIC FLOW AS A SEASONAL STOCHASTIC TIME SERIES PROCESS
104 Citations1999Billy M. Williams, L A Hoel
The findings presented in this report establish the traffic flow prediction superiority of seasonal time series methods, especially seasonal ARIMA modeling, over the recently developed methods.
Learning for Larger Datasets with the Gaussian Process Latent Variable Model
102 Citations2007Neil D. Lawrence
The latest techniques in sparse Gaussian process regression (GPR) to theGaussian process latent variable model (GPLVM) are applied and how they may be implemented in the context of the GP-LVM is discussed.
Neural Information Processing SystemsMulti-task Gaussian Process Learning of Robot Inverse Dynamics
91 Citations2008Christopher K. I. Williams, Stefan Klanke +2 more
A multi-task Gaussian process prior for handling multiple loads, where the inter-task similarity depends on the underlying inertial parameters, and generally improves performance over either learning only on single tasks or pooling the data over all tasks.
Collaborative multi-output Gaussian processes
63 Citations2014Trung Nguyen, Edwin V. Bonilla
The collaborative multi-output Gaussian process (GP) model for learning dependent tasks with very large datasets achieves superior performance compared to single output learning and previous multi- output GP models, confirming the benefits of correlating sparsity structure of the outputs via the inducing points.
Bayesian Warped Gaussian Processes
63 Citations2012Miguel Lázaro-Gredilla
It is shown that it is possible to use a non-parametric nonlinear transformation in WGP and variationally integrate it out and the resulting Bayesian WGP is able to work in scenarios in which the maximum likelihood WGP failed.
arXiv (Cornell University)Variational Gaussian Process Dynamical Systems
56 Citations2011Andreas Damianou, Michalis K. Titsias +1 more
arXiv (Cornell University)Manifold Relevance Determination
49 Citations2012Andreas Damianou, Carl Henrik Ek +2 more
This paper presents a fully Bayesian latent variable model which exploits conditional nonlinear (in)-dependence structures to learn an efficient latent representation and introduces a relaxation to the discrete segmentation and allow for a "softly" shared latent space.
Neural Computing and ApplicationsA review of deterministic approximate inference techniques for Bayesian machine learning
43 Citations2013Shiliang Sun
This paper reviews typical deterministic approximate inference techniques and attempts to identify open problems that may be helpful for future investigations in this field.
TUbilio (Technical University of Darmstadt)Expectation Propagation in Gaussian Process Dynamical Systems
36 Citations2012Marc Peter Deisenroth, Shakir Mohamed
MPG.PuRe (Max Planck Society)Switched Latent Force Models for Movement Segmentation
35 Citations2010Mauricio A. Álvarez, Jan Peters +2 more
This paper introduces an extension of the basic latent force model, that switches between different latent functions and potentially different dynamical systems, that creates a versatile representation for robot movements that can capture discrete changes and non-linearities in the dynamics.
Pattern Recognition LettersGaussian Process Dynamical Models for hand gesture interpretation in Sign Language
29 Citations2011Nuwan Gamage, Ye Chow Kuang +2 more
The main objective of this paper is to introduce Gaussian Process Dynamical Model as an alternative machine learning method for hand gesture interpretation in Sign Language and presents the experimental results for Gaussian process Dynamical model against a database of 66 hand gestures from the Malaysian Sign Language.
Gaussian process dynamical models for nonparametric speech representation and synthesis
24 Citations2012Gustav Eje Henter, Marcus Frean +1 more
Gaussian process dynamical models (GPDMs) are proposed as a new, nonparametric paradigm in acoustic models of speech that demonstrate subjectively better quality of synthesized speech than from comparable HMMs.
arXiv (Cornell University)Variational Inference for Uncertainty on the Inputs of Gaussian Process Models
24 Citations2014Andreas Damianou, Michalis K. Titsias +1 more
A Bayesian method for training GP-LVMs by introducing a non-standard variational inference framework that allows to approximately integrate out the latent variables and subsequently train a GP-LVM by maximizing an analytic lower bound on the exact marginal likelihood.
Efficient Gaussian Process Inference for Short-Scale Spatio-Temporal Modeling
24 Citations2012Jaakko Luttinen, Alexander Ilin
The proposed Gaussian process inference scheme is compared to the standard approach using the sparse Cholesky decomposition and it is shown to be much faster and computationally feasible for 100–1000 times larger datasets.
arXiv (Cornell University)Sequential Inference for Latent Force Models
16 Citations2012Jouni Hartikainen, Simo Särkkä
neural information processing systemsPhoneme Classification using Constrained Variational Gaussian Process Dynamical System
13 Citations2012Hyunsin Park, Sungrack Yun +3 more
An acoustic model based on the variational Gaussian process dynamical system (VGPDS) to overcome the limitations of classical hidden Markov models in modeling speech and demonstrate the effectiveness of the proposed model with three experimental results.