Learning subjective representations for planning

Published 30 July 2005

Dana Wilkinson, Michael Bowling, Ali Ghodsi

Citations4

TL;DR

A non-linear embedding technique called Action Respecting Embedding is used to construct a subjective representations for planning that are learned directly from agent observations and actions (requiring no initial domain knowledge).

Abstract

Planning involves using a model of an agent’s actions to find a sequence of decisions which achieve a desired goal. It is usually assumed that the models are given, and such models often require expert knowledge of the domain. This paper explores subjective representations for planning that are learned directly from agent observations and actions (requiring no initial domain knowledge). A non-linear embedding technique called Action Respecting Embedding is used to construct such a representation. It is then shown how to extract the effects of the agent’s actions as operators in this learned representation. Finally, the learned representation and operators are combined with search to find sequences of actions that achieve given goals. The efficacy of this technique is demonstrated in a challenging robot-vision-inspired image domain. 1

Keywords

Computer Science

Journal of the American Statistical AssociationMarkov Decision Processes: Discrete Stochastic Dynamic Programming.

8,422 Citations1995Kasra Hazeghi, Martin L. Puterman

Markov Decision Processes covers recent research advances in such areas as countable state space models with average reward criterion, constrained models, and models with risk sensitive optimality criteria, and explores several topics that have received little or no attention in other books.

Artificial IntelligenceStrips: A new approach to the application of theorem proving to problem solving

4,073 Citations1971Richard Fikes, Nils J. Nilsson

A new problem solver called STRIPS is described that attempts to find a sequence of operators in a space of world models to transform a given initial world model into a model in which a given goal formula can be proven to be true.

MPG.PuRe (Max Planck Society)Learning with Kernels

2,532 Citations2002Bernhard Schölkopf

View PDF

International Journal of Computer VisionUnsupervised Learning of Image Manifolds by Semidefinite Programming

700 Citations2006Kilian Q. Weinberger, Lawrence K. Saul

An algorithm for unsupervised learning of image manifolds by semidefinite programming that computes a low dimensional representation of each image with the property that distances between nearby images are preserved.

View PDF

PsychometrikaFitting One Matrix to Another Under Choice of a Central Dilation and a Rigid Motion

402 Citations1970Peter H. Schönemann, Robert M. Carroll

Adaptive BehaviorEfficient Learning and Planning Within the Dyna Framework

205 Citations1993Jing Peng, Ronald J. Williams

Neural ComputationObservable Operator Models for Discrete Stochastic Time Series

181 Citations2000Herbert Jaeger

A novel, simple characterization of linearly dependent processes, called observable operator models, is provided, which leads to a constructive learning algorithm for the identification of linially dependent processes.

Elsevier eBooksLearning by Observation and Practice: An Incremental Approach for Planning Operator Acquisition

127 Citations1995Xuemei Wang

Empirical results show that the system learns operators in this domain well enough to solve problems as effectively as human-expert coded operators.

Learning low dimensional predictive representations

92 Citations2004Matthew Rosencrantz, Geoff Gordon +1 more

This work provides an efficient principal-components-based algorithm for learning a transformed predictive state representations (TPSRs), and shows that TPSRs can perform well in comparison to Hidden Markov Models learned with Baum-Welch in a real world robot tracking task for low dimensional representations and long prediction horizons.

Learning and discovery of predictive state representations in dynamical systems with reset

70 Citations2004Michael R. James, Satinder Singh

The first discovery algorithm and a new learning algorithm for linear PSR-based models for the special class of controlled dynamical systems that have a reset operation are provided and experimental verification of these algorithms are provided.