Towards Human-Level Semantics Understanding of Human-Centered Object Manipulation Tasks for HRI: Reasoning About Effect, Ability, Effort and Perspective Taking
International Journal of Social RoboticsPublished 13 July 2014
Amit Kumar Pandey, Rachid Alami
Citations18
SJR quartileQ1
SJR score1.16
SNIP1.94
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is argued that such symbolic level understanding of a task, which is not bound to trajectory, kinematics structure or shape of the robot, facilitates generalization to novel situations as well as ease the transfer of acquired knowledge among heterogeneous robots.
Abstract
International audience
Keywords
Computer ScienceEngineering
Robotics and Autonomous SystemsA survey of robot learning from demonstration
3,256 Citations2008Brenna Argall, Sonia Chernova +2 more
A comprehensive survey of robot Learning from Demonstration (LfD), a technique that develops policies from example state to action mappings, which analyzes and categorizes the multiple ways in which examples are gathered, as well as the various techniques for policy derivation.
Machine LearningExplanation-Based Learning: An Alternative View
835 Citations1986Gerald DeJong, Raymond J. Mooney
Six specific problems with the previously proposed framework for the explanation-based approach to machine learning are outlined and an alternative generalization method to perform explanation- based learning of new concepts is presented.
IEEE Transactions on Robotics and AutomationLearning by watching: extracting reusable task knowledge from visual observation of human performance
680 Citations1994Yasuo Kuniyoshi, Masayuki Inaba +1 more
A novel task instruction method for future intelligent robots that learns reusable task plans by watching a human perform assembly tasks is presented, which results in a hierarchical task plan describing the higher level structure of the task.
Developmental PsychologyCopying actions and copying outcomes: Social learning through the second year.
418 Citations2006Mark Nielsen
The present work documents how the logic of a model's demonstration and the communicative cues that the model provides interact with age to influence how children engage in social learning.
Cambridge University Press eBooksCultural transmission in the tool use and communicatory signaling of chimpanzees?
355 Citations1990Michael Tomasello
Not all population differences are cultural in origin: For example, naturalistic observations have revealed that one population of wild rats dives into the Po River for mollusks, whereas another population of rats living on that same river does not.
Ecological PsychologyVisually Perceiving What is Reachable
313 Citations1989Claudia Carello, Alexis Grosofsky +3 more
Robotics and Autonomous SystemsTeaching and learning of robot tasks via observation of human performance
304 Citations2004Rüdiger Dillmann
An approach for teaching a humanoid robot is presented that will enable the robot to learn typical tasks required in everyday household environments and the main focus is on the knowledge representation in order to be able to abstract the problem solution strategies and to transfer them onto the robot system.
IEEE Transactions on Systems Man and Cybernetics - Part A Systems and HumansEnabling Effective Human–Robot Interaction Using Perspective-Taking in Robots
303 Citations2005J. Gregory Trafton, Nicholas L. Cassimatis +4 more
A fully integrated system that instantiates the theoretical framework for performing perspective-taking and uses the same frames of references that astronauts do to facilitate collaborative problem solving with a person is shown.
KNOWROB — knowledge processing for autonomous personal robots
294 Citations2009Mori Tenorth, Michael Beetz
KINDROB is a first-order knowledge representation based on description logics that provides specific mechanisms and tools for action-centered representation, for the automated acquisition of grounded concepts through observation and experience, for reasoning about and managing uncertainty, and for fast inference — knowledge processing features that are particularly necessary for autonomous robot control.
PubMedThe development in very young children of tacit knowledge concerning visual perception.
286 Citations1977Jacques D. Lempers, Eleanor R. Flavell +1 more
It was found that the majority of 1-year-olds produced and comprehended pointing, and would sometimes hold out a toy to show it, but did little else, and the role of the other's eyes in seeing appeared to be quite well understood at least by age 2-2 1/2.
On Advances in Robot Kinematics
282 Citations2004M. M. Staniisic, Jadran Lenarčič
This book presents the most recent research advances in the theory, design, control and application of robotic systems, which are intended for a variety of purposes such as manipulation, manufacturing, automation, surgery, locomotion and biomechanics.
The International Journal of Robotics ResearchA Hybrid Approach to Intricate Motion, Manipulation and Task Planning
257 Citations2009Stéphane Cambon, Rachid Alami +1 more
The stimulated and unstimulated salivas from aged subjects showed significant reductions in concentrations of both MG1 and MG2, as quantitated in mucin dye-binding units, possible associations of these reductions with the aging process are discussed.
How Children Think and Learn: The Social Contexts of Cognitive Development
178 Citations1988David Wood
Machine LearningA Study of Explanation-Based Methods for Inductive Learning
154 Citations1989Nicholas S. Flann, Thomas G. Dietterich
An empirical study evaluates three methods for solving the problem of identifying a correct concept definition from positive examples such that the concept is some specialization of a target concept defined by a domain theory, and concludes that the new method, IOE, does not exhibit these shortcomings.
Proceedings of the Royal Society B Biological SciencesObservational learning in chimpanzees and children studied through ‘ghost’ conditions
154 Citations2008Lydia M. Hopper, Susan P. Lambeth +2 more
It is concluded that chimpanzees can and will display emulation learning when the task is as simple as the present one, which contrasts with a failure to do so in a more complex manipulative task tested earlier.
ORO, a knowledge management platform for cognitive architectures in robotics
142 Citations2010Séverin Lemaignan, Raquel Ros +3 more
An embeddable knowledge processing framework, along with a common-sense ontology, designed for robotics, that enables in turn reasoning and the implementation of other advanced cognitive functions like events, categorization, memory management and reasoning on parallel cognitive models.
Robotics and Autonomous SystemsUsing perspective taking to learn from ambiguous demonstrations
117 Citations2006Cynthia Breazeal, Matt Berlin +3 more
This paper presents a system where the robot is modeled as a socially engaged and socially cognitive learner that is able to correctly learn from “flawed” demonstrations by taking the visual perspective of the human instructor to clarify potential ambiguities.
The International Journal of Robotics ResearchLearning Non-linear Multivariate Dynamics of Motion in Robotic Manipulators
116 Citations2010Elena Gribovskaya, Seyed Mohammad Khansari-Zadeh +1 more
An algorithm to estimate multivariate robot motions through a mixture of Gaussians, which allows a generalization of motion to unseen context and provides fast on-line replanning of the motion in case of of spatio-temporal perturbations.
Affordance-based imitation learning in robots
111 Citations2007Manuel Lopes, Francisco S. Melo +1 more
An imitation learning algorithm for a humanoid robot on top of a general world model provided by learned object affordances, which is used to recognize the demonstration by another agent and infer the task to be learned.
Child DevelopmentYoung Children's Knowledge about Visual Perception: Hiding Objects from Others
106 Citations1978John H. Flavell, Susan G. Shipstead +1 more
These and other recent findings indicate that children of this age can be both nonegocentric and skillful at estimating what other people do and do not see under various viewing conditions.
International Journal of Advanced Robotic SystemsRobot Learning from Demonstration: A Task-level Planning Approach
104 Citations2008Staffan Ekvall, Danica Kragić
This paper proposes two techniques for constraint identification, and aims to learn the goal of the task and use a task planner to reach the goal given different initial states of the world.
Journal of Physiology-ParisRobotics-based synthesis of human motion
102 Citations2009Oussama Khatib, Emel Demircan +4 more
A new method for the real-time reconstruction of human motion trajectories using direct marker tracking, a task-driven muscular effort minimization criterion and new human performance metrics for dynamic characterization of athletic skills are presented.
IEEE Transactions on Industrial ElectronicsExtraction of essential interactions through multiple observations of human demonstrations
96 Citations2003Koichi Ogawara, Jun Takamatsu +2 more
A new approach on how to teach a robot everyday manipulation tasks under the "Learning from Observation" framework is described, which is examined by using a human-form robot, which successfully imitates human demonstrations of everyday tasks.
Move3D: A generic platform for path planning
92 Citations2002Thierry Siméon, Jean‐Paul Laumond +1 more
Reports on Move3D, a software platform dedicated to collision-free path planning, focuses on results obtained in logistics for industrial installations, in graphics animation and in mobile robotics as well.
Springer tracts in advanced roboticsDemonstration-Guided Motion Planning
88 Citations2016Ye Gu, Ron Alterovitz
Task-level imitation learning using variance-based movement optimization
86 Citations2009Manuel Mühlig, Michael Gienger +3 more
An imitation learning framework is presented, which allows the robot to learn the important elements of an observed movement task by application of probabilistic encoding with Gaussian Mixture Models and shows that the proposed system is suitable for transferring information from a human demonstrator to the robot.
An analysis of rule evaluation metrics
86 Citations2003Johannes Fürnkranz, Peter Flach
All commonly used heuristics, including accuracy, weighted relative accuracy, entropy, Gini index and information gain, are equivalent to one of two fundamental prototypes: precision and cost-weighted difference between covered positive and negative examples.
International Journal of Advanced Robotic SystemsPerceptual Perspective Taking and Action Recognition
74 Citations2005Matthew Johnson, Yiannis Demiris
Simulation of visual perception is used to re-create the visual Egocentric sensory space and egocentric behaviour space of an observed agent, and through this increase the accuracy of action recognition.
Which one? Grounding the referent based on efficient human-robot interaction
73 Citations2010Raquel Ros, Séverin Lemaignan +5 more
This work proposes a set of strategies that allow a robot to identify the referent when the human partner refers to an object giving incomplete information, i.e. an ambiguous description, and proposes the use of an ontology to store and reason on the robot's knowledge to ease clarification, and therefore, improve interaction.
Handling of multiple constraints and motion alternatives in a robot programming by demonstration framework
72 Citations2009Sylvain Calinon, Florent D'halluin +2 more
An approach based on Hidden Markov Model (HMM) and Gaussian Mixture Regression (GMR) is proposed to extract redundancies across multiple demonstrations, and build a time-independent model of a set of movements demonstrated by a human user.
Human Movement ScienceScaling affordances for human reach actions
68 Citations2004Hyeg Joo Choi, Leonard S. Mark
The findings indicate that Cesari and Newell's method can identify a complex relationship between geometric and dynamic constraints that determine the affordances for different reach actions.
Towards grounding concepts for transfer in goal learning from demonstration
62 Citations2011Crystal Chao, Maya Çakmak +1 more
This work presents the scenario of a social robot that learns task goals from human demonstrations without prior knowledge of high-level concepts, and shows how the learning curve improves when transferring the knowledge of grounded concepts to future tasks.
Archives of Internal MedicineLearning and Instinct in Animals.
58 Citations1965Louis L. Lunsky
This is the second edition of this popular volume on the discipline of ethology, which has evolved a large body of data which has implications concerning evolution, phylogenesis, and instinct theory.
Robotics and Autonomous SystemsA cognitive framework for imitation learning
53 Citations2006Antonio Chella, Haris Dindo +1 more
The purpose of the following discussion is to show how this Conceptual Area can be employed to efficiently organize perceptual data, to learn movement primitives from human demonstration and to generate complex actions by combining and sequencing simpler ones.
Learning probabilistic relational planning rules
50 Citations2004Hanna Pasula, Luke Zettlemoyer +1 more
Modeling affordances using Bayesian networks
49 Citations2007Luis Montesano, Manuel Lop +2 more
A general affordance model based on Bayesian networks linking actions, object features and action effects is proposed that is able to deal with uncertainty, redundancy and irrelevant information.
Robotics and Autonomous SystemsEfficient models for grasp planning with a multi-fingered hand
47 Citations2011Jean-Philippe Saut, Daniel Sidobre
The purpose is to compute a context-independent and dense set or list of grasps, instead of just a small set ofgrasps regarded as optimal with respect to a given criterion, to let the robot quickly choose a grasp adapted to a specific situation.
Mightability maps: A perceptual level decisional framework for co-operative and competitive human-robot interaction
46 Citations2010Ajay Kumar Pandey, Rachid Alami
Interestingly Humans are able to maintain rough estimations of visibility, reachability and other capabilities of not only themselves but of the person they are interacting with as well as that of the human partner.
Learning actions from human-robot dialogues
46 Citations2011Rehj Cantrell, Paul Schermerhorn +1 more
A novel algorithm for learning meanings of action verbs through dialogue-based natural language descriptions is proposed and demonstrated to be effective in a scenario where a human explains to a robot the meaning of an action verb unknown to the robot and the robot is subsequently able to carry out the instructions involving this verb.
Springer tracts in advanced roboticsLearning from Experience in Manipulation Planning: Setting the Right Goals
44 Citations2016Anca D. Dragan, Geoffrey J. Gordon +1 more
A method of improving trajectory optimization based on predicting good initial guesses from previous experiences and predicting qualitative attributes of the trajectory that place the initial guess in the basin of attraction of a low-cost solution is described.
International Journal of Social RoboticsTowards a Task-Aware Proactive Sociable Robot Based on Multi-state Perspective-Taking
42 Citations2013Amit Kumar Pandey, Muhammad Ali +1 more
This paper enables robots, equipped with visuo-spatial perspective-taking capabilities, to behave proactively based on reasoning ‘where’ its human partner might perform a particular task with different effort levels to enrich multi-modal interaction and cooperation capabilities of the robot.
Ecological PsychologyHow Do Task Characteristics Affect the Transitions Between Seated and Standing Reaches?
39 Citations2001Douglas L. Gardner, Leonard S. Mark +2 more
The outcome of this investigation indicates that people choose actions that effect the necessary postural control required to complete the task.
Proceedings of the AAAI Conference on Artificial IntelligenceLearning from Demonstration for Goal-Driven Autonomy
39 Citations2021Ben Weber, Michael Mateas +1 more
This work presents a method using case-based reasoning and intent recognition in order to build GDA agents that learn from demonstrations and shows that integrating the GDA conceptual model into the agent greatly improves its win rate.
Towards a platform-independent cooperative human-robot interaction system: I. Perception
38 Citations2010Stéphane Lallée, Séverin Lemaignan +7 more
The resulting system is demonstrated to perform robust object and action learning and recognition on two distinct robotic platforms and it is demonstrated that knowledge acquired about action recognition with one robot can be directly imported and successfully used on a second distinct robot platform for action recognition.
Towards planning Human-Robot Interactive manipulation tasks: Task dependent and human oriented autonomous selection of grasp and placement
36 Citations2012Amit Pandey, Jean-Philippe Saut +2 more
This paper will explore essential human oriented constraints on grasp and placement selections and present a framework to incorporate them in synthesizing key configurations of planning basic interactive manipulation tasks.
A Task Planner for an Autonomous Social Robot
32 Citations2009Samir Alili, Rachid Alami +1 more
An overall description of HATP is provided and the features of its plans evaluation mechanisms are discussed and its performance on a real scenario achieved by a robot in interaction with a human partner in a realistic setup is illustrated.
Linguistic transfer of human assembly tasks to robots
31 Citations2012Neil T. Dantam, Irfan Essa +1 more
This work extends efforts showing the utility of linguistic models in verifiable robot control policies by now performing real visual analysis of human demonstrations to automatically extract a policy for the task.
UPCommons institutional repository (Universitat Politècnica de Catalunya)Integrating task planning and interactive learning for robots to work in human environments
29 Citations2014Alejandro Agostini, Carme Torras +1 more
An interface for interleaved symbolic-geometric planning and backtracking
29 Citations2013Lavindra de Silva, Amit Kumar Pandey +1 more
This paper presents a meaningful interface, and insights into a methodology for developing interwoven symbolic-geometric domains, and presents a prototype implementation of the combined system on a PR2 robot.
HAL (Le Centre pour la Communication Scientifique Directe)ORO, a knowledge management module for cognitive architectures in robotics
29 Citations2010Séverin Lemaignan, Raquel Ros +3 more
An embedded cognitive kernel, along with a common-sense ontology, designed for robotics, that enables in turn reasoning and the implementation of other advanced cognitive functions like events, categorization, memory management and reasonning on parallel cognitive models.
A user-friendly high-level robot teaching system
27 Citations2005Anthony Levas, Mallory Selfridge
An implemented robot system, TL, is taught using a new technique which is easy to use yet produces powerful high-level plans, and an example of it learning to assemble a simple electric switch is presented.
Towards shared attention through geometric reasoning for Human Robot Interaction
25 Citations2009Luis F. Marín-Urías, Emrah Akin Sisbot +3 more
A geometric reasoning mechanism that employs psychological concepts of “perspective taking” and “mental rotation” in order to reason what the human sees, what the robot sees and where the robot should focus to share human's attention is presented.
Affordance graph: A framework to encode perspective taking and effort based affordances for day-to-day human-robot interaction
22 Citations2013Amit Pandey, Rachid Alami
Two complementary aspects of affordances are merged: from agent-object perspective, what an agent afford to do with an object, and from agents-agent perspective,What an agent can afford toDo for other agent are presented, and a unified notion of Affordance Graph is presented.
Towards life-long learning in household robots: The Piagetian approach
18 Citations2007Michael Pardowitz, Rüdiger Dillmann
A method for the autonomous recognition of different task classes in the robots experiences is proposed and one possibility, how this task knowledge can be exploited for incremental learning of sequential reordering features of a task is given.
International Journal of Social RoboticsLearning of Planning Models for Dexterous Manipulation Based on Human Demonstrations
17 Citations2012Rainer Jäkel, Sven R. Schmidt-Rohr +4 more
In this work, planning models for dexterous tasks are learned based on multiple human demonstrations using a general feature space including automatically generated contact constraints, which are automatically relaxed to consider the correspondence problem.
International Journal of Humanoid RoboticsCOLLABORATING WITH HUMANOID ROBOTS IN SPACE
15 Citations2005Donald Sofge, Magdalena Bugajska +9 more
By using shared representations between the human and robot, and enabling the robot to assume the perspectives of the human, the humanoid robot may become a more effective collaborator with a human astronaut for achieving mission objectives in space.
A new approach to combined symbolic-geometric backtracking in the context of human-robot interaction
15 Citations2014Lavindra de Silva, Mamoun Gharbi +2 more
This work presents a new approach to interleaved backtracking, where the symbolic planner backtracks to try alternative action branches that naturally map to different geometric solutions, which eliminates the need to “protect” certain symbolic conditions when backtracking at the geometric level.
International Joint Conference on Artificial IntelligenceCausal learnability
15 Citations2011Loizos Michael
Effective Robot Task Learning by focusing on Task-relevant objects
6 Citations2009Kyu Hwa Lee, Jinhan Lee +2 more
This paper proposes a biologically-inspired computational model that enables the robot to focus on the task-relevant objects, and exhibits recognition capabilities compatible with the way that humans demonstrate.
Learning flexible, multi-modal human-robot interaction by observing human-human-interaction
5 Citations2010Sven R. Schmidt-Rohr, Martin Lösch +1 more
A model is generated using the proposed technique with symbolic states and actions, representing the scope of the observed mission, and used for dynamic action selection in HRI situations.
Artificial Intelligence ReviewExplanation-Based Learning: A survey
5 Citations1992Judith Wusteman
An introductory survey of Explanation-Based Learning is provided to define EBL's position in AI by exploring its relationship to other AI techniques, including other sub-fields of machine learning.
Journal of the Robotics Society of JapanIngredients and a Framework of Dexterous Manipulation Skills for Robots in Human Centered Environment and HRI
1 Citations2014Amit Kumar Pandey, Rachid Alami
The presented control architecture mainly focuses on grasp, and a handcentric dexterous manipulation taxonomy has been presented, which can be used to identify a manipulation strategy for executing a certain task.