Multimodal interaction with a wearable augmented reality system
IEEE Computer Graphics and ApplicationsPublished 1 May 2006
Mathias Kölsch, R. Bane, Tobias Höllerer, Matthew Turk
Citations57
SJR quartileQ3
SJR score0.33
SNIP1.11
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
An augmented reality system enhances a mobile user's situational awareness and provides new visualization functionality. The custom-built multimodal interface provides access to information encountered in urban environments. In this article, we detail our experiences with various input devices and modalities and discuss their advantages and drawbacks in the context of interaction tasks in mobile computing. We show how we integrated the input channels to use the modalities beneficially and how this enhances the interface's overall usability
Keywords
Computer Science
Toolglass and magic lenses
1,149 Citations1993Eric A. Bier, Maureen Stone +3 more
ToolglassTM widgets are new user interface tools that can appear, as though on a transparent sheet of glass, between an application and a traditional cursor, and form a see-through interface that offers many advantages over traditional controls.
Personal and Ubiquitous ComputingA touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment
939 Citations1997Steven Feiner, Blair MacIntyre +2 more
A prototype system that combines the overlaid 3D graphics of augmented reality with the untethered freedom of mobile computing is described, to explore how these two technologies might together make possible wearable computer systems that can support users in their everyday interactions with the world.
A touring machine: prototyping 3D mobile augmented reality systems for exploring the urban environment
272 Citations2002Steven Feiner, Blair MacIntyre +2 more
Robust hand detection
268 Citations2004Mathias Kölsch, Matthew Turk
This paper presents a frequency analysis-based method for instantaneous estimation of class separability, without the need for any training, and builds detectors for the most promising candidates, their receiver operating characteristics confirming the estimates.
Fast 2D Hand Tracking with Flocks of Features and Multi-Cue Integration
184 Citations2005Mathias Kölsch, Matthew Turk
This paper introduces "Flocks of Features," a fast tracking method for non-rigid and highly articulated objects such as hands that combines KLT features and a learned foreground color distribution to facilitate 2D position tracking from a monocular view.
Mutual disambiguation of 3D multimodal interaction in augmented and virtual reality
173 Citations2003Ed Kaiser, Alex Olwal +6 more
An approach to 3D multimodal interaction in immersive augmented and virtual reality environments that accounts for the uncertain nature of the information sources and fuses symbolic and statistical information from a set of 3D gesture, spoken language, and referential agents is described.
Structured visual markers for indoor pathfinding
114 Citations2003Michael Kalkusch, Thomas Lidy +4 more
A mobile augmented reality (AR) system to guide a user through an unfamiliar building to a destination room using a structured marker re-use scheme based on graph coloring has been developed.
Interactive Tools for Virtual X-Ray Vision in Mobile Augmented Reality
102 Citations2005R. Bane, Tobias Höllerer
Four interactive tools are presented: the tunnel tool and room selector tool directly augment the user's view of the environment, allowing them to explore the scene in direct, first person view, and the room in miniature tool allows the user to select and interact with a room from a third person perspective.
Multimodal system processing in mobile environments
91 Citations2000Sharon Oviatt
Although speech recognition as a stand-alone failed more often during mobile system use, the results confirmed that a more stable multimodal architecture decreased this error rate by 1935% and were replicated across different types of microphone technology.
Vision-based interfaces for mobility
69 Citations2004Mathias Kölsch, Matthew Turk +1 more
A robust real-time hand gesture recognition system that is capable of being the sole input provider for a demonstration application and achieves usability and interactivity even when both the head-worn camera and the object of interest are in motion.
Virtual RealityGlove Based User Interaction Techniques for Augmented Reality in an Outdoor Environment
66 Citations2002Bruce H. Thomas, W. Piekarski
A set of pinch glove-based user interface tools for an outdoor wearable augmented reality computer system and a set of augmented reality information presentation techniques to support direct manipulation.
Oxford University Press eBooksInput Devices and Interaction Techniques for Advanced Computing
63 Citations1995I. Scott MacKenzie
Speaking and listening on the run: design for wearable audio computing
51 Citations2002Nitin Sawhney, C. Schmandt
This work demonstrates the use of auditory cues, spatialized audio, and speech I/O in the wearable interface for passive awareness, scaleable notification and navigation/control.
Speech and gesture multimodal control of a whole Earth 3D visualization environment
42 Citations2002David M. Krum, Olugbenga Omoteso +3 more
VizWear-Active: Towards a functionally-distributed architecture for real-time visual tracking and context-aware UI
5 Citations2003T. Kato, Takeshi Kurata +1 more
A visual augmented memory application based on real-time visual tracking by the wearable active camera module and arm gesture recognition by the wristwatch UI module is implemented and demonstrated.