Communicating expressiveness and affect in multimodal interactive systems
IEEE MultimediaPublished 1 January 2005
Antonio Camurri, Gualtiero Volpe, Giovanni De Poli, Marc Leman
Citations137
SJR quartileQ1
SJR score0.68
SNIP0.94
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work presents the multilayer conceptual framework of MIees, algorithms for expressive content analysis and processing, and MIEEs-based art applications.
Abstract
Multisensory integrated expressive environments is a framework for mixed reality applications in the performing arts such as interactive dance, music, or video installations. MIEEs address the expressive aspects of nonverbal human communication. We present the multilayer conceptual framework of MIEEs, algorithms for expressive content analysis and processing, and MIEEs-based art applications.
Keywords
Computer ScienceEngineering
PsycEXTRA DatasetAffective computing
5,051 Citations1997Rosalind W. Picard
Key issues in affective computing, " computing that relates to, arises from, or influences emotions", are presented and new applications are presented for computer-assisted learning, perceptual information retrieval, arts and entertainment, and human health and interaction.
CognitionPerceiving affect from arm movement
613 Citations2001Frank Pollick, Helena Paterson +2 more
The results suggest that the corresponding activation of perceived affect is a formless cue that relates directly to the movement kinematics while the pleasantness of the movement appears to be carried in the phase relations between the different limb segments.
Computer Music JournalEyesWeb: Toward Gesture and Affect Recognition in Interactive Dance and Music Systems
204 Citations2000Antonio Camurri, Shuji Hashimoto +5 more
A modular system for the real-time analysis of body movement and gesture, with a particular focus on the understanding of affect and expressive content in gesture, is developed.
Developmental PsychologyChildren's decoding of emotion in expressive body movement: The development of cue attunement.
179 Citations1998R. Thomas Boone, Joseph G. Cunningham
Children as young as 5 years were shown to rely on emotion-specific movement cues in their decoding of anger and happiness intensity, and the theoretical significance of these effects across development is discussed.
Lecture notes in computer scienceMultimodal Analysis of Expressive Gesture in Music and Dance Performances
175 Citations2004Antonio Camurri, Barbara Mazzarino +3 more
Research outputs include computational models of expressive gesture, validation by means of continuous ratings on spectators exposed to real artistic stimuli, and novel hardware and software components for the EyesWeb open platform, such as the recently developed Expressive Gesture Processing Library.
Computer Music JournalEmotional Coloring of Computer-Controlled Music Performances
166 Citations2000Roberto Bresin, Anders Friberg
A complex, ecological-predictive ANN was designed that was able to learn a professional pianist's performance style at the structural micro-level and listeners were able to recognize the intended emotional colorings.
Musicae ScientiaeCan Dance Reflect the Structural and Expressive Qualities of Music? A Perceptual Experiment on Balanchine's Choreography of Mozart's Divertimento No. 15
141 Citations1997Carol L. Krumhansl, Diana Lynn Schenck
Lecture notes in computer scienceAnalysis of Expressive Gesture: The EyesWeb Expressive Gesture Processing Library
100 Citations2004Antonio Camurri, Barbara Mazzarino +1 more
A collection of algorithms and related software modules for real-time analysis of expressive gesture in full-body human movement are presented for the EyesWeb open architecture (freely available from www.eyesweb.org).
Journal of New Music ResearchPrediction of Musical Affect Using a Combination of Acoustic Structural Cues
77 Citations2005Marc Leman, Valery Vermeulen +3 more
The results indicate that musical affect attribution can partly be predicted using a combination of acoustical structural cues, and manual structural cues worked better than acoustically structural cues.
Proceedings of the IEEEModeling and Control of Expressiveness in Music Performance
63 Citations2004Sergio Canazza, Giovanni De Poli +3 more
This paper discusses a model that applies a smooth morphing among performances with different expressive content, adapting the audio expressive character to the user's desires, and introduces an original method for the spectral processing of audio.
Journal of New Music ResearchAn Abstract Control Space for Communication of Sensory Expressive Intentions in Music Performance
55 Citations2003Sergio Canazza, Giovanni De Poli +2 more
A new model is suggested for computer understanding of sensory expressive intentions of a human performer and both theoretical and practical applications are described for human-computer interaction, perceptual information retrieval, creative arts and entertainment.
The Journal of Visualization and Computer AnimationApplication of multimedia techniques in the physical rehabilitation of Parkinson's patients
49 Citations2003Antonio Camurri, Barbara Mazzarino +4 more
The aim is to develop a computational open architecture in which it is possible to integrate modules for gesture analysis and recognition and for interactive construction of therapeutic exercises based on multimedia stimulation in real time.
Cognition Technology & WorkExpressive interfaces
38 Citations2004Antonio Camurri, Barbara Mazzarino +1 more
Some recent developments in the research on expressive interfaces are described by presenting computational models and algorithms for the real-time analysis of expressive gestures in human full-body movement.
User-dependent taxonomy of musical features as a conceptual framework for musical audio-mining technology
23 Citations2003Micheline Lesaffre, Marc Leman +4 more
The need for user-dependent taxonomy development is addressed and an experiment in spontaneous user behavior is described, and how categories and concepts from the statistical analysis have been used for the refinement of taxonomies that address user interactive interfacing and feature extraction are described.
PRESENCE Virtual and Augmented Reality“It/I”: A Theater Play Featuring an Autonomous Computer Character
20 Citations2002Cláudio Pinhanez, Aaron Bobick
The interval script paradigm used to program the computer character and the ACTSCRIPT language for communication of actions and goals can successfully address the control and management of any virtual environment with a complex temporal structure or a strong underlying narrative.
Journal of New Music ResearchSynthesizing Expressive Music Through the Language of Conducting
18 Citations2002Teresa Marrin Nakra
The Conductor's Jacket software system is described, a sensor interface that gathers its wearer's gestures and physiology and generates real-time expressive effects by shaping the note onsets, tempos, articulations, dynamics, and note lengths in a musical score.
Proceedings of the IEEERobotic Interface for Embodied Interaction via Dance and Musical Performance
15 Citations2004Kenji Suzuki, S. Hahimoto
Four musical platforms utilizing robotic technology and information technology in different circumstances are introduced, effective designing environments for artists such as musicians, composers, and choreographers, not only for music creation but also for media coordination including motion and visual effects.