Interaction between auditory and visual processing in impressional evaluation of a piano performance

Abstract

To investigate the interaction between auditory and visual processing in impressional evaluation of a piano performance, a pianist was asked to play the same musical excerpt in three different expressive levels and these were recorded on a video. Three kinds of sounds and three kinds of body movements were combined and these nine performances were presented to musically trained students and less trained students. They rated the expressivity of each performance on a five-point scale in three modes: sound and vision together, sound alone, and vision alone. When sound and vision were presented, they also rated their subjective acceptability on a five-point scale. The result was that both musically trained students and less trained students could discern among the levels of a pianist’s expressive intent when sound alone or vision alone was presented. However, when sound and vision were presented, it was found that vision had a greater influence on the impressional evaluation. This tendency was more remarkable for less trained students than musically trained students. When sound and body movements were at different levels, performances were rated lower by musically trained students but not by less trained students.