The Sonification Project

Participants: Alexandre Savard, Vincent Verfaille (SPCL), Marcelo M. Wanderley

Former Participant: Oswald Quek

This project is about sonification of musicians' ancillary gestures using data provided by motion capture system such as the Vicon, the Optotrak or the Polhemus. The analysis of human motion is challenging first due to high dimensionality and high variability (intra-subject, inter-subject) but also because of non-linear relationship among data. Recent researches applied an alternate representation that combines visual and auditory stimuli to enhance perception of human movements.

We aim at developing sonification tools in Max/MSP and MATLAB to provide researchers a way to study the complex relationships between gestures. Our goal is to develop a modular framework that allows for different combinations of sounds designed to identify specific gestures once they occur. This topic has been explored in a recent publication on the sonification of clarinetists' ancillary gestures. The next step is to apply a coherent sonification strategy to other instruments/performers that present their own gestural patterns.

Publications

Vincent Verfaille, Oswald Quek, and Marcelo M. Wanderley. “Sonification of Musician’s Ancillary Gestures.” In Proc. of the 2006 International Conference on Auditory Display (ICAD’06), London, England, 2006, pp. 194–7.

Vincent Verfaille and Marcelo M. Wanderley. “Mapping Strategies for Sound Synthesis, Digital Audio Effects and Sonification of Performer Gestures.” Paper presented during the Acous. Soc. Am. Meeting, Rhodes Island, June 2006.