Description:
This project is about the development of an automated method to visually detect and recognize fingering gestures of the left hand of a guitarist. The choice of computer vision to perform that task is motivated by the absence of a satisfying method for realtime guitarist fingering detection. The development of this computer vision method follows preliminary manual and automated analyses of video recordings of a guitarist. These first analyses led to some important findings about the design methodology of such a system, namely the focus on the effective gesture, the consideration of the action of each individual finger, and a recognition system not relying on comparison against a knowledge-base of previously learned fingering positions. Motivated by these results, studies on three important aspects of a complete fingering system were conducted. One study was on realtime finger tracking, another on string and fret detection, and the last on movement segmentation. Finally, these concepts were integrated into a prototype and a system for left-hand fingering detection was developed. Such a data acquisition system for fingering retrieval has uses in music theory, music education, automatic music and accompaniment generation and physical modeling.
IDMIL Participants:
External Participants:
Barbara Mazzarino (InfoMus Laboratory, Genoa, Italy)
Gualtiero Volpe (InfoMus Laboratory, Genoa, Italy)
Antonio Camurri (InfoMus Laboratory, Genoa, Italy)
Research Areas:
Funding:
- NSERC
Publications:
- Burns, A. (2007). Computer Vision Methods for Guitarist Left-Hand Fingering Recognition. In M.A. Thesis, McGill University (pp. 114). Montreal, Canada.
- Burns, A., Wanderley, M. M. (2006). Visual Methods for the Retrieval of Guitarist Fingering. In Proceedings of the 2006 International Conference on New Interfaces for Musical Expression (NIME 2006) (pp. 196–199). Paris, France.
- Burns and Marcelo Wanderley, A. M. (2006). Computer Vision Method for Guitarist Fingering Retrieval. In Proceedings of the 2006 Sound and Music Computing Conference (SMC2006). Marseille, France.
- Burns, A., Mazzarino, B. (2005). Finger Tracking Methods Using EyesWeb. .