Description:

This project explores the use of video cameras and projectors and piezoelectric sensing to enable multiple-user musical interaction focused around a tabletop. For the initial prototype a mirror is mounted above a table at an angle such that a video projector mounted behind the table projects onto the table’s surface. A webcam is also mounted behind the table and also uses the mirror to view the same surface. Using video tracking, objects placed on the table – and the hands of the users – may be used to interact with the system.

The software developed during initial exploration of the table interface allows an arbitrary number of objects or users to affect the system simultaneously. Rather than performing video blob tracking, video processing is used to generate a vector field from the web cam input, by which the dynamics of virtual objects or systems projected on the table may be influenced.

Three software demos using the interactive table were prepared for the Eureka! Science Fair in Montréal, where CIRMMT had a room for discussing and demonstrating research projects to the public. Two of the interaction modes use DIMPLE to generate virtual physical dynamics – although DIMPLE is intended for use with haptic interfaces, here the vector field is used to influence the virtual dynamics. The third interaction mode uses the the vector field to animate a control-rate digital waveguidesystem used for scanned synthesis. A video showing DIMPLE- and waveguide-based interaction is below:

IDMIL Participants:

• Joseph Malloch
• Marcelo M. Wanderley
• Stephen Sinclair

Research Areas:

• Interface Design

Funding:

• NSERC