Body Music is a physically interactive system that extracts expressive motion features from the players full-body movements and gestures in a room setting. The values of these motion features are mapped both onto acoustic parameters for the real-time expressive rendering of a piece of music, and onto real-time generated visual feedback projected on a screen in front of the player.

Body Music: teaching pitch

Users standing far apart creates a lower frequency sound, and as they move closer together the frequency raises. (higher pitch)


This intuitive way of exploring music provides a deeper sensitization of the nature of music itself, such as frequency, amplitude, tempo and time signature, and hence lowers the barriers in appreciating music. This can also inspire more people to learn music and music theory as well as develop newfangled approaches to music.


Body Music: teaching time signature

(a) Three people in the space triggers the melody to play in 3 / 4 time (b) Four people in the space triggers the melody to play in 4 / 4 time (c) Six people in the space triggers the melody to play in 6 / 8 time


This system was built using simple computer vision techniques and precedes the position/gesture sensing technologies that are prevalent today (Kinect, Leap Motion, etc.)

Body Music: back end built in MAX/MSP

System in operation: MAX/MSP program on the right and the visual output on the left


Khoo, E., Merritt, T., Fei, L.F., Liu, W., Rahaman, H., Prasad, J., Marsh, T. 2008. Body Music: Physical Exploration of Music Theory. In Proceedings of the 2008 ACM SIGGRAPH Symposium on Videogames (Los Angeles, California, August 10, 2008). Sandbox ’08. ACM, New York, NY, 35-42.