Graphonic Interface

The Graphonic Interface is an interactive surface for music performance that captures gestures from a set of whiteboard pens and translates them into sound as you draw. The system uses a commercial product called a whiteboard tracker that converts input from pen movement to real-time digital data that can then be analyzed by the computer to produce corresponding music.



Instead of using a whiteboard as the drawing surface, the Graphonic Interface employs a large plate of plexiglass as the performance interface (and as a flat-panel loudspeaker membrane), allowing the musician to stand behind the instrument and face the audience while playing it. The audio that is generated is amplified and played back through the same plexiglass surface, thereby providing tactile feedback to the musician and effectively turning the Graphonic Interface into a pseudo-acoustic musical instrument.

While the first prototype of the Graphonic Interface utilized a clear piece of plexiglass, the use of a semi-transparent (frosted) plexiglass plate would allow for video projection onto the performance surface, and/or highlight the silhouette of the performer with back-lighting from a stage light.

There are 2 directions in which I’d like to pursue the development of this system:
1 - As a performance instrument, where in the performer uses the Graphonic Interface as a musical instrument in a concert setting.
2 - As an installation in a public space where the Graphonic Interface is set up as an exhibit that allows users to create music themselves.

There are also many interesting research issues, like gesture analysis, automated performance environments, animated musical scores, and networked performance that I plan to explore and document with the project. However, this project proposal is not intended to address those issues directly, or aim to solve any related problems.

Below is a short video clip showing an additive synthesis algorithm by Lance Putnam, played by Satoshi Morita. In this performance the tactile element of the interface was not used.

  • Graphonic Interface - 10MB Quicktime movie

    Publications

    The Sonic Scanner and the Graphonic Interface
    Proceedings of the International Computer Music Conference (Miami, Florida, 1-6 November 2004).

    Visually Controlled Synthesis using the Sonic Scanner and the Graphonic Interface
    Proceedings of the 117th Audio Engineering Society Convention (San Francisco, California, 28-31 October 2004).