Gesture In Reactive Soundplace

Sonic Tai Chi - Auditory Gestures in Reactive Soundspaces Joanne Jakovich

Kirsty Beilharz

Key Centre of Design Computing University of Sydney, City Road, Sydney, Australia +61-2-9351-2053

Key Centre of Design Computing University of Sydney, City Road, Sydney, Australia +61-2-9351-4031

[email protected]

[email protected]

ABSTRACT Sonic Tai Chi is an interactive soundlab for exploring the relationship between environmental intelligence and human interaction. Spaces that intelligently respond to our actions are increasingly part of our work, play and learning environments. However the use of sound for both interfacing and specifying these environments is still in stages of early exploration. The Sonic Tai Chi soundspace extends the function of a typical interactive installation. It is a lab for developing the foundations for intelligent interactive environments whereby sound is used not as an ambient measure or indication of human interaction or environmental intelligence, but actively employed as a spatio-temporal medium for designing space and spatial experience using gestures. Thus the central purpose of the lab is to develop an understanding about human gestural control of sound. By building a detailed understanding of the particular propensities of gesture toward certain patterns of sound generation it is conceived that a model to describe auditory gesture in interactive soundspaces can be constructed.

Figure 2. Screen stills showing optical flow visualization The movement information is analysed in realtime to extract the features and clusters of the gestural activity within the soundspace.

The processing system developed in MaxMSP and Jitter uses the Horn-Schunk method to estimate optical flow of movement in captured using a simple Internet conference camera. Hence gloves or markers are not necessary to capture accurate gestural interaction in space.

Figure 3. Realtime video data signal of gestural activity Users explore the spatiality of the reactive soundspace using gestural interaction. Visualisation reinforces the interaction between the reactive sounds (cellular automata) and gestures.

Figure 1. Processing environment in MaxMSP and Jitter Figure 4. Video stills of Sonic Tai Chi in gallery context

Copyright is held by the author/owner Proceedings of the Second Australasian Conference on Interactive Entertainment, Sydney, Australia ISBN 0-9751533-2-3/05/11

Please see http://www.jakovich.net/sonic

243