Electroacoustic_music_as_embodied_cognit.pdf

ELECTROACOUSTIC MUSIC AS EMBODIED COGNITIVE PRAXIS DENIS SMALLEY’S THEORY OF SPECTROMORPHOLOGY AS AN IMPLICIT THEORY OF EMBODIED COGNITION Brian Bridges, Ulster University, Northern Ireland Ricky Graham, Stevens Institute of Technology, NJ, USA EMS2015 University of Sheffield

OUTLINE 1. INTRODUCTION: CONNECTING THE DISCOURSES OF ELECTROACOUSTIC MUSIC AND COGNITIVE SCIENCE 2. EMBODIED–ECOLOGICAL MODELS: COMPARING EMBODIED IMAGE SCHEMA THEORY (LAKOFF AND JOHNSON) WITH SPECTROMORPHOLOGY 3. SOUNDED AFFORDANCES: BEYOND CANONICAL IMAGE SCHEMAS TO THE ATTRIBUTES OF SOUNDED GESTURE– TEXTURES IN ELECTROACOUSTIC MUSIC 4. FORMS, DIMENSIONS AND COMBINATIONS: HOW SCHEMAS AND THEIR ATTRIBUTES MAY RELATE TO FORM (HOPEFULLY!) SOME CONCLUSIONS

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

e.g. two modes/models of ‘perception and cognition’

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

e.g. two modes/models of ‘perception and cognition’

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

e.g. two modes/models of ‘perception and cognition’

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

e.g. two modes/models of ‘perception and cognition’

‘top–down’

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

e.g. two modes/models of ‘perception and cognition’

‘top–down’

‘bottom–up’

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

e.g. two modes/models of ‘perception and cognition’ formal model

‘top–down’

‘bottom–up’

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

e.g. two modes/models of ‘perception and cognition’ formal model note distances, no. of points, etc.

‘top–down’

‘bottom–up’

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

e.g. two modes/models of ‘perception and cognition’ formal model note distances, no. of points, etc. match to known template

‘top–down’

‘bottom–up’

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

e.g. two modes/models of ‘perception and cognition’ formal model note distances, no. of points, etc. match to known template then (maybe) interact

‘top–down’

‘bottom–up’

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

e.g. two modes/models of ‘perception and cognition’ object with affordances

formal model note distances, no. of points, etc. match to known template then (maybe) interact

‘top–down’

‘bottom–up’

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

e.g. two modes/models of ‘perception and cognition’ object with affordances

formal model note distances, no. of points, etc.

key features are obvious... environment guides cognition

match to known template then (maybe) interact

‘top–down’

‘bottom–up’

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

e.g. two modes/models of ‘perception and cognition’ object with affordances

formal model note distances, no. of points, etc.

key features are obvious... environment guides cognition

match to known template then (maybe) interact

‘top–down’

‘bottom–up’

=> watch out for sharp edges!

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

•

From reduced listening to source–bonding and sonic–gestural ecologies

e.g. two modes/models of ‘perception and cognition’ object with affordances

formal model note distances, no. of points, etc.

key features are obvious... environment guides cognition

match to known template then (maybe) interact

‘top–down’

‘bottom–up’

=> watch out for sharp edges!

1. INTRODUCTION: ENVIRONMENTAL MATERIALS, STRUCTURES, DISCOURSES •

Environmental materials in music & ‘bottom–up’ revolution in cognitive sci.

•

From reduced listening to source–bonding and sonic–gestural ecologies

•

Sound gestures, ecological/embodied grammars, spectromorphology

e.g. two modes/models of ‘perception and cognition’ object with affordances

formal model note distances, no. of points, etc.

key features are obvious... environment guides cognition

match to known template then (maybe) interact

‘top–down’

‘bottom–up’

=> watch out for sharp edges!

tonal center (e.g. triadic) will cause movement of voices towards the center. More tonally distant materials (e.g. chromatic) will cause movement towards the periphery. Many musicians will be familiar with these general structuring principles of tonal relationships in common practice music and will recognize the flocking behaviors as broadly predictable and related to their own understanding of musical macrostructures. Thus, the use of embodied correlates between input and output provides a means of managing emergent complexity via conceptual mapping [4,7,14,16]. • Cross–domain mapping: image

domains, facilitating accessible and immersive designs for NIME. Schacher et al. [13] use the term ecological relationships for a related idea, the blending of human interaction gestures and system responses. Brower [2,3] and Johnson [8] have previously applied image schema theories to tonal music. They have also been investigated for their practical significance in music software design [16,17]. Some of the key basic image schemas are illustrated below (figure 3).

2. EMBODIED–ECOLOGICAL: IMAGE SCHEMAS AND SPECTROMORPHOLOGY the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as making a circling motion with one's head produces a pronounced degree of cyclical individuation/arpeggiation123 ofone's a head produces a pronounced degree of cyclical individuation/arpeggiation123 of a number of the higher voices. In addition, a repetition of Gann's experiment of tugging number of the higher voices. In addition, a repetition of Gann's experiment of tugging on an earlobe does bring about perceptible changes of a high frequency (but pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched)

component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of

Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below.

the experience. Continuous repetitive movement such as making a circling motion with

the experience. Continuous repetitive movement such as making a circling motion with

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

number of the higher voices. In addition, a repetition of Gann's experiment of tugging

number of the higher voices. In addition, a repetition of Gann's experiment offorce tuggingof

on an earlobe does bring about perceptible changes of a high frequency (but pitched)

on an earlobe does bring about perceptible changes of a high frequency (but gravity pitched)

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

of acoustical factors which influence the perception of the frequency structures of

of acoustical factors which influence the perception of the frequency structures of

schemas (Lakoff and Johnson) as common sensorimotor patterns ‘imported into cognition’

An Ecological Melodic ! Pitch Space Young's installations in various ways. These factors are summarised in figure 30, below.

Young's installations in various ways. These factors are summarised in figure 30, below.

high tendency values mapped to speed ‘matching instinct’ the experience. Continuous repetitive movement such as making a circling motion with steering behaviors relative to neighboring one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a boids (e.g. pc 2number -> pc0) of the higher voices. In addition, a repetition of Gann's experiment of tugging

low melodic ! attraction (expectancytension) (e.g. pc1 -> pc6) such as making a circling motion with the experience. Continuous repetitive movement flock towards periphery one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a high implicative denial values

X

number of the higher voices. In addition, a repetition of Gann's experiment of tugging Figure 30: The relationship between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, listener HRTF and the

•

Image schemas (and related ideas) in music research:

on an earlobe does bring about perceptible changesresulting of a highpercept frequency (but pitched) in Young's sound installations

resulting percept in Young's sound installations

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

on an earlobe does bring about perceptible changes of a high frequency (but pitched) component (perhaps in the region of 4 kHz). In summary, there appear to be a number

of acoustical factors which influence the perception of the frequency structures of

123

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

CENTRE–PERIPHERY CENTER–PERIPHERY

CONTAINER

Young's installations in various ways. These factors are summarised in figure 30, below.

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207

• • • • • • •

VERTICALITY

of acoustical factors which influence the perception of the frequency structures of

Young's installations in various ways. These factors are summarised in figure 30, below. 123

207

Brower (2000); Adlington (2003) Godøy (2003; 2006) 8ve Kendall and Ardillia (2008) Wilkie et. al. (2010) Our own previous work (2013–15) Roddy and Furlong (2014) Barrett (2015)

X

the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as making a circling motion with

one'sroom, headlistener produces a pronounced Figure 30: The relationship between source material, HRTF and the degree of cyclical individuation/arpeggiation resulting percept in Young's sound installations

123

123 ofone's a head produces a pronounced degree of cyclicalFigure individuation/arpeggiation of a source material, room, listener HRTF and the 30: The relationship between

number of the higher voices. In addition, a repetition of Gann's experiment of tugging number of the higher voices. In addition, a repetitionresulting of Gann'spercept experiment of tugging in Young's sound installations on an earlobe does bring about perceptible changes of a high frequency (but pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched) component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number

123

of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of 123

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207 These factors are summarised in figure 30, below. Young's installations in various ways. Young's installations in various ways. These factors are summarised in figure 30, below.

207

high melodic ! attraction (e.g. pc1 -> pc0) flock towards center

CYCLE

Figure 30: The relationship between source material, room, listener HRTF and the

Figure 30: The relationship between source material, room, listener HRTF and the

resulting percept in Young's sound installations

resulting percept in Young's sound installations

Key Q: Is spectromorphology compatible with image schema Figure 2: Dynamic tonal–spatial mappings of initial system theory? •

123

Major Ionian Framework! 207 ‘Ionian Space’

123

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

123

resulting percept in Young's sound installations

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.123 This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207

These image schemas may be applied to goal–oriented tonal tension/release (center–periphery, container, source–path– BALANCE along with hierarchical pitch goal) or cyclical structures, relationships (verticality). [2,3] suggest common practice tonal structures may be based on these image schemas]. Our previous Image schemas: embodied conceptual metaphors work [6] has attempted to unify these ideas with cognitively– based theories of tonality [10,11], culminating in the tonal– spatial mappings discussed above, which adopt center– 207

Figure 30: The relationship between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, listener HRTF and the resulting percept in Young's sound installations

Figure 3: Key embodied image schemas

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

iteration, ‘animating’ a tonal hierarchy and attraction model derived from Lerdahl [10,11] using a boids algorithm to control the spatialization of voices

These approaches have provided the basis for a further

207

SOURCE––PATH––GOAL

tonal center (e.g. triadic) will cause movement of voices towards the center. More tonally distant materials (e.g. chromatic) will cause movement towards the periphery. Many musicians will be familiar with these general structuring principles of tonal relationships in common practice music and will recognize the flocking behaviors as broadly predictable and related to their own understanding of musical macrostructures. Thus, the use of embodied correlates between input and output provides a means of managing emergent complexity via conceptual mapping [4,7,14,16]. • Cross–domain mapping: image

domains, facilitating accessible and immersive designs for NIME. Schacher et al. [13] use the term ecological relationships for a related idea, the blending of human interaction gestures and system responses. Brower [2,3] and Johnson [8] have previously applied image schema theories to tonal music. They have also been investigated for their practical significance in music software design [16,17]. Some of the key basic image schemas are illustrated below (figure 3).

2. EMBODIED–ECOLOGICAL: IMAGE SCHEMAS AND SPECTROMORPHOLOGY the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as making a circling motion with one's head produces a pronounced degree of cyclical individuation/arpeggiation123 ofone's a head produces a pronounced degree of cyclical individuation/arpeggiation123 of a number of the higher voices. In addition, a repetition of Gann's experiment of tugging number of the higher voices. In addition, a repetition of Gann's experiment of tugging on an earlobe does bring about perceptible changes of a high frequency (but pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched)

component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of

Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below.

the experience. Continuous repetitive movement such as making a circling motion with

the experience. Continuous repetitive movement such as making a circling motion with

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

number of the higher voices. In addition, a repetition of Gann's experiment of tugging

number of the higher voices. In addition, a repetition of Gann's experiment offorce tuggingof

on an earlobe does bring about perceptible changes of a high frequency (but pitched)

on an earlobe does bring about perceptible changes of a high frequency (but gravity pitched)

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

of acoustical factors which influence the perception of the frequency structures of

of acoustical factors which influence the perception of the frequency structures of

schemas (Lakoff and Johnson) as common sensorimotor patterns ‘imported into cognition’

An Ecological Melodic ! Pitch Space Young's installations in various ways. These factors are summarised in figure 30, below.

Young's installations in various ways. These factors are summarised in figure 30, below.

high tendency values mapped to speed ‘matching instinct’ the experience. Continuous repetitive movement such as making a circling motion with steering behaviors relative to neighboring one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a boids (e.g. pc 2number -> pc0) of the higher voices. In addition, a repetition of Gann's experiment of tugging

low melodic ! attraction (expectancytension) (e.g. pc1 -> pc6) such as making a circling motion with the experience. Continuous repetitive movement flock towards periphery one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a high implicative denial values

X

number of the higher voices. In addition, a repetition of Gann's experiment of tugging Figure 30: The relationship between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, listener HRTF and the

•

Image schemas (and related ideas) in music research:

on an earlobe does bring about perceptible changesresulting of a highpercept frequency (but pitched) in Young's sound installations

resulting percept in Young's sound installations

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

on an earlobe does bring about perceptible changes of a high frequency (but pitched) component (perhaps in the region of 4 kHz). In summary, there appear to be a number

of acoustical factors which influence the perception of the frequency structures of

123

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

CENTRE–PERIPHERY CENTER–PERIPHERY

CONTAINER

Young's installations in various ways. These factors are summarised in figure 30, below.

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207

• • • • • • •

VERTICALITY

of acoustical factors which influence the perception of the frequency structures of

Young's installations in various ways. These factors are summarised in figure 30, below. 123

207

Brower (2000); Adlington (2003) Godøy (2003; 2006) 8ve Kendall and Ardillia (2008) Wilkie et. al. (2010) Our own previous work (2013–15) Roddy and Furlong (2014) Barrett (2015)

X

the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as making a circling motion with

one'sroom, headlistener produces a pronounced Figure 30: The relationship between source material, HRTF and the degree of cyclical individuation/arpeggiation resulting percept in Young's sound installations

123

123 ofone's a head produces a pronounced degree of cyclicalFigure individuation/arpeggiation of a source material, room, listener HRTF and the 30: The relationship between

number of the higher voices. In addition, a repetition of Gann's experiment of tugging number of the higher voices. In addition, a repetitionresulting of Gann'spercept experiment of tugging in Young's sound installations on an earlobe does bring about perceptible changes of a high frequency (but pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched) component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number

123

of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of 123

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207 These factors are summarised in figure 30, below. Young's installations in various ways. Young's installations in various ways. These factors are summarised in figure 30, below.

207

high melodic ! attraction (e.g. pc1 -> pc0) flock towards center

CYCLE

Figure 30: The relationship between source material, room, listener HRTF and the

Figure 30: The relationship between source material, room, listener HRTF and the

resulting percept in Young's sound installations

resulting percept in Young's sound installations

Key Q: Is spectromorphology compatible with image schema Figure 2: Dynamic tonal–spatial mappings of initial system theory? •

123

Major Ionian Framework! 207 ‘Ionian Space’

123

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

123

resulting percept in Young's sound installations

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.123 This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207

These image schemas may be applied to goal–oriented tonal tension/release (center–periphery, container, source–path– BALANCE along with hierarchical pitch goal) or cyclical structures, relationships (verticality). [2,3] suggest common practice tonal structures may be based on these image schemas]. Our previous Image schemas: embodied conceptual metaphors work [6] has attempted to unify these ideas with cognitively– based theories of tonality [10,11], culminating in the tonal– spatial mappings discussed above, which adopt center– 207

Figure 30: The relationship between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, listener HRTF and the resulting percept in Young's sound installations

Figure 3: Key embodied image schemas

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

iteration, ‘animating’ a tonal hierarchy and attraction model derived from Lerdahl [10,11] using a boids algorithm to control the spatialization of voices

These approaches have provided the basis for a further

207

SOURCE––PATH––GOAL

al mapping [4,7,14,16].

ion123 of a

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

of voices domains, facilitating accessible and immersive force of designs for NIME. gravity erials (e.g. Schacher et al. [13] use the term ecological relationships for a DESCENT (ground/stability) An Ecological Melodic ! hery. Many related idea, the blending of human interaction gestures and VERTICALITY Pitch Space structuring system responses. high tendency values gorithm resulting sound structure. our system combines mapped to More specifically,ASCENT X ennecting music and physical Brower [2,3] and Johnson [8] havethe previously applied image X speed ‘matching instinct’ motion–tracked movements with figurative steering behaviors 2 predictable schema theories to tonal music. They have also been relative to neighboring ’s tonal gestures [6] of musical macrostructures, such as pitch contours boids (e.g. pc 2 -> pc0) SOURCE-----------PATH------------------GOAL of musical and rhythmic investigated for (see their practical music software ee figure groupings figure 1). Thesignificance shared gestural in space PLANE tes between Some of the mapping, key basic close to is the design unifying[16,17]. ‘glue’ connecting input, and image output schemas are VERTICALITY [A] standard (linear) gestures [B] cycle gestures (rotation spin) CONTAINER CENTER–PERIPHERY (RECIPROCAL = COMBINATIONS OF and THE ABOVE) domains, facilitatingbelow accessible and 3). immersive designs for NIME. g voices emergent [VERTICALITY illustrated (figure + SOURCE–PATH–GOAL] [EMBODIED DYNAMICS OF DIFFERENT CYCLE RATES]

f tugging

number of the higher voices. In addition, a repetition of Gann's experiment of tugging

pitched)

on an earlobe does bring about perceptible changes of a high frequency (but pitched)

a number

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

res of

SPECTROMORPHOLOGY AND IMAGE SCHEMAS of acoustical factors which influence the perception of the frequency structures of

e 30, below.

Young's installations in various ways. These factors are summarised in figure 30, below.

e movement such as making a circling motion with

egree of cyclical individuation/arpeggiation123 of a

motion with tion, a repetition of Gann's experiment of tugging

123 changes of a high frequency (but pitched) ptible ion of a

kHz). In summary, there appear to be a number f4 tugging

the experience. Continuous repetitive movement such as making a circling motion with one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

number the higher In addition, a repetition of Gann's experiment of tugging ip between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, of listener HRTFvoices. and the

the perception of the frequency structures of pitched) g's sound installations

resulting percept in Young's sound installations

s. These factors are summarised in figure 30, below.

on an earlobe does bring about perceptible changes of a high frequency (but pitched)

a number

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

res of

of acoustical factors which influence the perception of the frequency structures of

e 30, below.

ls (e.g. y. Many ucturing usic and 8ve edictable tendency values ped to musical d ‘matching instinct’ ng behaviors between ve to neighboring mergent (e.g. pc 2 -> pc0)

Young's installations in various ways. These factors are summarised in figure 30, below.

Schacher et al. [13] use the term ecological relationships for a related idea, theforceblending of human interaction gestures and of VERTICALITY gravity DESCENT system responses. ASCENT (ground/stability) Brower [2,3] and Johnson [8] have previously applied image schema theories to tonal music. They have also been investigated for their practical significance Xin music softwareX design [16,17]. Some of thePLANE key basic image schemas are illustrated below (figure 3). SOURCE-----------PATH------------------GOAL

application of what Bregman (1990) terms the 'old-plus-new heuristic’.123 This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

the experience. Continuous repetitive movement207 such as making a circling motion with

207

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a number of the higher voices. In addition, a repetition of Gann's experiment of tugging

rotation=slow (internal) rotation path is (still) clear

on an earlobe does bring about perceptible changes of a high frequency (but pitched)

such as making a circling motion with

s repetitive movement such as making circling with the experience. Continuous component (perhaps in thearegion of 4motion kHz). In summary, there appear to be arepetitive number movement such as making a circling motion with

ical individuation/arpeggiation123 of a

123 123 nounced cyclical factors individuation/arpeggiation ofone's a ofhead produces structures a pronounced individuation/arpeggiation of a source material, room, listener HRTF and the and the degreeofofacoustical 30: The relationship between which influence the perception the frequency of degree of cyclicalFigure

tion of Gann's experiment of tugging

Young's installations in various ways.of These factors are of summarised figure 30, below. a repetitionresulting es. addition, a repetition of Gann's experiment tugging number the higherinvoices. In addition, of Gann'spercept experiment of tugging in Young's sound installations es ofIna high frequency (but pitched)

ummary, there appearchanges to be a number bout perceptible of a high

on of the frequency structures of

X

frequency (but pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched)

e region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number

ors are summarisedthe in experience. figure 30, below. Continuous repetitive movement such as making a circling motion with

h influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of 123

heuristic’.

ro ro un re co cy im

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

207 These factors are summarised in figure 30, below. arious ways. Young's installations in various ways. These factors are summarised in figure 30, below.

207

number the higher In addition, a repetition of Gann's experiment of tugging source material, room, of listener HRTFvoices. and the

high ! about perceptible changes of a high frequency (but pitched) on an melodic earlobe does bring attraction (e.g. pc1 -> pc0) component (perhaps in the region of 4 kHz). In summary, there appear to be a number flock towards center number of the higher voices. In addition, a repetition of Gann's experiment offorce tuggingof the experience. Continuous repetitive movement such as making a circling motion with

nstallations

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

ROTATION (CYCLE) ! CYCLE VERTICALITY CONTAINER CENTER–PERIPHERY DESCENT [system is stable]!! (RECIPROCALFigure = COMBINATIONS OF THE ABOVE) 30: The relationship between source material, room, listener HRTF and the Young's installations in various ways. These factors are summarised in figure 30, below. of acoustical factors which influence the perception of the frequency structures of

on an earlobe does bring about perceptible changes of a high frequency (but gravity pitched)

and the

component in thetheregion of 4 kHz). In summary, there appear to be a number f what Bregman(perhaps (1990) terms 'old-plus-new heuristic’.

(ground/stability) 207

resulting percept in Young's sound installations

of acoustical factors which influence the perception of the frequency structures of

Young's installations in various ways. These factors are summarised in figure 30, below.

!

! ! SPIN (CYCLE w/ sustained fas SOURCE––PATH––GOAL !

!

[prefigures change of state/syst

Figure 3: Key embodied image schemas

Both are CYCLE schemas, but different embodied rotation=fast represents accumulated X These imageassociations/dynamics...SPIN schemas may to goal–oriented ton rotationbe path applied is energy, ROTATION indicates unclear/rate of more of in incremental energy tension/releaseinput/output (center–periphery, container, source–pat repetition of

cy values

rotation=slow X hing instinct’ This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’. eheuristic’. movement suchIonian as makingrepetitive a circling motionsuch withas making a circling motion with Major Framework! the experience. Continuous movement (internal) aviors 207 207 ghboring one's produces a pronounced degree of cyclical a egree of head cyclical individuation/arpeggiation of aindividuation/arpeggiation Figure 30: TheSpace’ relationship between source material, room,oflistener HRTF and the ‘Ionian rotation path is c 2number -> pc0) of the higher voices. In addition, a repetition Gann's experiment of tugging rial, room, listener HRTF and the SOURCE-----------PATH------------------GOAL tion, a repetition of Gann's experiment of tugging resulting percept in Young's soundofinstallations (still) clear

X

123

123

!

123

complete goal) or cyclical structures, cycle is along with hierarchical pit al–spatial mappings of initial system VERTICALITY CONTAINER CENTER–PERIPHERY (RECIPROCAL = COMBINATIONS OF THE ABOVE) relationships (verticality). important [2,3] suggest common practice ton g’ a tonal hierarchy and attraction associated embodied dynamics? structures may be based on these image schemas]. Our previo rdahl [10,11] using a boids algorithm e.g. ground/stability versus height/instability? rotation=fast rotation=slow ROTATION (CYCLE) ! ! ! ! SPIN (CYCLE w/ sustained fast rotation) CYCLE SOURCE––PATH––GOAL X[6] work has attempted to unify these ideas with cognitivel rotation path is (internal) he spatialization of voices [system is stable]!! ! ! ! [prefigures change of state/system] e.g. force–based (momentum/inertia) associations unclear/rate of rotation path is Figure(still) 3:clear Key embodied schemas repetition of basedimage theories of tonality [10,11], culminating in the tona complete cycle is are CYCLE schemas, but different embodied spatial mappings discussed above, which adopt cente ve provided the basis for aBothfurther important on an earlobe does bring about perceptible changes of a high frequency (but pitched)

ptible changes of a high frequency (but pitched)

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

4 kHz). In summary, there appear to be a number

of acoustical factors which influence the perception of the frequency structures of

the perception of the frequency structures of 123

Thisinoccurs because the application what Bregman (1990)30, terms the 'old-plus-new heuristic’. Young's installations various ways. of These factors areofsummarised in figure below.

n (1990) terms the 'old-plus-new heuristic’.

207

s. These factors are summarised207 in figure 30, below.

ip between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, listener HRTF and the

g's sound installations

resulting percept in Young's sound installations

such as making a circling motion with

30: 123 The between source material, room, icalFigure individuation/arpeggiation of relationship a source material, 30: TheFigure relationship between room, listener HRTF and the listener HRTF and the

tionresulting of Gann'spercept experiment of tugging in Young's sound in installations Young's sound installations application of whatresulting Bregmanpercept (1990) terms the 'old-plus-new heuristic’.123 This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

es of a high frequency (but pitched)

ummary, there appear to be a number

on of the frequency structures of

207

207

al mapping [4,7,14,16].

ion123 of a

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

of voices domains, facilitating accessible and immersive force of designs for NIME. gravity erials (e.g. Schacher et al. [13] use the term ecological relationships for a DESCENT (ground/stability) An Ecological Melodic ! hery. Many related idea, the blending of human interaction gestures and VERTICALITY Pitch Space structuring system responses. high tendency values gorithm resulting sound structure. our system combines mapped to More specifically,ASCENT X ennecting music and physical Brower [2,3] and Johnson [8] havethe previously applied image X speed ‘matching instinct’ motion–tracked movements with figurative steering behaviors 2 predictable schema theories to tonal music. They have also been relative to neighboring ’s tonal gestures [6] of musical macrostructures, such as pitch contours boids (e.g. pc 2 -> pc0) SOURCE-----------PATH------------------GOAL of musical and rhythmic investigated for (see their practical music software ee figure groupings figure 1). Thesignificance shared gestural in space PLANE tes between Some of the mapping, key basic close to is the design unifying[16,17]. ‘glue’ connecting input, and image output schemas are VERTICALITY [A] standard (linear) gestures [B] cycle gestures (rotation spin) CONTAINER CENTER–PERIPHERY (RECIPROCAL = COMBINATIONS OF and THE ABOVE) domains, facilitatingbelow accessible and 3). immersive designs for NIME. g voices emergent [VERTICALITY illustrated (figure + SOURCE–PATH–GOAL] [EMBODIED DYNAMICS OF DIFFERENT CYCLE RATES]

f tugging

number of the higher voices. In addition, a repetition of Gann's experiment of tugging

pitched)

on an earlobe does bring about perceptible changes of a high frequency (but pitched)

a number

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

res of

SPECTROMORPHOLOGY AND IMAGE SCHEMAS of acoustical factors which influence the perception of the frequency structures of

e 30, below.

Young's installations in various ways. These factors are summarised in figure 30, below.

e movement such as making a circling motion with

egree of cyclical individuation/arpeggiation123 of a

motion with tion, a repetition of Gann's experiment of tugging

123 changes of a high frequency (but pitched) ptible ion of a

kHz). In summary, there appear to be a number f4 tugging

the experience. Continuous repetitive movement such as making a circling motion with one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

number the higher In addition, a repetition of Gann's experiment of tugging ip between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, of listener HRTFvoices. and the

the perception of the frequency structures of pitched) g's sound installations

resulting percept in Young's sound installations

s. These factors are summarised in figure 30, below.

on an earlobe does bring about perceptible changes of a high frequency (but pitched)

a number

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

res of

of acoustical factors which influence the perception of the frequency structures of

e 30, below.

ls (e.g. y. Many ucturing usic and 8ve edictable tendency values ped to musical d ‘matching instinct’ ng behaviors between ve to neighboring mergent (e.g. pc 2 -> pc0)

Young's installations in various ways. These factors are summarised in figure 30, below.

Schacher et al. [13] use the term ecological relationships for a related idea, theforceblending of human interaction gestures and of VERTICALITY gravity DESCENT system responses. ASCENT (ground/stability) Brower [2,3] and Johnson [8] have previously applied image schema theories to tonal music. They have also been investigated for their practical significance Xin music softwareX design [16,17]. Some of thePLANE key basic image schemas are illustrated below (figure 3). SOURCE-----------PATH------------------GOAL

application of what Bregman (1990) terms the 'old-plus-new heuristic’.123 This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

the experience. Continuous repetitive movement207 such as making a circling motion with

207

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a number of the higher voices. In addition, a repetition of Gann's experiment of tugging

rotation=slow (internal) Both are CYCLE rotation path is s c(still) h e clear mas, but

on an earlobe does bring about perceptible changes of a high frequency (but pitched)

such as making a circling motion with

s repetitive movement such as making circling with the experience. Continuous component (perhaps in thearegion of 4motion kHz). In summary, there appear to be arepetitive number movement such as making a circling motion with

ical individuation/arpeggiation123 of a

123 123 nounced cyclical factors individuation/arpeggiation ofone's a ofhead produces structures a pronounced individuation/arpeggiation of a source material, room, listener HRTF and the and the degreeofofacoustical 30: The relationship between which influence the perception the frequency of degree of cyclicalFigure

tion of Gann's experiment of tugging

Young's installations in various ways.of These factors are of summarised figure 30, below. a repetitionresulting es. addition, a repetition of Gann's experiment tugging number the higherinvoices. In addition, of Gann'spercept experiment of tugging in Young's sound installations es ofIna high frequency (but pitched)

ummary, there appearchanges to be a number bout perceptible of a high

on of the frequency structures of

different embodied associations/ dynamics...

frequency (but pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched)

e region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number

ors are summarisedthe in experience. figure 30, below. Continuous repetitive movement such as making a circling motion with

h influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of 123

heuristic’.

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

207 These factors are summarised in figure 30, below. arious ways. Young's installations in various ways. These factors are summarised in figure 30, below.

207

number the higher In addition, a repetition of Gann's experiment of tugging source material, room, of listener HRTFvoices. and the

high ! about perceptible changes of a high frequency (but pitched) on an melodic earlobe does bring attraction (e.g. pc1 -> pc0) component (perhaps in the region of 4 kHz). In summary, there appear to be a number flock towards center number of the higher voices. In addition, a repetition of Gann's experiment offorce tuggingof the experience. Continuous repetitive movement such as making a circling motion with

nstallations

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

ROTATION (CYCLE) ! CYCLE VERTICALITY CONTAINER CENTER–PERIPHERY DESCENT [system is stable]!! (RECIPROCALFigure = COMBINATIONS OF THE ABOVE) 30: The relationship between source material, room, listener HRTF and the Young's installations in various ways. These factors are summarised in figure 30, below. of acoustical factors which influence the perception of the frequency structures of

on an earlobe does bring about perceptible changes of a high frequency (but gravity pitched)

and the

component in thetheregion of 4 kHz). In summary, there appear to be a number f what Bregman(perhaps (1990) terms 'old-plus-new heuristic’.

(ground/stability) 207

resulting percept in Young's sound installations

of acoustical factors which influence the perception of the frequency structures of

!

ro ro un re co cy im

! ! SPIN (CYCLE w/ sustained fas SOURCE––PATH––GOAL

! SPIN !represents ! [prefigures change of state/syst

Figure 3: Keyaembodied image schemas ccumulated

e schemas, n e r but g different y , embodied Both are CYCLE rotation=fast X Rrotation O Tbe A TisI represents O N toaccumulated These imageassociations/dynamics...SPIN schemas may goal–oriented ton path applied energy, ROTATION indicates more of in incremental energy unclear/rate of indicates more of tension/releaseinput/output (center–periphery, container, source–pat repetition of

Young's installations in various ways. These factors are summarised in figure 30, below.

cy values

rotation=slow X hing instinct’ This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’. eheuristic’. movement suchIonian as makingrepetitive a circling motionsuch withas making a circling motion with Major Framework! the experience. Continuous movement (internal) aviors 207 207 ghboring one's produces a pronounced degree of cyclical a egree of head cyclical individuation/arpeggiation of aindividuation/arpeggiation Figure 30: TheSpace’ relationship between source material, room,oflistener HRTF and the ‘Ionian rotation path is c 2number -> pc0) of the higher voices. In addition, a repetition Gann's experiment of tugging rial, room, listener HRTF and the SOURCE-----------PATH------------------GOAL tion, a repetition of Gann's experiment of tugging resulting percept in Young's soundofinstallations (still) clear

X

123

123

X

123

an incremental

complete goal) or cyclical structures, along with hierarchical pit cycle is i/o al–spatial mappings of initial system energy VERTICALITY CONTAINER CENTER–PERIPHERY (RECIPROCAL = COMBINATIONS OF THE ABOVE) relationships (verticality). important [2,3] suggest common practice ton g’ a tonal hierarchy and attraction associated embodied dynamics? structures may be based on these image schemas]. Our previo rdahl [10,11] using a boids algorithm e.g. ground/stability versus height/instability? rotation=fast rotation=slow ROTATION (CYCLE) ! ! ! ! SPIN (CYCLE w/ sustained fast rotation) CYCLE SOURCE––PATH––GOAL X[6] work has attempted to unify these ideas with cognitivel rotation path is (internal) he spatialization of voices [system is stable]!! ! ! ! [prefigures change of state/system] e.g. force–based (momentum/inertia) associations unclear/rate of rotation path is Figure(still) 3:clear Key embodied schemas repetition of basedimage theories of tonality [10,11], culminating in the tona complete cycle is are CYCLE schemas, but different embodied spatial mappings discussed above, which adopt cente ve provided the basis for aBothfurther important on an earlobe does bring about perceptible changes of a high frequency (but pitched)

ptible changes of a high frequency (but pitched)

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

4 kHz). In summary, there appear to be a number

of acoustical factors which influence the perception of the frequency structures of

the perception of the frequency structures of 123

Thisinoccurs because the application what Bregman (1990)30, terms the 'old-plus-new heuristic’. Young's installations various ways. of These factors areofsummarised in figure below.

n (1990) terms the 'old-plus-new heuristic’.

207

s. These factors are summarised207 in figure 30, below.

ip between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, listener HRTF and the

g's sound installations

resulting percept in Young's sound installations

such as making a circling motion with

30: 123 The between source material, room, icalFigure individuation/arpeggiation of relationship a source material, 30: TheFigure relationship between room, listener HRTF and the listener HRTF and the

tionresulting of Gann'spercept experiment of tugging in Young's sound in installations Young's sound installations application of whatresulting Bregmanpercept (1990) terms the 'old-plus-new heuristic’.123 This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

es of a high frequency (but pitched)

ummary, there appear to be a number

on of the frequency structures of

207

207

,7,14,16]. redictable musical s between emergent

schema theories to tonal music. They have also been investigated for their practical significance in music software designOTHER [16,17]. Some of theSCHEMA/ key basic image schemas are IMAGE illustrated below (figure 3).

123 one'sThe head produces a pronounced degreeour of cyclical of aalgorithm innovation within ownindividuation/arpeggiation usage of the boids

resulting sound structure. More specifically, our system combines physical motion–tracked movements with the figurative melodic attraction and inertia values from Lerdahl’s tonal gestures2 [6] of musical macrostructures, such as pitch contours gravity on an earlobe does bring about perceptible changes of a high frequency (but pitched) In summary, there appear to be a number model [10,11] to in–kind spatial flocking parameters (see figure and rhythmic groupings (see figure 1). The shared gestural space component (perhaps in the region of 4 kHz). In summary, there appear to be a number 2, below). In this mapping, melodic intervals that are close to is the unifying ‘glue’ connecting input, mapping, and output dic ! of acoustical factors which influence the perception of the frequency structures of tonal center (e.g. triadic) will cause movement of voices domains, facilitating accessible and immersive designs for NIME. ception of the frequency structures of Young's installations ways. More These factors are summarised in figurematerials 30, below. towards the in various center. tonally distant (e.g. Schacher et al. [13] use the term ecological relationships for a highintendency values chromatic) related idea, the blending of human interaction gestures and factors are summarised figure 30, below. will cause movement towards the periphery. Many mapped to musicians will be familiar with these general structuring system responses. X speed ‘matching instinct’ of tonal relationships in common practice music and principles Brower [2,3] and Johnson [8] have previously applied image the experience. Continuous repetitive movement such as making a circling motion with steering behaviors will recognize the flocking behaviors as broadly predictable schema theories to tonal music. They have also been The innovation within our own usage of the boids algorithm resulting sound structure. More specifically, our system combines 123 relative to neighboring one's produces a pronounced degree of own cyclical individuation/arpeggiation of a musical andhead related to their understanding of for their practical with significance in music software is its metaphorical mapping of musical forces, connecting physical investigated motion–tracked movements the figurative boids (e.g. pc 2number -> pc0) macrostructures. Thus, thea repetition usefrom ofofembodied correlates design [16,17]. Some of thesuch key as basic schemas are the higher In addition, Gann's experiment oftonal tugging between F and the Figure 30: The relationship betweenmelodic source material, room, of listener HRTFvoices. and the attraction and inertia values Lerdahl’s gestures2 [6] of musical macrostructures, pitchimage contours input and output provides a parameters means of (see managing emergent illustrated below (figure model [10,11] to in–kind flocking and rhythmic groupings (see figure3). 1). The shared gestural space on an earlobe does bringspatial about perceptible changes of a high frequency (butfigure pitched) resulting percept in Young's sound installations complexity via conceptual mapping [4,7,14,16]. 2, below). In this mapping, melodic intervals that are close to is the unifying ‘glue’ connecting input, mapping, and output

hanges of a high frequency (but pitched) is itsof the metaphorical mapping ofof Gann's musical forces, number higher voices. In addition, a repetition experiment offorce tuggingconnecting of

the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as making a circling motion with one's head produces a pronounced degree of cyclical individuation/arpeggiation123 ofone's a head produces a pronounced degree of cyclical individuation/arpeggiation123 of a number of the higher voices. In addition, a repetition of Gann's experiment of tugging number of the higher voices. In addition, a repetition of Gann's experiment of tugging on an earlobe does bring about perceptible changes of a high frequency (but pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched)

component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of

SPECTROMORPHOLOGY PAIRINGS Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below.

the experience. Continuous repetitive movement such as making a circling motion with

the experience. Continuous repetitive movement such as making a circling motion with

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

component (perhapsmovement in the region of 4 kHz). In summary, there appear be a number motion with the experience. Continuous repetitive such as making a tocircling tonal center (e.g. triadic) will cause movement of voices domains, facilitating accessible and immersive designs for NIME. number of the higher voices. In addition, a repetition of Gann's experiment offorce tuggingof

number of the higher voices. In addition, a repetition of Gann's experiment of tugging

VERTICALITY distant materials (e.g.

on an earlobe does bring about perceptible changes of a high frequency (but gravity pitched)

towards the center. More tonally Schacher et CENTER–PERIPHERY al. [13] use the term ecological relationships for a An Ecological Melodic ! 123 Pitch Space Young's in various ways. towards These factors are summarised in figure 30, below. one's head produces a pronounced degree of cyclical individuation/arpeggiation of a idea, the blending of human interaction gestures and chromatic) will installations cause movement the periphery. Many related w heuristic’. This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’. high tendency values low melodic ! mapped to 207 207 X attraction musicians willtension) be(e.g.(expectancyfamiliar with these general structuring system responses. speed ‘matching instinct’ pc1 -> pc6) steering behaviors flock towards periphery principles of high tonal relationships in common practice music relative andtoof Brower [2,3] and Johnson [8] have previously applied image number of the higher voices. In addition, a repetition of Gann's experiment tugging neighboring implicative denial values boids (e.g. pc 2 -> pc0) will recognize the flocking behaviors as broadly predictable schema theories to tonal music. They have2 also been Spectromorphology 9 investigated for their practical significance in music software to Visual their Sound-Shapes own understanding of musical on an earlobe doesand bringrelated aboutThe perceptible changes of of a high frequency (but pitched) VERTICALITY CONTAINER CENTER–PERIPHERY macrostructures. Thus, use design [16,17]. Some of the key basic image schemas are g motion with the experience. Continuous repetitive movement such as making a circlingthe motion withof embodied correlates between input and output provides123 a means of managing emergent illustrated below (figure 3). iation123 ofone's a component head produces a pronounced degree of cyclical individuation/arpeggiation of a 30: The between source material, room, listener HRTF andto the be a number (perhaps in the Figure region ofrelationship 4 kHz). In summary, there appear complexity via conceptual mapping [4,7,14,16]. X on an earlobe does bring about perceptible changes of a high frequency (but pitched)

of acoustical factors which influence the perception of the frequency structures of component (perhaps in the region of 4 kHz). In summary, there appear to be a number

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

CONTAINER

The innovation within our own usage of the boids algorithm resulting sound structure. More specifically, our system combines is its metaphorical mapping of musical forces, physical motion–tracked movements with the figurative force connecting of melodic attraction and inertia values from gravity Lerdahl’s tonal gestures [6] of musical macrostructures, such as pitch contours X model [10,11] to in–kind spatial flocking parameters (see figure and rhythmic groupings (see figure 1). The shared gestural space 2, below). In this mapping, melodic intervals that are close to is the unifying ‘glue’ connecting input, mapping, and output cussed. Macro-composites of acoustical factors which influence the perception of the frequency structures of domains, facilitating accessible and immersive designs for NIME. activitytonal in wholecenter phrases (e.g. triadic) will cause movement of voices mposing a structure of this Young's installations in various ways. More These factors are summarised in figurematerials 30, below. towards the center. tonally distant (e.g. Schacher et al. [13] use the term ecological relationships for a ty to engage with, as there to co-ordinate over longerwill cause movement towards the periphery. Many ency values chromatic) related idea, the blending of human interaction gestures and Figure 3: Key embodied image schemas danger of being overo>the musicians will be familiar with these general structuring system responses. with X pc0) extended time frames These image schemas may be applied to goal–oriented tonal CYCLEmusic SOURCE––PATH––GOAL ntching develop other inherent instinct’ principles of tonalSPIRAL relationships in common practice and Brower [2,3] and Johnson [8] have previously applied image tension/release (center–periphery, container, source–path– ! = aCYCLE + CENTRE/PERIPHERY (+ SOURCE-PATH GOAL) er time the that might notContinuous be experience. repetitive movement such as making motion goal) or cyclical structures, along Xwith hierarchical pitch Figure 2: Dynamic tonal–spatial mappings ofbroadly initial systemwith ehaviors the flocking behaviors ascircling predictable schema theories to tonal music. Theycombines have also been Figure 3: Key embodied image schemas out. Inwill some recognize instances, relationships (verticality). [2,3] suggest common practice tonal iteration, ‘animating’ and attraction innovation within our own usage ofa tonal thehierarchy boids algorithm resulting sound structure. More specifically, our system 123 structures may be based on these image schemas]. Our previous cular large-scale shape or neighboring one's head produces a pronounced degree of cyclical individuation/arpeggiation of a model derived from Lerdahl [10,11] using a boids algorithm and related to their own forces, understanding of musical investigated for their practical with significance in music software mapping of tomusical connecting physical movements the figurative work [6] has attempted tomotion–tracked these ideas with cognitively– bemetaphorical contrary to the implied control the spatialization of voices (AGAIN, CHANGE OF STATE/FUNCTION IS IMPLIED BY THIS, unify NEW CENTRE/GOAL .g pc 2number ->given pc0) based theories of tonality [10,11], culminating in the tonal– 2 design macrostructures. Thus, the use of embodied correlates between [16,17]. Some of the key basic image schemas are from a sound’s of the higher voices. In addition, a repetition of Gann's experiment of tugging material, room, listenerand HRTF and the [+ SOURCE–PATH–GOAL?] dic attraction inertia values havefrom Lerdahl’s [6] applied of musical macrostructures, such as pitch contours PROVIDED) spatial gestures mappings above, which adopt center– These maydiscussed be to goal–oriented tonal These approaches provided the basisimage for a tonal furtherschemas ork! ‘bottom up’ approach to periphery, cycle, and verticality schemas. input and output provides a means of managing emergent illustrated below (figure 3). investigation of musical parameters the (see perspective of a Figureand 3: Keyrhythmic embodied imagegroupings schemas l [10,11] to in–kind flocking (see figure 1). The shared gestural space entially compromised overbringspatial on an earlobe does about perceptible changesmappings of a highfrom frequency (butfigure pitched) ons wider range of embodied tension/release metaphors. Our initial approach (center–periphery, container, source–path– showingcomplexity why vocabulary-via conceptual mapping [4,7,14,16]. investigated relatively obvious parallels between various These image schemas may be applied to goal–oriented tonal ow). In this mapping, melodic intervals that are close to 2.2.2 isUnifying the mappings unifyingfrom‘glue’ input, mapping, and output musical connecting movement of acoustical factors which influence the perception of the frequency structures of

Young's installations in various ways. These factors are summarised in figure 30, below.

123

of acoustical factors which influence the perception of the frequency structures of

Young's installations in various ways. These factors are summarised in figure 30, below.

the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as making a circling motion with one's head produces a pronounced degree of cyclical individuation/arpeggiation123 ofone's a head produces a pronounced degree of cyclical individuation/arpeggiation123 of a one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a number of the higher voices. In addition, a repetition of Gann's experiment of tugging number of the higher voices. In addition, a repetition of Gann's experiment of tugging number of the higher voices. In addition, a repetition of Gann's experiment of tugging number the higher In addition, a repetition of Gann's experiment of tugging Figure 30: The relationship between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, of listener HRTFvoices. and the on an earlobe does bring about perceptible changes of a high frequency (but pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched) on an earlobe does bring about perceptible changesresulting of a highpercept frequency (but pitched) on an earlobe does bring about perceptible changes of a high frequency (but pitched) in Young's sound installations resulting percept in Young's sound installations component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below. 123 123 This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207

207

the experience. Continuous repetitive movement such as making a circling motion with

the experience. Continuous repetitive movement such as making a circling motion with

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as making a circling motion with number of the higher voices. In addition, a repetition of Gann's experiment of tugging number of the higher voices. In addition, a repetition of Gann's experiment offorce tuggingof 123 123 one'sroom, headlistener produces a pronounced a head produces a pronounced degree of cyclicalFigure individuation/arpeggiation of a source material, room, listener HRTF and the Figure 30: The relationship between source material, HRTF and the degree of cyclical individuation/arpeggiation ofone's 30: The relationship between on an earlobe does bring about perceptible changes of a high frequency (but pitched) on an earlobe does bring about perceptible changes of a high frequency (but gravity pitched) number of the higher voices. In addition, a repetitionresulting of Gann'spercept experiment of tugging resulting percept in Young's sound installations number of the higher voices. In addition, a repetition of Gann's experiment of tugging in Young's sound installations component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number on an earlobe does bring about perceptible changes of a high frequency (but pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched) of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below. of acoustical factors which influence the perception of the frequency structures of tendency of acoustical factors which influence the perception of the frequency structures of 123 123 high values

t of tugging number of the higher voices. In addition, a repetitionresulting of Gann'spercept experiment of tugging in Young's sound installations

8ve

=

ut pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched)

An Ecological Melodic ! Pitch Space

be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’. low melodic ! 207

+

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207

mapped toinstallations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below. Young's

attraction (expectancytures of of acoustical factors which influence the perception of the frequency structures of tension) (e.g.123 pc1 -> pc6)

speed ‘matching instinct’

high melodic the experience. Continuous repetitive movement such as making a circling motion with This occurs because of the application of what Bregman (1990)! terms the 'old-plus-new heuristic’. steering behaviors

the experience. Continuous repetitive movement such as making a circling motion with

ure 30, below. Young's installations in various ways.

flock towards periphery high implicative values These factorsdenial are summarised

attraction (e.g. pc1 -> relative pc0) to neighboring one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a 207 flock towards center boids (e.g. pc 2 -> pc0)

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

in figure 30, below.

CYCLE

VERTICALITY

of acoustical factors which influence the perception of the frequency structures of

of acoustical factors which influence the perception of the frequency structures of

+

X

SOURCE––PATH––GOAL

number of the higher voices. In addition, a repetition of Gann's experiment of tugging number the higher In addition, a repetition of Gann's experiment of tugging Figure 30: The relationship between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, of listener HRTFvoices. and the Figure 30: The relationship between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, listener HRTF and the on an earlobe does bring about perceptible changesresulting of a highpercept frequency (but pitched) on an earlobe does bring about perceptible changes of a high frequency (but pitched) in Young's sound installations resulting percept in Young's sound installations resulting percept in Young's sound installations resulting percept in Young's sound installations component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number

CENTER–PERIPHERY

the experience. Continuous repetitive movement such as making a circling motion with the experience. repetitive movement Young's installations Continuous in various ways. These factors are summarised in figure 30,such below. as making a circling motion with

CONTAINER

Young's installations in various ways. These factors are summarised in figure 30, below. 123

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.123 This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207 207 This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’. This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’. Major Ionian Framework! 123 207 one's head produces a pronounced degree of cyclical207individuation/arpeggiation123 ofone's a head produces a pronounced degree of cyclical individuation/arpeggiation of a ‘Ionian Space’ 123

123

number of the higher voices. In addition, a repetition of Gann's experiment of tugging number of the higher voices. In addition, a repetition of Gann's experiment of tugging

Figure 30:theThe relationship between source material, room, listener HRTF and the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as making a circling motion with 8ve on an earlobe does bring about perceptible changes of a high (but pitched) on an earlobe does bring about perceptible changes ofone's a frequency head produces a pronounced degree of cyclical individuation/arpeggiation of a source Figure 30: The relationship between material, room, listener HRTF and the of a high frequency (but pitched) resulting percept in Young's sound installations number of the higher voices. In addition, a repetition of Gann's experiment of tugging number of the higher voices. In addition, a repetitionresulting of Gann'spercept experiment of tugging in Young's sound installations

one'sroom, headlistener produces a pronounced Figure 30: The relationship between source material, HRTF and the degree of cyclical individuation/arpeggiation

123

123

resulting percept in Young's sound installations

component on(perhaps in the region of 30: 4 The kHz). In summary, thereroom, appear to be component (perhaps in the region of 4andkHz). In summary, there appear to be a number Figure relationship between source material, listener HRTF and a thenumber Figure 30: The relationship between source material, room, listener HRTF the an earlobe does bring about perceptible changes of a high frequency (but pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched)

SPIRAL

resulting percept in Young's sound installations resulting percept in Young's sound installations component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number

of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of

123

of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of 123

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207

207

Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below. 123

123

This occurs because of the

123

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’. This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’. high melodic ! 207 207 attraction (e.g. pc1 -> pc0) flock towards center application of what Bregman (1990) terms the 'old-plus-new heuristic’.

CYCLE

SOURCE––PATH––GOAL

207

Figure 30: The relationship between source material, room, listener HRTF and the

Figure 30: The relationship between source material, room, listener HRTF and the

resulting percept in Young's sound installations

resulting percept in Young's sound installations

the experience. Continuous repetitive movement such as making a circling motion with

the experience. Continuous repetitive movement such as making a circling motion with

4 kHz). Inand summary, there appear be aasnumber more component successfully(perhaps imple- in the region control ofparameters embodied analogues, the tension/release (center–periphery, container, source–path– goal) oroftosuch cyclical structures, along with hierarchical pitch appings of initial system force models along Figurecontours 11. Dissipation. es, where a greater degree melodic to spatial trajectories. We have goal) or and cyclical structures, with hierarchical pitch center (e.g. triadic) will oftonal–spatial cause movement domains, facilitating accessible and immersive designs for NIME. Figure 2: mapping Dynamic mappings of initial system voices Smalley’s influential theories of electroacoustic music [15] of acoustical macrofactors iteration, which influence thetoperception of the and frequency ofrelationships (verticality). [2,3] [2,3] suggestsuggest common practice tonal since sought extend our attraction modelsstructures to consider the ed. Nevertheless, VERTICALITY ‘animating’ arefine tonal and hierarchy relationships (verticality). common rarchy and attraction couldbeSchacher be seenonas anticipating such an ds center. tonally distant materials et CENTER–PERIPHERY al. [13] use the termpractice ecologicaltonal relationships for a CONTAINER An Ecological Melodic !algorithm structures may based these image schemas]. Ourembodied–structural previous structural theories of embodied cognition, namely the (e.g. image derived from Lerdahl [10,11] using a boids uld notthe be ruled out, as model in More usage throughto their implicitly embodied discourse. These Pitch Space work [6] has attempted unify these ideas with cognitively– schema theories of Lakoff and Johnson [8,9]. We will now to control the spatialization ofsummarised voices installations in various ways. towards These factors the are in figure 30, below. anusing lead Young's towill some interesting structures may be based on these image schemas]. Our interaction previous gestures and matic) cause movement periphery. idea, the of human theories see archetypal envelope formsblending (termed energy–motion a boids algorithm based theories ofrelated tonality [10,11], culminating in the tonal– present these embodied image schema theories andMany related egman (1990) terms the 'old-plus-new heuristic’. profiles) as the key above, generative dynamics of electroacoustic spatial mappings discussed which adopt center– discourses from the perspective of informing the design of These approaches have provided the basis for a further 207 X cians be familiar with thesefrom general structuring system responses. work has attempted totheorizes unify thesedynamics ideas with cognitively– Smalley these generative in terms of digital musical mappings instruments, interfaces, and[6] mappings periphery, music. cycle, and verticality schemas. ion ofwill voices investigation of musical the perspective of a [5,6,16]. embodied associations such as force and coherence of path or wider range of embodied metaphors. Our initial approach ples of tonal relationships in common practice music and Brower [2,3] and Johnson [8] have previously applied image frames causality. [10,11], For example, attack phases are considered in the tonal– based theories tonality culminating Imageobvious Schemas, Musical andof apparent investigated2.2relatively parallels betweenRelations, various 2.2.2 Unifying mappingswith from musical movement to haveschema associations departure, launching, or emergence; recognize the flocking behaviors as analogues, broadly theories to spatial tonal music. They have also been control parameters and Mapping embodied such predictable as the ual macro-composite disEmbodied Strategies phases provide impressions of which passage;adopt and forcecontinuation models discussed above, center– mapping offurther melodic contours to spatialspatial trajectories. Wemappings have the basis for a consider this morphology 2.2.1 Unifying mappings from embodied image or release are associated with The Visual Sound-Shapes of Spectromorphology 9 investigated Smalley’s terminations influential theories of phases electroacoustic musicpractical [15] arrival or significance in music software related to their own understanding of musical for their since sought to refine and extend our models to consider the time frames, ranging from closure (see table 1). These ideas may further inform the design VERTICALITY CONTAINER CENTER–PERIPHERY could and be seen verticality as anticipating such an embodied–structural schemas: performance gesture ecologies VORTEX CYCLE+CENTRE/PERIPHERY+VERTICALITY (+SOURCE-PATH-GOAL) periphery, schemas. structural theories embodied cognition, namely the imagecycle, X om the of a =and[6]Johnson ostructures. Thus, the use ofofLakoff embodied correlates between design [16,17]. SomeTheseof the key basic image schemas are phrase evenperspective an of mapping strategies forembodied hyper instruments. usage through their implicitly discourse. Wewith have discussed howWe a variety of approaches ment such or as making a entire circling motion schema theories of previously [8,9]. will now theories see archetypal envelope forms (termed energy–motion duration. to musical structure, gesture tracking, mappings present these image of schema theories and and control related andOur output provides aembodied means managing emergent belowof (figure 3). rs. initial approach profiles) as the illustrated key generative dynamics electroacoustic may be unified within an informing embodied model. We propose that racted the30: motion and discourses from perspective of the design of and Figure Motion and growth processes (Smalley 1997: 116).the cyclicalfrom individuation/arpeggiation of12. athe Figure The relationship between source material, room, listener HRTF such a usage of embodied models can be of significant helpmusic. for Smalley theorizes these generative dynamics in terms of digital musical instruments,[4,7,14,16]. interfaces, and mappings [5,6,16]. figure 12) to conceptual inform the lexity via mapping rallels between various embodied associations such as force and coherence of path or X designers and creative practitioners in unifying such diverse ge it should be mentioned 2.2.2 Unifying mappings from musical movement epetition of Gann's experiment of tugging resulting percept in Young's sound installations causality. For example, attackpertain phasestoare considered 8ve domains of performance gestures and control mappings. apparent We Figurative gestures may a fictive movement or 2.2 Image Schemas, Musical Relations, and does not solely define a to have associations departure, or emergence; analogues, such as referMapping to the this type of integrated model as a performance gesture motion with perceived by launching, the listener, including musical Embodied Strategies On the of contrary, this comcontinuationmetaphors phases provide impressions of figurations spatial passage; hanges a high frequency (but pitched) ecology: a shared gestural space that conveys spatial–relational and abstracted musical such as melodic force models An Ecological Melodic ! fromand 2.2.1 Unifying mappings embodied image of sounds) will inherently terminations or release phases are associated with arrival or and chord structures. dynamics of performance gestures, resulting mapping strategies, and ial qualities: trajectories. WePitch have Space closure (see table 1). These ideas may further inform the design onic for example, schemas: performance gesture ecologies In summary, there appear to be a number Smalley’s influential theories of electroacoustic music [15] of mapping strategies for hyper instruments. We have previouslythe [6] discussed how a variety of approaches cy within spectraltospace, r models consider to musical structure, gesture tracking, and control mappings ontinuant termination), ception of and the frequency structures of could be thatseen as anticipating such an embodied–structuralX may be application unified within an Bregman embodied(1990) model. Wethepropose This occurs of the of what terms 'old-plus-new heuristic’. namely the image sition, is also true for thebecause image such a usage of embodied models can be of significant help for 207 CYCLE discourse. SOURCE––PATH––GOAL eviously introduced shapes factors are summarised indesigners figure 30, their implicitly embodied These andbelow. creative practitioners inusage unifying such through diverse [8,9]. We will now rson purposes of discussing domains of performance gestures and control mappings. We Figurative gestures may pertain to a fictive movement or Figure 3: Key embodied image schemas pation (figure 11) indicates see archetypal energy–motion refer to this type of integrated model astheories a performance gesture motion perceivedenvelope by the listener,forms including (termed musical ma theories and a related ecology: shared gestural space that conveys spatial–relational metaphors and abstracted musical figurations such as melodic y. Increasing the shape’s VERTICALITY CONTAINER CENTER–PERIPHERY dynamics of performance gestures, resulting mapping strategies, and profiles) as theand chord keystructures. generative dynamics of electroacoustic provides a further ‘handle’ informing the design of These image schemas may be applied to goal–oriented tonal Major Ionian Framework! ly realising a sound-shape. CYCLE SOURCE––PATH––GOAL music. Smalley theorizes these tension/release generative dynamics in terms of ‘Ionian Space’ rates a dissipation informed [5,6,16]. (center–periphery, container, source–path– and mappings 123 one's head produces a pronounced degree of cyclical ofIonian a This occurs because of the application individuation/arpeggiation of what Bregman (1990) terms the 'old-plus-new heuristic’. Major Framework! 207 ‘Ionian Space’ number of the higher voices. In addition, a repetition of Gann's experiment of tugging 123

123

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a

207

number of the higher voices. In addition, a repetition of Gann's experiment offorce tuggingof on an earlobe does bring about perceptible changes of a high frequency (but gravity pitched)

on an earlobe does bring about perceptible changes of a high frequency (but pitched)

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

component (perhaps in the region of 4 kHz). In summary, there appear to be a number

Figure 30: The relationship between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, listener HRTF and the

resulting percept in Young's sound of acoustical factors which influence the perception of the frequency structures ofinstallations

of acoustical factors which influence the perception of the frequency structures of

resulting percept in Young's sound installations

F and the Figure 30: The relationship between source material, room, listener HRTF and the

=

Young's installations in various ways. These factors are summarised in figure 30, below.

resulting percept in Young's sound installations

+

+

Young's installations in various ways. These factors are summarised in figure 30, below.

high tendency values low melodic ! mapped to attraction (expectancy‘matching instinct’ the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as makingspeed a circling motion with tension) (e.g. pc1 -> pc6) the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as making a circling motion with steering behaviors 123 one's head producesperiphery a pronounced degree of cyclical individuation/arpeggiation ofone's a head produces a pronounced degree of cyclical individuation/arpeggiation123 of a flock towards relative to neighboring one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a 123 high implicative denial values of the higher voices. In addition, a repetition of Gann's experiment number of the higher voices. In addition, a repetition of Gann's experiment of tugging w heuristic’. This occurs because of number the application of what Bregman (1990) terms the 'old-plus-new heuristic’. of tugging boids (e.g. pc 2number -> pc0) number of the higher voices. In addition, a repetition of Gann's experiment of tugging of the higher voices. In addition, a repetition of Gann's experiment of tugging 123

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.123 This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207

207

207

Figure 30: The relationship between source 207 material, room, listener HRTF and the Figure 30: The relationship between source material, room, listener HRTF and the on an earlobe does bring about perceptible changes of a high frequency (but pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched) on an earlobe does bring about perceptible changesresulting of a highpercept frequency (but pitched) on an earlobe does bring about perceptible changes of a high frequency (but pitched) in Young's sound installations resulting percept in Young's sound installations component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below. 123 123

VORTEX

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207

207

123

ce. Continuous repetitive movement such as making a circling motion with

the experience. Continuous repetitive movement such as making a circling motion with

roduces a pronounced degree of cyclical individuation/arpeggiation123 of a

one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a the experience. Continuous repetitive movement such as making a circling motion with the experience. Continuous repetitive movement such as making a circling motion with 2 he higher voices. In addition, a repetition of Gann's experiment of tugging number of the higher voices. In addition, a repetition of Gann's experiment offorce tuggingof 123 123 one'sroom, headlistener produces a pronounced a head produces a pronounced degree of cyclicalFigure individuation/arpeggiation of a source material, room, listener HRTF and the Figure 30: The relationship between source material, HRTF and the degree of cyclical individuation/arpeggiation ofone's 30: The relationship between e does bring about perceptible changes of a high frequency (but pitched) on an earlobe does bring about perceptible changes of a high frequency (but gravity pitched) number of the higher voices. In addition, a repetitionresulting of Gann'spercept experiment of tugging resulting percept in Young's sound installations number of the higher voices. In addition, a repetition of Gann's experiment of tugging in Young's sound installations (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number on an earlobe does bring about perceptible changes of a high frequency (but pitched)on an earlobe does bring about perceptible changes of a high frequency (but pitched) l factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of component (perhaps in the region of 4 kHz). In summary, there appear to be a number component (perhaps in the region of 4 kHz). In summary, there appear to be a number allations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. These factors are summarised in figure 30, below. of acoustical factors which influence the perception of the frequency structures of of acoustical factors which influence the perception of the frequency structures of 123 123 high tendency values

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’. This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’. low melodic ! 207 These factors are summarised in figure 207 mapped toinstallations in various ways. These factors are summarised in figure 30, below. Young's installations in various ways. 30, below. Young's traction (expectancyspeed ‘matching instinct’ 123 sion) (e.g.repetitive pc1 -> pc6) such as making a circling motion with high melodic ! ce. Continuous movement the experience. Continuous repetitive movement such as making a circling motion with steering behaviors ock towards periphery attraction (e.g. pc1 -> relative pc0) to neighboring roduces a pronounced degree of cyclical individuation/arpeggiation123 of a one's head produces a pronounced degree of cyclical individuation/arpeggiation123 of a plicative denial values flock towards center boids (e.g. pc 2 -> pc0)

he higher voices. In addition, a repetition of Gann's experiment of tugging number the higher In addition, a repetition of Gann's experiment of tugging Figure 30: The relationship between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, of listener HRTFvoices. and the 2 Figure 30: The relationship between source material, room, listener HRTF and the Figure 30: The relationship between source material, room, listener HRTF and the e does bring about perceptible changesresulting of a highpercept frequency (but pitched) on an earlobe does bring about perceptible changes of a high frequency (but pitched) in Young's sound installations resulting percept in Young's sound installations

resulting percept in Young's sound installations (perhaps in the region of 4 kHz). In summary, there appear to be a number

resulting percept in Young's sound installations component (perhaps in the region of 4 kHz). In summary, there appear to be a number

l factors which influence the perception of the frequency structures of

of acoustical factors which influence the perception of the frequency structures of

allations in various ways. These factors are summarised in figure 30, below. 123

Young's installations in various ways. These factors are summarised in figure 30, below.

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.123 This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

123

This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’.

207

207

207123 This occurs because of the application of what Bregman (1990) terms the 'old-plus-new heuristic’. 207

+

IMAGE SCHEMAS & MUSICAL CASES •

Verticality and path schemas (are fairly obvious/ubiquitous); centre– periphery or balance (in spectromorphological terms) is implied by any number of spectral and temporal gestures which establish dynamic range...

•

Iterative gesture to cycle/loop to spin/spiral (or in reverse): a musical affordance at least as old as Stockhausen’s Kontakte...

•

Container schemas: e.g. Klang (Harrison, 1982); a range of transformed sounds emerge from manipulation of source’s sonic ‘container–affordance’

•

Furthermore, our tendency to spatialise these spectromorphologies (schemas) makes explicit (concrete!) a conceptual metaphor which Johnson (2007) has proposed...the moving music/music–as–moving–force metaphor.

•

Considering spectromorphologies as combinations of image schemas elucidates dynamic connections (via embodied–cognitive ‘forces’)...may shed light on the macro–level gestural syntaxes of musical structures

3. SOUNDED AFFORDANCES: BEYOND CANONICAL IMAGE SCHEMAS TO SOUNDED GESTURE–TEXTURES •

Key Q: Do we need sound–specific schemas (rather than relying solely on existing image schemas)?

•

How might they differ from canonical image schemas?

•

Can we think of examples? flocking/streaming, rupture/breaking, stretching, bouncing, various granular/iterative gestural schemas (crackling, multiple cracks, etc.)...slow modulation ‘breathing’ schemas... If so, what is their musical/structural function?

•

These are common audible structures and affordances of our audio processes...though many relate quite closely to existing image schemas, their temporal dynamism and plasticity of form/state makes them distinctive

SOUND(ING) SCHEMAS •

Flocking/streaming...grouping and segregation (growth/integration/ dissolution)

•

Rupture/breaking/glitch (break/sudden change of state, foregrounded event/act)

•

Stretching (extension/sustaining tension, investigating limits of system/ gesture)

•

Bouncing (equilibrium/balance schemas; bounce–back, echoes, decay, inertial effects, coming to rest, new stable state...cue for new event entries)

•

Slow oscillation/breathing (cycle, balance; pace, affective qualities via emotional arousal model, etc.)

•

Dilation/diffusion–to–point source (expansion/contraction/coverage, changing density)

•

Note that these are plastic, state–change schemas: the ‘objects’ in question change form rather than simply moving

SOUNDED AFFORDANCES AND MATERIAL METAPHORS •

Many existing image schema theories rely heavily on path/movement metaphors (c.f. the image schema Master Metaphor List (Lakoff et al., 1994)) for changes of states, but certain contemporary musics may be better served by focus on change of state metaphors themselves (Adlington, 2003)

•

The lack of plasticity/animation and consideration of domain–specific affordances in canonical image schema cases may sometimes limit their utility in other domains (e.g. music)

•

From path to material metaphors and the details of gestures, not just their overall form...

•

So, modified sounded schemas may imply new relational dynamics (whilst still maintaining compatibility with existing image schema theories)

4. FORMS, DIMENSIONS AND COMBINATIONS: SCHEMAS, GESTURES, COMPOSITES AND FORM (a) Qualities of movement as well as forms (and their implications) (b) Spectromorphology, gesture and timbre–space (c) Combinations of movement and gestural ecologies (d) Sounded schemas, conceptual metaphors and larger–scale structures

accessible connections between figurative gestures and spatial metaphors via tonal hierarchy models. Connections between musical structures and embodied spatial domains may be urther explored and strengthened through the extraction and mapping of additional gesture data, specifically indirect ancillary/accompaniment gestures [3]. These are bodily movements that a performer may consciously or unconsciously execute alongside sounded performance gestures. They may be conceptualized as embodied accompaniments to musical structures. Although such usage entails the presence of additional input controls, we consider the application of such gestures to mappings to be broadly compatible with our initial aim of maintaining clear connections with familiar performance gestures. In •addition, such (2007) gestures are already broadly Johnson extended image accessible as by–products of established performance practice. schema theory a focusby on They are therefore less likely to bewith experienced the performer as contributing to fragmented control based and impeding aesthetic implications on heir musical execution. details of gesture Structures based on accompaniment gestures are obtained using a combination of the Xbox Kinect and the third-party application, Synapse [16]. This provides skeletal point–based data, including•values for velocity and acceleration. The result Parallel between the embodied of this extension is that we are now able to access a of his qualitative combination of associations gestural types––(1) physically small–scale sounding gesturesdimensions (either as individual note articulations of movement and and composite figurative gestures) alongside (2) more expansive some of Smalley’s archetypalmore bodily accompaniment/ancillary gestures––facilitating holistic interpretations and–motion mappings via force metaphors (see energy profiles Figure 4).

In addition, figurative gestural materials also articulate the rate–effort–to–density approach via the mapping of average note–inter–onset to granular density and grain size. Fundamentally, this mode of interaction can be seen as relational: functional dynamics of engagement with the environment are embodied within performance gestures. Johnson [15] has proposed a typology of qualitative dimensions of movement: tension, projection and linearity. These dimensions deal with the connection between the manner of the movement’s initiation and the form of the resulting gesture. We believe it is significant that this typology bears a striking resemblance to Smalley’s [5] account of energy–motion profiles in electroacoustic music see table 1, below.

(A) QUALITATIVE DIMENSIONS OF ENERGY–MOTION PROFILES

•

Table 1. Comparison of Johnson’s dimensions of movement with Smalley’s energy–motion profiles and embodied associations Johnson Tension Projection Linearity

583

Smalley Motion rootedness Motion launching Contour energy/inflection

Tension and motion–rootedness are correlated with an embodied expectation (force–dynamic) of the effort required to overcome inertia: the persistence of a system’s grounded/stable state (in our case, primarily associated with tonal and spatial

vs 2

Connecting the two gives us key gestural syntax cases

Embodied Association Rate–effort=>overcoming inertia Sudden rate-change / transient movement Coherence of path

http://www.youtube.com/watch?v=5wEgpVuB9-w

TENSION

PROJECTION

LINEARITY

motion rootedness

motion launching

contour energy/ inflection

(B) SPECTROMORPHOLOGY AND (EMBODIED) TIMBRE–SPACE •

Spectromorphology by its nature is a spatiotemporal model of timbral gesture...conceptualised initially in the frequency/time domain

•

It is an embodied theory of timbral dynamics...timbre seen not just as some abstract perceptual map which is a by-product of simple perceptual acts of transcoding...spectromorphology is a guide to timbre as a structural forming principle in sound environments (composed or otherwise)

•

Worth examining its connection with other theories of timbre–space/ representation...e.g. Patton (2007) attempts to extend it as a general model/descriptor of timbral structure across 3 dimensions (Smalley’s basic form is two–dimensional, but various cases imply further dimensionality)

amental entional ense not er space ly tuned s a band even an ed to be xis as in enting a xis then ures can granular f a cello inly can sounds.

Another way to look at timbre is as a Klangfarben, or tone colour (Vaggione 1994). Timbres can have a consistent recognition, regardless of the listener’s ability to attribute a source. Smalley recognises this through his idea of the source-cause texture, a complex multilayered experiential ‘sonic physiognomy whose spectromorphological ensemble permits the attribution of an identity’

(B) SPECTROMORPHOLOGY AND (EMBODIED) TIMBRE–SPACE Morphological notation for interactive electroacoustic music

125

Figure 1. MN representations of a viola note and a piano

phology cluster. Smalley Figure 3. General schema of the MN system. se, and n of the Grey (1977) classic timbre–space studies alley 1994). For the MN system, within a single instrumentalists, the representation of electroacoustic he most timbre must include the acoustic source. Any composite of music, when referring to a timbre that re-occurs, Z axis is ‘flipped’ in comparison with reference the timbre with can no be identified with colour, or fill pattern. timbre that includes a live source must (note sounding body. The representation of electroacoustic greatcan importance for interactive music is the idea transient/noise treatment in Patton) oise processes must reflect the source, as well as the ource bonding. In his analysis Smalley offers a z-axis is processing to the degree of presence of each. archical typology of instrumental source-cause Although the MN system proposed attempts to COMBINE THESE IDEAS TO stinuum. and gives evidence that in electroacoustic music CAN WE articulate electroacoustic processes in such a way as to be lend itself to this hierarchical source-cause .not Sound clear to an instrumentalist, the source sound will, in WORK TOWARDS ct of timbral identity. ‘In electroacoustic music Figure 2. Spectral motion of pitch-shifted paper crushing. nedness) Patton morphological notation some way, always be present – even if simply sounding e source-cause links are severed,(2007) access to any

er, primal, tensile level is not mediated by sourcee texture’ (Smalley 1994). ith interactive music, however, the sounding body urned to the listener. The self-referential aspect of umental performance combined with the presence ectroacoustic sound and processing present the

EMBODIED TIMBRE–SPACE MODELS?

simultaneously with a triggered sample. This is not rooted in what Smalley described as the ‘umbilical security of instrumental source-cause coherence’, nor does this represent a ‘hesitant reserve about cutting loose in order to pursue a freer exploration’ (Smalley 1994). By assimilating duration, spectral space or harmoni-

(B) SPECTROMORPHOLOGY AND (EMBODIED) TIMBRE–SPACE x= synchronicity of entry/ exit of sound elements

y= relative frequency y= spectral centroid position/distribution => embodied => 2 embodied cases/cases scales [1] contour energy/ inflection (overall [1] heightclarity contour energy/ of stable inflection position (overallwithin clarity of stable position within verticality schema)

verticality schema) [2] motion rootedness tensionand [2] motionversus rootedness (verticality schema tension (verticality aspect)=‘degree of schema aspect)=‘degree effort required to of effort required to move/ move/launch’ from launch’ from current current stable vertical stable vertical position position

two profiles of launching/ projection gestures: [a] stable contour energy [b] unstable contour energy a

=>embodied scales (L/R)

b

instability of contour energy vs. stability

From motion launching/ projection (rapid dynamic change) To gradual contour energy/ inflection (relative stability) z= presence/absence of transients/noise x= synchronicity of entry/exit => of partials embodied scales (F/B) =>embodied cases

z= presence/ absence of attack transients/noise

Towards an embodied timbre– space (various gestural cases and embodied associations) => embodied cases From motion rootedness/tension (grounded contact schema/inertia condition) To stable (floating/ flying/ungrounded)

From motion rootedness/ From gradual contour tension (grounded contact energy/inflection schema/inertia (relative stability) condition...noise) To motion launching/ projection To stable (floating/flying/ (rapid dynamic ungrounded) change)

sustained/smooth trajectory

(B) (EMBODIED) TIMBRE–SPACE AND SPATIAL IMPLICATIONS

The Visual Sound-Shapes of Spectromorp

Z

shape’s movement, behaviour, interaction pancy in space, while advancing this vi three-dimensional images promotes consid spatial depth and interplay between late space positions. Spectromorphology’s sound-shapes offe and are adaptable to the individual. Custo approach through vocabulary additions tions, personal preference and creation of images will inspire a whole array of sonic and will hopefully support the realisat composer’s creative intensions. A final note embraces the subjective Figure 27. Depth perspective. visual sound-shapes. The vast variety of and sonic possibilities of this illustr domain representation is also compatible with tions depth influenced by spectromorphology are undo perspective model noted in Blackburn (2011) greatest advantages.

(transients/noise will aid spatial discrimination)

REFERENCES

Figure 28. Spectral space (Smalley 1997: 121).

Blackburn, M. 2009. Composing from morphological Vocabulary: Proposed Pedagogy and Metadata, www.ems-network proceedings.html. Patton, Kevin. 2007. Morphological Notatio

4. (C, D) SCHEMAS, ECOLOGIES AND FORM • The

key benefit of this type of model is that it provides the starting point for an ecological/embodied theory of timbral relations which foregrounds musical macrostructural/ perceptual macrostructural contexts... (i.e. forming principles) (c) Combinations of movement and gestural ecologies; potentially aiding the creation of timbrally-sophisticated (and gesturally sophisticated) musical interfaces or DMIs (d) The embodied dynamics and extended conceptual metaphors of combinations of sounded schemas may explain the gestural connections of larger–scale form

(SOME) CONCLUSIONS •

Spectromorphology’s ‘embodied space’ may be extended via image schema theory to provide insight into a broader range of musical/ sonic spaces...has important pragmatic utility in terms of our connections with other fields (HCI, DMIs/NIMEs, sonification)

•

Also, certain common affordances/gestures in electroacoustic music embody distinctive conceptual metaphors and may extend image schema theory

•

Considering spectromorphology via the gestural dimensions and force dynamics implied by image schema theories may extend generalised composition theories and models of larger–scale structures/form etc.

•

Also, if spectromorphology is an embodied cognitive theory, electroacoustic music is embodied cognitive praxis and may extend current theories of embodied cognition

SELECTED REFERENCES •

Adlington, R. (2003). Moving beyond motion: Metaphors for changing sound. Journal of the Royal Musical Association, 128(2), 297-318.

•

Barrett, N. (2015). Creating tangible spatial-musical images from physical performance gestures. Proc. NIME 2015. Louisiana State University.

•

Blackburn, M. (2011). The Visual Sound-Shapes of Spectromorphology: an illustrative guide to composition. Organised Sound, 16(01), 5-13.

•

Brower, C. (2000). A Cognitive Theory of Musical Meaning. Journal of Music Theory, 44,2, pp.323–379

•

Godøy, R. I. (2006). Gestural-Sonorous Objects: embodied extensions of Schaeffer's conceptual apparatus. Organised Sound, 11(02), 149-157.

•

Graham, R., & Bridges, B. (2014). Gesture and Embodied Metaphor in Spatial Music Performance Systems Design. Proc. NIME 2014. Goldsmiths University of London.

•

Grey, J. M. (1977). Multidimensional perceptual scaling of musical timbres. The Journal of the Acoustical Society of America, 61(5), 1270-1277.

•

Johnson, M. (2007). The Meaning of the Body: Aesthetics of Human Understanding. University of Chicago Press.

•

Lakoff, G. & Johnson, M. (1980). Metaphors We Live By. Chicago, University of Chicago Press.

•

Patton, K. (2007). Morphological notation for interactive electroacoustic music. Organised Sound, 12(02), 123-128.

•

Roddy, S., & Furlong, D. (2014). Embodied Aesthetics in Auditory Display. Organised Sound, 19(01), 70-77.

•

Wilkie, K., Holland, S. & Mulholland, P. (2010). What Can the Language of Musicians Tell Us about Music Interaction Design? Computer Music Journal. Winter 2010, Vol. 34, No. 4, pp. 34–48

THANKS...& QUESTIONS • Further

questions/comments/suggestions?

• Brian

Bridges: [email protected]

• Ricky

Graham: [email protected]