Matteazzi Master's Thesis 2009

DIPARTIMENTO DI INGEGNERIA DELL’INFORMAZIONE

CORSO DI LAUREA SPECIALISTICA IN INGEGNERIA INFORMATICA

TESI DI LAUREA

MUSIC PERFORMANCE WITH DELAYED AUDITORY FEEDBACK: AN EMBODIED PERSPECTIVE

RELATORE: Prof. Giovanni De Poli

CORRELATORE: Prof. Marc Leman (Ghent University)

LAUREANDO: Marco Matteazzi

Padova, 20 ottobre 2009

ANNO ACCADEMICO 2008 – 2009

2

© Copyright by Marco Matteazzi 2009

3

dedicata a mia madre, a mio padre, e ad Elisa

4

5

Noi non abbiamo un corpo, noi siamo un corpo.

We don’t have a body, we are a body.

Pier Paolo Pasolini

6

7

ABSTRACT

Research on music playing with delayed auditory feedback (DAF) shows that timing asynchronies between action and perception profoundly impair performance, to the extent that musical execution may be interrupted. New technologies for music production are often affected by significant latencies, causing playing music to be difficult and unsatisfactory. This scenario calls for the search of useful techniques for playing music in presence of DAF, overcoming the difficulties introduced by the mismatch between action and perception. In this thesis, I approach this issue from an ecological, embodied perspective, considering the role of human body as a mediator between the sonic energy in the environment and the inner world of the performer. I investigate the effects of DAF on piano performance under three different auditory conditions (normal auditory feedback, absence of auditory feedback, DAF), focusing on variations in MIDI parameters and in performers’ body movement, captured through sensors. Results confirm that DAF significantly impairs music performance while absence of auditory feedback does not. Moreover, a diminution in body movement is found in the absence of feedback, whereas under DAF embodied responses seem to depend strongly on the players’ personal attitude.

8

9

ACKNOLEDGMENTS

First, I’d like to thank my supervisor prof. Giovanni De Poli for giving me the opportunity to undertake this extraordinary experience. I am very grateful to all the people at IPEM. Primarily, I whish to thank prof. Marc Leman for his great helpfulness and for the continuous stimuli his ideas provide to me. I especially thank the three people who helped me more during this work, Michiel Demey, Frank Desmet, and Dirk Steenbrugge. The first for his indispensable support; the second for the great job he did with the ANOVA analysis of Paragraph 5.1, as well as for having introduced me to the world of statistics (“there are three kinds of lies: lies, damned lies, and statistics”); the third for his precious assistance with regard to pipe organs. I whish to thank the other people at IPEM who kindly helped me in some occasions: Ivan Schepers, Micheline Lesaffre, Pieter Coussement and Frederik Styns. I’d like to thank all the students of the Systematic Musicology class 2008 at Ghent University for their hospitality and their sympathie. In particular, I’d like to thank the three special friends with which this project started: Imke De Hert, Renske Witteveen, and Pieter-Jan Maes. I thank very much all the people who helped in some ways during this work: Jan Vanlerberghe, Ivo Delaere, Bart Meynckens, Tine Allegaert, Lukas Huisman, Clara Van der Bremt, Frederic Lamsens, Marianne Van Boxelaere, Anke Steenbeke, Herman Streulens, Yves Senden, Delphine Grandsart, Kersten Cottyn, Johan Wijnants, Lize Raes, Giovanni Bruno Vicario, Massimo Grassi, Federico Avanzini, Enrico Marchetto, Luca Mion. Finally, I’d like to thank Valentina Munaro who beautifully corrected the thesis, Nicola Barban and Elisabetta De Cao for their improvised statistics assistance at the No Dal Molin Festival (and not only), Lucie Jurystova for the picture in Figure 1.1, and Alessio Guzzano for making me know the Pasolini’s quote.

10

11

CONTENTS

Abstract

7

Acknoledgements

9

Chapter 1 - Introduction

15

1.1 Music performance

14

1.1.1 Music performance and sensory feedback

14

1.1.2 Music performance as a timed sequence of motor acts

18

1.2 The role of feedback in movement execution

18

1.2.1 Closed-loop vs open-loop models

18

1.3 Auditory feedback in music performance

19

1.3.1 Feedback deprivation

20

1.3.2 Delayed auditory feedback (DAF)

21

Critical interval vs relative time hypothesis

22

Break-point interval

23

Theoretical implications

24

1.3.3 The role of auditory feedback in musical sequence production 25 1.4 New scenarios for the research on music performance affected by DAF

26

1.5 An ecological, embodied approach to the study of music performance with DAF 1.6 Thesis organization

Chapter 2 – Empirical review of music performance with DAF

27 30

31

2.1 Literature survey

31

2.2 Summary

35

2.2.1 Accuracy

36

2.2.2 Timing

36

2.2.3 Dynamics

37

2.2.4 Expressivity

37

12

2.2.5 Body movement

Chapter 3 - Method

37

39

3.1 Subjects

40

3.2 Stimulus material

41

3.3 Conditions

44

3.4 Equipment

44

3.5 Procedure

46

Chapter 4 – Data analysis

49

4.1 MIDI data

49

4.2 Movement data

51

4.2.1 Synchronization of MIDI and movement data streams

51

4.2.2 Intensity of movement

52

4.2.3 Head pitch periodicity

53

Chapter 5 – Results

55

5.1 One-way ANOVA of tempo, dynamics, and intensity of movement against the experimental condition as factor

55

5.1.1 Exploratory data analysis

55

5.1.2 Homogeneity of variances

58

5.1.3 ANOVA table

58

5.1.4 Tamhane's T2 post hoc test

61

5.1.5 Homogeneous subset tables

64

5.1.6 Means plots

65

5.1.7 Summary of the one-way ANOVA results

67

5.2 Correlations

67

5.3 Timing and dynamics profiles

70

5.4 Individual results

72

5.4.1 Tempo and velocity

72

5.4.2 Intensity of movement

76

5.4.3 Periodicity of the head pitch movement

77

13

Chapter 6 – Discussion and conclusions

81

6.1 Timing

81

6.2 Dynamics

82

6.3 Expressivity

83

6.4 Body movement

83

6.5 Conclusions

85

Chapter 7 – Reference bibliography

87

Appendix 1 – Individual subjects’ graphics

97

A1.1 Subject 2

98

A1.2 Subject 3

100

A1.3 Subject 4

102

A1.4 Subject 5

104

A1.5 Subject 6

106

A1.6 Subject 7

108

A1.7 Subject 8

110

A1.8 Subject 9

112

A1.9 Subject 10

114

A1.10 Subject 11

116

A1.11 Subject 12

118

A1.12 Subject 13

120

A1.13 Subject 14

122

A1.14 Subject 15

124

A1.15 Subject 16

126

Appendix 2 – Measurement of the delay times of the church organ of St. Anna in Ghent

129

A2.1 Method

132

A2.2 Results

135

A2.3 Discussion

138

A2.4 Conclusions

139

A2.5 Reference bibliography

140

14

15

CHAPTER 1 - INTRODUCTION

Music performance is a highly skilled human activity in which action and perception are tightly coupled. While playing music, if the primary brain area dedicated to motor control activates, the auditory area activates too, and vice versa. Coupling of action and perception permits the musician to feel immersed in the environment, and to be able of modifying it to his or her taste just moving on the instrument. Such experience of immersion strongly relies on sensory feedback, which contributes to the perception of the action-relevant values of the physical energies in the environment. In particular, auditory feedback has an important role in the matching of produced actions and perceived ones. In fact, as many studies on delayed auditory feedback (DAF) report, alterations of auditory feedback timing that introduce asynchronies between action and perception can profoundly impair performance, to the extent that musical execution may be interrupted: this phenomenon seems due to the need for congruency between what is produced and what is perceived. New technologies for music production are often affected by significant DAF, causing the musical performance to become more difficult and unsatisfactory. Examples may be suites for network playing, in which the large communication distances produce unavoidable latencies. This scenario calls for the search of useful techniques for playing music with DAF, overcoming the difficulties introduced by the mismatch between action and perception. I will approach this problem from an ecological, embodied perspective, considering the role of human body as a mediator between the sonic energy in the environment and the inner world of the performer, made of intentions, meanings, and significations. In this chapter, I introduce music performance as one of the ultimate human motor skills (Paragraph 1.1), requiring high motor and cognitive capabilities, and strongly relying on sensory feedback (Subparagraph 1.1.1). Subparagraph 1.1.2 briefly presents the cognitive view of music performance as a timed sequence of motor acts: according to this approach, research on feedback results central in understanding perception-action coupling mechanisms. In Paragraph 1.2, I discuss the role of feedback in the execution of motor acts in general: open-loop and closed-loop models of motor control are presented. In Paragraph 1.3, I focus on the role of auditory feed-

16

back in music performance, reporting the results coming from investigations on music performance without auditory feedback (Subparagraph 1.3.1) and under DAF (Subparagraph 1.3.2). Subparagraph 1.3.3 resumes these findings. Paragraph 1.4 shows how latency affects many electronic instruments and interfaces, so that DAF contexts are frequently found in the field of music production. Paragraph 1.5 presents the adopted ecological, embodied approach to the study of DAF affected music performance. Lastly, Paragraph 1.6 describes the organization of the next chapters.

1.1 Music performance Music performance is a highly skilled activity that involves both cognitive and motor capabilities and demands for a strict connection between them. The ability of skilled musicians to coordinate fine body movements to produce complex meaningful sequences is often considered as one of the ultimate examples of human motor skills (Bernstein, 1967; Lashley, 1951). Some musicians are capable of carrying into action extreme tasks, showing impressive abilities in the control of hand and finger movements. Skilled pianists, for example, can produce movements at rates that exceed visual reaction times (e.g., in the execution of trills; Lashley, 1951), playing up to 30 sequential notes per second for sustained periods (Rumelhart & Norman, 1982). High-level playing is based on long-term and intensive rehearsal of motor patterns, that has the aim of forming an inner space of automatic motor trajectories to be recalled and generated without paying too much conscious attention to them (Leman, 2007). Decades of regular practise are necessary to completely automate the motor patterns: the hours of training needed can be roughly estimated at 10,000 (Ericsson, Krampe, & Tesch-Römer, 1993; Howe, Davisdon, & Sloboda, 1998). The acquired experience permits good players to focus on musicality - the transmission of the expressive intentions - rather than on movements and technique. Thoughts, emotions, aesthetic forms and ideas can thus be communicated to the audience through sounds.

1.1.1 Music performance and sensory feedback In music performance, a strict correspondence between player’s intentions and generated sound is necessary for the transmission of the correct musical message. Therefore, musicians continuously monitor their performance through sensory feedback,

17

also when they’re playing solo. Some musical activities like group playing (e.g., orchestral music) require in addition the player to coordinate with other musicians, so that visual and auditory feedback plays a more central role (for the interdependency between musicians during ensemble performances see Rasch, 1979; 1988). In some playing styles based on group improvisation (e.g., jazz ensembles, see Figure 1.1) feedback is absolutely necessary: musicians are often called to decide in real-time what and how to play basing themselves on the information they collect about what their colleagues are playing at the same time (e.g., in standard jazz improvisation, “interdependent routines such as call and response, propagating motifs, supporting and contrasting dialogs, and a higher level of leader/follower dynamics”; Weinberg, 2002, p. 21). These observations point out the necessity of investigations of the role of feedback in music performance: is feedback always necessary for playing? Is feedback used for error correction? Are musicians able to cope with distorted sensory feedback? In the following paragraphs, relying on the results in literature, I will try to answer these questions limiting to the case of solo music performance, in which feedback is not used for inter-subjects coordination.

Figure 1.1 A jazz quartet performance. In improvised group music auditory and visual feedback have a very central role.

18

1.1.2 Music performance as a timed sequence of motor acts A common approach to the study of human serial behaviours like music performance or speaking implies a focus on the role of feedback. This kind of serial behaviours can be viewed as examples of timed sequences of motor acts: research on feedback investigates how such timed sequences are actually produced, which is indicated by Lashley (1951) as one of the central problems of cognition. The manner in which perceived consequences of actions influence the production of subsequent ones is studied, with the aim of clarifying both planning and production mechanisms. In general, understanding the role of feedback is crucial for the comprehension of the relationships between action and perception (see Pfordresher, 2006): this two components of human behaviour are simultaneously involved in sequence production, and require a certain degree of congruency to keep the production fluent and correct. In the last analysis, understanding the role of feedback is important to clarify the complex relationships between the inner subjective world of human beings, expressed through action, and the external reality (even if, as we will see, such a strict subdivision between inner and outer is somewhat outdated).

1.2 The role of feedback in movement execution 1.2.1 Closed-loop vs open-loop models The control of motor acts such as musicians’ movements can be explained by two main theoretical models: the closed-loop model, in which, during a movement, feedback is used to control if the goal is being achieved, and the open-loop model, in which feedback is not used for correctness control. In the closed-loop model, sensory feedback is necessary since movement control is totally depending on the peripheral information (feedback control hypothesis): the execution and the completion of an action is guided by a centralized comparison between the intended movement and the feedback information. Vice versa, in the open-loop model, execution is centrally leaded by an abstract representation of the motor sequence stored in memory (motor program), and feedback can have a role in determining or triggering possible responses, but not in the guidance of the current movement. An early variant of the open-loop theory is the response-chaining hypothesis (James, 1890), in which

19

movement is composed by a chain of muscular contractions, and the feedback from one contraction (response-produced feedback) serves as a stimulus for the next contraction in the chain. In this approach, sensory feedback, as a trigger, is necessary for the execution of the movement, in contrast with other open-loop theories according to which feedback doesn’t have a role at all in the execution phase. It may be of interest for the present discussion to notice that the response-chaining hypothesis provides an account for the timing among the contractions, very important in skilled activities like music performance: such timing, called relative timing, would be determined by the temporal delays in the various sensory processes (i.e., responses propagation). Each motor control theory relies on some experimental evidences and is opposed by others. In general, closed-loop theories give the better account for longer duration movements, such as driving a car, in which the possibility of error correction is evident, whereas open-loop theories seem more suitable for explaining rapid movements. The hypothesis of the cohabitation of different types of motor control relies on the limited velocity of feedback transmission. In fact, though some studies (Bowman & Combs, 1969; Cohen, Goto, Shanzer & Weiss, 1965; Fuchs & Kornhuber, 1968) have shown that some kinds of very fast responses – 5 to 10 ms – are possible, the elapsed time between the error detection and the start of the correction is estimated on average at 200 ms. During the execution of fast movements, a response time of 200 ms is too long to permit feedback to have an active role in error correction, as stated by closed-loop theories: open-loop theories seem therefore likely, at least for this class of movements.

1.3 Auditory feedback in music performance In order to investigate the role of feedback in sound production tasks like music and speech, audition is the most studied feedback channel. In these kind of tasks, in fact, the output of the system is sound: therefore, though in music performance visual, tactile and proprioceptive feedback are very important (for the importance of vision in music performance see Sloboda, 1982; Banton, 1995), audition is the only feedback channel which consents a direct comparison between the produced action (e.g., a keypress) and the desired goal. Experimenting with auditory feedback is therefore an

20

appropriate solution to test whether open-loop or closed-loop systems are involved in music production. The role of auditory feedback in sound sequence production has been investigated mainly through two experimental approaches. The first one implies a study of the effects of feedback removal: an impairment of production, in the absence of feedback, would indicate the necessity of auditory feedback, at least for a correct realization of the production aspects disrupted by such absence. A second approach consists in studying the effects of feedback alteration: each sound resulting from a produced action is altered, so that the coordination of the auditory feedback with actions is modified. This experimental paradigm is known as altered auditory feedback (AAF). Alterations can occur in the dimensions of time and pitch, where timing alterations introduce an asynchrony between the onset of a produced action and the onset of the corresponding feedback, whereas in pitch alterations the pitch associated with a produced action is not the pitch that is normally associated with that action. Both kinds of alteration can be simultaneously present, giving rise to a combination of onset asynchrony and unexpected feedback content. The AAF paradigm means to investigate the relationships between action and perception, and, in particular, to answer the questions about how action and perception are bounded together.

1.3.1 Feedback deprivation Studies on feedback deprivation seem to show that auditory feedback is not strictly necessary in music sequence production. In fact, though auditory feedback is shown to be important in the learning phase (Finney & Palmer, 2003; Highben & Palmer, 2004), other researches indicate that its absence doesn’t significantly impair the production of learned sequences (Gates & Bradshaw, 1974; Banton, 1995; Finney, 1997; Repp, 1999), even for untrained performers (Pfordresher, 2005). However, auditory feedback may still be necessary in some kinds of fine control, since Repp (1999) reported small effects of its absence on expressive parameters of production. The fact that auditory feedback doesn’t appear to be necessary in music performance supports Lashley’s open-loop theory (Lashley, 1951), that, based on the trilling speed of concert pianists, argued for the impossibility of a role of feedback in motor control during very fast movements (see also Keele, 1968). However, in the above mentioned studies on auditory feedback deprivation, the remaining feedback channels were not

21

inhibited, so that it could be argued that visual, tactile and proprioceptive feedback can still guide players’ movement for a correct execution. Nevertheless, feedback does not have much time to affect execution, which testifies against this hypothesis, as well as the fact that some kind of activities seem to be executable in absence of kinesthetic feedback (Keele & Summers, 1976; Lashley, 1951). To sum up, relying on the results in literature, it seems likely that motor acts in music performance are memorized through sensing during training, to form an inner space of motor trajectories (Leman, 2007). These trajectories can be recalled without the aid of auditory feedback, even if, in this case, a small degradation of fine performance parameters is possible. For what concerns motor control, a motor program (open-loop) model is adopted, at least for fast movements: comparison between feedback and intended results may be used to adapt subsequent actions, but not to guide the current. On the other hand, closed-loop models can be applied to slower movements, for which error correction is possible.

1.3.2 Delayed auditory feedback (DAF) The most extensively studied AAF paradigm consists of introducing a certain delay between the onset of a produced action and the onset of the corresponding feedback: this experimental condition is called delayed auditory feedback (DAF). It is well investigated how DAF strongly disrupts sequence production in many tasks including music performance, speech, and rhythmic tapping. Two early studies on DAF speech (Lee, 1950; Black, 1951) reported significant slowing of production rate, increased sound level, and increased articulatory errors, with a predominance of insertions and repetitions. Many studies confirmed these and other negative effects of DAF on the various kind of sequence production tasks. A review of the studies on music performance under DAF is given in Chapter 2. The introduction of a delay between note onsets and feedback onsets may lead to three different situations, depending on the relationship between the amount of delay (delay length) and the inter-onset-intervals (IOIs) duration. First, when the delay length is shorter than the IOIs duration, the feedback onset of a produced event i occurs before the produced event i  1 : in this case, only the timing of production is altered. The second case occurs when the delay length is equal to the IOIs duration, so that the feedback onset of a produced event i is simultaneous to the produced

22

event i  1 : in such situation, only the pitch contents are altered. The third case occurs when the delay length is longer than the IOIs duration, and the feedback onset of a produced event i succeeds the produced event i  1 : in this case, both timing and pitch alterations are present. In general, DAF disrupts both temporal relationships between note onsets (timing) and notes correctness (accuracy). Recently, Pfordresher (2003) showed that alterations of feedback timing disrupt the timing of sequencing more than the accuracy, whereas alterations of feedback content without asynchrony (e.g., pitch alterations that occur with DAF when the delay time is equal to the IOIs duration) disrupt accuracy but do not influence much timing. These findings suggest that a strict connection exists between the disrupted aspects of the performance (i.e., timing or accuracy) and the kind of feedback alteration (i.e., timing or pitch alterations). Critical interval vs relative time hypothesis A frequently discussed topic regarding DAF disruption is the kind of dependency from the amount of delay, and, in particular, the amount of delay that causes maximal disruption. In the early studies, disruption caused by DAF was found to increase with the amount of delay up to a certain point, called critical (delay) interval, or delay of maximal impairment, and then to reach asymptote (e.g., in music, Gates, Bradshaw, & Nettleton, 1974, found an asymptote around 270 ms,) or to decrease (e.g., in speech, Fairbanks & Guttman, 1958). These findings have given credit to the so called absolute time hypothesis, according to which the delay of maximal impairment occurs when the absolute temporal separation between a produced action ant its feedback onset falls within a certain temporal window, regardless of the production rate. Anyway, all experiments on music performance supporting this view have been using fixed delay lengths, so that, in case of tempo variations or of non-isochronous pieces, the phase relationships between the timing of produced actions and their relative feedback onsets would not be constant. Moreover, production rate was usually not controlled. Recently, Pfordresher & Benitez (2007), using a kind of delay called adjustable delay, which roughly preserve the phase relationships between actions and feedback onsets, found that disruption is best predicted by the relative phase location rather than by the absolute position of feedback onsets. This result gives support to

23

the relative time hypothesis, which states that perception and action are coordinated according to the rhythmic cycles formed by IOIs (cf. Jones, 1976; Robinson, 1972). Break-point interval Another discussed issue concerning DAF is which is the minimum delay length which causes auditory feedback to be actually perceived as delayed and DAF disruption to become significant. This topic is related to the investigation of temporal thresholds in perception and cognition. In a classical study on vision, Card, Moran & Newell (1983) showed that, if two events are connected by an immediate causality relationship and the perception of the second is progressively delayed, degradation of immediate causality starts for some subjects as early as 50 ms. Moreover, they found that, while the perception of immediate causality ends around 100 ms, perception of delayed causality begins at 50 ms, reaches a peak around 100 ms, and terminates around 160 ms, threshold after which the events are recognized as independent. Research on the temporal ordering of two distinct stimuli showed that the events required a minimum of 30 ms to be perceived as successive, regardless of sensory modality (Poppel, 1997). In the field of music perception, Rasch (1979) observed that listeners often judge ensemble performance as synchronous despite asynchronies of 30-50 ms. For what concerns playing an instrument under DAF, Finney (1997) reported that professional pianists may perceive delay lengths of under 10ms, so that this threshold is often suggested as the maximum latency for a music controller (Finney, 1997; Freed, Chaudhary, & Davila, 1997). However, a certain degree of tolerance to higher latencies is well-documented. Dahl & Bresin (2001), in a study on synchronization under DAF, individuated between 40 and 55 ms a possible break-point at which DAF begins to make the performance increasingly difficult. Mäki-Patola & Hämäläinen (2004), testing the threshold of latency tolerance of subjects playing a theremin (a continuous sound instrument without tactile feedback), reported that latency started to be perceived at 30 ms, when comparing to a reference with zero latency. With this delay length, subjects perceived latency with a high degree of uncertainty; conscious detection was found to start at 60 ms. Other studies addressed the break-point interval problem for network duet performances. In this situation, differently from solo playing, inter-subjects coordination is

24

required, so that the effect of DAF is supposed to be even stronger. However, results do not differ from what found as for solo playing. Chew, Sawchuk & colleagues (Sawchuk et al., 2003; Chew et al., 2004) found latency tolerance in network playing to be dependent on both the piece and the instrument played. In general, in such situations, they suggested a general threshold for latency tolerance at 50 ms. Chafe, Gurevich, Leslie, & Tyan (2004), quantifying the effects of latency on rhythmic clapping network performance, showed that the performance is at its best when the round-trip bi-directional latency is comprised between 20 and 30 ms, and degrades with higher latencies. Basing on these empirical results, the interval of delay lengths in the range of 30-60 ms seems therefore to be a plausible break-point for the correct execution of music performance under DAF. A certain variability is to be taken into account, mostly due to the kind of instrument and the piece played. Theoretical implications In a recent review, Pfordersher (2006) resumed the empirical results on music performance under AAF, and DAF in particular, discussing their theoretical implications. What clearly emerges from studies on DAF is that, despite the fact that auditory feedback is not necessary for musical sequence production, a certain match between auditory feedback and produced actions is required when feedback is present. In other words, congruency between action and perception is needed: miscoordination between them causes disruption of performance. In particular, it seems that high impairment from DAF is due to the fact that the feedback sequence is equal in structure to the planned events sequence, but it is also in conflict with it, since each feedback onset occurs simultaneously to the production of events with a different position in the planned sequence. In other words, it is likely that DAF disrupts production by virtue of the interfering effect of perception of events planned for the past on the current activation of subsequent events for production. This view is strengthened by research on AAF paradigms different from DAF: manipulations of feedback contents resulting in a sequence of events highly dissimilar to the planned sequence (extraneous feedback) do not disrupt performance as much as DAF does (Gates & Bradshaw, 1974; Finney, 1997; see Finney 1999 for a review of similar results in speech production). Further support comes from the reduced disruption reported by Finney

25

(1997) and Pfordresher (2003) for combined alterations of timing and pitch: instead of a summation of the different effects of asynchrony and changed contents, overall impairment decreased, suggesting that combined alterations cause the feedback sequence to be perceived as unrelated to the planned sequence. Given these findings, Pfordresher (2006) hypothesizes a new theoretical framework in which action and perception share a common representation of sequence structure in memory guiding both the planning of actions and the interpreting of the perceived consequences of those actions. This theory is consistent with neurophysiological evidence for the so-called mirror neurons, that respond in the same way when humans or monkeys execute an action and when they perceive the same action executed by another individual (Rizzolatti, Fogassi & Gallese, 2001; see Leman, 2007, pp. 95-96, for a brief summary on this topic). Moreover, Pfordresher’s model assumes a basic functional separation between timing and sequencing (see also Krampe, Mayr & Kliegl, 2005; MacKay, 1987), thus accounting for the fact that different kinds of feedback alteration cause different kinds of disruption. Similarly to the research on feedback deprivation, studies on AAF provide evidence against the feedback control theories. In fact, if feedback was used for error correction, any kind of feedback alteration would signal that an error has occurred, thus causing disruption of performance. Instead, as said, not all alterations of pitch contents strongly affect production. In Pfordresher’s opinion (Pfordresher, 2006), the problem with classical closed-loop theories is that they focused on the relationships between feedback and planning limiting the field to individual events, whereas it is plausible that these relationships involve sequences of actions and concurrent feedback sequences.

1.3.3 The role of auditory feedback in musical sequence production To conclude, research on music performance shows that, in general, auditory feedback does not function as a “feedback”, since it is not strictly necessary for musical sequence production; rather, it is a “recurrent auditory information” (Howell, 2004) that facilitates learning (Finney & Palmer, 2003) and control of fine nuances of performance (Repp, 1999). However, studies on DAF show that a congruence between auditory feedback and actions is needed. Citing Pfordresher (2006, p. 195), this congruence perhaps reflects “a more general sensitivity to statistical regularities in the

26

environment, including the relationships between actions and correlated perceptual events”. In fact, according to motor program theories, an internal model of movement trajectories guides the execution of actions, relying on the expected consequences of these actions: incongruences between perceived and planned consequences of actions disrupt production, presumably because of the shared representation of action and perception in memory. In conclusion, “the way in which people coordinate perception and action during music performance builds on a general tendency for people to relate action plans to perceptual information” (Pfordresher, 2006, p. 195).

1.4 New scenarios for the research on music performance affected by DAF

Normally, as for traditional acoustic instruments, feedback onsets are perceived as simultaneous with the actions generating them: the sound is directly produced by the player’s movements, without time-consuming technological mediations, and it is generated close to the player, at distances on the order of few decimetres or few metres. One remarkable exception is given by church organ, in which mechanical factors may cause considerable delays – sometimes of hundreds ms (see Appendix 2) between the player’s actions and the diffusion of sound. An example of source of latency in the mechanics of church organs is the pressure transmission from the keyboard to the pipes, especially in the organs in which such transmission is pneumatic, rather than mechanical or electronic. Recent developments in digital technologies have opened new prospects in which a DAF condition may be inevitable. In the last two decades, a proliferation of music instruments and interfaces in which the auditory content is processed, or transmitted over a network, has forced musicians, instrument makers and developers to cope with significant latencies: as with church organs, amount of DAF may be on the order of hundreds ms (e.g., in the field of vocal controllers, Hämäläinen, Mäki-Patola, Pulkki, & Airas, 2004). An important and promising context in which latency is necessarily present is that regarding systems for collaborative network playing such as duet at a distance (Sawchuk et al., 2003; Chew et al., 2004; Chafe, Gurevich, Leslie, & Tyan, 2004; Bartlette et al., 2006; Kapur, Wang, Davidson, & Cook, 2005; for re-

27

views see Barbosa, 2003; Follmer, 2002; Weinberg, 2002). Other examples of latency-affected systems for music production may be physical models instruments, gestures-driven systems for sound generation and control, handheld devices wirelessly linked to the Internet and audio services (see Smith, 2001), vocal controllers for edutainment (Hämäläinen, Mäki-Patola, Pulkki, & Airas, 2004) or sound synthesis (Smith, 2001; Janeer, 2008). In such systems, the introduction of delays is mainly due to two kinds of constraint (cf. Smith, 2001): first, physical constraints limit the speed of signal transmission through the various media; second, practical limits affect the speed of signal processing. In suites for network playing, transmission constraints are often predominant over processing limits, due to the long distances and to the fact that round-trip latency also has to be taken into account (Kapur, Wang, Davidson, & Cook, 2005). Instead, in other latency-affected systems like those cited above, computation time is usually predominant (e.g., see Smith, 2001). The total latency of a system is given by the sum of all the delays from the source to the user. Sources of latency are numerous: examples are delays due to conversions (e.g., analogue to digital), features extraction (e.g., pitch detection from an audio stream), buffering (e.g., of input samples in audio drivers and APIs), internal latencies of subparts of the system (e.g., of applications or sensors), operating systems activities (e.g., context switching, inter-thread communication). The total latency of a system may be variable in time, especially in the case of systems for network playing, for the reason that “long-distance networks with Internet Protocol (IP) routing often result in asymmetry, jitter and packet loss” (Chafe, Gurevich, Leslie, & Tyan, 2004, p. 4).

1.5 An ecological, embodied approach to the study of music performance with DAF

We saw in Paragraphs 1.3.2 and 1.3.3 that studies on DAF give support to the hypothesis according to which perception and action share a common representation in memory. This hypothesis merges into the more general notion of coupling of perception and action, on the basis of which perception it is not merely the sensing of the physical properties of reality (distal stimuli) through the effect of these properties on sensory input (proximal stimuli; Brunswik, 1956); instead, perception is considered as a simulated action, because aspects of the outer world are directly captured in

28

terms of their action-relevant value (called affordance; Gibson, 1979). Duality between an individual and its environment is therefore overcome by the fact that the individual can have a direct access (i.e., not mediated by inference and judgement) to the energies in the environment. This view relies on the observation according to which knowledge of the outer world itself does not emerge from passive perception, “but from the need to act in an environment” (Leman, 2007, p.43; this point is well illustrated by a classic study on kittens by Held & Hein, 1963). The ability to relate sensory features with the cause that generated them is therefore due to the fact that perception and action in individuals are not only linked, but they evolved together: “cognitive structures emerge from the kinds of recurrent sensorimotor pattern that enable action to be perceptually guided” (Varela, Thompson & Rosch, 1991, p.176). Many experimental results are giving support to the idea that perception and action are inseparable in lived cognition, to the extent that they share a common neuronal events code (cf. Hommel, Müsseler, Aschersleben, & Prinz, 2001). Research areas go from observations on imitation in newborn infants to the discovery of the mirrorneurons in human and monkeys (for a brief review of results in these fields, see Leman, 2007, pp. 89-91). In particular, a tight coupling of perception and action is observed in brain activity during music performance: when the primary motor area is activated, the primary auditory area activates too, and vice versa (Lotze et al., 2003; Kristeva et al., 2003; Langheim, Chakarov, Schulte-Mönting, & Spreer, 2002; Hickok, Buchsbaum, Humphries, & Muftuler, 2003; Haslinger et al., 2005). Playing music seems therefore “embedded in a goal-directed ontology of involvement with music” (Leman, 2007, p. 96). Research on DAF, as we showed, brings further arguments in favour of these views. Given that cognition of human beings is determined by the need to act in an environment, with bodies gifted with sensorimotor capabilities and subjected to social and cultural constraints, Varela, Thompson & Rosch (1991) suggested that the study of cognition should concern not the recovery or projection of physical features, but the embodied action. This theoretical approach, called embodied cognition paradigm, overcomes the classic cognitive tradition, “criticized for its neglect of the action component in the subject’s involvement with the environment” (Leman, 2007, p. 43). For what concerns the study of music, Leman (2007) transferred the embodied approach of Varela, Maturana, and others, to the systematic musicology methods, giv-

29

ing born to the paradigm called embodied music cognition. In embodied music cognition, individuals engage with music thanks to the fact that, while perceiving sonic energy, they simulate or imitate moving sonic forms and corporeal articulations: these evoked bodily gestures have a meaning for the listener “due to his or her personal history as an active participant within a cultural environment” (Keller & Janata, 2009, p. 289). The aspects of the outer world, music included, are captured in terms of embodied resynthesis. Thus embodied music cognition focused on the relationships between subjects and their cultural and physical environment, and can be therefore considered an ecological approach to music research. As the term “embodied” underlines, human body has a central role in this paradigm: physical body is considered as a mediator between the mind, which sees music in terms of intentions, meanings, and significations, and the energies in the environment, sonic and not. The possibility of a direct perception of the affordances in the environment through body gestures causes musical involvement to be based on corporeal articulations. Thereby, the study of all musical activities is centred on the role of body movement. Among the aims of the embodied approach to musicology is the search for a mediation technology that links bidirectionally musical energies to gestures. The search is motivated by new digital technologies, in particular real-time interactive music systems (e.g., sensor-based instruments) and information retrieval techniques. In light of what is said above, I will approach the study of music performance under DAF from an embodied perspective: namely, I will investigate the role of body movement in DAF affected music performance, in particular in helping the musician to keep the desired timing profile. The hypothesis is that under DAF musician will attempt to accentuate the feedback modalities important for performance which are not altered, i.e. visual, tactile and kinesthetic. Stressing movements, concentrating on the notation, increasing playing intensity may be useful strategies to give less importance to the auditory channel, through which the misleading information is received. I will check for possible changes in the periodicity of body movement, in order to investigate the effect of DAF on this aspect of corporeal articulation. Moreover, I will verify whether there are individuals capable of playing fluently under DAF, and, in case, which strategies they adopt. For this reason, some of the participants in the experiment were chosen among expert church organists, who are used to play with

30

relevant amount of DAF. The underlying motivation for this research is that embodied strategies may be an important aid to all the musicians who encountered significant DAF while performing on latency-affected electronic interfaces and instruments, such as those mentioned in Paragraph 1.4.

1.6 Thesis organization

The organization of this thesis is the following: Chapter 2 summarizes the existing empirical results on music performance under DAF. Chapter 3 describes the experimental method followed. In Chapter 4, the extraction of the parameters to analyze is explained. Chapter 5 provides the experimental results, while Chapter 6 discusses them and draws some conclusions. Chapter 7 reports the reference bibliography. Two appendices conclude the thesis: Appendix 1 reports graphics concerning the experimental results of the single subjects, while Appendix 2 describes the measurements of the amount of DAF of St. Anna church organ, in the city of Gent, Belgium.

31

CHAPTER 2 – EMPIRICAL REVIEW OF MUSIC PERFORMANCE WITH DAF

In this Chapter, I summarize the experimental results concerning music performance with DAF. Paragraph 2.1 provides a survey of the existing studies, briefly indicating the performance parameters studied, the stimulus materials, the experimental conditions, the relevant points about the procedures, and the salient results. In the survey, only the studies in which the stimulus is a musical piece or a melody are considered: single-note or tapping performances are not taken into account. In Paragraph 2.2 a summary of the results is given, considering each of the following aspects of music performance: accuracy, timing, dynamics, expressivity, body movement. Before proceeding, I will provide some clarifications about the glossary adopted in the following sections and in the rest of the thesis. With normal condition, I refer to an experimental condition in which the auditory feedback is not altered, in contrast with altered auditory feedback paradigms such as DAF or extraneous auditory feedback (cf. Paragraph 1.3). Analogously, silent condition refers to an experimental condition of feedback deprivation, whereas delayed condition to a DAF paradigm. Lastly, key velocity (or simply velocity) indicates the MIDI parameter that represents the force with which a keyboard key is pressed, thus providing a measure of the playing dynamics.

2.1 Literature survey The first studies on the effect of DAF on music performance were carried out by Kalmus, Denes & Fry (1955) in a research on clapping under DAF. In their work, they reported that “exploratory experiments showed that whistling and the playing of musical instruments were, in fact, strongly influenced by delayed acoustic feedback”. The first research explicitly designed to investigate this issue was accomplished by Havlicek (1968), who tested the effect of DAF on the number of errors during sight-reading of unfamiliar musical compositions, as well as the difference in susceptibility among woodwind, strings, brass and piano performers. With a delay time of approximately 200 ms, performance under DAF was found to be louder than

32

in a normal condition, it showed increased IOIs, and included more errors. Performance was disrupted for all instruments used. Gates, Bradshaw and colleagues have performed a number of studies concerning DAF and music playing. As reported by Finney (1999) in his review, in their experiments subjects were asked to play “as fast as possible” a practiced piece from notation. The dependent variable was the total elapsed time necessary to perform the piece. In a study focused on the left-right differences between ears, Bradshaw, Nettleton & Geffen (1971) tested the effects of different delays on a piano performance combining normal auditory feedback with DAF. 200 ms delay was found to be more disruptive than 400, 550, 750 and 1100 ms delays. Gates & Bradshaw (1974), studying subjects performing an etude on an electronic organ, found that a performance under 180 ms DAF took longer than a performance with normal feedback. Another aim of this study was to find out whether a difference between genders subsist in auditory perception, with the result that no significant difference was found. Gates, Bradshaw and Nettleton (1974) compared the effect on keyboard performance of 12 different delay times, equally spaced between 100 and 1000 ms, in a task combining normal auditory feedback with DAF. They found that disruption in total elapsed time increased with delay length and reached asymptote at 270 ms. In this regard, it is important to point out that the individual rate of playing was not controlled, though a post-hoc analysis suggested that the slower and the faster players were similarly affected. Another result was the reported tendency to repeat individual notes or to insert an extra note that, for instance, extended a scale passage one note beyond its correct conclusion. Insertions and repetitions never comprised more than a single note. This fact can be correlated with the increased total elapsed time, insofar as insertions and repetitions can be viewed as two of the factors which cause the performance to be completed in a longer period of time. Finney (1997) examined performances of Bach pieces by trained pianists under 250 ms DAF condition. Individual subjects were asked to play at their preferred tempo “without expressive variations”. Performances were significantly disrupted by DAF in different ways including note errors, total elapsed time, key velocity and interhand coordination: more errors were made, production rate was slower, and key velocity was higher. Approximately 60% of the total note errors under the DAF condition were insertions. Finney also studied the case of DAF mixed with random pitch al-

33

terations, finding a reduced impairment in comparison with the simple DAF condition. Pfordresher and colleagues have performed a number of studies on the coordination of perception and action in music performance, providing different auditory feedbacks to subjects playing on an electronic keyboard. In contrast with the methods of the previous research, which used real piano pieces as stimulus material, they chose simple isochronous melodies of 8 or 12 quarter notes to be performed with the right hand only, appositely “designed to be easy to produce and repeatable without changes in hand position”. The musicians were asked to play the melodies in a “flat” way (mechanically), so that possible increased timing variability could be considered as an evidence of disruption. Pfordresher & Palmer (2002) focused on the effect of DAF on the timing of music performance, and in particular on the phase relationships between produced onsets and auditory feedback. In Experiment 1, they had pianists performing melodies at two production rates (600 ms and 500 ms IoIs) with different amounts of DAF (0, 100, 150, 200, 250, 300 and 350 ms). Timing variability of performances was measured through the coefficient of variation, CV (standard deviation of IOIs/mean IOI). The results show a general increase in timing variability with the amount of delay, with a relatively reduced timing variability for .5 phase ratio (twice the amount of DAF) for the 500 ms IOIs production rate (but not for the 600 ms). In Experiment 2, pianists had to choose the preferred tempo with 200, 250, 300 and 350 ms delays. Performers chose slower rates for larger delays, with the preferred rate approximated twice the amount of DAF (.5 phase ratio). In Experiment 3, Pfordresher & Palmer tested whether instructions to mentally subdivide produced events in two or three different blocks lead to a reduced disruption. The experiment included four delay conditions (none, 200, 300 and 400 ms) at the 600 ms IOIs production rate. As in Experiment 1, they found that temporal variability increased with DAF. A second relevant finding was that a deliberate subdividing reduced the timing variability with longer feedback delays. Finally, they failed to demonstrate a reduction of DAF disruption when feedback onsets coincided with planned subdivisions, in contrast with the result of reduced disruption for .5 phase found in Experiment 1. Pfordresher (2003) used a new kind of delay, named phase shift (or adjustable delay), that adjusted to produced timing so that feedback onsets maintained a roughly

34

consistent relative phase position within produced IOIs. In Experiment 1, phase shifted tones occurred after their associated keystrokes but before the subsequent ones. The phase shifts used were of .33, .50 and .66, with respect to the expected IOIs. Results showed that, in comparison with a normal feedback performance, phase shifts increased timing variability and slowed production rate, while error rates were only marginally increased. However, error rates did increase with the amount of phase shift, with most errors occurring with a .66 phase shift. In Experiment 2, auditory feedback onsets and produced keystrokes were synchronous, but feedback pitches were altered to match pitches associated with earlier keystrokes by a lag of one, two or three events (serial shift). Alterations of feedback pitch without asynchrony were found to increase errors but not to influence produced timing. Experiment 3 incorporated combined phase and serial shifts, which caused a moderate disruption of timing and accuracy and revealed interactive effects of serial and phase shifts on production. Pfordresher & Palmer (2006) explored the effects of serial shifts on music performance in case of pitches matching events intended both for the past (delays) and for the future (prelays). The trials tested the feedback distance of 1 to 3 events in both feedback directions (past and future). All alterations disrupted the accuracy of the performance more than timing. Overall disruption was not influenced by feedback distance or by whether feedback events originated from past or future events. The specific kind of errors, anticipatory or perseveratory, was found to depend on feedback direction: future feedback increased perseveratory errors, while past feedback increased anticipatory errors. Pfordresher & Benitez (2007) investigated whether disruption from DAF reflects the phase location of feedback onset or the absolute temporal separation between actions and sounds. Non skilled participants had to play simple isochronous melodies or to tap an isochronous beat at the production rates of 330, 500 and 660 ms IOIs. In Experiment 1 fixed delays of 330, 500 and 660 ms and adjustable delays of 66%, 100% and 132% of the produced IOIs were used. Experiment 2 presented the same conditions as Experiment 1, except for shorter delays, appositely “designed to form a distribution around lengths that should cause maximal disruption according to the absolute time hypothesis” (see Paragraph 1.3.2): 165, 250 and 330 ms for fixed delays and 33%, 50% and 66% of produced IOIs for adjustable delays. For both delay types,

35

results showed that disruption was best predicted by the phase location of feedback onsets, and it decreased when feedback onsets formed harmonic phase ratios (phase synchrony). In contrast with a finding of Pfordresher & Palmer (2002, Experiment 2), no relative advantage was found for .5 phase ratio. Finally, different movement tasks (melody production versus tapping) led to slightly different patterns of disruption across phase. Moelants, Demey and Leman (2009), in a research for which the present study was a preliminary investigation, had 10 professional musicians performing four different pieces of piano music on an digital piano, with DAF of 200 and 300 ms. The pieces were “Für Elise” by L. van Beethoven, “Sonatina op.36/1” by M. Clementi, “Bulgarian Rhythm (Microcosmos 113)” by B. Bartok, and “Sarabande in g-minor” by G. F. Händel. The analyzed aspects were dynamics, tempo per measure, amount of asynchrony (between notes notated on the same point in the score), errors (order errors, deletions, wrong notes, insertions), and upper body movements. In all pieces they reported a significant increase, with the DAF conditions, in both key velocity and measures duration. In three pieces out of four, other significant differences with respect to the normal condition were observed: increased measures duration variability, increased amount of asynchrony, with a larger asynchrony in the 300 ms delay, and increased errors (insertions in particular). In the slowest piece, the Händel Sarabande, no significant effects of DAF on these parameters were found. Head movements showed a significant increase in the delayed condition in three of the pieces played, with exception of the piece by Clementi.

2.2 Summary All the studies evidenced that DAF profoundly disrupts performance, “to the extent that a skilled performer sounds like a beginner” (Pfordresher 2006). I will now summarize the reported effects of DAF on some specific aspects of music performance keyboard performance in particular - in comparison to normal auditory feedback performances. These aspects include accuracy, timing, dynamics, and performer’s body movement.

36

2.2.1 Accuracy Many studies in which fixed delays and real musical pieces were used (Havlicek, 1968; Gates, Bradshaw & Nettleton, 1974; Finney, 1997; Moelants, Demey & Leman, 2009) reported an increased number of pitch errors in performance under DAF, with a preponderance of insertions/repetitions. Other kinds of pitch errors accentuated by DAF are deletions, wrong notes, order errors (Moelants, Demey & Leman, 2009). It seems therefore proved that DAF disrupts the accuracy of music performance. Despite that, Pfordresher (2003, Experiment 1) showed that the number of errors in an performance increased only marginally with .33, .50 and .66 phase shifts. A likely explanation for this apparent contradiction is that Pfordresher used adjustable delays instead of fixed ones, as well as simple isochronous melodies instead of real pieces, reaching thus a rough control over the phase location of the feedback onset, otherwise difficult to guarantee. It seems indeed that error rates increase with phase shift from the expected feedback onset (Pfordresher 2003, Experiment 1), and that significantly higher error rates are found when feedback onsets are synchronous with subsequent produced events (serial shifts, see Pfordresher, 2003). Therefore, the position of feedback onsets relative to produced time intervals influences the number of errors more than their absolute temporal position (cfr. Pfordresher & Benitez, 2007; the dependence of error rates on the piece-delay length combination in Moelants, Demey, & Leman, 2009, seems to give further support to this thesis).

2.2.2 Timing All previous studies reported that DAF disrupts timing aspects of music performance. In particular, a striking and well-documented effect of the DAF condition on instrument playing is the slowing of the production rate (Havlicek, 1968; Gates & Bradshaw, 1974; Gates, Bradshaw & Nettleton, 1974; Finney 1997; Moelants, Demey & Leman, 2009). Another relevant effect is the increase in timing variability (Pfordresher & Palmer, 2002; Pfordresher, 2003; Moelants, Demey & Leman, 2009). As noticed in the previous paragraph, in some studies timing disruption and increase in error rates may be correlated since repetitions and insertions would cause longer performances. Nevertheless, researches in which error affected performances are re-

37

moved from the analysis show that timing aspects of the performance are disrupted anyway (Pfordresher & Palmer, 2002; Pfordresher, 2003; Pfordresher & Benitez, 2007). For what concerns the dependence of timing disruption on the amount of delay, once again the relative position of feedback onsets with respect to produced IOIs predicts disruption better than their absolute position (Pfordresher & Benitez, 2007). In general, timing disruption increases with the temporal separation of feedback onsets from produced actions, it reaches its maximum when feedback onsets approach the time of the subsequent produced action, and significantly decreases when feedback onsets and produced actions are in phase synchrony. Pfordresher & Palmer (2002) indicated a reduced impairment also for .5 phase ratio (anti-phase coordination), but this result was not confirmed by other experiments (Pfordresher, 2003; Pfordresher & Benitez, 2007).

2.2.3 Dynamics All research on DAF and music playing which considered dynamics aspects of performance (Havlicek, 1968; Finney, 1997; Moelants, Demey & Leman, 2009) reported significant increase in the playing intensity.

2.2.4 Expressivity At the current state of the art, no research has explicitly focused on the effects of DAF on expressivity. However, it can be argued that expressiveness is also debilitated by DAF, at least because it can be considered as a sum of timing and dynamics deviations from a “flat” performance (constant tempo and intensity), and both timing and dynamics are shown to be disrupted by DAF.

2.2.5 Body movement The only study focusing on body movement (Moelants, Demey & Leman, 2009) reports a significant increase in head movement in three out of four of the tested musical pieces.

38

39

CHAPTER 3 - METHOD

In this Chapter, I present the experimental method followed in the investigation. 16 keyboard players were asked to play a piece on an electronic piano under three different auditory conditions: first under a normal auditory feedback condition, then without any auditory feedback, in the end with a feedback delay of 300 ms. Executions were recorded in MIDI format. Body movement was recorded with two sensors, one positioned on the forehead and the other on the chest of the subjects. All the experiments took place at IPEM in Ghent, Belgium. Figure 3.1 shows a participant during the experiment.

Figure 3.1 A screenshot from the video recording of the experiment.

40

3.1 Subjects 16 adult musicians, numbered from 1 to 16, took part in the experiment. They were given some University Cinema tickets as an incentive. The data from subject 1 were discarded because MIDI and sensors data relative to his performances were not accurately synchronized. Therefore, only the data of subjects from 2 to 16 were examined. 8 subjects were male, 7 were female. The average age was 30.6 years, ranging from 18 to 71. It is possible to divide participants into two subgroups, depending on whether they played the organ or not. A first group of eight participants (subjects from 2 to 9) was composed of pianists recruited at the Conservatory and at the University of Ghent, Belgium. None of them had experience with organ playing. The remaining seven participants (subjects from 10 to 16) were organists from the Ghent and the Antwerp communities. For three subjects of this second group, organ was not the only keyboard instrument played: subject 13 and 15 were also pianists, and subject 16 was also a harpsichord player. Only three subjects, within this group, had a long-lasting experience with organ: subject 10, 11 and 12. Instead, subjects from 13 to 16 had studied the organ for less than three years. The average years of keyboard studying among all the 15 subjects, regardless of the kind of keyboard instrument played, was 18.1 years, ranging from 2 to 60. These data are represented in Table 3.1.

41

Subject

Age

Sex

Piano

Pipe Organ

Harpsicord

Keyboard Instruments

Years

Years

Years

Years

2

35

M

28

28

3

19

F

10

10

4

22

M

9

9

5

20

M

12

12

6

26

F

16

16

7

23

F

15

15

8

31

M

11

11

9

23

F

12

12

10

51

M

25

25

11

71

M

60

60

12

43

M

31

31

13

18

F

1

7

14

27

M

2

2

15

28

F

2

20

16

22

F

7 20

2

13

13

Mean

30.6

18.1

St. Dev.

14.5

14.1

Table 3.1 Subjects’ ID, age, sex, and years of keyboard instruments studying.

3.2 Stimulus material An excerpt of the well-known “Sarabande”, from the “Harpsichord Suite IV in D Minor HWV 437” by George Friedrich Händel, served as stimulus material. The sarabandes are slow and grand Baroque stylized dances in triple meter, very popular in the European courts of the 17th and 18th century. They were characterized by accents on the second beat of each measure. The second and the third beats were often tied, giving the dance a distinctive rhythm of quarter and halves notes in alternations. Händel’s “Sarabande” was written around 1703-06 and was first published in 1733. Originally composed for harpsichord, it can be played on the piano with excellent results, and transcriptions for organ are common too. The excerpt used as stimulus was constituted by the first 16 measures of the piece, a period made of two parallel phrases: the first phrase ends with a half cadence, while the second ends with a perfect authentic cadence. Every phrase can be divided in 4 sub-phrases of 2 measures

42

each, with a rhythm typical of the sarabandes of Händel’s early years (cf. Burrows, 1997): . The piece was notated in the key of D minor and featured a 3/2 meter. It had to be performed with both hands. The score given to the musician is showed in Figure 3.2. The chosen tempo of the execution, 56 bpm per half note, was also indicated.

Figure 3.2 The excerpt of the “Sarabande” by Händel used as stimulus material.

The total number of notes to play was 165, the total number of IOI 64. The maximum IOIs values were found in connection with quarter notes: in this cases, the nominal IOI value at 56 bpm was 536 ms.  The piece was chosen after a pilot experiment in which a skilled pianist executed three pieces with different characteristics: besides the “Sarabande”, “Für Elise” and “33 Veräderungen uber einen Walzer von A. Diabelli” by Ludwig van Beethoven. “Sarabande” was chosen for various reasons. First, it is a Baroque piece for harpsichord (sustain pedal is not needed), but commonly played on organ and piano too. Therefore, every keyboard player can perform it regardless of the specific instrument studied. The second reason is that the “Sarabande” is an easy piece to play, within the capacity of every fairly skilled keyboard player. Third reason, from the analysis of the pilot study “Sarabande” was found to be easier to analyze than the other pieces. In fact, it is a slow and majestic piece, in which different chords and notes are clearly detectable, so that it is straightforward to divide the piece in different subunits, referring to the score. For the same reason, note errors are less frequent and, if present, easy to find. The fourth and last advantage of using the “Sarabande” is that

43

at the chosen tempo of 56 bpm the maximum nominal IOI is of 536 ms: considering a delay time of 300 ms for the DAF condition (the delay time actually used), IOIs are always longer than the delay time. Therefore, every feedback onset associated with a produced event i is asynchronous with i but precedes the produced event i  1 (see Figure 3.3). This fact is important because the opposite situation, namely the feedback onset of a produced event i following the produced event i  1 (IOIs shorter than delay time), would have caused feedback to be altered not only in phase but also in pitch. Pfordresher (2003, cf. Paragraph 1.3.2) demonstrated that DAF disruption in the case of pitch shifts is different from disruption in case of phase shifts: while the first primarily disrupts sequencing, the second primarily disrupts timing. If a faster piece like “Für Elise” had been chosen, both kind of alterations would have been present. In such circumstances disruption may reflect both feedback timing and pitch, in this way complicating the analysis of the influencing factors. The choice of the slow “Sarabande” as stimulus material permitted us to avoid this problem.

1071 300

871

1607 300

  0

1407



300



1071

1607

536 (=)

Time (ms)

Produced Onsets / Feedback Onsets Intervals

236







Planned IOIs

536



Feedback Onsets



Produced Onsets

536

Figure 3.3 Relationships between produced onsets and feedback onsets within the main rhythmic figure of the “Sarabande”, assuming a 56 bpm tempo

44

3.3 Conditions Each subject performed the chosen piece under three different auditory condition: normal feedback, silent feedback and delayed feedback (DAF). Under the DAF condition, the sound of each keystroke was heard at a fixed delay time of 300 ms after the absolute time of the event production. Being the piece not isochronous, the relationships between IOIs and delay time are variable (see Figure 3.3): at the chosen tempo of 56 bpm, delay time corresponded to 56% of quarter notes IOIs, to 28% of half notes IOIs, and to 19% of dotted half notes IOIs. The delay time was chosen after a basic statistical analysis of the data of the pilot experiment (cf. Paragraph 3.2), in which the pianist tested the pieces with 8 different delay times: 90, 140, 190, 240, 290, 340, 490, and 890 ms. From the data analysis values around 300 ms (290, 340 ms) resulted to work well with the slow “Sarabande”. In fact, lower values had less influence on the performance, whereas higher values caused the problem of the overlapping of phase shifts and pitch shifts discussed in the previous paragraph.

3.4 Equipment The subjects performed the piece on a Yamaha P60 weighted-key digital keyboard, which simulates the feel of an acoustic piano. The keyboard was connected to a personal computer via a M-Audio MIDI USB Interface. The computer used was a Fujitsu Siemens laptop endowed with a AMD Sempron 3000+ processor clocked at 798 MHz and 992 MB of RAM. The audio card was an external Creative Sound Blaster Audigy 2 NX. An electronic Korg MA-30 metronome was used to suggest subjects the performance tempo. No sustain pedal was used. Under normal conditions, the subjects heard the auditory feedback directly from the piano speakers. Under this circumstance, the selected sound patch was Grand Piano 1, a preset patch simulating a standard acoustic piano timbre. The sound intensity was kept at the same level with all the subjects. The silent condition was obtained just lowering the volume control to zero (the feedback due to the physical key noise from the keyboard wasn’t taken into consideration). The DAF condition was obtained writing a Pure Data patch (see Figure 3.4) which synthesized the incoming MIDI notes after a settable time interval. To set up the system and create such condition, the interval was first set equal to

45

zero: in this way it was possible to measure the intrinsic delay of the system formed by piano, MIDI interface and computer audio card. Then, appropriate values were added to this intrinsic delay, in order to obtain the desired delay times. The notes synthesized by the audio card were played by two speakers positioned over the keyboard ones. The synthesizer sound timbre was chosen and balanced in order to be as similar as possible to the Yamaha P60 Grand Piano 1. The volume was kept approximately equal to the volume of the piano speakers under normal conditions. Keystroke MIDI data were collected using the same Pure Data patch used to generate the delayed auditory feedback condition (Figure 3.4).

Figure 3.4 The Pure Data patch used to record MIDI data and to add the delay time

Movement data were detected through two MT Xsens sensors capable of calculating the Tait–Bryan angles yaw, pitch and roll in real time, as well as outputting calibrated 3D linear acceleration; one sensor was positioned on the forehead of the sub-

46

jects, the other one on the chest. In Figure 3.5 a representation of the Tait-Bryan angles relative to the human head movements is shown.

Figure 3.5 The Tait–Bryan angles yaw, pitch and roll relative to the human head movements.

One more sensor was used to synchronize MIDI and sensors data. Data extraction was performed with Matlab R2007b, whereas data analysis was performed with Matlab R2007b and SPSS 15.0. All the experiments were recorded with a JVC MG30 digital camera.

3.5 Procedure After the selection, each subject was given a few days to learn the score of the excerpt (Figure 3.2). At that moment no explanation was given about the specific experiment to be conducted. Subjects were then tested individually. First, they were given a pre-questionnaire to examine their musical training and background. Sensors were then positioned on forehead and chest of the players using elastic bands. Before starting the measurements, each player had the possibility of training some minutes, with the musical notation present. Only in this training phase, a metronome beat indicated the players the suggested tempo of 56 bpm. They weren’t asked to strictly

47

keep the tempo during the execution, but to use it just as a starting reference, and then add all the timing expressiveness (accelerando, rallentando) they considered appropriate. They were also told to avoid the use of non-written ornaments such as trills or passing notes. Another indication given was not to stop in case of errors, but to continue the execution until the end of the piece, trying to keep the desired execution rate. After the training phase, the metronome was stopped, while the notation was left available on the note-holder. The three auditory conditions (normal, silent and delayed feedback) were executed in succession, with short breaks in between to consent the necessary changing in the experimental apparatus. Each condition was described to the subjects just before the start of the respective recordings. For each condition, the start of the recordings was given pressing sharply the A0 key of the piano with a sensor. This action had the specific function of consenting the subsequent finding of a temporal relationship between the MIDI and the sensors data streams: during the data extraction phase, the A0 keystroke time recorded by the MIDI was bound to the time of the vertical acceleration peak recorded by the sensor, thus obtaining a sufficiently precise temporal coordination between the two streams. In the continuation of the thesis, I will refer to the sensor used to strike the A0 key as to the coordination sensor. If a player stopped in the middle of a recording, the execution under that condition was restarted from the beginning, and, in case, restarted again, until a complete execution was recorded for each condition. After the measurements phase, a post-questionnaire was given to subjects, to express their subjective impressions about the experienced conditions. In particular, questions were asked about the difficulty of the tasks, the emotions felt in the various conditions, the degree of expressiveness added to the piece, and the strategies adopted to cope with the delay.

48

49

CHAPTER 4 – DATA ANALYSIS

In this chapter, I describe the techniques adopted to extract the parameters to be analyzed, starting from the recorded data. A Matlab script was written for this purpose. Such script first imported both MIDI and movement data, respectively from the .txt files outputted by the Pure Data patch and from the .log files outputted by the MT software. MIDI and movement data where then synchronized following the method described in Subparagraph 4.2.1. Working on the imported data, the Matlab script extracted all the parameters of relevance for the subsequent analysis. Some of these parameters are calculated with reference to musical measures: such parameters are tempo, average key velocity, and average intensity of movement (IoM) of head and chest. The calculation of tempo and average key velocity is described in Paragraph 4.1, whereas calculation of IoM is explained in Subparagraph 4.2.2. For what concerns the periodicity of the movement, the Fourier spectra of the head pitch signals were calculated. The focus was placed on the spectral magnitude of three particularly relevant frequencies: the frequencies corresponding to 1-beat, 1-measure, and 2measures periodicities. These spectral values are of interest because possible peaks in correspondence to them would signify regular movements of the head in relation to the metrical subdivision of the piece. The Fourier analysis is described more in detail in Subparagraph 4.2.3.

4.1 MIDI data The Matlab script automatically extracts from the MIDI data the first note of each measure, to obtain a subdivision per measure of all the notes of the performance (see Figure 4.1). For each measure, the first note was chosen as the first note played between the notes belonging to the nominal first chord of the measure. Subdivision per measure was then checked manually, to correct the errors due to missing or wrong notes.

50

Figure 4.1 MIDI piano-roll view of an execution of the Händel’s “Sarabande”. The vertical dotted lines indicate the starting point of each measure.

Starting from such subdivision, average tempo (in bpm) and key velocity were calculated for each measure. Given a measure k, the average tempo for this measure was calculated with the following formula, where sp (k ) indicates the starting point (in s) of the k-th measure relative to the start of the performance, and QPM indicates the number of quarters per measure (in our case, QPM is constant and equal to 3):

tempo(k ) 

1 sp (k  1)  sp (k ) 

 60 .

(4.1)

QPM

The average key velocity of each measure was calculated as the average of the key velocity of all the notes belonging to the measure.

51

4.2 Movement data 4.2.1 Synchronization of MIDI and movement data streams As written in Paragraph 3.5, before every performance a striking of the A0 key with the coordination sensor was used to relate MIDI data with movement data streams. The timing of the peak in the vertical acceleration of the coordination sensor (Figure 4.2), approximately indicating the moment of contact between the key and the keybed (cf. Goebl & Palmer, 2008, case of pressed touch), was taken as starting point for the movement recording, and bound together with the timing of the corresponding keystroke recorded by the MIDI.

Figure 4.2 The peak in the vertical acceleration of the coordination sensor due to the striking of the A0 key.

Having established in this way a temporal relationship between movement and MIDI data, the starting points of each measure, extracted from the MIDI, were bound with

52

the corresponding samples in the movement data stream, so that a subdivision per measure of the movement data was obtained (Figure 4.3).

Figure 4.3 Subdivision per measure of the head angles data stream. Dotted vertical lines indicate the starting point of each measure.

4.2.2 Intensity of movement Moving from the 3D linear acceleration data stream, the absolute value of the jerk (i.e., the temporal derivative of the acceleration) was taken into consideration as a measure of the intensity of the movement, IoM:

2 2  da  da x   da y   da z    IoM  j       ,   dt  dt   dt   dt  2

 where j is the jerk. IoM values were then averaged over each measure.

(4.2)

53

4.2.3 Head pitch periodicity For what concerns the movement angles, only the pitch of the head has been taken into consideration during the analysis. The reason for this choice is that normally updown movements (pitch) of the player’s head are more related to the time-keeping process and to the transmission of expressivity than left-right head movements (yaw) and lateral head oscillations (roll). This point was experimentally confirmed by the fact that head pitch shows much clear periodicities than yaw and roll, and that these periodicities are more related to the subdivision of the music in measures and beats (e.g., see Figure 4.3, in which pitch exhibits a periodicity related to measures). For analogue reasons, chest angles were not considered. Before proceeding with the Fourier analysis, some manipulations were done on the original head pitch signals. First, head pitch data were resampled with linear interpolation so that each measure of each performance contained the same number of samples. Second, signals were averaged to zero, to cancel the 0 Hz components of their Fourier spectra. Third, signals were zero-padded using a zero-padding factor (zpf) of 4, which was found to be the minimum zpf needed to obtain spectral values in the three frequencies of interest for our investigation. It has to be noticed that, thanks to the precise subdivision per measure operated on each performance (cf. Paragraph 4.1), as well as to the fact that after the resampling each measure has the same number of head pitch samples, calculated frequencies for 1-measure and 2-measures are directly proportional to the actual measure frequency of performances. On the other hand, beat frequency is calculated dividing the 1-measure frequency for QPM, therefore resulting in an approximation of the actual beat frequency in the performances. 4-zero-padding resampled head pitch signals were then transformed with a FFT algorithm. The magnitude of the Fourier transforms was normalized so that the sum of all the samples of each transform was one. For the three relevant frequencies, a weight w was calculated as the ratio between the spectral magnitude for that frequency and the overall standard deviation of the spectrum. An example of the resulting Fourier spectrum is shown in Figure 4.4.

54

Figure 4.4 Normalized FFT of the resampled head pitch signal of Figure 4.3. The highest peak is in correspondence to the 1-measure frequency. The weight (w) of the relevant frequencies indicates the ratio between the signal magnitude for such frequencies and the overall standard deviation.

55

CHAPTER 5 – RESULTS

In the present chapter, I will proceed with the statistical analysis of the effect of silent and delayed auditory feedback on the extracted parameters, first considering, for each experimental condition and each parameter, the data from all the subjects as a single distribution. Although there are individual differences between pianists, and it can be expected that the capability to cope with delay will depend on personal factors (e.g. skills or education), it is assumed that most pianist will respond in a similar way depending on the condition. The technique chosen to investigate this hypothesis is a one-way analysis of variance (ANOVA). In Paragraph 5.1, the one-way ANOVA will be performed on those parameters for which we extracted values per measure (cf. Chapter 4): tempo, average key velocity, average head and chest IoM. In Paragraph 5.2, for each of the expressive parameters of the performance (tempo and average velocity), the correlations between the various conditions will be calculated, to check how similar these parameters result in each pair of corresponding measures. In this way, we intend to investigate possible dissimilarities in the expressive content of the performances in the different conditions. Another way to look for different expressive contents is to compare the average timing and dynamics profiles: this will be done in Paragraph 5.3, only with a descriptive purpose, due to the small sizes of the considered distributions. Finally, in Paragraph 5.4, individual results will be summerized. Subparagraph 5.4.1 will focus on tempo and velocity, Subparagraph 5.4.2 on the intensity of movement, while Subparagraph 5.4.3 will report the results of the periodicity analysis.

5.1 One-way ANOVA of tempo, dynamics, and intensity of movement against the experimental condition as factor 5.1.1 Exploratory data analysis In Figure 5.1, the boxplots of tempo, average key velocity, head and chest IoM by experimental condition are shown. The graphics indicate that, in the case of tempo, velocity, and chest IoM, the normality of the distributions can be accepted, although

56

with some uncertainty. This is not the case, however, of the movement of the head: especially in the delayed case, a large amount of extreme values and outliers is present. After an inspection of the individual sequence plots, it was found that there are 3 participants (subjects 8, 11 and 12) who moved a lot more their head in the delayed case, in comparison with the other participants (Figure 5.2). The effect of these 3 subjects on the distributions can be seen when they are excluded from the analysis (Figure 5.3).

a

b

c

d

Figure 5.1 Boxplots of tempo (a), average velocity (b), head IoM (c) and chest IoM (d) by experimental condition (from left to right: normal, silent delayed).

57

Figure 5.2 Example of difference in the head movement (left subject 5, right subject 8).

a

b

c

d

Figure 5.3 Boxplots of tempo (a), average velocity (b), head IoM (c) and chest IoM (d) by experimental condition (from left to right: normal, silent delayed), without subjects 8, 11 and 12.

In the following sections, a one-way ANOVA is used to look for effects of the experimental condition on the measured variables. In order to have an idea about the

58

impact of subject 8, 11 and 12, the ANOVA is performed on the whole dataset and on the dataset excluding subjects 8, 11 and 12.

5.1.2 Homogeneity of variances In order to choose the correct settings for the analysis, a Levene's test of homogeneity of variances is performed (see Table 5.1). All significances are lower than 0.05 for both analyses. This means that homogeneity of variance cannot be assumed. Therefore, a Tamhane's T2 post hoc test instead of a Tukey test is used in the ANOVA analysis (the Tamhane’s test uses the Welch procedure for determining degrees of freedom for the standard error of the contrast; this test is based on the Student’s t distribution and the Sidak procedure is applied to find the alpha level).

Test of Homogeneity of Variances - All Cases Levene Statistic

df1

df2

Sig.

Tempo

6,528

2

672

,002

Average velocity

4,855

2

672

,008

HD_Iom

42,367

2

672

,000

CH_Iom

12,334

2

672

,000

Test of Homogeneity of Variances - 8, 11 and 12 excluded Levene Statistic

df1

df2

Sig.

Tempo

4,366

2

537

,013

Average velocity

8,778

2

537

,000

HD_Iom

3,563

2

537

,029

CH_Iom

9,847

2

537

,000

Table 5.1 Levene’s tests.

5.1.3 ANOVA table Table 5.2 and 5.3 are respectively the ANOVA tables for the all cases and for the selected cases (subject 8, 11 and 12 excluded). The low significance values provide evidence of significant differences between the experiments for both analyses.

59

Sum of Squares Tempo

Average velocity

Head_Iom

Chest_Iom

Between Groups

df

Mean Square

5818,287

2

2909,143

Within Groups

40511,627

672

60,285

Total

46329,914

674

Between Groups

11524,866

2

5762,433

Within Groups

77915,005

672

115,945

Total

89439,871

674

Between Groups

1,178

2

,589

Within Groups

7,473

672

,011

Total

8,651

674

,259

2

,130

Within Groups

3,398

672

,005

Total

3,657

674

Between Groups

F

Sig.

48,256

,000

49,700

,000

52,971

,000

25,652

,000

Table 5.2 Anova table comprising all cases.

Sum of Squares Tempo

Average velocity

Head_Iom

Chest_Iom

Between Groups

df

Mean Square

5943,931

2

2971,966

Within Groups

35782,539

537

66,634

Total

41726,470

539

6736,527

2

3368,264

Within Groups

53095,874

537

98,875

Total

59832,401

539

,131

2

,066

Within Groups

2,652

537

,005

Total

2,783

539

,122

2

,061

Within Groups

2,323

537

,004

Total

2,444

539

Between Groups

Between Groups

Between Groups

F

Sig.

44,601

,000

34,066

,000

13,279

,000

14,060

,000

Table 5.3 Anova Table for the selected data (subject 8, 11 and 12 excluded).

60

Some explanations will be now given about the values present in the tables. The total sum of squares ( SS tot ) is defined as the sum of the squares between groups ( SS bg ) and the sum of the squares within groups ( SS wg ): SS tot  SSbg  SS wg ,

(5.1)

from which k

ni

 y i 1 j 1

k

k

ni

ij  y    ni  yi  y     yij  yi  , 2

i 1

2

2

(5.2)

i 1 j 1

with k the number of trials (or conditions), here 3, and ni the number of measurements for condition i, here 15  15  225 for all the conditions. The degrees of freedom (df) for a total number of measurements N are calculated as follows: df tot  N  1 (here 675  1 ),

(5.3)

df bg  k  1 (here 3  1 ),

(5.4)

df wg  N  k (here 675  3 ).

(5.5)

The mean square (MS) values are calculated as the sum of squares divided by the corresponding df, and are a measure of the variance. For example, in the case of tempo and considering the all subjects (Table 5.2), we can see that the largest amount of variance is explained by the condition. The F  value is then the ratio between the mean square between groups and the mean square within groups: F  value  MSbg MS wg .

(5.6)

Again considering the case of tempo and all data, by way of example, the F  value is 2909.1 60.3  48.3 . In this case, the critical F  value Fcrit at   0.05 , with

df (2,672) , is equal to 3: F  Fcrit , therefore the null hypothesis that the variance is due to chance is rejected in favour of the hypothesis that there is a difference among the means in the 3 conditions. The ANOVA tables provide evidence for an effect of condition on the mean values of the investigated variables. In the next subparagraph, in order to explain which conditions differ from others, post-hoc tests are used.

61

5.1.4 Tamhane's T2 post hoc test Table 5.4 shows the results of the Tamhane’s post hoc test for the all data. In the case of tempo, the table shows that the effect of delay is negative, meaning that delay causes most players to decrease in tempo. The velocity, on the contrary, reveals a positive effect: due to the delay a higher average velocity occurs. The intensity of movement variables shows significant differences between all 3 experiments.

62

Dependent Variable

(I) Experim. (J) Experim.

Tempo

Normal

No audio Delayed

No audio

Delayed

Average

Normal

velocity No audio

Delayed

HD_Iom

Normal

No audio

Delayed

CH_Iom

95% Conf. Interval

Mean Difference

Normal

No audio

Delayed

Normal

(I-J)

Std. Error

Sig.

-,72280

,68454

,645

-2,3638

,9182

5,83511

*

,71823

,000

4,1131

7,5571

,72280

,68454

,645

-,9182

2,3638

Low. Bound Up. Bound

Delayed

6,55791

*

,78940

,000

4,6659

8,4499

Normal

-5,83511*

,71823

,000

-7,5571

-4,1131

No audio

-6,55791

*

,78940

,000

-8,4499

-4,6659

No audio

-1,32733

1,04678

,498

-3,8363

1,1816

Delayed

-9,35338

*

1,02917

,000

-11,8202

-6,8866

Normal

1,32733

1,04678

,498

-1,1816

3,8363

Delayed

-8,02604*

,96795

,000

-10,3459

-5,7062

Normal

9,35338*

1,02917

,000

6,8866

11,8202

No audio

8,02604*

,96795

,000

5,7062

10,3459

No audio

,027238*

,006643

,000

,01132

,04316

Delayed

-,071808*

,011396

,000

-,09916

-,04446

Normal

-,027238*

,006643

,000

-,04316

-,01132

Delayed

-,099046*

,011070

,000

-,12563

-,07247

Normal

,071808*

,011396

,000

,04446

,09916

No audio

,099046*

,011070

,000

,07247

,12563

No audio

,020980*

,006103

,002

,00635

,03561

Delayed

-,026916*

,007373

,001

-,04459

-,00924

Normal

-,020980

*

,006103

,002

-,03561

-,00635

Delayed

-,047895

*

,006574

,000

-,06366

-,03213

Normal

,026916*

,007373

,001

,00924

,04459

No audio

,047895

*

,006574

,000

,03213

,06366

*. The mean difference is significant at the 0.05 level.

Table 5.4 Tamhane's T2 post-hoc test, all data.

When excluding subjects 8, 11 and 12, the conclusions for tempo and average velocities remain the same. On the other hand, it is found that the intensity of movement is different in the case of no feedback (the subjects move less in the absence of feed-

63

back) and that there is no significant difference between the normal and delayed condition.

Dependent Variable

(I) Experim. (J) Experim.

Tempo

Normal

No audio Delayed

No audio

Delayed

Average

Normal

velocity

Normal

No audio

Normal

No audio

,771

-2,6432

1,2427

6,66167

*

,83962

,000

4,6466

8,6767

,70022

,80969

,771

-1,2427

2,6432

5,1364

9,5874

Normal

-6,66167*

,83962

,000

-8,6767

-4,6466

No audio

-7,36189

*

,92767

,000

-9,5874

-5,1364

No audio

-1,61622

1,10948

,377

-4,2782

1,0457

*

-8,16872

1,06624

,000

-10,7274

-5,6100

1,61622

1,10948

,377

-1,0457

4,2782

*

,96334

,000

-8,8636

-4,2414

*

1,06624

,000

5,6100

10,7274

*

,96334

,000

4,2414

8,8636

No audio

*

,027049

,007446

,001

,00918

,04491

Delayed

-,009802

,007763

,502

-,02843

,00882

*

,007446

,001

-,04491

-,00918

*

-,036852

,006993

,000

-,05363

-,02007

,009802

,007763

,502

-,00882

,02843

*

,006993

,000

,02007

,05363

No audio

*

,023426

,006276

,001

,00836

,03849

Delayed

-,012821

,007641

,257

-,03115

,00551

*

,006276

,001

-,03849

-,00836

*

-,036246

,006811

,000

-,05260

-,01989

,012821

,007641

,257

-,00551

,03115

*

,006811

,000

,01989

,05260

Normal

Normal

Normal

Normal

Normal Delayed

Delayed

,80969

,000

No audio CH_Iom

-,70022

Low. Bound Up. Bound

,92767

Delayed Delayed

Sig.

*

No audio Normal

Std. Error

7,36189

Delayed Delayed

(I-J)

Delayed

Delayed No audio

HD_Iom

95% Confidence Interval

Mean Difference

Normal No audio

-6,55250

8,16872 6,55250

-,027049

,036852

-,023426

,036246

*. The mean difference is significant at the 0.05 level.

Table 5.5 Tamhane’s posthoc test, selected data (subjects 8, 11 and 12 excluded).

64

5.1.5 Homogeneous subset tables The homogeneous subset tables enable to divide factors into subsets, resulting in an alternative way to look for differences or equalities between factors. By way of example, for what concerns tempo and all data, it can be seen that the delayed condition differs from the normal and the silent, which instead belong to the same subset. The conclusions which can be drawn from these tables correspond, as expected, with those of the Tamhane’s table.

Tempo

Average Velocity

Subset for alpha = 0.05 Experiment

N

1

Delayed

225

Normal

225

No audio

225

Experiment

2

56,7759

Sig.

1,000

Subset for alpha = 0.05 N 225

58,4158

62,6110

No audio

225

59,7431

63,3338

Delayed

225

,585

No audio

225 ,16033

Normal

225

Delayed

225

Sig.

1

2

,391

3

,25938 1,000

1,000

Chest Intensity of movement

,18757

1,000

67,7692

Sig.

Subset for alpha = 0.05 N

2

Normal

Head_Intensity of movement

Experiment

1

1,000

Subset for alpha = 0.05 Experiment

N

No audio

225 ,18425

Normal

225

Delayed

225

Sig.

1

2

3

,20523 ,23214 1,000

Table 5.6 Homogeneous subset table, all data.

1,000

1,000

65

Tempo

Experiment

Subset for alpha =

Subset for alpha =

0.05

0.05

N

Delayed

180

Normal

180

No audio

180

Average velocity

1

Experiment

2

56,3690

Sig.

58,7731

63,0307

No audio

180

60,3893

63,7309

Delayed

180

,695

Normal

180

Delayed

180

66,9418

Sig.

,272

1,000

Chest Intensity of movement

Subset for alpha =

Subset for alpha =

0.05

0.05

N 180

2

180

Head Intensity of Movement

No audio

1

Normal

1,000

Experiment

N

Sig.

1

Experiment

2

,16548

1,000

N

1

2

No audio

180

,19253

Normal

180

,21447

,20233

Delayed

180

,22729

,383

Sig.

,19105

1,000

,155

Table 5.7 Homogeneous subset table, selected data (subjects 8, 11 and 12 excluded).

5.1.6 Means plots The means plots visualize the results of the analysis showing the effects of the 3 factors on the 4 variables.

66

Figure 5.4 Marginal means plots, all data. Top left: tempo; top right: velocity; bottom left: head IoM; bottom right: chest IoM.

67

Figure 5.5 Marginal means plots, selected data (subjects 8, 11 and 12 excluded). Top left: tempo; top right: velocity; bottom left: head IoM; bottom right: chest IoM.

5.1.7 Summary of the one-way ANOVA results 1)

The ANOVA reveals that there is a significant effect on tempo and average velocity as a result of the effect of delay, considering all participants’ data. In particular, delay decreases the tempo and increases the average velocity.

2)

The intensity of movement is more complex: 3 of the 15 participants have a different behavior and move more intensively in the delayed context.

5.2 Correlations Now, the Pearson’s correlation coefficients r between the values of the expressive parameters of the performance, tempo and average velocity per measure, in the three experimental conditions, will be calculated. As in the previous paragraph, the data of all the subjects, for a given parameter and a certain condition, are considered as a sin-

68

gle distribution: we saw in Subparagraph 5.1.1 that for such distributions normality can be accepted. Results are shown in Table 5.8 and 5.9. Correlations - Tempo Tmp_N Tmp_N

Sig. (2-tailed) N Tmp_S

Pearson Correlation Sig. (2-tailed) N

Tmp_D

1

Tmp_S ,848(**)

Tmp_D ,621(**)

225 ,848(**)

,000 225 1

,000 225 ,574(**)

Pearson Correlation

Pearson Correlation Sig. (2-tailed)

,000

,000

225 ,621(**)

225 ,574(**)

,000

,000

N

225 ** Correlation is significant at the 0.01 level (2-tailed).

225

225 1 225

Table 5.8 Correlations between the values of tempo in the three experimental conditions (N = normal, S = silent, D = delayed). Correlations – Average Velocity Avel_N Avel_N

Avel_S

Avel_D

Pearson Correlation Sig. (2-tailed)

1

N Pearson Correlation Sig. (2-tailed) N Pearson Correlation

225 ,851(**) ,000 225

Sig. (2-tailed)

,689(**)

Avel_S ,851(**)

Avel_D ,689(**)

,000 225 1 225

,000 225 ,743(**) ,000 225

,743(**)

1

,000 N 225 ** Correlation is significant at the 0.01 level (2-tailed).

,000 225

225

Table 5.9 Correlations between the values of average velocity in the three experimental conditions (N = normal, S = silent, D = delayed).

Although all the parameters are significantly correlated ( Sig  0.01 ), we can see that the correlations between the values in the normal and the delayed conditions (tempo: r  0.621 ; velocity: r  0.689 ) are lower than the correlations between the values in

the normal and the silent (tempo: r  0.848 ; velocity: r  0.851 ). Another way to look at this differences is to draw the overlay scatter plots (Figure 5.6 and 5.7).

69

Figure 5.6 The relationships between tempo per measure in the normal and silent conditions (green) vs the relationships between tempo per measure in the normal and delayed conditions (blue), with best-fit lines. Normal condition values are represented by the abscissas.

70

Figure 5.7 The relationships between average velocity per measure in the normal and silent conditions (green) vs the relationships between average velocity per measure in the normal and delayed conditions (blue), with best-fit lines. Normal condition values are represented by the abscissas.

5.3 Timing and dynamics profiles Now, the average timing and dynamics profiles in the three experimental conditions will be considered. With timing profile we refer to the variation in tempo as a function of measure, whereas with dynamics profile we refer to the variation in average velocity as a function of measure. For a given condition, the profiles are calculated averaging each measure data over all the subjects. Before proceeding, it must be taken into account that, in our case, the number of samples for each measurecondition pair is low (15). Therefore, these statistics will have only descriptive purposes. In Figure 5.8 the average timing and dynamics profiles are shown.

71

Figure 5.8 The average timing (left) and dynamics (right) profiles, in the three conditions.

The results show that, while the average profiles of the normal condition are kept in the silent condition, they are disrupted in the delayed condition. In particular, a progressive slowing of production rate is evident in the timing profile, causing a remarkable deterioration in the timing shape of the performance. For what concerns the key velocity, although the dynamics shape results similar in all the three conditions, in the delayed it is higher than in the others by a noteworthy constant factor. In Figure 5.9, the differences between the profiles in the normal and delayed conditions are shown with error bars indicating the 95% confidence intervals. It can be noticed how, in the last four measures of the timing profile, the error bars of the two conditions are not overlapping. In the dynamics profile, instead, the error bars do not overlap only in one measure (the 9th).

72

Figure 5.9 The average timing (left) and dynamics (right) profiles, in the normal (blue) and delayed (green) conditions. Error bars indicate the 95% confidence intervals.

5.4 Individual results Now, some individual subjects results will be provided. As for the timing and dynamics profiles, the amounts of considered samples are low (15), therefore these statistics have only a descriptive aim. The complete dataset for each subject can be found in graphical form in Appendix 1.

5.4.1 Tempo and velocity Table 5.10 presents the individual subjects means and standard deviations for the expressive parameters (tempo and velocity) in the three conditions. Moreover, the Pearson’s correlation coefficients r between such parameters are shown for the two pairs of conditions, normal-silent and normal-delayed. In Figure 5.10, 5.11, and 5.12, the values of Table 5.10 are presented in graphical form.

73

Subject

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Condition TEMPO Normal Silent Delay Normal Silent Delay Normal Silent Delay Normal Silent Delay Normal Silent Delay Normal Silent Delay Normal Silent Delay Normal Silent Delay Normal Silent Delay Normal Silent Delay Normal Silent Delay Normal Silent Delay Normal Silent Delay Normal Silent Delay Normal Silent Delay

Mean Std. Dev. 55,64 2,48 55,04 1,66 50,52 2,64 62,04 2,26 65,12 2,09 60,10 4,06 61,75 1,91 66,03 1,27 47,78 6,21 64,13 3,79 67,25 3,34 63,64 2,42 64,79 2,67 69,88 3,17 62,90 6,75 63,50 1,90 66,49 1,70 57,56 2,53 65,65 3,41 68,33 3,42 58,33 4,21 67,27 3,79 70,89 4,67 54,79 7,29 69,76 3,18 67,27 2,83 62,89 4,07 57,47 3,04 58,41 2,91 51,23 2,14 59,68 2,09 58,50 1,79 65,65 3,91 73,12 3,06 74,77 2,37 65,77 5,94 49,43 3,31 44,99 6,87 37,24 6,47 66,08 4,19 61,21 3,72 56,62 2,31 58,86 1,82 55,82 2,19 56,62 2,31

VELOCITY r ,705 -,078 ,719 ,160 ,027 ,716 ,868 ,265 ,580 ,230 ,500 -,078 ,827 ,130 ,511 ,566 ,639 ,744 ,833 ,303 -,019 ,375 ,626 ,082 ,042 -,030 ,803 -,068 ,765 ,244

Mean Std. Dev. 53,05 13,65 59,74 9,21 69,75 5,13 46,67 5,58 46,36 5,48 54,70 5,34 64,02 8,56 59,24 7,94 72,01 6,79 43,97 7,02 51,13 6,84 63,56 8,03 55,14 7,02 63,52 5,44 77,53 3,43 49,72 4,67 54,79 3,76 54,98 3,66 58,48 7,01 53,34 5,52 71,60 2,23 54,14 4,64 50,76 5,21 59,33 4,83 76,48 2,75 74,36 2,95 76,51 2,21 69,35 9,07 72,43 9,04 86,28 7,91 43,12 4,87 45,71 5,58 55,36 6,09 69,08 2,71 66,73 2,68 72,09 3,39 66,47 4,20 70,63 2,09 66,38 5,18 62,73 5,86 63,47 3,58 68,23 3,73 63,81 4,32 63,95 3,35 68,23 3,73

r ,821 ,643 ,881 ,671 ,660 ,668 ,867 ,848 ,738 ,631 ,788 ,749 ,799 ,171 ,683 ,532 -,125 ,074 ,806 ,893 ,766 ,753 ,292 ,260 ,063 ,431 ,752 ,315 ,536 ,031

Table 5.10 Individual subjects means and standard deviations of tempo and velocity in the thee experimental conditions. In the r-labelled column, the Pearson’s correlation coefficients r between such parameters are reported for the normal-silent (Silent-

74

labbelled row) and normal-delayed (Delay-labelled row) condition pairs. The highlights evidence the results in the delayed condition, in comparison with the normal one: blue highlights indicate similarity between the values or high correlations, red highlights remark bad matches or low correlations, whereas yellow highlights refer to peculiar results.

Figure 5.10 Tempo: individual subjects means (left) and standard deviations (right) in the three experimental conditions.

Considering the means of tempo in Figure 5.10, it can be noticed that some subjects (3, 5, 6, 16) were able to keep approximately the same means in the normal and in the delayed conditions. The difference between their means in the two conditions is lower than 3 bmp. In general, however, most of the subjects show lower means values in the delayed condition, with the extreme cases of subjects 4, 9, and 14, for which the difference between the means is higher than 10 bmp. A remarkable exception is given by subject 12, who is the only one who played faster in the delayed condition. For what concerns the standard deviations, individual results are separable in two groups, with subjects 2, 5, 7, 8, 10, 11 and 16 keeping similar values both under the normal and in the delayed conditions, whereas subjects 3, 4, 6, 9, 12, 13, 14, and 15 show higher values in the delayed case. In particular, extreme cases in the two groups are subjects 2, 7, 8, 10, and 11 (difference lower than 1) for the first, and subjects 4, 6, 9, and 14 (difference higher than 3) for the second.

75

Figure 5.11 Average velocity: individual subjects means (left) and standard deviations (right) in the three experimental conditions.

The individual mean values of key velocity (Figure 5.11 left) show that almost each individual played louder with DAF than in the normal condition, with the exceptions of subjects 10 and 14. For six subjects (2, 5, 6, 8, 11, 12), the difference between the two conditions was really marked (more than 10 points). For what concerns the standard deviations, three subjects (2, 6, 8) show a much lower variability in the delayed condition in comparison to the normal (the difference in standard deviations is higher than 3). It is peculiar the case of subject 2, who had a very high dynamics variability in the normal performance, much lower in the delayed condition (13.6 versus 5.1). On the other hand, five subjects (3, 9, 10, 13, 14) have almost identical dynamics variability (difference in standard deviation less than 1).

76

Figure 5.12 Individual subjects correlations (blue: normal-silent; green: normaldelayed) for tempo (left) and velocity (right).

For what concerns the correlations in tempo between the normal and the delayed cases (Figure 5.12 left green), only three subjects (4, 9, 10) show values higher than 0.5. The other subjects present low correlations values, with four subjects (2, 7, 14, 15) having negative correlations. Considering now the correlations between the normal and the delayed case, as for velocity (Figure 5.12 right green), the individual results are higher than those for tempo, with nine subjects having values superior to 0.5. Two of them (subjects 5 and 11), in particular, present correlations higher than 0.8.

5.4.2 Intensity of movement In Figure 5.13, the means of head and chest IoM per subject are shown. The high increase in the head movement for subject 8 and 12 in the delayed condition can be noticed (left).

77

Figure 5.13 Individual subjects means of head (left) and chest (right) IoM in the three experimental conditions.

5.4.3 Periodicity of the head pitch movement For what concerns the periodicity of the head pitch, the calculated weights w for the three frequencies of interest are reported in Table 5.11. The red highlights indicate very significant peaks ( w  20 ), whereas the yellow highlights indicate medium range values ( 10  w  20 ). All individual subjects head pitch graphics can be seen in Appendix 1, together with the plots of the corresponding resampled signals FFTs.

78

Subject Period.

Subject Period. Normal

2

3

4

5

6

7

8

9

2-bars 1-bar 1-beat 2-bars 1-bar 1-beat 2-bars 1-bar 1-beat 2-bars 1-bar 1-beat 2-bars 1-bar 1-beat 2-bars 1-bar 1-beat 2-bars 1-bar 1-beat 2-bars 1-bar 1-beat

14,6 3,7 0,3 4,5 7,5 3,2 9,7 3,6 0,5 13,9 19,2 1,6 20,9 27,0 3,8 15,2 15,0 9,8 21,9 28,3 0,9 17,8 18,2 1,0

Silent

20,4 7,3 0,2 5,5 6,4 1,6 18,5 5,8 0,7 16,8 20,3 0,8 12,8 21,4 2,7 11,7 11,3 11,2 24,7 27,5 1,8 25,7 14,0 0,4

Delay

7,9 25,3 2,1 12,9 13,6 3,3 8,8 9,9 0,9 9,2 6,0 4,3 21,6 6,9 2,9 11,3 6,2 11,4 17,8 24,5 1,2 17,4 17,9 1,1

Normal

10

11

12

13

14

15

16

2-bars 1-bar 1-beat 2-bars 1-bar 1-beat 2-bars 1-bar 1-beat 2-bars 1-bar 1-beat 2-bars 1-bar 1-beat 2-bars 1-bar 1-beat 2-bars 1-bar 1-beat

14,1 12,2 1,0 8,0 24,6 2,0 15,0 31,5 2,1 16,5 23,2 5,2 8,4 28,1 5,4 9,3 22,3 1,7 28,6 9,8 1,0

Silent

20,6 21,4 1,0 4,2 25,3 1,8 4,2 22,0 11,0 10,8 15,9 3,2 0,3 26,8 4,7 14,7 13,8 1,1 12,8 8,5 0,5

Delay

8,2 18,1 1,6 7,8 28,2 4,1 3,7 9,2 34,1 13,4 6,4 3,3 14,1 27,0 4,6 12,5 19,8 2,2 15,1 6,9 1,8

Table 5.11 Weights w of the spectral magnitude for the frequencies related to 2measures, 1-measure, and 1-beat. Red highlights indicate the very relevant peaks ( w  20 ), whereas yellow highlights indicate medium range values ( 10  w  20 ).

A consideration of the weights in Table 5.11 and of the graphics of Appendix 1 makes clear that most subjects changed qualitatively or quantitatively the periodicity of their head pitch in the different conditions. The Table 5.12 tries to point out these changes when comparing the delayed case with the normal.

79

Subject

Periodicity 2-bars

1-bar

1-beat

Body as a reference under DAF (questionnaires)

2 3 4

x =

x

5 6

x =

x

7

=

8 9

x x

=

=

10 11

=

12

x

13

x (trunk)

14

=

=

15 16

Table 5.12 The changes in the individual head pitch periodicities when passing from the normal condition to the delayed one. Vertical arrows indicate diminutions/increases in the corresponding weights, horizontal arrows indicate changes in the main peaks, whereas equal signs signal stationary situations. Thick arrows indicate drastic changes. Cells are left blank when both conditions show no periodicity for the correspondent frequency. On the right column, subjects who reported the use of the body as a reference under DAF are marked.

Although the results show a great variability due to individual differences, two trends seem to emerge in the delayed case: a tendency to reduce the relation between the

80

head pitch periodicity and the musical structures (subjects 5, 6, 7, 8, 13, and 16), and a tendency to relate more with higher frequencies than with low ones, in comparison to the normal condition (subjects 2, 4, 5, 7, 10, 11, and 12 ). Only three subjects (3, 14, 15) showed behaviours in opposition to these tendencies. Noteworthy is the behaviour of subjects 2 and 12, who drastically changed their periodicity in the delayed case: subject 2 switched from very low-frequency movements (see Figure A1.2 in Appendix 1) to a well-defined 1-measure periodicity (1-measure w passed from 3.7 in the normal case to 25.3 in the delayed), whereas subject 12 switched from a very prominent 1-measure periodicity (31.5 in the normal case) to a equally prominent 1beat periodicity (34.1 in the delayed case). According to the questionnaires results, subject 2 did not deliberately choose to change his movement under DAF, while subject 12 did.

81

CHAPTER 6 – DISCUSSION AND CONCLUSIONS

In this Chapter, the experimental results will be discussed, with regards to each of the considered aspects of music performance. Paragraph 6.1 will focus on timing, Paragraph 6.2 on dynamics, Paragraph 6.3 on expressivity, and Paragraph 6.4 on body movement. In Paragraph 6.5, some final conclusions will be drawn.

6.1 Timing The ANOVA analysis of tempo confirmed the results found in literature. The absence of feedback does not have a significant effect on the distribution of the tempo values, in comparison to normal conditions (the means are 62.6 bpm in the normal condition and 63.3 bmp in the silent). This is in line with what found in previous studies by Gates & Bradshaw (1974), Finney (1997), Repp (1999), Moelants, Demey & Leman (2009). On the contrary, DAF has a strong effect on tempo, causing players to significantly slow their production rate (the mean in the delayed case is 56.8 bmp). This slowing effect of DAF, extensively studied with regard to speech tasks, has been previously reported in music performance by Havlicek (1968), Gates & Bradshaw (1974), Gates, Bradshaw & Nettleton (1974), Finney (1997), and Moelants, Demey & Leman (2009). Considering the analysis of the all subjects’ correlations and timing profiles, performances under DAF resulted much more dissimilar with respect to normal ones rather than performances in the absence of feedback. In particular, DAF performances tended to slow in tempo with the advance of the measures, to the extent that, in the last four measures of the piece, the error bars of the normal and delayed condition timing profiles do not overlap (cf. Figure 5.9) . This finding seems to indicate an additive effect of timing disruption by DAF, so that the decrease in tempo depends on the metrical positions. In other words, the players’ confusion caused by DAF seems to increase measure after measure. However, the individual timing profiles in Appendix 1 suggest that this effect may depend strongly on the individuals’ personal capabilities: although most subjects slowed down during the performance, with subjects 4, 8 and 9 as extreme cases, other subjects (5, 10, 11, 15, 16) did not show this tendency at all. In general, personal abilities seem to play an impor-

82

tant role in determining the effect of DAF on timing. While some subjects (5, 10, 11, 16) were capable of keeping the various aspects of timing almost at the same level as in a normal performance, most of the performances degraded in one or more timing aspects. To conclude, noteworthy is the case of subject 12, who is the only one who played faster in the DAF condition. As we will see in the Paragraph 6.4, this can be interpreted as a consequence of a deliberately adopted embodied strategy.

6.2 Dynamics The ANOVA analysis on the average key velocities provide further support to the literature results. The auditory feedback deprivation did not seriously affect the dynamics of the performances (the mean in the silent condition is 59.7, against the 58.4 value in the normal condition), as previously reported by Finney (1997) and Repp (1999). The DAF, instead, caused a significant increase in loudness (the mean value in this case is 67.8), in accordance with the findings of Havlicek (1968), Finney (1997), and Moelants, Demey & Leman (2009). A possible explanation for this fact is that, under DAF conditions, the players attempted to rely more upon tactile feedback, which was not altered. Another possible account for this effect is that players, having the impression that the notes were not “coming out”, tried to “force” them playing louder. The all subjects’ correlations and dynamics profile show that DAFaffected performances were less similar in the expressive dynamics to the normal conditions performances as opposed to the silent condition ones. In particular, a comparison of the dynamics profiles in the delayed and in the normal condition shows that, although the average dynamics shapes were very similar, an almost constant increase in loudness affected the values of each measure in the delayed case. The individual subjects’ correlations between the normal and the delayed condition were better than those regarding tempo, which may give support to the hypothesis that the dynamics shapes suffer DAF less than the timing shapes (cf. Figure 5.8). As in the case of tempo, however, a great individual variability seems to emerge.

83

6.3 Expressivity The expressivity of the performances was more impaired in the case of DAF than in the absence of auditory feedback. As for all subjects’ data, this is shown by the lower correlations coefficients in the normal-delayed pair (Paragraph 5.2), as well as, in a less significant but more impressive way, by the timing and the dynamics profiles (Figure 5.9). The specific effects of DAF on the single parameters that shape an expressive piano performance, i.e. the expressive timing and the expressive dynamics, are discussed in the previous paragraphs. An important support to the hypothesis that DAF significantly disrupts the expressivity of the musical performance comes from the analysis of the questionnaires. 13 subjects out of 15 affirmed they played expressively in the normal condition. Considering the silent condition, 7 subjects stated they could add the performance the same expressivity with respect to normal conditions, 3 subjects reported they added more, and 5 subjects affirmed they could not add the same expressivity. Under DAF, only 2 subjects (10 and 14) reported they could play as expressively as in the normal condition, while the other 13 affirmed they could not.

6.4 Body movement The ANOVA analysis of the intensity of movement provide two results. First, the absence of auditory feedback caused subjects to move significantly less than in a normal condition. Second, in the DAF condition three subjects (8, 11, and 12, cf. Figure 5.13 left) moved their head much more than in the normal condition, causing the all subjects’ values of the head IoM to deviate from normality: when they are excluded from the analysis, the ANOVA reports no significant differences between the normal and the delayed conditions, for both head and chest IoM. These results are in partial contrast with the findings of Moelants, Demey & Leman, (2009), who, in a similar study with 10 subjects, reported no differences in the amount of movement between the normal and the silent condition in all the four pieces played, and a significant increase of the head movement in the delayed condition in three out of four pieces. The importance of the individual inclinations, pointed out in this study by the ANOVA, may account for these different outcomes. The results concerning the periodicity of

84

the head pitch (Paragraph 5.4) seem to confirm the great variability in individual behaviours. However, although the dataset size does not permit to draw general conclusions in this regard, the periodic movement of the players seems to be strongly influenced by DAF. First, a tendency to reduce the periodicity components linked to the musical structure seems to emerge. This fact can be interpreted as a disrupted effect of DAF on body movement, in so far some subjects (5, 7, 8, and 13, cf. Table 5.12) manifested such a tendency despite a deliberate attempt to use the body as a reference. The second noticeable tendency is that of relating more on high-frequencies than on low, comparing with the normal condition. Two explanations may be taken into account for these results. The first explanation relies upon the observation that, under DAF, the expressiveness of the playing is highly impaired, as seen in the previous paragraph. In the DAF condition, most players found it difficult, if not even impossible, to express the emotional contents of the music. In the “Sarabande”, as seen in Paragraph 3.2, the musical sentences are based on sub-phrases of 2 measures each, which may explain why, in the normal condition, players often moved with periodicities of this length. In the DAF condition, on the contrary, this natural link between body movement and expressivity seems to fail, resulting in a diminution in the weights of the low-frequency periodicities. The second explanation relies upon the fact that, under DAF, 8 subjects out of 15 explicitly tried to use their body as a reference (cf. Table 5.12), in order to contrast its disruptive effect on timing. This embodied strategy may therefore account for the relative increase in the weight of the highfrequency periodicities, more related to the beats. Noteworthingly, this embodied strategy to cope with DAF was adopted by some subjects (2, 10, 11) who did not report it as a conscious deliberation. Lastly, we focus on the behaviour of subject 12, who, as we saw in Subparagraph 5.4.3, explicitly tried to keep the correct tempo moving regularly his head at every beat (see Figure A1.22). This fact may account for two results underlined before: his drastically increased IoM in the delayed condition (cf. Figure 5.13), and the fact that he is the only subject who played faster in the delayed condition (cf. Figure 5.10). In conclusion, DAF seems to influence the periodicity of body movement both in relation to its disruptive effect on the expressivity and because it triggers embodied responses, conscious or not, that are functional in contrasting its disruptive effects on timing.

85

6.5 Conclusions Embodied strategies, such as using the head movement as a reference, seem to have a strong potential for alleviating timing disruption caused by DAF. The relatively good results obtained by some subjects who adopted such strategies provide clues in this regard. Also experience with DAF may contribute to reduce its negative effects on music performance, as the results of the three long-lasting experienced organists seem to suggest. For what concerns the hypothesis that musicians would accentuate the feedback modalities which are not altered, this is shown to be true for the tactile feedback, whereas individual differences prevent from drawing general conclusions about body movement. In general, this study confirms that music playing is a matter of direct involvement with music, in the sense of “corporeal immersion in sound energy, which is a direct way of feeling musical reality” (Leman, 2007, p. 4). Indeed, the lack of congruency between action and perception provoked by DAF impedes the experience of behavioural resonance with the physical energies in the environment, fundamental to permit the players to convey expressive intentions through music. As a consequence, musical performance under DAF results to be impaired in all its expressive parameters. On the contrary, the absence of auditory feedback does not significantly impair performance. In this case, in fact, no misleading information is received through sensory feedback, and the auditory imagery may replace the missing information basing on previously built up statistics. Also in the absence of feedback, however, the normal behavioural resonance with the external energies is impeded: this fact results in some negative effects of auditory feedback deprivation on the expressive nuances of the performance (Repp, 1999). According to the embodied music cognition, the mediator between the physical energies in the environment and the inner space of the players is body movement. Therefore, it is not surprising that changes in the auditory feedback condition reflect on the players’ movement. In the case of DAF, the incongruence between action and perception introduced by the auditory feedback provokes embodied responses that seem to differ according to individual variability. Further investigations should attempt to clarify these relationships. In the case of auditory feedback deprivation, on the other hand, the lack of sound waves causes a significant decrease in the intensity of

86

movement. Once again, this fact may be explained with the impossibility of a normal resonance with the energies in the environment.

87

CHAPTER 7 – REFERENCE BIBLIOGRAPHY

Allen, D. R. (2007). Mental representations in clarinet performance: connections between auditory imagery and motor behaviours. Master’s thesis, University of North Carolina, Greensboro, NC.

Banton, L. J. (1995). The role of visual and auditory feedback during the sightreading of music. Psychology of Music, 23, 3-16.

Barbosa, A. (2003). Displaced Soundscapes: A Survey of Network Systems for Music and Sonic Art Creation. Leonardo Music Journal, 13, 53-59.

Bartlette, C., Headlam, D., Bocko, M., & Velikic, G. (2006). Effect of Network Latency on Interactive Musical Performance. Music Perception, 24, 49-62.

Bernstein, N. (1967) The Co-ordination and Regulation of Movements. New York: Pergamon.

Black, J. W. (1951). The effect of delayed side-tone upon vocal rate and intensity. Journal of Speech and Hearing Disorders, 16, 56-60.

Bortoli, L., & Robazza, C. (1990). Apprendimento motorio: concetti e applicazioni. Rome: Edizioni Luigi Pozzi.

Bowmann, J. P., & Combs, C. M. (1969). Cerebellar responsiveness to stimulation of the lingual spindle afferent fibers in the hypoglossal nerve of the rhesus monkey. Experimental Neurology, 23, 537-543.

Bradshaw, J. L., Nettleton, N. C., & Geffen, G. (1971). Ear differences and delayed auditory feedback: Effects on a speech and a music task. Journal of Experimental Psychology, 91, 85-92.

88

Brunswik, E. (1956). Perception and the Representative Design of Psychological Experiments. Berkeley: University of California Press.

Burrows, D., ed. (1997). The Cambridge Companion to Handel. New York: Cambridge University Press.

Card, S., Moran, T., & Newell, A. (1983). The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.

Chafe, C., Gurevich, M., Leslie, G., & Tyan, S. (2004). Effect of Time Delay on Ensemble Accuracy. In Proceedings of the International Symposium on Musical Acoustics. Nara, Japan.

Chew, E., Zimmermann, R., Sawchuk, A. A., Kyriakakis, C., Papadopoulos, C., François, A. R. J., Kim, G., Rizzo, A., & Volk, A. (2004). Musical Interaction at a Distance: Distributed Immersive Performance. In Proceedings of the MusicNetwork Fourth Open Workshop on Integration of Music in Multimedia Applications, pp. 15-16. Los Angeles, CA.

Cohen, B., Goto, K., Shanzer, S., & Weiss, A. H. (1965) Eye movements induced by electrical stimulation of the cerebellum in the alert cat. Experimental Neurology, 13, 145-162.

Dahl, S. (2005). On the beat: Human movement and timing in the production and perception of music. Ph.D. dissertation, KTH, Royal Institute of Technology, Stockholm, Sweden.

Dahl, S., & Bresin, R. (2001). Is the player more influenced by the auditory than the tactile feedback from the instrument? In Proceedings of the COST G-6 Conference on Digital Audio Effects, pp. 194-197, Limerick, Ireland.

89

Ericsson, K. A., Krampe, R. Th., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100, 363-406.

Fairbanks, G., & Guttman, N. (1958). Effects of delayed auditory feedback upon articulation. Journal of Speech and Hearing Research, 1, 333-346.

Finney, S. A. (1997). Auditory feedback and musical keyboard performance. Music Perception, 15, 153-174.

Finney, S. A. (1999). Disruptive effects of delayed auditory feedback on motor sequencing. Doctoral dissertation, Brown University, Providence, RI.

Finney, S. A. & Palmer, C. (2003). Auditory feedback and memory for music performance: Sound evidence for an encoding effect. Memory & Cognition, 31, 51-64.

Finney, S. A., & Warren, W. H. (2002). Delayed auditory feedback and rhythmic tapping: Evidence for a critical interval shift. Perception & Psychophysics, 64, 896-908.

Föllmer, G. (2002). Musikmachen im Netz Elektronische, ästhetische und soziale Strukturen einer partizipativen Musik. Ph.D. thesis, Martin-Luther-Universität Halle-Wittenberg, Germany.

Freed, A., Chaudhary, A., & Davila, B. (1997). Operating Systems Latency Measurement and Analysis for Sound Synthesis and Processing Applications. In Proceedings of the International Computer Music Conference. Thessaloniki, Hellas: ICMA.

Fuchs, A., & Kornhuber, H. (1969). Extraocular muscle afferents to the cerebellum of the cat. Journal of Physiology, 200, 713-722.

90

Gates, A., & Bradshaw, J. L. (1974). Effects of auditory feedback on a musical performance task. Perception and Psychophysics, 16, 105-109.

Gates, A., Bradshaw, J. L., & Nettleton, N. (1974). Effect of different delayed auditoryfeedback intervals on a music performance task. Perception and Psychophysics, 14, 21-25.

Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Boston: Houghton Mifflin.

Goebl, W., & Palmer, C. (2008). Tactile feedback and timing accuracy in piano performance. Experimental Brain Research, 186, 471-479.

Hämäläinen, P., Mäki-Patola, T., Pulkki, V., & Airas, M. (2004). Musical computer games played by singing. In G. Evangelista & I. Testa (eds), Proceedings of the Seventh International Conference on Digital Audio Effects. Naples, Italy.

Haslinger, B., Erhard, P., Altenmuller, E., Schroeder, U., Boecker, H., and CeballosBaumann, A.O. (2005). Transmodal sensorimotor networks during action observation in professional pianists. Journal of Cognitive Neuroscience, 17, 282293.

Havlicek, L. L. (1968). Effects of delayed auditory feedback on musical performance. Journal of Research in Music Education, 16, 308-318.

Held, R., & Hein, A. (1963). Movement-produced stimulation in the development of visually guided behaviour. Journal of Comparative & Physiological Psychology, 56, 872-876.

Hickok, G., Buchsbaum, B., Humphries, C., & Muftuler, T. (2003). Auditory-motor interaction revealed by fMRI: speech, music, and working memory in area Spt. Journal of Cognitive Neuroscience, 15, 673-682.

91

Highben, Z., & Palmer, C. (2004). Effects of auditory and motor mental practice in memorized piano performance. Bulletin of the Council for Research in Music Education, 159, 58–65.

Hommel, B., Müsseler, J., Aschersleben, G. & Prinz, W. (2001). The theory of event coding (TEC): A framework for perception and action planning. Behavioral and Brain Sciences, 24, 849-937.

Howe, M. J. A., Davidson, J. W., & Sloboda, J. A. (1998). Innate gifts and talents: Reality or myth? Behavioural & Brain Sciences, 24, 849-937.

Howell, P. (2004). Effects of delayed auditory feedback and frequency-shifted feedback on speech control and some potentials for future development of prosthetic aids for stammering. Stammering Research, 1, 34-43.

James, W. (1890). The principles of Psychology. New York: Holt.

Janer, J. (2008). Singing-driven Interfaces for Sound Synthesizers. PhD thesis, Universitat Pompeu Fabra, Barcelona, Spain.

Jones, M. R. (1976). Time, our lost dimension: Toward a new theory of perception, attention, and memory. Psychological Review, 83, 323-355.

Kalmus, H., Denes, F., & Fry, D. B. (1955). Effect of delayed acoustic feed-back on some non-vocal activities. Nature, 175, 1078.

Kapur, A., Wang, G. E., Davidson, P., & Cook, P. R. (2005). Interactive Network Performance: a dream worth dreaming? Organised Sound, 10, 209-219.

Keele, S. W. (1968). Movement control in skilled motor performance. Psychological Bulletin, 70, 387-403.

92

Keele, S. W., & Summers, J. J. (1976). The structure of motor programs. In G. E. Stelmach (Ed.), Motor Control: Issues and Trends (pp. 109-142). New York: Academic.

Keller, P. E., & Janata, P. (2009). Review of “Embodied Music Cognition and Mediation Technology” by Marc Leman. Music Perception, 26, 289-292.

Krampe, R. T., Mayr,U., & Kliegl, R. (2005). Timing, sequencing, and executive control in repetitive movement production. Journal of Experimental Psychology: Human Perception and Performance, 31, 379-397.

Kristeva, R., Chakarov, V., Schulte-Mönting, J., & Spreer, J. (2003). Activation of cortical areas in music execution and imagining: A highresolution EEG study. NeuroImag, 20, 1872-83.

Langheim, F. J. P., Callicott, J. H., Mattey, V. S., Duyn, J. H., & Weinberger, D. R. (2002). Cortical systems associated with covert musical rehearsal. NeuroImage, 16, 901-908.

Lashley, K. S. (1951). The problem of serial order in behaviour. In L. A. Jeffress (Ed.), Cerebral Mechanism in Behaviour: The Hixon Symposium (pp. 112136). New York: Wiley.

Lee, B. S. (1950). Effects of delayed speech feedback. Journal of the Acoustical Society of America, 22, 824-826.

Leman, M. (2007). Embodied Music Cognition and Mediation Technology. Cambridge, MA: MIT Press.

Lotze, M., Scheler, G., Tan, H. R. M., Braun, C., & Birbaumer, N. (2003). The musician’s brain: Functional imaging of amateurs and professionals during performance and imagery. Neuroimage, 20, 1817-29.

93

MacKay, D. G. (1987). The organization of perception and action. New York: Springer-Verlag.

Mäki-Patola, T. (2005). Musical Effects of Latency. Suomen Musiikintutkijoiden, 9, 82-85.

Mäki-Patola, T., & Hämäläinen, P. (2004). Latency Tolerance for Gesture Controlled Continuous Sound Instrument without Tactile Feedback. Proceedings of the International Computer Music Conference. Miami, FL.

Mereu, S. W. (1995). Improving Depth Perception in 3D Interfaces with Sound. Master’s thesis, University of Waterloo, Ontario, Canada.

Michels, U. (1994). Atlante di Musica. Milan: Sperling & Kupfer.

Moelants, D., Demey, M., & Leman, M. (2009). Performing music with delayed auditory feedback. In Proceedings of the 2009 European Society for the Cognitive Sciences of Music Conference (ESCOM). Jyväskylä, Finland.

Nicoletti, R. (1992). Il controllo motorio. Bologna: Il Mulino.

Palmer, C. (1997). Music Performance. Annual review of psychology, 48, 115-138.

Pfordresher, P.Q. (2003). Auditory feedback in music performance: Evidence for a dissociation of sequencing and timing. Journal of Experimental Psychology: Human Perception and Performance, 29, 949-964.

Pfordersher, P. Q. (2004). Can altered auditory information affect planning? Evidence from music performance. Stammering Research, 1, 51-53.

Pfordresher, P. Q. (2005). Auditory feedback in music performance: The role of melodic structure and musical skill. Journal of Experimental Psychology: Human Perception and Performance, 31, 1331-1345.

94

Pfordresher, P. Q. (2006). Coordination of perception and action in music performance. Advances in Cognitive Psychology, 2, 183-198.

Pfordresher, P. Q., & Benitez, B. (2007). Temporal coordination between actions and sound during sequence production. Human Movement Science, 26, 742-756.

Pfordresher, P. Q., & Palmer, C. (2002). Effects of delayed auditory feedback on timing of music performance. Psychological Research, 66, 71-79.

Pfordresher, P. Q., & Palmer, C. (2006). Effects of hearing the past, present, or future during music performance. Perception & Psychophysics, 68, 362-376.

Pöppel, E. (1997). A hierarchical model of temporal perception. Trends in Cognitive Sciences, 1, 56-61.

Rasch, R. A. (1979). Synchronization in performed ensemble music. Acustica, 43, 121-131.

Rasch, R. A. (1988). Timing and synchronization in ensemble performance. In J. Sloboda (Ed.), Generative Processes in Music (pp. 70-90). Oxford: Clarendon Press.

Repp, B. H. (1999). Effects of auditory feedback deprivation on expressive piano performance. Music Perception, 16, 409-438.

Rizzolatti, G., Fogassi, L., & Gallese, V. (2001). Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Reviews Neuroscience, 2, 661-670.

Robinson, G. M. (1972). The delayed auditory feedback effect is a function of speech rate. Journal of Experimental Psychology, 95, 1-5.

95

Rumelhart, D. E., & Norman, D. A. (1982). Simulating a skilled typist: A study of skilled cognitive-motor performance. Cognitive Science, 6, 1-36.

Sawchuk, A. A., Chew, E., Zimmermann, R., Papadopoulos, C., & Kyriakakis, C. (2003). From remote media immersion to distributed immersive performance. Proceedings ACM SIGMM Workshop on Experiential Telepresence (ETP). Berkeley, CA.

Schmidt, R. A., & Lee, T. D. (1999). Motor control and learning: a behavioral emphasis (3rd ed.). Champaign, IL: Human Kinetics.

Sheft, S. (2008). Envelope Processing and Sound-Source Perception. In W. A. Yost, R. R. Fay, & A. N. Popper (Eds), Auditory Perception of Sound Sources (pp. 233-279). New York: Springer.

Sloboda, J. A. (1982). Music performance. In D. Deutsch (Ed.), The psychology of music (pp. 479–496). New York: Academic Press.

Smith, A. D. (2001). WAI-KNOT (Wireless Audio Interactive Knot). Master’s thesis, MIT Media Laboratory, Cambridge, MA.

Thompson, W. F., Dalla Bella, S., & Keller, P. E. (2006). Music Performance. Advances in Cognitive Psychology, 2, 99-102.

Varela, F. J., Thompson, E., & Rosch, E. (1991). The Embodied Mind. Cambridge, MA: MIT Press.

Weinberg, G. (2002). Interconnected Musical Networks: Bringing Expression and Thoughtfulness to Collaborative Music Making. Ph.D. dissertation. MIT Media Laboratory, Cambridge, MA.

96

97

APPENDIX 1 – INDIVIDUAL SUBJECTS’ GRAPHICS

In next pages, all the graphics concerning the experimental results of the individual subjects are reported.

98

A1.1 Subject 2

2 - Tempo (bpm) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

2 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

2 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

2 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.1 Subject 2: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

99

Figure A1.2 Subject 2: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

100

A1.2 Subject 3

3 - Tempo (bpm) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

3 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

3 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

3 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.3 Subject 3: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

101

Figure A1.4 Subject 3: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

102

A1.3 Subject 4

4 - Tempo (bpm) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

4 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

4 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

4 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.5 Subject 4: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

103

Figure A1.6 Subject 4: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

104

A1.4 Subject 5

5 - Tempo (bmp) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

5 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

5 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

5 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.7 Subject 5: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

105

Figure A1.8 Subject 5: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

106

A1.5 Subject 6

6 - Tempo (bpm) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

6 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

6 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

6 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.9 Subject 6: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

107

Figure A1.10 Subject 6: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

108

A1.6 Subject 7

7 - Tempo (bpm) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

7 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

7 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

7 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.11 Subject 7: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

109

Figure A1.12 Subject 7: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

110

A1.7 Subject 8

8 - Tempo (bpm) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

8 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

8 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

8 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.13 Subject 8: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

111

Figure A1.14 Subject 8: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

112

A1.8 Subject 9

9 - Tempo (bpm) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

9 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

9 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

9 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.15 Subject 9: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

113

Figure A1.16 Subject 9: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

114

A1.9 Subject 10

10 - Tempo (bpm) per Measure 80 70 60

Normal

50

No Audio Delay

40 30 20 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

10 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

10 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

10 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.17 Subject 10: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

115

Figure A1.18 Subject 10: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

116

A1.10 Subject 11

11 - Tempo (bpm) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

11 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3 4

5

6 7

8

9 10 11 12 13 14 15 16

11 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

11 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.19 Subject 11: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

117

Figure A1.20 Subject 11: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

118

A1.11 Subject 12

12 - Tempo (bmp) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

12 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3 4

5

6 7

8

9 10 11 12 13 14 15 16

12 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

12 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.21 Subject 12: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

119

Figure A1.22 Subject 12: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

120

A1.12 Subject 13

13 - Tempo (bpm) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

13 - Average MIDI Key Velocity per Measure 90 80 70 60

Normal

50

No Audio

40

Delay

30 20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

13 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

13 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.23 Subject 13: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

121

Figure A1.24 Subject 13: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

122

A1.13 Subject 14

14 - Tempo (bpm) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

14 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

14 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

14 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.25 Subject 14: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

123

Figure A1.26 Subject 14: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

124

A1.14 Subject 15

15 - Tempo (bpm) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

15 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

15 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

15 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.27 Subject 15: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

125

Figure A1.28 Subject 15: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

126

A1.15 Subject 16

16 - Tempo (bpm) per Measure 80 70 60 50

Normal

40

No Audio

30

Delay

20 10 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

16 - Average MIDI Key Velocity per Measure 100 90 80 70 60 50 40 30 20 10 0

Normal No Audio Delay

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

16 - Average Head IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

16 - Average Chest IoM (m/s^3) per Measure 0,9 0,8 0,7 0,6

Normal

0,5

No Audio

0,4

Delay

0,3 0,2 0,1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

Figure A1.29 Subject 16: tempo, average velocity, head and chest intensity of movement in the three experimental conditions.

127

Figure A1.30 Subject 16: head pitch and normalized FFT of the 4-zero-padded resampled head pitch in the three experimental conditions.

128

129

APPENDIX 2 – MEASUREMENT OF THE DELAY TIMES OF THE CHURCH ORGAN OF ST. ANNA IN GHENT

In this section, I will present a report on an experiment that took place in St. Anna church in Ghent, Belgium. Aim of the experiment was to measure the temporal intervals that elapse between the keystrokes and the feedback perception when playing the organ of the church (Figure A2.1).

Figure A2.1 Prospect of the St. Anna church organ.

130

The peculiarity of the St. Anna organ is the tubular-pneumatic key action (here the term action means system of moving part; key action is the system which allows the wind to blow into a pipe): the opening of the pipe valves is due to pressure changes caused by the moving of small puffs of air through the tiny tubes which link the keys to the windchests. The mechanism is well described by Figure A2.2 and Figure A2.3, that show the effect of a keystroke on the air inside the organ. When the key is pressed, a puff of air moves into the tube, reaching the end close to the pipes, where the tube is mechanically connected to the back pressure channel situated below the pipe toe-holes and the stop channels (see Figure A2.2). The back pressure channel, full of compressed air, is separated from the stop channels and the pipe toe-holes by a membrane which is kept pressed on by the air. The puff pressure in the tube causes the back pressure channel to deflate, so that the membranes under the pipes deflate too (Figure A2.3). The compressed air in the stop channels is therefore allowed to flow into the back pressure channel, and consequently into the pipes, producing the sound.

Figure A2.2 A membrane drawer of the organ with the key action in rest position (adapted from Steenbrugge, 2005).

131

Figure A2.3 A membrane drawer of the organ when the key is pressed (adapted from Steenbrugge, 2005).

The tubular-pneumatic action is a major source of delay in sound production. An important role, in particular, is played by the transmission of the key pressure through the tubes. Steenbrugge (personal communication) estimated the time needed for the tubular transmission in the St. Anna organ at approximately 30 ms. This estimate is based on the fact that the tubes are approximately 10 metres long, a distance that pressure waves cover in about 30 ms, considering a velocity of propagation in the air at normal temperature of 340 m/s. A longer transmission time is needed for a group of stops, such as the Principal 8’, whose pipes are positioned some metres farther than the others from the keyboards. Other amounts of delay are due to the time needed for the deflation of the back pressure channel and the membranes. Lastly, an amount of delay should be proportional to the pipes dimensions. The experiment consisted in the measurement of the time that elapses between keystrokes and sound perception, for three different keys (a low, a middle, and a highfrequency note) and seven different stop combinations.

132

A2.1 Method

Essentially, measurement of the delays was performed calculating the difference between the moment in which the keys were pressed and the onset of the corresponding notes. A contact microphone was positioned over the key to be tested (see Figure A2.4), to record the noise due to the keystrokes, whereas a Shure microphone was positioned close to the organ pipes (see Figure A2.5), to record the produced sound. The positioning of the Shure microphone close to the pipes, instead of close to the console, was chosen to facilitate the signal analysis: the consequent introduction of an additional amount of delay will be neglected in this study, due to considerations about its order of magnitude. The two microphones were synchronized and recorded via a Max/MSP patch.

Figure A2.4 A contact microphone is positioned over the key to be tested, to record the noise due to the keystrokes.

133

Figure A2.5 A Shure microphone is positioned close to the pipes, to record the organ sound.

For what concerns the keystrokes noise, I took as production moments the onsets of the waveforms recorded by the contact microphone (Figure A2.6), which correspond to the start of the keystrokes. Considering the fact that the pneumatic transmission pulse starts as soon as a key is approximately 1 mm down (Steenbrugge, personal communication), that is when the key has covered about 15% of its trajectory, taking the onsets as the start instants introduces a certain error in the measurement. To minimize such error, keypressing movements were executed quickly. On average, keystroke movement from the upper position to the lower one lasts around 10 ms: the resulting error is so therefore approximately around 1-2 ms for each measurement. For what concerns the sound of the organ recorded by the Shure microphone, the moments of sound production were extracted via onset detection on the corresponding waveforms (Figure A2.7). Onset detection was performed manually with the help of Sonic Visualiser and Audacity software.

134

Figure A2.6 Waveform of the noise generated by a keystroke. The vertical marker indicates the onset moment.

Figure A2.7 Waveform of a Bourdon 8’ g3 recording. The vertical marker indicates the onset moment.

Tested keys were the lowest pitch key of the keyboard, a C, the higher pitch one, a G, and the middle C. I will refer to these keys with the notation adopted in Steenbrugge (2005): C for the lowest C, c1 for the middle C, and g3 for the higher G. The pitch of the corresponding notes depends on the kind of stop: for 8’ stops, C, c1 and g3 keys correspond respectively to C2, C4 and G6 notes in the scientific pitch notation; for 4’ stops, which sound an octave above, they correspond to C3, C5 and G7. The keys no-

135

tation and their relative pitches for 8’ and 4’ stops are presented schematically in Table A2.1 and A2.2, together with the theoretical open-pipe lengths (calculated with the formula length  172 frequency , where frequency refers to the fundamental frequency; Steenbrugge, personal communication).

Key

Pitch (scientific notation)

Frequency (Hz)

Theoretical pipe length (m)

C

C2

65.406

2.630

c1

C4

261.63

0.657

g3

G6

1568.0

0.110

Table A2.1 Notation, pitch, fundamental frequency and theoretical open-pipe length of the tested keys in the case of 8’ stops.

Key

Pitch (scientific notation)

Frequency (Hz)

Theoretical pipe length (m)

C

C3

130.81

1.315

c1

C5

523.25

0.329

g3

G7

3136.0

0.055

Table A2.2 Notation, pitch, fundamental frequency and theoretical open-pipe length of the tested keys in the case of 4’ stops.

Tested stops were the single stops Montre 8’, Bourdon 8’, Gamba 8’, Prestant 4’, and Principal 8’, plus the two stop combinations Montre 8’ plus Prestant 4’ and Montre 8’ plus Prestant 4’ plus Gamba 8’ plus Bourdon 8’. In total, the different stop combinations tested were 7. For each key-stop combination, three delay measurements were performed, giving a total of 63 measurements (3 measurements  3 keys  7 stop combinations).

A2.2 Results

Results are reported in Table A2.3 and graphically shown in Figure A2.8, Figure A2.9, and Figure A2.10. Figure A2.8 shows average values and standard deviations

136

of the delay measurements for the 5 single stops, whereas Figures A2.9 and A2.10 show a comparison between the average values and the standard deviations for the two stop combinations and the single stops composing them.

Stops

Average Delay per key

St. Deviation per key

C

C

c1

g3

c1

G3

Montre 8’

0,141

0,106

0,085

0,004

0,005

0,005

Prestant 4’

0,152

0,105

0,065

0,004

0,004

0,003

Principal 8’

0,139

0,134

0,132

0,004

0,004

0,004

Bourdon 8’

0,145

0,101

0,080

0,005

0,004

0,005

Gamba 8’

0,124

0,102

0,065

0,006

0,004

0,004

Montre+Prestant

0,147

0,112

0,067

0,011

0,006

0,004

Montre+Prestant+Gamba+Bourdon

0,147

0,104

0,092

0,006

0,010

0,006

Table A2.3 Average values and standard deviations of all the delay measurements.

Figure A2.8 Mean values and standard deviations of the delay measurements for the 5 tested single stops. Each error bar is two standard deviations long.

137

Figure A2.9 Mean values and standard deviations of the delay measurements for the Montre, Prestant, and Montre plus Prestant stops. Each error bar is two standard deviations long.

Figure A2.10 Mean values and standard deviations of the delay measurements for the Montre, Prestant, Bourdon, Gamba, and Montre plus Prestant plus Bourdon plus Gamba stops. Each error bar is two standard deviations long.

138

A2.3 Discussion

Results show that delay values vary in the range of 60-160 ms, with higher values for the low-frequency key (C) and lower values for the high-frequency key (g3). The investigation regarding the precise nature of the dependency of the delay on the key and the stop is beyond the scope of this experiment. Indeed, in order to clarify this relationships, more keys should have been studied, as well as more information about the pneumatic mechanisms and the pipes dimensions collected. However, some observations can be made. First, a dependency of the delay on the played keys is evident for all stops but the Principal 8’, for which the differences between the keys are very small. Second, a dependency of the delay on the different stops is also clear. In particular, considering the single stops, Principal 8’ differs from the other stops in the high-frequency and in the middle-frequency keys, in which case it is affected by delays around 135 and 130 ms, whereas the other stops show delays inferior to 115 ms (Montre 8’ and Bourdon 8’) or 90 ms (Prestant 4’ and Gamba 8’). Gamba 8’ presents a shorter delay than the other stops in the low-frequency key, while for the highfrequency key Gamba 8’ and Prestant 4’ both present values lower than the others. Montre 8’ and Bourdon 8’ are the only two stops that show a very similar behaviour over all the three keys. For what concerns the stop combinations, somehow unexpected is the fact that the two combinations show opposite behaviours: delay values for the Montre plus Prestant combination are similar to the lower between the values of the single stops composing it, whereas delay values for the Montre plus Prestant plus Bourdon plus Gamba are similar to the higher. The third observation is that a constant amount of delay seems to be present over all the measurements, arguably due to the transmission time. Again, an investigation of the reasons underlying the dependency of the delay on keys and stops is beyond the scope of the present study. In general, a kind of proportionality between the delay and the diameter of the pipes (see Figure A2.11) should be expected. This relation, anyway, should be not straightforward, as the measurements seem to confirm. In particular, a striking difference has been registered between the delay values of the Montre 8’ and the Principal 8’, which have almost identical pipe dimensions. A partial explanation for this difference lies in the fact that the Principal pipes are positioned farther than the other pipes, therefore they need a

139

longer transmission time. Nevertheless, this fact does not account for the apparently very different asymptotical behaviours.

Figure A2.11 External diameters of the pipes for Montre 8’, Prestant 4’, Principal 8’ and Gamba 8’ (values from Steenbrugge, 2005).

A2.4 Conclusions

The experiment shows that the amounts of delays in the St. Anna pneumatic organ are prominently high. Referring to the discussion in Subparagraph 1.3.2, all measured delays are beyond the break-point interval, individuated as the delay length for which auditory feedback starts to be perceived as delayed and DAF disruption becomes significant. In particular, delay values for the low-frequency notes are very close to the 170-200 ms window which was found to be very damaging in many studies on music performance (Havlicek, 1968; Bradshaw, Nettleton & Geffen, 1971; Gates & Bradshaw, 1974; Pfordresher & Benitez, 2007; Moelants, Demey & Leman, 2009), and often considered as the critical interval in speech (Black, 1951; Fairbanks, 1955; Butler & Galloway, 1957; Fairbanks & Guttman, 1958; MacKay, 1968; Howell, Powell, & Khan, 1983; Fabbro & Darò, 1995). Moreover, when playing the organ, two more factors contribute to make performance very difficult. First, attack

140

times have also to be taken into account: for low-frequency notes, attack phase can last more than 100 ms (e.g., for the Bourdon 8’ stop), so that the overall time needed for a clear pitch recognition may reach 250 ms or more. Second, the strong reverberation of the sound waves in the church further complicate the relationships between played notes and their perception. Playing St. Anna’s organ therefore constitutes a very challenging task, at least for players who don’t have experience under analogous conditions, and pneumatic organs such as St. Anna’s one can be taken into consideration as appropriate instruments for ecological experiments on music performance under DAF.

A2.5 Reference bibliography Bello, J. L., Daudet, L., Abdallah, S. A., Duxbury, C., Davies, M. E., & Sandler, M. B. (2005). A Tutorial on Onset Detection in Music Signals. IEEE Transactions on Speech and Audio Processing, 13, 1035-1047.

Black, J. W. (1951). The effect of delayed side-tone upon vocal rate and intensity. Journal of Speech and Hearing Disorders, 16, 56-60.

Bradshaw, J. L., Nettleton, N. C., & Geffen, G. (1971). Ear differences and delayed auditory feedback: Effects on a speech and a music task. Journal of Experimental Psychology, 91, 85-92.

Butler, R. A., & Galloway, F. T. (1957). Factoral analysis of the delayed speech feedback phenomenon. Journal of the Acoustical Society of America, 29, 632635.

Fabbro, F., & Darò, V. (1995). Delayed auditory feedback in polyglot simultaneous interpreters. Brain and Language, 48, 309-319.

Fairbanks, G. (1955). Selective vocal effects of delayed auditory feedback. Journal of Speech and Hearing Disorders, 20, 333-346.

141

Fairbanks, G., & Guttman, N. (1958). Effects of delayed auditory feedback upon articulation. Journal of Speech and Hearing Research, 1, 333-346.

Gates, A., & Bradshaw, J. L. (1974). Effects of auditory feedback on a musical performance task. Perception and Psychophysics, 16, 105-109.

Havlicek, L. L. (1968). Effects of delayed auditory feedback on musical performance. Journal of Research in Music Education, 16, 308-318.

Howell, P., Powell, D. J., & Khan, I. (1983). Amplitude contour of the delayed signal and interference in delayed auditory feedback tasks. Journal of Experimental Psychology: Human Perception and Performance, 9, 772-784.

MacKay, D. G. (1968). Metamorphosis of a critical interval: Age-linked changes in the delay in auditory feedback that produces maximal disruption of speech. Journal of the Acoustical Society of America, 43, 811-821.

Michels, U. (1994). Atlante di Musica. Milan: Sperling & Kupfer.

Moelants, D., Demey, M., & Leman, M. (2009). Performing music with delayed auditory feedback. In Proceedings of the 2009 European Society for the Cognitive Sciences of Music Conference (ESCOM). Jyväskylä, Finland.

Pfordresher, P. Q., & Benitez, B. (2007). Temporal coordination between actions and sound during sequence production. Human Movement Science, 26, 742-756.

Shannon, J. R. (2009). Understanding the Pipe Organ: A guide for Students, Teachers, and Lovers of the Instrument. Jefferson, NC: McFarland & Company.

Steenbrugge, D. (2005). Het Pierre Schyven orgel in de Gentse Sint-Annakerk. Orgelkunst, 111.

142