Northern Lights: Film And Media Studies Yearbook: Volume: 5 | Issue: 1

3 7–24

Introduction Arild Fetveit and Gitte Bang Stald Mixed media: from digital aesthetics towards general communication theory Klaus Bruhn Jensen

25–38

Remediation and the language of new media Jay David Bolter

39–56

Alan Kay’s universal media machine Lev Manovich

57–74

Convergence by means of globalized remediation Arild Fetveit

75–88

The website as unit of analysis? Bolter and Manovich revisited Niels Brügger

89–104

Gameplay as design: uses of computer players’ immaterial labour Adam Arvidsson and Kjetil Sandvik

105–118

On transdiegetic sounds in computer games Kristine Jørgensen

119–140

Power and personality: politicians on the World Wide Web Ib Bondebjerg

141–158

Online debate on digital aesthetics and communication Lev Manovich, Jay David Bolter, Klaus Bruhn Jensen, Arild Fetveit and Gitte Bang Stald

159

Film & Media Studies Yearbook 2007

intellect Journals | Film Studies

05

1

www.intellectbooks.com

intellect

9 771601 829000

Northern Lights cover.indd

Volume 5

Northern Lights

Contributors

ISSN 1601-829X

Volume Five

Volume 5 – 2007

Northern Lights: Film & Media Studies Yearbook | Volume Five

Northern Lights

ISSN 1601-829X

8/22/07

4:51:16 PM

Northern Lights.qxd

3/9/07

10:47 am

Page 1

Northern Lights: Film and Media Studies Yearbook Volume 5, 2007 Digital Aesthetics and Communication Northern Lights: Film and Media Studies Yearbook is a peer-reviewed international yearbook started in 2002 and dedicated to studies of film and media. Each yearbook is devoted to a specific theme. In addition, every volume may include articles on other topics as well as review articles. The yearbook wants to further interdisciplinary studies of media with a special emphasis on film, television and new media. Since the yearbook was founded in Scandinavia, the editors feel a special obligation towards Scandinavian and European perspectives. But in a global media world it is important to have a global perspective on media culture. The yearbook is therefore open to all relevant aspects of film and media culture: we want to publish articles of excellent quality that are worth reading and have direct relevance for both academics in the broad, interdisciplinary field of media studies in both humanities and social sciences and for students in that area. But we also want to appeal to a broader public interested in thorough and well-written articles on film and other media.

Editorial Board: Ib Bondebjerg, Editor-in-chief (Department of Media, Cognition and Communication, Section of Film and Media Studies, University of Copenhagen), e-mail: [email protected] Torben Kragh Grodal (Department of Media, Cognition and Communication, University of Copenhagen, Section of Film and Media Studies) Stig Hjarvard (Department of Media, Cognition and Communication, University of Copenhagen, Section of Film and Media Studies) Anne Jerslev (Department of Media, Cognition and Communication, University of Copenhagen, Section of Film and Media Studies) Gunhild Agger (Department of Communication, University of Aalborg) Jens Hoff (Department of Political Science, University of Copenhagen)

Journal Editors Ib Bondebjerg Department of Media, Cognition and Communication Section of Film and Media Studies University of Copenhagen Njalsgade 80, 2300 Copenhagen S Phone: +45 35328102 Fax: +45 35328110 Mobile: +4524421168 Email: [email protected] Web: www.mef.ku.dk

Guest Editors Arild Fetveit, Department of Media, Cognition and Communication, University of Copenhagen, Email: [email protected] Gitte Bang Stand, IT-university of Copenhagen, Email: [email protected]

Corresponding Editors: Daniel Biltereyst (Ghent University, Belgium), Edward Branigan (University of California – Santa Barbara, USA), Carol Clover (University of California – Berkeley, USA), John Corner (University of Liverpool, UK), John Ellis (Royal Holloway University of London, UK), Johan Fornäs (Linköping University, Sweden), Jostein Gripsrud (University of Bergen, Norway), Andrew Higson (University of East Anglia, UK), Mette Hjort (Lignan University, Hong Kong), Dina Iordanova (University of St Andrews, Scotland), Steve Jones (University of Illinois, USA), Sonia Livingstone (London School of Economics, UK), Ulrike Meinhof (University of Southampton, UK), Guliano Muscio (University of Palermo, Italy), Janet Murray (Georgia Institute of Technology, USA), Horace Newcomb (University of Georgia, USA), Roger Odin (l’Université de Paris 3., France), Dominique Pasquier (CEMS, France), Murray Smith (University of Kent, UK), Trine Syvertsen (University of Oslo, Norway), William Uricchio (MIT, USA), Lennart Weibull (Göteborg University, Sweden), Espen Aarseth (Copenhagen, Denmark), Eva Warth (University of Bochum, Germany)

Northern Lights is published once a year by Intellect, The Mill, Parnall Road, Bristol, BS16 1JG, UK. The current subscription rates are £30 (personal) and £140 (institutional). A postage charge of £8 is made for subscriptions outside of Europe. Enquiries and bookings for advertising should be addressed to: [email protected] © 2007 Intellect Ltd. Authorisation to photocopy items for internal or personal use or the internal or personal use of specific clients is granted by Intellect Ltd to libraries and other users registered with the Copyright Licensing Agency (CLA) in the UK or the Copyright Clearance Center (CCC) Transactional Reporting Service in the USA, provided that the base fee is paid directly to the relevant organisation.

ISSN 1601-829X

Printed and bound in Great Britain by 4edge Ltd. Hockley. www.4edge.co.uk

Northern Lights.qxd

3/9/07

10:47 am

Page 2

Notes for Contributors Editorial process All articles submitted for NL must be original works not published or considered for publication elsewhere. The journal is a refereed, international journal, and the editors and two anonymous referees will evaluate all articles submitted for the journal. Anonymity is also accorded to authors. Format Articles must not exceed 8000 words (50,000 characters, including space), including notes and references – but introduction, keywords, abstract not included. Author-name, Institutional affiliation, address, and e-mail of the author(s) on a separate title page only. Author-CV: On same page: short cv of author, max 150 words All articles should be made in Word. Font: Times New Roman size 12. Top of article: authors name in italics. Before article: short introduction, in italics, max. 75 words. Keywords – six words, or two-word phrases, that are at the core of what is being discussed. There is a serious reduction in an article’s ability to be searched for if the keywords are missing. Insert abstract after notes and references, in italics, max 150 words. Format specifications Headings, Paragraphs and sections Bold is used for title of article (bold, size 14). Bold is also used for headings (size 12) in the article. By sub-headings, use italics (size 12). If further level is needed, use normal (size 12). A new paragraph is indicated by a carriage return and one tabulator indent. A new section is indicated by two carriage returns (a blank line). Orthography The Journal follows standard British English. But standard American spelling may be used. Word language checking for UK-English or American can be used. Use ‘ize’ endings in stead of ‘ise’, when there is an option for that. References All references in the text should be according to the Harvard system, e.g., (Bordwell, 1989: 9). Book titles are italicized, with the main words capitalized. The titles of articles are placed in double quotation marks, with the main words capitalized, e.g., Gunning introduces these ideas in an article from 1983, “An Unseen Energy Swallows Space.” See also the sample references below. Works mentioned Titles of films, TV-program, literary works etc. must be italicized. Works like this must be followed by year. Original title in other language than English must be given, title in English after year in italics, if original title in English exists, otherwise translation to English in double quotation marks, e.g. Italiensk for begyndere (2000, Italian for Beginners), or Barnet (1940, “The Child”). Quotations NL’s style for quotations embedded into a paragraph is single quote marks, with double quote marks for a second quotation contained within the first. All long quotations (i.e. over four lines or 40 words long) should be ‘displayed’ – i.e. set into a separate indented paragraph with an additional one-line space above and below, and without quote marks at the beginning or end. Brief quotations within the main text are indicated by double quotation marks. Quotations of more than 50 words are treated as a separate section (blank line before and after, no quotation marks, no indent). ‘Scare quotes,’ highlighting or questioning the use of a term, are indicated by single quotation marks, also within an actual quotation, e.g: As Bordwell states, “To speak of ‘interpretation’ invites misunderstanding from the outset” (Bordwell 1989: 1).

Punctuation marks should always be placed within quotation marks. All omissions in a quotation are indicated thus: (...) Italics may be used (sparingly) to indicate key concepts. Images, Tables and Diagrams All illustrations, photographs, diagrams, maps, etc. should follow the same numerical sequence and be shown as Figure 1, Figure 2, etc. The source has to be indicated below. Copyright clearance should be indicated by the contributor and is always the responsibility of the contributor. When they are on a separate sheet or file, an indication must be given as to where they should be placed in the text. Reproduction will be in greyscale (sometimes referred to as ‘black-and-white’). If you are supplying any article images as hard copy, these should be prints between 10–20 cms wide if possible, and preferably greyscale if being submitted as illustrations for articles. However, colour prints, transparencies and small images can be submitted if you need to supply these. Photocopies are never advisable, but may be okay for diagrams. They are never acceptable for photographs. Line drawings, maps, diagrams, etc. should be crisp, clear and in a camera-ready state, capable of scanning and reduction. Although not ideal, slides are certainly acceptable. If images are supplied electronically, all images need to have a resolution of at least 12 dpm (dots per millimetre) – or 300 dpi (dots per inch). The figure showing the number of pixels across the width of the image, a figure independent of millimetres, centimetres or inches, is reached by multiplying the width of the image in millimetres required for reproduction in the journal by 12, or in inches by 300. This is the actual information available that allows the production team to offset resolution (dpm or dpi) against width. Images sent in as e-mail attachments should be greyscale to save time uploading and downloading. Tables should be supplied either within the Word document of the main text or as separate Word documents. These can then be extracted and reproduced. Reproducing text within images supplied separately is difficult: they need a high final resolution – around 48 dpm. An additional Acrobat PDF document is encouraged. The PDF is a good proof copy that can also be used for reproduction if the table is exactly as it should be, but if editing is necessary, this can be done in Word if there is a small spelling error or if a statistical error is identified later. Diagrams are difficult to construct in Word. Diagrams are best constructed in an object-oriented computer program rather than a text-oriented one. Diagrams can be supplied to us as JPEG, TIFF or Acrobat PDF documents. If a mistake is identified in a diagram, make the amendments and re-supply. Bullets and numbered lists NL prefer that you use bullet points when listing is necessary. If a numbered list is used they should be formatted as 1. 2. 3. Etc. Notes Notes may be used for comments and additional information only. Do not use footnotes for simple reference-purposes. Use the Word-program for footnotes, and please do not use endnotes. Notes should be used only in very special cases and only as footnotes. Footnotes must not exceed 30 words. Dates 21 March 1978 1970s, 1980s 1964–67; 1897–1901 nineteenth century, twentieth century, twenty-first century Numbers one to twenty (words); 21–99 (figures); 100, 200 thirty, forty, fifty (if expressed as an approximation) 15 years old 3 per cent, 4.7 per cent, 10 per cent, 25 per cent pp. 10–19, 19–21; 102–07, 347–49 16mm, 35mm

Abbreviations ibid., op. cit., Ph.D., BBC, UN, MA, PAR (practice as research) Foreign names Capitalized proper names of organizations, institutions, political parties, trade unions, etc. should be kept in roman type, not in italics. Specific Names Names of art exhibitions, film festivals, etc. should be in roman type enclosed in single quote marks. References All references are listed at the end of the article, alphabetically and beginning on a separate page. A blank line is entered between references. The reference list must follow the Harvard style of reference, more specifically the APA-standard (http://www.apastyle.org) that should comply with End Note and other electronic standard reference programs. The following samples indicate conventions for the most common types of reference: Anon (1931). Les films de la semaine. Tribune de Genéve, p. 15 (January 28). Cabrera, D. (1998a). Table Ronde de l’APA. La Faute á Rousseau: ‘Le secret’, 18 (1), pp. 28-29. Cabrera, D. (1998b). Une chambre á soi. Trafic, 26 (1), 28-35. Flitterman-Lewis, S. (1990). To Desire Differently: Feminism and the French Cinema. Urbana and Chicago: University of Chicago Press. Grande, M. (1998). Les Images non-dérivées. In Fahle, O (ed.), Le Cinéma selon Gilles Deleuze. Paris: Presse de la Sorbonne Nouvelle, pp. 284302. Gibson, R., Nixon, P. & Ward, S. (eds.) (2003). Political Parties and the Internet: Net Gain?. London: Routledge. Hayward, S. (1993). French National Cinema. 2nd edn. New York and Paris: Routledge. Hottel, R. (1999). Including Ourselves: The Role of Female Spectators in Agnés Varda’s ‘Le bonheur and L’une chante, l’autre pas. Cinema Journal, 38(2), 52-72. Roussel, R. (1996), Locus Solus, Paris: Gallimard. (Originally published 1914). Stroöter-Bender, J. (1995). L’Art contemporain dans les pays du ‘Tiers Monde’. (trans. O. Barlet). Paris: L’Harmattan. Mendoza, A. (1994). Las communicaciones en ingles y espanol [Communications in English and Spanish]. Madrid: Universidad de Madrid.? (So this is for titles in other languages you want to translate to English, and where an official English version doesn’t exist, list title in roman and in square brackets). Website references are similar to other references. There is no need to decipher any place of publication or a specific publisher, but the reference must have an author, and the author must be referenced Harvard-style within the text. Unlike paper references, however, web pages can change, so there needs to be a date of access as well as the full web reference. In the list of references at the end of your article, the item should read something like this: Bondebjerg. (2005). Web Communication and the Public Sphere in a European Perspective. At www.media.ku.dk, accessed February 15, 2005.

Northern Lights.qxd

3/9/07

10:47 am

Page 3

Northern Lights Volume 5 © 2007 Intellect Ltd Editorial. English language. doi: 10.1386/nl.5.1.3/2

Introduction Arild Fetveit and Gitte Bang Stald Media proliferate and migrate across new technological devices in an ongoing digital revolution. This involves changing aesthetics, refigured communication and distribution patterns, challenges to copyright holders, and a new surge for audience creativity and sociality. The development raises a number of questions for scholars as well as for media practitioners and cultural commentators. Are ‘new media’ new in a more fundamental way than previous media? To what extent and in which ways are media converging? What happens to other media when the computer is positioned as a metamedium, one that can handle and display most previous media? How does design and creativity develop in the game industry and to what extent is user-driven innovation becoming a factor in the assessment of productivity? Are players increasingly coming to produce the computer games they play, and in that case, how does this phenomenon relate to a new economic logic characteristic of Web 2.0? How are statesmen and -women dressing up their websites, and do these sites add to our democracies? And, to what extent do classics like Remediation: Understanding New Media (1999), and The Language of New Media (2001) still hold up, and how could they be updated to address current developments? This volume aims at addressing such questions, and it also hopes to raise even more that can be productive for further research in the field. But to what extent is there a field, and how should it be conceived? From early attempts to view ‘new’ and ‘digital’ media as entirely different from ‘old media’, new-media studies, to the extent that such a field exists, has come of age through being able to historicize and to see how earlier media forms in various ways revisit the new. Such an approach seems productive in a number of areas, whether it be the study of computer games, political discourse, websites, or other matters of interest in the realm of digital aesthetics and communication. One of the merits of an historicizing and comparative approach is also that it may gradually help overcome the tendency to treat ‘new media’ as a separate field of inquiry deserving its own special methods and theories secluded from other approaches employed in the study of aesthetics, culture, technology, and social life. Besides, as most media in some way are coming to employ digital technologies in some aspect of their production, distribution and reception processes, an erosion of the concepts ‘digital media’, and ‘new media’ is about to take place. This may create an opening for a further integration between established research fields and the field of ‘new media’. Two of the classics, mentioned earlier, of the study of ‘new media’, which both incidentally use this concept in their titles, are Remediation: Understanding New Media (1999), by Jay D. Bolter and Richard Grusin, and The Language of New Media (2001), by Lev Manovich. Both books, in

NL 5 3–6 © Intellect Ltd 2007

3

Northern Lights.qxd

1.

3/9/07

However, it is not always evident that the double logic of immediacy and hypermediacy is vital to the way remediation takes place. This suggests that the concept of remediation has wider relevance than what the terms immediacy and hypermediacy would initially suggest. According to their preface, the origin of the book was a seminar on multimediacy (which later became hypermediacy) offered by Grusin, in which Bolter visited discussing immediacy. The concept of remediation was then developed later. This decent explains the central position that immediacy and hypermediacy have been granted. In fact, a theory of remediation itself could work well also by allocating a less prominent role to these terms, although the distinction remains a powerful one, as well as the related distinction between looking at and looking through (see Bolter and Grusin 1999: iii, 41).

10:47 am

Page 4

their particular ways, take a historical and comparative approach, exploring continuities and relations between past and present rather than complete breaks. Thus, they position themselves against an early tendency to dismiss the importance of historical precedents for digital media in the interest of emphasizing their revolutionary newness. Bolter and Grusin, as well as Manovich, counter such tendencies in their own particular ways, and seek to position ‘new media’ in an aesthetic and cultural history of western visual culture spanning all the way back to the Renaissance, in the case of Bolter and Grusin, and with a major interest in how cinematic language is continued in digital media, in the case of Manovich. In the beginning of his book, Manovich (2001: xv) proposes an examination of the idea that ‘cinematic ways of seeing the world, of structuring time, of narrating a story, of linking one experience to the next, have become the basic means by which computer users access and interact with all cultural data’. Critics have generally taken this to indicate that Manovich has a skewed view of the importance of cinema, rather than realizing the potential usefulness of pushing such an idea for its explorative purposes, understanding that the cinematic paradigm is first of all prominent in the opening and the closing of the book, as Manovich points out in the discussion ending this volume. Bolter and Grusin pursue a comparative perspective, not by opting for one particular medium and exploring how it informs ‘new media’, but through historicizing mediation across any previous divide between the art world and the world of popular culture. Within this general perspective, they develop a tool for examining the ways in which media histories revisit new media in the concept of ‘remediation’. Remediation entails ‘the representation of one medium in another’, they claim, and is ‘a defining characteristic of the new digital media’ (Bolter and Grusin 1999: 45). Through this definition, the concept structurally sets up a comparative project of assessing ways in which the past revisits the present, well aware that this is a two-way street, so that, for example, televisual features not only reappear in various ways in websites but television itself also adopts elements from the newer media. Bolter and Grusin further define the concept of remediation as being based on a double logic of immediacy and hypermediacy. Where the former represents an effort to escape mediation and access reality or whatever is mediated directly, the latter represents an opposite logic. In this case, the experience of mediation is sought, and multiplied, in ways that bring attention to the process of mediation itself.1 Bolter and Grusin’s concept also involves remediation in the sense of repairing, improving, and making something better. The works of Bolter and Grusin, and that of Manovich, have been productive for the field, not only in offering concepts and insights, but also in raising issues in need of further consideration. The following volume is therefore conceived in part as a dialogue with these books, and with the work of Bolter and Manovich in general. By naming the volume ‘Digital Aesthetics and Communication’, by adding two concepts to that of the digital, the multiplicity of the field is acknowledged. But perhaps even more important, a possible tension between an aesthetic approach and an approach focusing on communication is suggested. Invoking such a tension, Klaus Bruhn Jensen opens this volume by offering an article in which he finds Remediation (Bolter and Grusin 1999), 4

Arild Fetveit and Gitte Bang Stald

Northern Lights.qxd

3/9/07

10:47 am

Page 5

and The Language of New Media (Manovich 2001) to represent a limiting tradition of ‘digital aesthetics’. He finds this tradition useful for researching the interface and assessing modes of representing reality. But, he argues, it fails to assess interactivity in terms of the different social meanings it generates, and to understand how digital media ‘enter into social interaction beyond the interface’. Bruhn Jensen proposes instead to place digital media within the larger framework of a general communication theory, which allows greater attention to the social uses of media. A key challenge for Bruhn Jensen’s paradigm, as for most paradigms addressing digital media, is to explain how digital media are different from earlier media. In order to address this challenge, he proposes to replace the distinction between unmediated and mediated communication with a more refined three-layer concept which distinguishes between media of first, second, and third degree. Jay David Bolter is also preoccupied with historicizing digital media, but from within a computer-science perspective informed by crossdisciplinary aesthetics. His contribution reviews the way in which the concept of remediation took shape, as well as how his and Grusin’s work relate to that of Manovich. In his current contribution, Manovich locates an effort to activate and use earlier medial forms directly in the stated aspirations of computer pioneers like Ivan Sutherland, Douglas Engelbart, Ted Nelson, and Alan Kay. In this context, Kay is the most interesting because he conceives of the computer as a metamedium capable of handling ‘already-existing and not-yet-invented media’. But Manovich also emphasizes how these media get new properties with new functionalities in their digital remediations. Arild Fetveit also seeks to address this addition of properties in his interrogation of media convergence. The manipulability of digital data has been taken to facilitate convergence and to effect an erosion of differences between media. Fetveit questions these assumptions by showing how there are considerable obstacles against such manipulability as well as against a convergence that will erase the differences between media, one of them being our affection for a variety of specific media aesthetics. He proposes a conception of convergence not so much as resulting from an erasure of differences among media, as resulting instead from an addition of new properties and affordances that tend to be similar. This tendency, he claims, promotes convergence by means of a globalized remediation. The character of digital media and how they develop can also be studied more specifically by looking at a single phenomenon like the website. This is what Niels Brügger offers in an article which reviews Bolter’s and Manovich’s contributions to understanding the website. He finds that an important tension in discussions of the website revolve around whether it is embedded in a coherent structure, or subject to fragmentation. He makes a case for coherence. One of the ways in which websites have developed over the last decade is by facilitating new forms of sociality and productive cooperation. Online games have been among the major vehicles to promote such developments. Adam Arvidsson and Kjetil Sandvik investigate game design fuelled by immaterial labour, and argue that the creative activity of computer players is being put to work by the game industry. Their argument feeds into larger issues concerning the way in which the Web is

NL 5 3–6 © Intellect Ltd 2007

5

Northern Lights.qxd

3/9/07

10:47 am

Page 6

developing, and the much talked about shift to a Web 2.0 logic, in which users become producers, exemplified by Wikipedia, and in part by YouTube, MySpace and other sites based on social software. In spite of these interesting developments in the social area, there is still a need to understand how digital media like games draw upon and employ elements of previously developed audio-visual language like that of the cinema, in new and different ways. Kristine Jørgensen explores the case of sound. She identifies what she calls a transdiegetic sound space in computer games, sounds that neither stem from a source within the story (diegetic), nor are quite external to the story (extradiegetic). These sounds may seem extradiegetic, but the fact that they communicate actively about the story events to the player who then comes to act upon them, allows Jørgensen to label them transdiegetic. The stakes in digital media are not merely aesthetic, social, economic or technological; they are also political. And this aspect is most explicitly brought out in Ib Bondebjerg’s analysis of websites serving state leaders. As politics and politicians are increasingly remediated onto the Web, it is important to explore how these new media are used, and what bearing they may have on the functioning of democracy. The many questions concerning digital aesthetics and communication prompted the editors to arrange a debate addressing challenges relating to Web 2.0, the issue of remediation, how ‘old media’ now tend to fill up ‘new media’, and how the concept of media itself is affected by the instalment of the computer as a metamedium. The debate also addressed the relevance of Bolter and Grusin’s Remediation (1999) and that of Manovich’s The Language of New Media (2001), especially in light of the Web 2.0 paradigm in which social software gain increased prominence. An interesting aspect in this debate proved also to be a bridging of the possible tension between digital aesthetics and a general communication theory, prompted by the recent developments on the Web. The move towards social software, Web 2.0, and the number of sites aiming to generate sociality, as well as the social move within the art world itself, makes for a situation where researchers coming from an aesthetics and a communication-theory paradigm are challenged to explore new forms of sociality and communication, and where perspectives interrogating immaterial labour might well be considered more closely. The dynamism in the chat format made the discussion touch on a number of questions. We hope that the discussion will prove to be all the more valuable for inspiring future debates.

6

Arild Fetveit and Gitte Bang Stald

Northern Lights.qxd

3/9/07

10:47 am

Page 7

Northern Lights Volume 5 © 2007 Intellect Ltd Article. English language. doi: 10.1386/nl.5.1.7/1

Mixed media: from digital aesthetics towards general communication theory Klaus Bruhn Jensen Abstract

Keywords

During the last decade, many studies have reconsidered the definition of ‘media’, frequently emphasizing how ‘new’digital media may be reproducing or reformulating ‘old’ analogue media. Through a critical examination of two key contributions – Bolter and Grusin (1999) on remediation and Manovich (2001) on the language of new media – this article suggests that much current work under a heading of ‘digital aesthetics’, approaching media as modes of representing reality, rather than as resources for acting in and on reality, is missing not one, but two opportunities – one of exploring interactivity at the level of meaning as received and interpreted, the other of specifying how the discourses of digital media enter into social interaction beyond the interface. Digital media should be understood in the wider context of general communication theory, including issues of ‘mediated’ and ‘unmediated’ social interaction.

Action Communication theory Digital media History of communication Modality

Introduction In 1996, one of the world’s main professional organizations for communication research changed its name. The abbreviation, IAMCR, which used to denote the International Association for Mass Communication Research, came to refer to the International Association for Media and Communication Research. Founded in 1957, at a time when the ‘old’ mass media, with television at the forefront, were consolidating themselves as social institutions, the IAMCR, like communication research at large, was coming to terms with another major shift in its object of analysis. New media of the digital and interactive variety had challenged the field of research to reconsider the very definition of (mass) communication and (mass) media. During the ten years since 1996, a wide variety of studies have addressed this foundational issue, frequently emphasizing the question of how ‘new’ media may be reproducing or reformulating ‘old’ media. This article reviews some of the answers, identifying disciplinary as well as ideological fault lines, and proposing an agenda for continued interdisciplinary theory development. In his important history of the idea of communication, John Durham Peters showed how communication as a general category, including faceto-face interaction, ‘became thinkable only in the shadow of mediated communication. Mass communication came first’ (Peters 1999: 6). During the past few decades, the ongoing differentiation of mediated forms of communication appears to have made a general category of media

NL 5 7–24 © Intellect Ltd 2007

7

Northern Lights.qxd

3/9/07

10:47 am

Page 8

thinkable, as well. As in the case of communication, the reconceptualization of ‘media’ involves reconsideration, not just of information and communication technologies, but of the very distinctions and interrelations between humans, technological artefacts, and social contexts. The mass media, arguably, came first. At present, research is struggling to explain what comes after mass media. Following a brief genealogy of the concept of media, this article departs from two key contributions to recent media theory – Bolter and Grusin (1999) on remediation and Manovich (2001) on the language of new media – which provided some of the first comprehensive and most widely influential accounts of how the discursive forms of new media differ from those of old media. A critical analysis of the two volumes serves to identify a premise that is commonly shared in much current work under a heading of ‘digital aesthetics’, approaching media as modes of representing reality, rather than as resources for acting in and on reality. This article suggests that such a premise may lead the broadly humanistic, text-oriented stream of media and communication research to miss not one, but two opportunities in the face of new media – one of exploring interactivity at the level of meaning as received and interpreted, the other of specifying how the discourses of digital media enter into social interaction beyond the interface. The last part of this article outlines an approach to reinserting digital aesthetics into general communication theory, drawing on a wider repertoire of (new) media studies. First, while media show and tell, they also enable their users to do things in the world. All media, new and old, are vehicles of information, channels of communication, and means of both interpersonal and institutionally organized action. Second, no medium is created equal to any other in all of these respects, having been shaped in an interplay of the modalities of human experience, the historically available technologies, and the institutional conditions of communication. In order to locate new media within contemporary culture, the final section distinguishes three prototypes of media, each of which is programmable to different degrees and in different respects, including a very old medium – humans communicating in the flesh (Lakoff and Johnson 1999).

The means in the middle The Oxford English Dictionary Online (OED) (accessed 5 January 2006) notes that while classical Latin ‘medium’ referred to some middle entity or state, in post-classical Latin and in British sources from the twelfth century onwards, ‘medium’ and ‘media’ also came to denote the means of doing something. On the one hand, a medium can be understood as a more or less incidental presence, linking natural phenomena or, for the spiritually inclined, this world and the hereafter. On the other hand, a medium can serve as an intentional instrument of human action in a modern sense. In the latter respect, the OED distinguishes two conceptions – medium as an artistic modality, material, or technique; and medium as a channel of mass communication – both of them from the mid-nineteenth century, when the idea of communication took hold (Peters 1999). By the mid-twentieth century, medium in the sense of ‘any physical material (as tape, disk, paper, etc.) used for recording or reproducing data, images, or sound’ became common, presumably accelerated by digital media with diverse input and 8

Klaus Bruhn Jensen

Northern Lights.qxd

3/9/07

10:47 am

Page 9

output options. All three senses – mode of expression, material of recording, and means of transmission – can be retraced in the media-studies literature. In order to understand what are increasingly hybrid or mixed media, it is helpful to begin to unmix definitions of media. In his Keywords (1983), Raymond Williams reminded researchers that the changing meanings of, for instance, ‘media’ bear witness to the cultures using (and studying) them. Williams himself noted three senses of medium, including a middle entity and a technical means of transmission, adding ‘the specialized capitalist sense’ in which it is ‘a medium for something else, such as advertising’ (Williams 1983: 203). With or without the critical twist, the term has remained not just contested, but ambiguous. In a recent overview, Ryan (2004: 16) noted the persistence in parallel of the two midnineteenth-century senses – mode of expression and means of transmission. Whereas social scientists commonly give priority to media as technological and institutional infrastructures (means of transmission), scholars originating from the arts and humanities still tend to privilege media discourses as aesthetic forms (modes of expression). Digital media provide one more opportunity for research to consider the potential of an interdisciplinary, integrative ‘third culture’ (Brockman 1995) of media studies. One of the first movers behind the personal computer, Alan Kay, early on compared computing to music-making (Kay 1999: 129). Comparing phenomena such as media is the business of scholarship. According to Beniger (1992: 35), ‘all social science research is comparative’ because it compares across time, space, cultures, individuals – and media. Scholarly comparisons, in turn, depend on the available concepts and theories for the job, which vary with historical context. It was not until the early 1960s that ‘the media’ presented themselves as one phenomenon (Scannell 2002: 194), the elements of which called for comparative analyses. Since the seminal contributions of Marshall McLuhan (1962, 1964), research has been expected to account for different media in terms of their distinctive and complementary contributions to contemporary culture. Also outside academia, it is a common assumption that the media make up a networked cultural environment that conditions and frames social interaction as well as individual existence. As such, the media constitute the publicly accessible components of the contemporary control society (Beniger 1986), which is increasingly dependent on information and communication technologies to regulate and reproduce itself. Regardless of terminology – control, information, media, or network society (Castells 1996) – social and cultural theory is asking how material networks of communication afford and constrain imagined networks (Anderson 1991). The material channels of communication set the terms for who knows what and when (Rogers 1962); the prevalent modes of expression shape how people come to know. While research on who, what, and when in the ‘social shaping and social consequences’ (Lievrouw and Livingstone 2002) of new media still predominates, the how of communication has preoccupied a great deal of new-media theory, yielding findings with an audience far beyond the arts and humanities, and into engineering circles and boardrooms. The ongoing differentiation of media formats is challenging traditional transmission models of communication – corporate

NL 5 7–24 © Intellect Ltd 2007

9

Northern Lights.qxd

3/9/07

10:47 am

Page 10

research entities can no longer depend on old-style development processes, from lab to launch, in the attempt to generate context-sensitive and, hence, viable products and services. Enter ordinary users, creative artists, and digital aesthetics.

Remediation revisited Situated within a historical perspective of medium theory (Meyrowitz 1994), emphasizing the implications of shifting media forms for human consciousness and culture, the volume by Bolter and Grusin (1999) offered a vocabulary in which to examine new media discourses. Citing McLuhan’s famous quip, that ‘the “content” of any medium is always another medium’, the authors set out to specify ‘a more complex process of borrowing’, rejecting any ‘simple repurposing’ of one medium in another. To Bolter and Grusin, instead, ‘one medium is incorporated or represented in another’. As it turns out, this terminology provides a key to the theoretical argument – small discursive differences make a difference, in metatheory as in media discourse. A few lines on, representation is preferred over incorporation in a central definition: ‘we call the representation of one medium in another remediation, and we will argue that remediation is a defining characteristic of the new digital media’ [original emphasis] (Bolter and Grusin 1999: 45). Whereas incorporation might suggest functional integration, representation rather privileges formal simulation – surface versus substance. Remediation manifests itself, according to Bolter and Grusin, as a dual logic involving two general forms of representation, namely, immediacy and hypermediacy. Immediacy is the transparency of media as windows on the world, informed by ‘the belief in some necessary contact point between the medium and what it represents’ (Bolter and Grusin 1999: 30), and exemplified by linear perspective as well as photorealist computer graphics. Hypermediacy, in contrast, interferes with the subject’s line of sight, as in modernist art seeking to defamiliarize the spectator’s comprehension of what is being represented, not least through the form of the artwork. In an art-historical perspective, the authors note, ‘the logic of immediacy has perhaps been dominant in Western representation’, and at the end of the twentieth century, hypermediacy still was in a subordinate position, even if it ‘has never been suppressed fully or for long periods of time’ (Bolter and Grusin 1999: 34). The central accomplishment of the volume was the application of this dual logic in a series of close analyses of new media genres and discourses – from digital photography to virtual worlds – through a non-sectarian postmodernist lens of study. In subsequent publications, Bolter has extended some of the points to design practices (Bolter and Gromala 2003), as well as reconceiving his ‘history of writing’ (Bolter 1991) in a second edition with a subtitle referring to ‘the remediation of print’ (Bolter 2001). Acknowledging that ‘the computational device’ only became a medium when it acquired aesthetic forms and ‘social and cultural functions’, Bolter and Grusin (1999: 66) were early contributors to that growing body of research that has challenged commercial as well as scientific hype assuming the technological determination of culture and society, what Carey and Quirk (1988) referred to as a fascination with the technological 10

Klaus Bruhn Jensen

Northern Lights.qxd

3/9/07

10:47 am

Page 11

sublime. As a theory for interpreting and explaining such social and cultural functions, however, for ‘understanding new media’ – the subtitle of the book – Remediation presented several ambiguities. The first issue concerns the systematics of the theoretical framework. Elaborating on the relationship between media and remediation, Bolter and Grusin (1999: 65) offer ‘this simple definition: a medium is that which remediates’. But, media do not merely or primarily represent each other. And, if remediation is, indeed, the defining characteristic of new media, it is not clear what old media used to do. In some passages, the authors seem hard pressed to defend an immanent analysis of media representations, for example, when they assert that ‘there is nothing prior to or outside the act of mediation’ (Bolter and Grusin 1999: 58). A few sections do consider material, economic, and other social aspects of media, in part to claim a parallel space for aesthetic and formal studies: The social dimension of immediacy and hypermediacy is as important as their formal and technical dimensions. However, there is no need to deny the importance of the latter in order to appreciate the former, no need to reduce the technical and psychological dimensions to the social. (Bolter and Grusin 1999: 73)

Still, the theory of remediation tends to choose sides, inviting an analytical gaze at the surface of the interface, bracketing technologies, users, and social contexts. Second, the place of history – the history of media, but also the history of explanatory concepts – is in question. In support of the previous argument, that media are essentially remediators, it is said that ‘a medium in our culture can never operate in isolation’ (Bolter and Grusin 1999: 70). Yet, the analytical examples in the volume cover much of the history of western arts, raising questions of whether it might not be necessary to consider several different kinds of remediation – discursive, technological, and institutional – in order to capture the processes by which human experience has been shaped and cumulated through shifting media forms. Bolter and Grusin do recognize the historical contingency of their approach, to such a degree, in fact, that readers may wonder what kind of explanatory value is being assigned to the framework. Having emphatically subordinated objects to representations as the field of study, the authors next relativize the concepts that serve as their lens of study. What remains, appears to be a set of ad hoc analytical surfaces or terms – with immediacy and hypermediacy as the central nodes – regarding the things people do with media: ‘we see ourselves today in and through our available media’ (Bolter and Grusin 1999: 231). Importantly, we are meant to include researchers trying to make sense of the media and signs of our times. Today amounts to a rather brief window of opportunity through which contemporary media provide access to cultural history: ‘at this extended historical moment, all current media function as remediators and […] remediation offers us a means of interpreting the work of earlier media as well’ (Bolter and Grusin 1999: 55). Surely, the framework of Remediation, elaborating insights from Russian formalism onwards concerning the (de)familiarizing functions of media, has more lasting relevance; the

NL 5 7–24 © Intellect Ltd 2007

11

Northern Lights.qxd

3/9/07

10:47 am

Page 12

question is how its internalist perspective may be complemented to substantiate conclusions beyond the discourses of the media that are new here and now. A final, related ambiguity has to do with the pragmatics of remediation – what are the claims being made regarding the effects or implications of new media? Bolter and Grusin go on to draw quite far-reaching inferences about the impact of new media on users in terms of a ‘remediated self’ (Bolter and Grusin 1999: 230). They further identify a ‘psychological economy of remediation’ (Bolter and Grusin 1999: 236), which is grounded in the processes and stages of Lacanian psychoanalysis. While this is in keeping with the tradition of textual media studies spanning art history, film theory, and digital aesthetics, which infers from media representations to audience responses, the line of argument appears problematic if one seeks to account for the distinctive features of specific historical media forms. In some sections, the authors briefly consider other positions, including what amounts to an alternative hypothesis, namely, that immediacy and hypermediacy might constitute different aspects or moments of one reception process. This is suggested by evidence presented by, for example, Messaris (1994: 73), that non-western spectators quickly learn to interpret and ‘see through’ unfamiliar, hypermediated images. The relative merits of this and other approaches, however, are not pursued. In an additional reference to the psychological experiments by Reeves and Nass (1996), showing that people relate to media in the same way that they relate to other people, Bolter and Grusin (1999: 58) find that this ‘supports and complements our contention that media and reality are inseparable’. Given the radically different epistemologies and methodologies of the two approaches, it remains to be seen in which senses media and reality might be inseparable. On the dustjacket of Remediation (1999), the reader learns that the volume challenges ‘the modernist myth of the new’ assuming that new media require ‘a new set of aesthetic and cultural principles’. Cover texts are not necessarily penned by authors; ‘modernist’ is a contested term. In reference to modernity, the text nicely captures the historically reflexive perspective of the volume on media as open-ended cultural forms. In reference to modernism, however, the premise concerning the dual logic of immediacy and hypermediacy, operative since at least the Renaissance (Bolter and Grusin 1999: 21), embraces rather than challenges the modernist mainstream of contemporary art history and (digital) media aesthetics, seeking new insights and, perhaps, new forms of social organization in the cracks and crevices of aesthetic artefacts. Remediation, similarly, depends on internalist perspectives on media in order to substantiate conclusions about cultural history as well as audience psychology. New-media studies need perspectives gazing through the interface in both directions – into machines and humans in context.

The functionalities of new media Approaching the machine architecture behind the computer interface, Manovich’s The Language of New Media (2001) offered another important contribution to new-media theory. Preparing his agenda for computer aesthetics, Manovich identifies five principles of new, digital media. First, 12

Klaus Bruhn Jensen

Northern Lights.qxd

3/9/07

10:47 am

Page 13

regardless of their immediate appearance, they are the product of numerical representation or digital code. Second, new media are subject to modularity on a different scale than analogue media, being recomposable at the site of production as well as in the context of use. Third, these first two principles allow for ‘the automation of many operations’, be it ‘creation, manipulation, or access’ [emphasis added] (Manovich 2001: 32). Fourth, a further consequence of numerical and modular computing is variability, for example, interactivity as a form of user-driven variability. Fifth, new media enable cultural transcoding, or a translation back and forth between ‘a cultural layer’ of familiar objects in recognizable forms and ‘a computer layer’ processing these according to a digital common denominator (Manovich 2001: 46). To Manovich (2001: 45), this is ‘what is in my view the most substantial consequence of the computerization of media’. It is also where his position has the strongest affinities to that of Bolter and Grusin: transcoding and remediation have a family resemblance, even if they do not share all the same theoretical ancestors. Manovich cites Bolter and Grusin approvingly when he, too, seeks to distance his position from ‘a modernist view that aims to define the essential properties of every medium’ and from ‘old metaphors’ concerning interfaces in traditional human-computer interaction research (Manovich 2001: 89). Manovich’s argument joins two components ‘that today can be found in most areas of new media’. On the one hand, both the Internet and computers as such constitute a database, ‘a collection of documents’, that has been taken to a different, digital degree. On the other hand, access to the database takes place through ‘a navigable space’, specified as ‘a virtual interactive 3D space, employed in computer games, motion rides, VR, computer animation, and human-computer interfaces’ (Manovich 2001: 214). One of Manovich’s main points is that display and narrative are becoming less central in new media, compared to their role in classical arts and traditional mass media. In Manovich’s strong formulation (2001: 225), ‘database and narrative are natural enemies’, even if he recognizes that digital narratives result from the user’s interaction with games or interactive fiction. Perhaps database and narrative were cultural enemies in some previous media. Digital media facilitate links between databases and interfaces, which further enable users to communicate and act. The links between the two constituents of new media, however, are understood less as means of doing than as ways of showing. From Manovich’s perspective, cinema is experiencing a second coming as a model of digital representation: ‘To summarize, the visual culture of a computer age is cinematographic in its appearance, digital on the level of its material, and computational (i.e., software driven) in its logic’ [original emphasis] (Manovich 2001: 180). Even intuitively, however, it is questionable whether cinema, in some definition, can account for the range of representations in computer interfaces. The GUI (graphic user interface) is clearly home to variants of cinema, television, and video; it is also a point of access to other virtual 3D spaces. But, cinematography is hardly a sufficient principle when it comes to matters of, for example, the layout or navigation of a database. In the last part of the volume, Manovich elaborates on his conception of cinematography and film theory, as informed by aesthetics and semiotics.

NL 5 7–24 © Intellect Ltd 2007

13

Northern Lights.qxd

3/9/07

10:47 am

Page 14

With reference to the basic semiotic matrix of paradigms and syntagms, he argues that even if interactive interfaces present users with several simultaneous paradigms from which to choose, ‘the end result is a linear sequence of screens that […] unfolds along a syntagmatic dimension’. The resulting syntagms are described, further, as a ‘language-like sequencing’ which, to Manovich, suggests that new media ‘follow the dominant semiological order of the twentieth century – that of cinema’ (Manovich 2001: 232). Leaving aside the issue of whether cinema might qualify as the dominant cultural order of the last century, again it is intuitively far from clear that the common experience of watching several screen images, interstitched by paradigmatic choices, resembles anything like cinema, or television, or animation, for that matter. In theoretical terms, moreover, it is quite a stretch to batch verbal language and computer interfaces with cinema as sequential vehicles of meaning under a heading of ‘languagelike’ characteristics. Especially against the background of film theory, which has had notorious struggles with the metaphor of film as language (Metz 1974), it is surprising to find the metaphor reinstated at this level of generality for the field of new-media studies. In specific analyses of interactive genres, especially games, the volume does recognize the various communicative interchanges linking system and users, beyond their cinematic identity as spectators gazing at silver and other screens. In a key section examining the ways in which database and interface map onto each other during an interchange, Manovich begins to focus the performative aspect of using new media. Having noted the potential conflict between efficient access to information and the users’ psychological involvement, he generalizes the point in italics: ‘Along with surface versus depth, the opposition between information and “immersion” can be thought of as a particular expression of the more general opposition characteristic of new media – between action and representation’ (Manovich 2001: 216). The implication seems to be that the category of action is associated with immersion or engagement – virtual action. Action in the sense of interactivity with a database of content, with other users, or with the system of communication itself, is not theorized explicitly and on a par with the other pole of ‘the more general opposition’ of new media – representation. And, the everyday actions that people perform with computers – from social networking and netbanking, to cultural engagement and political mobilization – fall outside the perspective of this cinematographic theory of new media. Compared to the approach of Bolter and Grusin, Manovich appears relatively more cautious in inferring from media formats to their consequences for users and historical contexts. Still, in addition to conceiving of cinema as the dominant cultural code of the last century, he also assumes that cinema holds the key to understanding twenty-firstcentury media, returning in his last chapter to André Bazin’s question, ‘What is cinema?’ Manovich’s answer is that what we used to think of as ‘cinema’s defining characteristics are now just default options, with many others available’ (Manovich 2001: 293). More ambitiously, cinema is taken to provide both the default option and the source code for other options. Having reviewed how cinema was born from animation, which then became marginalized, the author restates the question, ‘What is digital 14

Klaus Bruhn Jensen

Northern Lights.qxd

3/9/07

10:47 am

Page 15

cinema?’: ‘Digital cinema is a particular case of animation that uses liveaction footage as one of its many elements’ [original emphasis] (Manovich 2001: 302), a notion that Manovich has explored in a creative project on ‘soft cinema’ (Manovich and Kratky 2005). Most important perhaps, a particular subset of cinema is said to triumph with the computer: One general effect of the digital revolution is that avant-garde aesthetic strategies came to be embedded in the commands and interface metaphors of computer software. In short, the avant-garde became materialized in a computer […] collage reemerged as the ‘cut-and-paste’ command, the most basic operation one can perform on digital data. (Manovich 2001: 306f)

Modernism is back, not just as aesthetic logic, but as technological form. On the very last page of the book, Manovich adds that ‘cinema, along with other established cultural forms, indeed becomes precisely a code. It is now used to communicate all types of data and experiences, and its language is encoded in the interfaces and defaults of software programs and in the hardware itself’ (Manovich 2001: 333). Leaving aside again the strong and surprising claim that cinema is already encoded in the hardware of computers, the present discussion has suggested that cinema will account for only certain dimensions of how digital media articulate information, enable communication, and facilitate action. Cinema, undoubtedly, is the source of some subset of the codes that are currently being reworked in the software of digital media. Cinema may, or may not, have a language. But, it is the functionalities and practices that link databases and users via interfaces that a theory of new media, above all, must account for. Manovich and Bolter and Grusin, in related ways, have begun to explore how new media show and tell at the interface. Media also do things beyond the interface.

Media showing, telling, and doing The position of digital aesthetics, as informed by cinematography and art history in the works of Manovich and of Bolter and Grusin, can be summarized with reference to recent interdisciplinary research that focuses, not on visuals, but on sound (Bull and Back 2003). Sound serves as a reminder concerning the multimodal nature of new media and human communication as such. Examining sound in cinema and other screen media, Chion (1994) identified three modes of listening. Causal listening seeks the source of a sound, for example, a human voice. Semantic listening interprets its message in terms of a code, i.e. a particular verbal statement. And, reduced listening, a term coined by Pierre Schaeffer, focuses on ‘the traits of the sound itself, independent of its cause and of its meaning […] its own qualities of timbre and texture’ (Chion 1994: 29–31). In real-life settings, causal and semantic listening can be expected to predominate; people listen in order to orient themselves and understand events in context. In arts settings, and in the meta-analysis of sound by musicologists or acousticians, reduced listening is the defining practice. Digital aesthetics has given priority to reduced listening and viewing.

NL 5 7–24 © Intellect Ltd 2007

15

Northern Lights.qxd

3/9/07

10:47 am

Page 16

Over the last decade, much other research has sought to establish links between the social, technological, and aesthetic aspects of new media (e.g. Bell and Kennedy 2000; Lister et al. 2003), notably studies arising from the Association of Internet Researchers, as reported in the Internet Research Annual (2004ff). In order to advance an interdisciplinary dialogue on the several necessary constituents of a theory of new media, it is helpful to return to some of the basics of communication theory. Media are vehicles of information; they are channels of communication; and they are means of both interpersonal and macrosocial action (Jensen, 2006). While all of these remain contested – as terms, concepts, and phenomena – together they offer a set of what Blumer (1954) called ‘sensitizing concepts’ in configuring the domain of inquiry. The conceptual pair of ‘information’ and ‘communication’, first of all, is familiar from several fields of research in various terminological guises. Philosophy traditionally distinguishes between proposition and modality, i.e. a potential reference and the reality status being assigned to it in an assertion (Audi 1996). In structuralist literary and film theory, enoncé covers a work as a statement or message, whereas enonciation refers to the act of enunciation (Stam et al. 1992: 105). And, in speech-act theory (Austin 1962; Searle 1969), a distinction was introduced between locution (propositional components), illocution (a social act being performed, for example, a promise or a threat), and perlocution (the received implications of the act). In combination, information and communication enable socially coordinated actions – from discussion and voting, to consumer purchases and investments, to political and aesthetic involvement. Each of the three constitutive concepts can be exemplified with reference to sound: • Information: Sound serves as an explicit and regularized vehicle of delimited items of information. This is the case in oral narratives, with fire alarms (no warning without an implied object of attention), as well as for jingles and other ‘program music’ that seeks to generate ideas or values in the listener. • Communication: Sound supports intersubjective relations of communication. An oral narrative engages its listeners, young and old. A fire alarm, when activated by a person or by smoke, addresses a warning to the inhabitants of a building. And, program music produces, however tendentially and momentarily, some level of understanding and orientation in the audience. • Action: Sound accomplishes physical as well symbolic actions, over and above the (speech) act being performed in and of communication – sound becomes action as it is embedded in established social practices and institutions. Storytelling is a classic part of primary socialization; fire alarms accomplish evacuations; and program music reactivates imagined communities (Anderson 1991), ranging from nationalism to consumerism. Media, new and old, enable and constrain these uses, functions, or characteristics in different ways and shifting configurations. Information can be thought of as the potential articulation of insights and ideas, lending 16

Klaus Bruhn Jensen

Northern Lights.qxd

3/9/07

10:47 am

Page 17

itself to externalization and dissemination, through the modalities of human experience and communication technologies of human making; in more formal terms, ‘information is data that have been organized and communicated’ (Porat 1977: 2). Artworks, digital and otherwise, may be understood as information waiting to make its mark on the world through some medium. Communication, next, minimally requires a mode of expression and a channel of transmission, as noted, both of which are programmable in different respects and to varying degrees. The act of communication produces some more or less stable tokens to which two parties make themselves available and, to a degree, internalize. Finally, through informational representation and communicative interaction, the communicators engage in action, cumulatively enacting themselves, their significant others, and the social system of which both are components. This potential widening of the field of media studies next suggests the question: what is not a medium? Anthropology, sociology, and other adjoining fields note that people continuously ascribe significance to natural objects, cultural artefacts, and social institutions. Even the boundary between ‘nature’ and ‘culture’ is negotiated from a position within culture through the historically available media. As pinpointed by Watzlawick et al. (1967), humans cannot not communicate – the body shows itself, and it sounds. Equally, social arrangements from business transactions to interior decorating have, or are given, meanings. The media that form the objects of analysis for media and communication research are distinguished by their ‘programmability’, being flexible resources for the articulation of information and communicative interaction as part of social structuration (Giddens 1984). The definition of media in terms of programmability can be specified in three respects. First, media comprise modalities that make possible the rendering of and interaction with worlds, past and present, real and imagined. Modalities amount to semiotic registers of verbal language, music, still and moving images, etc. enabling an immensely varied repertoire of discourses and genres, and engaging the human senses in selective and culturally conventional ways. Second, media depend on a material substratum for articulating and presenting information, as commonly associated with modern technologies of communication. (The next section considers the human body in context as a medium.) Like modalities, technologies lend themselves to diverse aesthetic and social adjustments – across time, space, and possible worlds. Third, media communicate to, about, and on behalf of social institutions. Media and societies mutually shape – programme – each other in the course of prevalent communicative and cultural practices (Meyrowitz 1994). The agenda of new-media studies may be clarified with reference to these three aspects, particularly how the modalities, technologies, and institutions of digital media relate to those of earlier (mediated) communication.

Media of three degrees The traditional dichotomy of ‘mediated’ and ‘unmediated’ communication, as mentioned in the introduction, assumes that the human body does not qualify as a medium of contact and exchange, but somehow communicates directly. As argued by Peters (1999: 264), neither messages nor people have NL 5 7–24 © Intellect Ltd 2007

17

Northern Lights.qxd

3/9/07

10:47 am

Page 18

a simple, immediate presence in the world – even face-to-face ‘dialogue may simply be two people taking turns broadcasting at each other’. With the rise of many more differentiated types of communicative interaction, the dichotomy is increasingly in question. At the present stage of research, it is helpful to distinguish conceptually and analytically between three degrees of media (see further Jensen 2002a, 2006). Media of the first degree can be defined, briefly, as the biologically based, socially formed resources that enable humans to articulate an understanding of reality, for a particular purpose, and to engage in communication about it with others. The central example is verbal language, or speech, as constitutive of oral cultures and subcultures (e.g. Scribner and Cole 1981) – additional examples include song and other musical expression, dance, drama, painting, and creative arts generally, often relying on comparatively simple, mechanical techniques such as musical instruments and artistic or writing utensils as necessary elements. Importantly, such media depend on the presence of the human body in local time-space. While one might identify (spoken) language, or the human voice, as the medium, it seems helpful to differentiate between, for example, speech and song as media with reference to their different modalities, sharing the same material substratum, but commonly addressing different social institutions, contexts, and practices. Frequent references to the ongoing ‘mediatization’ of politics and culture tend to obscure the fact that embodied speech, music, and other sounds remain constitutive of everyday life. As noted by one standard textbook of media studies (McQuail 2005: 18), the total number of face-toface interactions that occur within the micro-coordination of daily life by far outnumber those communicative events that are technologically mediated. Moreover, speech became an integral part of the modern mass media, notably radio and television, further stimulating conversations about and around media (Gumpert and Cathcart 1986; Scannell 1991). Indeed, Ong (1982) argued that the technological re-embedding of speech had produced a new form of ‘secondary orality’. Speech delivers not just the contents, but also many of the forms that have been remodelled as media genres – the town crier as news announcer, the court jester as talkshow host. Theorizing digital media, it is essential to consider not just the reworking of analogue into digital media, whether in the sense of ‘remediation’ or ‘new languages’, but equally the human body as a source and medium of representation and interaction. Compared to a tendency in some cybercultural and digital aesthetics (e.g. Haraway 1997; Hayles 1999; Stone 1991) to discursify the body, it seems time for new-media studies to examine users as historical and biological individuals, not just as abstractions and represented surfaces. Media of the second degree come under the heading of Benjamin’s technically reproduced and enhanced forms of representation and interaction (1977) which support communication across space and time, irrespective of the presence and number of participants. Whereas Benjamin placed the emphasis on photography, film, and radio, media of the second degree range from early modern examples including the standardized reproduction of religious and political texts by the printing press (Eisenstein 1979), to television and video. The common features are, first, 18

Klaus Bruhn Jensen

Northern Lights.qxd

3/9/07

10:47 am

Page 19

one-to-one reproduction, storage, and presentation of a particular content and, second, radically extended possibilities for dissemination across time and space. In this regard, the technologies were key to a re-embedding, both of media of the first degree and of people in relation to distant others, issues, and arenas. At the same time, the specific adaptability or programmability of these media had important consequences for major social institutions – from the Catholic Church to the nation state. And, modalities from media of the first degree were reworked – remediated: in radio talk shows, conversation took on new conventions, just as acting styles were adapted from the theatre stage to cinema and television. (A further question is whether handwriting, fixing, for instance, speech and music in comparatively stable forms, should be understood as a separate category of media. In the present context, handwriting is considered within media of the first degree: the production of manuscripts is embodied and local, laborious and error-prone, and their distribution is selective, commonly within established institutions, as supported by oral commentary.) Media of the third degree are the digitally processed forms of representation and interaction and, accordingly, of particular interest here. Digital technology enables reproduction and recombination of all media of the second degree on a single platform – computers, thus, can be understood as metamedia (Kay and Goldberg 1999) with an unprecedented degree of technical programmability, between as well as within previous media. The central current example is the networked personal computer,

Media of first degree

Media of third degree

Media of second degree

Figure 1: Media of three degrees.

NL 5 7–24 © Intellect Ltd 2007

19

Northern Lights.qxd

3/9/07

10:47 am

Page 20

although this interface, as well as that of mobile telephones, is likely to change substantially as technologies are adapted further to the human senses, and integrated into both common objects and social arrangements. Whereas classic mass media, such as illustrated magazines and television, combined modalities to a considerable degree, the scale and speed with which digitalization facilitates the incorporation and reconfiguration of second-order modalities, supports the view that already the personal computer may represent a qualitative shift from media of the second degree that is comparable to the shift from first-degree to second-degree media. The interrelations of digital technology and multimodality with the institutions of contemporary society are still in the making, with implications to be determined through empirical research and in historical perspective. One characteristic of media of the third degree is their re-enactment or simulation of face-to-face interaction. Computer networks enable forms of interaction that are more similar to interpersonal than to mass communication, as exemplified by the informality of e-mail, chat, and gaming. In certain respects, humans are media; in certain respects, digital media can substitute for the social roles of humans. Figure 1 seeks to illustrate the interrelations of the media of three different degrees as a wheel of culture. The media types do not replace each other – they recirculate the forms and contents of shifting cultural traditions, and they remain elements of the same historical media environment. They do, however, constitute different and ascending degrees of combined programmability in terms of adaptable technologies, differentiated modalities, and institutions transcending time, space, and social actors. While communication has always been pervasive, digital technologies are making information and interaction more accessible and applicable across contexts. Why communicate so much? As noted by Aristotle (Clarke 1990: 11), words allow humans to consider that which is at least temporarily absent – in space, in time, and from one’s immediate experience – through thought experiments and dialogue. Media can represent what is absent from, but imagined within, face-to-face encounters, opening up universes of what is not yet, what might be, as well as what ought never to come to pass. Why not communicate less? We cannot not communicate (Watzlawick et al. 1967), because we are copresent with others in the real world, and necessarily share a culture. In a discussion of communication and culture in relation to music, Meyer (2001: 348f) noted that we keep social complexity manageable through culture: ‘what most significantly shaped human behaviour and gave rise to human cultures was not the presence, but the absence of adequate innate constraints. It is because evolution resulted in such an animal that human cultures became indispensable.’ Culture is not icing on the layercake of evolution and history; it is the preliminary outcome of communication in managing extreme social and cognitive complexities for endless practical purposes. We need all the media we can get, occasionally to appreciate their aesthetics, but mostly to get by and go on.

20

Klaus Bruhn Jensen

Northern Lights.qxd

3/9/07

10:47 am

Page 21

Conclusion Media and communication research is positioned to renew its theory development, having been challenged by digital technologies to reconsider its core concepts of ‘media’ and ‘communication’. This article has argued for an inclusive agenda, incorporating interdisciplinary concepts and concerns from several decades of humanistic as well as social-scientific research, as well as addressing humans as media. The traditional divides between interpersonal, organizational, and mass communication studies are increasingly counterproductive. Media content itself – from Frankenstein (1818) via Blade Runner (1982) to current massively multiplayer online role-playing games (MMPORPG) – provides a cultural laboratory regarding the status of humans and the realities about which they communicate. As a second-order laboratory or institution-to-think-with (Jensen 2002b), research – from hard-nosed artificial intelligence (e.g. Boden 1996) via semi-soft actor-network theory (e.g. Latour 1993) to postmodern philosophy dissolving the category of being human (e.g. Hayles 1999) – equally is at pains to define who or what communicates. ‘Mixed media’ that combine materials in more or less innovative ways are a familiar format in artistic practice and criticism. The aesthetic gaze and the camera eye, as developed by Bolter and Grusin (1999) and by Manovich (2001), are valid perspectives on new, mixed media, as well. Appearing half a decade after the popular breakthrough of the Internet, the two volumes offered some of the first elaborate theories regarding digital technologies as media, and have contributed to digital aesthetics as a separate sub-speciality of study. In order to account for the wider implications of mixed media today, however, as they reconfigure modalities, materials, as well as institutions, digital aesthetics need to reconsider their interfaces with other explanatory models. From within the art domain, the tradition of contemplative appreciation of media and culture has recently been countered, for example, by Summers (2003) in a ‘post-formalist art history’, which examines the arts as thoroughly practical enterprises in a material environment of really existing media and humans. The larger field of media and communication research itself is ripe with approaches to the texts and contexts of new media, from their role in everyday life (Wellman and Haythornthwaite 2002) and sociocultural communities (Baym 2000) to their place in the infrastructures of economy and politics (Castells 1996). In conjunction, these approaches may begin to address the key question regarding any new medium for policy-makers, business leaders, cultural activists, little boys and, increasingly, little girls: what does it do? If the idea of communication has been a century and a half in the making (Peters 1999), it is not surprising that the definition of media has continued to pose significant challenges for research since the 1960s, as restated by digital media during the 1990s. The media of three degrees provide a framework in which to approach the distinctive affordances (Gibson 1979; Hutchby 2001) of different media, with implications for human communication and action over the longues durées of history. Mixed media fill up art museums; metamedia saturate the everyday across platforms and contexts. In order to focus historical and empirical studies of the social uses and implications of new media, further research is needed to unmix theoretical definitions of media. NL 5 7–24 © Intellect Ltd 2007

21

Northern Lights.qxd

3/9/07

10:47 am

Page 22

References Anderson, B. (1991), Imagined Communities: Reflections on the Origin and Spread of Nationalism, 2nd edn., London: Verso. Audi, R. (ed.) (1996), The Cambridge Dictionary of Philosophy, Reprinted edn., Cambridge: Cambridge University Press. Austin, J.L. (1962), How To Do Things With Words, Oxford: Oxford University Press. Baym, N.K. (2000), Tune In, Log On: Soaps, Fandom, and Online Community, Thousand Oaks, CA: Sage. Bell, D. and Kennedy, B.M. (eds) (2000), The Cybercultures Reader, London: Routledge. Beniger, J. (1986) The Control Revolution, Cambridge, MA: Harvard University Press. ———— (1992), ‘Comparison, yes, but – the case of technological and cultural change’, in J.G. Blumler, J.M. McLeod and K.E. Rosengren (eds), Comparatively Speaking: Communication and Culture Across Space and Time, Newbury Park, CA: Sage. Benjamin, W. (1977 [1936]), ‘The work of art in the age of mechanical reproduction’, in J. Curran, M. Gurevitch and J. Woollacott (eds), Mass Communication and Society, London: Edward Arnold. Blumer, H. (1954), ‘What is wrong with social theory?’ American Sociological Review, 19, pp. 3–10. Boden, M.A. (ed.) (1996), Artifical Intelligence, San Diego, CA: Academic Press. Bolter, J.D. (1991), Writing Space: The Computer, Hypertext, and the History of Writing, Hillsdale, NJ: Lawrence Erlbaum. ———— (2001), Writing Space: Computers, Hypertext, and the Remediation of Print, 2nd edn., Mahwah, NJ: Lawrence Erlbaum. Bolter, J.D. and Gromala, D. (2003), Windows and Mirrors: Interaction Design, Digital Art, and the Myth of Transparency, Cambridge, MA: MIT Press. Bolter, J.D. and Grusin, R. (1999), Remediation: Understanding New Media, Cambridge, MA: MIT Press. Brockman, J. (ed.) (1995), The Third Culture, New York: Simon & Schuster. Bull, M. and Back, L. (eds) (2003), The Auditory Culture Reader, Oxford: Berg. Carey, J.W. and Quirk, J.J. (1988), ‘The mythos of the electronic revolution’, in J.W. Carey (ed.), Communication as Culture, New York: Unwin Hyman. Castells, M. (1996), The Rise of the Network Society, Oxford: Blackwell. Chion, M. (1994), Audio-Vision: Sound on Screen, New York: Columbia University Press. Clarke, D.S. (1990), Sources of Semiotic: Readings with Commentary from Antiquity to the Present. Carbondale, IL: Southern Illinois University Press. Eisenstein, E.L. (1979), The Printing Press as an Agent of Change: Communication and Cultural Transformation in Early-Modern Europe, Cambridge: Cambridge University Press. Gibson, J.J. (1979), The Ecological Approach to Perception, London: HoughtonMifflin. Giddens, A. (1984), The Constitution of Society, Berkeley, CA: University of California Press. Gumpert, G. and Cathcart, R. (eds) (1986), Inter/media: Interpersonal Communication in a Media World, New York: Oxford University Press. Haraway, D.J. (1997), Modest_Witness@Second_Millennium. Femaleman_Meets_ Oncomouse: Feminism and Technoscience, New York: Routledge.

22

Klaus Bruhn Jensen

Northern Lights.qxd

3/9/07

10:47 am

Page 23

Hayles, N.K. (1999), How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics, Chicago: University of Chicago Press. Hutchby, I. (2001), Conversation and Technology: From the Telephone to the Internet, Cambridge: Polity. Jensen, K.B. (2002a), ‘Introduction: The state of convergence in media and communication research’, in K.B. Jensen (ed.), A Handbook of Media and Communication Research: Qualitative and Quantitative Methodologies, London: Routledge. ———— (2002b), ‘The social origins and uses of media and communication research’, in K. B. Jensen (ed.), A Handbook of Media and Communication Research: Qualitative and Quantitative Methodologies, London: Routledge. Jensen, K.B. (2006), ‘Sounding the media: An interdisciplinary review and a research agenda for digital sound studies’, Nordicom Review, 27(2), 7-33. Kay, A. (1999 [1984]), ‘Computer software’, in P.A. Mayer (ed.), Computer Media and Communication: A Reader. Oxford: Oxford University Press, pp. 129-138. Kay, A. and Goldberg, A. (1999 [1977]), ‘Personal dynamic media’, in P.A. Mayer (ed.), Computer Media and Communication: A Reader, Oxford: Oxford University Press, pp. 111–19. Lakoff, G. and Johnson, M. (1999), Philosophy in the Flesh: The Embodied Mind and its Challenge to Western Thought, New York: Basic Books. Latour, B. (1993), We Have Never Been Modern, Hemel Hempstead: Harvester Wheatsheaf. Lievrouw, L. and Livingstone, S. (eds) (2002), Handbook of New Media: Social Shaping and Social Consequences, London: Sage. Lister, M., Dovey, J., Giddings, S., Grant, I. and Kelly, K. (2003), New Media: A Critical Introduction, London: Routledge. Manovich, L. (2001), The Language of New Media, Cambridge, MA: MIT Press. Manovich, L. and Kratky, A. (2005), Soft Cinema: Navigating the Database, Cambridge, MA: MIT Press. McLuhan, M. (1962), The Gutenberg Galaxy, Toronto: University of Toronto Press. ———— (1964), Understanding Media, New York: McGraw-Hill. McQuail, D. (2005), McQuail’s Mass Communication Theory, 5th edn., London: Sage. Messaris, P. (1994), Visual ‘Literacy’: Image, Mind, and Reality, Boulder, CO: Westview Press. Metz, C. (1974), Language and Cinema, The Hague: Mouton. Meyer, L.B. (2001), ‘Music and emotion: Distinctions and uncertainties’, in P.N. Juslin and J. Sloboda (eds), Music and Emotion: Theory and Research, Oxford: Oxford University Press, pp. 341–60. Meyrowitz, J. (1994), ‘Medium theory’, in D. Crowley and D. Mitchell (eds), Communication Theory Today, Cambridge: Polity Press. Ong, W. (1982), Orality and Literacy, London: Methuen. Peters, J.D. (1999), Speaking into the Air: A History of the Idea of Communication, Chicago: University of Chicago Press. Porat, M. (1977) The Information Economy: Definition and Measurement, Washington, DC: Government Printing Office. Reeves, B. and Nass, C. (1996), The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places, New York: Cambridge University Press. Rogers, E. M. (1962), The Diffusion of Innovations, Glencoe, IL: Free Press.

NL 5 7–24 © Intellect Ltd 2007

23

Northern Lights.qxd

3/9/07

10:47 am

Page 24

Ryan, M.-L. (2004), ‘Introduction’, in M.-L. Ryan (ed.), Narrrative Across Media: The Languages of Storytelling, Lincoln: University of Nebraska Press. Scannell, P. (ed.) (1991), Broadcast Talk, London: Sage. ———— (2002), ‘History, media, and communication’, in K.B. Jensen (ed.), A Handbook of Media and Communication Research: Qualitative and Quantitative Methodologies, London: Routledge. Scribner, S. and Cole, M. (1981), The Psychology of Literacy, Cambridge, MA: Harvard University Press. Searle, J.R. (1969), Speech Acts, London: Cambridge University Press. Stam, R., Burgoyne, R. and Flitterman-Lewis, S. (1992), New Vocabularies in Film Semiotics, London: Routledge. Stone, A.R. (1991), ‘Will the real body please stand up? Boundary stories about virtual cultures’, in M. Benedikt (ed.), Cyberspace: First Steps, Cambridge, MA: MIT Press. Summers, D. (2003), Real Spaces: World Art History and the Rise of Western Modernism, London: Phaidon. Watzlawick, P., Beavin, J.H. and Jackson, D.D. (1967), Pragmatics of Human Communication: A Study of Interactional Patterns, Pathologies, and Paradoxes, New York: Norton. Wellman, B. and Haythornthwaite, C. (eds) (2002), The Internet in Everyday Life, Malden, MA: Blackwell. Williams, R. (1983), Keywords, London: Fontana.

24

Klaus Bruhn Jensen

Northern Lights.qxd

3/9/07

10:47 am

Page 25

Northern Lights Volume 5 © 2007 Intellect Ltd Article. English language. doi: 10.1386/nl.5.1.25/1

Remediation and the language of new media Jay David Bolter Abstract

Keywords

Many new-media enthusiasts have inherited from modernist aesthetic theory the assumptions of essentialism and absolute originality. They assume that each medium is constituted by a unique set of essential characteristics and that the task of designers is to explore these characteristics by creating artefacts that will ‘define the medium’. Bolter and Grusin’s Remediation is a study of intermedial relationships that rejects modernist aesthetics and calls these assumptions into question. Manovich’s The Language of New Media does not simply accept these assumptions, but it does seek to derive new-media artistic practice from modernism.

Remediation Intermediality Essentialism Avant-garde Modernist aesthetics New-media theory

Remediation is a study of the relationships between ‘new media’ and traditional media. Richard Grusin and I argued that these relationships were often ignored by popular new-media enthusiasts, who insisted on the utter novelty of digital technology as a medium. Their unstated assumption was (and is) that complete originality is a necessary condition for true creativity. In assuming that the goal of the artist or designer is (and should be) to reinvent the medium, new-media writers were adopting a popularized version of the rhetoric of high modernism. The attitude was typified by the following quotation from Steven Holtzman’s Digital Mosaics (Holtzman 1997), in which the author characterized repurposing (a kind of remediation) as a passing phase in the development of a new medium: ‘Repurposing is a transitional step that allows us to get a secure footing on unfamiliar terrain. But it isn’t where we’ll find the entirely new dimensions of digital worlds. We need to transcend the old to discover completely new worlds of expression’ (Holtzman 1997: 15). It is revealing that, even in the title of a book on the originality of new media, Holtzman referred to the ancient medium of the mosaic. Holtzman’s title was an unwitting acknowledgement that remediation is an unavoidable element in the process of mediation. But Holtzman was certainly not alone in the modernist assumption of utter originality, which is implied in the term ‘new media’ itself. By giving a name (remediation) to the process of borrowing among media forms, we hoped to challenge this assumption and encourage readers to examine the complex intermedial relationships of digital media forms to such older forms as film, television, radio and photography. The term ‘remediation’ indicates a particular kind of intermedial relationship, characterized by what Harold Bloom referred to long ago as the ‘anxiety of influence’ (Bloom 1997). In Remediation, we used a

NL 5 25–37 © Intellect Ltd 2007

25

Northern Lights.qxd

3/9/07

10:47 am

Page 26

shorthand when we claimed that one piece remediates another or even that one media form (computer games) remediates another (narrative film). We were not trying to suggest that media are autonomous agents that act on each other or on other aspects of our mediated culture. Remediation is a process that is realized in and through the creative practices of individual producers, designers, and artists. Sometimes this remediation is conscious and intended; sometimes individual designers may not acknowledge their dependence on earlier media even to themselves. But in all cases they are engaging in a dialogue with their audience, for it is the audience who will construct the meaning of the remediation. In the remediation of one form by another, there is always a combination of homage and rivalry. A remediating form pays homage by borrowing representational practices of an older one. At the same time, the newer form is trying to surpass the older one in some way, for the simple reason that it must justify its claim on our cultural attention. There are several ways in which a new form might justify such a claim. For example, it might offer itself as a cheaper or more efficient alternative. However, such technical improvements per se do not seem to constitute new media forms; all the alphabetic soup of different wireless protocols (such as GPRS, EDGE, WCDMA, and so on) is still regarded as the same medium for mobile telephony and data exchange. In order to constitute a new medium or a significant new form within an existing medium, designers must produce a significant change in representational practice with the tacit or explicit suggestion that this change offers an experience that is more compelling, more ‘authentic’, even more ‘real’. If we look at media innovations since photography, we see that remediations with significant cultural impact often claim to provide a more faithful representation of our experience of nature or human relations. In this discussion of the real, I am not making my own claim about the ontological status of any medium, as Andre Bazin did, for example, in his essay on photography (Bazin 1980). I wish to focus on cultural constructions of the real – on the belief that media can achieve a status of immediacy. We can call this a desire for transparency, and even in our sophisticated media age this desire remains pervasive. One need only read the film reviews in popular newspapers to understand the tenacity of an uncritical desire for transparency, when reviewers regularly complain that a character in a film never ‘comes to life’ or praise a film for putting us in touch with the true emotions of the characters. Remediation is meant above all to describe the competition among various media forms over the construction of the real. This competition is not the only aspect of the relationships among media, and remediation is not the only approach to examining such relationships. Remediation can be regarded as part of the larger project of intermediality, a version of comparative media studies that is now quite vigorous in Europe, particularly Germany, and in Canada, although not perhaps in the United States. Intermediality can encompass a broad range of relationships, although scholars who address intermediality have tended to focus on formal relationships and aesthetic effects. (This is in contrast to those who characterize their work as the cultural studies of visual media.) Among many authors in intermediality, we can name Jürgen E. Müller (1996), 26

Jay David Bolter

Northern Lights.qxd

3/9/07

10:47 am

Page 27

Yvonne Spielmann (1998 and 2005), Irina Rajewsky (2002), all of whom have written books with this title. There is also a journal entitled Intermedialités published by the University of Montreal. Remediation was never meant to be a purely formal concept, however, for the borrowing of representational practices always has social, ideological, and economic as well as aesthetic dimensions. With The Lord of the Rings, for example, we could explore how Peter Jackson used various filmic techniques to blur the class distinctions sanctioned in Tolkien’s original books, or, more generally, how Jackson used the techniques of blockbuster film-making (and advertising) to broaden the film’s appeal to an audience large enough to pay for the enormous costs of production – costs which of course the novels did not entail. These are aspects of remediation to which we alluded, but did not devote enough space in our book. Each filmic technique (e.g. the long take or the jump cut), every affordance in a computer interface (e.g. clickable icons or the use of the joystick) – every significant formal practice embedded in popular media artefacts contributes to a cultural construction of the real. And this construction has implications for the production and the consumption of media. Borrowing a technique in a new medium may change the construction. So for example, the long take in film was for Bazin an indication of authenticity: as a sign of an auteur, it was a technique that simultaneously ensured the realism and the aesthetic quality of film. It became associated with directors who were regarded as artists with important vision. And yet in computer games the remediation of the cinematic long take is the relentless first-person point of view that has come to be identified with a violent genre, known as the so-called ‘first person’ shooter. This genre is often regarded as a cultural embarrassment and is not held to be either realistic or the work of auteurs. In studying the remediation of the real in various media forms, Grusin and I identified two representational strategies: transparency and hypermediacy. This is obviously not a new idea with us. Many art historians and media theorists had presented dichotomies that influenced our thinking. Above all, we were influenced by the standard interpretations of modernism, by Greenberg and many others, which set up a dichotomy between the transparency of illusionistic painting and the reflexive practices (hypermediacy) of modern art. There is also McLuhan’s much maligned distinction between hot and cool media (where hot media might be thought of as transparent). And there is Benjamin’s distinction between auratic and non-auratic art. With transparency and hypermediacy, we wanted to describe in particular the ways in which designers and artists mask or acknowledge their debt to earlier media forms, as they seek to mobilize their audience’s sense of the real or the authentic. Scholars who have picked up our dichotomy have shown more interest in the notion of hypermediacy than transparency. Ours is an age of hypermediacy: with forms ranging from handwriting and print to film and 3D games, our diverse media economy welcomes hybrids. Digital technology makes it relatively easy for designers to hybridize different media. Since its formation in the first part of the twentieth century, the avant-garde has usually pursued strategies of hybridity and hypermediacy,

NL 5 25–37 © Intellect Ltd 2007

27

Northern Lights.qxd

3/9/07

10:47 am

Page 28

and today even popular media forms (MTV, rock-music concerts, mobilephone applications) often favour these strategies. Even in this era of MySpace and YouTube, however, we should not discount the influence of the desire for transparency or immediacy (the erasure of the medium). In recent work, my colleagues at Georgia Tech and I have argued for a relationship between immediacy and Benjamin’s notion of aura (Bolter, MacIntyre, Gandy and Schweitzer 2006). The desire for immediacy can also be understood as the desire for presence. At least since Derrida, deconstructing our culture’s desire for presence has been a part of the poststructuralist and postmodern project. Theorists in these traditions tend to treat the issue as if it were settled: that we cannot attain presence through any technology of representation. But critical theory has never had much influence on either the popular imagination or the academic community outside of the humanities and some social sciences. The notion of presence lives on, for example, in computer science, among those specializing in virtual reality: Presence (published by MIT Press) is the name of the most important journal devoted to the study of virtual reality (VR) and user perception. Many popular new-media writers still like to imagine a perfected form of VR as the ultimate medium for achieving a transparent form of art and entertainment. The profound difference over the notion of presence – it is not an active disagreement because the two sides do not belong to the same discourse community – reminds us of the intellectual diversity among the various groups working in digital media. The most important division among these groups is the one between theorists and practitioners. Those who design and program applications and games tend to read only the most popular and accessible media theorists. Those working in the traditional entertainment world (television and film) read new-media authors who offer new media as a ‘respectful remediation’ of film and television. Meanwhile, digital artists give us various blends of theory (art and design theory as well as critical theory) with practice. There are profound differences even among new-media academics. Specialists in human–computer interaction (HCI), who are now studying digital mediation in the workplace and in social life, generally ground their work on the literature and techniques of cognitive science or the social sciences. Communication-studies researchers have their own vast literature with both empirical and theoretical approaches, and often examine digital media from the perspective of traditional mass media. Humanists with literary or art history backgrounds are the ones most likely to bring postmodern theory into the discussion of new media. What Grusin and I have identified as the distinction between transparency and hypermediacy expresses itself in different ways in each of these communities. No author could hope to find a language that would speak to them all.

The many languages of new media I would like to bring The Language of New Media into the discussion, keeping in mind the question: what community does Manovich belong to and which communities does he address? I would characterize his book (not surprisingly) as a study in remediation, because a key argument is that 28

Jay David Bolter

Northern Lights.qxd

3/9/07

10:47 am

Page 29

digital media today borrow and extend the representational practices of avant-garde cinema of the 1920s. In his prologue, Manovich specifically relates the key qualities of digital media to those of cinema as practised by Dziga Vertov in The Man with a Movie Camera (1929). This is exactly a remediating relationship: avant-garde cinema provides an authenticity that Manovich is seeking to appropriate for digital art. In a remediating relationship the new medium not only borrows from the old, but also offers something new, and for Manovich the element that distinguishes digital media from avant-garde film (and all other media) is ‘transcoding’. As we shall see, transcoding is the process by which old media forms are transformed into software. When I claim that The Language of New Media conceives of digital art in important ways as a remediation of avant-garde film, I am not trying to diminish the scope of this work. With his commanding knowledge of the media technologies and media theory in the twentieth century, Manovich has given us the first convincing genealogy of new media. Manovich realizes that the original avant-garde offered a highly influential definition of the artist for the twentieth (and now the twenty-first) century: the task of the artist is to offer critical alternatives to the representational practices of mainstream media. In The Language of New Media Manovich shows how these avant-garde alternatives re-emerge in the interface, the operations, and the code of new media. Historians of twentieth-century visual art such as Rosalind Krauss (1985) might object to this generalization of the impact and meaning of the avant-garde (and of course this is my formulation here and not necessarily Manovich’s). Hal Foster might argue that Manovich pays too little attention to the neo-avant-garde such as Andy Warhol or the minimalists such as Frank Serra and Robert Morris (Foster 1996). For Foster, the minimalists and other artists of the 1960s constitute a second key moment in the history of the avant-garde’s questioning of established practice; in this period what was called into question was the practice of the high modernists with their abstraction and insistence on the purity and isolation of their art. But The Language of New Media is not trying to be a comprehensive history of the avant-garde in the twentieth century. Much of the art of the neo-avant-garde does not bear the same relation to its contemporary media technologies that the original avant-garde bore to film. There is one important exception, however: experimental film and video of the 1960s and 1970s, to which Manovich does not perhaps do justice. He mentions Hollis Frampton and Stan Brakhage, but he has almost nothing to say about the video art of Nam June Paik, Woody and Steina Vasulka, and others (Spielmann 2005). Yet these video artists were also exploring a technological innovation. Just as the 1920s avant-garde was responding to narrative film of its day, video artists were offering an alternative to broadcast television in theirs. According to Spielmann they were insisting that recorded video was its own medium with its own representational practices, separate from those of television. In his brilliant genealogy of the screen (Manovich 2001a: 95–115), Manovich seems interested in almost every kind of screen (film screen, radar screen, computer screen) other than the video or television screen. I suggest that an examination of video art of the mid-twentieth century as the third moment between avant-garde film of the 1920s and

NL 5 25–37 © Intellect Ltd 2007

29

Northern Lights.qxd

3/9/07

10:47 am

Page 30

digital art today would strengthen Manovich’s case. For example, the techniques of the video artists could enrich the analysis of operations in Chapter 3, as Manovich (2001a: 149–52) suggests in a short section on video compositing. I would like to comment on another question raised by Manovich’s appeal to the avant-garde: the distinction between elite and popular forms, between art and entertainment. In the second half of the twentieth century, the boundary between serious art and entertainment has blurred. High modernists, such as Clement Greenberg, one of whose most famous essays was entitled ‘Avant-Garde and Kitsch’, still believed in the superiority of elite art over consumer-driven entertainment (Greenberg 1939). Even if the claim of superiority can no longer be sustained, there is certainly a significant gap between contemporary (digital and analogue) art that grows out of the avant-garde tradition and the diverse strands of popular entertainment. No one today could fail to see the differences in representational practices and in audience reception between Manovich’s own Soft Cinema project and the traditional cinema of Spielberg or James Cameron – anymore than anyone in the 1920s would have had difficulty appreciating the differences between Chien Andalou and Chaplin’s The Gold Rush. Once again, these differences are operative in the digital world, in part because of the diverse communities pursuing the making and critiquing of digital artefacts. They become clear if we place The Language of New Media alongside a book that appeared a few years earlier and is perhaps equally influential today, Janet Murray’s Hamlet on the Holodeck (Murray 1997). Although both of these books set out to describe the possibilities of digital technology as an expressive medium, they have few intertexts in common. It is not surprising that as a literary critic Murray would draw on literary models, where Manovich draws on the visual arts. But most of Murray’s literary references are to canonical authors (Shakespeare, Dickens). The literary avant-garde does not figure prominently. Her references to visual media forms come mostly from popular film (e.g. Casablanca) or mainstream television (Star Trek). Indeed the ‘holodeck’ in her title refers to the VR machine in the television and film series Star Trek – a series that, were he alive today, Greenberg would identify as archetypal kitsch. Murray has no interest in relating digital media to the visual avantgarde. She takes it for granted that film and television are narrative media forms, because narrative forms have constituted the popular tradition in each of those media. Although the premise of her book is that the digital medium will be the new cinema, this does not mean, as it does for Manovich, that digital technology will bring forth a new avant-garde language that will fragment and reconstitute our conventional narrative strategies. Instead, for Murray digital technology will allow the viewer to insert herself seamlessly into traditional Hollywood narrative. Murray’s view of the digital medium is therefore popular and influential outside of the communities of the visual arts and media studies. She offers a vision of smooth and seamless interactive narrative that appeals to the entertainment industry itself, because she implies that those brought up in the world of mainstream film and television can simply extend their aesthetic principles into the digital realm. Murray’s case for interactive 30

Jay David Bolter

Northern Lights.qxd

3/9/07

10:47 am

Page 31

narrative is particularly convincing for those in Hollywood trying to introduce user participation into conventional broadcast television (socalled ‘enhanced TV’ or ‘interactive TV’) and those in the computer- and video-game industry who make games that repurpose film and sports. With her view of digital narrative, Murray shows us the alternative to Manovich: the mainstream against which Manovich’s avant-garde is in revolt. It is hard, for example, to imagine a conference in which Murray and Manovich would both be asked to keynote. Nevertheless, there is one important area of apparent agreement in their work – on the question of whether (new) media have essential qualities.

Essentialism and new media Essentialism remains strong in popular media theory today, where the essentialist strategy is to identify certain key properties that a medium possesses – properties that are supposed to emerge from the technology itself. These properties then serve to justify arguments about how the medium can or should be used. Essentialist arguments receive a sympathetic hearing from many in new media, especially those computer specialists who develop and advance the technology. On the other hand, those working in cultural studies have long argued against essentialism and determinism in communications technologies as elsewhere in technoscience. In the 1970s Raymond Williams offered a well-known and compelling critique of technological determinism in the work of McLuhan (Williams 1974). A straightforward reading of McLuhan does suggest an essentialist – most famously in his characterization of media as hot and cool. Film is hot, for example, because of the high resolution of the analogue photographic image; television is cool because the low-resolution video image requires the viewer’s participation to complete the picture (McLuhan 1964). Murray too is an essentialist. She identifies four ‘essential properties’ of digital environments: they are the procedural, the participatory, the spatial, and the encyclopaedic. These properties function in her view both descriptively and prescriptively (Murray 1997: 71–90). They describe how digital artefacts function, and at the same time it is the task of the digital designer to explore these qualities in their applications. For Murray, the mandate of the digital age is to develop the latent possibilities of the medium by bringing these qualities to fruition. Manovich is subtler: he seems to me to be pulled in two directions here, probably because he is so widely read both in modernism and in recent art and critical theory. The admirer of the modernist avant-garde is drawn to the notion of essential qualities to be identified and explored, but the student of postmodern art and theory is concerned to avoid a charge of determinism. It is useful to compare Manovich’s five ‘principles’ (numerical representation, modularity, automation, variability, transcoding) with Murray’s four essential properties, for the lists are quite different. At least two of Murray’s qualities are really popular cultural metaphors for the experience offered by digital media. To say that the computer is spatial or encyclopaedic is really to say that these are metaphors that we as a culture have chosen (or might choose) to characterize our experience with computers: they are the metaphors of cyberspace and the digital library. NL 5 25–37 © Intellect Ltd 2007

31

Northern Lights.qxd

3/9/07

10:47 am

Page 32

Manovich’s principles are more abstract, and his focus is on the ways in which producers (such as artists) manipulate digital technology as the material for their expressive efforts. Manovich rejects ‘participation’ as an essential characteristic: he objects to the notion of interactivity as being too broad to be useful (Manovich 2001a: 55–61). Murray and Manovich seem to agree on one foundational principle: the coded nature of computer applications. Murray calls this quality ‘procedurality’, while Manovich refers to ‘transcoding’. But even here the differences are important. Manovich’s concept is reflexive in a way that Murray’s is not. With ‘transcoding’ he is identifying a process by which not only earlier media objects but perhaps whole media forms are transformed into the new language of computer code. He suggests that the introduction of the computer as code marks a break in the history of media: This perspective [comparing properties and aesthetics of various media] is important and I am using it frequently in this book, but it is not sufficient. It cannot address the most fundamental quality of new media that has no historical precedent – programmability. Comparing new media to print, photography, or television will never tell us the whole story […]. To understand the logic of new media, we need to turn to computer science. It is there that we may expect to find the new terms, categories, and operations that characterize media that became programmable. From media studies, we move to something that can be called ‘software studies’ – from media theory to software theory. [italics in original] (Manovich 2001a: 47–48)

From my perspective, this may be the single most important passage in The Language of New Media, because it captures precisely Manovich’s claim for the remediating power of new digital media. Like other remediations, the digital must ultimately surpass its predecessors by promoting some quality that is unique to it. As we have said, for Manovich that quality is not interactivity (the most commonly made claim for the digital); it is the fact that all digital artefacts are ultimately software. The code that underlies a digital artefact is what gives it its uniqueness (‘no historical precedent’) and therefore its authenticity. In remediating avant-garde film of the 1920s, digital art not only borrows the representational practices of that period, but also reimagines them by embodying them in code. Manovich’s own recent project in the remediation of avant-garde film is called Soft Cinema to acknowledge both its roots and its uniqueness (www.manovich.net/ softcinemadomain). Manovich appears to be coming down on one side of a divide by taking what we can call the ‘code view’.

Code and interface Manovich’s insight into the relationship of new media and the avant-garde is perhaps unique among new-media theorists. However, the code view in various forms is widely shared by computer specialists who actually write applications. The rhetoric of the code view sometimes turns up among digital poets and artists as well. I would argue, however, that there is an alternative construction of the digital, widely held but less clearly articulated: an ‘interface view’. If the code is what lies beneath the surface of a digital artefact, the interface is the surface that the artefact presents to 32

Jay David Bolter

Northern Lights.qxd

3/9/07

10:47 am

Page 33

its viewer or user: the input (what the user types on the keyboard, how the user manipulates the joystick, and so on) and the output (what the user sees on the screen or in a headset and what she hears through the speakers). The interface is all that most users ever know of any application: it constitutes the digital experience for them. Over the past thirty years the emergence of more sophisticated and varied interfaces has been the critical element in the defining of new digital media forms. The graphical user interface (GUI), developed at Xerox PARC in the 1970s and perfected in the Apple computers of the 1980s, was the single most important step in our culture’s changing understanding of what the computer could be. (The development of the GUI was not predetermined: it was in part the result of the creative decisions of such developers as Englebart and Kay.) Manovich in fact devotes two chapters to exploring the digital interface that grew out of the GUI together with the operations (menu items, filters, cutting, pasting, and so on) that the digital artist carries on through that interface. Many, perhaps most, digital artists today (including Manovich himself) are really artists of the interface: that is, their concern is with the experience that the application offers to the user, not the code that constitutes the application. Every programmer knows that there are many different ways to achieve the same result: that many different sequences of code can produce the same interface and therefore the same digital experience. Many digital artists and even application designers and game designers do not program themselves, but instead trust their ideas to others to code. Furthermore, most genres of computer games are almost all interface: the algorithms that drive the play action are usually trivial or at least well known. (The great exception would be the graphics algorithms – techniques for making the graphics more responsive and more photorealistic. Yet computer graphics is a kind of coding that is completely in the service of the interface.) What matters to them is simply that the program should render the design operative – that it should deliver the graphics, the sounds, and the interactions. The code and interface views reflect different aesthetic and cultural criteria. The interface view is almost by definition hybrid, multiple, concerned with effects rather than technological essences. The interface view also suggests a focus on the relationship of the viewer (user) with the artefact, or on the triangular relationship of viewer, artefact, and artist. The code view in its purest form focuses on the program itself as a texture of symbols, and those who favour the code view in this form are drawn to a traditional aesthetics of the artefact. They speak of the beauty, simplicity, and elegance of the code (Bolter and Gromala 2006). Among new-media enthusiasts and theorists, the code view and essentialism often go together. This seems to be particularly true of those who work in the area of ‘interactive narrative’, whose research focuses on narrative ‘engines’ – that is, on algorithms and data structures that will produce a dynamic story as output. Some researchers regard the form of the output, the visual expression of the story, as secondary. Some suggest that the same engine could drive various outputs: a prose version, a screenbased video version, and (ultimately) a virtual reality. The game designer Chris Crawford’s massive Storytron project (www.storytron.com) is one example of a system in which the engine is vastly more important than the

NL 5 25–37 © Intellect Ltd 2007

33

Northern Lights.qxd

3/9/07

10:47 am

Page 34

form of the output. Michael Mateas and Andrew Stern’s interactive drama, Façade, also seems to me to privilege the code over the interface (Mateas and Stern 2005). Façade contains a sophisticated engine, which shapes the dialogue as a series of beats within a larger story arc; what the user sees on the screen are two animated figures, whose stiff gestures would fool no user into thinking that she was really sitting in the home of a couple of old friends. Façade has been praised as a step on the way to the holodeck, but its visual style is compelling and ironic, precisely because it calls into question the claim of transparency that Façade makes. Digital artists trained in the traditions of the visual arts, on the other hand, are unlikely to emphasize the code at the expense of the visual, aural, or tactile experience that their work offers. It should be no surprise that my two categories (the code and interface views) are not always neatly observed and cannot capture every aspect of varied digital designers and artists. For example, some digital artists actually make the code part of the interface – part of what the viewer/user sees and experiences. This is true of such digital poets as John Cayley, who calls some of his work ‘code poetry’ (Engberg 2005; Hayles 2002). But digital artefacts that display their code are promoting an aesthetic of hybridity, not the seamless transparency that belongs for example to interactive narrativists. They in fact form part of Manovich’s new media avant-garde. The Language of New Media seems to me to be seeking to reconcile what I am calling the code and interface views. Throughout the book, Manovich contends that the notion of software supersedes (absorbs, transcodes) the traditional notion of media. This is an argument that he also makes explicit in the essay ‘For a Post-media Aesthetics’ (www.artmargins.com/content/eview/manovich5.html). In a sense we could say that software becomes the remediation of media. Manovich’s ‘software view’ is more encompassing than the code view, for software consists of both the interface and the code. Nevertheless, by insisting that the discourse of software should now replace the traditional discourse of media, Manovich runs the risk of validating a hierarchy that the computerscience community has accepted for decades, in which the code is more important than the interface it generates. It is true, on the other hand, that new developments in the field of HCI (human–computer interaction) are gradually eroding that sense of hierarchy even within the computer-science community. The code view seems to be growing among new-media theorists at a time when it is losing ground in the field of computer science from which it emerged. Manovich wants to do justice to the hybridity of contemporary media yet without rejecting essentialism altogether; he wants to reconcile the essential principle of transcoding with a new cinematic practice that is hybrid and contingent. Chapter 5 (‘The Forms’) lays out the argument for a ‘database logic’ through which new media can offer an alternative to narrative cinema and literature. For Manovich the database represents a new media aesthetic, founded on the computational dichotomy of data and algorithm. Database logic applies even to computer games, which often only appear to have a narrative structure:

34

Jay David Bolter

Northern Lights.qxd

3/9/07

10:47 am

Page 35

This [the event-driven experience of a computer game] is another example of the general principle of transcoding discussed in the first chapter – the projection of the ontology of a computer onto culture itself […] The world is reduced to two kinds of software objects that are complementary to each other – data structures and algorithms. (Manovich 2001a: 223)

By Chapter 6, which returns to the comparison of new media and cinema, it has become clear that database cinema is a remediation of traditional cinema. Manovich wants to underline the historical significance of this move to a new media form, while avoiding the charge of determinism. Even before listing his principles in Chapter 1, he writes that: ‘Not every new media object obeys these principles. They should be considered not as absolute laws but rather as general tendencies of a culture undergoing computerization’ (Manovich 2001a: 27). So unlike Murray, who commits herself firmly to the mainstream media aesthetic (transparency) and the mainstream technophilosophy (essentialism), Manovich is aware of an interplay of dichotomies (transparency–hybridity, essentialism–contingency). I think he is seeking to promote this interplay in an atmosphere freed of technological determinism, as he notes in his culminating distinction between the narrative and database forms: Rather than trying to correlate database and narrative forms with modern media and information technologies, or deduce them from these technologies, I prefer to think of them as two competing imaginations, two basic creative impulses, two essential responses to the world. (Manovich 2001a: 233)

I do too. I would also like to suggest that narrative and database as aesthetic forms in fact align with the two representational strategies of transparency and hypermedia. Although there are narrative styles in the twentieth century that are highly sophisticated and self-referential, mainstream narrative film and the current modest attempts at digital interactive narrative are usually promoted as transparent forms, which permit the viewer/user to become lost (or ‘immersed’) in the story (or the story world). Avant-garde film as well as postmodern and digital art are almost by definition self-aware and in my terms hypermediated. Manovich identifies the classic avant-gardist Dziga Vertov and the contemporary film-maker Peter Greenaway both as practitioners of his ‘database cinema’ (Manovich 2001a: 237–43). The very fact that the database form can be expressed in more than one medium shows that Manovich does not want to insist too strongly on the power of the computer medium to determine aesthetic principles.

Theory and practice I would like to close by touching briefly on another question: the relationship of theory and practice in new-media studies. In the past decade many universities in North America and Europe have instituted new-media programmes, most of which pay at least lip service to the notion that theory

NL 5 25–37 © Intellect Ltd 2007

35

Northern Lights.qxd

3/9/07

10:47 am

Page 36

and practice should be closely combined. Because I teach in one such programme, it is fair to ask whether Remediation has succeeded in bringing together theory and practice – whether this monograph on media history and theory has practical value for designers and artists. Although I hoped this would be the case, the practical impact of Remediation has been small. I hope that some artists and designers have taken some inspiration from a book that seeks to validate the notion of working across media and denies the need for purity in digital media forms. But Remediation is analytical rather than prescriptive, and its argument is that artists and designers borrow from and refashion other media forms all the time, whether they are aware of this process or not. Their audiences receive their work in this spirit as well, consciously or unconsciously comparing the new media artefacts to those in other media that they know. In this sense, there is no need for artists and designers to change their practice. The only further argument I could make is that a conscious knowledge of the practices of remediation would help media producers to articulate what they already know intuitively. It is at least plausible that the concepts of remediation might help in the teaching of media design, and I believe the book may be read in some new-media programmes for this reason. The same question may be posed for The Language of New Media. It is a book of history and theory, not a practical manual. Manovich has had more success in crossing the gulf between theory and practice, above all because he is himself a practising artist. The book can be read as a historically sophisticated manifesto for the database art that Manovich himself creates: specifically, for his Soft Cinema project. The fact that Manovich himself can produce art that grows out of his theory makes the theory itself more convincing. References Bazin, A. (1980 [1967]), ‘The ontology of the photographic image’, in A. Trachtenberg (ed.), Classic Essays in Photography, New Haven: Leete’s Islands Books, pp. 237–44. Bloom, H. (1997), The Anxiety of Influence: A Theory of Poetry, 2nd edn., New York: Oxford University Press. Bolter, J. D., MacIntyre, B., Gandy, M. and Schweitzer, P. (2006), ‘New media and the permanent crisis of aura’, Convergence, 12: 1, pp. 21–39. Bolter, J. D. and Gromala, D. (2006), ‘Transparency and reflectivity: digital art and the aesthetics of interface design’, in P.A. Fishwick (ed.), Aesthetic Computing, Cambridge, MA: MIT Press, pp. 369–82. Engberg, M. (2005), Stepping into the River: Experiencing John Cayley’s River Island, available from http://www.dichtung-digital.com/2005/2/Engberg/index.htm. (Last accessed February 1st. 2007) Foster, H. (1996), The Return of the Real: The Avant-garde at the End of the Century, Cambridge, MA: MIT Press. Greenberg, C. (1939), ‘Avant-garde and kitsch’, Partisan Review, 6: 5, pp. 34–49. Hayles, N.K. (2002), Writing Machines, Cambridge, MA: MIT Press. Holtzman, S. (1997), Digital Mosaics: The Aesthetics of Cyberspace, New York: Simon and Schuster. Krauss, R. (1985), The Originality of the Avant-Garde and Other Modernist Myths, Cambridge, MA: MIT Press.

36

Jay David Bolter

Northern Lights.qxd

3/9/07

10:47 am

Page 37

Manovich, L. (2001a), The Language of New Media, Cambridge, MA: MIT Press. ———— (2001b), ‘For a post-media aesthetics’, Art Margins, available from http://www.artmargins.com/content/eview/manovich5.html. Mateas, M. and Stern, A. (artist) (2005), Façade: A One-Act Interactive Drama. http://forums.adventuregamers.com/showthread.php?t=2076 (Last accessed February 1st. 2007) McLuhan, M. (1964), Understanding Media: The Extensions of Man, New York: New American Library Inc. Müller, Jürgen, E. (1996), Intermedialität, Formen modener kultureller Kommunikation, Münster: Nordus Publikationen. Murray, J. (1997), Hamlet on the Holodeck, Cambridge, MA: MIT Press. Rajewsky, I. (2002), Intermedialität, Stuttgart: UTB. Spielmann, Y. (1998), Intermedialität: Das System Peter Greenaway. München: Wilhelm Fink. ———— (2005), Video: Das reflektive Medium, Frankfurt am Main: Suhrkamp Verlag. Williams, R. (1974), Television: Technology and Cultural Form, London: Fontana.

NL 5 25–37 © Intellect Ltd 2007

37

Northern Lights.qxd

3/9/07

10:47 am

Page 38

^ciZaaZXi Wdd`h bZY^V  XjaijgZ

JcYZghiVcY^c\ i]Z
lll#^ciZaaZXiWdd`h#Xdb

CZl I^iaZ

GZVa^in IK! bV`ZdkZg egd\gVbbZh VcY \VbZ h]dlh/ XjggZcian i]ZhZ XdcXZeih VgZ ^cZhXVeVWaZ# I]Z XjaijgVa dWhZhh^dc l^i] heZX^[^X IK [dgbVih ]Vh [Vh]^dcZY V [dgb d[ bZY^V ]necdi^hb ^c gZXZci nZVgh# I]Z ZmeVch^dc d[ bjai^X]VccZa k^Zl^c\ ]Vh ^gdc^XVaan egZX^e^iViZY Vc ^cXgZVhZ ^c i]Z hVbZ \ZcZg^X h]dlh# IdYVn! cjbZgdjh XdbeVc^Zh egd[^i [gdb X]jgc^c\ dji ^ciZgcVi^dcVa kVg^Vi^dch d[ i]Z hVbZ IK XdcXZei# rel="nofollow">c i]^h XdcX^hZ VcY lZaa"gZhZVgX]ZY hijYn! i]Z Vji]dgh ZmVb^cZ i]Z \adWVa iZaZk^h^dc [dgbVi Vh Vc Zci^in ^c ^ihZa[ VcY bdc^idg i]Z YZkZadebZciVa hiV\Zh [gdb XdcXZei^dc id Y^hig^Wji^dc# ;dXjh^c\ dc i]Z bVg`Zi^c\ d[ XjaijgVa YZbVcY! i]Z IK [dgbVi ^h h]dlc id ]VkZ ZkdakZY ^cid V XdbbdY^in WajZeg^ci! l]^X] ^h i]Zc ^b^iViZY! bVg`ZiZY VcY hdaY [dg bVhh Xdchjbei^dc#

^ciZaaZXi# EjWa^h]Zgh d[ dg^\^cVa i]^c`^c\# lll#^ciZaaZXiWdd`h#Xdb

^ciZaaZXi# 7dd`h  ?djgcVah ED 7dm -+'! 7g^hida 7H.. &9:! Jc^iZY @^c\Ydb dgYZgh5^ciZaaZXiWdd`h#Xdb

Northern Lights.qxd

3/9/07

10:47 am

Page 39

Northern Lights Volume 5 © 2007 Intellect Ltd Article. English language. doi: 10.1386/nl.5.1.39/1

Alan Kay’s universal media machine Lev Manovich Abstract

Keywords

While new-media theorists have spend considerable efforts in trying to understand the relationships between digital media and older physical and electronic media, the important sources – the writing and projects by Ivan Sutherland, Douglas Englebart, Ted Nelson, Alan Kay, and other pioneers working in the 1960s and 1970s – remain largely unexamined. What were their reasons for inventing the concepts and techniques that today make it possible for computers to represent, or ‘remediate’ other media? I suggest that Kay and others aimed to create a particular kind of new media – rather than merely simulating the appearances of old ones. These new media use already existing representational formats as their building blocks, while adding many new previously non-existent properties. At the same time, as envisioned by Kay, these media are expandable – that is, users themselves should be able to easily add new properties, as well as to invent new media. Accordingly, Kay calls computers the first ‘metamedium’ whose content is ‘a wide range of already-existing and not-yet-invented media’.

Simulation Remediation Metamedium New-media theory Alan Kay Xerox PARC

Medium: Definition 8. a. A specific kind of artistic technique or means of expression as determined by the materials used or the creative methods involved: the medium of lithography. b. The materials used in a specific artistic technique: oils as a medium. (American Heritage Dictionary, 4th edition) (Houghton Mifflin, 2000) The best way to predict the future is to invent it. (Alan Kay)

Appearance versus function As a result of the adoption of the GUI (graphical user interface) in the 1980s, software has replaced many other tools and technologies for the creative professional and it has also given hundreds of millions of people the ability to create, manipulate, sequence, and share media – but has this led to the invention of fundamentally new forms of culture? Today, computer scientists along with media companies are busy inventing electronic books and interactive television; consumers are happily purchasing (or downloading for free) music albums and feature films distributed in digital form, as well as making photographs and video with their digital cameras and cell phones; office workers are reading PDF documents that imitate paper. And even at the futuristic edge of digital culture – inhabited by smart objects and ambient intelligence – traditional NL 5 39–56 © Intellect Ltd 2007

39

Northern Lights.qxd

3/9/07

10:47 am

Page 40

forms persist: Philips showcases ‘smart’ household mirrors which can hold electronic notes and videos, while its Director of Research dreams about a normal-looking vase which can hold digital photographs. In short, the revolution in means of production, distribution, and access to media has not been accompanied by a similar revolution in the syntax and semantics of media. It is Alan Kay and his collaborators at PARC (the Palo Alto Research Centre, formerly ‘Xerox PARC’) that we must call to task for making digital computers imitate older media. By systematically developing easy-to-use GUI-based software to create and edit familiar media types, Kay and others appear to have locked the computer into being a simulation machine for ‘old media’. Technologies developed at PARC, such as the bitmapped colour display used as the main computer screen, laser printing, and the first page description language which eventually led to Postscript, were conceived to support the computer’s new role as a machine for the simulation of physical media. To put these developments in terms of Bolter and Grusin’s very influential book Remediation: Understanding New Media (1999), we can say that GUI-based software turned digital computers into what they might call ‘remediation machines’. Bolter and Grusin define remediation as ‘the representation of one medium in another’ (Bolter and Grusin 1999). According to their argument, new media always remediate older forms and therefore, we should not expect that computers would function any differently. This perspective emphasizes the continuity between computational media and earlier media forms. Rather than being separated by different logics, all media – including computers – follow the same logic of remediation. The only difference between computers and other media lies in how and what they remediate. As Bolter and Grusin put this in the first chapter of their book: ‘What is new about digital media lies in their particular strategies for remediating television, film, photography, and painting.’ In another place in the same chapter, they make an equally strong statement that leaves no ambiguity about their position: ‘We will argue that remediation is a defining characteristic of the new digital media.’ If we consider today all the digital media created both by consumers and by professionals – digital photography and video shot with inexpensive cameras and cell phones, the contents of personal blogs and online journals, illustrations created in Photoshop, feature films cut on AVID, etc. – in terms of its appearance, digital media indeed often looks to the casual observer exactly the same way it did before it became digital. Thus, if we limit ourselves to looking at the surfaces of media, the remediation argument accurately describes much of what goes on with computational media. But rather than accepting this condition as an inevitable consequence of the universal logic of remediation, we should ask why this is the case. In other words, if contemporary computational media imitates other media, how did this become possible? There was definitely nothing in the original theoretical formulations of digital computers by Turing or Von Neumann about computers imitating other media such as books, photography, or film. The conceptual and technical gap that separates the first room-sized computers – used by the military to calculate the shooting tables of antiaircraft guns or to crack German communication codes – versus the contemporary small desktop and laptop computers – used by ordinary 40

Lev Manovich

Northern Lights.qxd

3/9/07

10:47 am

Page 41

people to store, edit, and share media – is vast. The contemporary identity of a computer as a media processor took about forty years to emerge, if we count from 1949 when MIT’s Lincoln Laboratory started to work on its first interactive computers, to 1989 when the first commercial version of Photoshop was released. It took generations of brilliant and very creative thinkers to invent the multitude of concepts and techniques that today make it possible for computers to ‘remediate’ other media so well. What were their reasons for doing this? What was their thinking? In short, why did these people dedicate their careers to inventing the ultimate ‘remediation machine’? I cannot consider the thinking of each of the key figures in the history of media computing in the space of one article. However, we can take a closer look at one place where the identity of a computer as a ‘remediation machine’ was largely put in place – Alan Kay’s Learning Research Group at Xerox PARC in operation during the first part of the 1970s. We can ask two questions: first, what exactly did Kay want to do, and second, how did he and his colleagues set about to achieve their aims?1 The brief answer – which will be expanded below – is that Kay wanted to turn computers into a ‘personal dynamic media’ that could be used for learning, discovery, and artistic creation. His group achieved this by systematically simulating most existing media within a computer, while simultaneously adding many new properties to these media. Kay and his collaborators also developed a new type of programming language that, at least in theory, would allow users to quickly invent new types of media using the set of general tools already provided for them. All these tools and simulations of already existing media were given a unified user interface designed to activate multiple mentalities and ways of learning, including the kinaesthetic, the iconic, and the symbolic. Kay conceived of ‘personal dynamic media’ as a fundamentally new kind of media with a number of historically unprecedented properties, such as the ability to store all of the user’s information, simulate all types of media within a single machine and, as Kay and Adele Goldberg put it, ‘involve the learner in a two-way conversation’ (Kay and Goldberg 2003: 399).2 These properties enable new relationships between the user and the media she may be creating, editing, or viewing on a computer. And this is essential if we want to understand the relationships between computers and earlier media. Briefly put, while visually computational media may closely mimic other media, these media now function in fundamentally different ways. For instance, consider digital photography that often does in fact imitate the appearance of traditional photography. For Bolter and Grusin, this is an example of how digital media ‘remediates’ its predecessors. But rather than only paying attention to their appearance, let us think about how digital photographs can function. If a digital photograph is turned into a physical object in the world – an illustration in a magazine, a poster on the wall, a print on a t-shirt – it functions in the same ways as its predecessor.3 But if we leave the same photograph inside its native computer environment – which may be a laptop, a network storage system, or any computer-enabled media device such as a cell phone which allows its user to edit and move it to other devices or the Internet – it can function in ways which, in my view,

NL 5 39–56 © Intellect Ltd 2007

41

1. Kay has expressed his ideas in a few articles and a large number of interviews and public lectures. The following have been my main primary sources: Alan Kay and Adele Goldberg, ‘Personal Dynamic Media’, IEEE Computer, Vol. 10, No. 3 (March 1977). My quotes are from the reprint of this article in New Media Reader, eds. Noah Wardrip-Fruin and Nick Montfort (The MIT Press, 2003); Alan Kay, ‘The Early History of Smalltalk’, (HOPL-II/4/93/MA, 1993); Alan Kay, ‘A Personal Computer for Children of All Ages’, Proceedings of the ACM National Conference, Boston, August 1972; Alan Kay, Doing with Images Makes Symbols (University Video Communications, 1987), videotape (available at http://www.archive.org); Alan Kay, ‘User Interface: A Personal View’, p. 193, in Brenda Laurel, ed., The Art of HumanComputer Interface Design (Reading, Mass.’ AddisonWesley, 1990), pp. 191–207; David Canfield Smith et al., ‘Designing the Star user Interface’, Byte, issue 4 (1982). 2. Since the work of Kay’s group in the 1970s, computer scientists, hackers, and designers added many other unique properties. For instance, we can quickly move media around the Net and

Northern Lights.qxd

3.

4.

3/9/07

share it with millions of people using Flickr, YouTube, and other sites. However, consider the following examples of things to come: ‘Posters in Japan are being embedded with tag readers that receive signals from the user’s “IC” tag and send relevant information and free products back’ (Hoshimo 2005: 25). The emphasis in this and all following quotes from this article is mine – L.M.

10:47 am

Page 42

make it radically different from its traditional equivalent. To use a different term, we can say that a digital photograph offers its users many more affordances that its non-digital predecessor could. For example, a digital photograph can be quickly modified in numerous ways, and equally quickly combined with other images; it can be instantly moved around the world and shared with other people; and it can be inserted into a text document, or an architectural design. Furthermore, we can automatically – by running the appropriate algorithms – improve its contrast, make it sharper, and even in some situations remove blur. Note that only some of these new properties are specific to a particular media – in this case, a digital photograph (i.e. an array of pixels represented as numbers). Many other properties are shared by a larger class of media species: for instance, at the current stage of digital culture, all types of media files can be attached to an e-mail message. Still others display more general features of the current GUI paradigm (which was developed thirty years ago at PARC): for instance, the fast response of the computer to its user’s actions assure that there will be, as Kay and Goldberg put it, ‘no discernable pause between cause and effect’ (Kay and Goldberg 2003: 394). Still other features are enabled by network protocols such as TCP-IP that allow all kinds of computers and other devices to be connected to the same network. In summary, we can say that only some of the ‘new DNA’ of a digital photograph is due to its particular place of birth, i.e. inside a digital camera. Many of its other features are the result of the current paradigm of network computing in general.

‘Simulation is the central notion of the Dynabook’ While Alan Kay has articulated his ideas in a number of articles and talks, his 1977 article co-authored with one of his main PARC collaborators, computer scientist Adele Goldberg, is a particularly useful resource if we want to understand contemporary computational media. In this article, Kay and Goldberg describe the vision of the Learning Research Group at PARC in the following way: to create ‘a personal dynamic medium the size of a notebook (the Dynabook) which could be owned by everyone and would have the power to handle virtually all of its owner’s information-related needs’ (Kay and Goldberg 2003: 393).4 Kay and Goldberg ask the readers to imagine that this device ‘had enough power to outrace your senses of sight and hearing, enough capacity to store for later retrieval thousands of page-equivalents of reference materials, poems, letters, recipes, records, drawings, animations, musical scores, waveforms, dynamic simulations and anything else you would like to remember and change’ (Kay and Goldberg 2003: 394). In my view, ‘all’ in the first statement is important: it means that the Dynabook – or the computational media environment in general, regardless of the size or form of the device in which it is implemented – should support the viewing, creating , and editing of all possible media that have traditionally been used for human expression and communication. Accordingly, while separate programs to create works in different media were already in existence, Kay’s group for the first time implemented them all together within a single machine. In other words, Kay’s paradigm was not to simply create a new type of computer-based media that would 42

Lev Manovich

Northern Lights.qxd

3/9/07

10:47 am

Page 43

coexist with other physical media. Rather, the goal was to establish the computer as an umbrella, a platform for all already existing expressive artistic media, which Kay and Goldberg dub the ‘metamedium’. This paradigm changes our understanding of what media is. From Lessing’s Laocoon; or, On the Limits of Painting and Poetry (1766) to Nelson Goodman’s Languages of Art (1968), the modern discourse about media depends on the assumption that different media have distinct properties and in fact should be understood in opposition to each other. Putting all media within a single computer environment does not necessarily erase all differences in what various media can represent and how they are perceived – but it does bring them closer to each other in a number of ways. Some of these new connections were already apparent to Kay and his colleagues; others became visible only decades later when the new logic of media set in place at PARC unfolded more fully; some are perhaps still not visible to us today because they have not been given practical realization. One obvious example of such connections is the emergence of multimedia as a standard form of communication: web pages, PowerPoint presentations, multimedia artworks, mobile multimedia messages, media blogs, and other communication forms which combine media. Another is the rise of common interface conventions and tools which we use in working with different types of media regardless of their origin: for instance, a virtual camera, a magnifying lens, and, of course the omnipresent copy, cut, and paste commands.5 Yet another is the ability to map one media into another using appropriate software – images into sound, sound into images, quantitative data into a 3D shape or sound, etc. – used widely today in such areas as DJ/VJ/live cinema performance and information visualization. This situation is the direct opposite of the modernist media paradigm of the early twentieth century, which was focused on discovering the unique language of each artistic medium. All in all, it is as though different media are actively trying to reach towards each other, exchanging properties and letting each other borrow their own unique features. Alan Turing theoretically defined a computer as a machine that can simulate a very large class of other machines, and it is this simulation ability that is largely responsible for the proliferation of computers in modern society. But as I have already mentioned, neither he nor other theorists and inventors of digital computers explicitly considered that this simulation could also include media. It was only Kay and his generation that extended the idea of simulation to media – thus turning the Universal Turing Machine into a Universal Media Machine, so to speak. Accordingly, Kay and Goldberg write: ‘In a very real sense, simulation is the central notion of the Dynabook’ (Kay and Goldberg 2003: 399). When we use computers to simulate some process in the real world – the behaviour of a weather system, the processing of information in the brain, the deformation of a car in a crash – our concern is to correctly model the necessary features of this process or system. We want to be able to test how our model would behave in different conditions with different data, and the last thing we want is for the computer to introduce some new property into the model that we ourselves did not specify. In short, when we use computers as a general-purpose medium for simulation, we want this medium to be completely ‘transparent’.

NL 5 39–56 © Intellect Ltd 2007

43

5. 2.

This elevation Kieslowksi hasof the techniques of referred to Welles as particular media to a status of general interface conventions can be understood as the further unfolding of the principles developed at PARC in the 1970s. First, the PARC team specifically wanted to have a unified interface for all new applications. Second, they developed the idea of ‘universal commands’ such as ‘move’, ‘copy’, and ‘delete’. As described by the designers of the Xerox Star personal computer released in 1981, ‘MOVE is the most powerful command in the system. It is used during text editing to rearrange letters in a word, words in a sentence, sentences in a paragraph, and paragraphs in a document. It is used during graphics editing to move picture elements, such as lines and rectangles, around in an illustration. It is used during formula editing to move mathematical structures, such as summations and integrals, around in an equation’ (David Canfield Smith et al., ‘Designing the Star user interface’, Byte, 4: 1 (1982), pp. 242–82).

Northern Lights.qxd

6. 2.

3/9/07

Emphasis Kieslowksimine has – L.M. referred to Welles as

10:47 am

Page 44

But what happens when we simulate different media in a computer? In this case, the appearance of new properties may be welcome as they can extend the expressive and communicative potential of these media. Appropriately, when Kay and his colleagues created computer simulations of existing physical media – i.e. the tools for representing, creating, editing, and viewing these media – they ‘added’ many new properties. For instance, in the case of a book, Kay and Goldberg point out, It need not be treated as a simulated paper book since this is a new medium with new properties. A dynamic search may be made for a particular context. The nonsequential nature of the file medium and the use of dynamic manipulation allows a story to have many accessible points of view. (Kay and Goldberg 2003: 395)6

Kay and his colleagues also added various other properties to the computer simulation of paper documents. As Kay referred to this in another article, his idea was not to simply imitate paper but rather to create ‘magical paper’ (Kay 1999: 199). For instance, the PARC team gave users the ability to modify the fonts in a document and create new fonts. They also implemented another important idea that was already developed by Douglas Engelbart’s team in the 1960s: the ability to create different views of the same structure (I will discuss this in more detail below). And both Engelbart and Ted Nelson also already ‘added’ something else: the ability to connect different documents or different parts of the same document through hyperlinking – i.e. what we now know as hypertext and hypermedia. Engelbart’s group also developed the ability for multiple users to collaborate on the same document. This list goes on and on: e-mail in 1965, newsgroups in 1979, the World Wide Web in 1991, and so on. Each of these new properties has far-reaching consequences. Take ‘search’, for instance. Although the ability to search through a page-long text document does not sound like a very radical innovation, as the document gets longer, this ability becomes more and more important. It becomes absolutely crucial if we have a very large collection of documents – such as all the web pages available today. Although current search engines are far from being perfect and new technologies will continue to evolve, imagine how different the culture of the Web would be without them. Or take the capacity to collaborate on the same document(s) by a number of users connected to the same network. While it was already widely used by companies in the 1980s and 1990s, it was not until early 2000s that the larger public saw the real cultural potential of this ‘addition’ to print media. By harvesting the small amounts of labour and expertise contributed by a large number of volunteers, social software projects – most famously, Wikipedia – created vast and dynamically updatable pools of knowledge which would be impossible to create in traditional ways. In a less visible way, every time we do a search on the Web and then click on some of the results, we also contribute to a knowledge set used by everybody else – since in deciding the sequence in which to present the results of a particular search, Google’s algorithms take into account those among the results of previous searches for the same words people found most useful. 44

Lev Manovich

Northern Lights.qxd

3/9/07

10:47 am

Page 45

Studying the writings and public presentations of the people who invented interactive media computing – Sutherland, Engelbart, and Kay – makes it clear that they did not come with new properties of computational media as an afterthought. On the contrary, they knew that they were turning physical media into new media. In 1968, Engelbart gave his famous demo at the Fall Joint Computer Conference in San Francisco before a few thousand people that included computer scientists, IBM engineers, people from other companies involved in computers, and funding officers from various government agencies (Waldrop 2001: 287). Although Engelbart had ninety minutes, he had a lot to show. Over the few previous years, his team at the Research Centre for Augmenting Human Intellect had essentially developed the modern office environment as it exists today (not be confused with the modern media design environment which was developed later at PARC). Their computer system included word processing with outlining features, documents connected through hypertext, online collaboration (i.e. two people at remote locations working on the same document in real time), online user manuals, an online project planning system, and other elements of what is now commonly called the ‘computer-supported collaborative work’. This team also developed the key elements of the modern user interface that were later refined at PARC: the mouse and multiple windows. Paying attention to the sequence of this demo reveals that while Engelbart had to make sure that his audience would be able to relate the new computer system to what they already knew and could use, his focus was on a completely new set of features available in computer-simulated media. Engelbart devotes the first segment of the demo to word processing, but as soon as he briefly demonstrated text entry, cut, paste, insert, naming, and saving files – in other words, the set of tools that make a computer into a more versatile typewriter – he then goes on to show at more length the features of his system which no writing medium had before: ‘view control’.7 As Engelbart points out, the new writing medium could switch at the user’s wish between many different views of the same information. A text file could be sorted in different ways. It could also be organized as a hierarchy with a number of levels that can be collapsed and expanded – like the outline tools included in modern word processors such as Microsoft Word. In his demo, Engelbart next shows another example of view control, which today, forty years later, is still not available in popular document management software. He makes a long ‘to do’ list and organizes it by locations. He then instructs the computer to display these locations as a visual graph (i.e. a set of points connected by lines). In front of our eyes, representation in one medium changes into another medium – text becomes a graph. But this is not all. The user can control this graph to display different amounts of information – something that no image in physical media can do. As Engelbart clicks on different points in a graph corresponding to particular locations, the graph shows the appropriate part of his ‘to do’ list. This ability to interactively change how much and what information an image shows is particularly important in today’s information visualization applications. Next, Engelbart presents ‘a chain of views’ which he prepared beforehand. He switches between these views using ‘links’, which may

NL 5 39–56 © Intellect Ltd 2007

45

7. 2.

The complete Kieslowksi hasvideo of Douglas referred to Welles as Engelbart’s 1968 demo is available at http://sloan.stanford. edu/MouseSite/1968 Demo.html.

Northern Lights.qxd

2.

3/9/07

Kieslowksi has referred to Welles as

10:47 am

Page 46

look like hyperlinks the way they exist on the Web today – but they actually have a different function. Instead of creating a path between many different documents à la Vannevar Bush’s Memex (often seen as the precursor to modern hypertext), Engelbart is using links as a method for switching between different views of a single document organized hierarchically. He brings up a line of words displayed in the upper part of the screen; when he clicks on these words, more detailed information is displayed in the lower part of the screen. This information can, in its turn, contain links to other views that show even more detail. In this way, rather than using links to drift through the textual universe associatively and ‘horizontally’, we move ‘vertically’ between general and more detailed information. Appropriately, in Engelbart’s paradigm, we are not ‘navigating’ – we are ‘switching views’. We can create many different views of the same information, and switch between these views in different ways. And this is what Engelbart systematically explains in this first part of his demo. He demonstrates that one can change views by issuing commands, by typing numbers that correspond to different parts of a hierarchy, by clicking on parts of a picture, or on links in the text. Since new-media theory and criticism emerged in the early 1990s, endless texts have been written about interactivity, hypertext, virtual space, cyberspace, cyborgs, and so on. But I have never seen anyone discuss ‘view control’. And yet this is one of the most fundamental and radical new techniques for working with information and media available to us today. ‘View control’, i.e. the ability to switch between many different views and kinds of views of the same information, is now implemented in multiple ways not only in word processors and e-mail clients, but also in all ‘media processors’ (i.e. media editing software) such as AutoCAD, Maya, After Effects, Final Cut, Photoshop, InDesign, and so on. For instance, in the case of 3D software, it can usually display the model in at least half a dozen different ways: in wireframe, fully rendered, etc. In the case of animation and visual-effects software, since a typical project may contain dozens of separate objects, each having dozens of parameters, it is often displayed in a way similar to how outline processors can show text. In other words, the user can switch between more and less information. One can choose to see only those parameters on which the user is working at present. One can also zoom in and out of the composition. When this is done, parts of the composition do not simply get smaller or bigger – they show less or more information automatically. For instance, at a certain scale, the user may only see the names of different parameters; but when one zooms into the display, the program may also display the graphs that indicate how these parameters change over time. As we can see from the examples analysed above, the aim of the inventors of computational media – Engelbart, Nelson, Kay, and colleagues with whom they have worked – was not to simply create accurate simulations of physical media. Instead, in every case, the goal was to create ‘a new medium with new properties’ which would allow people to communicate, learn, and create in new ways. So while today, the content of these new media may often look the same as its predecessors, we should not be fooled by this similarity. The newness lies not in the content but in the software tools used to create, edit, view, distribute, and share this 46

Lev Manovich

Northern Lights.qxd

3/9/07

10:47 am

Page 47

content. Therefore, rather than only looking at the ‘output’ of softwarebased cultural practices, we need to consider the software itself – since it allows people to work with media in a number of historically unprecedented ways. To summarize: while on the level of appearance, computational media indeed often remediate (i.e. re-presents) previous media, the software environment in which this media ‘lives’ is very different. Let me add one more item to the examples discussed above – Ivan Sutherland’s ‘Sketchpad’ from 1962. Created by Sutherland as a part of his Ph.D. thesis at MIT, Sketchpad deeply influenced all subsequent work in computational media (including that of Kay) not only because it was the first interactive media-authoring program, but also because it made it clear that computer simulations of physical media can add many exciting new properties to the media being simulated. Sketchpad was the first software that allowed its users to interactively create and modify line drawings. As Noah Wardrip-Fruin points out, it ‘moved beyond paper by allowing the user to work at any of 2000 levels of magnification – enabling the creation of projects that, in physical media, would either be unwieldly large or require detailed work at an impractically small size’ (Wardrip-Fruin 2003:109). Sketchpad similarly redefined graphical elements of a design as objects which ‘can be manipulated, constrained, instantiated, represented ironically, copied, and recursively operated upon, even recursively merged’ (Wardrip-Fruin 2003:109).. For instance, if the designer defines new graphical elements as instances of a master element and later makes a change to the master, all these instances would also change automatically. Another new property that perhaps demonstrates most dramatically how computer-aided drafting and drawing differed from their physical counterparts was Sketchpad’s use of constraints. In Sutherland’s own words, ‘The major feature which distinguishes a Sketchpad drawing from a paper and pencil drawing is the user’s ability to specify to Sketchpad mathematical conditions on already drawn parts of his drawing which will be automatically satisfied by the computer to make the drawing take the exact shape desired’ (Sutherland [1963] 2003). For instance, if a user drew a few lines, and then gave the appropriate command, Sketchpad automatically moves these lines until they are parallel to each other. If a user gives a different command and selects a particular line, Sketchpad moves the lines in such a way that they are parallel to each other and perpendicular to the selected line. Although we have not exhausted the list of new properties that Sutherland built into Sketchpad, it should be clear that this first interactive graphical editor was not only simulating existing media. Appropriately, Sutherland’s 1963 paper on Sketchpad repeatedly emphasizes the new graphical capacities of his system, marvelling how it opens new fields of ‘graphical manipulation that has never been available before’ (Sutherland [1963] 2003: 123). The very title Sutherland gives to his Ph.D. thesis foregrounds the novelty of his work: Sketchpad: A Man-Machine Graphical Communication System. Rather than conceiving of Sketchpad as simply another medium, Sutherland presents it as something else – a communication system between two entities: the human and the intelligent machine. Kay and Goldberg will later also foreground this dimension of

NL 5 39–56 © Intellect Ltd 2007

47

2.

Kieslowksi has referred to Welles as

Northern Lights.qxd

2.

3/9/07

Kieslowksi has referred to Welles as

10:47 am

Page 48

communication by referring to it as ‘a two-way conversation’ and calling the new ‘metamedium’ ‘active’ (Kay and Goldberg 2003: 394). We can also think of Sketchpad as a practical demonstration of J.C. Licklider’s idea of the ‘man-machine symbiosis’ applied to image-making and design (Licklider [1960] 2003).

Permanent extendibility As we saw, Sutherland, Nelson, Engelbart, Kay, and other pioneers of computational media have added many previously non-existent properties to media that they have simulated in a computer. The subsequent generations of computer scientists, hackers, and designers have added many more properties – but this process is far from finished. And there is no logical or material reason why it will ever be finished. To add new properties to physical media requires modifying its physical substance. But since computational media exists as software, we can add new properties or even invent new types of media simply by changing existing, or writing new, software. Adding plug-ins and extensions, as programmers have been doing with Photoshop and Firefox, is another way to innovate. One can also combine existing software together. For instance, at the moment of this writing in 2007, programmers keep extending the capacities of mapping media by creating software mash-ups which combine the services and data provided by Google Maps, Flickr, Amazon, other sites, and media uploaded by users. In short, ‘new media’ is ‘new’ because new properties (i.e. new software techniques) can always be easily added to it. Put differently, in industrial, i.e. mass-produced media technologies, ‘hardware’ and ‘software’ were one and the same thing. For example, the pages of a book were bound in a particular way that fixed the order of pages. The reader could change neither the order nor the level of detail displayed à la Engelbart’s ‘view control’. Similarly, the film projector combined hardware and what we now call ‘media player’ software into a single machine. In the same way, the controls built into the twentieth-century mass-produced camera could not be modified at the user’s will. And although today the user of a digital camera similarly cannot easily modify the hardware of her camera, as soon as she transfers the pictures into a computer, she has access to an endless number of controls and options for modifying her pictures via software. In the nineteenth and twentieth centuries, there were two types of situations when the normally rigidly fixed industrial media was more less fixed. The first type of situation was when a new media was being first developed: for instance, the invention of photography in the 1820s–1840s. The second type of situation was when artists would systematically experiment with and ‘open up’ already industrialized media – such as the experiments with film and video during the 1960s, which came to be called ‘Expanded Cinema’. What used to be separate moments of experimentations with media during the industrial era became the norm in software society. In other words, computers have legitimized experimentation with media. Why is this so? What differentiates a modern digital computer from any other machine – including industrial media machines for capturing and playing media – is 48

Lev Manovich

Northern Lights.qxd

3/9/07

10:47 am

Page 49

the separation of hardware and software. It is because an endless number of different programs performing different tasks can be written to run on the same type of machine, this machine – i.e. a digital computer – is used so widely today. Consequently, the constant invention of new and the modification of existing media software are simply two examples of this general principle. In other words, experimentation is a default feature of computational media. In its very structure, it is ‘avant-garde’ since it is constantly being extended and thus redefined. If in modern culture ‘experimental’ and ‘avant-garde’ are opposed to the normalized and the stable, this opposition has largely disappeared in software culture. And the role of the media avant-garde is performed no longer by individual artists in their studios, but by a variety of players, from very big to very small – from companies such as Microsoft, Adobe, and Apple, to independent programmers, hackers, and designers. But this process of the continual invention of new algorithms does not move in just any direction. If we look at contemporary media software – CAD, computer drawing and painting, image editing, audio and video remixing, word processing – we see that most of their fundamental principles were already developed by the generation of Sutherland and Kay. As new techniques continue to be invented, they are layered over the foundations that were gradually put in place by Sutherland, Engelbart, Kay and others in the 1960s and 1970s. Of course, we are not dealing here only with the history of ideas. Social and economic factors – such as the dominance of the media software market by a handful of companies, or the wide adoption of particular file formats –– also constrain possible directions of software evolution. Put differently, today, software development is an industry and as such it is constantly balanced between stability and innovation, standardization and exploration of new possibilities. But it is not just any industry. New programs can be written and existing programs can be extended and modified (if the source code is available) by anybody who has programming skills and access to a computer, a programming language, and a compiler. In other words, today, software is fundamentally modifiable in a way that physical industrially produced objects usually are not. Although Turing and Von Neumann already formulated this fundamental extendibility of software in theory, its contemporary practice – thousands of people daily involved in extending the capabilities of computational media – is a result of a long historical development. This development took us from the few early room-sized computers that were not easy to reprogram, to a wide availability of cheap computers and programming tools decades later. Such democratization of software development was at the core of Kay’s vision. Kay was particularly concerned with how to structure programming tools in such a way that would make development of media software possible for ordinary users. For instance, at the end of the 1977 article I have already been extensively quoting, Kay and Goldberg write: ‘We must also provide enough already-written general tools so that a user need not start from scratch for most things she or he may wish to do.’ Comparing the process of continuous media innovation via new software to the history of earlier, pre-computational media reveals a new logic at work. According to a commonplace idea, when a new medium is

NL 5 39–56 © Intellect Ltd 2007

49

2.

Kieslowksi has referred to Welles as

Northern Lights.qxd

8. 2.

3/9/07

We should also Kieslowksi has mention theWelles work on referred to as SAGE by MIT Lincoln Laboratory, which by the middle of the 1950s, had already established the idea of interactive communication between a human and a computer via a screen with a graphical display and a pointing device. In fact, Sutherland developed Sketchpad on the TX-2, which was the new version of a larger computer MIT had constructed for SAGE.

10:47 am

Page 50

invented, it first closely imitates already existing media before discovering its own language and aesthetics. Indeed, the Bibles first printed by Gutenberg closely imitated the look of handwritten manuscripts; early films produced in the 1890s and 1900s mimicked the presentational format of theatre by positioning the actors on an invisible shallow stage and having them face the ‘audience’ represented by the fixed camera. Slowly, printed books developed a different way of presenting information; similarly cinema also developed its own original concept of narrative space. Through repetitive shifts in points of view presented in subsequent shots, the viewers were placed inside this space – thus literally finding themselves inside the story. Can this logic apply to the history of computer media? As theorized by Turing and Von Neumann, the computer is a general-purpose simulation machine. This is its uniqueness and its difference from all other machines and previous media. This means that the commonplace idea – that a new medium gradually finds its own language cannot apply to computer media. If this were true, it would go against the very definition of the modern digital computer. This theoretical argument is supported by practice. The history of computer media so far has not been about arriving at some standardized language – the way this, for instance, happened with cinema – but rather, it seems to be about the gradual expansion of uses, techniques, and possibilities. Rather than arriving at a particular language, we are gradually discovering that the computer can speak more and more languages. If we are to look more closely at the early history of computer media – for instance, the way we have been looking at Kay’s ideas and work in this text – we will discover another reason why the idea of a new medium gradually discovering its own language does not apply to computer media. The systematic practical work on making a computer simulate and extend existing media (e.g. Sutherland’s Sketchpad, the first interactive word processor developed by Engelbart’s group, etc.) came after computers were already put to multiple uses – performing different types of calculations, solving mathematical problems, controlling other machines in real time, running mathematical simulations, simulating some aspects of human intelligence, and so on.8 Therefore, when the generation of Sutherland, Nelson, and Kay started to create ‘new media’, they built it on top of, so to speak, what computers were already known to be capable of. Consequently, they added new properties into the physical media they were simulating right away. This can be very clearly seen in the case of Sketchpad. Understanding that one of the roles a computer can play is that of a problem solver, Sutherland built in a powerful new feature that never before existed in a graphical medium – satisfaction of constraints. To rephrase this example in more general terms, we can say that rather than moving from an imitation of older media to finding its own language, computational media was from the very beginning speaking a new language. In other words, the pioneers of computational media did not have the goal of making the computer into a ‘remediation machine’ that would simply represent older media in new ways. Instead, well knowing the new capabilities provided by digital computers, they set out to create fundamentally new kinds of media for expression and communication. 50

Lev Manovich

Northern Lights.qxd

3/9/07

10:47 am

Page 51

These new media would use as their raw ‘content’ the older media, which already served humans well for hundreds and thousands of years – written language, sound, line drawings and design plans, and continuous tone images like paintings and photographs. But this does not compromise the newness of new media. For computational media uses these traditional human media simply as building blocks to create previously unimaginable representational structures, creative and thinking tools, and communication options. Although Sutherland, Engelbart, Nelson, Kay, and others developed computational media on top of already existing developments in computational theory, programming languages, and computer engineering, it would be incorrect to conceive the history of such influences as only going in one direction – from already existing and more general computing principles to particular techniques of computational media. The inventors of computational media have had to question many, if not most of the already established ideas about computing. They have defined many new fundamental concepts and techniques of how both software and hardware work, thus making important contributions to hardware and software engineering. A good example is Kay’s development of Smalltalk, which for the first time systematically established a paradigm of object-oriented programming. Kay’s rationale in developing this programming language was to give a unified appearance to all applications and the interface of the PARC system and, even more importantly, to enable its users to quickly program their own media tools. Kay cites an interesting example in which an object-oriented illustration program written in Smalltalk by a particularly talented 12-year-old girl was only a page long (Kay 1987: v). Subsequently, the object-oriented programming paradigm became very popular and object-oriented features have been added to most popular languages such as C++ and Java. Looking at the history of computer media and examining the thinking of its inventors makes it clear that we are dealing with the opposite of technological determinism. When Sutherland designed Sketchpad, and Nelson conceived hypertext, or Kay programmed a paint program, each new property of computer media had to be imagined, implemented, tested, and refined. In other words, these characteristics did not simply come as an inevitable result of a meeting between digital computers and modern media. Computational media had to be invented, step-by-step. And it was invented by people who were looking for inspiration in modern art, literature, cognitive and educational psychology, and theory of media as much as technology. For example, Kay recalls that reading McLuhan’s Understanding Media led him to a realization that the computer can be a medium rather than a mere tool (Kay 1990: 192–93). So far, I have talked about the history of computational media as a series of consecutive ‘additions’. However, this history is not only a process of the accumulation of ever more options. Although, in general, we have more techniques at our disposal today than twenty or thirty years ago, it is also important to remember that many fundamentally new techniques were conceived but never given commercial implementation, or were poorly implemented and did not become popular. Or perhaps they were not marketed properly. Sometimes the company making the software might go

NL 5 39–56 © Intellect Ltd 2007

51

2.

Kieslowksi has referred to Welles as

Northern Lights.qxd

2.

3/9/07

Kieslowksi has referred to Welles as

10:47 am

Page 52

out of business. At other times, the company that created the software might be purchased by another company that in turn would ‘shelve’ the software so it would not compete with its own products. And so on. In short, the reasons why many new techniques have not become commonplace are multiple, and are not reducible to a single principle such as ‘the most easy to use techniques become most popular’. For instance, one of the ideas developed at PARC was ‘project views’. Each view, according to Kay, ‘holds all the tools and materials for a particular project and is automatically suspended when you leave it’ (Kay 1990: 200). Although currently, in 2007, there are some applications that implement this idea, it is not a part of most popular operating systems, that is, Windows, MAC OS X, and Linux. The same holds true for the contemporary World Wide Web implementation of hyperlinks. The links on the Web are static and one-directional. Ted Nelson, who is credited with inventing hypertext around 1964, conceived it from the beginning as having a variety of other link types. In fact, when Tim Berners-Lee submitted his paper about the Web to the ACM Hypertext 1991 conference, his paper was only accepted for a poster session rather than the main conference program. The reviewers saw his system as being inferior to many other hypertext systems that were already developed in the academic world over the previous two decades (Wardrip-Fruin and Montford 2003)..

Computer as metamedium As we have established, the development of computational media runs contrary to previous media history. But in a certain sense, the idea of a new media gradually discovering its own language actually might apply to the history of computational media after all. And just as it was the case with printed books and cinema, this process has taken a few decades. When the first computers were built in the middle of the 1940s, they could not be used as media for cultural representation, expression, and communication. Slowly, through the work of Sutherland, Engelbart, Nelson, Papert, and others in the 1960s, the ideas and techniques were developed which made computers into a ‘cultural machine’. One could create and edit text, make drawings, move around virtual objects, etc. And finally, when Kay and his colleagues at PARC systematized and refined these techniques, placing them under the umbrella of GUI, which made computers accessible to multitudes, a digital computer was finally, in cultural terms, given its own language. Or rather, it became something that no other media has been before. For what has emerged was not yet another medium but, as Kay and Goldberg insist in their article, something qualitatively different and historically unprecedented. To mark this difference, they introduce a new term – ‘metamedium’. This metamedium is unique in a number of different ways. One of them we already discussed in detail – it could represent most other media while augmenting them with many new properties. Kay and Goldberg also name other properties that are equally crucial. The new metamedium is, they assert, ‘active – it can respond to queries and experiments – so that the messages may involve the learner in a two-way conversation’. For Kay, who was strongly interested in children and learning, this property was 52

Lev Manovich

Northern Lights.qxd

3/9/07

10:47 am

Page 53

particularly important since, as he and Goldberg puts it, it ‘has never been available before except through the medium of an individual teacher’ (Kay and Goldberg 2003: 394). Further, as noted above, the new metamedium can handle ‘virtually all of its owner’s information-related needs’. It can also ‘serve as a programming and problem solving tool’, and ‘an interactive memory for the storage and manipulation of data’ (Ibid.: 393). But the property that is the most important from the point of view of media history is that the computer metamedium is simultaneously a set of different media and a system for generating new media tools and new types of media. In other words, a computer can be used to create new tools for working in the media it already provides, as well as to develop new not-yet-invented media. Using the analogy with print literacy, Kay motivates this property in the following way: ‘The ability to “read” a medium means you can access materials and tools generated by others. The ability to write in a medium means you can generate materials and tools for others. You must have both to be literate’ [original emphasis] (Kay 1990: 193). Accordingly, Kay’s key effort at PARC was the development of the Smalltalk programming language. All media-editing applications and GUI itself were written in Smalltalk. This made all the interfaces of all applications consistent, facilitating the quick learning of new programs. Even more importantly, according to Kay’s vision, the Smalltalk language would allow even beginning users to write their own tools and define their own media. In other words, all media-editing applications, which would be provided with a computer, were to serve also as examples, inspiring users to modify them and to write their own applications. Accordingly, a large part of Kay and Goldberg’s paper is devoted to a description of software developed by the users of their system: ‘an animation system programmed by animators’, ‘a drawing and painting system programmed by a child’, ‘a hospital simulation programmed by a decision-theorist’, ‘an audio animation system programmed by musicians’, ‘a musical score capture system programmed by a musician’, ‘an electronic circuit designed by a high school student’. As can be seen from this list that corresponds to the sequence of examples in the article, Kay and Goldberg deliberately juxtapose different types of users – professionals, high-school students, and children – in order to show that everybody can develop new tools using the Smalltalk programming environment. This sequence of examples also strategically juxtaposes media simulations with other kinds of simulations in order to emphasize that the simulation of media is only a particular case of computer’s general ability to simulate all kinds of processes and systems. This juxtaposition of examples gives us an interesting way to think about computational media. Just as a scientist may use a simulation to test different conditions and play different what/if scenarios, a designer, a writer, a musician, a film-maker, or an architect working with computer media can quickly ‘test’ different creative directions in which the project can be developed, as well as see how modifications of various ‘parameters’ might affect the project. The latter option is particularly easy today, since the interfaces of most mediaediting software not only explicitly present these parameters, but also simultaneously give the user the controls for their modification. For

NL 5 39–56 © Intellect Ltd 2007

53

2.

Kieslowksi has referred to Welles as

Northern Lights.qxd

9. 2.

3/9/07

http://www. Kieslowksi has processing.org. referred to Welles as 10 http://www. processing.org/ reference/environment/. 11 http://www. processing.org/faq/.

10:47 am

Page 54

instance, when the formatting palette in Microsoft Word shows the font used by the currently selected text, it is displayed in a column next to all the other fonts available. Trying a different font is as easy as scrolling down and selecting the name of a new font. To give users the ability to write their own programs was a crucial part of Kay’s vision for the new ‘metamedium’ he was inventing at PARC. According to Noah Wardrip-Fruin, Engelbart’s research program was focused on a similar goal: ‘Engelbart envisioned users creating tools, sharing tools, and altering the tools of others’ (Wardrip-Fruin and Montfort 2003: 232). Unfortunately, when Apple shipped the first Macintosh in 1984, which was to become the first commercially successful personal computer modelled after PARC system, it did not have an easy-to-use programming environment. HyperCard, written for the Macintosh in 1987 by Bill Atkinson (who was a PARC alumni) gave users the ability to quickly create certain kinds of applications – but it did not have the versatility and breadth envisioned by Kay. Only recently, as general computer literacy has widened and many scripting languages have become available – Perl, PHP, Python, ActionScript, Vbscript, JavaScript, etc. – more people have started to create their own tools by writing software. A good example of a contemporary programming environment – which is currently very popular among artists and designers and which, in my view, is close to Kay’s vision – is Processing.9 Built on top of the Java programming language, Processing features a simplified programming style and an extensive library of graphical and media functions. It can be used to develop complex media programs and also to quickly test ideas. Appropriately, the official name for Processing projects is ‘sketches’.10 In the words of Processing initiators and main developers Ben Fry and Casey Reas, the language focuses ‘on the “process” of creation rather than end results’.11 Another popular programming environment that similarly enables the quick development of media software is MAX/MSP and its successor PD developed by Miller Puckette.

Conclusion The story I have just related could also be told differently. It is possible to put Sutherland’s work on Sketchpad in the centre of computational media history; or Engelbart and his Research Centre for Augmenting Human Intellect, which throughout the 1960s, developed hypertext (independently of Nelson), the mouse, the window, the word processor, mixed text/graphics displays, and a number of other ‘firsts’. Or we could shift focus to the work of the Architecture Machine Group at MIT, which since 1967 was headed by Nicholas Negroponte (and in 1985 became known as The Media Lab). We also need to recall that by the time Kay’s Learning Research Group at PARC fleshed out the details of GUI and programmed various media editors in Smalltalk (a paint program, an illustration program, an animation program, etc.), artists, film-makers, and architects were already using computers for more than a decade and a number of large-scale exhibitions of computer art were displayed in major museums such as the Institute of Contemporary Art, London; the Jewish Museum, New York; and the Los Angeles County Museum of Art. And certainly, in terms of advancing techniques for visual representation enabled by 54

Lev Manovich

Northern Lights.qxd

3/9/07

10:47 am

Page 55

computers, other groups of computer scientists had already made important advancements. For instance, at the University of Utah, which became the main place for computer-graphics research during the first part of the 1970s, scientists were producing 3D computer graphics much superior to the simple images that could be created on the computers being built at PARC. Next to the University of Utah, a company called Evans and Sutherland (headed by the same Ivan Sutherland who was also teaching at University of Utah) was already using 3D graphics for flight simulators – essentially pioneering the type of new media that is now called ‘navigable 3D virtual space’.12 The reason I decided to focus on Kay is his theoretical formulations that place computers in relation to other media and media history. While Vannevar Bush, J.C. Licklider, and Douglas Engelbart were primarily concerned with augmentation of intellectual and, in particular, scientific work, Kay was equally interested in computers as ‘a medium of expression through drawing, painting, animating pictures, and composing and generating music’ (Wardrip-Fruin and Montfort 2003: 393). Therefore, if we really want to understand how and why computers were redefined as a cultural media, and how the new computational media is different from earlier physical media, I think that Kay provides us with the best perspective. At the end of the 1977 article that served as the basis for our discussion, he and Goldberg summarize their arguments in the phrase, which in my view is the best formulation we have so far of what computational media is artistically and culturally. They call the computer ‘a metamedium’ whose content is ‘a wide range of already-existing and not-yet-invented media’. In another article published in 1984, Kay unfolds this definition. By way of conclusion, I would like to quote this longer definition, which is as accurate and inspiring today as it was when Kay wrote it: It [a computer] is a medium that can dynamically simulate the details of any other medium, including media that cannot exist physically. It is not a tool, though it can act like many tools. It is the first metamedium, and as such it has degrees of freedom for representation and expression never before encountered and as yet barely investigated. [original emphasis] (Kay 1984: 41). References http://www.research.philips.com/newscenter/pictures/display-mirror.html. Accessed 20 January 2005. Bolter, Jay and Grusin, Richard (1999), Remediation: Understanding New Media. Cambridge, MA: The MIT Press. Canfield Smith, David et al. (1982), ‘Designing the Star user interface’, Byte, 4: 1, pp. 242–82. Engelbart, Douglas C. (1968), Live public demonstration of the NLS at the Fall Joint Computer Conference, Convention Center in San Francisco, available online at http://sloan.stanford.edu/MouseSite/1968Demo.html. Accessed 18 April 2007. Gassee, Jean-Louis and Rheingold, Howard (1991), ‘The evolution of thinking tools’, in Brenda Laurel (ed.), The Art of Human-Computer Interface Design, Boston, MA: Addison-Wesley Professional.

NL 5 39–56 © Intellect Ltd 2007

55

12. For more on 2. Kieslowksi has3D virtual navigable referred to Welles as space as a new media, or a ‘new cultural form’, see chapter on ‘Navigable Space’ in The Language of New Media.

Northern Lights.qxd

2.

3/9/07

Kieslowksi has referred to Welles as

10:47 am

Page 56

Hoshimo, Takashi (2005), ‘Bloom time out east’, ME: Mobile Entertainment, 9 (November), p. 25, available online from http://www.mobile-ent.biz. Accessed 18 April 2007. Kay, Alan (1972), ‘A personal computer for children of all ages’, Proceedings of the ACM National Conference, Boston, August. Kay, Alan (1984), ‘Computer Software’, Scientific American, 251(3):41-47, September 1984. ———— (1987), Doing with Images Makes Symbols, videotape, University Video Communications, available from http://www.archive.orgAccessed 18 April 2007. ———— (1990), ‘User interface: A personal view’, in Brenda Laurel (ed.), The Art of Human-Computer Interface Design, Boston, MA: Addison-Wesley Professional, pp. 191–207. ———— (1993), ‘The early history of Smalltalk’, (HOPL-II/4/93/MA). http://gagne.homedns.org/~tgagne/contrib/EarlyHistoryST.html Accessed 18 April 2007. Kay, Alan and Goldberg, Adele ([1977] 2003), ‘Personal dynamic media’, in Noah Wardrip-Fruin and Nick Montfort (eds), New Media Reader, Cambridge, MA: The MIT Press, pp. 391-404. Licklider, J.C. ([1960] 2003), ‘Man–machine symbiosis’, in Noah Wardrip-Fruin and Nick Montfort (eds), New Media Reader, Cambridge, MA: The MIT Press. Manovich, Lev (2001), The Language of New Media, Cambridge, MA: The MIT Press. Processing programming language, available from http://www.processing.org, http://www.processing.org/reference/environment/ and http://processing.org/faq/. Accessed 20 January 2005. Sutherland, Ivan ([1963] 2003), ‘Sketchpad: a man-machine graphical communication system’, in Noah Wardrip-Fruin and Nick Montfort (eds), New Media Reader, Cambridge, MA: The MIT Press. Waldrop, M. Mitchell (2001), The Dream Machine: J.C.R. Licklider and the Revolution That Made Computing Personal, London: Viking. Wardrip-Fruin, Noah and Nick Montfort (eds) (2003), New Media Reader, Cambridge, MA: The MIT Press.

56

Lev Manovich

Northern Lights.qxd

3/9/07

10:47 am

Page 57

Northern Lights Volume 5 © 2007 Intellect Ltd Article. English language. doi: 10.1386/nl.5.1.57/1

Convergence by means of globalized remediation Arild Fetveit Abstract

Keywords

Prophecies of media convergence have been a key component of recent discussions of digitalization. It has been claimed that the manipulability of digital data facilitates convergence and allows an erosion of differences between media. This article questions these assumptions by showing how there are also considerable obstacles against such manipulability, as well as against the erasure of differences between media. By examining empirical developments, as well as arguments put forward by theorists like Friedrich Kittler, Jay Bolter, Lev Manovich and Rosalind Krauss, it is maintained that what we are seeing is more a proliferations of media than a convergence leading to their unification. In part, this is due to our affection for a multiplicity of media. However, one way in which media do become similar is by increasingly being remediated on a digital platform. Thereby they become subject to a globalization effect by having their functionalities augmented by basic traits of the computer.

Convergence Remediation Digitalization Post-medium condition New media Media aesthetics

‘[w]hat used to be cinema’s defining characteristics have become just the default options, with many others available.’ (Lev Manovich, The Language of New Media, 2001: 293)

In the introduction to Grammophone, Film, Typewriter, the German media theorist Friedrich Kittler stipulates a convergence, which will erase previous differences among media. He writes: Once movies and music, phone calls and texts reach households via optical fiber cables, the formerly distinct media of television, radio, telephone and mail converge, standardized by transmission frequencies and bit information […]. The general digitalization of channels and information erases the differences among individual media. (Kittler [1986] 1999: 1)

The gist of Kittler’s prediction of twenty years ago still seems valid in important ways. Reports on media convergence have become a daily feature in the news. We hear about Yahoo expanding their services towards television and mobile phones, about Motorola making a wireless phone featuring a customized Google search service, about Apple’s new iPhone, which is a combination of mobile phone, video iPod, and Internet device, and so on. Media as well as media companies migrate ever more

NL 5 57–74 © Intellect Ltd 2007

57

Northern Lights.qxd

1. 2.

2.

3/9/07

The quality has of mobile Kieslowksi telephone is referred tocameras Welles as steadily increasing, especially in terms of storage and resolution. Lenses will in most cases not be allocated much space, however, and this is likely to remain the most important limitation of phone cameras. In general, producers will seek to increase the quality until it reaches a certain level of customer satisfaction. In terms of photographs, a new low-fidelity aesthetic partly promoted by the limited quality of the early mobile-phone photograph has lowered the demands on photographs. This might be seen as part of a bigger picture in which a low-fidelity aesthetic of mobility has gained prominence, for example, in television spearheaded by reality shows. The concept of remediation was introduced’ by Bolter and Grusin and they have kept its definition rather open. They describe remediation as ‘the representation of one medium in another’, and argue that ‘remediation is a defining characteristic of the new digital media’ (Bolter and Grusin 1999: 45). This allows remediation to be about the borrowing of formal traits, as well as the recirculation of stories and characters as in adaptations where material from

10:47 am

Page 58

seamlessly into neighbouring areas, and telephone companies do the same. One of the more salient illustrations of convergence is actually offered by the mobile phone. Through the years, a number of gadgets have been offered as metamedium machines carrying a wide set of media functions, but nothing has been as successful as the mobile telephone. Besides a telephone service, this universal media machine now often offers television clips, music, radio, e-mail, web browsing, and computer games. It takes photographs, records video, and handles the new media SMS and MMS, just as it may offer additional non-media features like an alarm clock, calendar, notebook, calculator, and even penlight. The scope is impressive considering its small size, but many of its functions offer a quality that is rather limited, much like the scissors on a Swiss Army knife. Regarding its photographic and web-browsing capabilities, for example, it remains more of a toy than a serious media machine. Thus, on a number of such levels, we need more specialized devices. Still, the mobile telephone compensates for its low-fidelity quality in a number of areas by means of versatility and portability.1 The remediation of these media to our mobile telephones makes picking up our photographic camera to shoot still images or our video camera to shoot moving images, much the same thing.2 When these devices are remediated to our phones as the same physical object, choosing between them now becomes a matter of selecting between two settings on the camera menu of the telephone. If differences among media are merely becoming a matter of alternative settings in the same software running on universal media machines, are the differences between individual media then disappearing? Kittler does not only claim an erasure of the differences among individual media. In continuation of the quote above, he also projects the erasure of the very concept of medium: Inside the computers themselves everything becomes a number: quantity without image, sound, or voice. And once optical fiber networks turn formerly distinct data flows into a standardized series of digital numbers, any medium can be translated into any other. With numbers, everything goes. Modulation, transformation, synchronization; delay, storage, transportation; scrambling, scanning, mapping – a total media link on a digital base will erase the very concept of medium. (Kittler [1986] 1999: 1–2)

In the following, my question will be: to what extent and in which ways could these predictions be said to describe the situation now emerging? To what extent is a convergence of media taking place and what form does it take? Do formerly distinct media converge so that the differences among them in fact disappear, and does this imply an erasure of the concept of medium? There has in fact been talk of a ‘post-medium condition’ and a ‘post-media aesthetics’. These notions have respectively been brought up by the art historian Rosalind Krauss (1999a, 1999b), and by the media theorist Lev Manovich (2001b). What do such notions entail, and what bearing do they have on the convergences addressed here and on the future of the concept of medium? 58

Arild Fetveit

Northern Lights.qxd

3/9/07

10:47 am

Page 59

Before addressing Krauss and Manovich’s notions, I will first confront the widespread perception that with digitalization, ‘everything goes’, in terms of data manipulation, and second, I will look at how media interact, in terms of competition, symbiosis, and remediation. On this background, I move on to address the concept of medium itself, before concluding with a reassessment of the shape that convergence actually seems to take today.

With digitalization – ‘everything goes’ The judgement that digitalization brings unlimited freedom, where ‘everything goes’, as Kittler claimed in the 1980s, is still surprisingly common, in spite of, among others, Manovich’s demonstrations (2001a: 138) of how laborious, ‘time-consuming and difficult’ digital compositing, for example, may be. Such a notion of freedom is, for example, still Mark B.N. Hansen’s bid on the question of what difference the digital makes, in his book New Philosophy for New Media (2004). His answer is that ‘digitalization allows for an almost limitless potential to modify the image, that is, any image’ (Hansen 2004: 31). The qualifier ‘almost’ could indicate some moderation to this claim. But after Hansen takes Manovich to task for ‘correlating new media with earlier media types’, like the cinema and, to see new media as strongly influenced by earlier media, Hansen goes on to characterize new media by their ‘total material fluidity’, because ‘rather than being anchored to a specific material support, new media are fully manipulable, digital data’ (Hansen 2004: 31). Thus, it appears that Hansen grounds his ‘new philosophy for new media’ on an empirically dubious and hyped conception of such media. In order to correct our perception of what digitalization has brought, it is time to reassess and clarify the view of new media as ‘fully manipulable’, and focus not only on the manipulability of digital data, but also on the various limitations to which such a manipulability is subject. The efforts and achievements of the companies producing digital effects for cinema give a good indication of how difficult digital data is to mould and shape. Rendering hair, water, storms, lava from volcanoes, not to mention altering and controlling expressions on the human face, still represent daunting and costly challenges for highly specialized digitaleffects companies. Each achievement tends to elicit explicit celebrations: inside the production environment (in journals like American Cinematographer and Cinefex), within professional organizations like SIGGRAPH (devoted to the development of computer graphics), as well as in public (when marketing new movies featuring spectacular effects).3 Thus, talk of digital media as ‘fully manipulable’ belies the fact that any meaningful alteration of a photographic image in some way comes up against the challenge of drawing. An exploration of to what extent the rich set of effects and tools developed for controlling the alteration of photographs can aid us in overcoming this challenge is worthy of a study in its own right. A second limitation to the manipulability of digital media files is related to the fact that institutional players in the field work to limit our freedom to modify and edit the material we can access via our computers. In order to get a clearer picture of this, it might be useful to consider key aspects where the computer differs from earlier media. NL 5 57–74 © Intellect Ltd 2007

59

2.

3.

one mediumhas is used Kieslowksi in another. referred to The Welles as concept, according to Bolter and Grusin, also ‘express[es] the way in which one medium is seen by our culture as reforming or improving upon another’ (Bolter and Grusin 1999: 59). This way of defining the concept also allows any digitalization of a medium to be a remediation of the previous version of the medium. In the following I will draw upon these meanings freely, without attempts to delimit its meaning beyond what the flow of the argument will effect. In fact, this means that creativity in the new prolific area between live-action cinema and animation to a great extent takes place by way of software development. This also has a bearing on the ways in which aesthetic ideas and visions are spread today. New code might be written for a special project, like The Matrix (1999) or Lord of the Rings (2001), only to reappear as a set of features in editing software, which secures viral spread of an aesthetic effect across visual culture, though often in diluted versions. The dynamics of this field, in which programmers seek to make digital data manipulable in aesthetically productive ways, is worthy of a study in its own right.

Northern Lights.qxd

4. 2.

3/9/07

Such technologies Kieslowksi has partly dilute the idea referred to Welles as of a unique original, rendering the ‘original’ just one among many. But no matter how many there are, they are still considered originals. Thus, the distinction between original and copy is another matter. It is not based upon mass production, but upon reproduction of an original by what is somehow regarded as inferior means. The copy may, for example, not be created from the original cast at the time of the others, not be signed by the artist, or it may derive from an alternative technology, like a photographic reproduction of an oil painting, for example (see Benjamin 1973).

10:47 am

Page 60

Medium of display and medium of storage The computer ensures a radical separation between what we might call the medium of display and the medium of storage, initiated by the advent of mechanical mass production. A traditional medium like painting knew no separation between the two. Velasquez’s painting Las Meninas (1656) is both displayed and stored by the singular painting hanging in the Museo del Prado in Madrid. Sculpture is a somewhat more complicated case: sculptures carved in stone, such as Michelangelo’s Statue of David (1504), combine the storage and display function in the original sculpture, like the painting, in this case to be inspected in the Accademia Gallery in Florence. With the development of mass-production technologies related to items like bronze sculptures, printed books, graphical prints, a separation between the storage and display mediums develops, to the extent that casts and presses are stored.4 In the case of photography, the Daguerreotype and the later Polaroid photograph still combine storage and display functions much as in a painting. But with the positive/negative technique made public by Fox Talbot in 1839, a separation occurred, such that the negative became the privileged medium of storage for photography, while the positive print took care of the display function (albeit the use of positive prints in a storage function – in family albums and elsewhere – should not be underestimated). In the case of sound, the gramophone record provides another transitory case. It provides storage in a way that leaves the sound on display in some form, since the sound can be inspected directly on the record for levels of intensity. This option disappears with the magnetic tape, where the medium of storage can still be held in our hand and inspected for possible damage, but where the information stored has moved beyond the range of human perception. The computer radicalizes this, especially when we have dispensed with floppy disks and the like, and increasingly use our hard disks for storage. Then we end up first of all seeing the stored works in terms of icons and file names. The development sketched above, I believe, has been vital in inspiring current talk of dematerialization. However, what the examples above clearly indicate is that any general notion of dematerialization is missing the point. What is dematerialized in a certain way is our media of storage, in the sense that they are no longer physical objects we can hold in our hands as much as they are data codes kept in computer files. This dematerialization and standardization allows data to be stored in a computer and enables smoother operations of translation between formats, as well as further possibilities for editing and manipulation. When a computational device also becomes connected to other devices through interconnected networks, file transfer becomes easy, and allows an instant materialization of various data on a number of connected display units across the globe, notably screens and speakers. This allows for an increased use of the computer as a general media player, which in turn provokes producers to secure a separation between the two features discussed above, that of the display function and what we in terms of the computer might call the storage and manipulation function, because what we can store in a computer we also, for the most part, can edit.

60

Arild Fetveit

Northern Lights.qxd

3/9/07

10:47 am

Page 61

Copyright and the wall between editing and display In the development of the computer towards a metamedium – with an increased ability to remediate other media and show us films, television programmes, newspapers, and journals – the storage and manipulation capabilities of the computer are often left unengaged or explicitly locked out. We are often left with the option of displaying media content that we cannot change: to play movies or television programmes, to display newspaper or journal articles, and so on. Sometimes we can store media files on our computer; at other times we can neither download, nor store them. Very often, even if we can store them, editing is cumbersome and difficult, much more so than the technological possibilities of the computer itself would suggest, and certainly more than notions of ‘total manipulability’ indicate. However, a number of software agents offer keys to unlock this wall, at least partly, thus allowing us to copy and share audiovisual and other files. The competence of users also varies rather dramatically in terms of how skilled they are in finding the right software or in hacking their way beyond the limitations of this wall. A feud is ongoing in this field between copyright holders and encryption experts on the one side, and users, software agents, and hackers on the other. The film and music industries, especially, are trying to ensure that their material is not distributed for free or re-edited and used in ways that they do not approve of. However, they are at the same time increasingly eager to stimulate user activity in a number of ways, in order to profit from the user-driven productivity sometimes talked about under the rubric Web 2.0. The strengthening of such an interest represents a major trend in the current entertainment industry. This trend produces a certain slack in terms of how forcefully copyright holders seek to prevent attempts to play with and manipulate their material. Audience activity takes place under the watchful eye of a producer who is both happy to see people supply their labour for free while adding value to his product – but is also nervous, since his lack of control means he cannot dictate exactly how the product will turn out, or where all the revenue might end up. A major challenge for media producers is to accommodate, absorb, and utilize this creative vogue empowered by the digital democratization of the means of production. Arvidsson and Sandvik discuss this challenge related to computer games in the present volume. Henry Jenkins (2003, 2006) has discussed it in terms of audience participation in popular culture. Relating to this, there is also a trend in the art world where generating and reworking social relations are at issue, as discussed by Nicolas Bourriaud (2002) under the term ‘relational aesthetics’. In short, ours is not only a time of convergence, but also of increased democratization of cultural production. In fact, the two are linked, in that the technological convergence makes it easier to mix, edit, and play with various files, in spite of the limitations I have pointed to above. But let us now take a closer look at what convergence entails, what shape it seems to take, and how far it appears to go. Convergence – competition and symbiosis Bolter and Grusin (1999: 47–48) note how ‘television and the World Wide Web are engaged in an unacknowledged competition in which each now seeks to remediate the other. The competition is economic as well as NL 5 57–74 © Intellect Ltd 2007

61

2.

Kieslowksi has referred to Welles as

Northern Lights.qxd

5. 2.

3/9/07

John Caldwell Kieslowksi haspoints to high ambitions referred to Welles in as the case of the early cross-platform work for the television series Homicide (NBC), but notes that this level of ambition has become rare (Caldwell 2003: 129). There is a tendency to use the Internet to send viewers to television and vice versa. Internet sites also allow the audience to engage with the story worlds in multiple ways, through games, discussion groups, and additions to the fictional world (like plot expansions and character background stories not used in the dramatic production). They might also allow producers to tap fan cultures for ideas and useful material. There have also been attempts to make the media equal in importance in the unfolding of a story, for example in the use of radio and the Internet (see Neumark 2006). 6. Jenkins (2001: 1) argues that, ‘Part of the confusion about media convergence stems from the fact that when people talk about it, they’re actually describing at least five processes.’ I second Jenkins’s call for distinctions between levels when addressing convergence, and three of his five levels coincide with the distinctions I have opted for. In Jenkins’s view, the convergence can be (1) technological, (2) economic, (3) social

10:47 am

Page 62

aesthetic; it is a struggle to determine whether broadcast television or the Internet will dominate the American and world markets.’ Much like the earlier struggle between television and film, the relationship between television and the World Wide Web is not only one of struggle, but also one of tactical cooperation and symbiosis. An illustration of this can be found in television’s rapid expansion to integrate mobile telephones and web pages into their programmes. This expansion gives us, not television per se, but a multimedial circuit of television/mobile/Web, which demonstrated its revenue-generating potential with Big Brother. This is a set-up that has been adopted in numerous programmes since, in later reality-based programmes like Pop Stars as well as in a host of other shows, enticing the audience to engage in SMS voting, thereby creating substantial extra revenue. Fictional programming has become multimedial too, through offering websites that draw the audience into web-based chats, games, podcasts and ancillary products of various kinds. In most cases this leaves a main medium like radio, film, or television in the leading position, being supported and expanded by new points of access and supplementary products.5 But the tendency is not only one of symbiosis between distinct media. It is also one in which a series of remediations allow established media like newspapers, photography, radio, film, and television to free themselves partly from the media technologies in which they were developed and migrate across any channel open to them. The technological convergence facilitates these media’s reappearance on the screens of our computers, our mobile telephones, our PDAs (personal digital assistants), our iPods or wherever they can be displayed. Texts, sounds, and images become detached from the media technologies that used to support them, and perhaps even gave birth to them (like in the case of photographic emulsion and the celluloid film strip), and migrate and multiply on technological platforms flattened by digital technology. At the same time, texts as well as media seem to excel in exercises of remediation, in which they mirror and cite each other and simulate each other’s characteristic features, as Bolter and Grusin note in terms of television adopting aspects associated with web pages and the reverse. What seems to appear is a double move in which the technological convergence supports a tendency toward remediation and recirculation which also entails elements of aesthetic convergence, but at the same time, medium specificity also seems to be maintained through the recirculation of specific medial characteristics. This prolific activity of remediation raises obstacles for any theory of convergence, leading to the end of media, a post-medium situation in which all media converge into one super medium capable of performing the functionality of every previous medium, or at least, capable of satisfying much of our urge for mediation. Now, to get a handle on the complexity and the conflicting tendencies characterizing convergence, it is useful to distinguish between the various levels on which it takes place.6 First, digitalization itself, of course, renders media texts into data files that, to a substantial degree, are easily combined and circulated. This is the technical convergence that forms an important basis for other kinds of convergence. But to which extent does this situation entail convergence on other levels as well – on the level of cultural forms; 62

Arild Fetveit

Northern Lights.qxd

3/9/07

10:47 am

Page 63

on the level of aesthetics; on the level of the physical devices that deliver our media; and on the level of industrial actors? Clearly, industrial actors are merging on a grand scale, AOL and Time Warner, Sony and Ericsson, Google and YouTube, to mention just a few, or they are making exclusive or semi-exclusive partnerships, like Apple with Google, Cingular, and others. The more thorny issues to assess are those of aesthetics and cultural forms, and the level of the physical devices. In terms of the last, it is clear that the computer is aspiring to the condition of a metamedium, a machine which can display and handle all other media in some sense, while the mobile telephone effectively demonstrates this multimediality with the help of its onboard mini-computer. On this background, industry operators are pondering whether consumers will be satisfied by just carrying a multimedia mobile phone around, as well as a digital music player and a PDA. Or, if we look a few more years ahead, will our present devices be replaced by, or grow into, portable computers the size of mobile phones, with virtual keyboards and roll-out screens?7 When it comes to our homes, will we continue to have radios, music systems/players, newspapers, magazines, televisions, and computers, or will we be satisfied with one or more multimedial super computer(s) taking care of it all? The possible future of a multimedial super computer that can replace the television and the computer by combining the functionalities of both is much a matter of cost and of practicalities. For an expansion of our present televisions to allow us to browse the Internet and to have similar functionalities as our computers, they need to be equipped with basic computer components, which would incur additional costs. When it comes to our computers, they already handle video and audio streams, so minimal adjustments would seem to be necessary, except for the reading of analogue signals (which requires an additional tuner), but as the analogue net is about to be replaced by digital, this is hardly an issue. This means that social preferences may be more important than technical possibilities for assessing the effects of the convergence. Thus, one of the key questions becomes, will we want to have our work desk also be the site for aesthetic experience and entertainment, or will we move the device between our workspace and our living-room relaxation area according to our needs? Actually, it seems more reasonable to assume that we will have different devices in different places, tailored to the various uses in those places, and that those devices will have a degree of specialization rather than full multifunctionality. The more complex issue is the convergence of aesthetics and cultural forms respectively. This is only to a limited degree a matter of digitalization, as cultural forms intermix and imitate each other throughout history. This issue has in fact been the subject of ongoing debates at least since Horace pointed to the correspondence between painting and poetry in his Ars Poetica, condensed in the phrase ‘ut Pictura Poesis’, (as is painting, so is poetry). This observation has provided a major reference point for such debates, especially after Gotthold Ephraim Lessing made it a key target of criticism in his Laocoön ([1766] 1984), followed up in Irving Babbitt’s The New Laokoon: An Essay on the Confusion of the Arts (1910), and with substantially greater impact in Clement Greenberg’s ‘Towards a Newer Laocoon’ ([1940] 1985), which may be the first powerful articulation of Greenberg’s view of modernism.8

NL 5 57–74 © Intellect Ltd 2007

63

2.

or organic, (4) Kieslowksi has cultural (5) as referred and to Welles global. His description of the processes is crisp and his idea of ‘social or organic convergence’ original (Jenkins 2001: 1). Social or organic convergence is a matter of what he calls consumers’ ‘multitasking strategies for navigating the new information environment’, and it occurs when ‘a high schooler is watching baseball on a bigscreen television, listening to techno on the stereo, word processing a paper and writing e-mail to his friends’ (Jenkins 2001: 1). This also relates to an economic convergence in which Jenkins locates the exploitation of ‘synergies’, and a cultural convergence where he finds ‘transmedia storytelling, the development of content across multiple channels’ (Jenkins 2001: 1). The processes of convergence are intertwined and entangled in Jenkins’s short description, but I think he does well in trying to separate them out. If a sixth process should be added to Jenkins’s scheme, aesthetic convergence could be a strong candidate.

Northern Lights.qxd

7. 2.

8.

3/9/07

The cost andhas the size Kieslowksi of computers are still referred to Welles as shrinking. But as the electronics of the computer gets smaller, the two parts that must also shrink are the keyboard and the screen. A virtual keyboard, which is projected on a surface (preferably a table), is already developed, so is a screen that may be rolled up when not in use. Screen projection could also be viable, but has the disadvantage of requiring a reasonably white surface whereupon the image may be beamed. Therefore, chances are that we will still use laptops like today’s, only lighter and with better screens, batteries, and bigger memory capacity. Chances are also that the divide between high-fidelity and lowfidelity performance will become more institutionalized, so that we have both a high-fidelity service from our laptops and stationary computers, as well as a lowfidelity version that can fit in our pocket, whether it be cameras, browsers, e-mail services and more. This also calls for better systems of integration between the two kinds of devices. Apple may have taken an important step in this direction by having the iPhone run OS X, the same operating system that runs on their computers. The convergence of aesthetics and

10:47 am

Page 64

Greenberg is of special interest in this context, because, as I soon will show, he develops a conception of modernity that seems surprisingly interlinked with Bolter and Grusin’s concept of remediation. Greenberg implicitly suggests that the convergence of aesthetic forms take place with shifting intensity historically, and that the defining aesthetic of a period might first of all be characterized by undoing the effects of such a convergence. This is actually how he defines modernism. The definition springs from a concern with the way in which art forms may compromise their own nature in an effort to absorb the effects of, and to imitate the characteristics of, other art forms, particularly the dominant art form of the period. He says, There can be, I believe, such a thing as a dominant art form; this was what literature had become in Europe by the 17th century. […] Now, when it happens that a single art is given the dominant role, it becomes the prototype of all art: the others try to shed their proper characters and imitate its effects. (Greenberg [1940] 1985)

Greenberg comes to define modernism as a process of undoing the effects of such imitation, of purging the arts and their media from the impurity caused by the borrowing and absorptions of effects from other media. This process, he points out, becomes one of consolidating or entrenching an art form in its own area of competence. In ‘Modernist Painting’, Greenberg writes, The essence of modernism lies […] in the use of the characteristic methods of a discipline to criticize the discipline itself, not in order to subvert it, but in order to entrench it more firmly in its area of competence. […] It quickly emerged that the unique and proper area of competence of each art coincided with all that was unique in the nature of its medium. The task of self-criticism became to eliminate from the specific effects of each art any and every effect that might conceivably be borrowed from or by the medium of any other art. Thus would each art be rendered ‘pure’ […]. (Greenberg [1960] 1993)

Thus, it seems that Greenberg is keenly interested in what we today are more likely to call ‘remediation’, which may be described as the adoption or absorption of the effects of one medium in another. In short, and from this perspective, it appears that Greenberg is describing modernism as remediation in reverse. The idea of modernism is to undo the effects of a previous remediation, which has levelled the differences between art forms, because these promiscuous arts thrive in imitating each other’s effects, that is, their artistic means. It appears that we find ourselves again in a promiscuous time where media borrow effects from each other, not only because we may still be under the influence of a postmodern counter-reaction against modernism’s penchant for the purity of media forms, but also because digital technology encourages the playful recombination of media. This is one of the key features computer pioneers like Alan Kay sought to develop. The capacity of the computer for sampling and recombining various textual elements, its 64

Arild Fetveit

Northern Lights.qxd

3/9/07

10:47 am

Page 65

enhanced potential for post-production remixing of previous material, has contributed to an aesthetic of sampling which we can also see as adding to cultural convergence, where for example the art world and the world of media entertainment come closer. However, in this field of intermix, media hardly disappear. In fact, the aesthetics of sampling even allow old and outdated media forms and modes to be reborn into new contexts and to thrive. Yet another reason why media do not disappear so easily is our love for them, even when they may seem old and outdated. Their aesthetics seem to live on as they are utilized and recirculated in our digital phone cameras for example, which may have settings for sepia or solarization. Old film stock and the Super 8 format are often found in films and music videos. These anachronistic appearances testify to some level of truth in Marshall McLuhan’s statement ([1964] 2001) that ‘the medium is the message’, in that, at times, the specific medial quality may be of key importance to the aesthetics developed. In order to communicate in a rich way, we mobilize and use a number of media formats, in part because they help us articulate ourselves in rich and interesting ways. Another obstacle for a generalized convergence, as envisioned by Kittler and others, is the lack of standardized formats, which brings us back to the very basis of a digitally driven convergence. Arriving at a common and practical format is just the first obstacle here. Agreeing on a standard often seems even more difficult, as major economic interests are at stake. Thus, we get the common situation of various forms of war between competing formats. Apple iPods, for example, employ a standard that carries more information than MP3 files, and therefore, sending an iPod music file to a PC-user with an MP3 player may leave the latter without a chance to hear the music. Similarly, the current battle between the Blu-ray Disc and HighDefinition television echoes the war over formats that took place with the establishment of VHS as the standard video technology in popular use. Computer games continue to confront each other across a number of competing consoles, like Playstation 2 and 3, Xbox, Nintendo Gamecube and Wii. There is also the silent war going on between telephone companies and emerging Internet telephone providers that may eventually offer mobile services (Apple’s partnership with Cingular might be seen as an attempt to avoid such a possible standoff, by bringing the biggest mobiletelephone company on board rather than risking countermeasures from their side). Industrial strategies set in motion to act against competitors and secure profit for key players also work in part to halt technical convergence. These strategies and behavioural logics also defy any ideal operation of the free market, and instead seek to create and benefit from market situations that allow higher prizes than a freely functioning market would. This is the context in which we can see European regulators recent claims that iTunes should be compatible with all MP3 players, so that music bought from the iTunes Store can also play on devices from Apple’s competitors. In short, the tendency to produce incompatible standards may be motivated to some extent by diverging convictions about the benefits of the various standards proposed, but it is also driven by the desire to undercut the workings of a free market and protect major companies from open competition. Thus, even on the technological level where digitalization

NL 5 57–74 © Intellect Ltd 2007

65

2.

cultural forms Kieslowksi hascould, of course,tobe referred Welles as described from a number of other perspectives. For instance, a Bakhtinian perspective would assess voices travelling across texts and media, which is often done by way of the concept of intertextuality. Anna Everett (2003) pursues such a perspective in the development of her concept of ‘digitextuality’, meant to describe the particular textual form encouraged by digital media.

Northern Lights.qxd

9. 2.

3/9/07

Krauss herself Kieslowksi haslocates such a reinvention of referred to Welles as the medium in James Coleman’s double portraits, which are photographic, but also invoke (or remediate) the medial forms of feature films and photo novels. A reinvention of the medium could also be claimed for Gerhard Richter’s photorealistic paintings, which seem to both redeem and re-actualize painting and photography at the same time as they are positioning themselves as neither painting nor photography. In fact, Richter himself claims to be doing photography, ‘by other means’. We must merely learn to accept that photographic emulsion and optical lenses are not integral to photography (see Richter 1995).

10:47 am

Page 66

would seem to ensure a common platform, the picture is much more complex and riddled with conflicts. Kittler’s conception of convergence implies a notion of a post-medium condition of some sort, an assumption that distinct media will become less important or even disappear, along with the concept of medium itself. Related notions are also put forward by Krauss and Manovich, respectively.

The post-medium condition and post-media aesthetics After Kittler, we seem to be confronted with two major attempts to articulate a ‘post-medium condition’, as Rosalind Krauss (1999) calls it, or that of a ‘post-media aesthetics’ as it is labelled by Manovich (2001b). In her article, ‘Reinventing the medium’, Krauss discusses the shifting status of photography in the art world and how photography in part became a tool for deconstructing an art practice based on the specificity of the medium, be it the specificity of painting, sculpture, graphic print, or, ironically, photography itself. She argues that photography enters the art world, or, in her words, ‘converges with art’ as a means of both enacting and documenting a fundamental transformation whereby the specificity of the individual medium is abandoned in favor of a practice focused on what has to be called art-in-general, the generic character of art independent of a specific, traditional support. (Krauss 1999b: 293–94)

Thus, photography is implicated in the enactment and the documentation of a change from a situation in which artists work within media (like painting, sculpture, graphical print and so on) to a situation that Krauss refers to as the ‘post-medium condition’. This casts photography in a new role, that of documenting various performances or ‘hors-media’ events where art practices have migrated beyond and outside established media forms to strategically and in ad hoc situations deploy various objects, actions, and events to convey conceptual ideas. Thus, in this ‘post-medium condition’, which Krauss also associates with postmodernism, practice, according to Krauss, ‘is not defined in relation to a given medium – sculpture – but rather in relation to the logical operations on a set of cultural terms, for which any medium – photography, books, lines on walls, mirrors, or sculpture itself – might be used’ (Krauss 1999: 289). This move away from the classic media of the academic tradition (which basically is thought of as painting, sculpture and graphic print), and from the modernism defined by Greenberg, which explored the medium specificity of painting and sculpture in particular, has according to Krauss left the artist in a ‘post-medium condition’ to explore ‘art-in-general’. However, this ‘post-medium condition’ does not so much take away the media from artists as it refigures their functions and multiplies them. Because now, any medium can be used and almost anything can become a medium; even ice cubes might become media in conceptual ephemeral events, as well as streams of air hitting the spectator’s head inside the gallery space. But elements of other medial practices could also be implicated in medial situations which in part may redeem an interest in their specificity, or in Krauss’s words, ‘reinvent the medium’, or have these media serve ad hoc strategic purposes of different kinds.9 66

Arild Fetveit

Northern Lights.qxd

3/9/07

10:47 am

Page 67

On this background, Krauss’s labelling of a situation in the modern art world – where artists no longer explore a single medium like painting and sculpture – as a ‘post-medium condition’ is somewhat ironic. The situation is partly defined by the fact that artists work across a range of different media, combining them pragmatically for various purposes and effects. We could see this as a move from the singular to the plural, from the medium artist faithful to the exploration of a specific means of expression, to the media artist defined by an adulterous pragmatism using whatever medium or combination of media that seem to work. Thus, ‘the post-medium condition’ might as well be labelled ‘the medium condition’, or ‘the condition of media proliferation’, for this ‘post-medium condition’ is one in which the artist constantly comes to choose certain media among the many possible. Whereas one medium conveniently can be called painting, or sculpture – most of the time without recourse to the concept of medium – a host of different medial options tends to render every one of them a ‘medium’. Under such a condition, most art might become ‘media art’, and most artists ‘media artists’ of some sort. A further reason why artists become media artists is related to the political and cultural interventionist project of jamming – re-figuring, reprogramming – important aspects of the culture articulated in ‘the media’. And by ‘media’, in this context, I mean major outlets like television, film, newspapers, media that in many ways define our culture, and that artists may feel compelled to comment on. This intervention will often employ the media formats that are being commented on and it may render the art ‘media art’, both in terms of its subject matter and its medial means.10 The challenge from Manovich’s notion of a post-media aesthetics is considerably greater in that it projects an end to media as well as to the concept of medium itself. This is first of all because Manovich’s conception of a post-media aesthetics is in part based on digitalization. But in his argument, an internal tension in the concept of medium also comes to the fore, a tension that goes some way towards explaining the persistence of the concept of medium allegedly faced with annihilation, for instance, the fact that the medium of film seems to have faint problems surviving the death of the celluloid filmstrip. Manovich starts his dismantling of the concept of medium by claiming that: In the last third of the twentieth century, various cultural and technological developments have together rendered meaningless one of the key concepts of modern art – that of a medium. However, no new topology of art practice came to replace media-based typology which divides art into painting, works on paper, sculpture, film, video, and so on. (Manovich 2001b: 1)

Manovich cites a number of reasons why the concept of the medium, in his words, is ‘rendered meaningless’. First, the proliferation of new artistic forms replacing traditional art threaten old typologies of media, sometimes with arbitrary combinations, sometimes even as dematerialized ‘conceptual art’. The introduction of modern media like video provides yet another challenge in that, as Manovich writes, the

NL 5 57–74 © Intellect Ltd 2007

67

10. the art 2. Within Kieslowksi hasworld itself, however, theas referred to Welles term ‘media art’ remains contested and riddled with uncertainty as to what it should mean. It has been used in a more specialized meaning to designate a particular kind of art where the logic of the medium itself is explored and where the medium involved is preferably digital. From this vantage point strategies of inclusion have been attempted to integrate ‘media art’ (or ‘new media art’) into the established art world (see Grau 2007). Such a conception of media art has come up against at least two problems. First, it risks holding onto a Greenbergian conception of art as an interrogation of the medium used. In the face of art’s social turn, this risks rendering ‘media art’ dated and without contemporary sensibility (see Bourriaud 2002). Second, the interpretation that all art uses some kind of medium and is therefore by definition media art risks a certain redundancy. The crisis for the concept of ‘media art’ has manifested itself in the removal of the concept of medium from the German art festival transmediale – international media art festival, which now is renamed transmediale – festival for art and digital culture.

Northern Lights.qxd

3/9/07

11. is one of 2. This Kieslowksi hasthe conceptual issues as referred to Welles Noël Carroll discusses in the three first chapters in Theorizing the Moving Image (1996) 12. For photography, the loss of the material support based on chemistry was especially powerful, because, as Barthes (1981: 80) notes, photography was not an optical invention in the wake of the Albertinian perspective, but a chemical invention. Thus, the digital change seemed to tamper with photography’s own defining characteristics. Moreover, photography had rested comfortably within its chemical ground through a number of other technical transitions, and had hardly been subject to very many revolutionary medial changes, at least, this I believe is how it was perceived by the late 1980s. What was eventually demonstrated by the digitalization of photography – that it was possible to lift the medium of photography away from its medium of invention and let it be remediated on a digital platform – had already been demonstrated by other media, like the LP which had become a CD, and continues to be demonstrated, without much public anxiety, by television. A comparative project might well be

10:47 am

Page 68

mass medium of television and art medium of video used the same material base […] and also involved the same condition of perception (television monitor). The only justifications of treating them as separate mediums were sociological and economic […]. Gradually, this sociological difference in the distribution mechanisms, along with other sociological differences […], became more important criteria in distinguishing between mediums than the distinctions in material used or conditions of perception. In short, sociology and economics took over aesthetics. [original emphasis] (Manovich 2001b: 2–3)

In order to strengthen his suit, Manovich does not play the digital card early in his discussion. Rather, he attempts to make a strong case for a crisis of the medium existing before digitalization. And he manages to do so to some extent, but even more, his argument comes to lay bare the inherent complexity in the concept of medium itself. His account starts off with an assumption that the concept designates the medium’s technological base, or what in the language of art history and criticism is usually designated ‘the material support’. Eventually, however, Manovich comes to describe what we might call a metamorphosis in which the concept of medium comes to be based as much on other parameters. The parameters Manovich nominates for distinguishing between media may actually form part of a richer conception of medium, made more urgent by these acts of disappearance and of migration away from the initial material supports on which the medium was conceived. Apart from ‘material used’, which otherwise can be referred to as media technology or material support, he mentions ‘distribution mechanisms’, ‘conditions of perception’, and lastly, the less clearly defined ‘other sociological differences’ (Manovich 2001b: 3). The key observation here, which may seem paradoxical, is precisely that the medium of film survives so effortlessly the death of its medium, in the sense of its technical support.11 Photography as well now seems to thrive on its new digital platform; in fact, it thrives to such an extent that it seems almost unreal to think of the repeated death sentences it received during the 1990s as its material support became destined for the technical museum.12 In the face of such changes, Manovich notes that ‘despite the obvious inadequacy of the concept of medium to describe contemporary cultural and artistic reality, it persists. It persists through sheer inertia – and also because to put in place a better, more adequate conceptual system is easier said than done’ (Manovich 2001b: 4). If the concept persists, it is because people persist in using it. And when the concept of film, for example, seems to survive even when films are often not shot on film, it is because the concept of medium is much more complicated than we often realize, and this is precisely what Manovich comes to articulate in the parameters above.

Revising the concept of medium Basically, the concept of medium seems to involve a lot more than the material support it is often associated with, as can be seen in a number of definitions, which Bruhn Jensen points out in his article ‘Mixed Media’ in this volume. If we take as a vantage point the parameters Manovich suggests, beyond the medium technology (or technical support), we have 68

Arild Fetveit

Northern Lights.qxd

3/9/07

10:47 am

Page 69

‘distribution mechanisms’, ‘conditions of perception’, and the less clearly defined ‘other sociological differences’. Without aiming for any fully coherent conception and avoiding internal overlap, we could perhaps add ‘sociocultural use’ along with the ‘aesthetic practices’ associated with such use. What complicates the picture, but also makes it more interesting, is that it seems that the concept of medium, the way that it is currently used, is capable of migrating between these parameters which make up the concept of medium, rendering some more important sometimes, and others more important at other times. Thus, the emphasis on the various aspects defining a medium may be shifting. At times, the technological aspect of the medium seems of central import, while at other times the formal, thematic, and aesthetic aspects, or ‘distribution mechanisms’ and ‘sociocultural uses’ come to the fore. W.J.T. Mitchell captures this complexity well, when he fits this difficult term with the following elusive description: An image appears only in some medium or other – in paint, stone, word, or numbers. But what about media? How do they appear, make themselves manifest and understandable? It is tempting to settle on a rigorously materialist answer to this question, and to identify the medium as simply the material support in or on which the image appears. But this answer seems unsatisfactory on the face of it. A medium is more than the materials of which it is composed. It is, as Raymond Williams wisely insisted, a material social practice, a set of skills, habits, techniques, tools, codes and conventions. (Mitchell 2005: 203)

In terms of the position of the concept, Mitchell adds, ‘[t]he concept of a medium […] seems […] to occupy some sort of vague middle ground between materials and the things people do with them’ (Mitchell 2005: 204). But the vagueness of this ‘ground’ can be clarified, I believe, by studying how the concept operates in actual cases. Then we might find that different aspects of the concept will be at play at different times. If we limit ourselves to what it is possible to say about the medium of film, we can get an illustration of the multifaceted nature of the concept of ‘medium’, and of how its various aspects can be actualized in almost paradoxical ways. It has been noted that the medium of film is dying because film stock is being replaced by digital video. In this case, the medium of film is referred to in terms of its technical support. If we counter that films survive because people simply love stories told in an audio-visual format, what is referred to is the medium of film as a cultural and aesthetic form. If we say that films will still die because theatres that show films lose their audiences and will soon have to close, we are conceiving of the medium of film in terms of its primary viewing practice in theatres, more than in terms of the technical support and the cultural and aesthetic form. These complexities relating to the concept of medium have led to recent attempts to clarify and to define the concept. In the article ‘Convergence? I Diverge’, Henry Jenkins proposes to distinguish between media, genres and delivery technologies. He claims: ‘Recorded sound is a medium. Radio drama is a genre. CDs, MP3 files and eight-track cassettes are delivery

NL 5 57–74 © Intellect Ltd 2007

69

2.

undertaken Kieslowksi here has to uncover to Welles what as referred to extent various media have changed their technical supports and with what consequences and possible difficulties.

Northern Lights.qxd

3/9/07

13. concepthas of 2. The Kieslowksi delivery technologies referred to Welles as serves to detach the concept of medium from the realm of technical support. The concept of genre works to protect the concept against shifting aesthetic and cultural practices. 14. From his initial attempt to clarify the concept, Jenkins has also found Lisa Gitelman’s work on the concept useful, without having that interest amalgamate into a coherent position. Gitelman proposes to ‘define media as socially realized structures of communication, where structures include both technological forms and their associated protocols, and where communication is a cultural practice, a ritualized collection of different people on the same mental map, sharing and engaging with popular ontologies of representation’ (2006: 7). She recognizes, that ‘[d]efining media this way admittedly keeps things muddy’ (2006: 7). In spite of saying, somewhat later that, ‘the “materiality” of media is one of the things that interest me most’, her definition is not clear as to what extent technological support or media materiality is admitted into her definition.

10:47 am

Page 70

technologies. Genre and delivery technologies come and go, but media persist as layers within an ever more complicated information and entertainment system’ (Jenkins 2001). This might seem clarifying. But if recorded sound is a medium, and not CDs, what other media are there and is it possible to invent new ones? These questions are implicitly answered by Jenkins five years later as he lists the following media, ‘spoken words’, ‘printed words’, ‘cinema’, ‘theatre’, television’, ‘radio’ (Jenkins 2006: 14). One could try to continue this list by adding ‘book’ and ‘newspaper’. But these seem partly superfluous when ‘printed words’ are already mentioned. In fact, ‘book’ and ‘newspaper’ could well be regarded as mere delivery technologies for the medium ‘printed words’. Likewise, radio might be regarded not as a medium, but as a delivery technology for the medium ‘spoken words’, or perhaps even better, for ‘recorded sound’. In short, the framework proposed by Jenkins is not entirely satisfactory, but it is productive in provoking us to rethink the concept of medium. The strategy behind making ‘recorded sound’ a primary example of what a medium is, could be to protect the concept of medium from the creative turmoil that takes place in the realms of technological and social inventions on the one side, and cultural and aesthetic inventions on the other.13 But the outcome is not so much to protect the concept as to undermine its relevance. For if it merely designates allegedly stable medial forms like ‘recorded sound’, ‘printed words’, and traditional media like radio, television, and cinema, it need not be on everybody’s lips, like it is today. Whatever the strategy, what is negated by making ‘recorded sound’ a primary example of what a medium is, is the multiple aspects of the concept, and its migratory flexibility in which different aspects seem to be emphasized at different times. Any attempt to admit the concept only one dimension, be it technical device, cultural or aesthetic practice, form of perception, or socio-economic mode of circulation, or medial form like ‘recorded sound’, belies this multiplicity.14 The inherent richness and flexibility in the concept of medium is an important reason why the predictions of Kittler and Manovich – that the concept will become obsolete in the wake of convergence – may not hold up. If the concept is ‘rendered meaningless’ (Manovich) in one of its aspects, it may still persist as long as it is meaningful in terms of other aspects. Another reason, of course, is that media forms do not so much disappear as they reappear, through acts of remediation and recirculation. The conception of ‘medium’ put forward here may also help us to explain and assess how creativity and renewal take place within a medium. Manovich suggests in his contribution to this volume that new media remain perpetually new because they are made up of software that is being renewed. This is a suggestive idea, but once we see the many aspects of the concept of medium, we see that updates to the software first of all renew one aspect of the medium, namely its technical basis. What implications this will have in terms of other aspects of the medium, its cultural and aesthetic forms, or the logic of production and distribution characterizing it, is uncertain. In fact, technical changes (for example, in terms of software additions) need to be accepted, and utilized, so that they have consequences on other aspects of the medium as well, for them to be truly effective in 70

Arild Fetveit

Northern Lights.qxd

3/9/07

10:47 am

Page 71

contributing to a change of the medium as a whole. This suggests that certain technical extensions will hardly be significant, whereas others may lay the basis for powerful reinventions of the medium, through their impact on aspects like aesthetic practices, sociocultural use, distribution mechanisms, and audience practices.

The globalization of software additions I started this discussion by referring to Kittler’s prediction that ‘[t]he general digitalization of channels and information erases the differences among individual media’. And I have used much of the space to raise objections against the powerful convergence envisioned by Kittler. I also suggested above, however, that the gist of Kittler’s prediction still seems succinct. Let me now qualify this beyond the obvious presence of the metamedia, which our computers and our mobile telephones now represent. When assessing the consequences of digitalization, instead of talking about the erasure of differences, and to judge the level of convergence by the amount of differences erased, it may be as productive to talk about the addition of similar traits, viewing this as a major source fuelling convergence. The remediation of most media on a digital platform is about to make all media ‘new media’, in the sense that they are all empowered by computer technologies in one way or another. Hence, they are also more or less subject to the addition of new properties, beyond the new affordances they may already have been granted as part of their digitalization. A basic logic of the computer is therefore coming to redefine both the production and the reception processes related to media, to the extent that computers are involved in media production and consumption. Manovich has commented on this in The Language of New Media where he says The logic of a computer can be expected to significantly influence the traditional cultural logic of media; that is, we may expect that the computer layer will affect the cultural layer. The ways in which the computer models the world, represents data and allows us to operate on it; the key operations behind all computer programs (such as search, match, sort, and filter); the conventions of HCI […] influence the cultural layer of new media, its organization, its emerging genres, its contents. (Manovich 2001a: 46)

In short, with the pervasive presence of computers in the handling of media, the latter have come to be affected by what we can call a ‘globalization effect’ in which their new affordances come to be ones which the computer can provide to most media, and which feature as standard menu options in most applications.15 In other words, what we seem to get is a form of convergence by means of a globalized remediation, a remediation where previous media have been remedied by expansions to their powers that tend to be similar. Simply put, by being more and more supported by and interwoven with the computer, media now are increasingly profiting from the enhancements the computer can offer. As the computer tends to offer similar enhancements to most media, a convergence by means of globalized remediation takes place. A couple of examples may lay this out in more detail.

NL 5 57–74 © Intellect Ltd 2007

71

15. are, ofhas course, 2. These Kieslowksi functions copy,as referred tolike Welles paste, search, save, but also a host of other more specialized options that only some users will end up using. An illustration of the potential in this can be found by looking at the plug-ins for the Firefox browser.

Northern Lights.qxd

3/9/07

16. has prepared 2. This Kieslowksi has the ground a vogueasof referredfor to Welles new digital-effects films, and for comicstrip heroes entering the movie theatres. Commenting on this development, Manovich (2001a: 295) somewhat provocatively states that ‘cinema can no longer be clearly distinguished from animation. It is no longer an indexical media technology but, rather, a subgenre of painting’. 17. Specific media materialities are thereby lost in this remediation except to the extent that they can be represented within the operative mode of the computer. It is, for example, difficult to tell that most of Gerhard Richter’s pictures are paintings rather than photographs in their photographically mediated reproduction on a computer screen.

10:47 am

Page 72

In its digital version, the New York Times offers photographic slide shows, film clips and music. The original newspaper, just like any printed matter, magazine or book, cannot display time-based media, but its remediated version on the Web can. And it becomes increasingly effective in doing so as the capacity of digital networks increases. Film provides another example of computer-augmented affordances. Animated films used to be able to depict events that were impossible to relay in live-action cinema. When live-action film is enhanced by digital production technologies, however, its affordances increasingly come to involve the stunning effects of animated films, and these effects can now be offered in photorealistic quality.16 The increased affordances generated through digitalization may be said simply to reflect the characteristics of the computer as a metamedium, its ability to handle most, if not all, previous media, albeit within the parameters of its interface.17 This sets in motion a development that may eventually make most media capable of displaying most other media. This tendency is illustrated by the broadcaster BBC which features newspaperlike articles on their websites, complementing their television and radio programmes, and newspapers like the New York Times which features television offerings on their website, complementing their printed matter. As Manovich puts it in his contribution to this volume, ‘it is as though different media are actively trying to reach towards each other, exchanging properties and letting each other borrow their unique features’. However, the ways in which this will pave out, and the actual forms it will take, still remain to be seen given the high level of dynamism and turmoil in the field.

Conclusion All in all, what we have before us is a complex picture in which convergence is a powerful tendency, continuously counteracted by powerful counter-tendencies holding back on a full convergence that might promise to erase the differences among individual media so as to have us end up with merely one mediatory device. The picture is one in which a number of obstacles – such as multiple formats, copyright holders’ resistance against copying, storage, and manipulation, and not least our attachment to media aesthetic diversity – continue resisting universal convergence. At the same time, however, convergence works powerfully to make files compatible, to have devices communicate with each other, and to endow media with globally shared features. This means that our many media do not disappear to be replaced by one, or even just a few. Nevertheless, this is no guarantee that our concept of medium will persist. As long as we have a multiplicity of media as competing alternatives to consult, to express ourselves in, or to comment on, we may find the concept of medium relevant. In the heyday of painting and sculpture in the art academy, when relatively few media alternatives were around, the need for a concept like ‘medium’ was hardly a matter of urgency. Some of the changes in the field of media – the occurrences of new means of communication, like SMS, MMS, Skype, Instant Messenger, etc. do not necessarily urge the use of the concept of medium. We may often be content talking about them as ‘applications’, or we may just refer to them using names like ‘Skype’, or ‘Messenger’. In fact, the proliferation of 72

Arild Fetveit

Northern Lights.qxd

3/9/07

10:47 am

Page 73

media in the guise of new applications, which digitalization has brought, may represent a new threat to the concept of ‘medium’, in that we just do not conceive of ‘medium’ as the proper descriptive word for these new media. Thus Kittler’s predictions about the erasure of ‘the very concept of medium’ may be well worth reassessing in another ten or twenty years. In the mean time, we are likely to see more bids on how the concept should be defined.

Acknowledgements I wish to thank Liv Hausken, Ulrik Ekman, Kiersten Leigh Johnson and Francesco Lapenta who have read the article in draft form and proposed valuable adjustments. I am also grateful for the productive interchanges offered by my colleagues in the Media Aesthetics research project at the University of Oslo, supported by the Norwegian Research Council through the programmes KULFO and KIM, and last but not least, I am thankful for great discussions with my inspiring students in the Department of Media, Cognition, and Communication, University of Copenhagen, and with my colleagues in the same department. References Arvidsson, Adam and Sandvik, Kjetil (2007), Gameplay as Design: Uses of Computer Players’ Immaterial Labor, London: Intellect Press. Babbitt, Irving (1910), The New Laokoon: An Essay on the Confusion of the Arts, Boston: Houghton Mifflin Company. Barthes, Roland (1981), Camera Lucida: Reflections on Photography, New York: Hill and Wang. Benjamin, Walter (1973), ‘The work of art in the age of mechanical reproduction’, in Illuminations, London: Collins (Fontana). Bolter, Jay and Grusin, Richard (1999), Remediation: Understanding New Media, Cambridge, MA: MIT Press. Bourriaud, Nicolas (2002), Relational Aesthetics, Paris: Les Presse Du Reel. Caldwell, John (2003), ‘Second-shift aesthetics’, in Anna Everett and John T. Caldwell (eds), New Media: Theories and Practices of Digitextuality, London and New York: Routledge. Carroll, Noël (1996), Theorizing the Moving Image, Cambridge: Cambridge University Press. Everett, Anna (2003), ‘Digitextuality and click theory: Theses on convergence media in the digital age’, in Anna Everett and John T. Caldwell (eds), New Media: Theories and Practices of Digitextuality, London and New York: Routledge. Gitelman, Lisa (2006) Always Already New: Media, History and the Data of Culture, Cambridge, MA: MIT Press. Grau, Oliver (2007), MediaArtHistories, Cambridge, MA: MIT Press. Greenberg, Clement ([1940] 1985), ‘Towards a newer Laocoon’, in Francis Frascina (ed.), Pollock and After: The Critical Debate, London: Paul Chapman. ________ ([1960] 1993), ‘Modernist painting’, in John O’Brian (ed.), The Collected Essays and Criticism, vol. 4, Chicago: University of Chicago Press. Hansen, Mark B.N. (2004), New Philosophy for New Media. Chicago: University of Chicago Press. Horace ([18 BC] 1989), Epistles Book II and Epistle to the Pisones (‘Art Poetica’), ed. Niall Rudd, Cambridge: Cambridge University Press.

NL 5 57–74 © Intellect Ltd 2007

73

2.

Kieslowksi has referred to Welles as

Northern Lights.qxd

2.

3/9/07

Kieslowksi has referred to Welles as

10:47 am

Page 74

Jenkins, Henry (2001), ‘Convergence? I diverge’, Technology Review, 1 June, available from http://www.technologyreview.com/Biztech/12434/. Accessed 19 February 2007. ———— (2003), ‘Quentin Tarantino’s Star Wars? Digital cinema, media convergence, and participatory culture’, in David Thorburn and Henry Jenkins (eds), Toward an Aesthetics of Transition, Cambridge, MA: MIT Press. ———— (2006), Convergence Culture, Cambridge, MA: MIT Press. Kittler, Friedrich A. ([1986] 1999), Grammophone, Film, Typewriter, Stanford, CA: Stanford University Press. Krauss, Rosalind (1999a), Voyage on the North Sea: Art in the Age of the Post-Medium Condition, London: Thames & Hudson. ———— (1999b), ‘Reinventing the medium’, Critical Inquiry, 25: 2, Chicago: University of Chicago Press, pp. 289–305. Lessing, Gotthold ([1766] 1984), Laocoön: An Essay on the Limits of Painting and Poetry, Baltimore: Johns Hopkins University Press. Manovich, Lev (2001a), The Language of New Media, Cambridge, MA: MIT Press. ———— (2001b), ‘Post-media aesthetics’, available from http://www.manovich.net/. Accessed 19 February 2007. McLuhan, Marshall ([1964] 2001), Understanding Media, London and New York: Routledge. Mitchell, W. J. T. (2005) What Do Pictures Want?: The Lives and Loves of Images. Chicago: University of Chicago Press. Neumark, Norie (2006), ‘Different spaces, different times: Exploring possibilities for cross-platform “radio”’, Convergence, 12: 2, pp. 213–24. Richter, Gerhard (1995), The Daily Practice of Painting: Writings 1962–1993, Cambridge, MA: MIT Press.

74

Arild Fetveit

3 7–24

Introduction Arild Fetveit and Gitte Bang Stald Mixed media: from digital aesthetics towards general communication theory Klaus Bruhn Jensen

25–38

Remediation and the language of new media Jay David Bolter

39–56

Alan Kay’s universal media machine Lev Manovich

57–74

Convergence by means of globalized remediation Arild Fetveit

75–88

The website as unit of analysis? Bolter and Manovich revisited Niels Brügger

89–104

Gameplay as design: uses of computer players’ immaterial labour Adam Arvidsson and Kjetil Sandvik

105–118

On transdiegetic sounds in computer games Kristine Jørgensen

119–140

Power and personality: politicians on the World Wide Web Ib Bondebjerg

141–158

Online debate on digital aesthetics and communication Lev Manovich, Jay David Bolter, Klaus Bruhn Jensen, Arild Fetveit and Gitte Bang Stald

159

Film & Media Studies Yearbook 2007

intellect Journals | Film Studies

05

1

www.intellectbooks.com

intellect

9 771601 829000

Northern Lights cover.indd

Volume 5

Northern Lights

Contributors

ISSN 1601-829X

Volume Five

Volume 5 – 2007

Northern Lights: Film & Media Studies Yearbook | Volume Five

Northern Lights

ISSN 1601-829X

8/22/07

4:51:16 PM