Game Set And Match

gamesetandmatch conference proceedings

Kas Oosterhuis

1 ISBN 90-5269-316-1

Gamesetandmatch Delft University of Technology Faculty of Architecture

proceedings of the GSM conference, 13 December, 2001 gamesetandmatch, real-time interactive architecture

conference director: Prof. Ir. Kas Oosterhuis edited by: Ir. Hans Hubers Misja van Veen Chris Kievid Remko Siemerink graphic design: Kas Oosterhuis Chris Kievid Misja van Veen

published by the publikatiebureau Bouwkunde, Faculty of Architecture, TU-Delft [[email protected]] printed by Drukkerij NKB - Bleisijk ISBN 90-5269316-1

scientific committee Prof. Dr. Antonino Saggio [University La Sapienza Faculty of Architecture] Ass. Prof. Marcos Novak [University of Santa Barbara Interactive Media] Ass. Prof. Bernard Cache [Arch EPFL, Berlage Institute] Ass. Prof. Mark Goulthorpe [MIT Department of Architecture]

© 2003 by the authors

Table of contents 0 1 2 3 4 5

6

Gamesetandmatch, Kas Oosterhuis Introduction, Ole Bouman ‘Yes we build spaceships’, Kas Oosterhuis Intermezzo 1 - Projects, Hyperbodies Swarm architecture, Kas Oosterhuis Ada: Buildings as organisms, Kynan Eng a.o. Intermezzo 2 - Different Perspectives to real-time, Edwin van der Heide Levelsbyarchitects, Maia Engeli

3 5 7 15 19 33 44 51

9

Videogames, exploring an emotional architecture, Xavier Boissarie Intermezzo 3 - Multiplayer Collaborative Design Internet Game, Hans Hubers Beyond Kinetic, Michael Fox Eliminate the interface, Ted Krueger Intermezzo 4 - Real-time structural analysis, Nils Addink Gamesetandmatch forum discussion

60 73 77 94 106 110

10

References

128

7 8

2

0

Gamesetandmatch Prof. Ir. Kas Oosterhuis Principal architectural offfice ONL, Director Hyperbody Research Group DUT

“Architecture is the masterly, correct and magnificent play of masses brought together in light” as Le Corbusier put it in his manifesto Towards A New Architecture. Everything has changed since then, and we are ready for a new paradigm: “Architecture is the programmable hyperbody played skilfully by its masters at the speed of light”. Le Corbusier gave shape and meaning to architecture in the era of the Industrial Revolution. Let’s now programme hyperreality in our era of the Digital Revolution. Let’s face it: virtual reality is in all respects more real than what we take to be natural reality. Virtual reality including any

software ever written for any platform is hyper-real. Simply because we know the stuff it is made of. We know every bit and byte. In the Digital Revolution reality is being rewritten from ground zero. Architecture becomes a game being played by its users. And not only architecture will be subject to the forces of real-time calculation. Planning, construction, interior design and landscape design are also ready to be developed as real-time games. During the design process the game is designed by the architect and played by all parties involved. During the life cycle of the building and the build environment, the game is played by their users, by the visitors and by the built environment itself. Visitors become participants in the experience economy. By playing the game the participants set the parameters. Each actors triggers an 3

0

Gamesetandmatch array of sensors writing the new data into a database, from where the building picks up the new data and starts reconfiguring itself, in shape, in content, or in both shape and content. Then the new configuration is matched to the desired conditions. It is fair to say that the building will find itself in a state of continuous operation. The building elements consists of numerous cooperating programmable elements, behaving like a swarm. The building elements will show flocking behaviour, always keeping an eye on the neighbouring actor and always ready to act and react. Hence we propose a new motto for the discipline of architecture: “Game set and match”. To be played over and over again. Architecture is turning wild.

Web of North-Holland, design by ONL

4

1

Introduction Ole Bouman Editor-in-chief of Archis

Just about everything that makes architecture more than a technical construction, an accommodation, and an investment, is under pressure in the digital era. What does it mean to draw boundaries in a society where entire environments are intermixed by means of sensor and display technology and interface design? What does it mean to occupy a place when at one and the same moment you can be everywhere and nowhere, via GSM and

Ole Bouman

GPS, via avatar and alias. What is left to represent by means of buildings when there is little collective meaning anymore and messages are becoming more and more individualistic? In an age where investments must yield a profit for a large group of anonymous financiers, return and functionality are the dominant motives. The territorial is reduced to security. It is difficult to represent values when there are no longer any shared values. Ambitious clients turn into risk-shy investment companies. Spatial unicity becomes impossible under a regime of extreme cost-consciousness. What remains is a building profession that falls back on its core activity and is at risk of losing its cultural relevance. Unless...

5

1

Introduction Unless architecture is able to redefine itself and to broaden the substance of its design activities. And that will only be possible when it no longer leaves the design of the digital environment to trained specialists but incorporates it into the design of the physical built environment. If the virtual world is threatening to usurp the cultural relevance of architecture, then it is logical that architecture should seek to connect with that world.

6

‘Yes we build spaceships’

2

Kas Oosterhuis interviewed by Cindy, a virtual friend Principle architectural offfice ONL, Director Hyperbody Research Group DUT

Kas Oosterhuis: We build spaceships. Cindy: Spaceships? But you’re an architect!

virtual friend Cindy

Kas Oosterhuis: In 1994 we made a building for processing garbage in the Twente region. This service building we designed as a spaceship making a soft landing at the site. The spaceship was first thought out and modelled in zerogravity digital space. It then transmitted signals, so to speak, to the site to say it was on its way. The site replied with something along the lines of ‘all right, but don’t forget I’m a bit soft on top so I’ll first get a sturdy bed organized.’ The site then passed on all the data necessary for the spaceship to make a successful landing, bed and all.

7

2

‘Yes we build spaceships’ Since then we’ve made manymore spaceships which have landed at numerous sites across the planet, or were intended to land somewhere successfully. One of the most recent is our design for the Queensland Gallery of Modern Art for Brisbane in Australia. This spaceship exists as a 3D model and as a presentation on our server, but unfortunately it’s not been able to make contact with the authorities in Brisbane. Deep in the spaceship we’ve planned an interactive protospace, a space where the public can be fully immersed in a virtual world. A genuine holodeck. Cindy: When you say you design spaceships, do you mean to say that you yourself are a sort of alien, a creature from outer space? Where are your spaceships built, Kas?

Kas Oosterhuis: We are space travellers on earth, what we do is deliberately assemble the unknown. We want to surprise our audience, we want to show what we can do with the current resources here and now. We also want to surprise ourselves in the design process. In 1990 Ilona Lénárd and I held a series of workshops entitled ‘Artificial Intuition’. Our motto was: ‘We must train our intuition to direct our logic’. Kas Oosterhuis: We build sculptures. Cindy: Sculptures? But a sculpture isn’t a building! Kas Oosterhuis: A building can be a sculpture and a sculpture can function wonderfully as a building. At ONL our work is informed by the notion of electronically fusing art and architecture. In 1994 we organized the Sculpture City project, where we used digital techniques to carry out the entire process from design to execution. We designed everything 8

2

‘Yes we build spaceships’ with the computer, modelled it all in 3D and rendered the animations, and then made the models - the Clouds series - with the milling robot from the Industrial Design department at the TU Delft. An implicit factor in the Sculpture City concept is that when art and architecture join forces this must happen at the same scale, and artist and architect have to work with equal budgets. In other words, not your one per cent for art in and around the building, but a one-to-one relationship, on equal footing. Some years later we were able to put this concept into practice when designing and building

Kas Oosterhuis

the water pavilion. There we expanded our team with the composers Edwin van der Heide and Victor Wentinck. The Saltwater Pavilion is a building with real-time behaviour, feeding as it does on the unprocessed data delivered to it in a continuous flow by a weather station mounted on a buoy in the North Sea. Kas Oosterhuis: We build concrete science fiction. Cindy: Science fiction? But science fiction is one thing you can’t build, I mean science fiction isn’t real! Surely you can only do that in your mind. Kas Oosterhuis: For my final-year project at the TU Delft in 1980 I launched my ‘principle 9

2

‘Yes we build spaceships’ of concrete science fiction’. I wanted to show that if you can describe a scheme in concrete terms you can realize it in practice too. My proposal was to have a ‘legislative strip’ 5 km wide and 250 km long float above the Netherlands, cutting clear across all the major traffic arteries and rivers. It hung like a kind of sword of Damocles above the Netherlands. The idea was that legislation, besides being a binding text, could also have its own form - the form of the law. So one of my laws was to declare an aesthetic control-free zone in the shadow of the strip. The government commission overseeing the ‘visual decency’ of buildings would

then have no jurisdiction within this tautly defined area. The upshot of this measure would be that, on crossing the border into this strip of land the world would look completely different, as though you had driven into another country. An important factor is that the form of the law fails to coincide with a municipal or national border, but has the form that fits the idea. So it is the idea that fixes form and content, not the prevailing political circumstances. All power to the imagination. Cindy: And WTC 911? What about that then? Wasn’t that concrete science fiction? Kas Oosterhuis: I reckon you’re right there. It was an extremely concrete act, and an 10

construction of the Web of North-Holland, design by ONL

2

‘Yes we build spaceships’ impressive example of realizing the unimaginable. This is the sort of scenario you’ll find everywhere in science fiction movies. In almost every SF film we are confronted with disasters and the threat of evil. The kamikaze flights into the proud WTC towers give a bizarre twist to the principle of concrete science fiction. The action adds up on all fronts to a hair-raising scenario for a SF film. But this time it’s for real. It’s acceptable in a film, in the theatre, in a book. Every writer is allowed to think up the most nail-biting, blood-spattering scenes, every producer may film them but it must never happen in reality. But though I am keen on situations where reality and virtual reality are difficult to tell apart, as an architect I’m not interested in destruction. Not in the deconstruction of architecture, and not in cynicism in art. We make new structures with their own personality and have an unshakeable faith in assembling things that are beautiful and good. If you want to combat a network like Bin

Laden’s, the way I see it you can only do that by ignoring it and neutralizing it with other, positive networks. All that the Americans are achieving now is the short-term destruction of a number of nodes in that network. The network will soon replace them and simply continue functioning, very likely stronger than ever. This works just like our brains. That’s the strength of a network: if one distribution point breaks down, its functions are taken over by other nodes of equal standing. We can learn a lot from the cube-shaped Borg ship in the series Startrek Voyager: every Borg space station is part of the swarm and has a self-reconstructing capacity of an incredible 70% of its own volume. Cindy: How do you think we should deal with the new techniques and the new media? Kas Oosterhuis: I suspect that we humans will have to evolve step by step in a most profound collaboration with technique. Digital life may well assimilate us completely in the 11

2

‘Yes we build spaceships’ end, so that we no longer need pretend that we are the be-all and end-all of evolution. After us, much smarter forms of life will develop that will reach much further into the micro and macro universes than we with our physical limitations could ever do. And not just that, I think those new life-forms are already evolving and that without knowing it we are devotedly adding to their evolution. Sitting at my computer with the data flowing, I myself am facilitating this digital life. I’m providing a little sustenance for the globally distributed network of collaborating computers. I may still feel my computer is doing more or less what I want it to do, but the opposite is every bit as true. I can only do things with the computer that the

hardware and software allow me to do. I live in a no-strings-attached, peaceful co-existence with my computers. Kas Oosterhuis: We build dream machines. Cindy: Dream machines? What do you mean by dream machine? Surely a dream is more like virtual reality. How can you build it as a machine? Kas Oosterhuis: As I see it, virtual reality is more real than what we normally understand by reality. We know all about even the tiniest particle of virtual reality. We know exactly how the hardware works, and how the software makes its way through the integrated circuits. 12

2

‘Yes we build spaceships’ Virtual reality is more a kind of hyperreality. And in our work, we seek to have reality and this hyperreality collaborate in real-time. Buildings and built environments are becoming programmable. Form and substance can both be driven. An interactive relationship will effortlessly grow between the users and the built environment, in the way that users and smart appliances are beginning to communicate now. Buildings will develop into a smart swarm of building parts in contact with each other and with their users. All building elements will then know each others’ position and influence each other in real-time. Compare this with the behaviour of birds in a swarm. A few simple rules programmed into the birds themselves see to it that they don’t fly too close to one another, but also that the swarm remains a single

entity. Birds are always in motion. Our real-time dream machine works this way too: users and their environment are the members of the swarm, they are always in motion. All the time, building parts and users are taking account of one another. Reality and virtual hyperreality melt together into a new world of experience. Cindy: Who do you build your dream machines for? Who is your audience? And who is the client? Kas Oosterhuis: We build for people who have learnt to play. For children for whom it is natural to move effortlessly between reality and imagination. And for people with an open prospect of the future, for those who have not forgotten how to play. Our clients are those who see that society is changing into an experience economy which will then further develop 13

2

‘Yes we build spaceships’ into a transformation economy. These entrepreneurs see that there can be a lot of pleasure and a lot of money in this. They produce plans for interactive dream machines to give their public an experience and to transform them; to play with them the game of programmable e-motive architecture. Cindy: You build spaceships that are sculptures at the same time, and even dream machines in a concrete form of science fiction. It seems as though you two really enjoy your work. Kas Oosterhuis: Damn right we do.

14

Mathew Nelson

Intermezzo 1 - Hyperbody Projects Students participating in projects of the Hyperbody Research Group DUT “We make our buildings, and our buildings make us.” Churchill “Now we make our networks, and our networks make us.” Mitchell “Buildings, like people, are both senders and receivers of information.” Oosterhuis I believe that we no longer live in the past of the first statement, nor do we fully practice in the future of the second. We exist in that excited intermediate state, like water just before it reaches the boiling point. Therefore, we must design our buildings to house and display intense networks of virtual information, and at the same time we must give built form and spatial voice to heretofore-invisible information. The design of this architectural element is meant to be representative of an Internet shopping experience. The Internet gives the user total freedom of movement, while at the same time the user is constantly bombarded with visual and audible information. If we are to design a game we must define the roles of the players, their goals and needs. In this case, the players have been defined as the consumer and the producer. Their goals and needs are both to simple and complex to completely examine here, but let me say that I have expressed their roles as dependent on one another. The playing field is the Lijnbaan shopping district in Rotterdam, The Netherlands. The arena is my design.

15

Living in the same place all the time can be annoying, just because everything is always the same…. There are people who like to change from environment constantly. As architecture is part of the environment I tried to apply it to make a change about that. The cubes in this whole building are the apartments that are moving slowly horizontally and vertically. In this way I wanted to create a more exciting and interesting experience on the way home. Isn’t it much more exciting and interesting to find things out every time going home; wondering where your home is, how to reach it and who your neighbours will be, how the view will be looking through your window, wondering if you will be on top of the building this time or will you have that nice little garden on the ground floor? Just ask (by speaking to the computer in the elevator) and the building answers you. Sounds, light beams, scrolling text and information on the wall; everything will lead you home and give you information about the whole place. All those signs are moving the person or visitor and it keeps him / her excited. Everyday is another experience! It’s just like a game, and you are playing the hero in it….

16

Fatima ElBouyahyaoui

Chris Kievid

A hyperbody, capable of dividing itself into a swarm of multiple morphable metaballs, transports itself in a “player-made” interactive environment. Moving through the open spaces left behind by the internal interactions of the other players, the body functions as an internal transportation device for both physical as non-physical data. It is a data processing machine (metabolism), absorbing the incoming information, processing that given information and delivering it in another form. This organism feeds on the fresh data from local parameters, the internet and from direct input by the users. This information settles in the hyperbody as a hypersurface, perceiving the spaces in and around the hyperbody as programmable and driven. This fresh data is the flocking parameter for the data-driven body to change its shape, content and position in real-time. That information of course travels as hypertext does; via warp holes from one universe to another. The body is in constant action, always reconfiguring itself. This body is movin’. It behaves as a unitary whole, maintaining its identity in space, resisting dissolution . . . neither a thing nor a concept, but a continual flux or process. It is a distributed memory that both perceives and remembers. The hyperbody can be distributed in 4 distinct facets: (1) the absence of imposed centralized control, (2) the autonomous nature of (temporarily) subunits, (3) the high connectivity between the subunits, and (4) the webby non-linear causality of peers influencing peers. It got self-consciousness. This body displays real-time behaviour! The data is transported from one position to another by the morphing, translating, rotating and merging of the metaballs (both single-vectorial as multi-vectorial). These bodies are made up of inter-connecting data streams, and in-between-surfaces, acting as activators for the movement. The data streams are the actual transport devices, both input as output, for the physical data (goods and people). The in-between-surfaces act as muscles, setting the data streams, through contraction and relaxation, in the wanted positions, making exchange possible. The body is docking, inter-lockin’ and up-rockin’.

17

to see the world in a grain of sand, and heaven in a wild flower, hold infinity in the palm of your hand, and eternity in an hour.´ - William Blake Imagine it is 21.00 and you are still busy working on this text but you get tired and a little inefficient. Wouldn’t it be nice if your space would notice this and gently turn your work mezzanine into your favourite relaxation-landscape with your favourite forest scrolling over you. It looks so beautiful. gently the rose turns up your beloved music and turns on a subtitle breeze. You love it when she does that. With a simple ´hmm´-sound you can deny a proposal and tell her what you really like. As you wonder you think of your wife who is on that biz-trip in Asia and you decide to contact her. You do not get up but tell the rose to render her on the soft floor next to you. She is also in her rose so you can not only see her hires but you can even feel her body move against yours (long-distance motion). One more week and she will be back. Would it not be wonderful... ´A rose is a rose is a rose´ - Rilke

18

Michael Bitterman

3

Swarm Architecture Prof. Ir. Kas Oosterhuis Principal architectural offfice ONL, Director Hyperbody Research Group DUT

Abstract: This paper gives the authors vision on recent insights in human and other animal behaviour and developments in Informatics and Communication Technology. Based on these insights a new approach to architecture is developed in which collaborative design plays an important role. Architecture should become a multiplayer interactive game. The idiot savant Struck by an article in the Scientific American about “Idiot Savants” I suddenly realized that the idiot savant is exactly what the computer is to us. An idiot savant is a person who is

proposal for Ground Zero, Max Protech Gallery New York, design by ONL

mentally retarded but has remarkable capabilities in a specific but limited field of knowledge. Some of these narrow-band geniuses can for example remember every single telephone number of a complete telephone book. Some of them can reproduce the complete contents of a Shakespeare novel, but without any understanding of what it is about. They appear to have direct access to their project database in their brains. Without the filters of insight which intelligent people are so proud of having. I was very excited to read this essay, because it has enormous implications for the understanding of our brain, and for the understanding of how we work with computers. Direct access to the project database Direct access seems equivalent to the “list” function in programmes like Autocad. It is an open tunnel to a table containing all registered data. In human brains these data come in through our eyes, ears and nose. Everything seems to be recorded, but normally we can remember 19

3

Swarm Architecture only a fraction of it. This is because we filter these data in order to be able to handle socially. It would be very inconvenient if we could literally reproduce everything, which was stored in our brains. Reproducing the data would take a lot of time (“you only live twice”) and we would not have time left to interact socially, we would not be able to form a society. Have you ever seen those piles of paper, which the computer produces when listing a complex calculation? You do not want to read this, you may have a look at it, maybe only to spot obvious inconsistencies, but not for actual reading or understanding. Having direct access to the project database is blocked for social reasons. The shells around the database form the interface between database and environment. So is the case with computers. The operating systems, the computer programmes form shells around the project data, to make it possible for us humans to communicate with these data. To make it possible for us to give meaning to the data, to make interpretations, to propose changes, to produce new data and to have

them sent back to the database. These shells represent our social transaction space. Dreaming and intuition Am I only dreaming? In our dreams we have direct access to our personal image-banks. Just like seeing is an action in real-time, also dreaming is an activity constructing storylines based on a vast database of images, relations, proportions, contours, colours, smells, texts, and other possible disguises of information. And it is my guess that deep down there is a generic storage medium running, storing all information – images, texts and relations alike – as raw data in a dynamic project database. In earlier days, briefing the participants in the workshops “Artificial Intuition” in Galerie Aedes and at the Delft University of Technology, Ilona Lénárd and I used to say: “We must train our intuition to operate our logic”. Now we understand that we were talking about direct access to our project databases, unfiltered, spontaneous, direct, actual. We wanted to establish a hotline between intuition and logic, like idiot savants. We know now and we knew then that our personal computers play a crucial role in this procedure. We found out ways to connect our intuition to the calculation speed of the computer. 20

3

Swarm Architecture Split second Training the intuition is top sport. A Formula 1 driver, a top tennis player, a soccer top scorer, they make decisions in split seconds. These decisions are way faster than thinking. What happens here in the brain? They must have these kind of hotlines also, these shortcuts between their motor system and their Formula 1 database, tennis database, soccer database.

Kas Oosterhuis

Deep down in their brains all possible situations and actions are etched, and in a split second they choose for the right action. Faster than lightning, more precise than extensive calculations, very direct and absolutely accurate. Being heavily involved in the actual design process, the top designer makes these split second decisions as well. Good design is top sport. And we believe that this sensitivity can be trained. Intuition can be trained. Which is an emancipator liberation process. One should allow oneselves to have access to the project database that directly. 21

3

Swarm Architecture PC computation power From Ray Kurzweil we learned that by the year 2050 one single Personal Computer may have the computing power of the total human population of the earth. Just imagine! How will

we humans relate to these small but extremely powerful machines? How will we work with these super idiot savants? This is in my opinion the central issue, which our society is facing the coming decades. We do believe that a well-trained intuition is necessary for a successful communicative relation between (wo)men and machine. It is obvious to us that it is useless trying to compete with the calculation speed of the computer. We should see them as friendly open extensions to our brains (exo-brains), where we can have access to in the way we want 22

3

Swarm Architecture it and when we desire so. Sometimes ultra fast, sometimes very slow and filtered. Sometimes immediate like lightning and sometimes as blurry and fuzzy as can be. Project database So we have numerous project databases deep inside our head. And we want to be free to swap from the idiot to the savant in the way we connect to these data. There are project databases in each computer, where the user wants to be free to connect to or to close off from. The wiring inside our heads and the information flow inside the computer are separate

the Web of North-Holland at the Floriade design by ONL

different layers within the same project database, influencing one another. Each design project forms its database both inside our own heads and inside our computers. The project must be stored as a 3D model where we can have different views upon. One view is the stereometric view (perspective), the other view is the flat cut (section, plan), the third view the numbers. Other possible views are the physical tactile model, the smell, the sound. Each view is another look at the same thing. The numbers represent exactly the same thing as the 3D model. Only the shell around the data is built different as to generate a specific view on the data. One can change the data through all the different views. If one changes the 3D model, the data change. If one changes the data, the 3D model changes. If one changes the plan, changes are seen in the 3D model and thus in the data. This concept is crucial for all the work our office is dealing with the last ten years; for all our proposals dealing with parametric 23

3

Swarm Architecture design, artificial intuition, real-time behaviour, active structures, multiplayer games and transaction spaces. All our work is based on the notion that it is data-driven. Data-driven process Information technology gave us the platform to communicate with other disciplines. Working together in a data-driven process of collaborative design and collaborative engineering the stakeholders behave like birds in a swarm. They are like bees, dynamically socializing ina hive, exchanging information in the flow, developing a hive mind. They are the swarm. All members of the swarm are data-carriers in disguise. Every member is a node in the information flow network, in their own disguise, playing their specific role. But in the end it all

comes down to exchanging data. The players are distributed beings, absorbing, processing and distributing data. All these data processing vehicles (yes, also people are vehicles, and as we will see later, buildings are vehicles too) operate in swarms, and all these swarms exchange information with other swarms. There are swarms on all social and physical levels and on all time-scales. Communication People communicate. When they talk they produce output in the form of spoken language. When they move they produce output that will influence other people’s movements. When they write things down they store the output in a buffer which may be activated later. People communicate with other people, with animals, but can people communicate with buildings? What sort of language must they speak to achieve that? And do different elements of the same building body communicate? And what sort of language will they use to establish the communication? 24

aluminium skin of TT-Monument design by Ilona Lénárd

3

Swarm Architecture Suppose that communication nothing more and nothing less than pure information flow. When people communicate with people the information flows from brain to mouth through air to ear to brain. When building elements communicate the information may flow through wires, or maybe even wireless through the ether. The sender sends the signals, the receiver absorbs the signals. Our building bodies are wired. Our cities are wired. Wires being any infrastructure, bringing fresh data in, and carrying concerted data out. Very much like the blood vessels and the lymphatic system of animal bodies. With interactive architecture we want to establish a two-way communication between the people (the users) and their environment (the house, the office).

Senders and receivers Some swarms move very slowly like geological processes, others configure themselves with the speed of light. Swarm architecture is based on the idea that all building elements are each one of them data-carriers as well, and that they all are members of a swarm. Swarm behaviour has been drastically evolved since the immersion of digital life into our daily lives and into the very fabric of building materials. Building components are now potential senders and receivers of information, exchanging data, processing incoming data, and proposing new configurations as the outcome of the process. People communicate. Buildings communicate. People communicate with people. People communicate with buildings. Buildings communicate with buildings.

25

3

Swarm Architecture The swarm In the Swarm the designers exchange information with their clients, and with the other stakeholders in the process of building dreams. They exchange information with other disciplines in the collaborative design process, they work together with visual artists, composers, graphic designers, planners, publishers, broadcasters of information, and with other architects. They exchange information with construction engineers, installation engineers, project managers and process managers. And they want to establish feedback loops, because they want to learn from each other. They feed upon data from the other parties.

The hive To facilitate the process of collaborative engineering we need to build rooms for group design and decision. We could call them the Hives. One such a Hive will be the second life of the pavilion the Web of North-Holland. Now a not so very interactive propaganda machine for the Province of North-Holland, but after the closure of the Floriade a professional fieldlab at the Delft University of Technology where students plug into the Hive mind with their laptops. The Web of North-Holland will then become a true transaction space. The students and the research staff of my Hyperbody Research Group wants to use the space as a Group Design Room. The Building Management group would be able to use the space as a Group Decision Room. Within this Hive multi-actor and multiplayer networks will be established. Collaboration and transactions in the Hive can only take place when there is a smooth two26

3

Swarm Architecture way interaction between the stakeholders, when all parties involved are active, and when all parties are willing to offer the best of their knowledge and intuition. Transactions are done by submitting data to the project database, in any conceivable disguise, through any interface. We could experiment with sensors, keyboards, num pads on mobile phones, GPS systems, speech recognition, mouse, joysticks, bitmap-tracking as negotiating components. We will absolutely not work with gloves and headsets because this we deem too restrictive for the freedom to move within the interaction space.

Hyperbodies The stakeholders exchange information with the producers of building components, with the producers of digital techniques. They exchange information with their computers, they work together with their digital instruments to build new possible realities. They are the players in the input > processing > output game. Each one of the players operates in their own personal distributed swarm, and all swarms are connected. Swarm architecture is naturally based upon parametric modelling and on parametric behavioural patterns. Swarm architecture is based on the communication between and within the Hyperbodies. First we need architectural vehicles to carry the data, we need synthetic bodies. Architectural bodies are like the uni-bodies of automotive vehicles. Architectural vehicles are e-motive vectorial bodies. The Hyperbodies are hyperlinked integrities, which are built and sustained to process and evolve their parametric formula’s. The parameters are submitted to and extracted from dynamic databases in real-time. 27

3

Swarm Architecture Transaction spaces Swarm architecture is a true transarchitecture since it builds new transaction spaces. It is at the same time emotive, transactive, interactive and collaborative. Swarm architecture feeds on data derived from social transactions, it is the hive mind of the new transformation economy. Swarm architecture is design, construct and operate in real-time. Architecture becomes the discipline of building transactions. That is what architects do: they build transaction spaces. Architects are step by step becoming conscious of the fact that they are the designers of vehicles, which execute a game of life and death. Architecture has no longer as a hidden agenda to resist to external and internal forces. Architecture now becomes the

science of dynamic structures and environments running in real-time. Architecture goes wild. In the meantime the other stakeholders in the collaborative design process are experiencing the coming-out of Swarm architecture. Swarm architecture manifests itself as the inevitable evolution of architecture and the building industry. Parametric design Building and Architecture has from now on two aspects to it: at the one hand one creates a physical environment, at the other hand one designs the behaviour, the rules of the game, the states of mind of the buildings and the environments, directly connected to the physical places. We like to label this as enhanced reality. The design work in both reality and enhanced reality heavily relies on the parametric basis. If not built parametrically one can not play with the parameters, and one is not able to interfere with it, to communicate with the 3D model and 28

TT-Monument design by Ilona Lénárd

3

Swarm Architecture the project database, neither in the design process nor in the life-cycle of the environment. Working with parametric models creates the communication space for the stakeholders in the building process to discuss the qualities of the proposed environments. It opens up the design process for collaborative engineering. It also opens up the design process for a possible and meaningful interaction with the clients and the users.

File-to-factory Parametric design is absolute compulsory for the file-to-factory process of making the architectural bodies. One must directly connect the 3D model of the design to the production techniques in the factory. The connection relies on a common language, which is spoken by both the machines of the designer and of the producer. Also here we need direct shortcuts, unfiltered. For the production process of the TT Monument we made a NURBS surface 3D model and sent it as an IGES file to the milling machine of the model-maker. For the productionof the Web of North-Holland we wrote an exact procedure how to take the separate elements from our 3D model and prepare them for the production with the cutting machines for the steel. We wrote in our office an Autolisp routine which describes step by 29

3

Swarm Architecture step the exact procedure. One parametric detail fits all. One building, one detail. One work of art, one detail. But always parametric. Each element is unique, but each element undergoes the same procedure. We are fully immersed now in the industrial production process of mass-customisation. The game To facilitate the concept of collaborative engineering we must build a game, which is a form of open architecture in real-time. We think it is extremely important that the designers not only talk about the process of collaborative engineering, but that they actually must make it work. First then they see and feel how beautifully complex it is, and how precise they must

act and think. They must think as programmers, writing code. The designers must deal with the simultaneous development of the design and the communication. The software, which we use for this purpose is based on a graphic interface, it is pure visual programming. Working with the Virtools software includes the design of the architectural environment and the structure of the communication process. Rules of the game The whole process of interaction, communication and collaborative design is a parametric game. The designers design the rules of the game, and at the end of the project, they play the game. They design the design. Playing the parametric game of architecture is experienced by the players as a form of serious fun. The design is the formula, playing the game means setting the parameters. The players start to realize that if you connect the 3D model of the architectural design to the databases (tables, arrays) the essence of the architecture is not 30

3

Swarm Architecture one arbitrary choice of how the environment should look, but actually a multitude of possible architectural schemes, which all are just as valid and beautiful as the other ones. Direct democracy In the end the concept of interactive gaming technology for collaborative engineering is a strong tool for direct democracy. Now we directly connect to the people we work for and we work with. Not only experts are participators in the process of direct democracy, but especially also our clients, the citizens, friends, accidental users, passers-by. Everyone becomes a player in the transaction space, either consciously as a participator in the

Variomatic design by ONL

design process, or unconsciously as a passenger whose presence matters for the real-time behaviour of the transaction space.

31

3

Swarm Architecture

ElhorstVloedbelt, garbagetransferstation design by ONL

32

4

Ada: Buildings as organisms Kynan Eng Institute of Neuroinformatics, University / ETH Zurich

Abstract: Buildings are usually conceived as expressing a static functionality where users are required to adapt to its pre-defined properties. This view of architecture is outdated. Buildings can be conceived and constructed as perceiving, acting and adapting entities not unlike biological systems. Although several existing projects have aimed at developing “smart” rooms or buildings, in most of these attempts information technology is used in a strictly local

artist impression of ADA

way. “Ada: Intelligent space” goes beyond this utilitarian perspective and aims to develop a space that can dynamically change its overall functionality and quality through an active dialog with its visitors. Named after Lady Ada Lovelace, one of the pioneers of computer science, Ada is conceived as an artificial organism that can interact and communicate with her visitors. She is based on current research in neuroinformatics. By means of her senses, i.e. vision, audition and touch, Ada will be able to locate and identify visitors. Her effectors, i.e. lights and sounds, will allow her to provide cues to visitors and express her internal states in an emotional language. Key functionalities of Ada are: to balance visitor density and flow; to identify, track and guide “interesting” visitors; to group selected visitors in space; and to play games with visitors. Ada has to achieve these behavioural goals by establishing active 33

4

Ada: Buildings as organisms interactions with her visitors. Ada is being constructed for the Swiss national exhibition Expo.02 (May 15-October 20, 2002) which is held once every 30 years. As an exhibition project, Ada

aims to initiate a public debate of the application and implication of brain-based technology on our future society. This paper describes the concepts behind Ada, some of her key sensor and effector technologies, and how this collection of devices is integrated using a neuronal control system. Examples will be discussed from recent tests of this system during the Zürich festival of science and Zürifaescht. 34

4

left: overview of activity at the Zurich Festival of Science.

Kynan Eng from the Institute for Neuro informatics

Ada: Buildings as organisms Introduction From the beginning of civilisation, humans have been installing infrastructure in their buildings. This infrastructure – physical, plumbing, electrical, environmental, communications – is designed to support the intended uses of that building. The users of the building serve as the regulatory intelligence that makes all of the subsystems work together to serve the users’ needs. Until the last years of the 20th century, the structure and organisation of buildings was sufficiently simple and static that intermittent control by users provided more than enough computational power to maintain building integrity and operation. However, this situation is changing. In many buildings, especially large multi-purpose buildings such as airports,

shopping malls and skyscrapers, the number of dynamically modifiable components is increasing sharply. Both the structure and the operation of these buildings are becoming dynamically reconfigurable. Such reconfiguration is expected to occur under constraints of power, resources and the functional requirements of individual users. The growing demand for dynamic reconfiguration of building services is placing increasing demands upon the people controlling those services. To reduce the amount of active human intervention needed to keep things running smoothly, more of the necessary data processing is being delegated to automatic devices. Some of these devices form part of the building’s permanent infrastructure and some are objects (or organisms) that pass through the building with varying residence times. Modern industrialised societies are rapidly filling their buildings with interconnected devices, with an ever-increasing number of subtle interactions and side effects. But what happens when the components of a building reach a critical density in 35

4

Ada: Buildings as organisms space and interconnectedness? Two things then occur: 1.

Users begin to observe correlations between the actions of different subsystems of the building, which reach a certain level of coherence.

2.

The overall operation of the building becomes opaque; non-experts are no longer able to control it. They can, however, perform actions on subsystems and observe the consequences, looking for correlations.

Under these conditions, the users of the building must engage in a dialogue with the building in order to achieve their goals, rather than simply controlling a few parameters. In the course

of this dialogue, the users begin to treat the building as a single entity, and attribute purposeful cognition to the processing underlying the building’s actions. From the viewpoint of the user, the building becomes more than the sum of its shell and subsystems; it appears to be alive. This change in our relationship with artefacts is not limited to buildings. In the future more of the objects we create will become more autonomous in completing more complex tasks. The project “Ada: intelligent space” is an exploration in the creation of living architecture. It can be distinguished from existing “smart office” or “smart building” projects in four main ways. Firstly, the level of behavioural integration being attempted is far more extensive than in other projects. Secondly, the user interaction with the space is immersive rather than invisible – the building does not quietly serve its users’ needs in the background, but becomes an active participant in their lives. Thirdly, the functionality of the space in its interactions with users is not fixed, but time-varying and adaptive. Finally, and most importantly, the space has its own goals which it actively tries to achieve by engaging its users. Ada is the product of interdisciplinary research into the functioning of the brain at the Institute of Neuroinformatics 36

4

Ada: Buildings as organisms (INI). The mission of the institute is to discover the underlying principles of brain function and to implement these principles in artificial systems that interact with the real world. The space and its users Ada is intended to stimulate public discussion of brain-like technologies and the social implications of their usage. “She” is being constructed as an exhibit at the Swiss Expo.02 in the town of Neuchatel, on a purpose-built platform over a lake bearing the same name as the town. She will function continuously for 10 hours a day over 6 months. The exhibit’s components are shown below. Region area description Conditioning tunnel (entry corridor) 65 m2 Visitor waiting area, including a staged visitor

introduction to the components of Ada and their basic functions Interaction space 175 m2 Octagonal room where all interaction with Ada occurs Voyeur area 81 m2 Corridor around interaction space so visitors can observe without interacting directly Brainarium 45 m2 Semitechnical display room showing internal processing states of Ada, using the observatory metaphor. Includes windows so that people can see back into the interaction space. Explanatorium 45 m2 Explanation and discussion of key technologies behind Ada Lab area 73 m2 Computer room, working area for operators The overall goal of the exhibit is to entertain and inform each visitor during an average 10-15 minute visit. This is a difficult task, as Ada’s visitors will have a wide variety of ages and educational/cultural backgrounds. What they do have in common is that they have paid to see Ada, and expect educational and entertainment value for money. Most will have little expertise in domains such as architecture and neuroinformatics, but many will have widely varying preconceived ideas and opinions about these topics, often based on public media hyperbole. Some of these opinions will be technically infeasible or considered irrelevant by “experts”. It is of paramount importance that Ada does not disturb her visitors’ preconceptions too much without giving them new food for thought. Ideally, each visitor will 37

4

Ada: Buildings as organisms leave Ada with a positive impression, gained from her entertainment value and a sense of having something novel and challenging to think about regarding the future of brain-like technologies. Sensors & effectors All organisms need some means for collecting information about the world. Ada will be equipped with sensors that mimic some of the capabilities of organisms: vision, hearing and touch. In the space the following sensors are applied:

38

-

Vision: Ada has a vision matrix – a grid of cameras mounted above the floor pointed vertically downwards that can be used to watch visitors everywhere in the space. A number of pan-tilt cameras called gazers are also available for Ada to use for attentional, focused interactions with specific individual visitors.

-

Hearing: There are clusters of three fixed microphones in the ceiling plane, with which Ada is able to identify and locate sound sources by triangulation. Directional microphones are also mounted on the gazers. Some forms of sound and word recognition will be available.

ADA at the Expo’02 in Neuchatel, Switzerland

4

Ada: Buildings as organisms -

Touch: Ada has a “skin” of hexagonal pressure-sensitive floor tiles (Delbrück 1999) that can detect the presence of visitors.

As well as sensing, Ada can also express herself and act upon her environment in the following ways: -

Visual: Ada uses a 360° ring of LCD projectors to express her internal states visually to visitors. There is also a ring of ambient lights for setting the overall visual emotional tone of the space. Local visual effects can be created using the RBG coloured neon lights in each floor tile in Ada’s skin, rather like a chameleon.

-

Audio: Ada is able to generate a wide range of sound effects. She expresses herself using sound and music composed in real-time on the basis of her internal states and sensory input. The composition is generated using a system called Roboser (Wasserman 2000). She is also able to perform a simple form of “baby talk” which imitates what she hears from her visitors.

-

Touch: Ada has a number of pan-tilt movable light fingers for pointing at individual visitors or indicating different locations in the space. For reasons of safety and human psychology, we have chosen not to have Ada act on her environment using forceful actuators such as robot arms.

39

4

Ada: Buildings as organisms Behaviours & Interactions Sensors and effectors alone are not enough for Ada to convince visitors that she shares properties of a natural organism. To achieve the feel of an organism, the operation of the space needs to be coherent, real-time, understandable to most visitors (the conditioning tunnel plays an important role here in providing initial visitor priming) and sufficiently rich in depth of possible interactions that visitors feel the presence of a kind of basic unitary intelligence.

To provide for a natural progression in visitor interaction, Ada incorporates at least four basic behavioural functions. First, she will track individual visitors or groups of visitors, possibly (but not necessarily) giving them an indication that they are being tracked. At the same time, she will identify those visitors who are more “interesting” than others because of their responsiveness to simple cues that Ada uses to probe their reactions. These people are encouraged to form a group in part of the space through the use of various cues. When the conditions are appropriate, Ada can then play a number of group-based games with her visitors. All the time she evaluates the results of her actions and expresses emotional states accordingly, and tries to regulate the distribution and flow of visitors. It is important 40

detail of a floorpanel used in ADA

4

Ada: Buildings as organisms to realise that the above scenario is not a static progression, but a set of interconnected, interdependent, simultaneously evolving internal processes. The underlying software is a mixture of simulated neural networks, agent-based systems and conventional procedural or object-oriented software. Integration & Testing Exploring unknown territory in organism-like architecture with a system as large as Ada could not occur without experience with smaller systems. As well as the continuously developing demo space at INI, since 1998 a number of increasingly large public system tests were run to evaluate the feasibility and scalability of the underlying architecture, gauge visitor

Visitors playing a group football game at Zürifaescht. They are trying to trap the bouncing white tile in the corner of the floor

impressions, and test different interaction scenarios. One of these tests was at the Zurich Festival of Science (Zurich main station, May 2001, 100,000 visitors), where a system called Gulliver was run for three days in collaboration with the Remote Sensing Laboratories in the Department of Geography of the University of Zurich. Gulliver contained light fingers that could follow visitors, and a floor used as a large collective joystick to control a simulated flight over Switzerland. The main goal of this test was to gain experience in the technical and logistical aspects of running an exhibition. Zürifaescht, the triennial Zurich city festival in July 2001, provided the setting for the following system test. This time a rather more Adalike system called Gulliver II was deployed over three days, including an approximation of a raised “brainarium” area, from where spectators could observe the space and some displays showing the internal operations of Gulliver. Some basic Ada functionalities were tested, such as visitor tracking with floor tiles and light fingers, displaying visitor trajectories, sound 41

4

Ada: Buildings as organisms localisation of handclap noises and a group football game. Overall, the two tests proved to be very useful in affirming what the team was doing well, such as providing good system stability and adaptability to on-site conditions. The tests also indicated areas where improvements were needed. The two key issues that stood out from the results of the tests were a need for effective visitor flow control, and the importance of communicating Ada’s intentions clearly through the use of effective cues and visitor pre-conditioning sequences.

Outlook In four months Ada will be operational when Expo.02 opens. But even then her development will not be complete. Apart from the usual tuning period that occurs with any public exhibit, Ada will be performance art and a living experiment. In the background, data collection related to fundamental research will occur using Ada as an experimental platform. The experiences and reactions of her visitors will lead to as-yet unknown changes to her behaviour, some introduced by Ada’s designers and some from Ada herself as she adapts to her visitors. Continuing system upgrades will incrementally increase Ada’s capabilities. So, over the course of Expo.02, Ada will interact with visitors, express herself and grow – just like 42

4

Ada: Buildings as organisms the other organisms living and working at the exhibition.

floorplan of ADA on Expo ‘02

Acknowledgments Organisations supporting the Ada project include the SNF SPP program, ETH Zurich, University of Zurich, Expo.02, Manor, and Rotronic.

43

Intermezzo 2 - Different perspectives to real-time Edwin van der Heide Composer/Performer/Designer of electronic music and sound installations Abstract: In this paper the metaphor of a musical instrument is used as a way of looking at interaction and games. The quality of the interaction depends on the actuators (sensors, keyboard, joystick etc.), the rules of the game, the public (individual or group), the instrument and many other aspects. Architecture is becoming more and more a real-time medium. I have a strong interest in relating to this aspect of architecture. In the following proceeding I’ll approach the real-time aspect from my background. This background originates from the electronic music tradition and has expanded itself more into an installational approach in which space and audience fulfil more active roles. There is a long tradition in the development of musical instruments. When we look at architecture as an instrument, there are many interesting comparisons to make. Traditionally there is a strong separation between the composer, the conductor, the performer and the audience. In Holland Dick Raaijmakers has developed extensive theories starting from this perspective and then transforming it into more appropriate models for contemporary art forms. The relationship between the different elements is very hierarchical. Because of this there is no real possibility for the audience to interact with the artwork itself. In this sense a traditional musical performance is a very closed art form. The audience is a recipient in stead of a participant. In certain musical forms the separation between the composer and performer does not exist. Improvised music is an important example of this. The medium is not the score but the performance becomes the medium itself.

44

fragments from film #11

Close up of Sensor used in SoundNet Edwin van der Heide

Personally I have a very strong interest in working with sound or to be more precise working inside sound. My work is not using a combination of sounds to create a piece; my work is the (transforming) sound itself. The traditional hierarchy in music disappears at the moment of the acceptance of this approach. I’ve been using, researching and developing different sensor based interfaces to investigate possibilities for instruments which have special qualities for controlling sound while it’s sounding. There is a lot of research being done at developing ‘alternative controllers’ at STEIM, Amsterdam and The Royal Conservatory in The Hague. The area you enter when you start doing this is quite complex. The separation between the composer and the performer disappears but there is more. You start developing your own instrument. Every instrument has it’s own special qualities. The problem, which appears now, is that the next step is not only about learning how to play this instrument but also about creating the (new) musical language that matches the instrument. Of course you can also work starting from the definition of the musical language and then design the instrument. In practice it most often is a process which goes both ways. In my opinion the language aspect is most important. It is through the language you communicate. I think it is very valid to look at a musical instrument as an advanced interactive system/installation. I call it advanced because I think a well-designed instrument should be able to be very consistent and very diverse in sound at the same time. By playing and practicing you can master the instrument more and more. However you can play many different pieces on the same instrument and different players can have their own styles and specialties. I think the described qualities are directly relating to qualities an interactive environment should have. The designed language of the interaction is very important. No language, no communication. At most acoustical instruments the human body is in direct contact with the sound production. With wind instruments it’s your breathing which is directly responsible for, and part of the sound production. The same counts for example for the violin and the guitar. Your fingers are touching the strings which produce the sound. The piano and the organ have a more distant approach from the human body. At the piano the finger touches the key. The hammer then touches the string. The keyboard with the hammers is the controller or actuator. The strings are the sound generators. When using a computer there is always the separation of action and reaction. In analogue systems a volume knob on an amplifier is part of the circuit. Action and reaction is the same thing. In digital systems it is not. When you’re building an electronic instrument that is sensor based, the information of the sensors is comparable to the keys of the piano. I would say that the sensors form only one small part of the

45

instrument. The second part is the definition of how the information is being interpreted and translated to control sound. The last part is the design of the sound generation itself. The information that comes from the actuator (sensors, keyboard, joystick, etc.) is not at all related to the response yet. When using a digital system the rules between input and output are open. This is what interaction design is really about. There is a lot to explore on the side of design of the actuators and even more on the design of the rules between action and reaction. SoundNet SoundNet is a live performance musical instrument of monumental proportions created by Sensorband. It is a giant web, 11 meters by 11 meters, created with 16mm thick rope. At the end of the ropes are eleven sensors placed that detect action, stretching and movement of it. The instrument is being performed by climbing it. Sensorband consists of Edwin van der Heide, Zbigniew Karkowski and Atau Tanaka. The trio was founded in 1994 after a concert in Berlin where each of us played solo performances. Each of us was using custom developed sensor based electronic instruments. It was at that concert that I thought it would be very interesting to confront the different instruments, and the different styles of playing them, with each other. Each of us is very interested in creating physical interfaces and relationships with electronic sound generation. Starting the trio was a real challenge because it’s very different to play your instrument solo then having to open a dialogue with other musicians. It’s a real ‘test’ to see how flexible, communicative and expressive the instruments are. SoundNet is inspired by ‘The Web’, a 1 meter diameter spider’s web created by the composer/performer Michel Waisvisz, at STEIM, Amterdam. The general concept behind The Web is to create a natural complexity between different parameters that are controlling sound. When you touch The Web at one point

46

the tension distribution over the whole web changes. Since the sensors are spread over the web one movement results in a complex change of parameters and therefore in a complex change of sound. SoundNet has the same complexity of control as The Web. However SoundNet is extreme in the relation of physical input and audible output. It’s totally the opposite of controlling sound with a mouse. Human physical limits become part of the performance and the sound. The sensors are continuously measuring the different tensions in the net. The values measured are used to trigger sound and more important to control and change the sounds while they are sounding. The data from the sensors goes into the computer. A computer program contains the rules for the relation between the sensor data and the control of the sound. Also the computer is generating the sound in real-time. There are no external synthesizers or samplers being used. Another important aspect is that SoundNet is not intended for one performer. We perform it with the three of us at the same time. Since we all climb the same web that interconnects us we’re all controlling the same sound. It’s like playing one instrument with three people at the same time. It’s not like playing the piano with six hands but like playing the flute where the breathing of three of players is being added and controls one flute. We don’t only interact with the sound but through sound we interact with each other. Sometimes you gain control, sometime you lose control. Ted Krueger and Edwin van der Heide (right)

I think it’s a very valid approach to look at SoundNet as interactive architecture. When you’re designing interactivity it’s very important to be aware of how to deal with multiple participants. Are the results of

47

different independent systems added (this is similar to a music ensemble) or are multiple participants responsible for one output (this is similar to SoundNet). In the last case it becomes a very interesting question how to make participants aware of it and more important how to give meaning to it. SoundNet has become a performance instrument and not an installation, because in the case of SoundNet you have to be trained in order to be able to interact with each other. Spatial Sounds (100 dB at 100km/h) Spatial Sounds is a collaborative work together with Marnix de Nijs. De Nijs his work focuses on installations that experiment with the physical limits of perception. Spatial Sounds is an interactive installation; it’s a horizontally rotating arm with a loudspeaker at the end of it. The movement of the arm and the sound of the speaker are generated in real-time by custom developed software on the computer. There are two ultrasound sensors build into the speaker. The ultrasound sensors measure the distance from the speaker to objects in front of the speaker. There is another sensor on the bottom of the rotating axis that continuously measures the angle of the axis. The data from the sensors is being communicated to the computer. The installation first learns the shape of the space that it is in. This means that for every angle the measured distances are being stored. After this process the installation can recognize visitors in the space because they generate different distances then the already stored shape of the space. Spatial Sounds has different ways of reacting to visitors in the space. When nobody is being seen for a while the installation rotates slowly scanning for people that enter the space. After it detects people it starts to react actively on them. There are different forms of reacting. How the installation reacts depends on what people do. The installation reacts very direct to individual people through sound. When a person is being detected in front of the loudspeaker it reacts with sound. The installation reacts also with movement. It’s able to follow people, swing around people and move away from people. When people are very active the installation becomes over excited and starts to spin very fast with a loud complex sound. In this case the speed depends on the distance of the people to the installation. The installation builds up a relation with the people in the space. The visitors really become participants instead of audience. Spatial Sounds reacts very clear on a single participant. The behaviour becomes more complex with more participants however it doesn’t lose its directness. Many interactive installations have a clear separation between the actuator (sensors, joystick, etc.) and the generator (speakers, video projection, etc.). Often you operate an interface which creates a result outside of the interface. Since Spatial Sounds has the sensors build in the speaker there is no separation between the actuator and the generator. The participant communicates with the sensors in the speaker

48

Performance by Sensorband on SoundNet

Spatial Sounds (100dB at 100 km/h)

and the speaker communicates back with sound and movement. The different rules of interaction between the input and output define the behaviour of the installation. Since one rule by itself is still able to generate different qualities of output I would like to use the word interactive language. By interacting with the installation you start a dialogue with the interactive language of the installation. Waterpavilion The concept of the water pavilion (architecture: Freshwater building – NOX Lars Spuybroek, Saltwater building – Oosterhuis associates, music: Edwin van der Heide & Victor Wentink) is based on the idea of creating a communicating architecture. The architecture expresses itself through its construction, light, images, water and sound. The behaviour of the architecture is based on literal processes and metaphors about water. The fluid structure of the inside of the building is a shell for a continuously flowing and transforming world of water realized both with real water and virtual environments.

Waterpavilion Neeltje Jans music by Edwin van der Heide and Victor Wentick

All the sounds are completely electronically produced. The speakers are placed in such a way that it is the building which is sounding. This is the opposite approach from the traditional where the architecture provides the conditions for sounds to be heard individually. The building is alive and expresses itself via sound. There are 60 speakers distributed over the whole building. Each individual sound has its own character of movement and speed over the speakers. You can distinguish two different musical

49

environments. The freshwater space is based on the metaphor of a river. The sounds are streaming over the spine starting at the entrance of the building. The saltwater space presents a virtual sounding sky based on metaphors of weather and sea. The sound material inside the saltwater part is interactive with the weather circumstances outside the building. The music in the two different spaces is not a fixed composition but is composed on the moment it is sounding. The rules for how sounds can be combined are defined; the actual decision of what sounds, is made at that moment. In this way the music will always be. Partly the visitors can influence the processes via sensor based interfaces in the building. The behaviour of the composition is again related to behaviours found in water. The goal of a traditional concert is to give the audience a uniform experience of the performance. Water pavilion has the opposite approach. The idea is to promote individual experiences. Two persons visiting the building should have different experiences. Also when you visit the building the next time the experience should again be different. This is partly realized in the compositional rules and partly by using weather conditions influencing the compositional rules. Since April 2002 the above-described approach has been removed from the building. The new director of Waterland Neetje Jans was not able to understand the quality of having a communicating architecture. The building is still there and has become a shelter for a traditional exhibit about whales.

50

5

Levelsbyarchitects Maia Engeli Architecture & CAAD, ETH Zürich, Switzerland

Abstract: Ego-shooter games, like Quake and Unreal, are about moving fast through different 3D environments (called “Levels” or “Arenas”) and coping with a variety of challenges and enemies. There are a few editors for the design of levels available. We have used some of these to explore architectural themes in the context of Ego-shooter games. In an architectural context shooting is not the only imaginable action and has actually disappeared as a primary

Maia Engeli

focus when thinking about the player’s behavior , a fact that is heavily criticized by hard-core players. For architects and architectural students the design of game levels creates a number of interesting challenges and twists compared to the usual design process “Architecture is the art of moving through space.” Ted Naos To navigate in real-time through sophisticated 3D environments, to master a variety of challenges, and to cope with enemies with different strengths and abilities are the main characteristics of egoshooter games like Quake and Unreal Tournament. An interesting aspect of these games is that there are editors for the design of the 3D environments (so-called “Levels”) available. We have used some of these to explore architectural themes in the context of ego-shooter environments. 51

5

Levelsbyarchitects At the edge of CAAD CAAD (computer-aided architectural design) traditionally is concerned with modelling, simulating and communicating architectonic ideas. The field has undergone major changes in the past decades; more and more sophisticated CAAD tools have been developed, new technologies and media—like the WWW—have emerged, the cultural and technological context of architecture has changed, and the functions of and functionality within buildings has expanded to include also autonomous behaviour. Thus creating a demand for new tools and media for its expression and realisation.

The aim of the research at the chair for Architecture and CAAD at ETH Zurich has shifted in two ways: 1.

from looking at paradigms for modelling architecture to exploring means for designing, expressing, and communicating architecture.

2.

from the representation of physical “bricks” architecture to the design of digital “bits” architecture and mixed physical-digital “bits and bricks” architecture, buildings with thousands of sensors, processors and software integrated in their structure (Schmitt, 2001).

Designing levels for ego-shooter games allows the modelling and integration of behaviours in addition to geometry, colours, light, and textures. The levels are thzen explored in a highly immersive fashion by the player. This makes it possible to express action within buildings, to 52

Level 5

5

Levelsbyarchitects design and explore “bits and bricks” buildings, and to develop digital architecture better than with state-of-the-art CAAD software. Nevertheless, we do not regard ego-shooter level editors as the ultimate design instruments for these new aspects. We appreciate them as valuable

The Hobbit or There and Back again

means for a fast exploration of ideas and the formulation of desired characteristics for future instruments.

53

5

Levelsbyarchitects Subversions and conflicts Using ego-shooter environments to explore architectonic themes leads to a subversion of both the ego-shooter game as well as the architectural discourse. Qualities as well as conflicts can arise: -

Aesthetics: Ego-shooter game levels have a particular aesthetic; they often have a “medieval” look because of the rough stone textures and sparse, low-energy light source that are generally applied to the environments. The aesthetic possibilities are restricted to allow for fast graphics, but there is still a lot possible regarding form, textures and

illumination. “count” in the Level5 example goes beyond known ego-shooter aesthetics by introducing a countdown as an animated texture.

54

-

Shooting: In an architectural context shooting is not the only imaginable action in an ego-shooter environment and has actually disappeared as a primary design focus when thinking about possible experiences and challenges for the player – a fact that is heavily criticized by hard-core players. On the other hand shooting is part of such environments and may as well be integrated as a possible action in the design of a level.

-

Community: Ego-shooter games depend on a community; people that play games together and against each other. Matches can be held online or as a LAN-parties (local networks that get set up specifically for a match also called frag-fests,) among amateur as well as professional players. For designers the interesting aspects of the

5

Levelsbyarchitects gaming community are: 1. the people that produce free levels, skins (game figures) or shareware level editors, 2. online discussions on new features, technical questions and new hardware, and 3. the possibility to invite players into a level that one has created. -

Design approach: Our examples cover a number of approaches to level design each focussing on exploring a specific issue. In Level5 we explored visual and behavioural architectonic qualities. NextLevel focussed on the implementation of a hyperenvironment, with spaces linked beyond physical geographic possibilities. The “Hobbit” example is the replication of a story using the ego-shooter level as a new media. Zound Garden uses the ego-shooter editor as a tool to place sounds spatially. New projects

will focus on interaction design and further exploring the behavioural capabilities of egoshooter environments. Characteristics of the process and the result For architects and architectural students the design of game levels introduces a number of interesting new features and possibilities compared to the usual design process: -

The user / client is the player. The player wants entertainment and fun. The player needs to be able to understand what all of the possible actions are. Motivation becomes a key concern. Design is about creating an interactive experience and about enabling a dialog between player and environment. 55

5

Levelsbyarchitects

-

Levels get built – 1:1 – real – not virtual – they are more then images or models of a design – levels are real game environments for real players.

-

Levels can be modified. Modifications in response to comments from players become

part of the design and/or learning process. The quality of the final design increases through the enhancements that are introduced in response to the players’ feedback.

56

-

Levels are dynamic and reactive. They may look like stone, but they are made of bits. They can behave, trigger the player’s actions, and respond in various ways.

-

Level design is also interesting as a simulation of built or planned architecture. Rough

nextLevel / Through the Looking Glass

5

Levelsbyarchitects 3D models can be created and explored very fast, thanks to the optimised 3D rendering capabilities of the game engine. Real size avatars can inhabit the spaces and help to detect qualities as well as problems. Examples by Architecture & CAAD, ETH Zürich Level5 / nextLevel In a study week in 1999 we explored with a group of students the design and perception of architectural spaces in computer games including behavioural elements. The students worked in pairs and designed

levels to explore possibilities like moving parts, animated texture maps, invisible elevators, organic forms, and the remodelling of real architecture. Each group focussed on one of the above themes in combination with a focussed architectural aim. The resulting levels were made accessible through an overall entrance level. After a few challenges the player arrives at the gallery from where the different contributions can be explored. In this study week we used the shareware editor QuArK (Quake Army Knife) to create the levels for the Quake II ego-shooter environment. QuArK is easy to learn and offers a nice visual interface. It includes viewing of players, monsters and models in the level preview mode. nextLevel / Through the Looking Glass (Exhibition Contribution, 2000, Patrick Siba). nextLevel investigates the relationship between game, architecture and narratives. It is a reinterpretation of the novel ‘Through the Looking Glass’ by Carroll Lewis. Alice, the story’s main character, imagines different worlds for the different characters of the chess game. The design of the game level was not done as a direct translation of the novel, but “aiming at understanding the essence of the idea and then creating meaningful interpretations” (Zehnder, 2001.) The environment is hybrid and hyper-linked,

57

5

Levelsbyarchitects it contains contrasts on the visual as well the behavioural level. This makes the journey through Alice’s world a real adventure with unpredictable encounters, like: Gravity does not always function as expected; speed of movement is relative; or what seams to be an enemy may turn out to be one’s own mirror image. The group that has implemented this level decided to work with Unreal Tournament and the editor that is delivered with the game software. The reason for this choice was mainly because of the better control of the influences on the player that are possible in Unreal Tournament as compared to Quake. A

working strategy was chosen that allowed each of the team members to create 2-3 worlds and in the end combine them into one level with a chessboard as access plane. Other links between worlds were also introduced, to create the linked hyper-environment. At three occasions the level was shown at exhibitions. In the installations the computer screens were surrounded by wall size projections of the levels from the perspectives of different players. These views multiplied the prominent theme of the mirrors of the novel. The Hobbit or There and Back Again (Postgraduate project, 2000, Silke Lang) The world of The Hobbit (J.R.R. Tolkien) is a collection of locations, emotions, and riddles which—when traversed—leads the viewer on a journey of discovery, both of place and self.

58

nextLevel / Through the Looking Glass

5

Levelsbyarchitects “The purpose of this level is not the hyperkinetic frenzy of mayhem and destruction found in traditional Unreal Tournament Worlds. Just as Bilbo’s adventure is of both place and time, the world created here is about experiencing the space and solving the riddles. The overall goal of the final world is to recover a sense of this place, just as sympathizer Bilbo is trying to help recover the homeland for the Dwarfs.” (Silke Lang) This level was also implemented for Unreal Tournament. The project is much smaller than nextLevel and focuses on creating a linear path illustrating the places Bilbo is travelling through. At the end the player arrives in the space called “Memories”, filled with a line of boxes that allow looking back into the different spaces and revisiting them.

Zound Garden

Zound Garden (Postgraduate project, 2001, Richie Jindal, Johannes Nöldeke) ZoundGarden is a ‘New Kind of Place’ designed with sounds, where a person can challenge his or her sense of aural perception and imagination to create a virtual space in mind. It provides recreation, provokes associations and is pure joy, just like a traditional Zen garden. It is essentially a Cartesian space composed wholly of sound and can be explored and experienced only by navigating through it. (Jindal, Nöldeke 2001.) The students describe the design process as ‘carving space out of noise’. The result is not a game level, but an ‘aureality’ that has to experienced in a dark room. The editor and environment of Unreal Tournament were chosen because of the possibility to place sounds spatially and play them in a surround sound environment. The ease of the implementation allowed concentrating on the design process and exploring the combination of different sounds and possible behaviours of the ‘aureal’ environment.

59

6

Videogames, exploring an emotional architecture Xavier Boissarie Computer game designer at Carapace Sofware company in Paris

Abstract: A game is an emotional Journey, architecture in games tends to organize this Journey through two topics: playground architecture and rules. The core notion in videogames is “the gameplay”, this gameplay has something in common with the pace of the game, it’s an uninterrupted flow of emotions. Like in a movie, you take the player at the beginning of the game and you won’t release its attention before the end. Everything is precisely organized. The sensation of liberty can be great but paradoxically there is not much. Therefore the game designer is more concerned about what happens in the mind of the player than about what

happens in the game itself. Games are not artificial and autonomous worlds, but supports of emotions. This affirmation triggers a question: to which extend could videogames be a new space of creation for architects? Let’s explore first the differences between games and hard reality. In games the playground and the rules are well-known contrary to non-game domains. It means that all the energy of the gamer can focus on the game without being disturbed by the multiple non-predictable events that make life. Most of the videogames are built on the same emotional structure: The emotional structure of a gamer can be described as his relationship with his goal. This goal is linked with a reward: the pleasure. So, the first job for a game architect / designer is to make things clearly 60

Dragon’s Lair

6

Videogames,exploring an emotional architecture understandable: what is the goal of the game, how can I reach it, what is my relative position to the goal, my assets and weaknesses... The second goal is to keep the attention of the player all the game long. The player is demanded by his girlfriend, TV, sport partners, keeping his attention is an exploit. In this effort, gamedesigners are helped by a huge heritage as most videogames are based on classical game structures. Strategy-games rely on games like Chess or Go. Platform games are derived from obstacle race, sports games are directly derived from their live expression, RPG and Doom likes are played at school without computers or playstations. Rules and information architecture exist before the design of a game. So, games are a space of liberty for players and a space of constraints for designers. There are several

Xavier Boissarie

ways for an architect to enter the game adventure: -

Learning the efficient game architectures to build a new whatever-like with some improvements

-

Exploring the brain architecture to feed neglected areas with surprises and innovations.

-

Putting confidence in the rules and the players: maybe they can build the final architecture themselves.

Who never dreamed to get as much pleasure in life as in his favourite game? To feel free to progress toward a clear goal; to focus all his energy on his activity without being disturbed by anything? The game is a protected space in which actions and desires can grow. The player knows the playground, he accepts rules and goals. Hurdles and difficulties reinforce the final 61

6

Videogames,exploring an emotional architecture release of pleasure. The game offers a sentiment of freedom, which helps to release energies and desires. We should not speak about “the pleasure”, as they are in fact several types. Our body offers this reward in several circumstances: -

It is the direct pleasure activated through the use of a body function, you can increase it while intensifying the action till the physical limits. As you reach pain you can stop your effort and you feel relieved. Direct pleasure is not always accessible, or can appear too easy, a distance to pleasure can trigger more motivation: what is difficult to reach gives a greater reward. You can choose a reward as a goal and progress in its direction.

-

If you stay below the limits, you get the subtle pleasure of well-being until you get bored; you can then free yourself and taste the vertigo of wilderness.

-

You can grow in this empty space untill you feel too big, you can either loose weight or

organise your things so that they don’t take so much room. -

Relationships make you feel alive, close people transmit their feelings to you, you wish them to feel good and happy; until you feel tied, you can then cut relationships and relish difference and autonomy, you are then able to compare your performances with others through competition.

-

Your experiences make you identify working scenarios to reach pleasure, you play back those scenarios as close as possible to the first time.

This first exploration shows us several families of pleasures:

62

-

Direct pleasure linked with an activity: running, hearing music, dancing.

-

State pleasure linked with a situation: a balanced situation gives some well-being pleasure, a chaotic situation gives some vertigo pleasure.

-

Reflexive pleasure linked with a sentiment: I’m the best, I’m recognised, I’m strong, I’m

6

Videogames,exploring an emotional architecture pretty, I progress, I’m powerful, I’m free… -

Distant pleasures organise a more complex relationship. The time distance between the player and his goal can be structured in many ways.

Whether a pleasure is accessible or not will define the type of relationship the player will have with it. Accessible pleasures can be intensified with a risk of boredom; you can also explore them deeply, as their variations are often infinite. They have limits you can accept, push or break. The game organises the relationship between the player and pleasure through time architecture. let’s explore some time architectures used in games and see the game families

they correspond to. -

Accessible direct pleasure is linked with “deep time”, the game lies in “intensifying the sensation”.

-

Non accessible direct pleasures take place in a “furtive time”, their occurrence is not predictable: the game is: “create the event conditions”.

-

Accessible state leads to “abolished time”, the game is “keep this state”.

-

Differed Pleasures can be reached through a progression in an “incremental time”, the game is “reach that goal”; or at a precise moment of a process through a cycling time, the game is “play a story”.

-

A differed state is linked with binary time (order/chaos for instance), the game could be “solve a problem”. 63

6

Videogames,exploring an emotional architecture Those game structures can take place in any field: body activity, social activity, and brain activity… It means that several players with different goals could interact in a multi-level game. We can now summarise the different characteristics of a game: -

A game defines a specific kind of reward.

-

The playground is a protected space in which the desire toward this reward can develop without perturbation.

-

This desire gives the player his energy for playing the game.

-

The game organises the relationship between the player and his reward through time

architecture. -

Time architecture maintains the player in a specific consciousness level. This continuity of Consciousness State distinguishes game from reality.

Among those game patterns, one is over-represented in videogames: “the progression games”; let’s study this pattern. Most of the videogames are built on this pattern. It can be simply described as a relationship between the player and his goal. This goal is linked with a reward: the pleasure. The progression to the goal must be measurable more power, more territory, and more money. The progression becomes itself a source of pleasure and regression a source of pain. The final pleasure equals the sum of energy spent to reach it.

64

6

Videogames,exploring an emotional architecture Some important topics in stock-game designing to keep the players attention

Mario Brothers

-

Make the goal attractive

-

Set the Play-Ground, the rules and the means available for the player

-

Create difficulties to reach the goal

-

Prove it is possible to reach it

-

Make progression info reachable at any time

-

Tune things so that the relationship between the player and its goal changes constantly

65

6

Videogames,exploring an emotional architecture

All those topics must achieve to generate emotions constantly in the player in order to stimulate his energy and keep his full attention. All the game long, the player stays in the same consciousness level: “incremental time”.

“Progression games” are close to the most important values that rule occidental societies. Progression sentiment is a key notion for a majority of people. Those games are good training areas for winners. The risk is to become Mono-oriented, it means that if your games are close to your life activities, they reinforce your emotional structure and you won’t evolve much. Second consequence: players who seek the same goal enter a competition. This situation generates a pyramidal structure based on performance. After a while, the basis 66

6

Videogames,exploring an emotional architecture of the pyramid abandons the game to admire the top ones. Losers don’t play anymore but attend a show. The notions of progression and reward are omnipresent in videogames and build a real paradigm. Most publishers won’t accept projects that are not formatted on those principles. I do think that there are in fact strong alternatives. Let’s explore some of them: Contractual relationships vs addiction Most of the games don’t exist without players. They don’t have autonomy. The game needs the player to exist, it develops (through the designers) much seduction to keep the player

in its net as long as possible. Some profitable relationships between player and game exist. They are on a contractual kind more than a seduction kind. The game can for example reject the player if it is not good for him. To reach that, the designer can give some autonomy to his game. Open Rules vs Given Rules A few games offer some open rules to the players. In fact, liberty may trigger a vacuum feeling. Some Role-Playing Games let you define your own goals. New kinds of roles emerge from the game without connection with the imagination of the designers: the “Player killers” who destroy everything, the “Dealers” who only buy and sell goods and services. Those new roles enrich the player community and give much interest to the game. Multiple Goals cohabit in Role-Playing Games. Several games interact on the same playground like in reality. This combination generates very rich situations. 67

6

Videogames,exploring an emotional architecture Dynamic Architecture vs Static Architecture The architectural approach in “progression like games” is static architecture. It means that at any time, the game designer controls the situation. The designer decides all possible situations. Present game structures don’t offer much liberty for architects and designers, as the constraints are numerous. But the designer can also forget the ambition of total control and open an undetermined space. Thus, he offers more liberty to the player as this architecture relies on flux and constant changes. Dynamic architecture is a very good way to personalise experiences and increase the intensity of relationship between the player and the event. You enter the world of living things, in contrast to the world of repetition. The player can give an orientation to the flux making the generated world a game. This kind of architecture is very close to a game as it deals with constraints and rules. The player can complete this

fuzzy world with his own thoughts, desires and dreams. He can also navigate on a frontier between chaos and order. The Ball The Ball is a project inspired by the following considerations. Two years ago I worked on a construction game called “Machines Poetiques”, built with Virtools; the player constructs a sculpture by connecting animated elements to a central heart piece. The heart transmits pulsation to the other pieces. When we put sounds on behaviour, we noticed that the sculpture generates a kind of music. Till that time, I wanted to create a game that gives as much surprises to the designers as to the players. I’m a basic tango dancer and noticed that when you have even a basis in a dance technique like tango, you can reach very intensive moments while dancing. It happens now, you feel floating and, there is no reflexivity. The body is the boss. With “the Ball”, I wanted to share this 68

6

Videogames,exploring an emotional architecture feeling with my colleagues and with the players through a game process. I gathered a little team with Veronique Caraux for her poetic design and Jean Jacques Birgé and Bernard Vitet, who are very skilled in generative music through a rich free jazz experience. “The Ball” takes place in an outdoor Ballroom. The player is attracted by noises. Objects are growing in peripheral areas. The player controls five fairy flies, they pick up an object and bring it on the scene. The object moves, his movement generate sounds. The Flies are taking control; they orientate the movements and the sounds. The dancer behaviour becomes more

The Ball by Xavier Boissoire

complex and the sound constructs into music. The player orientates music. The dancer can receive a hat to make a duo. Then he will be able to seduce another dancer and make a trio or a quatuor. All the purpose is to bring the player to intense moments. -

In “The Ball”, the player faces an empty space comparing to most videogames: there is no Goal, and little information. Things never behave twice the same, events die as they happen, The game doesn’t offer any progression information, it doesn’t need any specific skill and doesn’t give any reward to the player. He can’t record his performance; memory is in the hand and in the head. We try to disconnect the rational part of the brain and to open the way to sensitivity, intuition, contemplation, increased emotions and liberty sensation. 69

6

Videogames,exploring an emotional architecture

-

The player of “The Ball” enters then in a furtive time; music constructs itself as you hear it and vanishes, takes new forms. The player orientates this flux, his senses are widely opened upon what happens. His mind gives attention to surprises and nuances.

-

“The Ball” offers infinite variations through generative structures. The player can have a dance and live new experiences at each ball session.

-

Good surprises are not predictable, unlike in the linear recorded music for instance

To achieve that, we try to fulfil several demands: -

Build an intuitive interface as the adaptation period must be very short.

-

Open the possible range of actions with the gathering of experience.

-

Build a complex world on very simple rules to reach a good coherence and offer much surprise.

“The Ball” uses several generative structure types: L-systems for the terrain and plants generation. 70

Combinatory structures: you can create 100 different type of dancers while combining

Machines Poetiques by Xavier Boissoire

6

Videogames,exploring an emotional architecture legs and hats and 10 000 couples. Music is created with the combination of structures: melody, rhythm sequence, tonality, and tonal scale.

The Ring by Xavier Boissoire

-

Variations around structures: Each of those structures evolves upon appropriate rules.

-

Interactions: the dancer communicates and shares information. Their interactions build several kind of relationships: solo/accompaniment, imitation, alternative soli, duo…

-

Emotional layer: dancers have a simple emotional life that constructs an emotional representation of the world. They react to events and express their feelings: happiness, sadness…

The result is rather surprising for the designers themselves. This rapid exploration over the gaming world shows us several things: human beings can get some pleasure in an incredible range of situations and activities. “Progression games” are widely introduced in life to liberate energies. Nowadays, those transpositions take place in the world of competition under the myth of continuous progress. This paradigm undergoes some strange mutations: Players don’t necessary have to share the same goal; The renunciation to progression sentiment leads to very rich game experiences ; A game can take autonomy from its designers; a game can free its players instead of making them addicted; people playing different games can interact in the same space. I hope that this new game context will be able to give the opportunity for players to live a wider range of game experiences, access several types of level of consciousness and enrich their lives. The magic 71

6

Videogames,exploring an emotional architecture of a game architecture is that its time dimension is finite, it has a begin and an end. You can escape it easily and enter a new one. Sharing a game with other players helps to understand their feelings and desires, to communicate with them. It opens new dimensions.

Xavier Boissoire

72

Screenshots from Counter Strike

Intermezzo 3 - Multiplayer Collaborative Design Internet Game Ir. Hans Hubers Building Engineer at Faculty of Architecture - TUD Abstract: As an intermezzo Hans with his two sons show the state of the art of multiplayer internet games. This kind of technology can be used for simulations in architecture; not only for evaluation of the design by the future users, but also as a means of communication for the experts in a collaborative design team. Some problems and solutions of collaborative design are discussed. On two projectors, my two sons, Jonathan (17) and Joris (15) show the state of the art of multiplayer games on the internet. They are no whizzkids. Their friends do the same. This generation will expect from our universities that education in architecture will be as interactive as this game. The game is called Counter Strike. It uses the game engine of Half Life. It is a so called shoot-um-up game. Jonathan and Joris just logged in on the game server and joined the counter terrorist forces. With some key strokes they can switch cameras. The free look mode camera (Fig. 1) shows them both. Jonathan is standing in the back. With this camera you can move freely through the game when you are dead and learn from the others. It is also great to discover the architecture. They first choose their weapons, depending on the money they earned. Killing a terrorist or diffusing a bomb make you earn money. Your budget is on the bottom right of the screen. Other information on the bottom shows how much life you still have, how bad your protection is damaged, how much time left and how many players. You can communicate with team members through standard messages like “Follow me” or “Cover me” by just typing a key combination. But you can also type in free text, which is shown over the camera view at the left. Recently it became possible to even talk to each other using a head set, as Jonathan and Joris show. The green circle at the top left is a radar showing where your team mates are. Of course it is not for fun that we show this game. At least not only. What strikes me is that we see here real-

73

time interactivity over the web with 16 characters and amazing quality geometry, texturing, special effects, speech and behavioural information. Building professionals will soon be able to do the same in the game of designing architecture. No matter where they are. We call this collaborative design. Collaborative design Since architecture and building in general becomes more and more complex, many different specialists will work together. In the time of the cathedrals an architect had complete control over the design and the execution. Now, with complex buildings like high rise multi purpose buildings, specialists are needed for construction, installations, cost calculation etc. With interactive architecture it becomes even more complex. All these specialists must advise the architect in an early stage of the design process in order to optimise the concept. For this they need to exchange data to use in their computer programs. In the education of architects we must practice this collaborative design. There are some difficulties though. First: without a concept, what are we talking about. Early try outs show that meetings take for ever and ever. (Deiman, 1994) Discussions go in all directions and handle many different subjects, until finally the person with the most authority forces the meeting to his opinion. Not always the best. A solution might be found in a protocol of cooperation in which procedures per phase of the design process. E.g. what demands have to be fulfilled and what criteria must at least be used to evaluate proposals. Second: without a basic design, a specialist in construction or installation can not evaluate different solutions. If it is not known that the building will be small and high or large and low or if it will have a vaulted roof or columns or structural walls or a light or heavy facade or maybe dynamic loads, what sensible things a construction advisor can say? Well, of course he can say sensible things, but he can not calculate different solutions. A solution for this can be to work with a reference building (VCA. 1994). This building is chosen from a database filled with buildings of different type. And then roughly adapted to the needed square meters. So you don’t start with only a program of demands, but right a way with a building of which everything is known.

74

Then it becomes a re-design project. But really innovative buildings will never be developed this way. Third: the software -every specialist uses a different program-, is not made for collaborative design. Exchange of data is often very poor, if it works at all. Some 10 years ago in a national research program we used the STandard for the Exchange of Product model data (ISO/STEP) to define the data in a pilot project. It was about data exchange between an architect, a cost expert and an installation expert concerning a small standard office building. Prototypes in AutoCAD and Excel were developed using a central database in STEP format (VCA. 1994). The prototype worked fine, but the software developers didn’t want to make the necessary investments. A strategic quarrel started between the financing unions of the different building professionals about the central database with all the information of a building -the party that controls this database gets much power in the building process-. The co-operation stopped. In the meanwhile we learned a lot about data exchange. (O’Connor, 1998) (Rosenman,1997), (STEP,1991). We are convinced now that every involved profession must formulate the objects and attributes connected with those objects in a logical scheme, in fact a decomposition of building parts and items the way that profession uses it. Now one profession may give a name like “wall” to a vertical space delimiter, while an other profession may call it “separation”. One profession may only be interested in those elements if it is bearing loads and an other only if heat transmission goes through it. There for not only is it necessary to make a list of objects and attributes but also some agreement must be made between the professions about how they are related. This calls for classifications that must be possible to map on each other. Existing classifications can play an import role of course, but soon it becomes clear that there are some inconsistencies and that changes are needed and complementary elements. The organisations that maintain those classifications won’t do this right a way and need to be convinced. Which takes time and a lot of explaining about informatics and discussions. So after a while the professionals start to work without those agreements and have to make agreements for every project. This of course is not workable, because every time it is a little bit different. For instance in project X, an office building, the installation advisor needs the surface of the façade to calculate the heat loss. He not only wants the total surface, but specified for different materials (glass, bricks, wood etc.) and also the length of the connections and the quality and the isolation etc. This information does not just can be found in the output of the CAD-system of the architect. If the architect is using CAD-software that permits it, he or

75

she first has to put this information in. Who is going to pay the architect for this work? That is a big problem. But let’s imagine an architect that is willing to invest in this, e.g. because in this way he build a relation with partners that can generate work. Then the next problem is to store all the information in a database in which every partner can read and write his information. But since there is no agreement about the classifications the database must handle synonyms. Of course there is no software that can take care of that. So some whizzkid staff member will program a macro, which only he understands fully and maybe (not likely) the project will look like a bit collaborative design. Now project Y, a multipurpose building with some large meeting rooms and a theatre, must be set up. Maybe different partners? But even if the installation advisor is the same, now the heat loss of the walls are not relevant, because the building is underground or the heat loss is negligible compared to the isolation problem of air-conditioned large rooms, but acoustics are important and the advisor needs to know the kind and quantity of the materials inside the theatre. Again some whizzkid has to go to work. I think my point is clear. The lay-out of the central database changes every time and every team is doing the same kind of poor database programming. Still I think that both approaches are needed. Top-down by representing unions of the different professions and researchers and bottom-up by the professionals that know how things are done in practice. But since there is no money to make them cooperate there is still a long way to go. So let’s forget about collaborative design? No, we believe that in due time these problems will be solved. In our projects we experiment with multiplayer game development software and ways to work on the same project simultaneously. If we can do it in virtual reality it can be done in reality too.

76

7

Beyond Kinetic Michael A. Fox Kinetic Design Group, Department of Architecture, Design Technology, MIT

Abstract: This paper includes an overview of intelligent kinetic systems in architecture including precedent in embedded computation and kinetic architecture. The text underscores the work of the MIT Kinetic Design Group and the architectural practice roart / kdg founded by Michael Fox in 2001. The work explores intelligently responsive kinetic and mobile architectural systems; with the motivation in creating spaces and objects that can physically re-configure themselves to meet changing needs. Projects demonstrate both how responsive transformable kinetic objects occupy predefined physical space as well as how moving physical objects can share a common physical space to create adaptable spatial configurations. Also of concern are

Michael Fox

projects demonstrating automated kinetic response with respect to changing environmental conditions. The work explores a diverse range of controlled kinetic motion in architecture from simple biometric response to high-level self-learning control. Of particular interest is how responsive kinetic systems can integrate a heuristic or learning capacity into the control mechanism. Such systems can learn through successful experiential adaptation to optimize a kinetic system or spatial form of an environment in response to change. What is described is a structure as a mechanistic machine that is controlled by a separate non-mechanistic machine: the computer. In a sense, creating a building like a body with a system of bones and muscles and tendons and augmenting that body with a brain that knows how to respond.

77

7

Beyond Kinetic “If architects designed a building like a body, it would have a system of bones and muscles and tendons and a brain that knows how to respond. If a building could change its posture, tighten its muscles and brace itself against the wind, its structural mass could literally be cut in half.” - Guy Nordenson, Ove Arup and Partners “If a building could mediate our needs and the environment outside: Its demand on physical resources could be slashed. If it could transform to facilitate multi-uses: Its function would be optimized. If a building could adapt to our desires: It would shape our experience.” - Michael A. Fox

Introduction Our capabilities of utilizing kinetics in architecture today can be extended far beyond what has previously been possible. This paper looks at the potential of advanced kinetic architectural systems; what they are, what they can do for us, and how we can go about designing them. Advancement will only be accomplished when kinetic structures are addressed not primarily or singularly, but as an integral component of a larger system that takes advantage of today’s constantly unfolding and far-reaching technology. Necessary are the use of advanced computational design tools, material development and embedded computation. It is important to point out that this article shall remain safely grounded in science-fact and not science fiction. In other words, to make convincing extrapolations based on where we stand today through inclusively appreciating and marshalling correctly the existing facts with 78

detail Interactive Kinetic Facade

7

Beyond Kinetic respect to technological development. The irony is that from an architectural standpoint we are in a relative infancy even with respect to our extrapolations, further exacerbating the matter is the foolishness to name what we are experiencing in terms of general technological advance as a revolution; it is an evolution, to which an end cannot be predicted outside the parameters of political and economical entanglement.

Prior to explicitly defining why advanced kinetic architectural systems will be useful or even necessary, we will state simply that the motivation lies in creating spaces and objects that can physically re-configure themselves to meet changing needs with emphasis on the dynamics of architectural space. Such systems arise from the isomorphic convergence of three key elements: structural engineering, embedded computation and adaptable architecture as situated within the contextual framework of architecture.

79

7

Beyond Kinetic Kinetic Architecture: a definition Concerns in structural engineering will focus explicitly upon kinetic design. Kinetic architecture is defined generally as buildings and/or building components with variable mobility, location and/or geometry. The general implications of utilizing such systems in architecture include, but are not limited to: space efficiency, shelter, security, transportation, safety, economics, and aesthetics. The specific applications are enormously varied and include novel applications that arise out of changing patterns of human interaction with the built environment. Structural solutions in kinetic architecture must consider in parallel both the ways and means for operability. The ways in which a kinetic structural solution performs may include among others, folding, sliding, expanding, and transforming in both size and shape. The means by which a kinetic structural solution performs may be, among others, pneumatic, chemical, magnetic, natural or mechanical.

Kinetic structures in architecture are classified here into three general categorical areas: -

Embedded kinetic structures are systems that exist within a larger architectural whole in a fixed location. The primary function is to control the larger architectural system or building, in response to changing factors.

-

Deployable kinetic structures typically exist in a temporary location and are easily transportable. Such systems possess the inherent capability to be constructed and deconstructed in reverse.

-

Dynamic Kinetic Structures also exist within a larger architectural whole but act independently with respect to control of the larger context. Such can be subcategorized as Mobile, Transformable and Incremental kinetic systems.

The ways can be described diagrammatically as mechanical motions. Contemporary innovators such as Chuck Hoberman and Santiago Calatrava continue to demonstrate that the last word has not been spoken in novel kinetic implementation at an architectural scale. 80

7

Beyond Kinetic Yet, we as designers ought to focus our attention in this area upon the vast wealth of resources that have been accumulated over numerous centuries of engineering. There are many great scientists of a thousand years ago who would have had no difficulty understanding an automobile or an engine or a helicopter and certainly not the most advanced architectural system. The craftsmanship would have been astonishing but the principles straightforward with respect to an understanding of the novel material properties. Materiality will prove to be the one great promise for advancement in this area primarily as a result of technology providing both an unprecedented vision into microscopic natural mechanisms and advanced

general kinetic Typologies

manufacturing of high quality kinetic parts with new materials such as ceramics, polymers and gels, fabrics, metal compounds and composites with unprecedented structural properties. The integrative use of such materials in kinetic structures facilitates creative solutions in membrane, tensegrity, thermal, and acoustic systems. Embedded computation If we were to show the same great scientist of the past a television or a computer or a radar, it would have appeared magical to them. The difficulty for them would not have been one of complexity; but rather they would have been lacking in the mental framework required to conceptualize such non-mechanistic devices. Today it does not take much effort to extrapolate existing computation as a means for kinetic actuation. We are rapidly 81

7

Beyond Kinetic approaching a time where the integration of embedded computation and kinetic function becomes a practical and feasible reality. When we look at the major trends in computation we see a clear transition of three general human-computer relationships. Initially there was the use of one computer (mainframe) by many people. This evolved to a one on one relationship with computers. As this article is written, we are witnessing a trend towards one person using many computers. This relationship was coined “Ubiquitous computing” by Mark Weisner of Xerox PARC in

1988. Ubiquitous computing forces the computer to live out in the world with people but at the same time, will recede into the background of our lives. The concept of Ubiquitous computing integrates human factors, computer science, engineering, and social sciences. We are at a time when Metcalfe’s Law (which states that the network benefit increases as a square of the # of connected devices) is starting to replace Moore’s Law (which states that CPU performance doubles every 18 months) as the driving force behind the development of computational devices. What then, are the implications of ubiquitous computing to built form? From an architectural standpoint, embedded computation has taken an interesting foothold. Work in embedded 82

intersection of embedded computation and kinetic architecture, diagram by MIT KDG

7

Beyond Kinetic computation has arisen primarily out of the field of computer science, reaching into the sub disciplines of both artificial intelligence and robotics. The research has come out of both academia and the corporate world and there are currently numerous precedent examples of embedded computation in that have begun to define a field now known as intelligent environments (IE). The projects cover a diverse range of scales; from the Biosphere2

the major trends in computng, graph by XeroxPARK

project that is arguably the most sophisticated building constructed by man, to the Adaptive ro “Neural Network” House in Colorado. Ironically the most sophisticated intelligent environments built to date have been constructed for space travel where the environmental conditions are extreme yet relatively constant and yet a residential house in Phoenix, Arizona typically could not be identified as different from one in Anchorage, Alaska. The primary goal of intelligent kinetic systems should be to act as a moderator responding to change between human needs and environmental conditions. The primary target clientele for research in 83

7

Beyond Kinetic Intelligent Environments has been the military, the elderly and the handicapped, typically in that order. Not surprisingly, the vast majority of the work has been both highly tectonic in

dealing with recognition tasks such as Speech, Gesture, Motion, and the Environment while focused on managing human interactions and novel applications of Internet Technology. The motivation for such research lies in embedding computers in ordinary environments so that people can interact with them the way they do with other people; in other words it is aimed at enhancing everyday activities. The major problematic of what has been accomplished 84

Michael Fox

7

Beyond Kinetic outside the field of architecture is the myopic nature of enhancing everyday activities. When embedded computation is employed to control physical built form, the most obvious application should be to foster an extension of manual capabilities. With due respect to the advantages such systems can provide to the elderly and handicapped they are at best equalizing the current advantages of built form, and not extending them. Architects need to design with an understanding of the current capabilities of embedded computation that have attained sufficient maturity to act as independent subsystems that can be beneficially incorporated into kinetic design. A consequent result of this motivation has been to create a seamless integration of computation into the built environment. It transcends a question

moderating skylights

of aesthetics to ask if embedded computation should be hidden from the users that inhabit an intelligent space or if there is due an honest expression of form where computation is embodied. Perhaps the most applicable research to draw upon in designing intelligent kinetic systems lies in an area of study within Active Control Research that focuses upon the design of structures to control the movements of a building through a system of tendons or moving masses tied to a feedback loop to sensors in the building. Changes are brought about by both environmental and human factors and may include axial, torsion, flexural, instability and vibration and sound. Such systems have been successfully employed in numerous large buildings situated in high-wind or earthquake-prone locations.

85

7

Beyond Kinetic Controlling kinetic function by computational means Precedent embedded computation will serve as a foundation for the explicit means of controlling kinetic motion. Such means can be described diagrammatically as the controlled source of actuation. Specifically we are interested in addressing embedded computation as a control mechanism for kinetic function to accommodate and respond to changing needs. Such systems will be utilized to interpret functional circumstances and direct physical movements to adaptively better suit changing human needs. The issue of controlling kinetic motion is central to issues of design and construction techniques, kinetic operability and maintenance, as well as issues of human and environmental interaction.

Novel applications for kinetic adaptability While there may be many reasons for employing kinetic solutions in architecture we can always rest assured that they are a means to facilitate adaptability. Adaptability is taken in the broadest sense to include issues such as spatial efficiency, shelter, security and transportability. Such systems that are inherently deployable, connectable and producible are ideally suited to accommodate and respond to changing needs. An adaptable space flexibly responds to the requirements of any human activity from habitation, leisure, education, medicine, commerce and industry. Novel applications arise through addressing how transformable objects can dynamically occupy predefined physical space as well as how moving physical objects can share a common physical space to create adaptable spatial configurations. Applications may range from multi-use interior re-organization to complete structure transformability to response to unexpected site and program issues. Specific applications may include intelligent shading and acoustical devices, automobileparking solutions, auditoriums, police box stations, teleconference stations, devices for 86

servo controlled choreographed elevator doors

7

Beyond Kinetic ticketing and advertising, schools and pavilions, as well as flexible spaces such as sporting, convention and banquet facilities. Other spaces of consideration are those with necessary fixed exterior configurations such as airplanes, boats, transport vehicles and automobiles. Through the application of intelligent kinetic systems, we can also explore how objects in the built environment might physically exist only when necessary and disappear or transform when they are not functionally necessary. Kinetic adaptability further considers the rapidly changing patterns of human interaction with the built environment. New architectural types are emerging and evolving within today’s technologically developing society. These new programs present practical architectural situations for unique and wholly unexplored applications that address today’s dynamic, flexible and constantly changing activities.

interactive robotic flowers in the Secret Garden

Future human interaction with the built environment is extremely difficult to predict even as science-fact extrapolations because it is ensnared with contradictions. In the example set forth by Arthur Clarke: A really perfect system of communication would have an extremely inhibiting effect on transportation. Less obvious is the fact that if travel became nearly instantaneous, would anyone bother to communicate? Our cities are the result of our mastery over neither. A topic of great interest today is the effect of our current mastery of communication on urban built form. What would be the effect if our mastery over travel had preceded that of communications? More relevant to applications of intelligent kinetic systems is the still science fiction issue of planetary engineering or climate control. If climate control were localized by architectural means at an urban scale would there be any desire to investigate planetary engineering given the potentially adverse effects on terrestrial equilibrium?

87

7

Beyond Kinetic Intelligent kinetic systems as living mechanisms What we are describing then with advanced kinetic systems in architecture is a structure as a mechanistic machine that is controlled by a separate non-mechanistic machine: the computer. An interesting phenomenon can be observed when we look at the higher levels of control. The engineer Guy Nordenson describes the phenomenon in embedded kinetic systems as creating a building like a body: A system of bones and muscles and tendons and a brain that knows how to respond. In a building such as a skyscraper where the majority of the structural material is there to control the building during windstorms, a great deal of the structure would be rendered unnecessary under an intelligent static kinetic system. If the building could change its posture, tighten its muscles and brace itself against the wind,

its structural mass could literally be cut in half. In deployable and dynamic kinetic systems as well, much of the structure will be reduced through the ability of a singular system to facilitate multi-uses via transformative adaptability. Buckminster Fuller who coined it “Ephemeralization” first illustrated this concept of material reduction. Robert Kronnenberg aptly illustrates the advantage of such systems in that buildings that use fewer resources and that adapt efficiently to complex site and programmatic requirements are particularly relevant to an industry increasingly aware of its environmental responsibilities. Conclusion It is difficult to see if advanced kinetic architectural systems are far on the horizon or inevitably in the very near future. To extrapolate the existing into a future vision for architecture is a conundrum residing in the hands of architects directing the future of their profession. Adaptive response to change must intelligently moderate human activity and the environment and 88

diagram of intelligent kinetic systems

7

Beyond Kinetic build upon the task of enhancing everyday activities by creating architecture that extends our capabilities. Such systems introduce a new approach to architectural design where objects are conventionally static, use is often singular, and responsive adaptability is typically unexplored. Designing such systems is not inventing, but appreciating and marshalling the technology that exists and extrapolating it to suit an architectural vision. Architects will inevitably hear that “it cannot be done”, and to this should recall that commercialized electric light was not long ago thought impossible, that it was thought a man would suffocate on a locomotive if he were to travel at a speed exceeding 30 miles an hour and of course the impossibility of heavier than air flight. Architects need to grasp a vision that will harness technology transfer from “outside” fields and prevent contradictions in human interaction

with the built environment. To a great extent the success of creating intelligent kinetic systems in architecture will be predicated upon the real-world test-bed. Applications must consider the capability for such systems to yield real-world benefits. Actual construction and operation will allow architects to develop realistic consideration of human and environmental conditions, and to overcome simplified assumptions about the costs of manufacture and operations. The result will be architecture of unique and wholly unexplored applications that address the dynamic, flexible and constantly changing activities of today and tomorrow. Projects -

Moderating Skylights (project of MIT KDG) The Moderating Skylights is a networked system of individually responsive skylights that function together to optimize thermal and day lighting conditions. The unique aspect of the Responsive Skylights lies in kinetic function, human interaction and adaptive control 89

7

Beyond Kinetic under realistic operating conditions. Adaptive control being that the system would be capable of learning the usage patterns on a daily basis. Primary design considerations are to utilize natural daylight in the space where and when it is desired and to optimally take advantage of natural ventilation. -

Boeing Business Jet Interior (project of MIT KDG) The motivation for this project lies in creating interior design solutions that are flexible and adaptive, and at instances, responsive and intelligently active with respect to the

changing individual, social and climatic contexts. Accordingly, the goal was to provide a responsive interior space that can be configured as prescribed by the users prior to a specific flight as well as partially reconfigured in-flight. Such adaptability aims to meet the changing needs of the users and their activities/environment for comfort and optimum spatial efficiency. The design introduces to the interior three basic kinetic components, namely sectors, which display variable location (mobility) and variable geometry (transformability). The sectors can technically operate independently; as a complete system, they divide and define zones of the program in the interior. Each is equipped with/provides the technical and the physical/spatial apparatus necessary for various parts of the program.

90

moderating skylights

7

Beyond Kinetic -

Boeing Business Jet interior

Shutter Speed (project of MIT KDG | ROART) This Eight automated doors, four on each floor, operate in a choreographed sequence

revealing full-scale car racers from the fifties silk-screened on a glass wall. A 2 ½” thick laminated glass floor and a stainless steel mesh wall combine to create this composed mechanism on an automobile lift in the heart of New York City.

91

7

Beyond Kinetic -

Interactive Kinetic Façade (project of MIT KDG | ROART) A band of bars, ten feet above the sidewalk translates the activity of the street into a

moving physical image on the façade of the Roger Smith Hotel in New York City. The band of bars that wrap the building is constructed from a hundred and sixty 2’ by 2’ panels each composed of fifty horizontal kinetic bars. Sensors mounted beneath each row that monitor the presence of a moving person drive the wave-like rhythm. Once motion is detected, a bar will gradually point towards the moving object creating a ripple 92

full-scale working prototype of the interactive kinetic façade

7

Beyond Kinetic through the field. As the singular movement of each individual bar becomes part of a complex pattern, the activity of the street is rendered on the building as a whole. -

The Secret Garden (project of MIT KDG | ROART) The Secret Garden is a reading and contemplative place for the children of a New York Public School. It is a place to read in an environment filled with nature, science and art. It is a garden inspired by the imagination of children and the dreams of their parents and friends. Amongst the giant wood sculpted flower pots which serve as random seating

areas are numerous interactive robotic flowers which gently move to track the sun and turn to bend and follow the motions of the children playing in the garden.

MOBI: mobile wireless computer classroom

-

Mobi: Mobile Wireless Computer Classroom (project of MIT KDG | ROART) The project couples the technology of wireless computer carts with a transformable seating and working area to create a literal classroom without walls that can be transported easily to any location. The new Mobile Wireless Internet Classroom is small and light enough to be transported by one person and will integrate seating for twelve children and one instructor. It is engineered to store and charge up to 13 wireless mininotebook computers and one instructor flat-panel monitor. The seating transforms to various classroom configurations for centralized, linear, or roundtable instruction. 93

8

Eliminate the Interface1 Ted Krueger Associate Professor, Director Laboratory for Human Environment Interaction, Rensselaer Polytechnic Institute

Abstract: In order to craft the means by which humans and environments can interact, a proper characterization of both must be developed. Traditional understandings of each limit the potential of their interaction. This paper will attempt on the basis of contemporary research to set out a proper relationship. Preliminary design research that is based on these assumptions is also discussed. Technological developments especially in material science and information technologies combine and reinforce each other to profoundly change the properties of the built environment. Embedded control and communication devices, low-cost sensor technologies, and smart materials may be employed to build objects and

environments that are adaptive and responsive to both conditions in which they are employed and to their users. As the built environment acquires these biological characteristics, our relationship to it changes. Products of design are no longer fixed objects and relations but interactive and complex systems. At the same time, developments in cognitive science, perceptual psychology and philosophy have characterized the individual not as a fixed and insular being, but one deeply enmeshed in, influenced and assisted by and dependent upon the environments in which they exist. In addition, particularly in micro gravitational environments, the rapid and profound physiological and perceptual adaptivity of the individual has become clear. The image that emerges is one of an organism that is not the cohesive and integrated entity that we experience but is continually in flux and negotiation. It is by means of this flux that the constancy is achieved. In 94

1

An earlier version of this paper was published in Blank, D. ed. 2000. Proceedings of the Eleventh Midwest Artificial Intelligence and Cognitive Science Conference. Menlo Park, CA: AAAI Press

Ted Krueger

8

Eliminate the Interface many instances, the relations that must be developed between two dynamic systems cannot be obtained by means of determining the formal aspects of inert object as has been the traditionally done in the design disciplines. The game has changed. Our theories of computation have evolved, interacted with and informed our understanding of cognition. There is a strong tendency to see either in terms of the other. This reciprocal relationship has had a profound impact on the kinds of interfaces that have been developed to mediate between humans and computers. Transformations in this relationship will have

corresponding implications for the kinds of interfaces that need to be developed and the way in which they are implemented. The shift from computational to dynamical models of cognition has tended to emphasize both the embodiment and situatedness of the agent and to bring the environment into an active role in the cognitive process. Simultaneously, the widespread distribution of embedded processors has allowed a measure of responsiveness and interactivity to be generated from within the environment itself. This condition requires notions of interface to be reconsidered. The development of digital computation had symbolic logic as its foundation. Logic is an attempt to codify the proper structure of thinking independent of its content or context. The rule-based manipulation of symbols has enormous power and flexibility. Instantiations of 95

8

Eliminate the Interface this process have shown to be equivalent whether implemented by mechanical, electrical, hydraulic, or other means. It was widely supposed that biological, and in particular, human cognition is based on computational principles as well. The independence of process from substrate suggested that software could be considered apart from hardware and so also the mind apart from the body. Hardware independence lent plausibility to the project of extracting and transferring human cognitive capabilities to machines. Wearn (1998) notes it is common to consider cognition only within the context of the mental processes of an individual using as a model the human as an information-processing machine. Interfaces

within this perspective are surfaces of contact between two kinds of information processors -surfaces upon which the symbolic tokens reside and are manipulated. Text, graphic and speech interfaces attempt to capitalize on the richness of the modes of communication that have developed in the human social sphere. They suggest a humanness of the intelligent machine that is consistent with the traditional goal of A.I. - the duplication of human intelligence. They also accept an abstraction of humanness that is consistent with the cognitivist approach. The computational model of human cognition, especially in that it relies on notions of representation and logical operations, has been effectively challenged. We have come to understand ourselves as much more than this. But if the computational model is no longer convincing, then alternative interface strategies will need to be developed. There has been a consistent trend toward the diffusion of computing resources into the 96

8

Eliminate the Interface environment. Embedded computation is more ubiquitous and transparent, but also less theorized than interfaces to information processing machines. The interface required for interaction with these environments is not likely to be a screen, and may have no reason to make use of symbolic tokens or other abstract devices. Interaction in this new condition must be founded on an understanding of activities that are grounded in the world rather than on the processing of symbols that have only an abstract relationship to it. The human-machine interface issues that arise out of this new condition may be illuminated by reference to perspectives drawn from biology, robotics, dynamical models of cognition, and autonomous agents. These serve to characterize the agents involved and so enable

an understanding of the constraints and opportunities of the interface. A comprehensive understanding of the relationship of cognition to the environment is required in order to craft an engagement with embedded devices and the material substrates through which they operate. These relationships have implications for how we understand the process of cognition relative to the organism and therefore how we might begin building interfaces to the media in which we exist. Maturana and Varela (1980) have proposed that the distinguishing features of living organisms consist in their self-production and autonomy. They are essentially closed systems, self-organizing out of the gregariousness intrinsic to specific molecular organizations and the resources available to those processes. An organism has internal states based on its sensory apparatus and realizes changes of state based on the operation of its effectors. It is from the regularities in the relationships of sensor states to effector states 97

8

Eliminate the Interface by which the organism ‘brings forth the world’. Cognition develops not from a static internal representation of the environment as given by sensory data but by patterns of relations that develop between changes in sensory data in conjunction with changes in effector states. This leads Maturana (1985) to assert that “the mind is not in the head, the mind is in behavior”. Maturana and Varela (1992) employ the concept of structural coupling in which recurring structures of behavior may arise from the repeated interaction of agents with an environment which, in itself, becomes structured by this interaction (Agre 1995). “ ...Internal state is externalized through products and behavior that select and organize the surrounding world” (Oyama, 1985:22). In this characterization, it is impossible to understand cognitive processes apart from both the specific physical constitution of the agent and the media within which it operates.

Over the course of the last fifteen years, a substantial amount of research has been devoted to articulating a behavior-based approach to artificial intelligence. Brooks (1991) designates situatedness and embodiment as two of the primary concepts of this approach. Rather than attempting to build a cognitive machine by incorporating human-level intelligence in software programs for a robotic device, this approach builds the body first and allows the behaviors to emerge from the interaction between the robot and the environment in which it is operating. The processors implemented in these robots tend to be small-scale distributed devices that are dedicated to the control of specific effectors, the collection of sensory data or the mediation between them. The behavior-based approach to intelligence and the general implementation of embedded devices owes more to the intellectual tradition of cybernetics than it does to information theory. While it is recognized that these are neither wholly distinct nor oppositional perspectives, the notion of a reciprocal interaction or feedback loop that may make no recourse to symbolic operations is especially useful. The emphasis on interaction focuses the analysis, not on the process that takes place within an individual, but 98

8

Eliminate the Interface on the relationship between entities. It encourages an examination of the interaction with and within a context. As such, it allows a discussion of the socio-cultural implications of cognition and a consideration of the place that artifacts play with in this process. The reciprocal relationship between agent and its media suggests that the isolation of cognitive processes as phenomena of the brain is an oversimplification. Kirsh (1995) and Maglio (1994) have shown that an agent’s environment may be actively restructured in order to facilitate perceptual and cognitive tasks. While clearly some actions in the world are undertaken in an effort to achieve functional or pragmatic goals, this work indicates that

there is also much that is undertaken only to facilitate cognition. Indeed, there are levels of complexity where cognition can only occur by means of this structuring. Clark (1997) proposes that the concept of the mind be extended to include the environment in which cognitive operations are performed based on the active role that the environment plays in driving these cognitive processes. Much of the extensive modification of the environment undertaken by humans may be an effort to simplify cognitive tasks and to apply previously developed solutions in the pursuit of pragmatic goals. Kirsh (1999) notes that the boundary between the agent and the environment depends upon the nature of the explanation required and it’s level of focus or analysis. In some cases it may be quite useful to think of the mind as extending into and incorporating parts of the world. Conversely, it is possible to consider just those sorts of cognitive operations to take place in the hybrid condition of agent and environment. 99

8

Eliminate the Interface Hutchins (1995b) argues that the cognitive work that is required to land a plane takes place in part through the agency of the cockpit reference materials, markers and instrumentation. These serve not only a recording or memory function but, by their configuration, allow for the instantaneous calculation and processing of certain critical information relative to the operation

of the craft. His analysis shows that the processing required takes place not within the individuals nor in the machine but by the agency of the socio-technical system that they comprise. Hutchins concludes that the thinking that takes place during this activity resides in the hybrid condition of pilots and cockpit, not in one or the other. Here the unit of analysis has shifted from individual components to the conjunction of humans and artifact together. This translation of perspective is of more fundamental interest and wider applicability than the navigational situation discussed. It suggests that many human activities and the artifacts that support them, at a wide variety 100

MAXM Project media augmented experience machines

8

Eliminate the Interface of scales, are deeply implicated in cognition. Hutchins (1995a) considers cognition to be a cultural process and culture to be a cognitive one. As we come to develop our environments with increasing levels of autonomous, adaptive and intelligent behaviors, it becomes necessary to understand the nature of the relations that we structure into these environments. We can no longer consider ourselves, as individuals or as social groups, to be the sole locus of cognitive activities operating within an essentially inert physical context. Hutchins’ analysis points out that, at a social scale, cognitive functions may be distributed and that the status of active participant in these processes must be attributed to machines as well as humans. Within the context of this analysis, then machines are no longer inert objects or artifacts in the physical realm but must be regarded as participants in the socio-cultural realm, as well.

The designation of an artifact as a co-participant assumes that there is some parity of stature within their relation. This equivalence (though clearly not yet an equality) becomes possible by virtue of the autonomous stature acquired by interactive machines. Krueger (1999) argues that increasingly complex behaviors on the part of an agent can not be met with brute force programming. The requirement that all possible states of interaction be anticipated and provided for sets a practical limit on what can be achieved. These limitations are simply a matter of mathematics. Repeatedly, as the combinatorial limitations become manifested, the designer is required to develop some means by which the agent can independently evaluate its context and take action. A measure of autonomy must be granted to the machine in order to be able to deal effectively with the complexity of its interactions with the environment. Autonomy is a fundamental change in the nature of the artifact that in turn requires a reevaluation of roles that objects play in both the cultural and cognitive processes. It is this aspect of embedded intelligence that most profoundly alters our relationship to the products 101

8

Eliminate the Interface of our material culture and requires an accommodation within the interfaces that we design in an effort to craft an interaction with them. The production of intelligent behavior in synthetic constructions must begin with a reproduction of the necessary, though perhaps not sufficient, conditions from which cognition arises in biological systems. This implies that the cognitive capacities of artificial systems will develop out of the relations that are obtained between their physical architecture and their immersion in a dynamic context. Just as the cognitive capacities of humans depend upon both their sensory apparatus and their effectors, artificial systems will depend upon the specific sensor-effector functions that are enabled. There is no reason to expect, under these conditions, that if machines become capable of cognitive functions, that these

would bear a close resemblance to the cognition of humans. Contact between human and machine agents will be most efficiently realized by direct operations in the world allowing the structural coupling of each organism to its medium to operate as both media and message. Issues of translation then need not arise. Torrance and Stein (1997) propose that multiple interacting systems may extract structurally similar regularities from a shared environment and that these regularities form the basis for a shared grounding of their respective internal states which may be disparate. This understanding of shared grounding is consistent with and sympathetic to the notion of structural coupling of an autonomous entity and its medium as given above. It offers a framework in which to consider modes of communication that could be enacted between embodied autonomous synthetic agents and humans. A direct interaction between embodied entities may have no need to make use of the layers 102

8

Eliminate the Interface of abstraction associated with symbolic and representational schemes. This interaction may proceed by a direct manipulation associated with and grounded in the shared task, whether this task is pragmatic or epistemic in nature. Brooks (1991), in discussing the merits of embodied and situated robots, notes that ‘the world is its own best model’. He argues that it is unnecessary for the machine to contain explicit representations of its context if knowledge of the environment can be made available to it via its sensors. Information about the world is immediately and directly available in its current state. The robot may, in fact, communicate with itself via the environment. The results of an activity are given by changes to the robot’s context and may be directly perceived and made use of by other sensors. This is more efficient than passing the projected consequences of the action to a centralized

comprehensive model and then verifying the model relative to the actual context. The relationship between an action and changes of state in the medium can be perceived by other agent in the environment and may constitute a form of communication. Kirsh (1995) argues that humans use space as a stable repository for both information and organization. Humans actively structure their environments in order to simplify and facilitate their cognitive tasks. When the task domain is shared by several autonomous entities direct manipulation of the environment may be the most reliable and stable form of communication. Kirsh suggests that this is, in fact, a common occurrence during cooperative work. The structuring of the environment is often a clear indication of the nature of the task being undertaken and the manner or sequence in which it will proceed. Situated acts directly undertaken in the task domain may have no need for reference outside of the context of the interaction itself, but instead possess a high degree of situational meaning. This approach 103

8

Eliminate the Interface eliminates the surface between action and communication on which symbolic tokens exist and, by doing so, eliminates all errors and approximations of translation into and out of the symbolic medium. Perhaps most importantly it circumvents the problems associated with the relationship between symbolic structures and meanings for artificial systems. Cooperstock (1995) suggests that since people can infer the intentions of others from their actions that technical systems might be crafted to do the same. The intent is to reduce the cognitive load on the user by allowing the system to make context sensitive reactions to the user’s conscious actions. The underlying design principle is to reduce complexity by enabling interaction based on skills developed by the user in everyday activity of a lifetime in the everyday world.

Penny (2000), in describing interaction with the robot ‘Petit Mal’, uses the term ‘autopedagogic’ interface. Significantly, this is an autonomous and interactive machine operating in relation to humans and so displays exactly the kinds of dynamics that arise between autonomous embodied agents. Here there is no recourse to symbolic structures, but the users and the work together negotiate the interaction in real-time and real-space by means of the activity that takes place there. Embodied and situated actions in the world are both direct and immediate. Simultaneously productive, of epistemic and pragmatic goals, and communicative, they may also reduce the need for a metaphor. While interface metaphors allow for a relatively immediate accessibility to the means of accomplishing a task within a technical environment, they also become a limitation as interaction evolves. Interfaces are typically fixed environments for interaction. The notion that the interaction between machines and humans may proceed by means of a negotiation is a fundamentally different proposition. 104

8

Eliminate the Interface This kind of interaction is not beyond human experience however. It takes place on a regular basis as part of our social intercourse. This does not remove the interface from the concerns of the designer but instead shifts its focus. Maes (1995) notes that emergent phenomena are not the result of unforeseen or accidental occurrences rather the designer must set an interaction loop between an agent and its environment that converges on the appropriate behavior. The equivalence of an interface, the negotiated understanding between autonomous entities, may be an emergent rather than a designed phenomenon. In this case, the location of the design effort moves away from a concern with the proximal detail and instead describes the parameters that shape the space or set of possibilities

within which the interaction occurs. This kind of design does not have a long tradition and much of the work that needs to be done must be considered research and experimentation. There may be insights that can be drawn from ecology, social behavior, interactive arts and media, or a study of the choreography of improvisational dance. This paper is not intended to argue that action in the world is the only viable interface strategy or to suggest that the symbolic approach is unnecessary. There is far too much evidence to the contrary. Rather the proposal is that interfaces to embedded systems need to be thought through in radically different ways from those for general information processing machines. Just as the computational metaphor provided a common frame of reference for systems concerned with symbol processing, so autonomous embedded systems may be brought into relation by means of embodied and situated actions. 105

Intermezzo 4 - Real-time structural analysis Insight enhancing techniques for structural analysis applications Nils Addink Building engineer at Technical Design & Informatics, Faculty of Architecture DUT

Abstract: Using iterative methods of calculation for analysis of structures removes some basic shortcomings in existing educational applications, and offers the possibility for a more intuitive and insight enhancing way of analysis of structures. Introduction Structural analysis was one of the first academic areas in which the number-crunching power of computers was put to practical use. The early mainframe computers could only do simple mathematical operations, but could perform them repetitively without trouble. This catered perfectly to the solving of classic mechanics equations; that required only simple operations to be executed repetitively. Sadly, the initial lead has not led to simple and complete, state-of-the-art applications to be used today. With the staggering increase in computing power, the applications for applied mechanics grew stronger and stronger; nowadays a highly accurate analysis of a immense set of physical phenomena can be applied to huge, complex models. Though this power of accurately predicting physical behaviour of objects proves to be of great use for designers and architects, it does leave a lot to be desired. The main innovations for applications for structural analysis, are in the quality of

106

Screenshot from SFERE

the prediction itself, and not in the quality of the application’s feedback. The application’s abstraction level is often so high that a designer can not use the program without extensive training or assistance. This would not be much of a problem, if all users/designers would be aware of this fact... Unknown conditions; stability required With all mechanics applications it is assumed that the user is familiar with the basics of the subject. This is, of course, no problem for those schooled in mechanics, but it can lead to unnecessary complications for new students of the field. Although advanced dynamic mechanics are of an increasing importance in the lightweight (computer-) optimised structures of today, the initial interest of an architect is in the structure’s statistical mechanical behaviour. There are a lot of simple, ‘easy-to-use’ applications available that have been developed specially to cater to this interest. These same applications are used throughout the world to help students of architecture examine the mechanical behaviour of structures; and hopefully providing some insight into the behaviour. These standard, ‘easy-to-use’ applications all function in a similar manner: solve the equations that define statistical equilibrium, and then present the results. If the entered structure is stable this approach will work fine. However, problems arise when an instable structure is entered. In that case multiple solutions for the equations exist, and the equations can not be solved. Most applications will simply state that no equilibrium could be calculated. Some applications do inform the user of measures that could be taken to make the structure a stable one, but this advise rarely makes sense. The ‘mistake’ of entering instable structures is easily made by new students. The absence of any feedback on the actual behaviour of instable structures hardly helps the students develop a basic grasp of the mechanics of structures. An integral part of understanding mechanics is understanding what would happen to the structure if certain elements were not present. If applications could show the behaviour of instable structures, this would greatly increase their value for use in academic courses and exercises. Apart this academic bonus, there is an other reason to focus on analysis of instable structures. Some structures are instable by design: e.g. tethered structures. This analysis ability could also be used to do Form Finding, like Gaudi used to design the Sagrada Família, only this time without endlessly knotting string and little bags of sand. Nils Addink

107

Iterative analysis of instable structures Using an iterative approach to finding a static equilibrium, has proven to be a simple solution for accurately analysing both stable and instable structures. Because an iterative way of calculating is inherently slower than numerically solving a set of equations, it has long been ignored for use in simple, ‘easy-to-use’ applications. But nowadays, with the immense computing power of desktop computers, this reduction in calculation speed isn’t an issue anymore. So, without a big penalty in performance, the behaviour of instable structures can be determined. In fact, by deliberately reducing the speed of convergence, a pretty good indication of the actual movement of the structure can be generated by showing the results from intermediate iteration steps. This greatly enhances the insight into the behaviour of the structure. Added bonus of iterative analysis is that it’s very well suited for background processing. With a little effort an application can be built that allows the user to immediately see the effects of changes to the model and to play interactively with the different options available. Especially for creative design this interaction is a great feature, unexpected side-effects of a change often lead to new ideas, concepts and insights... User-friendly prototype To prove that iteratively finding equilibrium using background processing is possible, and to test it’s value in an academic course, I built a prototype application called Sfere2 (Structural Finite Element Research Environment version 2). Because this application was actually going to be used by students, the standard (read: non existent) interface of a technology pilot would not do. A user-friendly interface had to be built, allowing the students to evaluate the processing method and not be distracted by difficult interfaces. Besides applying the default techniques for creating an user-friendly interface (make functions easily recognizable; - conform to known standards; - uniformly functioning; - generating direct visual feedback / WYSIWYG), some additional innovative features were introduced. Flexible interface A lot of mechanics applications developers have a tendency to show off all the possibilities of their application by placing every option visibly onscreen. Although for experienced users this might not be a problem, for new users it is very hard to find the few buttons that they actually need. By optionally hiding rarely used options and windows, new users spend less time searching the option they need, and the risk of making errors using the advanced

108

screenshots from SFERE

screenshots from SFERE

interfaces is reduced as well. When the user gains experience, more and more options can be put onscreen or behind keyboard shortcuts. Exaggerating elastic behaviour The standard way of visualising deformation within the structure is to scale the displacement of the elements. Due to the three-dimensional shape of the structure this can distort the overall impression of the stresses involved: some elements can appear to lengthen, while they’re actually under pressure. In worse cases, the distortion can be so big that the original shape of the structure is hard to recognize. Another way to make the deformation visible is by reducing the moduli of elasticity of the materials used in the structure. This has some disadvantages, the calculated displacements while not be realistic, and second order effects, like buckling, will be exaggerated as well. But, the initial impression of the deformations is more natural, and therefore easier to read/understand. If the E-moduli are scaled linearly, the stresses calculated will still be quite accurate. Of course some errors are introduced by the larger second order effects, but the calculated stress values do give a good indication. There is an added bonus in exaggerating elastic behaviour: the convergence to equilibrium can be speeded up. As stated earlier, iteratively finding equilibrium can be quite a slow process, especially with structures consisting of a large number of elements. By lowering the E-modulus the reaction force to a deformation will be less. The time span per iteration step can be increased without the risk of diverging the iteration due to large reaction forces. If the user is experimenting with the basic mechanical behaviour of a structure, lowering the E-Moduli will give faster and better insight into the matter. Testing, testing Though the application is still in its beta-stage, it has already been successfully used in an exercise for firstyear architecture students. Afterwards the students were asked to fill out a questionnaire. The results of this were extremely positive, especially regarding the real-time feedback feature. The main negative points were about the current absence of an undo-option, and the hinge-settings which are currently ignored in the calculation. Some reactions: “Now you can follow the consequences of your design decisions, step by step” “Lots of things are recognizable, and are finally made visible. On paper it is hard to grasp. This provides more insight.” My favourite, from an educational point of view: “[...], but you still have to draw your own conclusions.”

109

9

Gamesetandmatch Forum Discussion The forum discussion of the GameSetandMatch conference was recorded on tape through the microphones on the tables. Unfortunately the audience had no mics and therefore it was sometimes impossible to hear the questions. The abbreviations mean: EH: Edwin van der Heide HH: Hans Hubers KE: Kynan Eng KO: Kas Oosterhuis ME: Maia Engeli MF: Michael Fox

NA: Nils Addink OB: Ole Bouman TK: Ted Krueger XB: Xavier Boissarie

OB: Thanks for the lectures. Most of you gave an impression of what might be the new mandate of architecture (addressing panel). I started at the beginning with a new time-based architecture. In the end it turns out more as a sort of ‘brain’ based architecture. Let’s come back to one of your statements that we should think about what we need to do in order to think. We should start right away with harvesting after such interesting lectures and to make a fruit-salad out of these banana’s, apples, avocados... We were talking about interaction design, kinetic structures, game design, hyperbodies, spatialisation and of course it’s very 110

9

Gamesetandmatch Forum Discussion difficult to have a coherent discussion within such a wide field of architectural practice. So before having a discussion between the speakers and you I should give the audience the opportunity to pose questions for 15 min. to the speakers. AUDIENCE: I have a feeling that applies to you all about the outputs of architecture or games. I have a feeling that virtual reality and physical reality are still far apart. As compared

to a physical reality, when it comes to reflection of light or sound, a computer can never compute the stream of information. All these buildings that try to imitate weightlessness or emulates organic / genetic processes to me don’t look very convincing. I have feelings why these two disciplines are so attractive to each other is that there is some transition phase going on which it is to big a shock to get from physical reality to the virtual reality. But I think in the end the most impressive images are the ones that are totally based on a generic program like the images that are made from the interactive music from Edwin van der Heiden. I like it more than photoshopping a texture on the wall. I’d like to get some reactions.

111

9

Gamesetandmatch Forum Discussion OB: Thank you, I think it’s an important intervention, and it touches right away on the core of this conference, I think KO should respond first. KO: I think that there is nothing more natural than that physical and non-physical realities come together. As far as virtual reality concerned I really do regard it as a form of hyperreality, it’s more real and more exact, exactly known and verifiable than normal reality, actually they have never been separated, so what is the problem in presentations like here is that you’re looking at a presentation. The emergence we are thinking about of reality and hyperreality is that you’re in it, so you feel yourself like the technology, which is embedded in it. There is no

other way than to do that, if you find it not so convincing I think it’s logical also, we are at the beginning of a vast and extensive progress. But at the same time it’s so tantalizing to work with and to make these connections and to really feel free as a bird in a swarm of connecting elements that we have to do it. We have to feel it, have to know how it feels to be in it instead of looking at it. AUDIENCE: But I was talking about the bridge between the different realities. KO: But you said they don’t like each other, I think they extremely like each other and they can’t do without each other. AUDIENCE: When they start to compete with each other, trying to copy the characteristics. 112

servo controlled choreographed elevator doors

9

Gamesetandmatch Forum Discussion KO: Yes, but that is correct, it’s a poor attitude trying to copy nature. I think that was also in Ada’s space, it was one of the things you said: that we are interested in Ada herself, not in Ada as a copy or mimic for something that nature already offers, no, it’s a new nature, and you’re in it. AUDIENCE: Isn’t it much cheaper and more convincing to put on your helmet and your

from left to right: Michael Fox, Xavier Boissarie, Kynan Eng, Kas Oosterhuis.

gloves and really... KO: Yes, but then you’re like in a medical treatment in a hospital and you have infusions, you’re not free to move. OB: I think were wasting some energy here, there are some people that want to respond to you’re basic issue, first Maia. ME: Yes, I kind of agree with what you’re saying. Especially if you think about simulating things, because a simulation is always poorer than the reality, so if in the virtual world you’re simulating the real world the experience will never be the same. I mean we don’t have the 113

9

Gamesetandmatch Forum Discussion computing power to achieve that and I hope this is the level where we can continue the discussion and where we have some agreement. I think it’s not like black and white. There is some gradual thing that you can do, you can import some things from reality in like what Xavier showed us, you have trees, you have things that you understand what you can do with and than you can start to play and start to create an experience, but they don’t have to look real and then in the very extreme part you have something which is very abstract but the more abstract it gets the harder it is to understand what it is. Right now I’m referring to the virtual world. In the other world where we have our environments and we start to put smart things in, there they don’t have to be hyper intelligent, they can be just a sensor that turns on the light. I think that is the way to make the environment sensitive. And then I think what we really want to get at is to merge these two things and to blur them. You can start from either

side. I’m trying to start from the virtual side and trying to push it out into the physical realm. And other people like with the Ada project are more starting in a physical environment and adding more intelligence to it. EH: It’s interesting to compare what you’re saying to what is happening in music. In music we are used to having sound from speakers for a long time. Partly with the invention of the tape-recorder we are speaking about reproduction. But then the computer is used a lot in music nowadays, in different ways. You can definitely say that the computer has lead to a new kind of music and that’s very logical because I would say using the computer, and therefore a virtual environment is simply a different medium with its own qualities and I think you should use those as best as possible. TK: One of the issues is I think, that with gloves and goggles, you don’t get much social interaction. One of the things they imply is a kind of very internal and hermetic thing, where in 114

9

Gamesetandmatch Forum Discussion the Ada project you get a lot of the activity of the people around you. The fact that in the demo they are trying to track the light in the corner means that there are some social operations that are generated by that environment as well. The other issue that has been brought up is that these technologies are used for representation, which I agree is the weakest mode. That’s why the visual stuff is usual the worst, for many of us it’s the most powerful sense. I mean going in and try to do something up against of what we have such great capability in is particular very difficult. NA: What I would like to add: It is basically a medium and if you think that the written word is also just a medium, it doesn’t mean that you cannot get a nice emotion from that. If you read a book

you can really feel things. The fact that the interface you have is somewhat limited doesn’t mean that your emotions are limited. XB: Simulation can be very powerful since it is already known to the user. One can do many things with simulation. Its difficult to speak about simulation in general, it’s only a choice between a huge amount of information and reality. There are many types of simulations and I think it might not interest many users, since they are more interested in the poetry of reality. So when we are looking at something which is not realistic, it is something new. I think we can oppose the concept of simulation and coherence. It means that in a non-realistic world stuff must be coherent but it does not have to look like reality. OB: I think it is also a matter of time we can also think of this as just a stage in a development, which has decades to come. What starts to be a kind of dialectical situation with al its 115

9

Gamesetandmatch Forum Discussion problems, might become a more synthetic situation in which we don’t talk about physical vis-à-vis and of virtual space anymore. But we will think about designing this interstitial space which only happens with interaction design. This is very important. Do you agree? AUDIENCE: I do not disagree so much with these experiments.

AUDIENCE 2: I would first like to express my relatively departs from what you are saying. We are not in an initial phase, I would say that we have always experienced multiple realities and it is connected to reality. The second point I want to make is that in the discussion it seems that there is a clear awareness of what the difference between the reality and virtual reality is. Then I would like to ask to the panel to if anyone is able to explain to me if we know that thought is part of reality, or if they agree that there is little that we can exclude from reality and that there is pure interaction. I wouldn’t know of anything in this universe, that is not in some way connected or influenced by another. So how could one make then a separation and which are the parameters that would explain: this is clearly virtual and this is clearly real. 116

9

Gamesetandmatch Forum Discussion Virtuality, as I’m taught, is that what is not actual, but what is not actual about an image for example or a story? ME: I just think that the term virtual reality has got a new meaning by the use of computers. Nowadays when we talk about virtual reality you are talking about a projection of images. This way it is easy to distinguish between physical and virtual reality. There are different definitions of virtual reality so it depends on the context which one to use.

from left to right: Ted Krueger, Edwin van der Heide, Ole Bouman

KO: I was very intrigued by what Ted said in his lecture of extended brains, extended awareness and extended self. I am not just this, I am also this room. All these realities happen at the same time. It’s a natural coexistence. I co evolve with my computer I co evolve this street pattern, since it is part of my awareness its part of my adaptation to whatever evolves. Digital evolution; I’m part of that, and digital evolution cannot do without us and vice versa. I would like to hear a professional opinion of neuro-morphology. KE: I guess for me this problem of virtual reality goes back to philosophical questions from Kant and Descartes whether anything actually exists beyond you’re own thought which nobody has solved yet. For me at least, it could be a transition point, I think of my own body 117

9

Gamesetandmatch Forum Discussion as existing and using that for a reference point for everything else. I’m not into the gloves, goggles, matrix type of thing where my own hand and feet don’t matter anymore. That’s one of the things that I find appealing about this conference and that architecture revolves about the concept of a human having a body, which I like to use as a starting point. Going to far towards matrix type of things requires people to change how they think about themselves which I don’t think we are ready for yet. TK: I don’t believe in the image of Hans Morovik that we will be able to download intelligence. It’s not just about the brain. It exists in a sub state. We are not even close to coming to a sub state that could support this kind of activities. If we have something that did that it would be so different that we wouldn’t know it, we wouldn’t communicate with it. We are made of wet stuff.

OB: This generation present in the panel is old enough to understand how we dealt with architecture in the old world and in the new, in which we cannot even communicate because it’s such an incredible new condition that we cannot discuss it in the speculating way in which we are doing now. So it might be the role of this generation: analysis, understanding and experiencing. TK: This is possible but the experiencing will always be done with the body. AUDIENCE: Should architecture not be based more on interaction than aimed at experience? We are often stuck in the experience and not dealing with interactivity. KO: What is interesting about interactivity is that it’s not only between humans, but in an environment which includes humans, technique and everything in it. What interests me is that there are multiple active parties necessary: actors. If you build interactivity between 118

9

Gamesetandmatch Forum Discussion humans and an environment, the environment must be an actor too; like the Ada space and Transports project. Creating interactivity is energizing the physical environment. It’s a natural thing. Technique is embedded in products; one can talk to his microwave. What we are doing, what we are talking about is, that you step into that product, or the whole environment. It becomes an intelligent being, it’s built from a number of elements working together. It’s a connection machine, in which lots of things are happening. Boundaries become more and more blurred. We are not looking at evolution but we are inside evolution. We are not looking at things we are part of them. AUDIENCE: inaudible

EH: I want to respond to your example of the switch which turns on the blue light. This question is very much about quality and not about the amount of interaction. Interaction is everywhere. If you should switch on a blue light it would have an effect on you which makes you perform a certain action. Which afterwards would make you switch on a red light. To give a simple example. This is real interaction. The quality of this interaction is how long you are fascinated by continuing that process. TK: One additional comment; I think that being concerned about the social realm is appropriate. The new technologies seem to me machines and the products of design are creeping into the social realm. So eventually you’ll be right but you’ll also will be dealing with machines in that social conversation. MF: Something that has been bothering me al day. There is a lot of similarity between virtual 119

9

Gamesetandmatch Forum Discussion and physical, since they are both closed systems. The physical environment outside should be the basis to what architecture has to respond. Influences like climatic changes should be the start for interaction, architecture should be a moderator between interacting with humans and interacting with these external environments. OB: But I think this role of a moderator is a very old one and to finish this discussion about how new it is anyway: if you define the role of moderator, we are discussing moderating situations in terms of social situations, spatial situations, etc. Now there are a couple of new innovations like fast computers that create a new situation for the architect for doing his

moderating role. That might make a shift why this conference could not have been held in the 1960s. I would now like to have different questions. AUDIENCE: What is really new about this form of architecture if you compare it to the Archigram exposition in Rotterdam, what is the difference? KO: That’s very simple; we do it in real-time. They were just thinking about it, we do it. When I presented my project to Constant who made new Babylon he didn’t understand it at al. At the same time he made proposals for cities that are always on the move. That’s a very strange situation: they thought about it, we also do it. NA: I think the difference is also that now these ideas are more realistic to build, the 120

9

Gamesetandmatch Forum Discussion technology has become more accessible and cheaper. KO: To elaborate on this; Archigram proposed walking cities. I know you have been in India,

you have seen factories that walk, clay being transformed into bricks as a continuous process, harvesting the landscape. The question is what do we gain from that. The things we do are real, they can be actually build, and from the process of making them you learn. References like walking cities should be evolved; otherwise you block your mind. We didn’t state that we are making new things we are working in the here and now, that’s what we are doing. 121

9

Gamesetandmatch Forum Discussion OB: Maybe defining these developments as new is because old questions are tackled by every generation in a new way. What is interesting if there are any new capacities to complement the speculations of the 1960’s. Can we speculate better now? I guess that there are sufficient reasons to say that there are new situations in terms of technology we can use. We can then also say that what happened in the 1960’s was very similar to the concept of ‘Totale Mobilmachung’ of Ernest Jünger in the 30’s and what Nietzsche did a 100 years ago about arbitrariness of form and situations, etc. So it is also part of the western tradition of speculating about the faith of modernity. There are long-term trends and short-term trends, I think we should focus on this new stuff.

HH: Why should it be new? Virtual reality may not change the world. What has changed the world profoundly is communication, look at the Internet revolution. Communication has become much easier and changed the world. MF: There are many advancements today also in materials witch can alternate, change colour in real-time which result in a de-mechanisation. Instead of using mechanics, real-time alterations to materials themselves are getting more common. That is a whole new horizon. AUDIENCE: Where is the border to this form of architecture? Commercial buildings and public space seem very suitable for it, but how about other forms of architecture? TK: I think it has to do with our expectations. Changes in technologies change expectations and as technology is moving into a social realm instead of a physical realm that kind of 122

9

Gamesetandmatch Forum Discussion change would be more like a change in attitude to what is creating the change. Now, technology is a servant, this will change, we will not be able to be in charge al the time, so we will have to find a way of negotiating, which is why I’m suggesting that things have to move into the social realm. ME: Interface design for computers deals with the same problem regarding trust. We prefer that when we start the computer the computer screen looks the same as when we had shut it down. It really is about when you trust somebody or something to do something for you. So in the end it comes down to you, you have to feel secure or know what you want. You either have to formulate what you want or you have to be sure enough to know that it’s going to be the right thing that will happen. So maybe its more about social relationships with machines as well as with people.

NA: Windows is an example, in that it restricts you’re freedom. You have to update to keep running the latest applications. It’s forced on the consumer. MF: Buildings will become intelligent simply because an optimised building safes money and is more efficient. Intelligence should however not be solely used to make a building economical more efficient. The efficiency also has to relate to the usage of the building by people. There has to be layers of adaptive learning in the system that can see patterns in you’re behaviour and the interaction is build onto this adaptive behaviour and in that way you are in control of the building but it can still optimise. KO: To refer back to communication again, what Hans introduced earlier, if we talk about connecting people to other people and environments. It’s not about controlling, if you say you have a nice house then you still refer to the fact that you are in control. I think we are heading to situations in which you are no longer in control, where your environment has a will of its own and makes suggestions that you did not suspect. When I talk to another person I 123

9

Gamesetandmatch Forum Discussion don’t know what he is going to say next. It always surprises me. AUDIENCE: What is good interaction? You were talking about emotions. Since you already know or presume how people will react, how do you work with that, so you will establish this clustering? XB: I think it’s a question of a strong relationship. Things exist more by their relations than by

themselves. In the games that I worked on I saw many kinds of emotions and reactions. Most of the games are oriented at a kind of emotion and a kind of relationship to get the right effect. It’s also a matter of identity. It determines the communication, the marketing, etc. I have a question about these relationships to an architect. I wonder if the extreme evolution between someone and his environment will be a dilution of somebody into the environment. Don’t you see the problem of identity? KO: You build up identity via communication and you build up you’re own identity if you get unexpected answers, because you have to deal with it. If you knew it beforehand why communicate anyway? 124

9

Gamesetandmatch Forum Discussion XB: I think this also creates a question; you indeed create identity via relationships but sometimes you don’t know were you are, if you are in the object, or here... TK: All you know is what’s inside you, and still there is a point where we attribute all this wood on the side of this room to be something outside ourselves. We come to know some kind of boundary. Where is this border of distinction? Simple: it’s the point where we see the other. There are things in the environment that we attach to ourselves and that we attribute to ourselves. If you’re driving a car, the car becomes very much a part of you’re awareness

but when I leave it, that changes. So I don’t think it will be a fixed boundary, I think it will be a boundary that we describe by our experience as it happens. I think that we will gradually understand ourselves as extending more and more the ideology of this kind of singular individual. In the end it’s all about our experience anyway, that’s all we’ve got. XB: I’m not sure because I think there is a risk of alienation in each strong relationship. If you for instance love you’re car very much, then do you become an agent of you’re car? AUDIENCE: How will decision making be changed in the future? KO: Hans’ Kids has experience with this, because what we are talking about is multi user and 125

9

Gamesetandmatch Forum Discussion multiplayer environments and then we can have group decision systems. OB: Not only the matter of decision making is important but also the method of organization. Are the people here within ten years part of a team, deciding altogether about designs? Maybe this is also something to have a conference about; maybe next year Kas, how to organize a collaborative design process. If these specialists that ten years ago would have never gotten together to discuss architecture, how can we transcend the situation in which an architect hires other specialists to fulfil his dream, to a situation in which the architect is a member of a team which needs the creativity of all the people involved. Might be an idea for next year... KO: Absolutely, but to answer the question, I think every discipline has grown in his own

direction and now everything is coming back by these tools which we have know. We need a sort of common database to connect to, this database cannot be visualized or animated or played in real-time in other ways than using these tools. These disciplines cannot work in another way together than by using these tools, they can sit on a table and they can talk but they cannot work together. This gives insight, although it is needed in real-time. Like Joris and Jonathan who play in real-time and they change parameters in real-time. HH: The rules that are used to make decisions must be considered. If you can compute in real-time the costs of a building you are making together, than you have an important rule already. AUDIENCE: cut of by OB. OB: let’s finalize what is happening here and start to think about the agenda of next year. 126

9

Gamesetandmatch Forum Discussion I hope there will be another conference on this topic here. I’ll raise the issue about the organization of the profession in terms of how we can make decisions in multi authorship. Another important issue is of course how to organize architecture education, if this is the practice of the future. Would it be possible to invite building technicians, civil engineers, media makers, people that specialize in algorithms, will this be part of architecture education or will it be a matter of just inviting these people ones in a while. An interesting question for directors of schools: how to organize the architecture profession these days. I would like to end this conference by saying, Game Set and Match. New architecture beats old architecture with a tie break. Thank you very much.

127

10 3

References Swarm Architecture, Kas Oosterhuis http://www.hyperbody.nl http://www.oosterhuis.nl http://www.trans-ports.com Kelly, K., 1994, Out of Control

4

Ada: Buildings as organisms, Kynan Eng a.o. Delbrück, T., Douglas, R. J., Marchal, P., Verschure, P., Whatley, A. M. 1999, A device for controlling a physical system, EU patent application 99120136.9-2215

Wasserman, K. C., Blanchard, M., Bernadet, U., Manzolli, J. M., Verschure, P. F. M. J. 2000, Roboser: An Autonomous Interactive Composition System. In: Zannos I. (Ed.), Proceedings of the International Computer Music Conference (ICMC), San Francisco, CA, USA: The International Computer Music Association, p. 531-534. Intermezzo 2 - Different perspectives to real-time, Edwin van der Heide http://www. knoware.nl/users/heide http://www.sensorband.com/soundnet http://www.steim.nl http://www.knoware.nl/users/heide/waterpavilion.html http://www.oosterhuis.nl http://www.lenard.nl 128

10

References http://www.knoware.nl/users/heide/spatialsounds.html http://marnix.v2.nl http://www.cycling74.com 5

Levelsbyarchitects, Maia Engeli http://server.gameover.org/playzone http://caad.arch.ethz.ch/teaching/semwo/level5 http://synworld.t0.or.at

Saltwater Pavilion interior

http://www.computerspielemuseum.de http://www.konsum.net/linx3D http://switch.sjsu.edu/CrackingtheMaze http://www.re-load.org http://www.risco.pt/quadrum/expos/jodi_ctrl-space-sod.html Baumgärtel, T. “Spiele erobern das Museum”, http://www.heise.de/tp/deutsch/special/game/3408/1.html Baumgärtel, T. “Experimentelle Software II”, http://www.heise.de/tp/deutsch/inhalt/sa/11107/1.html

129

10

References Jindal R., Nöldeke Johannes, “Zound Garden”, http://caad.arch.ethz.ch/teaching/nds/2000/prj2/pres_g3/index.html Lang, S., “The Hobbit or there and back again, Level design for an ego-shooter game”, http://caad.arch.ethz.ch/teaching/nds/2000/chall/ch_unreal Sibenaler, P., Zehnder, M. and Van der Mark E. „Through the Looking Glass”, http://caad.arch.ethz.ch/~patrick/LOCAL/research/playground/images.html Rushkoff, D., 1996, Playing the Future - How Kids’ Culture Can Teach Us to Thrive in an Age of Chaos, New York, USA. HarperCollins

Schmitt, G. 2001, Introduction to Bits and Spaces - Architecture and Computing for Physical, Virtual, Hybrid Realms - 33 Projects by Architecture and CAAD, ETH Zurich, Maia Engeli Ed., Basel, CH: Birkhauser Publishers, p. 6-7. Zehnder, M. 2001. “NextLevel”, in Bits and Spaces - Architecture and Computing for Physical, Virtual, Hybrid Realms - 33 Projects by Architecture and CAAD, ETH Zurich, Maia Engeli Ed., Basel, CH: Birkhauser Publishers, p. 198-20. 6

Video -games, exploring an emotional architecture, Xavier Boissarie http://www.theswapmeet.com/articles/boissarie.html Intermezzo 3 - Real-time structurl analysis, Hans Hubers Deiman, E., Hubers, J.C. e.a., 1994, NOBI/Bouwkundig model, Eindhoven University of Technology, faculty of Architecture, ISSN 1381-3129

10

References O’Connor, M.J., 1998, ‘Model-based collaborative engineering: an owner’s perspective’, in: ASCE Journal of Computing in Civil Engineering, june

Rosenman, .M. A., Gero, J. S., 1997, ‘Collaborative CAD modelling in multidisciplinary design domains’, in: M. L. Maher, J. S. Gero and F. Sudweeks (eds), Formal Aspects of Collaborative CAD, Key Centre of Design Computing, University of Sydney, Sydney, Australia, pp.387-404.

10

References STEP, 1991, Part 1: Overview and fundamental principles, Draft N14, ISO TC 184/SC4/ WG6. VCA. 1994 NOBI/BBB-project, VCA instituut voor bouwinformatica, Rijswijk, The Netherlands, http://counterstrike.sierra.com/ http://www.arch.usyd.edu.au/~mike/PAPERS/collabCAD.html http://www.wmich.edu/jcce/e1222.htm

8

Eliminate the interface, Ted Krueger Agre, P., 1995, Computation and Embodied Agency, Infomatica 19(4): 527-535 Brooks, R., 1991, Intelligence without Reason, Artificial Intelligence Laboratory Memo, M.I.T Clark, A., 1997, Being There: Putting Brain, Body and World Back Together Again. Cambridge, MA: M.I.T. Press Cooperstock, J., 1995, Making the User Interface Disappear: the Reactive Room. Proceedings of the IBM Center for Advanced Studies Conference, CASCON’95, Toronto, Ca. 241-250

132

10

References Hutchins, E.,1995a, Cognition in the Wild. Cambridge, MA: MIT Press Hutchins, E.,1995b, How a Cockpit Remembers Its Speeds. Cognitive Science 19: 265-288 Kirsh, D., 1995, The Intelligent Use of Space. Artificial Intelligence 73(1-2):1-30 Kirsh, D. and Maglio, P., 1994, On distinguishing Epistemic from Pragmatic Action. Cognitive Science 18: 513-549 Krueger, T., 1999, Intelligence and Autonomy, Convergence, Research Journal into

New Media Technologies, vol. 5, no. 1, Spring 1999, Luton, UK: University of Luton Press, p. 91-101 Maes, P., 1990, Designing Autonomous Agents. In Maes, P. ed. Designing Autonomous Agents: Theory and Practice from Biology to Engineering and Back. Cambridge, MA: MIT/Elsevier, p. 1 Maturana, H. , 1985, The Mind is Not in the Head. Department of Biology, Faculty of Sciences, University of Santiago, Casilla 653, Santiago, Chile. cited in Cull, J. In Search of the Mind. available at: www.ozemail.com.au/~jcull/articles/artmind.htm Maturana, H. and Varela, F.,1980. Autopoiesis and Cognition, Boston Studies in the Philosophy of Science, vol. 42. Dordrecht, NL : D.Reidel Pub. Co., Maturana, H. and Varela, F., 1992, The Tree of Knowledge. Boston, MA: Shambahala 133

10

References Oyama, S., 1985, Ontogeny of Information, New York: Cambridge University Press

Penny, S., 2000, Agents as Artworks and Agent Design as Artistic Practice, in Dautenhahn, K., ed. 2000, Human Cognition and Social Agent Technology. Amsterdam, NL. John Benjamins Co. 134

Maia Engeli

10

References Stein, L., 1997, Post-modular Systems: Architectural Principles for Cognitive Robotics. Cybernetics and Systems, 28(6): 471-487 Sept. 1997 Torrance, M. and Stein, L. , 1997. Communicating with Martians (and Robots). manuscript available at: http://www.ai.mit.edu/people/papers/torrance-stein-97.ps.gz Waern,Y. , 1998, Cognitive Ergonomics of Collaborative Work. in

Proceedings of the Ningth European Conference on Cognitive Ergonomics. available at http://www.cs.vu.nl/~eace/ECCE9/papers/waern.pdf Intermezzo 4 - Real-time structural analysis, Nils Addink http://www.addink.net/sfere2

135

gamesetandmatch conference proceedings

Kas Oosterhuis

1 ISBN 90-5269-316-1