Organic User Interfaces

Organic User Interfaces

Seminar on Post-Desktop User Interfaces

Seminar paper at the Media Computing Group Prof. Dr. Jan Borchers Computer Science Department RWTH Aachen University

Julian Krenge Vina Wibowo Advisor: Max Möllers Semester: Winter Semester 2008 Submission date: Jan 29th, 2009

iii

Contents

Abstract

ix

¨ Uberblick

xi

1

Introduction

1

2

Related Work

3

2.1

Organic Computing . . . . . . . . . . . . . . . . . . . . . .

3

2.2

User Interfaces We Know So Far . . . . . . . . . . . . . .

4

3

4

Organic User Interfaces Defined

7

3.1

Properties of Organic . . . . . . . . . . . . . . . . . . . . .

7

3.2

Design Principles . . . . . . . . . . . . . . . . . . . . . . .

8

3.3

Definition . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

3.4

Tangible UI Vs. Organic UI . . . . . . . . . . . . . . . . .

9

On the Way Towards Organic User Interfaces

11

4.1

Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

11

4.1.1

12

Interactive Surfaces . . . . . . . . . . . . . . . . . .

Contents

iv 4.1.2 4.2

4.3

4.4 5

6

7

Deformation Tracking . . . . . . . . . . . . . . . .

13

Flexible Displays . . . . . . . . . . . . . . . . . . . . . . .

14

4.2.1

Digital Paper . . . . . . . . . . . . . . . . . . . . .

14

4.2.2

Bendable Screens . . . . . . . . . . . . . . . . . . .

15

Shape Actuation . . . . . . . . . . . . . . . . . . . . . . . .

16

4.3.1

Physical 3D Displays . . . . . . . . . . . . . . . . .

16

4.3.2

Ferrofluid Displays . . . . . . . . . . . . . . . . . .

17

4.3.3

Volumetric Displays . . . . . . . . . . . . . . . . .

18

Combination of Technologies . . . . . . . . . . . . . . . .

18

An Alternative Approach to Organic User Interfaces

21

5.1

Data Presentation . . . . . . . . . . . . . . . . . . . . . . .

21

5.2

Data Manipulation . . . . . . . . . . . . . . . . . . . . . .

22

Evaluation

23

6.1

OUI in Everyday Life . . . . . . . . . . . . . . . . . . . . .

23

6.2

Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

24

Summary and Future Work

27

7.1

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27

7.2

Future Work . . . . . . . . . . . . . . . . . . . . . . . . . .

28

Bibliography

31

Index

35

v

List of Figures

4.1

SmartSkin . . . . . . . . . . . . . . . . . . . . . . . . . . .

12

4.2

TWEND . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13

4.3

E-Ink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

15

4.4

Flexible OLED . . . . . . . . . . . . . . . . . . . . . . . . .

16

4.5

Lumen . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

17

4.6

SnOil, a ferrofluid display . . . . . . . . . . . . . . . . . .

18

4.7

Gummi . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

18

6.1

Organic UIs interaction method . . . . . . . . . . . . . . .

24

7.1

The Nokia Morph Concept . . . . . . . . . . . . . . . . . .

29

vii

List of Tables

3.1

Differences between Tangible UIs and Organic UIs . . . .

9

ix

Abstract Nowadays, user interfaces are rigid and do not utilise human’s manipulation skill. They force users to learn certain methods to interact with them. However, in order to improve the richness of interfaces, they should conform to the way humans interact with their environment. This leads to the concept of Organic User Interfaces (UIs) in which interfaces are designed to imitate this interaction. This is achieved by developing interfaces so that they comply with the following principles: input equals output, form equals function and form follows flow. Currently, there is no interface, which follows all three principles above. Nonetheless, there are several technologies leading to the realisation of Organic UIs.

x

Abstract

xi

¨ Uberblick ¨ ¨ Heutige User Interfaces sind starr und schopfen die Moglichkeiten menschlicher Interaktion nicht aus. Stattdessen verlangen sie dem Benutzer ab, sich Methoden zur Kommunikation anzueignen. Um aber die Qualit¨at des Interfaces zu verbessern, sollten sie sich der menschlichen Art mit der Umwelt zu interagieren ¨ unterwerfen. Dies fuhrt zu dem Konzept der Organisches User Interfaces (Organische UIs), die diese Interaktion imitieren. Erreicht wird dies durch die Entwicklung von Interfaces, die folgenden Prinzipien entsprechen: Eingabe gleicht der Ausgabe, Form gleicht der Funktion und Form folgt Funktion. Im Moment ist noch kein Interface ¨ ¨ verfugbar, was allen drei Prinzipien genugt. Nichtsdestotrotz sind bereits einige ¨ Technologien erh¨altlich, die die Realisierung von Organischen UIs ermoglichen.

1

Chapter 1

Introduction “Because you don’t have to be green to be green.” —MTV Switch, The Green Song

Being organic is a trend that becomes more and more popular these days. From organic food to organic clothing, from organic agriculture to organic cleaner, everything is going green to save the earth environment and make humans healthier. Inevitably, the term organic has also inspired the computing field. Nevertheless, the purpose of organic differs from the purpose of organic in general. In organic computing, the term organic means to imitate the properties owned by organic beings such as adaptation. Apart from organic computing, there has been a silent development of socalled Organic User Interfaces (UIs). In which, interfaces are built to resemble the shape of nature, which is flexible and deformable, and the interaction between nature systems. This seminar paper presents an overview of the development of Organic UIs. Several work, which are related to the concept of Organic UIs, will be presented in chapter 2. The next chapter is dedicated to the definition of Organic UIs and their design principles. Chapter 4 gives several technologies that have a great contribution in the realisation of Organic UIs. In chapter 5 an alternative approach to Organic UIs is given. Chapter 6 discusses the interaction styles and issues that may arise by implementing Organic UIs in everyday life. Last but not least, summary and possible future work will be given in the last chapter.

2

1

Introduction

3

Chapter 2

Related Work “Art is either plagiarism or revolution.” —Paul Gauguin Research on Organic UIs is still in its infancy state. Currently, not many or even hardly any real UI is introduced as Organic UI. Nevertheless, several work related to Organic UIs have been done. One work includes organic computing, which has the same concept of imitating organic properties found in nature system. Others include several UIs, of which characteristics have similarity with of Organic UIs.

2.1

Organic Computing

Organic Computing and Organic UIs share the same organic term. Although both concepts are inspired by nature, they have different approaches in realising the nature or organic properties in a system. Organic Computing is a system that dynamically adapts to changes in the environment and probably interact with each other [Seebach et al., 2007]. The objective is to use principles observed in nature system ¨ for building technical system with organic properties [Muller-Schloer, 2004]. These include self-configuration, self-adaptation, self-healing and other self-x properties, as well as context-awareness. By combining these properties, the system is given with more degree of freedom to react upon component failures or environmental changes.

Dynamic Adaptation

Self-x Properties

2

4

2.2

Related Work

User Interfaces We Know So Far

Until today, a number of UIs have been introduced to the public. Although there are no revolutionary changes since good HCI design is evolutionary rather than revolutionary [Canny, 2006], each believes that its interaction is more intuitive than any of its predecessors. Below are some of UIs that can be seen as the building blocks of organic UIs. WIMP Interfaces

Adaptive Interfaces

Ambient Interfaces

PostWIMP

Graphical UIs, especially the WIMP Interfaces, are probably the most well-knowned interface ever. Since 1970s when WIMP Interfaces were introduced, all applications running on PCs are built to support them [Canny, 2006]. A WIMP Interface consists of windows, icons, menus, and is equipped with a pointing device, in this case a mouse. Many argue that this interface does not reflect daily objects interaction. Rekimoto in his article [Rekimoto, 2008] mentioned that by using a mouse users can only interact with ”one point” and either ”pressed” or ”hover”. Meanwhile, in reality people touch objects on ”multiple points” and even put a special ”pressure” to them, like handshaking for example. The popularity of Graphical UIs has made its way to smaller mobile devices such as PDAs. However, PDAs’ small screen makes it difficult for the users to browse through menus. Adaptive Interfaces have been introduced to address this issue. The idea is that the system adapts the interface according to the users’ needs, such as displaying only the features used most at the front page rather than all features. Although this proved to be viable in small displays, in large display these ”unpredictable menus” may lead to users’ confusion [Findlater and McGrenere, 2008]. Other interfaces similar with Adaptive Interfaces called Ambient Interfaces adapt their interface not based on their users but on their users’ physical environment such as light, sound, or movement [Gross, HCII, 2003]. They can be used for awareness information environment that aims to improve awareness among geographically separated team members. One example is by displaying pop-up window giving information regarding what the other co-workers are currently working on. Although Adaptive and Ambient Interfaces are improvements of classical Graphical UIs, the main interface is still in WIMP-style. Many researchers moved towards post-WIMP Interfaces to enrich user expe-

2.2

User Interfaces We Know So Far

5

rience in interacting with interfaces by introducing more natural way of interaction. Touching is the first natural thing people do when encountering new objects. By utilising this human instinct, Tactile and Haptic Interfaces were developed [Motamedi, 2007]. The idea of these interfaces is that the users are actually touching the interfaces when they interact with them and getting haptic feedback, such as vibration. The later feature adds a value to Tactile-Haptic Interfaces especially to visually impaired users [Kahol and Panchanathan, 2006]. The ability to grasp and manipulate physical objects inspired the development of Tangible UIs. Hiroshi Ishii, as one of the pioneers of Tangible UIs, defined Tangible UIs as interfaces, which give physical form to its digital counterpart [Ishii, 2008b]. Users manipulate the digital information by directly manipulating the physical object that represents the digital information and learn what interactions are possible from the physical affordances of the object itself. Both properties increase the directness and intuitiveness of interactions.

TactileHaptic Interfaces

Tangible User Interfaces

6

2

Related Work

7

Chapter 3

Organic User Interfaces Defined “To make something look real and alive, nothing can be symmetrical because nothing in real life is symmetrical. You have to make it look organic. ” —John Kricfalusi

Take a look at today’s PC hardware, for example an LCD. It is built to be planar and rigid in order to protect the electronics inside it. Compare this with paper. It can be folded, wrapped around, torn, even recycled. Such thing cannot be imagined to be done on the current LCD [Holman and Vertegaal, 2008].

3.1

Properties of Organic

Compared to Organic Computing, in which self-x properties of nature system becomes the fundamental key in developing a technical system, Organic UIs are inspired by the shapes of the nature system which are transformable, flexible, naturally adaptable, resilient and reliable [Vertegaal and Poupyrev, 2008]. Just like a leave, which bends instead of breaks to accommodate the reception of sunlight. It also grows and adjusts its shape to flexibly adapt with the environment [Holman and Vertegaal, 2008].

Nature Shape

3

8

Natural Interaction

Organic User Interfaces Defined

Another emphasis is on the analogue, continuous and transitional nature of physical and human interaction [Rekimoto, 2008] not on the physical objects or metaphors [Schwesig, 2008]. A successful Organic UI makes the users forgetting that they are operating machines to manipulate virtual data. A mouse, which is the most popular input device, is considered as the most inorganic interface. It is a tool to point and manipulate a certain (x, y) location of an object which is located on the display, a different device. It contradicts the true nature of human interaction in which a tool is rarely needed and the fact that people manipulate the shape of the object directly at multiple points at the same time.

3.2

Design Principles

Holman and Vertegaal [Holman and Vertegaal, 2008]defined three design principles, which can be used as guidelines, to develop Organic UIs. These include input equals output, form equals function and form follows flow.

Input Equals Output

Form Equals Function

Form Follows Flow

A person draws directly on the paper and views his/her drawing directly from the paper. This visualises a true physical interaction where input and output interaction happen at the same location, in this case the paper. To imitate this behaviour, Organic UIs should have a display which can sense [Holman and Vertegaal, 2008]; or to put it differently the input device acts also as the output device. The form of an object gives hints on what activities people can do with it [Holman and Vertegaal, 2008]. The flexible form of a piece of paper suggests that it can be folded, bent, crumpled or even torn up. Yet, it can still serve its original purposes: to be read or written on. Organic UIs should use their form as a physical representation of activity. A spring changes its shape to follow the movement of the person’s hand, which extends it. As the person lets the spring go, it will go back to its original shape. This implies that Organic UI should be able to either alter its shape to follow the flow of user interaction or to adapt its shape automatically for better context of use [Holman and Vertegaal, 2008].

3.3

3.3

Definition

9

Definition

Concluding from the three design principles above, a definition of Organic UI can be formalised. An Organic UI is a UI that more closely resembles natural human-physical and human-human interaction by using nonplanar displays, as input and output, that may actively or passively change shape following analogue physical input to adapt to user’s needs. [Rekimoto, 2008] [Holman and Vertegaal, 2008] [Vertegaal and Poupyrev, 2008] Although they are said to be organic, Organic UIs do not need to be made out of organic materials. The emphasis is to promote flexibility, enhance users satisfaction and allow users to be creative rather than productive. [Holman and Vertegaal, 2008].

3.4

Tangible UI Vs. Organic UI

By looking at the definition of Organic UIs defined above, similarities between Organic UIs and their predecessor, Tangible UI cannot be avoided. Organic UIs are Tangible UIs but Tangible UIs are not always organic. Tangible UIs are said to be the gate to Organic UIs. They inspire Organic UIs to use physical interaction to manipulate digital data. However, they lack of shape adaptation since the tool used to interact cannot change shape in real time. Adding shape actuating behaviour makes Tangible UIs may be considered as Organic UIs [Ishii, 2008a]. Table 3.1 describes further differences between Tangible UI and Organic UI [Rekimoto, 2008]. Characteristics Interaction Metaphor Orientation Representation Coverage

Tangible UIs using tool manipulation-oriented tool = digital info application specific

Organic UIs direct contact communication-oriented shape = activity generic

Table 3.1: Differences between Tangible UIs and Organic UIs

Physical Manipulation No Shape Adaptation

10

3

Organic User Interfaces Defined

11

Chapter 4

On the Way Towards Organic User Interfaces “Technology: No Place for Wimps!” —Scott Adams, Dilbert

As shown, Organic UIs are meant to be more intuitive than other UIs. To provide this, technologies enabling devices to feel organic are necessary. Input as well as output technologies are essential to make the interaction between human and computer feel natural. In this section, some of these technologies are described.

4.1

Input

First of all, the users should be provided with an intuitive way to manipulate data on the computer. Since nowadays most common input devices, such as mouse and keyboard, are very unnatural, a touchscreen can be said as an advancement of the nativeness of a user interface, even though a touch-screen is bound to the rigid and planar shape of a computer. Since rigidity does not occur in nature, Organic UIs should be without edges and corners as well.

4

12

4.1.1

On the Way Towards Organic User Interfaces

Interactive Surfaces

Three kinds of evolvement could be applied to a standard touch-pad. By these, a standard touch panel could be improved to an interactive surface, which enables intuitive interaction. Flexible Devices

Multiple Inputs

Gesture Detection

Firstly, the bondage of rigidity has to be broken. Not only should touch pads be capable to fit any form or shape but also to be deformed while being used. A first approach to this improvement is SmartSkin that could be produced to be flexible. Additionally, it could also be transparent to be applied on top of a display [Rekimoto, 2002]. Secondly, multiple inputs should be allowed, so that more than one hand can be used and even multiple users are able to interact at the same time. At the moment, multi-touch panels, which use different technologies, are available. The best-known examples are Apple products such as the iPhone or iPod touch. Thirdly, the sensing capabilities should be improved. Hovering as well as other gestures should be recognised. This would allow a more natural way of input as one can express complex instructions in uncomplicated ways. SmartSkin , which uses capacitive sensing, [Rekimoto, 2002] provide this capability. A more recent approach is ShapeTouch , in which the sensing is more precise [Cao et al., 2008]. By using optical recognition, ThinSight tracks hand gestures in short distance and through an overlying display [Hodges et al., 2007]. Toshiba recently released a notebook with a dedicated processor to recognise gestures using the built-in webcam [Toshiba DPD, 2008].

Figure 4.1: SmartSkin, an interactive surface - by Sony CSL

4.1

Input

Although these technologies are promising, there are still several issues to be addressed. Up to now there is no way to provide direct haptic feedback. Also, the coupling of touch panel and display has to be improved to enable thin devices. Using projectors decouples the input and output. Additionally, the use of gestures is a newly discovered field and effective ways of interaction have yet to be found. User studies have to be performed to discover how users would like communicating their commands.

4.1.2

13 Issues

Deformation Tracking

Pure bearing of deformation, as in the section before, might not be enough. For instance books allow input by deformation when searching for a specific page. It is likely to bend the pages in a way that they quickly flip over enabling the user to scroll through the book while getting a glance at every page. Yet, this is not the only example. Deformation is a very common way of interaction with objects. This leads to the conclusion that organic devices should be capable of tracking its own actual shape.

Deformation

TWEND, an input device, named by the terms ”twist” and ”bend”, provides this capability. Deformation regarding to the X- and Y-axis can be recognised. In this approach, optical bending sensors are used. Based on this technology, gestures can be defined: flipping one corner over for going to the next page or simulating a dog-ear for bookmarks [Herkenrath et al., 2008]. A more recent device providing similar features is Bookisheet [ichiro Watanabe et al., 2008].

State

Figure 4.2: TWEND, a device to track twisting and bending - by the Media Computing Group at RWTH Aachen

as Input

of the Art

4

14 Issues

On the Way Towards Organic User Interfaces

The problem when allowing users to interact via deformation is that the whole device has to be deformable. Flexible displays are available but the whole processing hardware has to be flexible as well. Useful gestures have to be found similar to the multi-touch panels. User studies on the interaction based on gestures have to be conducted to clarify how these features can be used efficiently.

4.2

Flexible Displays

Only creating devices, which are deformable, is not enough to create a flexible touch-pad or even to track the deformation. Since the whole device should be flexible, the display has to be bend- and twistable as well. In addition, this development is accompanied by a higher durability. There are two highly promising technologies.

4.2.1 High Contrast and Durability

State of the Art

Issues

Digital Paper

Formerly, the most used medium to provide information was paper, which is very different to computer screens. On one hand, it lacks of the ability to change its information easily. On the other hand, it is flexible and has a very high contrast. While computer screens are not readable any more when the sun shining on them, paper is immune to that effect. Closing this gap is electrophoretic ink displays, which are often associated with the brand E Ink. They combine advantages of both media. Also, they can be equipped with background lighting easily. In addition, digital paper has less power consumption than common Liquid Crystal Displays (LCDs) because due to their technique, their state is stable and only changes trigger the use of energy [Comiskey et al., 1998]. Recently, E Ink was improved such that it is unbreakable. It withstands extreme vibration as well as impacts of heavy objects. This broadens the application fields of electrophoretic displays.

Electrophoretic displays are already far developed and applicable in productive use. There are several products on the market using this display technology, usually developed as E-Book readers, such as the Amazon Kindle. However, digital paper still suffers from two main

4.2

Flexible Displays

15

Figure 4.3: A bendable E-Ink display - by E-Ink

problems. Firstly, its frame rate is low. Videos cannot be shown using this kind of display. This also hinders electrophoretic displays to be used in portable devices such as phones or PDAs because fluid menu navigation is not possible. Even so they convince their customers with their very low energy consumption. The Motorola F3 was the first cell phone using digital paper but had only basic features. Secondly, the chroma resolution is very low. Coloured displays can be produced by microcapsules containing red, green and blue droplets rather than white ones. Three capsules could be combined to one pixel.

4.2.2

Bendable Screens

While digital paper enhances the way the information can be altered easily, Organic LED (OLED) displays improve nowadays screens. By using OLED technology, full-coloured computer displays can be very thin and robust and therefore flexible. Compared to an LCD, the energy consumption is lower and the size of the borders is smaller. This enables OLED displays to be applied in the most convenient way.

Full-

Sony has already presented a flexible full-coloured display based on the OLED technology. Recently, Samsung SDI presented an OLED display with a thickness of only 0.05 mm, which actually flaps in the wind. These thin displays allow light shining through them and are flexible.

State

colour and Very Thin

of the Art

4

16

On the Way Towards Organic User Interfaces

Figure 4.4: A flexible Organic LED display - by Pioneer

Issues

Although OLED displays are far developed and already applicable in productive use they still suffer from several issues. At present only relatively small displays are available. Because of this, OLED displays are only available in small devices such as mp3-players. In further development, the screen size has to be enlarged. Another issue is the relatively short lifetime of Organic LEDs. Although they are still very high compared to LCD- or Plasma-displays, they still cannot compete with LEDs.

4.3

Shape Actuation

Two-dimensional displays are not the only way to visualise information. In several applications, the shape of an actual object represents digital data. Recent computers display a three-dimensional object by having several points of view at the object at the same time. This adds additional information to the data. Additionally, direct manipulation of an actual object would be more efficient and natural.

4.3.1 Actual Objects

Physical 3D Displays

The most efficient and convenient way of manipulating threedimensional objects would be a physical representation of the object of interest. This would grant direct haptic feedback and an instant view on the result. This overlaps with the field of Tangible UIs.

4.3

Shape Actuation

One of technology for shape alternation is so called shape memory alloys. Using this is Lumen. It is a 16x16 pixel display where a third dimension is added by enabling the pixels to alter their physical height. Therefore, every pixel provides not only information by its red, green and blue colour value, but also by its height [Poupyrev et al., 2004, 2007].

17 Simple Approach

Figure 4.5: Lumen, a simple three-dimensional display - by Sony CLS

At the moment the possibilities of shape alternation are limited. Lumen is a simple approach and cannot convey a lot of additional data via the height of the pixels. More advanced approaches would need more complex shape alternation.

4.3.2

Issues

Ferrofluid Displays

While physical 3D displays are still bound to the shape of their elements, ferrofluid displays are an approach to alternate shape in another way. Ferrofluid acts similar to iron, but is liquid. When a magnet approaches, it changes its shape.

Shapeable

An example for ferrofluid displays is SnOil. It is an implementation of the classic game Snake. Underlying electromagnets influence a basin filled with ferrofluid [Poupyrev et al., 2007]. Another approach is Protrude, Flow. It is not displaying information but focusing on aesthetics [Kodama, 2008].

State

Although ferrofluid displays are more versatile than nowadays physical 3D displays, they are not capable of representing any shape. They are bound to the possibilities of electromagnets. In addition, they cannot be touched.

Issues

Liquid

of the Art

4

18

On the Way Towards Organic User Interfaces

Figure 4.6: SnOil, a ferrofluid display - by Martin Frey at UDK Berlin

4.3.3 Holograms

Volumetric displays are not part of the field of shape actuation but also capable of displaying information three-dimensionally. A precise hand tracking could enable users to modify the projected objects directly using their hands. Since volumetric displays are encapsulated in glass, this concept is yet unrealisable [Grossman and Balakrishnan, 2006].

4.4 Building an Organic UI

Volumetric Displays

Combination of Technologies

When building an Organic UI it is not necessary to include all organic technologies mentioned before. As already explained an Organic UI has to be adjusted to its use very carefully. These technologies are to utilize that fit to the theme of the device that is about to be built.

Figure 4.7: Gummi, an organic digital map

4.4

Combination of Technologies

19

Gummi is the concept of a digital map as an Organic UI. The whole device is bendable and bending is used as input. In addition, a bendable touch-pad is positioned on the backside. Gummi is the first device combining a lot of organic aspects. It is flexible and also tracks the deformation to provide intuitive input. The user is not that aware of it as a computer but recognises its capability to display a street map [Schwesig et al., 2003, 2004]. Another example for the concept of a Organic UI is Morph, which will be explained in 7.2—“Future Work”.

First

Currently, Gummi is just a concept, the actual prototype does not consist of all aimed features. The prototype already allows evaluating the effectiveness of the interaction since all features are emulated. Gummi is not yet bendable but recognises applied pressure as if one was bending it.

Issues

Organic UI

20

4

On the Way Towards Organic User Interfaces

21

Chapter 5

An Alternative Approach to Organic User Interfaces “A different language is a different vision of life.” —Federico Fellini

Up to now, the focus lays on devices that enable users to interact with them in a natural way. The most important aspect on this was the flexibility of the devices. However, there is a different approach in viewing Organic UIs, that not only the device itself can be organic but also can be the software. There is a possibility for software to provide information in a natural way and still running on an ordinary computer.

5.1

Data Presentation

Data could be provided and shown in a natural way. Nowadays, presentation tools are based on the concept of a linear sequence of slides. Obviously the human mind is not organised in a straight proceeding. Dealing with this problem is Fly. It tries to map the thoughts of the presenter to the presentation. The information to be described is organised in any way the author wants it to be [Holman et al., 2006].

Organic Presentation Tool

5

22

An Alternative Approach to Organic User Interfaces

Another section of the field of organic data presentation is the data structuring. Apple recently developed iPhoto software to organise photos in a more natural way than any other software before. Photos are grouped into events that can be tagged by the users. By this, they do not have to remember the exact date, but only the people they met there. This is a good representation of the human memory.

5.2 Organic Video Navigation

Issues

Data Manipulation

Digital data could be manipulated in a more natural way. While the common way to navigate within a video is the timeline, the manipulation of objects in the video to browse through time would be more convenient. For example, one can jump to a previous part of the movie by dragging a person backwards in space and by this also in time. When a digital object is moved to the position where it was in a previous part, all other objects are reset to their previous positions as well. This is the concept of DRAGON [Karrer et al., 2008]. Although DRAGON is a very organic way of video navigation, it is not practical. For instance, when the scene changes, the objects on the frame change as well. Therefore, DRAGON could not be used for navigation in movies.

23

Chapter 6

Evaluation “I do not fear computers. I fear the lack of them.” —Isaac Asimov Several technologies in the field of Organic UIs have been described. However, it is questionable whether these approaches might be applicable to everyday life.

6.1

OUI in Everyday Life

A few years ago when PCs made its way to offices to store necessary business documents, employers have dreamed to cut business cost by realising a paperless office. Nevertheless until today, many employees still prefer to print out the document, hold the paper in hands and mark things up [Blevis, 2008]. With the emergence of thin and flexible displays, they are becoming more and more paper-like. In the near future, it is likely that displays can be treated as real paper and paperless office can finally be realised. By applying other continuous parameters such as pressure, one can improve the intuitiveness of the interface [Rekimoto, 2008]. Different pressure applied to the interface should result in different possible action. An example would be the bending interaction, which means zooming action. A softer pressure to the interface means that the zooming action is slower than when a harder pressure is applied.

Paperless Office

Continuous Parameters

6

24

Evaluation

Figure 6.1: Paper-like Organic UIs interaction method The development of Organic UIs is still in its infancy. Therefore, not many interaction methods could be thought up during this period. Emotionally- However, regardless on which interaction method introduced later, involved interaction with Organic UIs should make people emotionally involved in the interaction and forget that they are operating computers [SchweEveryday sig, 2008] and see them as ordinary everyday object [Holman and VerteObject gaal, 2008]. Alternative Interaction

As for the alternative approach of organic UIs, the interaction method might be different with the Organic UIs mentioned above. However, the emphasis is still to establish a natural interaction between the users and the interface. DRAGON, for example, allows users to navigate through a video by selecting the object inside the video instead of the video timeline [Karrer et al., 2008]. While Fly presents a new way to organise presentation slide as mind map rather than in linear order [Holman et al., 2006].

6.2

Issues

Nothing is perfect. The concept of Organic UIs gives a modern approach on how computers can integrate seamlessly in everyday life so that people forget that they are actually computers. Nevertheless, issues arise inevitably.

Flexible Hardware

Flexibility in Organic UIs arises issues concerning the hardware. Not only the displays, which should be flexible but also the processor and other electronics. Current technologies mentioned before revealed that there is no any approach, which tries to combine organic in- and output technologies so that they can be seen as one device. While TWEND does not possess a display, flexible displays such as E-Ink or Organic

6.2

Issues

25

LED supplies no input method. Gummi tried to combine these two technologies. However, they are still located in different positions (front and back). Recently developed technologies in the field of shape actuation suffer from another problem. Either they are bound to certain types of form manipulation and therefore cannot display any information or they are not suitable for human interaction as they can not be touched. The interaction techniques of Organic UIs are not complete yet. Only simple interaction techniques, such as bending for zooming action using both hands, have been introduced. It is questionable whether later the interaction can involve other modalities, such as eye gazing, blowing and entire body. Thus, more interaction techniques are still yet to be discovered. [Rekimoto, 2008] One property that makes Organic UIs differ from Tangible UIs is that Organic UIs are general rather than application-oriented. However, it does not mean that one Organic UI fits all since consistencies across activities and contexts might be difficult to realise [Holman and Vertegaal, 2008]. In graphic application, bending might mean zooming; while in video application bending might mean fast forwarding. Last but not least, enhancing computers in a way that they are not recognised as computers any more might lead to privacy problem. People might feel inconvenience by the thought of being observed by computers all the time and not being able to distinguish between digitally improved and ordinary objects in their everyday life.

Untouchable Interface

Incomplete Interaction Techniques

Interaction Inconsistencies

Privacy

26

6

Evaluation

27

Chapter 7

Summary and Future Work “Our imagination is the only limit to what we can hope to have in the future.” —Charles F. Kettering

Coming to the end of this seminar paper, several questions are still remained. How is the future of Organic UIs? Will they be as successful as Graphical UI? What improvements can be done to guarantee the future of Organic UIs?

7.1

Summary

Organic UI is a UI which more closely resembles natural humanphysical and human-human interaction by using non-planar displays, as input and output, that may actively or passively change shape following analogue physical input to adapt to user’s need 3.3— “Definition”. The definition states that Organic UIs should follow three design principles, which are input equals output, form equals function and form follows function 3.2—“Design Principles”.

Definition

Based on these definition and design principles, currently there is no real UI which follows these principles. Nevertheless, several technologies of input, flexible displays and shape actuators play an important role in the realisation of Organic UIs.

No

Design Principles

Organic UI

7

28 Alternative Approach

Issues

Summary and Future Work

Another approach in viewing Organic UIs from other point of view has been introduced. Rather than focussing on the shape of the device, the alternative approach focusses more on the interaction between the users and the application interface. Organic UIs aims to create interfaces, which are seamlessly integrated with everyday life objects. To resemble everyday life objects, Organic UIs need to be flexible. This property is fulfilled by the emergence of flexible displays and flexible input devices today. More research and experiments still need to be done in order to couple these two technologies. The seamlessly integration of the interface might raise privacy problem. People are afraid of being watched since they cannot differentiate between which objects are computers and which are not.

7.2

Future Work

Organic UIs is a fresh concept in interaction design. Since its official introduction published by ACM in April 2008, there has been no publication labelled “Organic UIs“ which leads to further development of Organic UIs. Nevertheless, by looking at the past and current technologies presented before, a possible future work can be concluded. Input + Output

Nokia Morph Nanotech

The in- and output technologies which lead to the realisation of organic UIs have been developed. However, there has not been any attempt to put these two technologies together such that they can be seen as one device. One possibility is by combining TWEND which is an input device and Flexible OLED which an output device. Both are flexible and possible to be put into one device. Introduced earlier this year was a concept by Nokia called Morph, [Nokia, 2008] which uses Nanotechnology as the basis for future mobile phones. Nanotechnology is a development and research on materials of which size ranges from 1 -100 nanometer (1 nm = 10−6 mm = 10−9 m) [Paull and Lyons, 2008]. By using Nanotechnology, the Nokia Morph demonstrates the possibility to have a mobile phone, which is flexible, stretchable and transparent. It is charged using solar power and has integrated sensors, which can sense the environment around the users.

7.2

Future Work

29

Figure 7.1: The Nokia Morph Concept

Further user studies should also be done to discover new interaction techniques for using organic UIs and to address the privacy issues, which might come out when a computer does not feel like a computer anymore.

New Interaction Techniques

30

7

Summary and Future Work

31

Bibliography Eli Blevis. Sustainability implications of organic user interface technologies: an inky problem. Commun. ACM, 51(6):56–57, 2008. John Canny. The future of human-computer interaction. Queue, 4(6): 24–32, 2006. Xiang Cao, Andrew D. Wilson, Ravin Balakrishnan, Ken Hinckley, and Scott E. Hudson. Shapetouch: Leveraging contact shape on interactive surfaces. Horizontal Interactive Human Computer Systems, 2008. TABLETOP 2008. 3rd IEEE International Workshop on, pages 129–136, Oct. 2008. Barrett Comiskey, J. D. Albert, Hidekazu Yoshizawa, and Joseph Jacobson. An electrophoretic ink for all-printed reflective electronic displays. Nature, 394:253–255, May 1998. Leah Findlater and Joanna McGrenere. Impact of screen size on performance, awareness, and user satisfaction with adaptive graphical user interfaces. In CHI ’08: Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, pages 1247–1256. ACM, 2008. Tovi Grossman and Ravin Balakrishnan. The design and evaluation of selection techniques for 3d volumetric displays. In UIST ’06: Proceedings of the 19th annual ACM symposium on User interface software and technology, pages 3–12. ACM, 2006. Gero Herkenrath, Thorsten Karrer, and Jan Borchers. Twend: Twisting and bending as new interaction gesture in mobile devices. In Extended Abstracts of CHI 2008 Conference on Human Factors in Computing Systems, Florence, Italy, April 2008. ACM Press. Student Research Competition 2nd Place. Steve Hodges, Shahram Izadi, Alex Butler, Alban Rrustemi, and Bill Buxton. Thinsight: versatile multi-touch sensing for thin form-factor

Bibliography

32

displays. In UIST ’07: Proceedings of the 20th annual ACM symposium on User interface software and technology, pages 259–268. ACM, 2007. David Holman and Roel Vertegaal. Organic user interfaces: designing computers in any way, shape, or form. Commun. ACM, 51(6):48–55, 2008. David Holman, Predrag Stojadinovi´c, Thorsten Karrer, and Jan Borchers. Fly: an organic presentation tool. In CHI ’06: CHI ’06 extended abstracts on Human factors in computing systems, pages 863–868. ACM, 2006. Jun ichiro Watanabe, Arito Mochizuki, and Youichi Horry. Bookisheet: bendable device for browsing content using the metaphor of leafing through the pages. In UbiComp ’08: Proceedings of the 10th international conference on Ubiquitous computing, pages 360–369. ACM, 2008. Hiroshi Ishii. The tangible user interface and its evolution. Commun. ACM, 51(6):32–36, 2008a. Hiroshi Ishii. Tangible bits: beyond pixels. In TEI ’08: Proceedings of the 2nd international conference on Tangible and embedded interaction, pages xv–xxv. ACM, 2008b. Kanav Kahol and Sethuraman Panchanathan. Distal object perception through haptic user interfaces for individuals who are blind. SIGACCESS Access. Comput., (84):30–33, 2006. Thorsten Karrer, Malte Weiss, Eric Lee, and Jan Borchers. Dragon: a direct manipulation interface for frame-accurate in-scene video navigation. In CHI ’08: Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, pages 247–250. ACM, 2008. Sachiko Kodama. Dynamic ferrofluid sculpture: organic shapechanging art forms. Commun. ACM, 51(6):79–81, 2008. Nima Motamedi. The aesthetics of touch in interaction design. In DPPI ’07: Proceedings of the 2007 conference on Designing pleasurable products and interfaces, pages 455–460. ACM, 2007. ¨ Christian Muller-Schloer. Organic computing: on the feasibility of controlled emergence. In CODES+ISSS ’04: Proceedings of the 2nd IEEE/ACM/IFIP international conference on Hardware/software codesign and system synthesis, pages 2–5. ACM, 2004. Nokia. The morph concept, December 2008. nokia.com/A4852062.

URL http://www.

Bibliography John Paull and Kristen Lyons. Nanotechnology: The next challenge for organics. Journal of Organic Systems, 3(1):3–22, 2008. Ivan Poupyrev, Tatsushi Nashida, Shigeaki Maruyama, Jun Rekimoto, and Yasufumi Yamaji. Lumen: interactive visual and shape display for calm computing. In SIGGRAPH ’04: ACM SIGGRAPH 2004 Emerging technologies, page 17. ACM, 2004. Ivan Poupyrev, Tatsushi Nashida, and Makoto Okabe. Actuation and tangible user interfaces: the vaucanson duck, robots, and shape displays. In TEI ’07: Proceedings of the 1st international conference on Tangible and embedded interaction, pages 205–212. ACM, 2007. Jun Rekimoto. Organic interaction technologies: from stone to skin. Commun. ACM, 51(6):38–44, 2008. Jun Rekimoto. Smartskin: an infrastructure for freehand manipulation on interactive surfaces. In CHI ’02: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 113–120. ACM, 2002. Carsten Schwesig. What makes an interface feel organic? Commun. ACM, 51(6):67–69, 2008. Carsten Schwesig, Ivan Poupyrev, and Eijiro Mori. Gummi: user interface for deformable computers. In CHI ’03: CHI ’03 extended abstracts on Human factors in computing systems, pages 954–955. ACM, 2003. Carsten Schwesig, Ivan Poupyrev, and Eijiro Mori. Gummi: A bendable computer. In CHI ’04, pages 24–29. ACM, 2004. Hella Seebach, Frank Ortmeier, and Wolfgang Reif. Design and construction of organic computing systems. In IEEE Congress on Evolutionary Computation 2007, pages 4215–4221. IEEE, 2007. Toshiba DPD. Toshiba qosmio world’s first laptop with cell processor technology, July 2008. URL http://explore.toshiba.com/ pressrelease/423413?fromPage=editorials. Roel Vertegaal and Ivan Poupyrev. Organic user interfaces. Commun. ACM, 51(6):26–30, 2008.

33

Typeset January 28, 2009