Information Interaction

case studies compiled by Ada-Nkem Juwah (text are direct quotes or based on quotes from sources)

object

text input punch card command line mouse keyboard digital pen joystick, wii wand, etc

gestural

cellphone, smartphone etc human-machine connection other

interaction ambient touchscreen

screen

display screen real time aggregated

environment visual thermal movement audio scent

real time aggregated

web

crowdsourcing

aggregated

data

HARDWIRED punch cards c://ommand line 

(graphical)

»» Even the old can be adapted to be new »» multitouch for the conventional keyboard »» keyboard accepts pressure-sensitivity data »» can infer user intentions »» Microsoft http://www.fastcompany.com/blog/ chris-dannen/techwatch/multitouchfuture http://www.youtube.com/ watch?v=PDI8eYIASf0

Data Visualization »» Ben Fry »» acquire – the matter of obtaining the data, whether from a file on a disk or from a source over a network. »» parse – providing some structure around what the data means, ordering it into categories. »» filter – removing all but the data of interest. »» mine – the application of methods from statistics or data mining, as a way to discern patterns or place the data in mathematical context. »» represent – determination of a simple representation, whether the data takes one of many shapes such as a bar graph, list, or tree. »» refine – improvements to the basic representation to make it clearer and more visually engaging. »» interact – the addition of methods for manipulating the data or controlling what features are visible. http://benfry.com/phd/dissertation-050312b-acrobat.pdf http://benfry.com/isometricblocks/

»» Narrative 2.0 »» visualizes music »» music was segmented in single channels »» the angle of the line changes according to the frequency of the channel, while the frequency reaching a high level, the »» channel becomes highlighted by orange »» music as an artist http://www.matthiasdittrich.com/projekte/ narratives/visualisation/index.html

»» We Feel Fine »» Jonathan Harris & Sep Kamvar »» harvesting human feelings from weblogs »» (real-time) every few minutes »» searches new blog entries with the phrases “I feel” and “I am feeling” »» identifies the “feeling” expressed in that sentence »» age, gender, and geographical location of the author can often be extracted »» local weather conditions also captured http://www.wefeelfine.org/index.html

»» visualizing intimate biometric data and emotional experiences using technology. »» Bio Mapping project »» Galvanic Skin Response (GSR), a simple indicator of the emotional arousal, is recorded in conjunction with one’s geographical location »» “Emotion Maps” were generated of the city in which the participants roamed around http://infosthetics.com/archives/2009/04/emotional_ cartography_implications_of_visualising_intimate_ biometric_data.html

»» “Abundance” »» Camille Utterback (2007) »» video installation »» converts the movement of pedestrians into colourful silhouettes and patterns that are projected onto a nearby 3-storey rotunda http://www.cartridgesave.co.uk/news/7incredible-computer-generated-works-of-art/

»» digg labs »» real-time data visualization

http://labs.digg.com/stack/

»» apple store display »» fake “real-time” data visualization of app purchases »» employs metaphor of ripple http://www.techcrunch.com/2009/06/08/apples-cool-matrix-style-app-wall/

»» Personas »» Aaron Zinman , MIT »» sophisticated natural language processing and the Internet »» data portrait of one’s aggregated online identity http://personas.media.mit.edu/

»» Ben Fry »» Darwin Origin of Species http://benfry.com/traces/

»» experiments with systems that monitor traffic patterns in real time and manage the use of lanes and access accordingly »» ability to map patterns as they happen, along with the alternatives and consequences. »» In Stockholm a system that tracks the movement of every car has reduced carbon emissions by 25% »» Pictured: Litmus (in UK) »» monitors traffic, tide level, wind turbine activity http://www.fastcompany.com/blog/michael-cannell/cannell/visualizationnew-frontier-design

http://www.youtube.com/watch?v=IkmpIXd9Q90&feature=related

»» environmental information »» movement »» speech patterns / conversation http://www.fastcompany.com/blog/michael-cannell/cannell/visualizationnew-frontier-design

http://www.youtube.com/watch?v=jEGwsfpHevU&feature=related

Data Interaction

»» touch »» gesture »» virtual

»» visualization in the public sphere »» CNN wordle of President Obama’s press conference »» CNN’s magic wall (developed by perceptive pixel) http://www.fastcompany.com/blog/michael-cannell/ cannell/visualization-new-frontier-design

http://www.perceptivepixel.com/applications.html

»» G-speak: a “spatial operating environment.” »» Oblong Industries (Minority Report's science advisor, John Underkoffler, is the founder) »» Spatial operating environment (SOE) »» combination of gesture-based input, networking, real-world displays »» operate with gloves »» free-hand gestural input that can include pointing, grabbing, and several other hand poses from several pairs of hands simultaneously »» Anything on-screen can be manipulated directly http://www.fastcompany.com/news/2008/11/16minority-report-g-speak.html http://www.youtube.com/watch?v=dyMVZqJk8s4

»» 'SixthSense' »» MIT »» wearable gestural interface »» augments the physical world around us with digital information »» natural hand gestures interact with information. http://www.pranavmistry.com/projects/sixthsense/

metaphor

play ... The interface sported almost no expanatory text or conventionally meaningful icons... She wasn’t intimidated, confused, or annyoyed; she seemed to consider the lack of instructions part of the game. She merely started the game and began clicking on objects...

“How do you know which blocks to hit?” “I just ... hit them.” “So how do you figure out what the rules are?” “Just play.” “Just play? And then what happens?” “You just ... play.”

Games are played for no other reason than for the experience of playing them—unlike a software application, in which the experience or enjoyment of the user is a by-product. If the experience of the interface is not pleasing, players will walk away. By contrast, the interfaces of many cell phones, software applications, digital cameras, microwave ovens, cars, and even wayfinding systems are maddening to use. In some situations—famously the VCR—the interface has been bad for so long that we expect operation to be frustrating and difficult. A good game interface will not bombard the user with information at the outset or rely on a complex instruction manual; it will teach the user everything he or she needs to know on a need-to-know basis. This convention is so entrenched, in fact, that gamers trust the system and never read the manuals. Figuring out how it works, whether it’s boosting your cyborg hero’s bomb-disposal skills or downloading a cheat code that makes her invisible to flying aliens, is part of the game. “A game’s system itself generates meaning, and the way it changes over time begins to modify your understanding of that system,” Salen says. “It’s a basic principle that can apply to all kinds of design.” The Principles of Play

http://www.metropolismag.com/story/20060911/the-principals-of-play

Sugar OS: http://laptop.org/en/laptop/software/index.shtml

kiosk system “head over heels” White Void http://www.whitevoid.com/portfolio.html

multi touch console “forward current” White Void http://www.whitevoid.com/portfolio.html

»» Media artist Toshio Iwai and Yamaha »» digital musical instrument »» TENORI-ON »» 16x16 matrix of LED switches allows everyone to play music intuitively, creating a "visible music" interface http://www.global.yamaha.com/tenori-on/what/index.html

»» Lemur, a multi-touch interface for creating real-time light shows and music mixes »» manipulate broad array of media in real time »» mimicking the analog experience of futzing with mixing boards and synthesizers »» clever musicians to become more like visual artists or stage directors, since the device can be linked up to light outputs—giving music a real time visual interpretation »» live performances at the hands of a single person http://www.jazzmutant.com/videos/lemurlight.mov

»» developed by Potion »»

The Jump Station for a special Nike exhibit at the 2008 Beijing Olympics

»» reactive piece that rewards physical performance with a larger-than-life payoff »» eight foot tall stick figure reacts to the person’s presence, begins warming up, and waits for the visitor’s next move »» a small jump is rewarded with only a small hop by the figure »» real effort is rewarded with a series of high flying animations like Michael Jordan style jump shots or acrobatic back flips http://www.potiondesign.com/index.php?page=2&project= 7§ion=0&gallery=0&listPage=0&capPage=0

Display

»» GE »» LED wallpaper »» roll-to-roll processing »» method of manufacturing organic LEDs (OLEDs) »» like a printing press »» not quite practical/costeffective presently »» tunable colour »» transparency (windows) http://dvice.com/archives/2008/04/ dvice_exclusive.php

»» programmable surfaces can be shaped and textured in more flexible ways than traditional LED displays »» modular tiles as building blocks »» displays can become an integral part of objects, structures and spaces. http://fluid.media.mit.edu/projects.php?action=details&id=23

»» MemTable »» MIT »» table with a contextual memory »» supports meeting annotation, process documentation, and visualization of group work patterns »» remember how it is used an provide an interface for contextual retrieval of information http://fluid.media.mit.edu/projects. php?action=details&id=71

Tangible Interfaces

»» DIS.PLAY »» group of German hackers »» Arduino and Processing! http://www.engadget.com/2009/02/24/foaminfused-impress-dis-play-encourages-touching/

»» TaPuMa; digital, tangible public map »» allows people to use their own belongings or the everyday objects they carry with them to access relevant, just-in-time information and to find locations of places or people »» table-top environment where map and dynamic content is projected on the table »» camera mounted above the table identifies and tracks the locations of the objects on the surface »» on the basis of identifications of the objects, the software program provides relevant information visualization to be shown on the table http://ambient.media.mit.edu/projects.php?action=details&id=53

»» Trackmate »» open source »» Trackmate Tracker allows any computer to recognize tagged objects and their corresponding position, rotation, and color information when placed on a surface »» Trackmate sends all object data via LusidOSC (a protocol layer for unique spatial input devices), allowing any LusidOSC-based application to work with the system http://trackmate.media.mit.edu/

»» SandScape »» MIT »» designing and understanding landscapes »» sand »» users view simulations as they are projected on the surface of sand model »» alter the form of the landscape model by manipulating sand while seeing the resultant effects of computational analysis generated and projected on the surface of sand in real-time http://tmg-video.media.mit.edu/sandscape/sandscape_352x240.mpg

Brain Machine Interface

»» Brain-Computer Interaction »» Emotiv Systems »» EPOC headset »» electroencephalography (EEG) technology »» measure the voltage produced by the combined activity of thousands of neurons in one area of the brain »» 16 electrodes can detect brain activity »» recognize emotions, commands, and facial expressions »» link them to operations or keystrokes in a wide variety of existing PC games and applications. »» emotion recognition can be used to continuously adjust the game’s environment »» i.e. the player is bored, the game can get harder; if the player gets excited, the background music can adjust accordingly www.emotiv.com http://seedmagazine.com/content/article/standing_on_the_shoulders_of_giants/

»» Toyota »» advanced brain sensing system that controls the movement of a wheelchair by reading a user’s thoughts alone »» processing patterns in brain waves »» the system can propel a wheelchair forward, as well as make turns »» virtually no discernable delay between thought and movement. »» return a response from a thought stimulus in just 125 milliseconds »» creating real-time responsiveness »» Five electroencephalography sensors stationed above the regions of the brain that deal with motor movement interpret patterns in the signals generated by the user »» Further, the software interpreting the signals adapts to a particular user’s patterns of thinking, achieving 95% accuracy after just one week of three-hour training sessions. »» Honda »» similar technology »» allows its Asimo robot to be manipulated via brain signals http://www.fastcompany.com/blog/clay-dillow/culture-buffet/mobility-mind-control-toyota-unveils-wheelchair-propelledthoughts-a

Questions based on a FastCompany article by David Sherwin

Can we listen & talk to the Internet? »»

“give me turn-by-turn directions to Pacific Place,”

Can a touch interface communicate through sense of touch?

»» Blind people should be able to ask their phone, “What’s the temperature going to be tomorrow?” and have the phone adjust its heat output in relation to today’s temperature to indicate the relative difference.

Can we feel directions?

»» I ask my phone (using my voice interface) how the traffic is on Road X. The steering wheel gets harder by 30%. Should I take Road Y then? The steering wheel softens dramatically. http://www.fastcompany.com/blog/david-sherwin/changeorder-business-process-design/move-beyond-gestural-interface-give-your-phon

http://www.fastcompany.com/blog/cliff-kuang/design-innovation/infographic-day-hype-cycle-computer-interface-design