Bjoern's Phd Defense Talk Slides
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Bjoern's Phd Defense Talk Slides as PDF for free.
More details
- Words: 1,514
- Pages: 106
stanford hci group
5/29/2009
Enlightened Trial and Error Gaining Design Insight Through Interaction Prototyping Tools
Björn Hartmann [email protected]
Prototyping Tools for Ubiquitous Computing
1
Enable Rapid Authoring Off the Desktop
2
Aid Iteration by Managing Feedback
3
Aid Exploration of Multiple Alternatives
2
Lawson, How Designers Think, Architectural Press Cross, Designerly Ways of Knowing, Springer Schrage, Serious Play, HBS Press Moggridge, Designing Interactions, MIT Press Buxton, Sketching User Experiences, Morgan Kaufmann 3
Exploration: Generate Parallel Alternatives
Redrawn from Buxton, Sketching User Experiences 4
Iteration: Incorporate Insight ANALYZE
EVALUATE
Dreyfuss, Designing for People, 1955; Lawson, How Designers Think, 1997; Cross, Designerly Ways of Knowing, 2005
PROTOTYPE
5
Prototypes for the Microsoft mouse From Moggridge, Designing Interactions, Ch2
7
8
Prototyping != Development Success criteria for interaction prototypes Rapid to construct Conveys user experience Uncovers design space Elicits the right feedback
(Some) success criteria for interactive products Robust Complete Polished
9
1
Enable Rapid Authoring Off the Desktop
2
Aid Iteration by Managing Feedback
3
Aid Exploration of Multiple Alternatives
10
Atkinson, Hypercard, 1985
Denim. Lin et al. CHI 2000 Also: SILK, Landay, 1996 DEMAIS, Bailey, 2001 11
© Chumby Industries © Medtronic
© Garmin
© Bayer
© Numark
© Nintendo
12
Phidgets, Greenberg, 2001
Bug Labs, 2009 Also: iStuff, Ballagas, 2003 Calder, Lee, 2004 DART, McIntyre, 2004
13
Fieldwork (11 designers, 3 companies)
Comprehensive prototypes are only built for presentation, not for design thinking.
14
1.
Leverage familiarity with storyboards: use states1 as level of abstraction 2. Tightly integrate of physical and digital design realms 3. Visualize hardware configuration & events in software, in real-time
d.tools
1: Harel,D. Statecharts: A visual formalism for complex systems. Science of computer programming (1987)
If |Buttons| < |Letters|: 1.
Select alphabet subset
2. Select letter from subset
Available Controls: 1.
X/Y Tilt data
2.
Discrete corner buttons
TiltType, Partridge et al., UIST 2002
16
Example: Tilt-based text entry 3-way navigation switches
2 axis accelerometer
17
18
19
Design Environment
GUI Editor
Design Environment Device Designer
Smart Hardware
Design Environment
Storyboard Editor
Accelerometers
IR/Ultrasonic Rangers
Force Sensitive Resistors
Phototransistors
24
Accelerometer X axis
//detect accelerometer peaks //read data sample xVal[t++]=readA2DValue(xPin);
Accelerometer Y axis
//look for changes in derivative if(((xVal[t]-xVal[t-1]) >= 0 && (xVal[t-1]-xVal[t-2]) < 0) || (((xVal[t]-xVal[t-1]) < 0 && (xVal[t-1]-xVal[t-2]) >= 0)) { //peak detected //send message oscSendMessageInt("/x/peak",1); } else { //no peak }
25
ML algorithms enable demonstration-based authoring. HCI: What are appropriate interaction techniques that make real-time demonstration and testing feasible for designers? Insight: Combine recognition and direct manipulation to leverage the designer’s knowledge 26
Authoring by Demonstration
[chi 2007] 27
28
29
30
Dynamic Time Warping (DTW) Demonstration Signal
Matching Input Signal
int DTWDistance(char s[1..n], char t[1..m], int d[1..n,1..m]) { declare int DTW[0..n,0..m] declare int i, j, cost for i := 1 to m DTW[0,i] := infinity for i := 1 to n DTW[i,0] := infinity DTW[0,0] := 0 for i := 1 to n for j := 1 to m cost := d[s[i],t[j]] DTW[i,j] := cost + minimum(DTW[ i-1, j ], DTW[ i , j-1 ], DTW[ i-1, j-1 ]) return DTW[n,m] }
Quadratic in time & space But: n is small @100Hz sampling
// insertion // deletion // match
[Sakoe, Chiba, ‘78]
31
32
What?
Why?
Usability Inspection
Analysis of d.tools as a visual notation
First-Use Laboratory Studies
Threshold and usability
Class & Industry Deployment
Real-world stress test
Used by authors as design tools for public exhibitions
Complexity ceiling for knowledgeable users
33
Study 1 Tool: d.tools Task: Build functional MP3 player prototype Time: 45 minutes N=10
Study 2 Tool: Exemplar Task: Create sensor-based controls for a provided game Time: 30 minutes N=12
Participants: graduate students in design programs + alumni
34
D.tools evaluation video analysis Task: Build a functional MP3 player prototype
D.tools post-test survey (n=10)
Class Deployment (In collaboration with Leapfrog)
39
d.tools / Exemplar used for prototyping at: Leapfrog Nokia SF IDEO Frog design
3471 d.tools downloads as of 5/27/09
40
d.tools successfully enabled non-programmers to prototype novel, sensor-based user interfaces. Techniques: 1. Plug&play hardware and visual, storyboard-based environment offer low threshold to authoring. 2. Programming by demonstration for sensor events raises complexity of possible interactions.
41
1
Enable Rapid Authoring Off the Desktop
2
Aid Iteration by Managing Feedback
3
Aid Exploration of Multiple Alternatives
42
43
Flickr Image (cc) brycej
I-Observe, Badre, 1995 EVA, Mackay, 1992
44
46
50
53
(Pixar Image from Buxton, Sketching User Experiences)
Movie production
Programming
54
Additions
Deletions
Sketched Graphics
Freeform Comments 55
56
57
12 participants
12 annotated images of d.tools diagrams
2 prototypes (camera, photo viewer) 12 d.tools diagrams revised with d.note
8 new participants
List of clear revisions
List of revisions requiring clarification
58
Making revisions Reported benefits: can make and test concrete changes Led to more careful suggestions
Interpreting revisions Fewer clarifications required for d.note diagrams (mean: 1.6 vs. 2.9 requests, p < 0.05)
(Deletions only prevalent with d.note)
59
Integrating analysis of test data into a design tool can radically shorten the time to work with test videos. Integration is enabled by a high-level visual representation of the software model. Interactive revision tools enable earlier testing of proposed changes, and those changes require fewer clarifications.
60
1
Enable Rapid Authoring Off the Desktop
2
Aid Iteration by Managing Feedback
3
Aid Exploration of Multiple Alternatives
61
Myers et al., VL/HCC 2008: Comparing working UI alternatives “very important” or “crucial” for 43% of surveyed interaction designers (N=210)
62
Design Galleries, Marks, 1997
Levoy, Spreadsheets for Images, 1994
Subjunctive Interfaces, Lunzer, TOCHI 08 Spreadsheets for Visualization, Chi, IEEE CGA 98 Parameter Spectrums & Parallel Pies, Terry, UIST 02 & CHI 04 Partials, Terry, Phd 2005
63
Flash
Arduino / Processing
Dreamweaver 64
Manage parameter variations Manage code alternatives Access variations & alternatives at runtime
65
Manage parameter variations Manage code alternatives Access variations & alternatives at runtime
Processing code brought in by an interviewee
66
Manage parameter variations Manage code alternatives Access variations & alternatives at runtime
67
68
69
70
71
Juxtapose editor for ActionScript 2. Generates Flash files.
72
73
74
Implementation: Longest common subsequence algorithm diff(A,B,…,N) [Toomim, VL/HCC 2004] 75
Harrison et al., UIST 2007
Progress bar test application Progress profiles
76
77
78
79
Baudisch, UIST 2006
80
81
Send variable info Juxtapose User Library
A X
B
Juxtapose Process
Inspect
User’s Application: //… Number A = 10; Boolean B = true;
User Process 82
Send tuning message Juxtapose User Library
A X
B
Juxtapose Process
Update
User’s Application: //… Number A = 10; Boolean B = true;
User Process 83
Search in 4D parameter space. Given recursive drawing code for this:
Produce these:
N=18; undergraduate + graduate HCI students 84
85
Mean Parameter Changes Tested per Minute 90 64.26
80
Changes/minute
70 60 50
Control Juxtapose
40 30 20 10
2.60
0
Tree Matching Task: Mean Completion Times by Tree 400 350 seconds
300 250 200
Control Juxtapose
150 100 50 0 Tree 1
not significant
Tree 2
not significant
Tree 3
p<0.01
Tree 4
p~0.01
88
89
Alternative Alternative Alternative 1 2 3
90
91
Arduino: 8bit Atmel AVR RISC chip programmed in C with gcc 92
How can we implement variable tuning in a static language without reflection?
93
int dly = 25; boolean blink=false; void setup() {...} //…
Var. Name “dly” “blink”
Type int boolean
Pointer &dly &blink
94
Auto-generated table
void initVarTable(void) { varTable[1].varName = PSTR("dly"); varTable[1].varType = VAR_TYPE_INT; varTable[1].varPtr = &dly; varTable[2].varName = PSTR("blink"); varTable[2].varType = VAR_TYPE_BOOLEAN; varTable[2].varPtr = &blink; } // plus a bunch of communication code... 95
Plans to integrate variable tuning into Arduino IDE
96
97
98
A structured approach to alternatives eliminates cruft and enables more exploration. Tool support requires connecting authoring and execution environments. 3 Techniques: Linked editing to manage code alternatives Execute set of programs side-by-side Auto-generate tuning interface
99
1
Enable Rapid Authoring Off the Desktop
Interaction techniques for prototyping physical user interfaces: Tight coupling between physical & digital realm through plug-and-play hardware, software models of hardware configuration Storyboard-based visual authoring with extensibility through scripting Demonstration-based definition of interaction events from raw sensor data 101
2
Aid Iteration by Managing Feedback
Integration of design, test, analysis and revision stages: Time code correlation between video and state model enables rapid review of usability videos Interactive revision tools for d.tools control flow diagrams enable testing of proposed changes and require fewer clarifications.
102
3
Aid Exploration of Multiple Alternatives
Techniques for authoring and running multiple UI alternatives in parallel: Selectively parallel source editing though and execution Automatic generation of tuning interface from source analysis & runtime reflection
103
104
stanford hci group
http://bjoern.org Funding Acknowledgments: Stanford MediaX/DNP Grant; NSF grant IIS-0534662 Stanford School of Engineerring & SAP Graduate Fellowships Equipment donations from Intel & Nokia
5/29/2009
5/29/2009
Enlightened Trial and Error Gaining Design Insight Through Interaction Prototyping Tools
Björn Hartmann [email protected]
Prototyping Tools for Ubiquitous Computing
1
Enable Rapid Authoring Off the Desktop
2
Aid Iteration by Managing Feedback
3
Aid Exploration of Multiple Alternatives
2
Lawson, How Designers Think, Architectural Press Cross, Designerly Ways of Knowing, Springer Schrage, Serious Play, HBS Press Moggridge, Designing Interactions, MIT Press Buxton, Sketching User Experiences, Morgan Kaufmann 3
Exploration: Generate Parallel Alternatives
Redrawn from Buxton, Sketching User Experiences 4
Iteration: Incorporate Insight ANALYZE
EVALUATE
Dreyfuss, Designing for People, 1955; Lawson, How Designers Think, 1997; Cross, Designerly Ways of Knowing, 2005
PROTOTYPE
5
Prototypes for the Microsoft mouse From Moggridge, Designing Interactions, Ch2
7
8
Prototyping != Development Success criteria for interaction prototypes Rapid to construct Conveys user experience Uncovers design space Elicits the right feedback
(Some) success criteria for interactive products Robust Complete Polished
9
1
Enable Rapid Authoring Off the Desktop
2
Aid Iteration by Managing Feedback
3
Aid Exploration of Multiple Alternatives
10
Atkinson, Hypercard, 1985
Denim. Lin et al. CHI 2000 Also: SILK, Landay, 1996 DEMAIS, Bailey, 2001 11
© Chumby Industries © Medtronic
© Garmin
© Bayer
© Numark
© Nintendo
12
Phidgets, Greenberg, 2001
Bug Labs, 2009 Also: iStuff, Ballagas, 2003 Calder, Lee, 2004 DART, McIntyre, 2004
13
Fieldwork (11 designers, 3 companies)
Comprehensive prototypes are only built for presentation, not for design thinking.
14
1.
Leverage familiarity with storyboards: use states1 as level of abstraction 2. Tightly integrate of physical and digital design realms 3. Visualize hardware configuration & events in software, in real-time
d.tools
1: Harel,D. Statecharts: A visual formalism for complex systems. Science of computer programming (1987)
If |Buttons| < |Letters|: 1.
Select alphabet subset
2. Select letter from subset
Available Controls: 1.
X/Y Tilt data
2.
Discrete corner buttons
TiltType, Partridge et al., UIST 2002
16
Example: Tilt-based text entry 3-way navigation switches
2 axis accelerometer
17
18
19
Design Environment
GUI Editor
Design Environment Device Designer
Smart Hardware
Design Environment
Storyboard Editor
Accelerometers
IR/Ultrasonic Rangers
Force Sensitive Resistors
Phototransistors
24
Accelerometer X axis
//detect accelerometer peaks //read data sample xVal[t++]=readA2DValue(xPin);
Accelerometer Y axis
//look for changes in derivative if(((xVal[t]-xVal[t-1]) >= 0 && (xVal[t-1]-xVal[t-2]) < 0) || (((xVal[t]-xVal[t-1]) < 0 && (xVal[t-1]-xVal[t-2]) >= 0)) { //peak detected //send message oscSendMessageInt("/x/peak",1); } else { //no peak }
25
ML algorithms enable demonstration-based authoring. HCI: What are appropriate interaction techniques that make real-time demonstration and testing feasible for designers? Insight: Combine recognition and direct manipulation to leverage the designer’s knowledge 26
Authoring by Demonstration
[chi 2007] 27
28
29
30
Dynamic Time Warping (DTW) Demonstration Signal
Matching Input Signal
int DTWDistance(char s[1..n], char t[1..m], int d[1..n,1..m]) { declare int DTW[0..n,0..m] declare int i, j, cost for i := 1 to m DTW[0,i] := infinity for i := 1 to n DTW[i,0] := infinity DTW[0,0] := 0 for i := 1 to n for j := 1 to m cost := d[s[i],t[j]] DTW[i,j] := cost + minimum(DTW[ i-1, j ], DTW[ i , j-1 ], DTW[ i-1, j-1 ]) return DTW[n,m] }
Quadratic in time & space But: n is small @100Hz sampling
// insertion // deletion // match
[Sakoe, Chiba, ‘78]
31
32
What?
Why?
Usability Inspection
Analysis of d.tools as a visual notation
First-Use Laboratory Studies
Threshold and usability
Class & Industry Deployment
Real-world stress test
Used by authors as design tools for public exhibitions
Complexity ceiling for knowledgeable users
33
Study 1 Tool: d.tools Task: Build functional MP3 player prototype Time: 45 minutes N=10
Study 2 Tool: Exemplar Task: Create sensor-based controls for a provided game Time: 30 minutes N=12
Participants: graduate students in design programs + alumni
34
D.tools evaluation video analysis Task: Build a functional MP3 player prototype
D.tools post-test survey (n=10)
Class Deployment (In collaboration with Leapfrog)
39
d.tools / Exemplar used for prototyping at: Leapfrog Nokia SF IDEO Frog design
3471 d.tools downloads as of 5/27/09
40
d.tools successfully enabled non-programmers to prototype novel, sensor-based user interfaces. Techniques: 1. Plug&play hardware and visual, storyboard-based environment offer low threshold to authoring. 2. Programming by demonstration for sensor events raises complexity of possible interactions.
41
1
Enable Rapid Authoring Off the Desktop
2
Aid Iteration by Managing Feedback
3
Aid Exploration of Multiple Alternatives
42
43
Flickr Image (cc) brycej
I-Observe, Badre, 1995 EVA, Mackay, 1992
44
46
50
53
(Pixar Image from Buxton, Sketching User Experiences)
Movie production
Programming
54
Additions
Deletions
Sketched Graphics
Freeform Comments 55
56
57
12 participants
12 annotated images of d.tools diagrams
2 prototypes (camera, photo viewer) 12 d.tools diagrams revised with d.note
8 new participants
List of clear revisions
List of revisions requiring clarification
58
Making revisions Reported benefits: can make and test concrete changes Led to more careful suggestions
Interpreting revisions Fewer clarifications required for d.note diagrams (mean: 1.6 vs. 2.9 requests, p < 0.05)
(Deletions only prevalent with d.note)
59
Integrating analysis of test data into a design tool can radically shorten the time to work with test videos. Integration is enabled by a high-level visual representation of the software model. Interactive revision tools enable earlier testing of proposed changes, and those changes require fewer clarifications.
60
1
Enable Rapid Authoring Off the Desktop
2
Aid Iteration by Managing Feedback
3
Aid Exploration of Multiple Alternatives
61
Myers et al., VL/HCC 2008: Comparing working UI alternatives “very important” or “crucial” for 43% of surveyed interaction designers (N=210)
62
Design Galleries, Marks, 1997
Levoy, Spreadsheets for Images, 1994
Subjunctive Interfaces, Lunzer, TOCHI 08 Spreadsheets for Visualization, Chi, IEEE CGA 98 Parameter Spectrums & Parallel Pies, Terry, UIST 02 & CHI 04 Partials, Terry, Phd 2005
63
Flash
Arduino / Processing
Dreamweaver 64
Manage parameter variations Manage code alternatives Access variations & alternatives at runtime
65
Manage parameter variations Manage code alternatives Access variations & alternatives at runtime
Processing code brought in by an interviewee
66
Manage parameter variations Manage code alternatives Access variations & alternatives at runtime
67
68
69
70
71
Juxtapose editor for ActionScript 2. Generates Flash files.
72
73
74
Implementation: Longest common subsequence algorithm diff(A,B,…,N) [Toomim, VL/HCC 2004] 75
Harrison et al., UIST 2007
Progress bar test application Progress profiles
76
77
78
79
Baudisch, UIST 2006
80
81
Send variable info Juxtapose User Library
A X
B
Juxtapose Process
Inspect
User’s Application: //… Number A = 10; Boolean B = true;
User Process 82
Send tuning message Juxtapose User Library
A X
B
Juxtapose Process
Update
User’s Application: //… Number A = 10; Boolean B = true;
User Process 83
Search in 4D parameter space. Given recursive drawing code for this:
Produce these:
N=18; undergraduate + graduate HCI students 84
85
Mean Parameter Changes Tested per Minute 90 64.26
80
Changes/minute
70 60 50
Control Juxtapose
40 30 20 10
2.60
0
Tree Matching Task: Mean Completion Times by Tree 400 350 seconds
300 250 200
Control Juxtapose
150 100 50 0 Tree 1
not significant
Tree 2
not significant
Tree 3
p<0.01
Tree 4
p~0.01
88
89
Alternative Alternative Alternative 1 2 3
90
91
Arduino: 8bit Atmel AVR RISC chip programmed in C with gcc 92
How can we implement variable tuning in a static language without reflection?
93
int dly = 25; boolean blink=false; void setup() {...} //…
Var. Name “dly” “blink”
Type int boolean
Pointer &dly &blink
94
Auto-generated table
void initVarTable(void) { varTable[1].varName = PSTR("dly"); varTable[1].varType = VAR_TYPE_INT; varTable[1].varPtr = &dly; varTable[2].varName = PSTR("blink"); varTable[2].varType = VAR_TYPE_BOOLEAN; varTable[2].varPtr = &blink; } // plus a bunch of communication code... 95
Plans to integrate variable tuning into Arduino IDE
96
97
98
A structured approach to alternatives eliminates cruft and enables more exploration. Tool support requires connecting authoring and execution environments. 3 Techniques: Linked editing to manage code alternatives Execute set of programs side-by-side Auto-generate tuning interface
99
1
Enable Rapid Authoring Off the Desktop
Interaction techniques for prototyping physical user interfaces: Tight coupling between physical & digital realm through plug-and-play hardware, software models of hardware configuration Storyboard-based visual authoring with extensibility through scripting Demonstration-based definition of interaction events from raw sensor data 101
2
Aid Iteration by Managing Feedback
Integration of design, test, analysis and revision stages: Time code correlation between video and state model enables rapid review of usability videos Interactive revision tools for d.tools control flow diagrams enable testing of proposed changes and require fewer clarifications.
102
3
Aid Exploration of Multiple Alternatives
Techniques for authoring and running multiple UI alternatives in parallel: Selectively parallel source editing though and execution Automatic generation of tuning interface from source analysis & runtime reflection
103
104
stanford hci group
http://bjoern.org Funding Acknowledgments: Stanford MediaX/DNP Grant; NSF grant IIS-0534662 Stanford School of Engineerring & SAP Graduate Fellowships Equipment donations from Intel & Nokia
5/29/2009
Related Documents
Bjoern's Phd Defense Talk Slides
May 2020 3
Giving A Talk Slides
November 2019 17
Halls Talk Sept 2006 Slides
August 2019 28
Phd
November 2019 33
Phd
November 2019 36