Directed Study: Novel Input Devices

DIGITAL MEDIA DESIGN DIRECTED STUDY PAPER KAREN KIELY 0643564 NOVEL INPUT DEVICES

CONTENTS ●

●

LITERATURE (in order of discussion)

A Brief History of Human-Computer Interaction Technology

•

•

Of Mice and Menus: Designing the User-Friendly Interface

•

•

INTRODUCTION

Touch Sensing Input Devices

Teaching Old Mice New Tricks: Innovations in Computer Mouse Design

•

Touch&Type: A Novel Pointing Device for Notebook Computers

•

Soap: A Pointing Device That Works in Mid-Air

●

PERSONAL STATEMENT

●

REFERENCE LIST

INTRODUCTION From the list of available topics, I chose 'Novel Input Devices'. I chose this because I felt that I would find the field interesting. We all use input devices every time we access a computer or other device. We cannot carry out as many activities on a computer without using input devices like a mouse or a keyboard. Because of this, I decided to focus on the mouse as an input device. It is the most common and most recognised input device in use today. Concentrating on the 'novel' aspect of the title, I researched into new improvements and developments in the mouse as well as generally in human-computer interaction. This provided the basis of my project from which I could stray into other related topics like multi-touch.

LITERATURE A Brief History of Human-Computer Interaction Technology B.A. Myers Introduction This article by Brad Myers is one in which he discusses the numerous interactions, applications, software tools and up-and-coming areas of human-computer interaction. It stays true to the title and provides a concise history of HCI under a number of headings. HCI today is seen everywhere when we use computers and other technologies. Myers documents a topic which spans a huge area. The article is one which provides the reader with an abundance of information on the topic of HCI. The article's purpose is to change people's thinking that important HCI research takes place at industry level. Myers states many people believe that if HCI research in universities is not supported, industry will continue in its place. In reality, this is not true. In my own opinion, I found the article an interesting read which aptly documented the research carried out over the years in the field of human-computer interaction. Critique Myers begins this article by declaring that “all software written today employs user interface toolkits and interface builders”. He then begins to state that the article will focus on summarising the important research developments in HCI technology. I felt that this introduction was exactly what an introduction should be. It was clear, concise and not technically-written. It didn't contain many technical details and just summarised the purpose of the article. It was one of the best introductions I came across while reading each paper. Focusing on the section entitled 'Direct manipulation of graphical objects', I found this interesting although in the context of my entire research, it provided repetition. Here, Myers simply discussed the Sketchpad. However, this was only a fraction of the section, and Myers went on to focus on AMBIT/G and Light Handles. The descriptions of these programs didn't contain much detail so I was a little in the dark on how they worked etc. A little more attention to detail on Myers' part wouldn't have done amiss. Although, having said that, I did do some substantial research in the Sketchpad so that is the reason why I didn't understand much on the other programs. Myers' discussion of the mouse is not very significant in the context of the whole article. It is a tiny paragraph containing little information. The most interesting point I found was that the first commercial mouse appeared in 1981 as part of the Xerox Star. Having read many papers on Engelbart and others' work on the mouse, I felt as if the mouse would have appeared commercially earlier.

Myers continues in the same way into a section on windows, explaining when windows were developed and when they commercially appeared as well as developments from tiled to overlapping windows. Continueing in the same vein through the sections on drawing programs, text editing, spreadsheets, hyper-text and CAD, Myers chronicles the inventions, developments and commercial appearances of such programs and applications. These sections, including that on windows, although not essential to my understanding of the history of the mouse, provided some interesting reading. Rounding up the article with descriptions of up-and-coming areas, Myers writes about gesture recognition, three dimensionality and multimedia as well as many other aspects. These were highly interesting although at times tough to read. I feel this is down to the fact that the programs and applications discussed were new and unusual. I found it easy to read on topics like the mouse as it is such an everyday device that I understand whereas this was not so for these topics. Even so, when I could understand, I found these exciting new prospects for technology in the future and even in the present. In the discussion at the end of the article, Myers argues that industry will not be able to carry on research in the field and research in universities must carry on. He goes on to explain that “university, corporate and government- supported research continue and be well-supported”. I can see his point as he explains that without sufficient academic funding, there would be “fewer doctoral graduates” and in turn, less graduates to work in the top-notch research labs. Summary Having read this article and Myers' views on the research in HCI, I feel I know much more about the problems behind the research process. He makes it clear that people should realise that research is carried out in university more so than in industry. Although this article was not very helpful in my research of the mouse, it did provide interesting reading in the sections regarding different aspects of HCI. Myers', at most times, easyto-understand language was the article's high-point. However, in some places, although very minimal, the article was a bit technically detailed, I still found it interesting and somewhat easy-toread.

Of Mice and Menus: Designing the User-Friendly Interface T. S. Perry, J. Voelcker Introduction Perry and Voelcker discuss throughout this article the development of the mouse and other aspects of HCI like windows and icons. The article contains a detailed section on the mouse and documents Engelbart's development of the device. The development of the mouse after Engelbart by companies such as Xerox and Apple is also discussed. Windows, icons and windows are also discussed in detail. The main topic of the article is important as windows, icons and menus obviously provide a huge basis for human computer interaction today. The mouse is an almost essential input device whose development throughout history is documented in a detailed manner in this paper. It provides the reader with an insight into the history of things we use everyday. The article's detailed and interesting analysis of the mouse as an input device as well as components like windows appealed to me as it was easy-to-read and understand. It contained information on a very interesting topic and provided an insight into the humble beginnings of many of the components we take for granted today.

Critique The writers begin this article by explaining the time frame in which certain technologies were invented, researched, developed and distributed. Perry and Voelcker provide background information on Ivan E. Sutherland's Sketchpad program as an introductory section. This provides some detail on the beginnings of the GUI which I had already read about in Sutherland's thesis. They go on to highlight the differences between the light pens of the 1960's and mice, declaring that mice overcome some downfalls associated with the light pen. For example, the mouse is used on a physical desktop while a light pen is used by the user lifting the arm from the table. This got tiring after a certain time, so mice were seen as more advantageous. A list of definitions is included in the first page, containing the meanings of words such as bitmap, graphical user interface, raster display and clicking. The concepts of some of the first mice are discussed in detail, including descriptions of the mechanics and technology behind them. In different sections on buttons and balls, Perry and Voelcker analyse the workings of mice regarding these aspects. A pictorial flow chart chronicling companies such as Xerox and Apple's research into mice, windows, icons and menus is provided. Further discussions of windows, menus and icons are found in the article. The article is concluded by a paragraph on lawsuits to “establish who owns what”. The introduction to the article is highly detailed, and almost too much so for an introduction. An easy-to-read and generalised introduction would have appealed to me more. Having said that, the information provided is interesting. The writers explain how prototypes incorporating windows, mice, icons and menus were distributed in the 1970's, going into detail, explaining “more than 1200 Altos were built and tested” by Xerox. Definitive facts like this, in my own opinion, would prove more useful in another section of the paper. I feel the introduction should have been more simplistic. The section of the article entitled 'Mouse Tales' was, I found, a very useful section. First, highlighting the disadvantages with the light pen, the writers instilled the faith in the reader that the mouse is a more useful device than it predecessors. Reading further, the writers begin to describe the mechanics of the first mice. This was, at times, hard-to-read due to some technical terms used but was not a great problem. The attention-to-detail was definitely an advantage and not a downfall of the style of writing. The section 'Going On The Ball' further incorporated technical terms which at times were hard-to-understand. 'Button Wars' was an interesting section, condemning Apple's decision to produce a mouse with one single button. I found the analysis on user-based experiments on various versions of the mouse highly engrossing. The contrast put across by the writers between mice with varying numbers of buttons was interesting. Multiple button mice (generally mice with more than two buttons) were concluded as being confusing for a user. Despite the fact millions of Apple single button mice are/were in use, William English, a computer engineer who helped Douglas Engelbart develop the first mouse, is mentioned as having stated one button mice were 'a bad idea'. This analysis of buttons and the user opinions of them interested me as it had a human factor in it. The sections entitled 'Window with view', 'Pop, pull and tear' and 'Iconography' each documented early research and developments in windows, menus and icons, respectively. The windows section didn't appeal to me as much as the others. It simply documented how windows developed over the years, from Sketchpad to overlapping windows. I felt it didn't contain any exciting or unusual point, unlike in the other sections. 'Pop, pull and tear', I felt, contained some interesting points. Research in the field of drop-down and single bar menus is discussed. User experiments showed how users would benefit from having a single bar menu at the top of a window as opposed to the bottom. This is why tool bar menus are located at the top of a window today. A supplement to menus, which I found interesting to read about, is key combinations, a “return to the earliest of user interfaces”. 'Iconography' details the beginnings of the concept of the icon. It contained an interesting point which stood out for me. On the NeXT computer system, deleted

objects are placed in a Black Hole as opposed to a trash can on a Macintosh. William English argues that the Black Hole idea is a confusing concept which users who don't read science fiction would find unusual and hard-to-understand. Summary Having read this article, I felt as if it contained useful information on different aspects of human-computer interaction. Perry and Voelcker focused on well-chosen topics as we all make use of mice, windows, menus and icons today. At times, the article was very technically written and in places was hard-to-understand. I had to re-read at some points. However, the exciting sections in the paper and the points made for great reading. I felt the points made on the user experience of different interfaces appealed to me most. I enjoyed learning about what users found easy-to-use and vice versa. These were some of the most interesting points I found in the paper. Overall, it was a well-written papers with an exciting focus, containing information on a widely-known topic.

Teaching Old Mice New Tricks: Innovations in Computer Mouse Designs Shumin Zhai & I. Scott MacKenzie Introduction This article is a short and simple one where Zhai and MacKenzie discuss the mouse, its human factors and some new “tricks” that are being incorporated into new models of mice. The topic of this article, namely the development of the mouse in recent years and in years to come, is a very interesting and important one. The writers aptly describe the new trends in mouse design technology. The three “tricks” they document seem like interesting slants on the technology of the mouse today. The simplicity of the article and its language help portray the new “tricks” in an appealing manner. I enjoyed the article as it was easy-to-read and had an exciting content. Critique This article is introduced with some fascinating statements which set the tone of the paper. Zhai and MacKenzie state that the mouse is one of the most frequently handled devices in many people's lives today. They also use interesting points to put across the message that the mouse is a large part of modern society, such as one which tells us that the term 'mouse' is now accepted by reputable dictionaries. They then go on to discuss the early history of the mouse which I had already researched in a number of articles. In the section on “Human Factors of The Mouse”, the writers argue that the mouse is superior to other input devices like touch pads, trackballs and pointing sticks. Dividing this section into four parts, they discuss in equal measure the 'Easy Control Mode', 'Button Integration', 'Flexibility of Using Different Muscle Groups' and 'Easy Device (Re)acquisition' of the mouse. These separate sections contain interesting facts on why the mouse is so useful so long after its invention. Following a short paragraph on 'Models for Mouse Research and Evaluation', Zhai and MacKenzie discuss the three new trends for mice: 'The Third Dimension and Six Degrees of Freedom', 'Making a Mouse Feel Able' and 'Second Stream of Input for Zooming and Scrolling'. The conclusion ends the article with a statement that it is possible to “teach old mice new tricks”. Personally, I found the article very intriguing. The style of writing of the entire piece appealed to me, more so than with any of the other articles I studied. In the section on human factors, the writers argue that the mouse often outperforms other devices when used in experiments

for pointing purposes. They put across clearly the point that the mouse is an almost indispensable input device that is so widely used that other devices pale in comparison. They continue to explain the downfalls of other devices like trackballs. They make clear the point that dragging and dropping is a difficult procedure with this device but not so with the mouse. This point interested me as it showed that the mouse is certainly a much superior device than others on the market today, despite its being on the market for more than twenty years now. In the section on new tricks of the mouse, the designs interested me. The different ideas were innovative and seemed exciting. 'Making a Mouse 'Feel Able'' is an interesting paragraph describing how classes of feel may be incorporated into a mouse. Vibration and 'force feedback' can be incorporated and the writers describe one such design developed by Immersion Comp. This section was exciting and although short and not very detailed, still put across the idea in an effective manner. The writers then go on to discuss new methods for zooming and scrolling. They argue that, perhaps, two-handed input would remedy the problem a user would have of having to switch between scrolling and pointing. They then tell us that of all the new mouse 'tricks' mentioned, scrolling is probably the most useful. I agreed with this as the paragraph on scrolling was the longest and contained the most detail, which obviously shows that this is an important 'trick'. As well as this, I can see the problem with scrolling and pointing more so than the problems put across in the other paragraphs. In the concluding section, Zhai and MacKenzie round up the article simply and efficiently, putting across appropriate points which easily portray the points put across in the paper. Summary This was one of my favourite articles I read on the topic of input devices. It was extremely easy-to-understand. The content was interesting as it described new innovations in mouse design. It was exciting and the writers put across all their points well.

Touch&Type: A Novel Pointing Device For Notebook Computers W. Fallot-Burghardt, M. Fjeld, C. Speirs, S. Ziegenspeck, H. Kreuger, T. Läubli Introduction The Touch&Type device is one which, again, incorporates touch-sensing into an existing device. Here, the keyboard is being used as a touch pad as well as a device on which to type. This article describes a user-study using a prototype device There is also a technically detailed article available. The user-centred process of testing and prototyping and the subsequent description which is detailed in this article is important to my understanding of touch-sensitive input devices. It gives the reader an insight into how a potential user perceives a new product, especially since similar products have already been on the market for twenty or thirty years. I found this article very interesting as the concept of a keyboard with touch-sensors integrated into it seems very exciting. The paper was very simplistic in language and easy-to-read. Critique The article begins with an interesting statement, declaring that computer users today are reluctant to use alternative, built-in pointing devices due to the widespread use of the mouse. The concept of Touch&Type, the writers go on to say, is that it is a novel input device which, they consider, is superior than any conventional pointing device. The study described in the paper is a comparison between the Touch&Type device, a mouse and a conventional touch pad.

Throughout the introduction of the article, the writers put across points highlighting the ineffectiveness of touch pads, pointing sticks and mice with keyboards. They make valid and interesting points which provide a sturdy base on which to develop the concept and research of this new input device. In the 'Touch&Type Approach' the writers describe the layout of the Touch&Type device, creating an image for the reader with each detail mentioned. They proceed to explain the expected advantages of the device. Moving on to the 'User Study' section, the writers inform the reader how they went about examining the performance of the three devices (mouse and keyboard, touch pad and Touch&Type) and the kind of tasks used in the examination. A series of graphs and charts document the results. Two lists show what the users found to be advantages and disadvantages of the device. The article finishes up with a summary and outlook where the writers explain the shortcomings of the device as well as documenting the aspects in which it outshone the other devices. Personally, I found the abstract and introduction to this article very interesting and informative, giving me a detailed foundation before delving into the concept of Touch&Type. The abstract, using simple language and describing effectively, gave a great insight into the basics of the device. The introduction, showing the disadvantages to existing input devices, gave detailed explanations regarding the other devices. I knew then, after reading such a clear and concise introduction and proceeding to read further, the downfalls of existing devices and that this new concept will be superior. The writers go on to discuss the Touch&Type device in a detailed manner. The attention to detail yet simplicity of language was one aspect of the text I found particularly effective. The expected advantages of the device are discussed, with an explanatory table accompanying. This makes the text all the more clearer to the reader, which I felt was very useful. The 'User Study' section appeared to be the core of the article, describing the writers' experiments with prototype devices. The results section, complete with tables and graphs, was clear and concise. I found this section helpful in my research and reading as the writers accurately and simply put across the results of the experiment in an easy-to-read fashion. In the summary, the writers argue that one of the shortcomings in the user experience results is down to the fact that users have much more prior experience with using mice, keyboards and touch pads. The linkage between the summary and the introduction is seen here. The discussion at the beginning of the article on the disadvantages of existing pointing devices mirrors this analysis that these same pointing devices provided a shortcoming in the experiment. This brings the article together and links the beginning and the end. Summary Overall, I found this article on the Touch&Type system highly interesting and exciting. It's a new slant on touch-sensing and input devices. Although the mouse will stay in prominent use due to its reliability and popularity, Touch&Type seems an exciting new prospect that, should it be developed and mass produced, would aid easy integration between man and machine. The article, overall, was produced in a very effective manner which I found very interesting and fascinating.

Soap: A Pointing Device That Works in Mid-Air P. Baudisch, M. Sinclair, A. Wilson Introduction This article on the pointing device 'Soap' is one which describes and demonstrates the actual device. The writers explain the concept behind the device and include lots of images to portray the exact concept of the device. They put across the advantages of Soap efficiently and well. The importance of this paper and the device on which it is based is very significant.

Researchers seem to always be trying to find ways of incorporating new methodologies into existing devices. This article is one which grabbed my attention as the concept of having a mouse working in mid-air is very innovative and exciting. There would be no need for a physical desktop nearby. I found the article very exciting and different. The writers put across their points concisely so the reader fully understands the workings of the device and the experiments carried out during its development. Critique The paper's abstract and introduction, well written and concise, put across the main points that provide the fundamentals for understanding the topic. The writers explain how other input devices, such as joystick-based presenter tools and game controllers, can be utilised in mid-air whereas the mouse can't, thus making a mouse “half an input device”. A mouse is unusable without a desk or mouse pad. The writers then go on to explain the device in a paragraph entitled 'Soap'. They explain the diagrams that accompany the paper. The device consists of two parts, the hull and the core. The core contains an optical sensor, like in a normal optical mouse. The hull surrounds it. Any relative motion between the hull and the core is picked up by the device and translated into on-screen movement. They argue that the device is more suited to interacting with large screens as it can produce very large motions and are “useful for getting across longer distances”. The writers then go on to describe the various user tests they put Soap up to. These were 'Controlling Microsoft Windows on a wall display', 'Soap as a presenter tool' and 'Playing first person shooter games'. The article is concluded with a paragraph on related work. This article, in my opinion, was one of the more exciting and interesting. Its concise documentation of the workings of the device appealed to me. It was easy to understand and comprehend, despite the fact I have never seen or used Soap. This paper was one of the more easy-to-understand papers and I never found myself struggling to understand what the writers were trying to put across to the reader. The concept of the device is fairly comprehendable and it seems like an easy device to manipulate and utilise. I really liked the idea of a pointing device that works in mid-air. This would be down to one reason and that is that it would be such a different experience to using a normal mouse or touch pad. Having said that, refering back to the paper on CHECK PAPER! It is obvious that the user's hand would get tired and/or sore after using such a device. For that reason, I believe that the device is not as useful as it may seem. However, the information put across by the writers on the usage of Soap as a pointing device for wall display or as a presenter tool is certainly valid. For these situations, personally, I believe that Soap would benefit the user. One strong point of this article is the use of images and diagrams. Putting such helpful and relevant images in the article helps the reader's understanding of the device a great deal. Personally, I found this to be the case. I understood the concept of Soap a little easier than if the article had no diagrams or images. Images of the user-experiments appealed to me most. These showed the device being used and provided some sort of reiteration that the device is effective and helpful in such cases as presenting and gaming. Summary To conclude, I found this article on Soap most interesting. The article content appealed to me as it is an exciting and innovative development in the field of HCI and input devices. I don't see Soap being used as a commercial product soon or if it is, it may only benefit people who will use it as a presenting tool or gaming controller. A similar device can already be seen in the game controller of the Nintendo Wii. Perhaps this is the beginnings of such devices being used by the consumer. Overall, the article was well written and easy-to-understand. The content was exciting and the images helped the article a great deal.

Personal Statement on the Topic As I have explained in my introduction, I chose the topic of novel input devices because input devices are widely used today. I felt the topic would be interesting, as the word 'novel' would allow me to research into new and exciting ways of using input devices. I decided to focus on the mouse as it is such a popular and indispensable input device. Novel ways of using the mouse interested me a great deal and I felt that researching into new innovations in the mouse would be a very exciting prospect. My aims beginning the project were to learn more about human-computer interaction and novel input devices as well as the relationship between touch sensing and existing input devices and how touch sensing can improve a regular input device like the mouse. I went about my research by using both the internet and the library. I made great use of websites such as Google Scholar,. ACM Digital Library and IEE Electronic Library. I found the university library quite poorly stocked for my topic of choice. I found only one book on the topic of input devices and at that, it was not a very helpful one. I choose to omit it from the documentation of my research in this paper. Having read a number of papers covering topics as diverse as history of HCI and new mouse models, I feel like I have learned a lot about human-computer interaction, among other things. Touch sensing was another slant on the topic I took and the papers I read on this were particularly interesting. The papers I found most useful in my research were: • • •

Soap: A Pointing Device That Works In Mid-Air Teaching Old Mice New Tricks: New Innovations in Computer Mouse Design Touch&Type: A Novel Pointing Device for Notebook Computers

These were the papers that I understood mostly. I also found them highly interesting. Their content was easy-to-understand. The papers were well written. The topic of each appealed to me, especially with Soap and Touch&Type as these were new mouse designs which I hadn't ever heard of or used before. As for the actual general topic of my study, personally I found the topic highly interesting. The studies I found relating to the topic of novel input devices were plentiful and the ones into which I choose to go into more detail were exciting and great to read. Studying the history of HCI and the work of Douglas Engelbart, I have a great respect for him as, essentially, the inventor of the mouse. I read and agreed with a point made in one paper that he is not appropriately rewarded as an inventor as not many people know his name. I feel novel input devices and in particular new mouse models have a place in technology in the future. Although the mouse may never be replaced, at least in the near future, new innovations in its design tell me that perhaps one day novel mouse designs may appear on the commercial market. In particular, I found the Touch&Type device very exciting and maybe more commercialfriendly than Soap, for example. It's only a matter of time before researchers in this field develop a new device that is worthy enough of replacing the mouse. Having said that, I feel this new device will still be based on the mouse. It may only be a simple mouse with a new tweak in its design to make it more efficient for the user of the future. To conclude, I'd like to say I enjoyed researching into the mouse's design and its history and predecessors. It was an exciting topic and I'm glad I chose it.

REFERENCE LIST Baudisch, P., Sinclair, M., Wilson, A. (2006) 'Soap: A Pointing Device That Works in Mid-Air', Symposium on User Interface Software and Technology, October 2006, 43-46 Fallot-Burghardt, W., Fjeld, M., Speirs, C., Ziegenspeck, S., Kreuger, H., Läubli, T. (2006) 'Touch&Type: A Novel Pointing Device For Notebook Computers', ACM International Proceeding Series vol. 189, 465-468 Myers, B. A. (1998) 'A Brief History of Human Computer Interaction Technology.' ACM interactions, 5(2), 44-54 Perry, T. S., Voelcker, J. (1989) 'Of Mice and Menus', PC/Computing, 2(4), 126-132 Zhai, S., & MacKenzie, I. S. (1998) 'Teaching old mice new tricks: Innovations in computer mouse design' Proceedings of Ergon-Axia '98 - the First World Congress on Ergonomics for Global Quality and Productivity, 80-83.