Paper Prototyping

Scenario Tasks Task 1: Answer the following question in the quiz, as shown on the slide. Task 2: Indicate that you wish to answer the question that your professor has posted verbally to the entire lecture. Task 3: Indicate that you have queries/questions about a particular slide that has passed, while the lecture is still in progress.

Observations Scenario Task 1: Our first user, Hei Wai performed the task smoothly. She pressed the button ‘5’ as the answer to the quiz question as shown in Figure 1.4. She briefly mentioned that this means she answered the question. She took a few seconds to realize the change in feedback light (green to blue) when we showed her how the clicker would respond to her action. The sketches flashed are as shown in Figure 1.2 and 1.3. The delay in response time is due to the limitations of paper prototyping which is the lack of colour. She had also mentioned that she thinks this change of colour in the feedback light means her response is accepted. Our second user, Sheng Rong also performed the task smoothly. She pressed the button ‘2’ as the answer to the quiz question in Figure 1.4. She mentioned that she did not clearly understand the change in colours of the feedback light but subsequently figured that it could represent that her answer has been received by the system. The limitation of the testing method used is also reflected here. Our third user, Jamie had queries on how to perform the task at first. She asked whether she could randomly press any button to answer the quiz question. She realized that the quiz question in Figure 1.4 had options 1 to 5, so she clicked button ‘2’ as her answer. She was confused about the blue light but subsequently felt that the change in colours of the light was not a concern or a problem. She felt that the screen was a good addition as it showed the answer that she has picked for the question. There were two usability problems. There was a few seconds delay in the response of first two users due to failure to recognise change in the colours of the feedback light. This suggests that the colour change or blue colour choice may not be intuitive to the users. This may be because it is inconsistent with familiar systems’ feedback light for response acceptance. However, the short delay and the users’ subsequent understanding without prompt suggest that this is a minor usability problem. Furthermore, the time delay may be inflated by the serial position of the tasks and paper prototyping limitation. There is a need to follow-up with high fidelity prototyping which provides real time interaction, and changing task sequence to certify the need for change in feedback light. Secondly, the feedback light may be redundant, as perceived by the third user. This is because the small screen also provides feedback to the response selection, resulting in multiple sub-feedback mechanisms. It occurs to us that the feedback sequence of the UI may be too long, thus there may be a need to shorten the sequence. We can consider the removal of the

blue feedback light as irrelevant features increases user’s cognitive load and impacts the design aesthetics. Also, we realized that all three of them did not notice that in the slide (Figure 1.4), there were two distinct sections in the toolbar (Figure 1.5) and that the responses section was activated while the other function was deactivated. Scenario Task 2: Our first user, Hei Wai performed the task without much difficulty. She said that the hand button looks most relevant in this task and thus she pressed the hand button. When the hand button lighted up in green, she said that this is an indication that it is in the usage of hand button mode. Her response indicates that the system effectively communicates the design model to the user, resulting in the correct user mental model. Once again, Hei Wai did not notice that the change in activation responses function for the quiz questions to the hand/participation function on the toolbar (Figure 2.4). Our second user, Sheng Rong, also performed the task smoothly. She pressed the hand button as she said it felt most intuitive and relevant to perform task 2 using this button. She also understood that when the hand button lighted up, it indicated that this button was in use. She briefly mentioned her concerns on how the professor would be using this piece of information. Like Hei Wai, Sheng Rong did not realize there was the toolbar to feedback to students about responses and queries and whether which mode is currently turned on or off. Our third user faced some problems with this task. Initially, she did not know how to use the clicker to perform the task. However, she said she would raise her hand in normal situations when she wishes to answer a professor’s question. Following this, she thought carefully and looked at the clicker prototype again and pressed the hand button. After she was presented with the final sketch showing the blue feedback light (Figure 2.2), she expressed some confusion about whether it represented that responses were received or an indication that one is picked to answer the question. There were two usability problems. The first would be the non-visibility of the toolbar in the slide (Figure 2.3). Users fail to notice the change in activation from the response mode to the participation mode. As explained earlier, this is likely to be a result of the use of paper prototyping sketches whereby we only use pencil descriptions, thereby being unable to capture users’ attention and to reflect the real time changes in such situations. The second problem is brought up by the third user, which is a change in the purpose and function of the hand button, resulting in mode error. She suggests distinct feedback lights to indicate the change in function. This solution may be feasible if the number of feedback light colours is not too large whereby users will have to learn them by rote, increasing their cognitive load and usability problems associated with their usage. Follow-up testing is required to verify this. An alternative solution would be to inform users about the function of feedback lights. This can be done via the instruction cards that come with the clicker which are usually placed on the back of the clicker. This is a convenient and easy way to provide help documentation without complicating the design of the clicker.

Scenario Task 3: Hei Wai had once again performed the task smoothly. She pressed the hand button as she felt that it was intuitive as students would normally raise their hands before posing a question to their professor, suggesting that the UI design matches the real world. After pressing the hand button, she paused when the word ‘Slide’ appeared on the screen of the clicker as seen in Figure 3.2. She mentioned that this was probably to guide users to key in the slide number. When the feedback light was changed to blue (Figure 3.4), Hei Wai wondered if this response is captured by the system automatically or by the lecturer manually. As mentioned in the previous two tasks, she did not notice the toolbar again or the change in functions in the toolbar. Sheng Rong did not perform the task as we have expected. She pressed the 0/J button to perform this task. She did it because she felt that this was an underused button and could likely be used for this task. She felt that the hand button did not appear intuitive as it did for the second task. We then presented her with the error sketch (Figure 4). She was confused at this moment and wondered out a loud that it should not be the hand button because she had already used it for the second task. Thus, the usage of multiple modes reduces the execution of tasks associated with the button. She hopes that there can be a separate button for the two different tasks. She decided to give up doing the task after the error occurred. As mentioned in the previous two tasks, she, like Hei Wai, did not notice the toolbar again or the change in functions in the toolbar. Jamie performed the task relatively smoothly. She pressed the hand button as she felt it was intuitive, as she would raise her hand before asking a question. After pressing it, the word ‘Slide’ appeared and she was a bit confused about which slide number to key in. However, she realized that she could figure out the slide number based on the current slide or even from her notes. After keying in the slide number, when we showed her the sketch with the feedback light in blue (Figure 3.4), she realized that this change in feedback light was an indication that her responses was received by the system. She questioned the ability of the system to cancel or change queries sent since she was able to change her responses to the quiz question. In all, there were two usability problems expressed by our users. The first is the occurrence of mode error in the hand button. It is used for two different tasks – sending queries and participating in class, resulting in error. After the performance of the second task, users have difficulty in associating the hand button with the third task. This may be because of the lack of labels in the hand button to indicate that the button serves more than one mode. Furthermore, the other buttons on the UI do not serve more than one function. Inconsistency may have resulted in the delayed or inability to recognize the modes of the hand button. There are two possible solutions. We can consider having two distinct buttons for participation and sending of queries – where each button serves one function only, consistent with the design of other buttons. There should be no user confusion or mode error but there is a space constraint in the current UI thus we do not want to reduce the size of current buttons in order to be able to add more buttons. This goes against the grain of an aesthetic and minimalist design, reduces relative visibility of buttons by increasing competing stimuli. Moreover, with careful consideration, the hand button does not represent a direct step to performing the two different tasks (Tasks 2 and 3). In fact, it represents only one function – that is to signal to the professor, matching the real world where raising our hand gets the attention of our teacher/professor, regardless of whether we

want to answer questions or ask questions. Therefore, we may consider the second solution which is to add a small label above the hand button to indicate multiple modes. The second problem is the violation of user control and freedom. Removal of responses is not inherent in the original design. However, users were able to overwrite responses to quiz questions which provide user control and freedom to some extent. One solution would be the addition of a button that enables user to undo their action. An alternative capitalizes on the consistency of other buttons which is to allow users to overwrite a submission with an unsubmission. For instance, users can perform the same task sequence for query sending or participation again. However, the system will prompt the users to remove the previous submission. A drawback to this solution is the occurrence of mode error from the multiple functions the buttons serve. Followup studies are needed to test the effectiveness of the proposed solutions. Overall, usability problems identified in this low fidelity prototyping procedure need to be considered in the context of its limitations. The lack of real time interaction and feedback as well as colours may have caused users to fail to notice details such as change in activation of modes displayed on the clicker and the slide. High fidelity prototyping is needed to confirm the findings in this testing.