Task Analysis

Problem and solution overview Our implementations to the current UI hope to improve instructor-student interaction in class and resolve current problems of subjective assessment of student participation and usability problems regarding the usage of the UI. We will modify the current design of UI lights and include a screen to improve feedback. [
] button will be included for entering of enquiries and for participation in class. The software and Edventure will be modified to allow participation to be recorded and displayed to the user. Interview Description and Results Our target users are the NTU student population who use the clicker regularly for their classes. The rationale behind our choice of target users is that they have experience using the clicker and thus design implementations would be relevant to them and improve their efficiency of clicker usage. We interviewed a total of three students who are currently using clickers in some classes that they attend via a video camera. All three of our interviewees are currently taking the course HP308 Psychology in the Workplace and are Psychology Year 3 majors. They all own mobile phones and mp3 players which contain Bluetooth or wireless devices and have used the remote control before. Our first and second interviewees Cherie and Nathalie’s language medium is English and are Singaporeans while our third interviewee, Sheng Rong, is a China PRC and she is more familiar with Chinese. All three of them use the Clicker during HP308 for their pop quizzes that occur on a weekly basis. They are required to select the correct response to the questions posed during a pop quiz, by pressing the correct button with the response they want on the clicker during the time frame with which they are given to answer the question. The interviews with them gave us insights of the current problems as well as possible solutions. We asked a total of thirteen questions and they are listed in the appendix. In summary, they all mentioned that there was a lack of feedback when they selected their answers and were unsure whether the answers got through or even whether they had clicked the correct answer. Thus, it had resulted in them clicking on the same answer repeatedly, leaving them frustrated. This is also applicable when they have selected a wrong answer, which in this case, they would quickly make amendments by clicking another button, for example, B, instead of A, repeatedly, to ensure that the new answer overwrites the previous response. Some themes common to all of them are the fear that their answers would not be accepted by the system as it would affect their assessment scores and the need for the accuracy in answering questions during a short frame of time. Both Cherie and Nathalie viewed the implementations suggested in our proposal favorably. They include the inclusion of a small screen, better feedback, the ability of clickers to tally students’ participation scores, for students to check their scores via the clicker and for students to post queries on various slides. However, for Sheng Rong, she was favorable of it in some sense, but had mentioned briefly that the tallying of participation scores is ideal but not necessary and that raising hands and speaking up directly in class rather than using the clicker to pose queries on particular slides would be better as it gives people the chance to exercise their communication skills. Task Analysis 1. Who is going to use the system?

This system was designed for both lecturers and students. On the part of students, they possess the clicker but not so for the lecturers. Thus we will be focusing on them instead of on the lecturers due to the focus on the student response unit rather than the TurningPoint software. 2. What tasks do they now perform? According to the interviews, they currently only perform the task of answering multiplechoice questions for general surveys and quizzes set by their lecturers. 3. What tasks are desired? Up to date, we feel that the invention of the clicker program is due in part to increase the interaction between both students and lecturers and to increase class participation and interaction. However, it has been shown to be single-directional from lecturers to students. Through the interview, we have gathered that some tasks that are desired could be the clicker recording their participation scores or rates and perhaps the introduction of using the clicker to pose queries or questions directly to the lecturer without disrupting the class. 4. How are the tasks learned? According to the interviews, the task that they now performed is learned over a period of time. Instructions were provided at the start of the class when clicker was first introduced and subsequently, students learned to use it through numerous practices. 5. Where are the tasks performed? The tasks are performed in settings as lecture theatres or seminar rooms, under the instructions or plans of the lecturers who make the decision on when students are required to use it. 6. What’s the relationship between user and data? The users that we are focusing on, in this case, are the students. The data that is eventually captured by the system through the use of the clickers by the students, reflect the eventual scores of their quizzes. Thus, the relationship between the users and that of the data are results that determine part of their final score in a module. 7. What other tools does the user have? The users have basic computing skills, experience using remote control devices (with number buttons on them) which are commonly used in the household. Most of the users have mobile phones and laptops with wireless devices and thus have a basic idea of how wireless devices work. 8. How do users communicate with each other? The term “users” refer to both students and lecturers. Therefore, the two different groups of users (students and lecturers), communicate via the clicker. However, this communication is one-way, that is from lecturers to students to answer the required questions for their quizzes through the clicker via a wireless network but none going from the students to the lecturers with the need for the latter to answer students’ queries. There is no form of communication available between student users. 9. How often are the tasks performed? The task of answering their quizzes via the clicker is performed on nearly a weekly basis, depending on the lecturer.

10. What are the time constraints on the tasks? The time allowed for answering the questions depends largely on the part of the lecturers who are responsible for setting the time limit to how much time the students are given to complete the task. A time limit of ten seconds is usually set for each question posed in the quiz. 11. What happens when things go wrong? Presently, there are no backup strategies for when task-related errors are committed. The lack of adequate feedback from the UI prevents users from noticing their errors. If the user forgets to change the channel formatted for his clicker or his response fails to be captured or a wrong response is sent, the only possible feedback is the number of responses logged and displayed on the instructor’s screen. However, the instructor usually does not announce or take attendance of the number of people in class or present thus number of responses is a weak feedback. He may only notice these errors when he views his updated scores on Edventure when it is too late. In addition, when the user fails to submit his response within the time frame given, he will be forced to accept a zero score for that question. Description and analysis of sample tasks Easy Task 1: You would like to select ‘A’: Yes regarding the question “Did you have MacDonald’s for lunch?” Submit your response ‘A’ using the clicker. This task is the submission of a response to a quiz or survey question using the clicker. It is well-practiced by our target users. It is automatically captured via a wireless network to the Turning Point software on the lecturer’s computer, as long as the answer is provided within the time limit. Submission and connection are automated. Easy Task 2: You pressed the 5/E button in order to respond to the question with the answer ‘none of the above’. However, there are only 4 options: A, B, C, D for this question. Change your answer to option B instead. Changing of responses occur frequently for users as many tend to select responses too quickly without reading the question thoroughly and the adequate time frame provides opportunities to correct an error submission. Submission of a new response automatically overwrites the previous response submission. Users need not perform additional tasks in order to do so. Moderate Task 1: Your lecturer is explaining some concepts on the current slide. You have some enquiries about it. Use the clicker to signal that to your lecturer. This is a task that our re-designed interface will support that was not supported in the original interface. It will involve function buttons which are new to the user but the action sequence required will be similar to that of answer submission. Submission and connection are also automated. Thus we classify this task to be of moderate difficulty. Moderate Task 2: Your lecturer posted a question to the class. You like to participate by answering the question. Use your clicker to do so. This is a task that our re-designed interface will support that was not supported in the original interface. It will involve function buttons which are new to the user but the action sequence required will be similar to that of answer submission. Submission and connection are also automated. Thus we classify this task to be of moderate difficulty. Difficult Task 1: You have some questions about a concept in a slide which has passed. Use the clicker to signal to your lecturer regarding that.

The original interface only supports response submission regarding the current slide displayed. The users have no experience in submitting responses regarding slides which have passed as they were told that they are not able to do so. This concept will be introduced by our new interface. In addition, the function buttons associated with posting enquiries are new to the user. This task requires an action sequence which is longer than answer submission. Thus it can be classified as a difficult task. Difficult Task 2: You have been using your clicker to ‘participate’ in class. Check your participation score thus far on Edventure. This task will be supported by our re-designed interface; it was not supported in the original interface. It is the only task which requires the usage of a computer, connection to the internet and logging into Edventure. Although they are familiar to users, the option to view participation scores on Edventure is new. The need to perform multiple steps in an unfamiliar action sequence makes this task challenging for users. Interface design We will introduce a small screen to the current UI. The screen will resemble that of a calculator screen which can display text and numbers. It will be dark grey in colour and display text in white. Its low cost makes it feasible for production in large numbers to be provided at low or no cost (the current UI) to the whole NTU student population. It will display the following: •

What channel is the device connected to

•

Which response was selected and whether it is accepted

•

Slide number

•

Errors in response selection i.e. “response is unavailable” with regard to current question on screen

By informing the user which channel the clicker is connected to (Fig. 1.1) will remind the user to change to the appropriate channel. This serves as a reminder and an error prevention measure especially if the user uses the clicker at many classes and has to change channels frequently.

Fig. 1.1

Informing the user which response is selected and whether it is accepted (Fig. 1.2) can improve the current user feedback immensely. It will reduce fear associated with the ambiguity of the system state and eliminate the need to submit response multiple times. Thus improve user experience of the clicker.

Slide number information (Fig. 1.2) is essential as the new interface allows the user to input response for slides which has passed. This implementation allows irreversible system functions to be reversible by extending time constraints from 10 seconds to the time period of the whole lecture. However, this will be available for participation and enquiries only, not for answering quizzes in order not to defeat the purpose of placing time constraints on a quiz. As this feature is novel, adequate feedback to system state will aid the user in performing associated tasks. This will allow the user to know which slide the information is being sent to. Current UI emits a blinking red and green light when it is activated for the input of channel code, a blinking green light when polling for responses is open, a yellow light when a response is sent. We believe changing the UI to emit a steady blue light when a response is sent is more appropriate as it is more distinct from green than yellow (Fig. 1.2). A clear feedback of a change in system state will be communicated to the user. Submission of a subsequent different response will cause the system to stop emitting light, the user has to reenter the response one more time for the system to emit a steady blue light. The double entry serves as a feedback to the user that the previous response is being removed and as a confirmation measure as to change of response. The blue colour is also appropriate as it (apart from green) universally represents openness and clearness and can be associated with positive acceptance (of response). The usage of knowledge in the head instead of rote learning (association of yellow to acceptance) is not only aesthetically pleasing but also makes the UI more intuitive to the user.

Fig. 1.2

Providing error messages (Fig. 1.3) due to submission of a response that is unavailable serve as an error prevention measure as well as a reminder to the user to double check the responses available and input the desired response. We also suggest that the clicker emit a blinking red light (Fig. 1.3) when an error selection is made. This is an error preventive measure as the current UI only provide feedback to the user after polling is closed; TurningPoint software displays selection for an unavailable option together with valid options in a bar chart. Furthermore, detection of the slip will only occur if the user can correctly associate the data on screen with his action which is difficult as feedback is poor. Moreover, when the user receives this feedback, error correction is no longer possible as polling is closed. Our implementation allows error correction to be made within time constraints. Blinking frequency is suitable as a warning signal as it can more easily capture the user’s attention than a steady light. In addition, the red colour is appropriate as it relies on the user’s knowledge in the head which is the association of the colour red to warning and error. It helps to maintain simplicity of design and eliminates the need for instruction or labels.

Fig. 1.3

In order to incorporate additional features to the current design without cluttering it with more function buttons, enquiries submission and participation will share the same button [
] on the UI (Fig. 1.4). This refinement contains the minimal critical features required for user recognition without creating unnecessary cognitive load. [
] effectively represents participation and enquiry as it allows the system to match the real world where the user participates and ask questions in class by raising hand. We will be replacing the current [?] button with [
] as the current [?] is commonly associated with “Help” function thus [
] is more appropriate. In consideration of the possibility of mode error to occur when one button serves multiple functions, the design of the UI is such that Participation and Enquiry are mutually exclusive. When the instructor posts a question, the user can press the [
] button to answer the question (Participation). Enquiries are not needed as the instructor will further elaborate the concept via the questions and answers. On the other hand, when a slide does not concur with any instructor questions, the user uses the [
] as an Enquiry button. The constraint placed on the system effectively prevents mode error. Unlike other buttons on the UI, this button is displayed is in the shape of a hand and lights up when pressed. Thus, when the user presses it, [
] will light up to feedback to the user (Fig. 1.4). In order to further differentiate the light from the background of the UI, the light will be a distinct green. Our colour choice capitalizes on user’s knowledge of the head as well because green is associated with positivity such as the traffic light and increment (stock market). We hope to use the green light as a subtle cue to encourage the user to participate and ask questions in class. [
] button will be salient against other buttons in the UI as it is a pictorial icon instead of alphabets or numbers. Search time will be reduced as the user can perform parallel search instead of serial search for this button. Our design hopes to encourage two-way interaction – student participation via the relative ease of usage of the [
] button.

Fig. 1.4

[
] button serves as a mode button. When pressed, it changes the default mode in the UI which is to answer questions to either Participation or Enquiry mode depending on whether the instructor selected Participation mode in the software. At default, all information from the clicker is sent to the current slide. Thus, in order to send Enquiry to a previous slide, the user should first press the [
] button to change mode then press the slide number. This information will be reflected on the small screen as well because we wish to provide adequate feedback to the users when this is novel to them (Fig. 1.5). This action sequence reflects a system match with the real world when the user raises his hand before speaking in class, and consistency with the existing action sequence where response numerals should come in last as the final response is the only response recorded by TurningPoint.

Fig. 1.5

Time constraints in the current UI albeit effective and appropriate for quizzes, discourages student participation and submission of enquiry, especially that of the latter. We will adjust TurningPoint software to allow networks to all slides to be turned on concurrently, allowing submission of responses after the slide has passed. The abolishment of time constraints allows the system to be better matched to the real world where there is no time limit to when students pose questions to the instructor. Also, in the real world, the user will require some time to digest the lecture content and decide that further explanation is needed and we will not want to deprive the user the opportunity to clarify lecture content. As the ‘clicker’ is not a standalone device, TurningPoint should complement the functions and feedback on the clicker for them to be effective. We suggest the natural mapping of function button layout to that of the screen. For instance, the recorded numbers of Participation and Enquiry should be displayed on the bottom right corner like that of the UI. This allows the user to immediately find and associate the data with that of Participation/Enquiry. Display of the number of participation and enquiry responses to student users is important as it provides some form of feedback that response was recorded, it also capitalizes on the bandwagon effect which is the tendency to perform a behavior when many other people do so. Thus by providing feedback to student users that many people have enquiries or wishes to participate in class will boost enquiry submission and participation, serving the goal of the system.

Fig. 1.6

We will also allow participation scores (clicker “
”) to be displayed under “View Grades” in Edventure (Fig. 1.7). This is based on the Gestalt law of similarity. At present, quiz scores are uploaded and displayed in “View Grades”. As participation assessment also contributes to course grades, the user will view participation and quiz as a group resulting in the automatic search for participation scores in the vicinity of quiz scores. By allowing the display of both items to be found under the same hyperlink and on the same page will ease user search of participation scores.

Fig. 1.7

Scenarios and Storyboards Easy Task: (Easy Task 1)

Your professor decides that there is a HP308 quiz for the day. There are a total of eight multiple-choice questions, with each question consisting of options from 1 to 5. Each question is flashed on the screen for a time period of ten seconds. Each question needs to be answered within the stipulated time. The first question is, ‘Fredrick Taylor is a psychologist who introduced Scientific Management during which period of time?’ The answer for this question could be one of these 5 options: 1-5. You need to answer the question and can do so by selecting the response that you deem most appropriate, by pressing one of the buttons: 1/A, 2/B, 3/C, 4/D or 5/E on the Clicker. You believe 1 is the correct answer. Answer this question using the clicker.

Turning on and setting channel

Flashing green & red Key in channel code Press and release

Moderate Task: (Moderate Task 2) The lecture is now in progress. As usual, your professor likes to ask a few questions to the entire lecture of forty students to ensure that all of you understand the concept. She is currently going through a slide on engineering psychology is a hybrid. The professor poses this question verbally: “Why is it that engineers and psychologists have to work together?”, and expects you to use your Clicker to signal that you want to attempt to answer her questions. To try to get chosen to answer the particular question, press [
] button on the Clicker, so that the professor will know that you wish to answer the question and that you may have the chance to do so.

Difficult Task: (Difficult Task 1) In the progression of the HP308 lecture, you begin to grapple with the concepts of ‘efficiency rules for manual labour’ in slide 5. Before you could fully comprehend the concepts and ideas to why steps are done in this concept (Slide 5), your professor has moved onto the next slide to discuss the following concept of ‘person-machine systems’. You feel that there is a need to clarify your queries in slide 5. You can do so by clicking the [
] button and entering the slide number, which in this case is 5, to let the professor know that there are queries on this slide.

Appendix A Interview Questions 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

12.

13.

Do you have a clicker? How many classes require you to use your clicker? What are those classes? How many times do you use your clicker a week? What tasks do you perform with your clicker? How did you learn to do them? What are the time constraints when you use your clicker? For instance, do you have to ‘click’ within a certain period of time? What is the time period? What are some problems you experience with your clicker? If you change your answer, does the clicker provide any feedback? i.e. are there any visible changes in the clicker? What do you do when problems happen? Do you think adding a small screen on the clicker to display which answer was chosen and whether it is accepted would improve the current design? If the clicker can be redesigned to include more functions, what would you like them to be? Given that we NTU students are usually shy about asking questions or providing our answers in class, Would you like to be able to ask and answer your lecturer’s questions using the clicker? For instance, signal your lecturer using the clicker when you want to participate or when you have a question about a certain slide in the lecture. Given that the current system for assessing class participation is subjective Would you like to have the clicker record and quantify the number of times you participated in class and allow you to view the tallied numbers via Edventure?