Session 9 Nuz Quadri

Exploring Students’ Views with Video Diaries

Nuzhat Quadri and Professor Peter Bullen University of Hertfordshire, [email protected], [email protected]

ABSTRACT Students’ views are often sought by institutions using conventional methods such as surveys, focus groups, and interviews. Students tend to be saturated by such methods and there is a strong possibility that their views are not fully reflected. In an attempt to gain a deeper insight into student behaviours, the Blended Learning Unit has implemented a form of feedback which could be perceived as enjoyable for participants as well as constructive. This feedback is the popular method of video diaries used in an educational setting. Five students were loaned a digital camcorder to show what their typical week is like at the University. Students were given advice on what areas to focus on but there were few restrictions on the content. Students provided feedback at the start and end of their day in order to reflect their normal routine, and for academics to gain understanding of student life. The general topics which were covered were learning and teaching preferences, perspectives on using technology in learning, as well as commenting on how the Managed Learning Environment (MLE) is an aid to learning. Although the number of students was small for this study, the project was a pilot and aimed to inform a second project using various media. Students handed in recorded tapes at the end of the week from which analysis was conducted. By viewing the videos, themes were developed and corresponding clips were edited. The method and initial findings are discussed in further detail, covering how video diaries can be implemented in an educational setting and used as an alternative form of feedback. The paper also covers an extension of this project involving video, audio and blog diaries.

KEYWORDS Video diaries, evaluation, feedback, students, research method.

BACKGROUND TO DIARIES: AS A RESEARCH METHOD Surveys and focus groups are very commonly used in institutions to measure students’ attitudes or gain insight into their views on various topics relating to the University. However, these methods can have limitations: surveys are very often restricted to a 5SOLSTICE 2007 Conference, Edge Hill University

1

point scale where respondents must choose a value that best represents their viewpoint; focus groups or interviews rely on the interviewer building good rapport to make the interviewee feel at ease when answering the questions. The questions which are set in both surveys and interviews must account for all perspectives to represent the response of a participant/ interviewee. With these possible limitations, it is arguable whether the true opinion of the respondent is achieved. Diaries as a form of research method have advantages over using surveys and focus groups because they provide an opportunity for an individual’s personal reflection. The content of such a document is likely to be more honest as the information is not being obtained through a face-to-face method. Thus “diaries may facilitate access to very personal and intimate information that may not emerge in a face-to-face interview” (Willig, 2001). There are fewer restrictions on the content using diaries; and the information obtained is in real time, so it “reveals how events unfold prospectively” (Willig, 2001). This means that data is not lost by details being forgotten, which is possible when commenting on topics retrospectively. The use of diaries as a research method has been used in qualitative research as part of health (e.g. Hufford, Stone, Shiffman, Schwartz and Broderick, 2002) and education fields (e.g. Clegg, Hudson and Mitchell, 2005). This method has been used because it provides “evaluators with ability to gain access to in-depth, detailed insights” (Cohen, Leviton, Isaacson, Tallia and Crabtree, 2006) into people’s views of the investigation area. In these studies, diaries were produced online as opposed to on paper, because it was considered “timely to look at developments using portfolios and to consider some current practice that is attempting to use digitised technology to support individual reflection” (Clegg et al., 2005). Cohen et al.’s (2006) study used online diaries as an opportunity to interact with the participants and to keep them engaged in the diary keeping. Online diaries are a common method where people taking part in research will produce what is now known as a weblog (often referred to as a blog). A blog is compiled of mainly text, but can be accompanied with other media such as photos, audio or video, however diaries can also be produced using audio or video alone.

VIDEO DIARIES AS A RESEARCH METHOD As well as online diaries, video diaries have also been used in health (Antoniu, 2003) and education sectors (Noyes, 2004). Noyes conducted a study with school children on their feelings of the transition between primary to secondary school. Video diaries were used as a method that complemented other research methods used in the study, namely observations and interviews. The data that was produced from the interviews and observations was limited because as Noyes explains, his “continued presence in the interviewing process understandably inhibited [this kind of] disclosure” (Noyes, 2004), where the pupils were not disclosing much information about their lives outside of the classroom. For this reason, video diaries were used so that the students could “talk more freely about their unseen day-to-day experiences” (Noyes, 2004). Even though Noyes had interviewed the students earlier in the study, the results of the video entries produced were described as being “far more compelling”. The children were more inclined to report feelings which they hadn’t reported in an interview session

SOLSTICE 2007 Conference, Edge Hill University

2

and so the findings were more in-depth. The results of the video clips have been described as being incommunicable on paper compared to watching the video clip, which highlights “the limitations of text/language for conveying knowledge” (Noyes, 2004). In the past decade, there has been an increase of this sort of diary use in the media, with the introduction of reality TV programmes where members of the public report their feelings using a video camera. One of these programmes is ‘Big Brother’, which is a flyon-the-wall approach to watching people living in a house together with certain restrictions. The most interesting aspect of this programme is participants commenting in a “diary room” where they share their thoughts and feelings with the public. Video diaries are also broadcast on Internet sites such as YouTube, where people can upload and share videos. YouTube has 100 million clips viewed each day, and 65,000 new videos are uploaded every 24 hours (Wikipedia, 2007). Video diaries is a very popular and powerful tool providing the potential for comprehensive feedback from students on their learning and teaching experiences and their use of technology This paper concentrates on video diaries as a form of feedback. The successes and challenges faced by the research team in two projects are examined, and implications of using this method in other institutions are discussed.

METHODOLOGY Two projects have been conducted which involved students providing feedback on various aspects of learning at the University. The aim was for students to provide feedback on University life using a form of media that would allow them to express themselves and to have fewer restrictions on the content. The two projects had the same basis for providing feedback, but there were differences in particular parts of the methodology. Here, the project that was conducted in 2005-06 will be described, followed by the project conducted in 2006-07. The first project involved five students from different discipline areas: Maths, Product Design, Accounting and Financial Management, Emergency Planning and Management, and Music Technology. This project was conducted as a pilot study to gauge how successful the use of video diaries would be as a form of feedback for the University. Students were recruited in the project via StudyNet, the University’s Managed Learning Environment (MLE) where a news item was posted for all students to view. Students who took interest in the project were asked to produce a five minute clip so that the research team could establish how well they could communicate using a video camera. In this five minute clip, students were asked to talk about why they wanted to take part in the project. The five students who were selected to take part in the project were asked to attend a meeting where they would be given more information about the study, such as how to use the basics of the video camera, and the nature of the project was explained in terms of what they were to do on a daily basis for the next five days (Monday to Friday). At the end of the meeting, the students were given a digital video camera which was their’s to use for the week, and three digital camera tapes. This allowed them two and a half hours recording time for the week.

SOLSTICE 2007 Conference, Edge Hill University

3

The students were given a set of guidelines to follow for the week so that their comments would be focussed on specific areas of interest for the research team. These guidelines were based on their expectations of studying on the day and for each study session; what they thought would be a positive/ negative part of the day; how much they used StudyNet/ technology in their learning; any improvements that they could suggest; and parts of their learning which they deemed were successful. The students were to use these guidelines to comment twice a day: once at the start of the day and once at the end of the day, and were asked to comment for no longer than fifteen minutes at each time. At the end of the week, the students handed back the video camera and the video tapes. The second project was implemented to make improvements on the method of using video diaries and to try to improve the quality of feedback that was received. One major change that was made to the project was to include other forms of media for students to use from the choice of either video, audio, or blogs. Fourteen students took part in this project and from a wider range of disciplines, including: Humanities, Psychology, Law, and Nursing. Students were again recruited by posting a news item on StudyNet. In this project, the methodology was changed to allow more students to take part, and to accommodate ease of analysis. Firstly, as mentioned above, students were given the choice of using one of three types of media: video cameras, audio recorders or blogs. This was so that students who wanted to take part in the project but did not want to use a video camera could have the opportunity. A second change was that the project was over a seven day period, from a Thursday to a Wednesday. This change was brought in because the project was conducted in collaboration with the Students’ Union who wanted to use the weekend for students to comment on their social activities. Thus, throughout the week, students had a set of general topics which were based on their academic experiences as well as their social experiences. A third change to the project was that rather than have guidelines that the students had to use throughout the week, which appeared to have become exhaustive for the students, different topics were set for each day. These were presented to students in seven different coloured envelopes, and students were sent a text message notifying them of which envelope to open each day. Students were asked to comment whenever they wanted but for no longer than thirty minutes each day. Once the students were selected to take part, a meeting was again held to give details on the project, to hand out the media to be used, and the seven envelopes for each day. There were more students who used the video cameras (eight students) compared to audio recorders (five students) and blogs (one student). It was rather disappointing to see that only one student opted for the blog, but it seemed that students would much rather use technology to keep a diary than write a blog, as one student comments: “I think a blog would be a lot more work (much quicker to point a camera and talk than get to a computer and type” (Humanities student, Year 2). Students who took part in the video diaries were given four tapes to use with the video camera. This gave them four hours of recording time for the week but they were to use a maximum of three and a half hours. Students who opted for the audio recordings of

SOLSTICE 2007 Conference, Edge Hill University

4

their diaries were given a digital voice recorder; and the student who blogged used StudyNet’s blogging facilities on a private setting. The blog was then copied into a Word document and handed to the research team. All of the students who took part in the project signed a consent form agreeing to take part, and also a form which had terms and conditions for using the equipment. This form was also signed for taking away and returning the equipment. The students in the second project verbally agreed to being sent a text message regarding which envelope to open.

ANALYSIS OF VIDEO CONTENT In the first project, the content and length of the diaries that were produced varied considerably. Some students would refer to the guidelines throughout the week, whereas others would talk about anything relevant to their University life; some commented for fifteen minutes twice a day, whereas others would comment for a couple of minutes and just once a day. The analysis of the videos was done by the research assistant who was part of the research team. This was a very time intensive process and was implemented by firstly watching the videos, and then content analysis was used to code each student’s video footage. Finally, themes were developed by examining all of the video footage. The amount of time spent analysing was proportionate to about one day for every hour of video footage. The videos were not transcribed into written comments because this would defeat the object of using video to receive feedback. However in this paper, student comments which were relevant to the analysis have been reproduced to give some idea of the feedback received. In the coding stage of analysis described above, the comments were typed in order to conduct the analysis, but the actual video footage was edited by a Video Production Officer, who is part of the TV Studio Team at the University. These clips were produced as individual files to use for dissemination purposes. The themes that emerged from this project were: learning enhancement by lecturers, StudyNet and University resources available, communication methods used by lecturers, study behaviours outside of class time, setbacks in students’ learning, support issues in their learning, and suggestions to enhance students’ learning. The analysis of the second project was conducted using content analysis as in the first project, and the analysis is still in progress.

FINDINGS FROM STUDY 1 In this section, comments that the students made are shown as examples under some of the themes, which are then summarised according to all of the students’ comments. Learning enhancement by lecturers This theme was related to the comments that students made about academic staff at the University, in terms of the help they offer to students, the technology they used in their classes, and the effects this had on their learning. The Maths student comments on a lecturer providing examples in class: The guy [lecturer’s name] is very good, he knows his stuff and his explanations are very accessible, everyone can understand and familiarise with them. If they SOLSTICE 2007 Conference, Edge Hill University

5

can’t, he’ll go over them in not exactly the same way that he did the first time, which is helpful, means he’s got back up examples in case you don’t get the first one. I’ve seen some lecturers who just have the same example, when you say I’m stuck they just tell you the same thing and expect you to understand it by hearing it repeated, which doesn’t work- parrot-fashioned, on this level isn’t very good (Maths, Year 2). From analysing the findings of this theme, it appeared that students expected lecturers to help with their learning, rather than just deliver content, as this has a big influence on how much students learn when in class, and also the independent study they engage in outside of class. The students were pleased that lecturers had the knowledge to deliver a lecture or seminar, as well as having good relations with the students. Any negative comments were made if lecturers were not committed with their time outside of classes. StudyNet and University resources available This theme related to any references made by students of StudyNet or other resources that could aid their learning. There was considerable use of StudyNet by students in their studies, either to search for journals or just to keep up-to-date with University life via the news items posted on the homepage. This theme also related to the technology used in classrooms, which generally students seemed impressed with and felt met their needs. Students commented that Over Head Projectors (OHPs) were used, even though lecture theatres are fitted with enhanced technology such as laptops and data projector screens. Other facilities that were in classrooms to help students learn were positively described such as using technology for a Music Technology course. A student describes different uses of StudyNet as part of his learning: There are two, possibly three main uses of StudyNet. One, email: I check it everyday. Two, I check the news, what’s been going on around the University. Three, when we have lessons we go into our specified folders… go into our subject area and all the documents are there like online worksheets. A fourth use is posting work on StudyNet instead of saving it to disc (Music Technology, Year 1). From this theme, it appears that there are sufficient resources available for these students to learn in class and to learn independently when using StudyNet. Communication methods used by lecturers This theme related to methods used by lecturers to communicate with students. Very often the method used was to replace face-to-face contact as this was not always possible. Email was the most common form of feedback, but StudyNet was also used to relay messages onto students. Students would check regularly to see if they had any meetings to attend, or even seminar or tutorial dates that had been set. One student describes how email is not always the best method to use to communicate with students, for example if a lecture has been cancelled:

SOLSTICE 2007 Conference, Edge Hill University

6

my day began with coming in to have my lecture with [lecturer’s name] but because he was ill… there was no lecture but the email got sent out at 9.29am, and I was already gone out of the house by that time so… [shaking head]… a bit kinda annoyed by that because it got sent out quite late, and I had to walk in, I basically got to University, and it’s like, no one was there (Product Design, Year 2). Study behaviours outside of class time This theme related to students’ studying behaviours when they were not in class, in terms of what they did independently to study, at what time, where, and so on. The students’ behaviours in relation to time was to work late into the night on essays or coursework that was due. One student describes what he will do for the week as independent study because he has no lectures: well this week, since I won’t be having formal lectures as such, I have to be quite self-motivated and I have to make sure I find myself in the LRC and ensure that I use the StudyNet facilities and print off some copies of notes, print off some journal articles by authors and use those to help me write my essay. I have a 3000 word essay due in May but I’m hoping to work on that earlier. So basically for the week, I will be getting up just after 12 o’clock or 1 o’clock, depending on how I feel, and then I will spend quite amount of hours in the LRC (MSc Emergency Planning and Management). Students seemed to have the facilities to be able to continue studying outside of lectures and would work either in groups or individually. The benefit of the Learning Resources Centre (LRC) being open 24 hours a day meant that students could study at any time. Setbacks in students’ learning This theme related to any factors that resulted in students not being able to study. The main setback that students experienced was the unavailability of connection to the Internet, or lack of Internet connection facilities, such as wireless or LAN ports. This lack of Internet connection meant that students were unable to use the online resources, and the lack of Internet facilities meant that students had to move to a different working environment. One student explains how this has an impact on his learning: the only thing which I suppose that quite annoyed me today was that I have a laptop and it’s got a LAN port, and I tried to connect it to the socket in the wall, well it looks like a LAN socket, which I suppose it is, it didn’t work, I don’t think it’s connected, which is quite annoying because… it would be great to have internet, in our workspace, so we could like, have a look at information whilst doing work, it would be quite good, but I’m gonna ask someone about that tomorrow, if they are able to turn it on, or I don’t know. It would be good to have internet points around the studio (Product Design, Year 2)

SOLSTICE 2007 Conference, Edge Hill University

7

The students reported that the lack of Internet connection did not happen very frequently but on occasions when it did, students were not able to continue working and had to move onto doing something else. One of the goals for the Blended learning Unit has been to provide capital funding to install wireless internet access in all teaching areas. To date approximately 50% of the teaching rooms have wireless connectivity installed. Other problems that students reported experiencing were upgrades of software resulting in saved work not being retrievable, computer work spaces not being free, and two hour lectures being held in two different classes due to timetabling issues.

INITIAL FINDINGS OF STUDY 2 The findings of the second project are currently in progress as the diaries were completed by the students very recently. The data produced by the students included video, audio and blogs which is very time intensive to conduct content analysis. The initial themes developed from the findings are the blend of technology used in learning; independent learning and how StudyNet supports this; improving staff use of e.learning and face-to-face learning; and communication between lecturers and students. The themes are based on the topics that were set for students over the seven days.

DISCUSSION From analysing the results of the diaries that were produced, it is apparent that the responses were much more in-depth and rich, providing data that may not have been obtained through a survey or an interview situation. The students had the chance to be alone and express their thoughts and feelings about their learning experience in a very truthful manner and in more detail. The diaries were an insight not only into students’ views but also into how students behave away from classroom situations, such as how they conduct their independent study and what they do outside of class time. Staff have responded to the diaries very positively and commented that this degree of feedback had not been previously received. An example of this was the comments made by the Learning and Information Services (LIS), who commented that “even if the comments are not complementary they are always enlightening”. The LIS team manage the LRCs and were extremely impressed by the input from the students about how they use the LRCs, from the swipe-card entry system to using the online journal facilities. Through the use of video, the audience, as opposed to just the interviewer, is able to view the students’ facial expressions when commenting on a topic. This made the comments more expressive, compared to reading text, and therefore offered a powerful influencing tool to help support learning and teaching strategy development and implementation. The students were also able to show what they were referring to, as they often took the cameras into the classroom. One of the students uses the camera to point to a set of workplaces in a design studio where there are no internet connections or computers. This is very powerful because the audience can see what the student is referring to when he says “I think you should stick computers here”, so that they don’t have to go to the LRC to use a computer or the Internet.

SOLSTICE 2007 Conference, Edge Hill University

8

However, using video diaries as a research method is not without its limitations. There is a chance that not all students will be willing to take part because this method can become very personal as the video camera is often used in the home. Alternatives can be used such as audio recordings, or blogs, however these do not have the same effect as they will not portray the picture which the video camera enables. Since it is a personal reflection, many irrelevant opinions can be disclosed. In the first project, a few students tended to give extensive detail when they talked about their experiences as there were few limits on what they had to say. Although some of this feedback was useful, at times it was difficult to understand the point of the comment which inevitably resulted in the data being more difficult to analyse. For this reason, the second project resulted in different questions being set for each day. Using questions also meant that there was a focus to what the students were saying. Another limitation is the inability to probe on what is being said. In an interview situation, the researcher is able to probe if the question is not answered fully or if the interviewer wishes to know more about the interviewee’s response. In watching the video diaries, if the topic was not covered in sufficient detail then it was not possible to ask the students to explain further. This can only be done in a follow-up with the individual but there may be some restrictions, such as them not turning up, or losing their train-of-thought from when making the comment. Diaries require commitment from the individual taking part in the research so it is important to ensure that the students do not drop out of the study, and that they comment when they are asked to. In these two studies, the students did not drop out during the week but there were times when the students did not comment when they were asked to. Students in the first project had to comment twice a day for the five days, but very often the students would miss out one session or even a whole day. Only one student missed entire days out and the other students only missed one session, but this resulted in data which was not complete. With the second project, the students would very often miss one day, then the following day they would start by answering the questions from the day missed, and then open the envelope for that day. Although this meant that data was not lost, because they answered the questions the next day, this lack of compliance did result in a loss of “real-time data” as explained earlier because two topics had to be discussed on the same day. This meant that the activities that happened the previous day was not included as part of the diaries. The commitment in this project was only for a week so perhaps this is why the students did not drop out, but if the project was over a longer period then the students may have started to drop out.

CONCLUSION The power of video diaries to provide a richer and more comprehensive feedback than more conventional tools, such as surveys, has been demonstrated. In using video diaries as a research method, there are many pros and cons to consider ensuring that the data produced is worthwhile. For institutions to use this as a research method, the research team need to ensure that the disadvantages outlined above are taken into consideration, such as constraining the questions so that the data is simple to analyse.

SOLSTICE 2007 Conference, Edge Hill University

9

This method has many advantages over other research but the analysis is very time intensive, so time should be allowed for such a project. To overcome this, proper guidelines should be given to students regarding what is expected otherwise some students can talk for long periods and make analysis unnecessarily long.

REFERENCES ANTONIU, S., 2003. Compliance with inhalatory therapy: an increasingly recognized clinical outcome Expert Review of Pharmacoeconomics & Outcomes Research, 3(4), pp. 449-456. CLEGG, S., HUDSON, A. and MITCHELL, A. 2005. The Personal Created Through Dialogue: Enhancing Possibilities Through the Use of New Media, ALT-J, Research in Learning Technology, 13(1), pp. 3-15 COHEN, J. D., LEVITON, L. C., ISAACSON, N., TALLIA, A.F., and CRABTREE, B.F. 2006. Online diaries for qualitative evaluation, American Journal of Evaluation, 27(2), pp. 163-184. HINE, C., 2000. Virtual Ethnography. London: Sage. HUFFORD, M., STONE, A., SHIFFMAN, S., SCHWARTZ, J. and BRODERICK, J. 2002. Paper versus electronic diaries: Compliance and subject evaluation, Applied Clinical Trials, 11(8), 3843. NOYES, A. 2004. Video diary: a method for exploring learning dispositions, Cambridge Journal of Education, 34(2), pp. 193-209. http://en.wikipedia.org/wiki/Wikipedia (accessed 2 March 2007) WILLIG, C. (2001) Introducing qualitative research in psychology. Buckingham: Open University Press.

SOLSTICE 2007 Conference, Edge Hill University

10