Sciencelearningengagementdigitalage-mobiletechadultengagementexperiencemuseum.pdf

Sarah Kounaves1, Louise Archer1, Heather King1 and Emma Pegram2 1

2

Natural History Museum, London, UK

Abstract. This study investigated how the use of mobile technologies in a natural history museum might affect the science engagement experiences of adults. A mixed methods approach was adopted over two phases. Phase 1 consisted of a questionnaire to adult visitors (n=60), seeking to explore attitudes towards mobile technology use in museums, and understand usage habits. The second phase consisted of an intervention. Adult visitors (n=20) were assigned to either a s using their personal mobile devices. Using multiple analytical methods including a modified Science Engagement Framework, preliminary findings suggest that contrary to findings in other studies, mobile devices may not be facilitating higher levels of science engagement in this context, and may instead be shaping visitor engagement experiences in other unforeseen ways. Given the current to re-evaluate claims of . Keywords: mobile technologies, science engagement, adults, museum learning

1 Introduction It has been argued that informal science learning settings such as museums have the potential to afford unique opportunities for adults to engage with and learn science (Bell et al.2009; Stocklmayer et al. 2010; Schwan et al. 2014). As personal mobile devices such as smartphones and tablets have become increasingly ubiquitous in these informal learning settings, it has become all the more significant for us to understand the ways in which these devices are interacting with the visitor learning experience. Although mobile learning research has been conducted in a wide array of settings, experts in the field of mobile learning have increasingly turned their attention towards places of informal science learning (e.g. museums, science centres, zoos, aquariums). Yet when set within these contexts, mobile learning research has primarily focused on children and students (Chen et al. itionally, many of the mobile learning and informal science learning studies have brought up the term engagement in conjunction with science learning, yet seldom investigated the concept of engagement itself, with few exceptions (Rennie et al. 2003; Barriault and Pearson 2010). Thus this research focused on investigating how the use of personal mobile technologies in a natural history museum context might affect the science engagement experiences of adults. From this the following three research questions were developed. RQ1. ds and perceptions of mobile technology use in museums? RQ2. How do adults naturalistically use mobile technologies in a museum context, and how does it compare to how they use them in their daily lives? RQ3. Does the prompted use of mobile technologies by adults in this context facilitate an increase in science engagement?

Two phases of research were conducted using a mixed-methods approach, as part of a collaborative reProvisional analysis of responses has shown that visitor dependence on and desire to use mobile devices decreases when at a museum. Perhaps more significantly, however, preliminary findings have suggested that, contrary to 361

findings in other studies, mobile devices may not be facilitating higher levels of science engagement, and instead may be shaping the overall visitor engagement experiences in other unforeseen ways.

2 Literature Review What is Mobile Learning? Mobile learning has been defined in a plethora of different ways since its inception. For example, some studies have defined mobile learning in more techno-centric terms (Stevens and Kitchenham 2011), while others have stressed the underlying learner experience (Sharples et al. 2007). Rather than defining mobile learning, it is perhaps more useful to identify some shared characteristics of mobile learning from the literature, which impact on this study that it is mobile, personalised, and collaborative. Mobility refers to the multiple contexts and on-the-go (Vavoula et al. 2009). Mobile learning is also personalised; learners are able to use mobile technologies in ways that cater to their own individual experiences and history and that are specific to the temporal and physical context of the learning moment (Kearney et al. 2012). Finally, mobile learning has the ability to be collaborative (Clough et al. 2008). Whether connecting with other people in the room or across the world, mobile devices have become increasingly capable of enabling collaboration. Regardless of which conception of mobile learning they have used, however, studies have overwhelmingly championed the use of mobile devices in enabling learning. For example, Hwang and Wu (2014) reported that 83% of studentlearning achievements.

Engagement with Science in Informal Learning Settings museum and informal science learning studies, research on the closely-related term engagement has been much less prevalent. This study takes the position that understanding engagement is a fundamental precursor to understanding learning. This has been echoed in the work of a small number of researchers who have argued for the significance of visitor engagement or highlighted a connection between learning outcomes and engagement (Rennie et al. 2003; Rennie and Johnston 2004; Barriault and Pearson 2010). Most influential in this study, has been the work of Barriault and Pearson (2010), who closely examined the meaning of engagement by presenting a framework of seven observable visitor engagement behaviours in science centres, grouped into three levels of engagement leading to learning. Modifications, however were necessary for this project in order to address the particular context of the research a natural history museum instead of a science centre (science centres have more of a focus on interactivity). Additionally, despite the framework bescience within their framework. Figure 1 presents the final modified version of the framework used to measure changes in the visitor engagement experience.

Figure 7. Modified Science Engagement Framework

362

3 Methods Data Collection The objective of this study was to investigate how the use of personal mobile technologies in a natural history museum context might affect the science engagement experiences of adults. Ethical approval was grante (LRS-14/15-0766) prior to conducting the research. Over the course of two phases a mixed-methods approach was used, specifically in order to attempt to increase the validity and reliability of the data through methodological triangulation (although other attempts to address validity and reliability were made as well). The research conducted in Phase 1 consisted of a paper questionnaire administered at the Natural History Museum in London, aimed at answering RQ1 and RQ2 while also helping to build an informed and effective Phase 2 intervention. The questionnaire was initially tested in a pilot phase (n=12). Pilot participants were briefly interviewed on their impressions and understandings of the questionnaire after filling it in. After revisions were made, the final questionnaire was administered to adult visitors around the museum (n=60) May through June 2015. Visitors had to meet three eligibility requirements, that they: were at least 18 years old, spoke relatively fluent English, and owned a personal mobile device such as a smartphone or tablet. ldevice to engage with an exhibit (such as taking a photo, making a note, looking up information, etc.), and were then instructed to attempt to use their own mobile devices to engage in at least three different ways within the exhibition. Twenty participants were recruited at the exhibition entrance, both pairs and individuals, February through April 2016. Participants had to meet the same three eligibility requirements as in Phase 1 (although visitors had to not only own a mobile device in Phase 2, but have it with them). Both groups were observed and audio recorded while visiting the exhibition, and interviewed for 20-25 minutes afterwards. The interviews sought to gain a better understanding of who the participants were, how they felt about science and technology, and how they might normally use their own devices. The interviews also investigated engagement experiences (mobile and non-mobile) in greater depth. Using the visit field notes and a series of exhibit photos path was re-traced through the exhibition. They were asked questions about exhibits where at least level 2 or 3 engagement behaviours had been noted, such as how they felt about the exhibits, why they stopped there, how they thought the exhibit related to science or natural history, and if they had used their mobile devices why they chose to use their device at that moment, if they found it distracting, etc. Finally, participants were sent email questionnaires 3-5 months later to determine if the participants recalled or had revisited any of their instances of engagement. To-date 8 followup questionnaires have been received.

Data Analysis Phase 1 data has begun to be analysed using basic statistical analysis, looking for overall trends in order to include t-tests and factor analysis. The analysis of Phase 2 field notes, visit recording transcripts, interview transcripts, and follow-up questionnaires has so far used two different approaches. First, they were each analysed using open coding analysis (using NVivo 11). This approach is somewhat related to grounded theory, although it also acknowledges prior review of the literature and that research questions were in mind when the analysis was carried out. Second, the interview transcripts, field notes, and visit recordings were analysed using the Modified Science Engagement Framework, looking for evidence of the observable engagement meta-

363

4 Preliminary Findings RQ1 and RQ2: Perceptions, Attitudes, and Usage Habits The majority of participants in Phase 1 reported feeling dependent on their personal mobile devices in general. However, when asked how dependent they felt while in a museum, the majority reported that they did not feel dependent on them (see Table 1). Participants in Phase 1 were also asked to report on how often they used their mobile devices in general and in the museum in eight different categories: social media, taking photos/videos, sharing photos/videos, communicating with others, looking up information, personal organisational purposes, entertainment, and learning (labelled a-h respectively in Figure 2). Although the most popular method of mobile device use changed whether the participants were responding about general usage or usage in the museum, in every category the overall reported usage decreased in the museum context.

Phase 1 and Phase 2 findings also illuminated the importance of photo-taking to participants. Nearly all of the participants in both phases reported using their personal mobile devices to take photos to some extent. In fact, in the majority of mobile engagement instances in Phase 2 (both prompted and unprompted) participants chose to involve photo-taking in some way (snapchat, Instagram, etc.) Finally, Phase 2 qualitative analysis also yielded further insight into participant attitudes towards mobile devices in museums. The idea that museums were not a place that participants felt they needed or wanted to use their devices was consistright? You

RQ3: Changes in Science Engagement Although the Phase 2 data is still in the process of being analysed, from the data analysed so far using the Modified Science Engagement Framework, there is a clear correlation between higher engagement levels and the use of personal mobile devices to engage. Comparing all of the instances of mobile engagement, the majority achieved either level 2 or 3 as the highest level of engagement. However, through further analysis, it was found that in the majority of those high level mobile engagement instances, that highest level was reached prior to the use of the mobile device, and no level increase was found during or after the usage, thereby eliminating a causal effect: mobile devices generally did not cause visitors to more deeply engage with the science content. Rarely was there any additional engagement with the science while or after using the device, and if there was, it was not usually a change in level, but rather the appearance of additional behaviours from that level. Additionally, during the in-person interviews participants were asked how they felt about the engagement instances, and specifically if they felt the mobile device usage changed their experience or enhanced their understanding of the science. In the majority of mobile engagement instances the participants reported that they did not feel as if the devices made any difference to their scientific understanding of the exhibits. This corresponded with the results of the Modified Science Engagement Framework analysis described above. Interestingly, preliminary results from the 8 post-visit questionnaires received so far (3-5 months afterwards), show that mobile devices might potentially be helping visitors to have more memorable experiences 364

overall. Those visitors who used their mobile devices to engage tended to be able to recall in greater detail general things they may have felt or observed, their memories of their visit to the exhibition, and how the exhibit related to their lives or other topics they were interested in. However, the responses did not seem to suggest that visitors using mobile devices were any better at recalling the science (such as scientific facts or concepts) they had come across and discussed during the interviews. In fact the majority of post-visit questionnaires did not include references to any science from the exhibition. Additional visitor post-visit questionnaire responses (still pending) will help to gain further insight into this preliminary finding.

5 Discussion and Conclusion Given the preliminary findings above, did the use of personal mobile devices change the science engagement experience of adults at the Natural History Museum? These results suggest that there is a correlation between mobile device use at the exhibits and higher levels of science engagement, however, the use of mobile devices did not facilitate an increase in the level of science engagement. One explanation for the correlation might be that visitors chose to use their mobile devices in instances where they were already interested, not instances where they felt they needed their device to learn. Perhaps visitors see their mobile devices as tools for expression of interest rather than facilitation. If so, this might explain why the devices were used predominantly durin change during or after mobile device use. Additionally, from analysis of interviews, it appeared as if participants used their personal devices with three specific motivations: to capture, personalise/take ownership of, and to share. Interestingly, these three motivations focussed predominantly on either the cultural, historical, artistic, or personal connections with the exhibits, suggesting that while mobile devices did not facilitate increased levels of engagement with scientific facts and concepts, perhaps they had a greater effect on more general engagement. The initial results of the post-visit questionnaires discussed above may indeed help to further support this notion. In addition to completing analysis, there are still many issues that need to be further explored from the perspective of both academia and museum practice. Is interest expression separate from engagement, or could expression just be another aspect of engagement? Should the practitioners shaping the informal science learning experience focus on helping people engage with science specifically, or, given this research, aim to simply help visitors broaden their overall engagement experiences? Should museums look to provide visitors with digital offerings that tap into the capturing, sharing, and personalisation motivations, or continue to focus on developing apps to try and facilitate science learning specifically (as opposed to general engagement)? Given the research highlights not only a need to potentially re-visit claims of mobile devices facilitating learning, but to re-evaluate the ways in which mobile technologies might best be utilised to improve engagement in informal science learning settings.

6 References Barriault C and Pearson D (2010) Assessing Exhibits for Learning in Science Centers: A Practical Tool. Visitor Studies, 13, 90-106. Bell P, Lewenstein B, Shouse AW, Feder, MA (2009) Learning Science in Informal Environments: People, Places, and Pursuits, National Academies Press. Bryman, A (2012) Social Research Methods, Oxford, Oxford University Press. Chen YS, Kao TC, Sheu JP (2003) A mobile learning system for scaffolding bird watcching learning. Journal of Computer Assisted Learning, 19, 347-359. Clough G, Jones AC, McAndrew P, Scanlon E (2008) Informal learning with PDAs and smartphones. Journal of Computer Assisted Learning, 24, 359-371. Evans C (2008) The effectiveness of m-learning in the form of podcast revision lectures in higher education. Computers & Education, 50, 491-498. Hedberg JG (2014) Extending the pedagogy of mobility. Educational Media International, 51, 237-253.

365

Hwang GJ, Wu PH (2014) Applications, impacts and trends of mobile technology-enhanced learning: a review of 2008 2012 publications in selected SSCI journals. International Journal of Mobile Learning and Organisation, 8, 8395. Kearney M, Schuck S, Burden K, Aubusson P (2012) Viewing mobile learning from a pedagogical perspective. Research in Learning Technology, 20. Rennie LJ, Feher E, Dierking LD, Falk JH (2003) Toward an agenda for advancing research on science learning in outof-school settings. Journal of Research in Science Teaching, 40, 112-120. Rennie LJ and Johnston DJ (2004) The nature of learning and its implications for research on learning from museums. Science Education, 88, S4-S16. Schwan S, Grajal A, LeWalter D (2014) Understanding and Engagement in Places of Science Experience: Science Museums, Science Centers, Zoos, and Aquariums. Educational Psychologist, 49, 70-85. Sharples M, Taylor J, Vavoula G (2007) A Theory of Learning for the Mobile Age. In: Andrews R and Haythornthwaite C (eds.) The Sage Handbook of Elearning Research. London: Sage. Stevens D and Kitchenham A (2011) An Analysis of Mobile Learning in Education, Business, and Medicine. In: Andrew K (ed.) Models for Interdisciplinary Mobile Learning: Delivering Information to Students. Hershey, PA, USA: IGI Global. Stocklmayer SM, Rennie LJ, Gilbert JK (2010) The roles of the formal and informal sectors in the provision of effective science education. Studies in Science Education, 46, 1-44. Vavoula G, Sharples M, Rudman P, Meek J, Lonsdale P (2009) Myartspace: Design and evaluation of support for learning with multimedia phones between classrooms and museums. Computers & Education, 53, 286-299.

366