Reframing Assessment Ed Media 2008

“Reframing Assessment: Using social software to collect and organise learning” Anne Bartlett-Bragg Faculty of Education University of Technology Sydney Sydney, Australia [email protected]

Abstract: Assessment is a fundamental ingredient in the teaching and learning process, yet there is an increasing body of literature expressing dissatisfaction with current assessment policies and practices in higher education that challenges the appropriateness of assessment to address the future needs of students in a rapidly changing information rich environment. This paper will provide a framework for designing assessment with social software and examples of options for alternative, authentic assessment methods that collect and organise evidence of learning, suggesting that this process can foster the development of self-directed learners, resulting in discretely embedded learning activities that promote problem-solving and knowledge development beyond the boundaries of the classroom and institution.

Conventional assessment practices tend to focus on students producing work that demonstrates knowledge of subject content, or a culture of evidence that is reinforced when learning is restricted to tight, cost efficient methods that relate only to subject matter taught (Shavelson, 2007). The process is supported by the rationale that to provide motivation, the learners’ attention is directed to content that is considered to be important within the subject being studied. This approach however, limits how and what is learned, and the process of learning itself (Boud & Falchikov, 2007). Concurrently, early implementations of e-Learning systems in educational institutions tended to focus upon the use of technology as an administrative tool to enhance efficiencies and processes with learner management systems testing and recording results of students. These strategies in themselves are irrelevant to the learning process and merely reinforce the controlled, transmission – acquisition approach to learning and assessment practices. In addition, there is little evidence to suggest that incorporating these strategies and technologies into existing learning environments has resulted in any significant change in the learning outcomes (Zemsky & Massy, 2004). The need to review assessment processes is becoming an expanding body of literature with two dominant perspectives: firstly – the literature that views assessment as the major influence on learning and how this effects student approaches to their work and studies. From this perspective arises the call to rethink the role of assessment and the learning processes, calling for self-assessment, peer feedback, portfolios, and group work that encourage learning for the future and as an act of informing judgement (Boud & Falchikov, 2007). And secondly – the integration of technology and associated impact on assessment processes. Extending now into calls for the use of social software to provide learners with digital spaces where they can interact, explore, and construct an individualised approach and manage their own learning. (Attwell, 2005; Bartlett-Bragg, 2007; Owen, Grant, Sayers & Facer, 2006;) Both perspectives align to the findings from my PhD research and teaching practice as a University lecturer. The process of students working with social software has evolved into more than a learning strategy, but also a multi-faceted assessment strategy that incorporated self-assessment, peer feedback, and the development of self-managed artefacts to be presented as a portfolio at the completion of a semester. In the University context, students tend to restrict their study activities to only what is required to be addressed by the assessment task, resulting in a short term point of reference with limited engagement beyond the semester timeframe (Boud, 2007). While research conducted into the use of self-publishing technologies (weblogs and wikis) has demonstrated the potential that social software integrated into the students’ learning activities presented an opportunity to extend the learning process and incorporate

assessment as a discrete activity, rather than the primary focus of the subject being studied (BartlettBragg, 2007). How can social software be used to reframe assessment processes? Self-publishing software, such as weblogs and wikis, enable the student to gather a collection of artefacts, develop critical reflection skills, receive peer feedback and potentially engage with a network beyond the classroom and educational institution. Additional social sharing software applications such as social bookmarking further encourages the students to collect, distribute and organise resources considered valuable not only to themselves, but also their network. Aggregation software provides a central collection platform where the student can subscribe and collect items of interest and be notified of updates that can be reviewed at a time convenient to them. Through the process of developing a personalised learning environment, the student is taking responsibility for the collection and presentation of items in a variety of modes for final assessment, while feedback and comments from peers and a network that may include participants beyond the boundaries of the classroom facilitates the potential for self-directed learning to continue beyond the timeframes of formal studies. How can social software foster self-directed, lifelong learning strategies? Social software can provide opportunities to engage with the content, not just the assessment task. Additionally, discretely embedded in learning activities are digital literacies, personal information management and self-directed strategies that are transferable to contexts other than the subject. The underpinning frameworks that have informed the development of pedagogical practices towards action-oriented, self-directed learning activities using social software applications are outlined below: 1. Pedagogical framework for self-publishing with social software – Bartlett-Bragg (2007) A pedagogical framework was developed and tested in my PhD research that provides learning and assessment strategies for the integration of social software into learning environments (Bartlett-Bragg, 2007). The objective of the pedagogical framework is to facilitate the development of independent learners allowing them to become proficient in the development of collective learning networks using weblogs and associated social software technologies. The framework can be viewed as an enabler where multi-linear pathways draw the focus not on the software or technology selected by the educator to create the learning environment, but on the social aspects of the learning process and strategies to support the learning experience. Refer to Figure 1 for a graphical representation.

5: DISTRIBUTED KNOWLEDGE

•Collaborating • Distributing • Networking 2: INTERPRETATION

• Structure •Personalisation

1:ESTABLISHMENT

3: REFLECTIVE MONOLOGUES

• Identity • Writing • Publishing

• Concepts • Context-examples • Set-up

4: REFLECTIVE DIALOGUES

• Writing • Socialising • Networking

Figure 1: A pedagogical framework for self-publishing with social software 2. Modes of Teaching – Baumgartner (2004) The educators’ existing pedagogical practices developed through formal studies or influenced by organisational structures can inhibit the development of learners towards participation within social software environments and informal learning activities. Baumgartner’s (2004) three prototypical models for teaching provide a valuable framework to review and reframe pedagogical strategies that enable the educator to understand their own practice and review their approach to designing learning and assessment. Refer to Table 1. Mode 1: Transfer (Directed Teaching) Programmed instruction

Mode 2: Tutor (Facilitated Learning) Problem solving

Mode 3: Coach (Informal Guide) Complex simulations

To teach, to explain

To observe, to help, to demonstrate

To co-operate, to support

Production of correct answers

Selection of methods and its use

Realisation of adequate action strategies

To know, to remember

To do, to practice

To cope, to master

Transfer of knowledge

Presentation of predetermined problems

Action in real situations (complex and social)

Table 1: Modes of Teaching Educators implementing learning and assessment activities in a Mode 1 mindset will be reinforcing a model that transmits content to an individual student in an asynchronous, self-paced style where the design is focused on outcome, results driven, directed learning. While educators designing learning and assessment activities in a Mode 3 mindset will be enabling the students to explore the potential presented by the integration of social software. 3. Scheme for developing informed judgement - Boud and Falchikov’s (2007) Boud and Falchikov’s (2007) scheme to develop informed judgement provides a framework for reviewing assessment tasks to emphasise evaluative expertise. The framework overlays the previous models and assists the educator to review their designs. The key elements are outlined as five overlapping components: 1) Identifying self as an active learner; 2) From known to need - identifying own level of knowledge and the gaps; 3) Practising testing and judging; 4) Developing judgement skills over time; 5) Embodying reflexivity and commitment. The key findings from the PhD study focused on the experiences of the adult learner using social software that was integrated into their learning and assessment processes and provides potential for educators to re-frame their pedagogical strategies to facilitate students’ development towards selfdirected learners. An important finding indicated that the inhibitors to effective implementation of social software into practice required pedagogical strategies to enable the educator to neutralise the potential negative effects on the learners. These inhibitors are categorised into three broad areas: organisational inhibitors, including cultural and technical infrastructure; educators’ inhibitors; and learners’ inhibitors. The frameworks mentioned previously support the educator with the design and implementation of alternative assessment strategies, and strategies to manage the learner’s inhibitors, but do not address the organisational cultural inhibitors. Table 2 outlines how the three frameworks can be overlayed to create processes for learning activities that directly inform assessment processes.

Bartlett-Bragg (2007) Pedagogical Framework

Baumgartner (2004) Modes of teaching

1. Establishment Learners are actively creating their personalised learning environments with social software eg. weblogs, wikis, social bookmarking and aggregation.

Mode 2: Educator is providing assistance but encouraging learners to build their own environments. Some strategies may require demonstrations of abstract concepts.

2. Interpretation Learners are developing a structure and adapting to their perceived needs. 3. Reflective Monologues Learners are publishing to their software platform and establishing their identity, or finding their voice.

Mode 2: Educator is observing and supporting through demonstration as learners adapt to their perceived needs. Mode 3: Educator is providing support and activities to assist learners to progress in a selfdirected manner.

4. Reflective Dialogues Learners are extending their learning environment by developing social networks.

Mode 3: The educator introduces activities to stimulate action towards further establishing connections with other learners.

5. Distributed knowledge artefacts Learners are collaborating with others, distributing their work, and gathering artefacts for review and reflection.

Mode 3: At this point the educator could be viewed as the conduit and a co-participant in the learner’s network – providing feedback towards future development.

Boud & Falchikov (2007) Scheme for developing informed justdgement 1. Identify self as an active learner The process of setting up the various software applications actively involves the learner in the selection and building of their environments – establishing engagement in the processes. 2. From known to need Learners are developing a structure within their software environment based on their perceived needs. 3. Testing and Judging As the learners perform writing and publishing tasks, feedback from the educator and peers informs further development and self-efficacy. 4. Developing skills over time A period confidence grows with the self-publishing techniques being practised, the learners establish more concrete practices. 5. Embodying reflexivity and commitment As both peer and educator feedback is received the learners continue to adjust and improve their work. While gathering artefacts for final assessment.

Table 2: Overlay of the 3 frameworks with examples Reframing assessment is not solely about integrating software into the process of assessment, it is also about reviewing current assessment philosophies and determining how assessment can foster learning. Social software affords educators the opportunity to personalise the overall learning experience, including assessment, to enable the learner to collect and organise information and artefacts from different contexts and situations, and demonstrate and reflect upon their skills and achievements (Attwell, 2007). It is timely, to reflect on these issues as our educational institutions espouse the values of lifelong learning and prioritise practices of collaboration, reflection, personalisation, knowledge sharing and networks into vision statements and strategic planning documents. Notwithstanding the importance of social software as the enabler in these processes, without re-framing our practice educators are not likely to realise the opportunity to incorporate social software and integrated communication networks that extends the learning beyond the physical boundaries and opinions of the classroom context.

References: Attwell, G. (2007). The Personal Learning Environments – the future of eLearning?, eLearning Papers, 2(1), 5. Retrieved February 10, 2007, from http://www.elearningpapers.eu/index.php?page=home&vol=2 Bartlett-Bragg, A. (2007). Reflections on pedagogy: Understanding adult learners' experiences of weblogs. In T. Burg, & J. Schmidt (Eds.), BlogTalks reloaded (pp. 119-241). Austria: Herstellung. Baumgartner, P. (2004). The zen art of teaching – Communication and interactions in e-Education. Workshop presented at ICL2004, Kassel University, Villach, Austria. Retrieved November, 2004, from http://www.elearningeuropa.info/extras/pdf/zenartofteaching.pdf

Boud, D. (2007). Reframing assessment as if learning were important. In D. Boud, & N. Falchikov (Eds.), Rethinking Assessment in Higher Education, (pp.14-25). London:Routledge. Boud, D. & Falchikov, N. (2007). Assessment for the longer term. In D. Boud, & N. Falchikov (Eds.), Rethinking Assessment in Higher Education, (pp. 4-13). London: Routledge. Owen, M., Grant, L., Sayers, S., & Facer, K. (2006). Social software and learning. Bristol, UK: Futurelab. Retrieved July 18, 2006, from http://www.futurelab.org.uk/resources/publications_reports_articles/opening_education_reports/Openi ng_Education_Report199 Shavelson, R. J. (2007). Assessing student learning responsibly: from history to an audacious proposal, Change, 39(1) (pp.26-33). Zemsky, R., & Massy, W.F., (2004). Thwarted innovation: What happened to e-learning and why. The Learning Alliance. Retrieved July 20, 2006 from http://www.thelearningalliance.info/Docs/Jun2004/ThwartedInnovation.pdf