Cleo Etec 500 Research Analysis Critique 2

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY 1

Computational Thinking and Scratch Jnr.: A Case Study-A closer look at student thinking. Research Analysis & Critical Review

Cristina Leo ETEC 500 University of British Columbia August 19, 2018

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY 2 Research Analysis 1. The purpose of this study was to explore young students’ thinking when completing basic coding tasks using Scratch Jnr. on the iPad. As governments push for the inclusion of coding into the curriculum, the researcher identified a gap that included empirical evidence and extensive literature review (knowledge) exploring benefits from programming to the enhancement of general thinking skills. Therefore, the researcher’s purpose was to “investigate the types of thinking young students used, and when they used them within computational tasks, to learn more about whether such tasks create useful opportunities for students to exercise and possible enhance an array of thinking capabilities.” The study was guided by the following research questions: “What types of general thinking skills were evident, and how were they applied, in these students’ computational work? What relationship exists between the exercise of general thinking skills and these students’ computational work?” (Falloon, 2016)

2. The most significant prior literature used to ground this study included Krathwohl’s (2002) revision of Bloom’s Taxonomy (Remember, Understand, Apply, Analyze, Evaluate and Create) and Brennan and Resnick’s (2012) computation thinking (CT) frameworks. As well, the prior research of Paper and Wing. Pea & Kurland (1984) and Mayer et al. (1986) work both discuss the lack of evidence determining whether the thinking skills used in coding are transferable to the enhancement of general thinking skills. The author’s most compelling argument for further study in this area is based upon this gap in research.

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY 3

3. The most significant construct was the computational thinking skills aligned with the frameworks of Krathwohl (Bloom’s revised taxonomy) and Brennan and Resnick (CT skills). This was operationally defined as the observable indicators of thinking skills through verbal responses embedded and aligned with the general thinking and main area of application. The following thinking skills, Remember, Understand, Apply, Analyze, Evaluate and Create were of particular importance.

4. The research was qualitative as data was collected by observing children’s dialogue and communication captured using a video recording app on the iPad’s. This non-intervention research design did not involve an intervention or experimental manipulation, but rather aimed to uncover relationships between computational thinking and frameworks in thinking, in particular what types of general thinking skills were used and how they were applied. This was a theory based research study utilizing previous theoretical frameworks.

5. The variables included the thinking skills present from the frameworks of Krathwohl (Remember, Understand, Apply, Analyse, Evaluate, Create) and Brennan and Resnick.

6. The research design was a qualitative, case study approach to investigating the questions of what and how. Relying on the theoretical framework of Krathwohl and Brennan and Resnick, this field study was conducted in two primary year 1 and 2 classrooms in New

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY 4 Zealand. Research was done in a non-controlled environment, where students were able to work uninterrupted, within a normal classroom environment structure and routine.

7. The researchers recorded data using an embedded display and audio capture app installed on the student’s iPad Air tablets. The app recorded the students’ on screen activity as well as their verbal discussions during their time coding the geometry task using Scratch Jnr. Using Studiocode video analysis software allowed for a double blind code of the sample data, to control for bias and any confounding variables.

8. The sample in this study included two primary year 1 and 2 classrooms from an elementary school in New Zealand. There were 32 students in total, making up the 16 pairs (14-year 1 students=2 months of schooling, and 18-year two students=14 months of schooling). None of the selected students had participated in any previous research conducted. A balance of data from boy/boy, girl/girl, and girl/boy pairs was collected.

9. The researcher stresses that because this study was not about measuring or assessing progression​ in thinking skills but rather the ​investigation​ into the ​types​ of thinking present and ​used​ while students completed a computational task (coding in a geometry lesson) the authors relied on two separate coders to identify specific thinking skills. Using the camera app as a tool to record data was reliable in accurately verifying language used. The Kappa results were rated as ‘good’ according to the Landis & Koch (1977) score.

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY 5 10. The researchers acknowledge the limitations of the scope of their study and point to the need for continued research into the study of transferability of thinking during computational tasks to general thinking. They do acknowledge that through this study, evidence suggests that benefits from working collaboratively on tasks, students benefit from coding not only for those interested in vocational aspirations but in the broader contributions towards student learning. Discussion around the benefits of coding in schools suggests links to the core competencies, such as communication and critical-thinking.

11. The sample data was coded using a double blind code using Studiocode video analysis software. To begin, of the 21 hours of recorded data, the coders selected and analyzed only 9 ½ hours of data. Selection was based upon the following criteria; balance of gender pairs (boy/boy, girl/girl, boy/girl), data from both year 1 and 2 paris, and data from work on all sets. First, research coders looked to identify if any evidence existed of students using Krathwohl’s revised Bloom Taxonomy and second, how they were linked to the components of Brennan and Resnick’s framework, they also noted the amount of times these thinking skills were displayed. Raw field data was converted into fractions of the day to allow for accurate percentages of the total time spent on each coding task. From the observed recorded data, evidence (major findings) suggests that students did display different types of thinking skills and how they were used was often in a collaborative nature to share information between pairs, to solve the coding activity tasks, through problem solving.

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY 6

12. The most important points in the discussion section of the research report speak to the fact that increased research is needed into the transferability of linking thinking skills during computational tasks towards generalized thinking skill development as a whole. The authors concluded that students who participated in the coding of the geometry tasks did demonstrate thinking aligned with the revised Bloom’s taxonomy by Krathwohl and the computational thinking framework of Brennan and Resnick. The author notes that important in this study was the identification of the need to explore the thinking that occurs during operation (when students are coding) or more specifically understanding the cognitive dimensions of programming. Also, the study revealed evidence that students engaged in collaborative, team-based work, that relied on self-management skills proved to be the foundational skills that led to higher order thinking. Overall, the author notes that when integrating coding into the curriculum, these additional benefits go beyond preparing students for a future vocation in programming, but rather developing skills for student success in learning in general.

Critical Review Purpose, Contribution, & Significance Within a global context, governments have recently made changes to the curriculum of education, emphasising the implementation of computational thinking and coding into the K-12 curriculum. Garry Falloon’s 2016 research article, “An analysis of young student’s thinking

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY 7 when completing basic coding tasks using Scratch Jnr. on the iPad” sheds light into the types of thinking students engage in when completing computational tasks.

The purpose of this study was “not to measure or assess progression in thinking skills development” but rather to “investigate the ​types​ of thinking young students used, and ​when​ they used them within computational tasks, to learn more about whether such tasks create useful opportunities for students to exercise and possibly enhance an array of thinking capabilities.” (Fallon, 2016)

Falloon recognizes the theoretical work of Papert and Wing regarding computational thinking skills, but also notes the research of Pea and Kurland (1984b) pointing to the “lack of empirical studies exploring possible transfer of problem solving in other domains.” From this knowledge gap, Falloon arrives at his research questions in which he investigates the ​types​ of thinking students display when solving computational tasks (e.g. coding a geometry task using Scratch Jnr. on iPad) and ​how​ these thinking skills are applied. The theoretical framework of Krathwohl’s revised Bloom’s Taxonomy and Brennan and Resnick’s framework for studying and assessing computational thinking form the basis of his analysis framework (General thinking and main area of application). He also seeks to identify if a relationship exists between general thinking skills and computational task work.

Methodology

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY 8 Embedded in the theoretical framework of thinking skills of Krathwohl and Brennan and Resnick, Falloon’s research methods include a qualitative case study, gathering data from two primary (K-6) year 1 and 2 classrooms (5 and 6 year olds) in a school New Zealand.

Students were asked to work in pairs and create shapes and letters using the Scratch Jnr. program on the iPads. Student’s voices demonstrating their thinking skills/discussions were recorded as well as screen captures of their work. Some prior coding experience was conducted before embarking on this computational task. From the 32 students, data was collected over 5 sessions ranging from 25 to 40 minutes each. Students worked in pairs due to the limited amount of iPad devices but also to provide opportunity for discussion and recordable thinking skills with the embedded display and audio capture app installed on the devices. Data samples included a balance of boy/boy, girl/girl, and girl/boy pairs, both year 1 and 2 students as well as data representing the student’s work on all the set tasks for analysis.

In regards to the coding of the data collected, “sample data was double blind coded using Studiocode video analysis software.” (Falloon, 2016) A total of 9 ½ hours of video of the 11 of 16 pairs were individually imported into the program. Researchers also separately “coded 3 hrs of the same data to enable an inter-rater agreement calculation”. Using only coding that both researchers agreed upon was used to calculate rater-agreement. Within case studies, one methodological issue based on the observations was what was not recorded while using the iPad app. It is possible that some discussions were missed due to the absence of researcher in the room. As well, increased opportunity for triangulation would have been useful.

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY 9 Weaknesses One of the weaknesses of this case study is in the sampling of participants. The author notes that one of the teacher’s had already participated in previous studies conducted by Falloon. This potential form of bias may have altered the way in which the teacher provided scaffolding during computational task time using the iPads. The lack of generalization of these findings to extend into other classrooms can also be seen as a weakness. The coders themselves as instruments can be seen as potential weaknesses of the research. In particular, the data that was not used due to lack of consensus, would have benefited from another coder to help with reliability and validity of observable data. Not mentioned in the study was the philosophy of grading of the assignment. This could have altered the mentality of the children to take-risks in their thinking. Considering the Hawthorne effect, it was not mentioned if students were aware that they were participating in a research study, potentially altering their behaviour. Finally, since this study was conducted in New Zealand, taking into account the cultural differences, learning styles, etc. may be considered an extraneous variable.

Strengths Evident in the final discussion and conclusion of the research, Falloon points to the fact that additional research is needed to check for generalizability of transfer of thinking skills while students are engaged in computational tasks. One strength in this research is the combination of general thinking and main area of application, noting that thinking skills don’t always occur strictly in isolation. Falloon’s identification of the need to research coding in education is timely in the event of changes in curriculum. Finally, another benefit of this research was the discovery

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY 10 that many students engaged in risk-taking, developing grit through team-work to problem solve the computational task. The findings also supported the importance of core competencies, such as collaboration and self-management skills, also being highlighted by government curriculum.

Summary Recommendations Overall, this research sought to discover the types of thinking skills students use when challenged with a computational task and revealed that more studies are needed to fill in the knowledge gap. Falloon’s work demonstrates that student’s thinking does align with the framework of Bloom’s revised taxonomy by Krathwohl, and Brennan and Resnick’s framework of computational thinking when coding, even at an early age. This research reveals that rather than focusing on transfer theories of thinking skills, which the author states act as a distraction, emphasis should be placed on studying the thinking skills that occur during student’s minds while they are coding (present tense), analysing their cognitive dimensions of programming. In conclusion, the value of this research takes into account the demands to include coding and computational thinking instruction in the classroom, and the value of this skill development in overall student learning.

Running head: COMPUTATIONAL THINKING AND SCRATCH JR: A CASE STUDY 11 References Brennan, K. & Resnick, M. (2012) New frameworks for studying and assessing development of computational thinking. ​Paper presented at AERA​, Vancouver, BC. Falloon, G. (2016). An analysis of young students' thinking when completing basic coding tasks using scratch jnr. on the iPad.​ Journal of Computer Assisted Learning, 32​(6), 576-593. doi:10.1111/jcal.12155 Krathwohl, D.R. (2002). A revision of Bloom’s taxonomy: An overview. ​Theory Into Practice, 41, ​212-225. Suter, W. N. (2012).​ Introduction to educational research: A critical thinking approach​ (2nd ed.). Thousand Oaks, California: SAGE Publications.