Plenary Lecture Presented At International Conference On Science And Mathematics Education 2008 By Dr.yeap Ban Har

The Role of the Calculator in Formal Mathematics Assessment Yeap Ban Har National Institute of Education Nanyang Technological University Singapore [email protected] Introduction As calculators become increasingly more inexpensive and prevalent, the debate on its use in mathematics assessment, especially in formal assessment, cannot be avoided. As informal assessment and instructional practices tend to take the cues from high-stakes formal assessment practices, this paper focuses on the role of the calculator in such high-stakes formal assessment which often take the form of national tests. In this paper, the term tests refers to standardized tests that are given under uniform conditions. Golden (2000) reported that teachers’ frequent use of the calculator in mathematics instruction reduced students’ ability on computations in end-of-year tests where calculators were not allowed. However, Hembree and Dessart (1986, 1992), in a meta-analysis of about 80 research studies that focused on the effects of using calculators on students achievement and attitude, found that in general average students who used calculators in tandem with traditional mathematics instruction did better in paper-and-pencil tests, both in computation as well as in problem solving. There was neither a positive nor negative effect on low-ability and high-ability students. Ellington (2003) similarly found in another meta-analysis, that the use of calculator did not hinder the development of students’ paper-and-pencil computation skills and facilitated skill development and a more positive attitude towards the subjects than students who did not use calculators. Smith (1997) earlier found similar positive effects of using calculators in a meta-analytic study on researches done in K-12 classrooms. Also, mathematics educators have long acknowledged the benefits of using calculators in mathematics teaching and learning. However, this does not necessarily translate into instructional practices (Brown, Karp, Petrosko, Jones, Beswick, Howe & Zwanzig, 2007). Wilson (2007), in a review on high-stakes mathematics testing, found that such tests had an impact on the implementation of the curriculum. In particular, one study that surveyed American teachers in different states found that teachers in states with high-stakes tests aligned their own tests with the state tests more than teachers in states with low-stakes tests (cited in Wilson, 2007). The same study also found that the high-stakes tests had made them teach in ways that contradicted their ideas of good mathematics teaching. This was especially true for teachers at the elementary grades. Thus, it is seen that mathematics educators have been advocating the benefits of appropriate use of the calculator in the classroom, even in lower grade levels. Meta-analyses of research on the calculator use generally support its role in mathematic teaching and learning. However, its use in the classroom, especially at the elementary grades, is yet to be prevalent and universal. As the effects of high-stakes national tests on instructional practices are well documented, introducing the use of the calculator in such tests may well induce teachers to use them appropriately in the mathematics classroom. The purpose of this paper is to discuss the role of the calculator in such high-stakes test.

Yeap, B.H. (2008). The Role of the Calculator in Formal Mathematics Assessment. Plenary lecture presented at the International Conference on Science and Mathematics Education, October 27 – 29, 2008 at the University of the Philippines.

The next section describes recent development in Singapore on the use of the calculator in national tests. The main part of this part is devoted to a discussion on how calculator can play an important role in helping teachers understand what mathematics instruction should include for it to matter to their students. This is the first step to helping teachers understand how to make mathematics instruction matter to their students. The paper concludes with a proposal of a programme of research into this area as we thread cautiously into introducing the calculator into high-stakes national tests. Recent Developments in Singapore Singapore has always been conservative and cautious in its approach to the use of technology in mathematics assessment. After years of consideration, deliberation and consultation with various stakeholders, the Ministry of Education of Singapore seems to have arrived at a position that while technology is essential in mathematics assessment, a moderate approach is equally crucial. This is illustrated by the introduction of the use of calculator in a significant part of the Grade 6 national examination (PSLE), of the use of calculator in the entire Grade 10 national examination (GCE O-Level), and of the use of graphing calculator in the Grade 12 national examination (GCE A-Level). Previously, calculators were not allowed in the Grade 6 national examination and allowed in only a portion of the Grade 10 national examination, and graphing calculators were not used in the Grade 12 national examination. Assessment guidelines for secondary schools indicate that calculators should be allowed in school tests from Grade 7. It should also be noted that the use of calculator in schools tests is not encouraged for Grades 4 and lower. Also, although it was considered, the use of graphing calculator in Additional Mathematics is not allowed at the Grade 10 national examination. In Singapore, students who have a high proficiency in mathematics opt to study Additional Mathematics which includes calculus. Key Competencies in Mathematics In 2004, eminent American economist Alan Greenspan told the Senate Banking Committee that the standard of living in the US is a function of ability of its people (Mukherjee, 2004). The warning is true not just for the US but also for every other nation. It was noted that in the next decade or so, “3.3 million U.S. service-industry jobs and $136 billion in annual wages will move to India, the Philippines, China and Malaysia, among other countries” according to a study by a US-based consulting company, Forrester Research Inc. (Mukherjee, 2004). By implications, this can only happen if the people in these other countries possess the ability the jobs require. What are the key competencies that our students need to develop for them to be able to participate and benefit from this global participation? We discuss these competencies in this section and we argue how teachers can be persuaded to include such competencies in mathematics instruction in the rest of the paper. There has been a lot of literature on key competencies in mathematics and how their notion has changed with technological advances. For example, the Organization for Economic Cooperation and Development (OECD) defined mathematical literacy as “an individual’s capacity to identify and understand the role that mathematics plays in the world, to make well-founded judgements and to use and engage with mathematics in ways that meet the needs of that individual’s life as a constructive, concerned and reflective citizen." (OECD, 2003, p.24). As Schoenfeld (2007) argued, while knowledge is a necessary part of mathematical proficiency, it is not sufficient. The ability to use problem-solving strategies, metacognitive ability as well as beliefs and dispositions must be part of mathematical proficiency that students develop through mathematics learning. Kilpatrick, Swafford and Findell (2001) described mathematical proficiency as comprising of inter-connected strands. Yeap, B.H. (2008). The Role of the Calculator in Formal Mathematics Assessment. Plenary lecture presented at the International Conference on Science and Mathematics Education, October 27 – 29, 2008 at the University of the Philippines.

They are conceptual understanding - comprehension of mathematical concepts, operations, and relations, procedural fluency - skill in carrying out procedures flexibly, accurately, efficiently, and appropriately, strategic competence - ability to formulate, represent, and solve mathematical problems, adaptive reasoning - capacity for logical thought, reflection, explanation, and justification, and productive disposition - habitual inclination to see mathematics as sensible, useful, and worthwhile, coupled with a belief in diligence and one’s own efficacy. In Singapore, the problem-solving curriculum has been in use since 1992. It has been revised twice, in 2001 and 2007. A comparison of the 2001 and 2007 versions showed that there are several differences. Firstly, visualization is included as a skill, suggesting its crucial role in mathematics learning. Secondly, reasoning, communication and connections as well as applications and modelling are included as processes. Previously, only thinking skills and heuristics were included. Thirdly, perseverance is added to a short list of essential attitudes, suggesting the inclusion of problems that require more time and exploration to solve. These revisions suggest that there are mathematical competencies that are becoming increasingly essential in mathematics teaching and learning and must received increased attention. Such competencies include conceptual understanding, metacognition, connections, modelling, visualization and communication. These competencies stand out as being human abilities – these are competencies that machines and computers do not possess. An emphasis on these competencies will necessarily imply a de-emphasis on competencies that are better executed using machines and computers, such as tedious computation and recall of a large amount of information. In the next few sections, examples from three categories of test items are used to show how the use of calculators during formal assessment can serve as a catalyst for teachers to include key competencies in instruction. Investigative-Type Items In test that prohibits the use of calculators, investigative-type items are seldom included as these items often require numerous computations and explorations, which are timeconsuming and tedious without the aid of a calculator. Figure 1 shows the item Reverse. It is based on the topic of multiplication of 2-digit numbers. In the Singapore curriculum, students learn to multiply, without the calculator, a 3-digit number with a 2-digit one in the fourth grade. The 2-digit numbers 13 and 62 are such that their product (13 × 62) and the product of the numbers formed by reversing the digits of 13 and 62 (31 × 26) are the same. The digits used, 1, 3, 6 and 2, are all different. a. Find five other such pairs of numbers. b. Describe how the digits are related to each other.

Figure 1: Test Item ‘Reverse’ In performing this task, it is necessary to perform numerous computations which can be tedious if a calculator is not used. When a calculator is used, the ability to solve this problem does not depend so much on the ability to perform the multiplication algorithm, which is done by the calculator, but on the ability to choose the appropriate digits for the ones and tens places during the guess-and-check process, which requires good number sense.

Yeap, B.H. (2008). The Role of the Calculator in Formal Mathematics Assessment. Plenary lecture presented at the International Conference on Science and Mathematics Education, October 27 – 29, 2008 at the University of the Philippines.

While it is entirely possible to assess students’ ability to describe relationships during a test that prohibits the use of calculator, the relationships that are available to the test item setters are limited to those that do not require tedious computations. In this test item, it shows how the use of calculator in formal mathematics assessment (a) shifts the focus from procedural understanding to conceptual understanding of large number multiplication, and (b) presents students with opportunities to deal with, and describe, a wider array of mathematical relationships. As Schoenfeld (2007) argued, “teachers feel pressured to teach to the test – and if the test focuses on skills, other aspects of mathematical proficiency tend to be given the short shrift” (p.72). Conversely, if the test focuses on conceptual understanding and includes a wide range of relationships then these will receive attention in classroom instruction. Investigative-type items tend to be somewhat open and require planning on the students’ part. Figure 2 shows the item Repeated Multiplication. It is based on the topic of combined operation. Students learn this topic in the Singapore curriculum in the fifth grade, when the use of calculator is encouraged during instruction and allowed during formal assessment. The product of 3 twos is 2 × 2 × 2 = 8. The tens digit in the product is 0 and the ones digit is 8. The product of 4 twos is 2 × 2 × 2 × 2 = 16. The tens digit in the product is 1 and the ones digit is 6. Find the tens digit and ones digit in the product of (a) 5 twos and (b) 2008 twos.

Figure 2: Test Item ‘Repeated Multiplication’ In completing the Repeated Multiplication task, students need to decide on the data to be collected (the tens and ones digits in the products of a small number of twos), to organize the data in a suitable way (use a table or list), to see a trend or pattern (the digits form a repeated pattern) and to make a generalization (for the tens and ones digits in the product of a large number of twos). Although it is not necessary for the use of a calculator for students to engage in investigative-type tasks, using a calculator provides a better balance between computation and problem solving. Also, it widens the range of problems that students can engage in. In the case of Repeated Multiplication, students get to realize that although the calculator can be used to generate data essential to the solution, it cannot provide the solution. The solution to the tens and ones digits in 2008 twos is found through the human mind’s ability to see patterns and make generalization. Authentic-Data-Type Items In test that prohibits the use of calculators, authentic data is seldom used. Often, the test designers choose numbers which students can handle without involving tedious computation. Figure 3 shows an example of such test item. It involves the percentage of a quantity. The computation can be done mentally. The car park charge is $3. There is a 7% tax added to it. Calculate the car park charge including the tax.

Figure 3: Test Item ‘Car Park Charges’ In itself, there is nothing wrong with using such items for formal assessment. It is only worrying when students are exposed to only this type of tasks simply because tasks such as Receipt (Figure 4) are not tedious computationally without calculator access. Yeap, B.H. (2008). The Role of the Calculator in Formal Mathematics Assessment. Plenary lecture presented at the International Conference on Science and Mathematics Education, October 27 – 29, 2008 at the University of the Philippines.

The receipt for a cup of latte is shown. The total amount includes a 7% tax. McDonald’s Boat Quay 1 South Canal Road S(048508) GST REGN NO:M2-0023981-4 TAX INVOICE 07/10/2008 11:21:46 QTY ITEM 1 Medium Latte

TOTAL 3.40

Eat-In Total (includes tax) Cash Tendered Change

3.40 5.00 1.60

Total includes tax of  Thank You

Find the amount of tax paid for the cup of latte.

Figure 4: Test Item ‘Receipt’ In performing this task, it is necessary to deal with authentic data. As the computation does not always involve clean computations, a calculator is necessary. Extraneous information may be present in authentic sources and students have to select the relevant information to use. Schoenfeld (2007) reasoned that an overdose of word problems that has little or nothing to do with the real world resulted in many students suspending their ability to make sense in solving problems involving the real world. Many typical textbooks problems are basic computation in disguise. As a result, although 70% of students in a study were able to do the computation in this task: An army bus holds 36 soldiers. If 1128 soldiers are being bussed to their training site, how many buses are needed?, only 23% gave the correct answer 32 (Carpenter, Lindquist, Matthews, & Silver, 1983). Even if the word problem takes the typical traditional form, the use of authentic data requires students to make sense of the computation results that the calculator yields. In solving the problem in Figure 5, many students would be able to formulate a correct number sentence ($50 ÷ $3.60). To be successful, they need to be able to make sense of the result (13.888...). Students who failed may give answers such as

, 13.9, 14,

, 27.8, 28 or 13. The use of

made-up data for the sake of clean computation when calculators are not used deprives students in developing this ability to formulate computation answers into problem solution. A store sold 2 tuna puffs for $3.60. Find the largest number of tuna puffs that can be bought with $50.

Figure 5: Test Item ‘Tuna Puffs’ Visualization-Type Items It is possible that there are students who are able to visualize that the required perimeter in the Flowerbed item (see Figure 6) is equal to the total of the length of the straight line and 1.5 times the circumference of a circle and that two shaded parts form a square of sides 7 cm, but unable to perform the computation

or

. The visualization required to

solve the problem cannot be done by a computer while the computation aspect of the problem Yeap, B.H. (2008). The Role of the Calculator in Formal Mathematics Assessment. Plenary lecture presented at the International Conference on Science and Mathematics Education, October 27 – 29, 2008 at the University of the Philippines.

can be done easily with a basic calculator. Students who have the visualization ability should not be handicapped just because he cannot do what an inexpensive calculator can. In tests that do not allow the use of calculators, such students would struggle and many would end up believing that they are not good in mathematics, when in actual fact they are just not good in performing tedious computations. The shaded figure shows a flowerbed which is formed by a straight line and 6 identical quarter circles.

(a) Find the perimeter of the flowerbed. (b) Find the area of the flowerbed. Take π = .

Figure 6: Test Item ‘Flowerbed’ (SEAB, 2008, p.32) Conversely, if a student is not able to perform the necessary visualization, the access to a calculator does not give the student any advantage in solving the problem. In this category, problems require students to have requisite visualization ability. Calculators serve as a computation tool. Conclude In this paper, three categories of test items were selected to illustrate the role of the calculator in formal assessment. Table 1 provides a summary of the preceding discussion. The role of the calculator can be perceived as cognitive, metacognitive and affective. In its cognitive role, the calculator allows repetitive and tedious computations to be done easily, thereby allowing more focus on other cognitive activities such as visualization, generating data, looking for connections, making generalizations and communication. This also allows the inclusion of a larger range of tasks that provides students with opportunities to engage in a wider range of activities, many of which are metacognitive in nature, such as selecting data and making sense of computation answers. In its affective role, the calculator helps students who struggle with computation algorithms develop confidence in mathematics and allows students to appreciate the limitations of the calculator and the superiority of the human mind. When calculators are not used in formal assessment, it is likely that investigative type problems and problems involving authentic data are excluded. As teachers are likely to expose students to problem-types in formal high-stakes test, such exclusion often means students are deprived from mathematics instruction that focuses on competencies that they require to function in a knowledge-based, technological world. The exclusion of calculators in formal assessment, I argue, has a significant impact on the quality of mathematics education students receive.

Yeap, B.H. (2008). The Role of the Calculator in Formal Mathematics Assessment. Plenary lecture presented at the International Conference on Science and Mathematics Education, October 27 – 29, 2008 at the University of the Philippines.

It is also possible that students who are good in visualization but less fluent in algorithmic computations may lose confidence and interest in mathematics. Table 1: Types of Calculator-Type Test Items and the Role of Calculator Category Investigative-Type illustrated by Reverse and Repeated Multiplication

Calculator’s Role The use of calculator allows students to deal with repetitive computations which are common in investigations. The use of calculator allows students to deal with a wider range of investigations including those that include tedious computations. Time saved by performing computations using the calculator is better used to engage students in communication, looking for relationships and making generalization. The calculator can be a tool to generate data for the investigation. The calculator also demonstrates its limits and inferiority to the human mind when generalizations beyond the capability of the calculator are required.

Authentic-Data-Type illustrated by Receipt and Tuna Puffs

The calculator allows the use of authentic data in problems as such data often result in computations which do not give neat answers. The calculator serves as a computation tool. The use of authentic data requires students to make sense of computation answers. Often they also need to handles extraneous information.

Visualization-Type illustrated by Flowerbed

The calculator serves as a tool to help students who are not good at performing routine but tedious computations. The calculator helps students develop a greater confidence in doing mathematics.

Avoiding the use of the calculator may result in students being exposed to a narrow range of problems or problems that do not require them to engage in interpretation of numerical computation. Also, much instructional time is likely to be used to teach and achieve fluency in computations, especially those which are tedious to perform, resulting is less attention to key competencies such as visualization, connections and communication. In conclusion, I argue that the benefits of allowing the calculator in formal assessment far outweigh the concerns associated with the issue. A Research Agenda A research agenda can be developed around the three-facet (cognitive, metacognitive and affective) roles of the calculator in formal assessment. I provide a list of research problems related to this domain. In each of the three problem categories discussed, what roles does the calculator play when students solve problems in paper-and-pencil tests? What are the differences in processes, cognitive and metacognitive, that students engaged in when they solve problems with and without access to the calculator? What are the effects of allowing calculator to be used in formal assessment? What is the impact on classroom instruction? What is the impact on student learning – mental Yeap, B.H. (2008). The Role of the Calculator in Formal Mathematics Assessment. Plenary lecture presented at the International Conference on Science and Mathematics Education, October 27 – 29, 2008 at the University of the Philippines.

computation, communication, problem solving and investigation, among others? What is the impact on students’ beliefs and attitudes? What is the impact on students of different achievement levels? References Brown, E.T., Karp, K., Petrosko, J.M., Jones, J., Beswick, G., Howe, C., & Zwanzig, K (2007). Crutch or catalyst: Teachers’ beliefs and practices regarding calculator use in mathematical instruction. School Science and Mathematics, 107(3), 102-116. Carpenter, T.P., Lindquist, M.M., Matthews, W., & Silver, E.A. (1983). Results of the third NAEP mathematics assessment: Secondary school. Mathematics Teachers, 76(9), 652-659. Ellington, A. (2003). A meta-analysis of the effects of calculators on students’ achievement and attitude levels in precollege mathematics classes. Journal for Research in Mathematics Education, 34, 433-463. Hembree, R. & Dessart, D.J. (1986). Effects of hand-held calculators in precollege mathematics education: A meta-analysis. Journal for Research in Mathematics Education, 17, 83-99. Hembree, R. & Dessart, D.J. (1992). Research on calculators in mathematics education. In J. Fey & C.R. Hirsch (Eds.), Calculators in mathematics education (pp. 23-32) Reston, VA: National Council of Teachers of Mathematics. Kilpatrick, J., Swafford, J., & Findell, B. (Eds.) (2001).Adding it up! Helping children learn mathematics. Washington, D.C.: National Academy Press. Ministry of Education of Singapore (2000). Primary mathematics syllabus. Singapore: Curriculum Planning and Development Division. Available at http://www.moe.gov.sg/education/syllabuses/sciences/files/maths-primary-2001.pdf. Retrieved on September 20, 2008. Ministry of Education of Singapore (2006). Mathematics syllabus: Primary. Singapore: Curriculum Planning and Development Division. Available at http://www.moe.gov.sg/education/syllabuses/sciences/files/maths-primary-2007.pdf. Retrieved on September 20, 2008. Mukherjee, A. (2004). Greenspan worries about math gap. Bloomberg.com (February 17, 2004). Available at http://www.bloomberg.com/apps/news?pid=10000039&refer=columnist_mukherjee&sid=ang rr7YVphls. Retrieved on September 20, 2008. Organization for Economic Co-operation and Development (OECD). (2003). The PISA 2003 Assessment framework: Mathematics, reading, science and problem-solving knowledge and skills. Available at http://www.pisa.oecd.org/dataoecd/46/14/33694881.pdf. Retrieved on September 20, 2008. Schoenfeld, A.H. (2007). What is mathematical proficiency and how can it be assessed? In A.H. Schoenfeld (Ed.), Assessing mathematical proficiency, pp. 59-73. New York: Cambridge University Press. Yeap, B.H. (2008). The Role of the Calculator in Formal Mathematics Assessment. Plenary lecture presented at the International Conference on Science and Mathematics Education, October 27 – 29, 2008 at the University of the Philippines.

Singapore Examination and Assessment Board (SEAB). (2008). PSLE Examination Questions 2003-2007 Mathematics. Singapore: Hillview Publications. Smith, B.A. (1997). A meta-analysis of outcomes from the use of calculators in mathematics education. (Doctoral dissertation, Texas A & M University, 1996). Dissertation Abstracts International, 58, 787A. Wilson, L.D. (2007). High-stakes testing in mathematics. In Lester, F.K. (Ed.), Second Handbook of Research on Mathematics Teaching and Learning, pp. 1099-1110. NCTM.

Yeap, B.H. (2008). The Role of the Calculator in Formal Mathematics Assessment. Plenary lecture presented at the International Conference on Science and Mathematics Education, October 27 – 29, 2008 at the University of the Philippines.