Test Data Analysesass_2

Test Data Analysis Compiled by P.J. Smit

Computer-based Assessment CIA 722

Department of Curriculum Studies Faculty of Education University of Pretoria

Test Data Analysis P.J. Smit

Acknowledgements Many thanks to Prof. JG Knoetze for supplying the needed raw data to complete the report. A big thank you to Prof. Tinus Kuhn for guiding the report compiler in completing the report.

Table of Contents Chapter 1.1 1.2 1.3 1.4

Abstract Introduction Nature of report Statement of

1.5 1.6 1.7 1.8 1.9 2.1

purpose Aims and objectives Test Analysis Central Tendency Results and findings References Appendix

Page 5 6 6 6 6 7 9 10 11 12

Abstract In this report the researcher tried to manipulate data into item analysis and descriptive statistics in order to asses his objective test item namely multichoice. The researcher measured the difficulty index, discrimination value as well as reliability coefficient. In the process the also calculated the Mode, established the mean as well as median. The completed literature study helped to narrow focus as well as exposed the researcher to new ways to convert data to visual representation. The report contains findings of the analysis as well as a summery of all the data.

Introduction Before we can use the data provided it is crucial that the data is ordered and grouped accordingly. The data is read into Microsoft Excel where formulas make it possible for us to make the most of the data. The data is managed much easier for it is now grouped and is manipulated to such an extend that the researcher can draw conclusions and graphs from what was once just numbers. Nature of report Descriptive statistics are used to describe the basic features of the data in a completed study. They provide simple summaries about the sample and the measures. Together with simple graphics analysis, they form the basis of virtually every quantitative analysis of data. Descriptive Statistics are used to present quantitative descriptions in a manageable form. Descriptive statistics help us to simplify large amounts of data in a sensible way. Each descriptive statistic reduces lots of data into a simpler summary.http://www.socialresearchmethods.net/kb/statdesc.php) Statement of purpose Why would we compile ‘n report like this? An analytic report like this would be great for a teacher or instructor to have, if that person would like to go forward in life and improve their questions. To have an objective look at your work is critical for improving the quality of your work. An item analysis provides information useful for improving the quality and accuracy of multiple-choice tests. This is extremely valuable in the classroom and practice. It is always good to know the quality of your questions. Your questionnaires can always be assessed to be perfected. http://www.socialresearchmethods.net/kb/statdesc.php) Aims and Purpose/ aims and objectives The aim now is to find out witch questions are too easy or to difficult. Lets look at the table below.

Amount of Learners correct

Amount Correct 25 20 15 Correct questions 10 5 0 1

2

3

4

5

6

7

8

9

10

11 12

13

14 15

16 17

18

19 20

Questions 1 - 20

Figure 1.1 It is very easy to identify right away the very easy questions. Question 1 and question 2 goes hand in hand. More than 20 learners answered the

questions correct. Questions 11 and question 16 is also very easy, because most of the learners answered correct. Question 14 and question 15 to a large extend answered correctly. All data used in this report used to draw up this charts are listed in the appendix. From figure it is clear that question 4, question7, question 10 and question 18 are amongst the ones that were answered the most incorrectly.

Amount of Learners wrong

Wrong answers 25 20 15 Wrong answers

10 5 0 1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Questions 1 - 20

Figure 1.2 A Conclusion based on the fact and info of Figure 1.1 and Figure 1.2 it is clear that there are some questions that are very easy and very difficult respectively. Tests Analysis Let us look at the data more analytically. The data can be converted to a difficulty index p(Figure 1.3). The difficulty index takes into consideration the amount of learners that got a question right. It also works out the percentage of learners who got it right. It becomes more clear witch questions was answered correctly the most. Questions 11 & 16 have a relatively low difficulty index because more or lest 90% of the class got the answer correct. The questions need to be replaced. Question dificulty index 100 Percentage

80 60

question dificulty index

40 20 0 1

2

3

4

5

6

7

8

9

10 11 12 13 14 15 16 17 18 19 20 Learners

Taking the analysis a step further. To get to a conclusion the researcher make use of a discrimination index (Figure 1.4). The discrimination index tells the test compiler whether he setup a good or a bad test. He looks at the data results and put them into a formula. That is determined by subtracting the amount of learners who answered correctly in the upper group from those in the lower group. Take that answer and divide that by the amount learners in the upper and lower group. Discrimination Index 1.2

D value

1 0.8 0.6

Discrimination Index

0.4 0.2 0 1

2

3

4

5

6

7

8

9

10 11 12 13 14 15 16 17 18 19 20

Question 1-20

In our case the reason for the discrimination index can be one of the following factors: Miskeying, Guessing or the ambiguity factor.

Central Tendency. 0.02 0.018 0.016 0.014 0.012 0.01

Series1

0.008 0.006 0.004 0.002 0 0

5

10

15

20

Figure 1.5

From figure 1.5we got the following information regarding central tendency. What is central tendency? The central tendency of a distribution is an estimate of the "center" of a distribution of values. There are three major types of estimates of central tendency namely Mean, Mode and Median. (http://www.robertniles.com/stats/dataanly.shtml) Mean 13.04

Mode 13

Median 13

The mean is the total average of the data where you add up all the numbers and then divide by the number of results. The mean for this specific report looking at the data was 13.04. The Median is the score found at the exact middle of the set of values. One way to compute the median is to list all scores in numerical order, and then locate the score in the center of the sample. In the case of the table below the median will be 13. The mode is the most frequently occurring value in the set of scores. To determine the mode, you might again order the scores as shown above, and then count each one. The most frequently occurring value is the mode. In our test data the mode will be 13.

Results and findings When looking at the given data and the results based on the test data we can make some necessary findings. Test items compiled in the given test was not of a very good standard. It is clear that there were no consistency on the difficulty level of the questions. Some questions were very easy and some were totally to complex and difficult. Conclusion & Recommendations One should take into consideration that not all learners are on the same cognitive level. Some might be on a level of brilliance and some barely understanding the new concept or language. The process of creating the perfect Test item questionnaire would be the ideal. Although this might sound impossible, this could be a timorously process and time consuming not the least, but with all the resources to our advantage any good educator must take the time to do this task.

References Research methods knowledge base; Accessed on 5 April 2008; http://www.socialresearchmethods.net/kb/statdesc.php RobertNiles.com Statistics Help for Journalists; Accessed on 4 April 2008; http://www.robertniles.com/stats/dataanly.shtml Tushar Mehta Copyright © 2000-2006; Accessed on 5 April 2008; http://www.tushar-mehta.com/excel/charts/normal_distribution/

Appendix