Report On Test Analysis

1.

Introduction

According to the Office of Educational Assessment (2005), Item analysis is a process which examines student response to individual test items/ questions in order to assess the quality of those items and of the test as a whole. This test item analysis report is based on a 20 multiple-choice test question administered to 25 students (see Appendix B). The quality of individual items is assessed by comparing student’s item responses to their total test scores. Statistical analysis has been shown to summarize the performance of the test as a whole (see Appendix A). Three types of graphical representation of statistical data are also shown in the report (A histogram, frequency polygon, a normal distribution curve and the Ogive).

2.

Purpose of the report

The purpose of this report is to disseminate information based on the descriptive statistics on 20 multiple-choice test items administered to 25 students.

3. 3.1

Test analysis Descriptive statistics

Descriptive statistics describes the basic features of the data that is, they describe what the data shows. They provide simple summaries about the sample and the measures and present quantitative descriptions in a manageable form. A set of test scores were considered to calculate the mean, mode, median and standard deviation, and a normal distribution graph for a distribution with a mean of 65, a median of 65 and a mode of 65 is drawn (see Figure 1).

Table 1: Descriptive statistics Mean Mode Median STDEV2 STDEV

65.79 65.00 65.00 479.57 21.90

This is a normal distribution because the measures of central tendency, the mean, the median and the mode are the same. If the distribution is truly normal (i.e., bell-shaped), the mean, median, and the mode are all equal to each other. Refer to Figure 1 to see the normal distribution curve

1

Figure 1: Normal distribution curve

0.0

0.1

0.2

0.3

0.4

Mean Median Mode

The numbers on the x-axis represent the standard deviations from the mean. The points where there is a change in curvature is one standard deviation on either side of the mean. Dark blue is less than one standard deviation from the mean. For the normal distribution, this accounts for about 68% of the set (dark blue) while two standard deviations from the mean (medium and dark blue) account for about 95% and three standard deviations (light, medium, and-4SD dark-3SD blue) account for about is symmetric. This is a -2SD -1SD 65 99.7%. +1SD The +2SDcurve +3SD +4SD heterogeneous distribution because the values are further away from the mean.

3.2

Frequency graphs

Table 2: Grouped frequency table H L Range Number of intervals Size of interval

100 15 85 10 8.5

Cumulative frequency is obtained by adding the frequency of a class interval and the frequencies of the preceding interval up to the class interval. This is explained in Table 3. The following frequency distribution table gives the marks obtained by 25 students:

2

Table 3: Cumulative frequency distribution Lower Limit

Upper Limit

Interval

Middle Value

Frequency

Cumulative Frequency

15.00 25.00 35.00 45.00 55.00 65.00 75.00 85.00 95.00

24 34 44 54 64 74 84 94 104

15-24 25-34 35-44 45-54 55-64 65-74 75-84 85-94 95-104

19.5 29.5 39.5 49.5 59.5 69.5 79.5 89.5 99.5

1 2 0 4 3 6 1 6 2

1 3 3 7 10 16 17 23 25

Figure 2:

Frequency histogram

Frequency

Frequency Histogram 7 6 5 4 3 2 1 0 15-24

25-34

35-44

45-54

55-64

65-74

75-84

85-94

95-104

Interval

Referring to Figure 2, a histogram is drawn to represent the class interval and the frequency in a form of a rectangle. The class intervals are marked on the horizontal axis (X-Axis) and the frequency is marked on the vertical axis (Yaxis). The intervals are equal; therefore the height of each rectangle is proportional to the corresponding frequency.

3

Figure 3:

Frequency polygon Frequency Polygon

7

Frequency

6 5 4 3 2 1 0 19.5

29.5

39.5

49.5

59.5

69.5

79.5

89.5

99.5

Middle values of intervals From Figure 3, the middle-values of the class interval of the given data are plotted against the corresponding frequencies and the points obtained are joined by straight lines. Points plotted (19, 1), (29, 2), (39, 0), (49, 4), (59, 3), (69, 6), (79, 1), (89, 6), and (99, 2). Refer to Table 3 to see the middle values of intervals and frequencies.

Figure 4: Cumulative frequency graph (An Ogive)

Cumulative Frequency Curve (An Ogive)

Cumulative Frequency

30 25 20 15 10 5 0 24

34

44

54

64

74

84

94

104

Upper Values

4

An Ogive is illustrated by an ‘S’ curve shape (see Figure 4). The graph is drawn by plotting the points with coordinates having abscissae (X-axis) as actual limits and ordinates(Y-axis) as the cumulative frequencies, (24,1), (34,3), (44,3),(54,7), (64,10), (74,16), (84,17), (94,23), and (104,25) are the coordinates of the points. Refer to Table 3 to see the data on cumulative frequency.

3.3

Reliability coefficient of a test

The reliability of a test refers to the extent to which the test is likely to produce consistent scores. It theoretically ranges in value from zero (no reliability) to 1.00 (perfect reliability). The KR20 measures test reliability of inter- item consistency. A higher value indicates a strong relationship between items on the test.

Table 4: Coefficients of reliability k k-1 Total pq Stdev

20 19 3.83 21.90

(Stdev)2

479.57

KR20

1.04

From Table 4, it is clear that the KR20 is 1.04 this means that the test is reliable or has a perfect reliability.

4.

Item analysis

Item analysis describes the statistical analysis which allows measurement of the effectiveness of individual test items.

4.1.1

Difficulty and discrimination indices of a set of test items

Using the questions and results from the test, the degree of difficulty of each question and the corresponding discrimination index was made (see Table 5 and Table 8).

5

Table 5:

Difficulty index (p) Difficulty index #Questions q1 q2 q3 q4 q5 q6 q7 q8 q9 q10 q11 q12 q13 q14 q15 q16 q17 q18 q19 q20

#Correct 21 22 17 12 21 17 11 12 13 8 23 19 15 21 20 22 15 8 13 16

#Answered 25 25 25 25 25 25 25 23 25 24 25 25 25 25 25 24 24 24 25 25

p 0.84 0.88 0.68 0.48 0.84 0.68 0.44 0.52 0.52 0.33 0.92 0.76 0.6 0.84 0.8 0.92 0.63 0.33 0.52 0.64

This shows a percentage of students who answered an item/question correctly. It is a measure of how difficult the question was to answer. The higher the difficulty index, the easier the question is. A value of 1.000 means that all of the students answered this correct response and this question may be too easy. If the p-value is greater than 0.75, the item is acceptable and if the p-value is less than 0, 25, then the item is difficult.

6

Table 6: Interpretation of the difficulty level of questions #Questions q1 q2 q3 q4 q5 q6 q7 q8 q9 q10 q11 q12 q13 q14 q15 q16 q17 q18 q19 q20

Proportion 0.84 0.88 0.68 0.48 0.84 0.68 0.44 0.52 0.52 0.33 0.92 0.76 0.6 0.84 0.8 0.92 0.63 0.33 0.52 0.64

Interpretation Unacceptable Unacceptable Acceptable Acceptable Unacceptable Acceptable Acceptable Acceptable Acceptable Acceptable Unacceptable Unacceptable Acceptable Unacceptable Acceptable Unacceptable Acceptable Acceptable Acceptable Acceptable

Reason Too easy Too easy Fine Fine Too easy Fine Fine Fine Fine Fine Too easy Too easy Fine Too easy Fine Too easy Fine Fine Fine Fine

From the Table, it shows that 35% of the questions (1, 2, 5,11,12,14, and 16) are unacceptable therefore it means that they were too easy and 65 %( 3, 4, 6, 7, 8, 9,10,13,15,17,18,19, and 20) are acceptable which shows that they were fine (see Table 6). Refer to Table 5 to see the interpretation on the difficulty level of questions. The discrimination index was used to measure the ability of items/ questions to distinguish between the lower and the upper group of students taking the test (see Table 7).

Table 7: Number of students in upper and lower group Upper Lower

15 10

This is the measure of ability of an item to discriminate or differentiates among students who have a high score on the test and those that get a low score on

7

the test. It is the difference between the percentage of correct responses in the upper group and the percentage of the correct responses in the lower group. Calculation procedures have been used to compare item responses to total test scores using upper and lower level groups of students. Refer to Table 8 to see the calculation procedures on discrimination index.

Table 8: Discrimination index (D) Discrimination index #U #L 15 6 15 7 14 3 8 4 15 6 12 5 9 2 10 2 10 3 8 0 14 9 14 5 12 3 15 6 14 6 15 7 12 3 5 3 12 1 11 5

D 0.60 0.53 0.79 0.50 0.60 0.58 0.78 0.80 0.70 1.00 0.36 0.64 0.75 0.60 0.57 0.53 0.75 0.40 0.92 0.55

This is a positive discrimination index because, the discrimination values are all positive therefore the item’s discrimination ability is adequate. A positive value for this index means that higher scoring student tended to select the response more often. Refer to Table 8 to see the discrimination values.

5.

Conclusion 8

Since the KR20 is 1.04 which shows that the test is reliable, it means that the questions of a test tended to “pull together”. Students who answered a given question correctly were more likely to answer other questions correctly. If a parallel test were developed by using similar items, the relative scores of students would show little change.

6.

References

1.

A Guide to Interpreting the Item Analysis Report. (2004). Retrieved September 12, 2007, from http://www.asu.edu/uts/InterpIAS.pdf

2.

Image: Standard deviation diagram.svg [Image] (n.d.). Retrieved 0ctober 11, 2007, From http://en.wikipedia.org/wiki/Image:Standard_deviation_diagram.svg#file

3.

Introduction to Statistical Inference. (2005). Retrieved September 11, 2007, from http://students.washington.edu/hdevans/lec_11.doc

4.

Kubiszyn, T., & Borich, G. (2007).Education testing and Measurement. Classroom Application and Practice (8th Ed).John Wiley &sons, Inc.United States of America.

5.

Normal Probability Distribution. (2005). Retrieved September 11, 2007, from http://palgrave.com/busines/taylor/taylor1/lectures/lectures/overheads/o chap5.doc

6.

Office of Educational Assessment (Understanding item analysis reports). (2005). Retrieved September 12, 2007, from http://personal.gscit.monash.edu.au/~dengs/teaching/GCHE/part33.pdf

7.

Test Item Analysis. (2005). Retrieved September12, 2007, From http://personal.gscit.monash.edu.au/~dengs/teaching/GCHE/part33.pdf

7.

Appendices 9

7.1

Appendix A

#Questions q1 q2 q3 q4 q5 q6 q7 q8 q9 q10 q11 q12 q13 q14 q15 q16 q17 q18 q19 q20

#Correct 21 22 17 12 21 17 11 12 13 8 23 19 15 21 20 22 15 8 13 16

#Incorrect 4 3 8 13 4 8 14 11 12 16 2 6 10 4 5 2 9 16 12 9

Prop Correct(p) 0.84 0.88 0.68 0.48 0.84 0.68 0.44 0.52 0.52 0.33 0.92 0.76 0.6 0.84 0.8 0.92 0.63 0.33 0.52 0.64

Prop Incorrect(q) 0.16 0.12 0.32 0.52 0.16 0.32 0.56 0.48 0.48 0.67 0.08 0.24 0.4 0.16 0.2 0.08 0.38 0.67 0.48 0.36 Total

pq 0.13 0.11 0.22 0.25 0.13 0.22 0.25 0.25 0.25 0.22 0.07 0.18 0.24 0.13 0.16 0.08 0.23 0.22 0.25 0.23 3.83

10

Appendix B Key St No

C Q1

B Q2

D Q3

D Q4

B Q5

C Q6

D Q7

A Q8

C Q9

B Q10

A Q11

C Q12

B Q13

D Q14

A Q15

A Q16

C Q17

D Q18

B Q19

C Q20

1 2 3 4 5 6 7 8 9 10 11 12 13

C C C C C C B C C C C C C

B B B B B A B B B B B B B

B D D D D D A D D B D D D

A D D B C D B B A A D D A

C B B B B C B B B B B B B

D D C C C C C C C C C C C

A A D B B A B B D D D D D

A A A A D B D D C A D A

D C C C C C D B B D C D C

D B B B D D D C D C B A B

A A A A A A A A A A A A A

D C C C C C C C C B C C C

A B B A B A B B B A B A B

A D D D D D D D D D D D D

A A A C A A C A A D A A A

A A A A A A A A A A A A A

C C C C A A A C C C C C A

B D B B B B D A B D D B B

D B D C B D D B D B B B B

B C C C C C C A A C C D C

14 15 16

C C C

B B B

D D D

A D D

B B B

C B C

D A D

A A A

C B C

B D B

A A A

C C C

B D B

D A D

A A A

A C A

A B C

B D

B D B

C D C

17 18 19 20 21 22 23

B C D C C B C

B B C B A B B

C B A D D A D

C D D D D B B

B B B B C B B

A A A C C C C

D D B D A B B

D D A A D B D

C D D C C D B

D C A D D C

A A C C A A A

D C D D C C C

B A A B A B B

D D A D D D D

A A D A A C A

A B A A A A

C B C A A C

C B B D B D A

A B A B D D B

D C B C C C A

24 25

C C

B B

B D

A D

C B

D D

A A

A

D C

D B

A A

D C

A B

A D

A A

A A

C C

B D

D B

B C

11