1
Introduction
The Office of Educational Assessment (2005) defines item analysis as a process, which examine student responses to individual test items/questions for the purposes of assessing their quality and the quality of a test a whole. This item analysis report is based on twenty multiple-choice test question administered to twenty-five students (Refer to Appendix B).The quality of individual test items/ question is assessed by comparing student’s item responses to their total scores. The summary of the performance of the test is shown in (Appendix A).
2
Purpose of the report
The purpose of this report is to disseminate information to parents; School management team and institutional support co-ordinators.This report is based on the descriptive statistics about twenty multiple choice test items administered to twenty students.
3
Procedures
The spread sheet was used to collate from twenty students. Twentyfive multiple choice test items were administered to twenty students. Microsoft Excel was used to calculate scores. The scores were used to calculate mean, median mode and standard deviation. These values were used to get a normal distribution graph. Three graphs were drawn using Microsoft Excel. Reliability coefficient was calculated using kuder-Richardson equation (i.e. K20).Difficulty index (p) and discrimination index were also calculated using questions and results from the test.
1
4 Test analysis 4.1 Descriptive statistics The descriptive statistics describe basic features of data. They describe summaries about the sample, measurements and quantities in a manageable form. A set of test scores were used to calculate the mean, median, mode and the standard deviation as shown in Table 1. Table 1:
Descriptive statistics Mean Mode Median STEDV2 STEDV
65 65 65 479.57 21.90
The table shows that the mean, median and mode are the same hence it is a normal distribution curve. The rational here is that if the distribution is truly normal, the mean, median, and mode are all equal to each other i.e. (bell-shaped) as referenced in Figure 1
0.0
0.1
0.2
0.3
0.4
Figure 1: A normal distribution curve
The values on the x-axis represent standard deviation from the mean. The area where there is a change in curvature is one standard
2 -4SD
-3SD
-2SD
-1SD
65
+1SD +2SD +3SD
+4SD
on either side of the mean, then follows two, three and four respectively. The curve is symmetrical, i.e it can be divided into two equal halves. It is a heterogeneous distribution.
4.2 Frequency graphs 4.2.1
Grouped frequency table
The highest score and the lowest score are used to find the range. The number of intervals is a choice. The size of the interval is from eighty (range) divided by ten (number of intervals). Table 2: Grouped frequency H (Highest value)
100
L (Lowest value)
20
Range
80
Number of intervals
10
Size of interval
8
4.2.2
Cumulative frequency
Cumulative frequency is calculated by adding the frequency of the class interval and the frequencies of the preceding interval until the class interval as indicated in Table 3.
Table 3:
Cumulative frequency distribution
3
Lower
Upper
Interva
Middle
Values
Values
l
Values
16
23
16-23
19.5
1
1
24
31
24-31
27.5
2
3
32
39
32-39
35.5
0
3
40
47
40-47
43.5
2
5
48
55
48-55
51.5
3
8
56
63
56-63
59.5
2
10
64
71
64-71
67.5
6
16
72
79
72-79
75.5
1
17
80
87
80-87
83.5
3
20
88
95
88-95
91.5
3
23
96
103
96-103
99.5
2
25
4.2.3
Frequency Cumulative Frequency
Frequency histogram
The frequency histogram is drawn from the class interval and frequency. The histogram is shown in Figure 2:
Figure 2
Frequency histogram
4
Frequency histogram 7 6 5 4 3 2 1 0
Fr e q u e n cy 4.2.4
23
31
39
47
55
63
71
79
87
95
103
16
24
32
40
48 56 64 Interval
72
80
88
96
Frequency polygon
The frequency polygon is drawn from cumulative frequency and the upper values. The frequency values and the upper values are shown in Table 3: The frequency polygon graph is illustrated in Figure 3: Figure 3
Frequency Polygon graph
Frequency Polygon 7 6 5 4 3 2 1 0
Frequency 1 Middle values 4.2.5
Cumulative frequency graph
5
Cumulative frequency is drawn by using cumulative frequency values and upper values. Cumulative frequency graph is shown in Figure 5. Figure 4
Cumulative frequency graph
Cumulative frequency graph (An Ogive)
30 25 20 15 10 5 0 3 10
87
71
55
39
23
Cumulative Frequency
Upper values
4.3 Reliability coefficient of a test The reliability coefficient of a test is defined as the extent to which the test is likely to give consistent scores (Borich & Kubiszyn).The values ranges from zero to one where zero is no reliability and one is perfect reliability. The kR20 is used to measure test reliability of interitem consistency. The high value indicates a strong relationship between items on the test.
The formulae for calculating KR20 is:
6
KR20 = (k/k-1) (1-∑pq/∂2) Where: K=No of items p=Proportion that passed q=Proportion that failed ∂2=Variance The values used to calculate KR20 are referenced in Table 4. Table 4:
Coefficient of reliability
K
20
K-1
19
Total pq
3.83
SD (standard deviation)
21.9
(SD)2
479.6
KR20
1.03
It is evident from the table that the value of KR20 is 1.04; this implies that the test has a perfect reliability.
5
Item analysis
7
Item analysis is described as statistical analysis which shows the effectiveness of individual test items (Borich & Kubiszyn).
5.1 Difficultly and discrimination indices of a set test items The questions and the results from the test were used to describe the difficulty of each question and the corresponding discrimination index as referenced in Table 5 Table 5:
Difficulty index (p) Difficulty index (p)
#Questions
#Correct
#Answered
p
Q1
21
25
0.84
Q2
22
25
0.88
Q3
17
25
.68
Q4
12
25
0.48
Q5
21
25
0.84
Q6
17
25
0.68
Q7
11
25
0.44
Q8
12
23
0.52
Q9
13
25
0.52
Q10
8
24
0.33
Q11
23
25
0.92
Q12
19
25
0.76
Q13
15
25
0.60
Table 5:
Difficulty index (p) (continued)
8
Difficulty index #Questions
#Correct
#Answered
p
Q14
21
25
0.84
Q15
20
25
0.80
Q16
22
24
0.92
Q17
15
24
0.63
Q18
8
24
0.33
Q19
13
25
0.52
Q20
16
25
0.64
The percentages of students who answered the item correctly is reflected in column (p).It measures the level at which the question was difficult to answer. If the difficulty index is higher then the question was too easy. The value 1.00 implies that all students answered this correct response and the question was too easy. If the p value is .75, the item is acceptable but if the p value is .25, then the item is too difficult. The interpretation of difficulty level of questions is listed in Table 6. Table 6:
Interpretation of difficulty level of questions
Questions
Proportions
Interpretation Reason
Q1
0.84
Unacceptable
Too easy
Q2
0.88
Unacceptable
Too easy
Q3
0.68
Acceptable
Fine
Q4
0.48
Acceptable
Fine
Q5
0.84
Unacceptable
Too easy
Table 6:
Interpretation of difficulty level of questions (continued)
9
Questions
Proportions
Interpretation Reason
Q6
0.68
Acceptable
Fine
Q7
0.44
Acceptable
Fine
Q8
0.52
Acceptable
Fine
Q9
0.52
Acceptable
Fine
Q10
0.33
Acceptable
Fine
Q11
0.92
Unacceptable
Too easy
Q12
0.76
Unacceptable
Too easy
Q13
0.60
Acceptable
Fine
Q14
0.84
Unacceptable
Too easy
Q15
0.80
Acceptable
Fine
Q16
0.92
Unacceptable
Too easy
Q17
0.63
Acceptable
Fine
Q18
0.33
Acceptable
Fine
Q19
0.52
Acceptable
Fine
Q20
0.64
Acceptable
Fine
It is evident from the Table 6.that 35% of the questions i.e. (question 1, 2, 5,11,12,14 and 16) are unacceptable and the implication is that they were too easy.65% of the questions i.e. (3, 4, 6, 7, 8, 9, 10, 14, 15, 17, 18, 19, and 20) are acceptable and fine as referenced in Table 6. The interpretation of the difficulty level of questions is referenced in Table 5.
10
The discrimination index is used to ability of test items to distinguish between the lower and the upper group of students taking the test as shown in Table 7 Table 7:
Number of students in the upper and lower group
Level
Value
Upper
15
Lower
10
These measures the ability of an item to discriminate of differentiate among students who got higher score to those who got lower scores. It is actually the difference between percentages of correct response in the upper group and the correct response in the lower group. Refer to Table 8 for calculations on the discrimination index. Table 8:
Discrimination index (D) Discrimination index
#U
#L
D
15
6
0.60
15
7
0.53
14
3
0.73
8
4
0.26
15
6
0.60
12
5
0.46
9
2
0.46
10
2
0.53
11
Table 8:
Discrimination index (D) (continued) Discrimination index 3 0.46 0 0.53 9 0.33 5 0.60 3 0.60 6 0.60 6 0.53 7 0.53 3 0.60 3 0.13 1 0.92 5 0.40
10 8 14 14 12 15 14 15 12 5 12 11
All the values obtained are positive therefore it is a positive discrimination index. The item discrimination index ability is adequate. A higher index implies higher scoring students tended to select the response more often as seen in Table 8.
5 Conclusion The value of KR20 is 1.04; this implies that the test is reliable. Students who answered a gives question correctly is likely to answer other questions correctly. The other implication is that should another test be administered using similar items, the relative scores of students would show very little change.
7.
References
12
1
A Guide to interpreting the item Analysis Report.(2004).Retrieved April 01 2008, from http://www.asu.edu/uts/InterplAS.pdf
2
Image: Standard deviation diagram.svg [Image] (n.d.).Retrieved April 01 2008, from http://en.wikipedia.org/wiki/Image:Standard deviation_diagram.svg#file.
3
Kubiszyn & Borich, G, (2007).Education testing and Measurement. Classroom Application and Practice (8th Ed).John Wiley & sons, inc.United States of America.
4
8 8.1
Appendices Appendix A
13
#Questi
#Corre
#Incorrec
Prop
Prop
ons
ct
t
Correct
Incorrec
0.84 0.88 0.68 0.48 0.68 0.44 0.52 0.52 0.33 0.92 0.76 0.60 0.84 0.80 0.92 0.63 0.33 0.52 0.64 0.68
t 0.16 0.12 0.32 0.52 0.16 0.32 0.56 0.48 0.48 0.67 0.08 0.24 0.4 0.16 0.2 0.08 0.38 0.67 0.48 0.36
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20
21 22 17 12 21 17 11 12 13 8 23 19 15 21 20 22 15 8 13 16
4 3 8 13 4 8 14 11 12 16 2 6 10 4 5 2 9 16 12 9 Total
14
pq
0.13 0.11 0.22 0.25 0.13 0.22 0.25 0.25 0.25 0.22 0.07 0.18 0.24 0.13 0.16 0.08 0.23 0.22 0.25 0.23 3.83
Key St No
C
B
D
D
B
C
D
A
C
B
A
C
B
D
A
A
C
D
B
C
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8
Q9
Q10
Q11
Q12
Q13
Q14
Q15
Q16
Q17
Q18
Q19
Q20
1 2 3 4 5 6 7 8 9 10 11 12 13
C C C C C C B C C C C C C
B B B B B A B B B B B B B
B D D D D D A D D B D D D
A D D B C D B B A A D D A
C B B B B C B B B B B B B
D D C C C C C C C C C C C
A A D B B A B B D D D D D
A A A A D B D D C A D A
D C C C C C D B B D C D C
D B B B D D D C D C B A B
A A A A A A A A A A A A A
D C C C C C C C C B C C C
A B B A B A B B B A B A B
A D D D D D D D D D D D D
A A A C A A C A A D A A A
A A A A A A A A A A A A A
C C C C A A A C C C C C A
B D B B B B D A B D D B B
D B D C B D D B D B B B B
B C C C C C C A A C C D C
14 15 16
C C C
B B B
D D D
A D D
B B B
C B C
D A D
A A A
C B C
B D B
A A A
C C C
B D B
D A D
A A A
A C A
A B C
B D
B D B
C D C
17 18 19 20 21 22 23
B C D C C B C
B B C B A B B
C B A D D A D
C D D D D B B
B B B B C B B
A A A C C C C
D D B D A B B
D D A A D B D
C D D C C D B
D C A D D C
A A C C A A A
D C D D C C C
B A A B A B B
D D A D D D D
A A D A A C A
A B A A A A
C B C A A C
C B B D B D A
A B A B D D B
D C B C C C A
24 25
C C
B B
B D
A D
C B
D D
A A
A
D C
D B
A A
D C
A B
A D
A A
A A
C C
B D
D B
B C
15
8.3
Appendix C Key St No 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1
C
B
D
D
B
C
D
A
C
B
A
C
B
D
A
A
C
D
B
C
Q1 1 1 1 0 1 1 0 1 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1
Q2 1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1
Q3 1 0 1 1 1 0 1 1 0 0 0 1 1 1 1 0 1 1 0 1 1 1 1 1 0
Q4 1 0 0 0 1 1 1 1 0 1 1 0 0 1 1 0 0 0 0 1 0 0 1 1 0
Q5 1 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 0
Q6 0 0 1 1 1 1 0 0 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0
Q7 0 0 0 0 0 1 0 1 1 1 1 1 0 1 1 1 1 0 0 0 0 0 1 0 0
Q8 1
Q9 1 0 0 0 0 1 1 0 1 1 0 1 1 0 1 0 0 0 0 1 1 1 1 1 0
Q10 1 0 0 0 0 0 0 0
Q11 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Q12 1 0 1 1 1 0 0 1 0 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0
Q13 1 0 1 1 0 1 0 0 1 1 0 1 1 0 1 0 1 1 1 0 1 0 1 1 0
Q14 1 0 1 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0
Q15 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 0 1 1 0 1 1 0 1 1 1
Q16 1 1 1 1 1 1 0 1
Q17 1 1 1 0 0 1 0 1
Q19
1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 0 0 0 1 1 1 1 1 0 0 0 1 1 1 1
Q18 1 0 0 1 0 1 0 0 0 1 0
Q20 1 0 1 0 1 1 0 1 0 1 0 1 1 0 1 1 0 0 1 1 1 1 1 1 0
0 0 0 1 0 1 1 1 0 1 1 1 0 0 0 0 0 1 1 1 1 1 1
1 0 1 1 0 1 0 0 0 0 0 0 1 1 1 0
16
0 0 1 1 0 0 1 0 0 0 0 1 0
0 1 0 0 1 0 1 0 1 0 1 1 1 1 1 0 1 0 0 1 0 0 1 0
Total 18 6 13 10 11 17 4 13 8 20 9 17 17 14 20 12 13 13 10 11 15 14 18 18 6
% 100 100 90 90 90 85 85 85 75 70 70 65 65 65 65 60 55 55 50 50 45 40 30 30 20