A TEST AND ITEM ANALYSIS REPORT
BY
JOSEPH A AZAGSIBA
TABLE OF CONTENTS Section 1 1.1 1.2 1.3
Introduction Purpose of report Approach Report structure
Page 6 6 6 7
2 2.1 2.2
Test Analysis Frequency histogram Descriptive statistics
7 7 9
3 3.1 3.2 3.3
Item Analysis Difficulty index Discrimination index Item reliability
10 10 12 13
4
Conclusion
13
APPENDIX A: CODED DATA APPENDIX B: DISCRIMINATION INDEX APPENDIX C: DIFFICULTY INDEX REFERENCES
15 17 18 19
2
List of Tables Table 1 2 3 4 5
Table name Report structure Measurements of values of central tendency, standard deviation and KR20 Difficulty index Discrimination index Reliability co-efficient
Page 7 9 10 12 13
3
List of Figures Figure 1 2
Figure name Histogram of grouped frequency distribution Normal distribution curve
Page 8 9
4
1. INTRODUCTION This is a test and item analysis report on the variability and reliability of twenty multiple-choice questions answered by twenty-five learners. It comprises the Introduction, the Test Analysis, the Item Analysis and a Conclusion. 1.1
Purpose of Report
The purpose of this report is to disseminate information on the measure of tendency and variability on a set of data of twenty multiple choice questions that have been answered by twenty-five learners. 1.2
Approach
The methodology employed to arrive at the purpose of this report consisted of the following steps: •
The relevant data was collected.
5
•
The collected data was recorded and tabulated using Microsoft Excel.
•
The mean, mode, median and standard deviation of the scores obtained by learners were calculated.
•
The difficulty index and the discrimination index were then calculated.
•
The reliability coefficient was also calculated using the Kuder-Richardson procedure (KR20).
•
The scores of learners were grouped into upper and lower levels.
•
A histogram of the data was drawn.
•
Conclusions were then made based on the analysed data.
1.3
Report structure
The structure of this report comprises an introduction, a test analysis, an item analysis and a conclusion. These sections are subdivided into subsections as shown in Table 1. Table1: Report structure Section 1
Heading Introduction
Subheading Purpose Approach Report structure
6
2
Test Analysis
Frequency graph Descriptive statistics
3 4
Item analysis
Difficulty index
Conclusion
Discrimination index Conclusion
1 TEST ANALYSIS 2.1 Frequency histogram The histogram or bar graph in Figure 1 displays statistical data about the number of learners who answered the multiple-choice questions correctly. From the histogram, one learner each fell within each of the score intervals of 20-28, 38- 46, 56-64, and 74-81; whilst two learners each were represented within the score intervals of 29-37 and 100-108. Five learners fell within the score interval of 47 and 55. No learner fell within the 91-99 score interval, whilst six learners were placed within the score intervals of 65-73 and 82-90. Frequency Histogram
I ntervals
100-108
91-99
82-90
74-81
65-73
56-64
47-55
38-46
Frequency histogram
29-37
7 6 5 4 3 2 1 0
20-28
1: Frequency
Figure
20-28 29-37 38-46 47-55 56-64 65-73 74-81 82-90 91-99 100-108
7
2.2 Descriptive statistics Table 2: Measurements of values of central tendency, standard deviation and KR20. Mean Mode Median STDEV #Upper #Lower STDEV2 #Question k-1 KR20
65.99 65 65 21.90 13 12 479.39 20 19 1.04
8
The mean (M), which is a measure of central tendency, is 65.99. The mode is 65 and the median (MDN) is 65. The mean, median and mode of this data are almost the same. This means that about half of the learners fell on either side of these values. The data distribution is therefore normal. Figure 2 illustrates a normal distribution curve. The standard deviation (SD) is 21.90. Figure 2: Normal distribution curve
3. ITEM ANALYSIS 3.1 Difficulty index The difficulty index for each item computed using the formula, p= Number of students selecting correct answer Total number of students attempting the item is given in Table 3. Table 3: Difficulty index DIFFICULTY INDEX
9
Quest 1 2 3 4 5 6 7 8 9 10 11 12
#Ans 25 25 25 25 25 25 25 23 25 24 25 25
#Cor 21 22 17 12 21 17 11 12 13 8 23 19
p 0.84 0.88 0.68 0.48 0.84 0.68 0.44 0.52 0.52 0.33 0.92 0.76
Remarks Too Easy Too Easy Moderate Moderate Too Easy Moderate Moderate Moderate Moderate Moderate Too Easy Moderate
10
Table 3: Difficulty index DIFFICULTY INDEX Quest #Ans #Cor 13 25 15 14 25 21 15 25 20 16 24 22 17 24 15 18 24 8 19 25 13 20 25 16
p 0.6 0.84 0.8 0.92 0.63 0.33 0.52 0.64
Remarks Moderate Too Easy Moderate Too Easy Moderate Moderate Moderate Moderate
From this information the level of difficulty or easiness of each item was determined. Items that have p values less than 0.20 are relatively difficult and those that have p values above 0.80, relatively easy. Thus six items or 30% of the items were relatively easy and 14 or 70% were moderately difficult.
11
3.2 Discrimination index The discrimination index for each item is provided in Table 4. Table 4: Discrimination index DISCRIMINATION INDEX Quest 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
U 15 15 14 8 15 12 9 10 10 8 14 14 12 15 14 17 12 5 12 11
L 6 7 3 4 6 5 2 2 3 0 9 5 3 6 6 7 3 3 1 5
D 0.69 0.54 0.85 0.31 0.69 0.54 0.54 0.62 0.54 0.62 0.38 0.69 0.69 0.69 0.62 0.62 0.69 0.15 0.85 0.46
Remarks Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable Desirable
12
A positive discrimination index (D) indicates that the item’s discrimination is adequate. All these items
show
positive
discrimination
index
values
implying
that
all
items
have
adequate
discrimination. 3.3 Item reliability The item reliability coefficient was calculated using the Kuder-Richardson procedure. The resulting coefficient ( KR20), 1.04, is provided in Table 5. Table 5: Reliability coefficient KR20
1.04
The reliability coefficient of 1.04 reflects the SD value of 21.9 as being a large variability score. With 30% of the test items being too easy, the test reliability is affected to some extent as when some items become very easy or very difficult, the test’s reliability goes down.
4. CONCLUSION From the data analysis all the test items are desirable as they all show positive discrimination abilities. However,of the twenty test items, six of them, items 1, 2, 5, 11, 14 and 16, have p values in excess of 0.80 thus rendering them relatively easy items. This enabled the students who did not even know their stuff to easily pick out the correct answers.
13
To prevent students who do not know their stuff from scoring undeserved marks therefore, it is recommended that items 1, 2, 5, 11, 14 and 16 be reconsidered and reconstructed to improve their difficulty levels.
14
Data coding Appendix A Key
C
B
D
D
B
C
D
A
C
B
A
C
B
D
A
A
C
D
B
C
Q1 1
Q1 2
Q1 3
Q1 4
Q1 5
Q1 6
Q1 7
Q1 8
Q1 9
Q2 0
#C or
# Inco r
#A ns
St No
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8
Q9
Q1 0
11
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
20
0
20
16
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
20
0
20
% 100.0 0 100.0 0
G roup
2
1
1
1
1
1
0
0
1
1
1
1
1
1
1
1
1
1
1
1
1
18
2
20
90.00
U
U U
3
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0
0
1
18
2
20
90.00
U
25
1
1
1
1
1
0
0
1
1
1
1
1
1
1
1
1
1
1
1
1
18
2
20
90.00
U
14
1
1
1
0
1
1
1
1
1
1
1
1
1
1
1
1
0
1
1
17
2
19
89.47
U
13
1
1
1
0
1
1
1
1
1
1
1
1
1
1
1
1
0
0
1
1
17
3
20
85.00
U
20
1
1
1
1
1
1
1
1
1
0
0
0
1
1
1
1
1
1
1
1
17
3
20
85.00
U
5
1
1
1
0
1
1
0
1
1
0
1
1
1
1
1
1
0
0
1
1
15
5
20
75.00
U
4
1
1
1
0
1
1
0
1
1
1
1
1
0
1
0
1
1
0
0
1
14
6
20
70.00
U
12
1
1
1
1
1
1
1
0
0
0
1
1
0
1
1
1
1
0
1
0
14
6
20
70.00
U
8
1
1
1
0
1
1
0
0
0
0
1
1
1
1
1
1
1
0
1
0
13
7
20
65.00
U
9
1
1
1
0
1
1
1
0
0
0
1
1
1
1
1
1
1
0
0
0
13
7
20
65.00
U
18
1
1
0
1
1
0
1
0
0
0
1
1
0
1
1
1
1
0
1
1
13
7
20
65.00
L
23
1
1
1
0
1
1
0
0
0
0
1
1
1
1
1
1
1
0
1
0
13
7
20
65.00
L
10
1
1
0
0
1
1
1
0
0
0
1
0
0
1
0
1
1
1
1
1
12
8
20
60.00
L
6
1
0
1
1
0
1
0
0
1
0
1
1
0
1
1
1
0
0
0
1
11
9
20
55.00
L
21
1
0
1
1
0
1
0
0
1
0
1
1
0
1
1
1
0
0
0
1
11
9
20
55.00
L
7
0
1
0
0
1
1
0
0
0
0
1
1
1
1
0
1
0
1
0
1
10
10
20
50.00
L
22
0
1
0
0
1
1
0
0
0
0
1
1
1
1
0
1
0
1
0
1
10
10
20
50.00
L
17
0
1
0
0
1
0
1
0
1
1
0
1
1
1
0
0
0
8
9
17
47.06
L
15
1
1
1
1
1
0
0
1
0
1
1
0
0
1
0
0
0
9
11
20
45.00
L
0
0
0
15
1
1
1
0
0
0
0
0
0
0
1
0
0
0
1
1
1
0
0
0
6
13
19
31.58
L
24
1
1
0
0
0
0
0
0
0
1
0
0
0
1
1
1
0
0
0
6
13
19
31.58
L
19
0
1
0
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
4
16
20
20.00
L
# Cor
1
21
23
17
12
21
17
11
12
13
8
23
19
15
21
20
22
15
8
13
16
4
2
8
13
4
8
14
11
12
16
2
6
10
4
5
2
9
16
12
9
25 0 .84
25 0 .92
25 0 .68
25 0 .48
25 0 .84
25 0 .68
25 0 .44
23 0 .52
25 0 .52
24 0 .33
25 0 .92
25 0 .76
25 0 .60
25 0 .84
25 0.8
24 0 .92
24 0 .63
24 0 .33
25 0 .52
25 0 .64
#U
15
15
14
8
15
12
9
10
10
8
14
14
12
15
14
15
12
5
12
11
#L
6 0 .69 0 .84 0 .16 0 .13
8 0 .54 0 .92 0 .08 0 .07
3 0 .85 0 .68 0 .32 0 .22
4 0 .31 0 .48 0 .52 0 .25
6 0 .69 0 .84 0 .16 0 .13
5 0 .54 0 .68 0 .32 0 .22
2 0 .54 0 .44 0 .56 0 .25
2 0 .62 0 .52 0 .48 0 .25
3 0 .54 0 .52 0 .48 0 .25
0 0 .62 0 .33 0 .67 0 .22
9 0 .38 0 .92 0 .08 0 .07
5 0 .69 0 .76 0 .24 0 .18
3 0 .69 0 .60 0 .40 0 .24
6 0 .69 0 .84 0 .16 0 .13
6 0 .62 0 .80 0 .20 0 .16
7 0 .62 0 .92 0 .08 0 .08
3 0 .69 0 .63 0 .38 0 .23
3 0 .15 0 .33 0 .67 0 .22
1 0 .85 0 .52 0 .48 0 .25
5 0 .46 0 .64 0 .36 0 .23
#Inc or # Ans p
D P Q PQ
U ppe r L owe r
13 12
3 .80
16
DIFFICULTY INDEX
Discrimination index Appendix B
Quest #Ans #Cor p 1 25 21 2 25 22 3 25 17 4 25 12 5 25 21 DISCRIMINATION INDEX 6 25 17 7 25 11 Quest 8 U 23 L 12 D 9 1 25 15 136 2 15 10 24 87 3 14 11 25 233 4 8 12 25 194 5 15 13 25 156 6 12 14 25 215 15 7 25 9 202 8 10 16 24 222 9 10 17 24 153 10 8 18 24 80 11 14 19 25 139 2012 25 14 165 13 12 3 14 15 6 15 14 6 16 17 7 17 12 3 18 5 3 19 12 1 20 11 5
Remarks 0.84Too Easy 0.92Too Easy 0.68Moderate 0.48Moderate 0.84Too Easy 0.68Moderate 0.44Moderate Remarks 0.52Moderate 0.69Acceptable 0.52Moderate 0.54Acceptable 0.33Moderate 0.85Acceptable 0.92Too Easy 0.31Acceptable 0.76Moderate 0.69Acceptable 0.6Moderate 0.54Acceptable 0.84Too Easy 0.54Acceptable 0.8Moderate 0.62Acceptable 0.92Moderate 0.54Acceptable 0.63Moderate 0.62Acceptable 0.33Moderate 0.38Acceptable 0.52Moderate 0.69Acceptable 0.64Moderate 0.69Acceptable 0.69Acceptable 0.62Acceptable 0.62Acceptable 0.69Acceptable 0.15Acceptable 0.85Acceptable 0.46Acceptable
17
Difficulty index
Appendix C
DIFFICULTY INDEX Quest #Ans 1 25 2 25 3 25 4 25 5 25 6 25 7 25 8 23 9 25 10 24 11 25 12 25 13 25 14 25 15 25 16 24 17 24 18 24 19 25 20 25
#Cor 21 22 17 12 21 17 11 12 13 8 23 19 15 21 20 22 15 8 13 16
p 0.84 0.92 0.68 0.48 0.84 0.68 0.44 0.52 0.52 0.33 0.92 0.76 0.6 0.84 0.8 0.92 0.63 0.33 0.52 0.64
Remarks Too Easy Too Easy Moderate Moderate Too Easy Moderate Moderate Moderate Moderate Moderate Too Easy Moderate Moderate Too Easy Moderate Moderate Moderate Moderate Moderate Moderate
References:
18
1.
Knoetze,
J.
(2007)
Test
data.
Retrieved
August
16,
2007,
from
Web
site
www.eSnips.com/web/CIA7222007. 2. Kubiszyn, T., & Borich, G. (2007). Educational Testing and Measurement: Classroom Application and Practice 8th ed. John Wiley & Sons Inc. United States of America
19