Item Analysis

Item Analysis Item analysis techniques are among the most valuable tools classroom teachers or rest developers can apply when attempting to improve the quality of their measuring devices. It has four general purposes: (1) to select the best available questions for the final form of an assessment; (2) to identify structural or content defects in the questions; (3) to detect learning difficulties of the class as a whole, identifying general content areas or skills that need to be reviewed by the instructor; and (4) to identify for individual students areas of weakness needing remediation. An item analysis has three main elements. One is examining the difficulty level of the items, that is the percentage of students responding correctly to each question in the assessment. Another is determining the discriminating power of each item. It refers to the relation of performance on each item to performance on the total test. The third element is examining the effectiveness of the distractors.

Preparing Data for Analysis 1. Arrange the answer sheets in order from high to low. This ranking is usually based on the individual’s total score on the test. 2. High and low-scoring groups are identified. For purpose of item analysis, these two extreme sets of examination paper are called criterion groups. The goal is to include enough people in the criterion groups to justify confidence in the results, and yet keep the criterion groups distinct enough to ensure that they represent different levels of ability. 3. Record separately the number of times each alternative was selected by individuals in the high and low groups. 4. Add the number of correct answers to each item made by the combined high and low groups.

5. Divide the total number of correct responses by the maximum possible, that is, the total number of students in the combined high and low groups, and multiply the result by 100. This percentage is an estimate of the difficulty index. Some test constructors allow items to be omitted, and the students inadvertently omit items. If all individuals have not attempted all items, item difficulty indices should be obtained by dividing the total number of correct responses by the number of individuals who attempted the item. 6. Subtract the number of correct answers made by the low group from the number of correct answers made by the high group. 7. Divide this number ( the difference, H-L ) by the number of individuals contained in the subgroup ( that is, the number in the high or low group ). This decimal number is the discrimination index.

Consideration of Question Difficulty An item’s difficulty level is important because it tells the instructor something meaningful about the comprehension of, or performance on, material or tasks contained in the item. It represents the percentage of the total number of respondents answering the item correctly. A Paradox exists in interpreting the item difficulty indices, if a teacher has done a good job in teaching a particular concept skill or fact, students will perform well on items related to that topic. Item difficulty values will be high in numerical value, for example, 85, meaning that a large percentage of the students did well. If most everyone did well then there is no room for the item(s) to discriminate.