Measuring Dosage And Quality Of Program Implementation

Practice:

Evaluate implementation to document what you are doing

Key Action:

Measure quality of implementation activities

TOOL: Measuring Dosage and Quality of Program Implementation

Purpose:

Both the dosage and the quality of your program activities affect your ability to produce your intended outcomes. You may find this table useful for ensuring that your evaluation of implementation activities includes appropriate indicators of quality and reach, and not just dosage for magnet program outputs. It can also help you determine which quality and reach indicators will provide the most useful information.

Instructions:

1. Identify what data you will collect to answer your implementation questions. 2. Compare your indicators for dosage and quality of program outputs to those on the checklist. Check to make sure you have included indicators from both columns. 3. Reflect on which indicators are most useful to you and why.

1

Practice:

Evaluate implementation to document what you are doing

Key Action:

Measure quality of implementation activities

Measuring Dosage and Quality of Program Implementation Magnet program output Magnet instruction received by students

Professional development for teachers

Recruitment of diverse body of students

Examples of indicators for dosage (How much, how many, how often?)  Number of new lessons, units, or courses created integrate magnet-theme content 

Number of hours of magnetrelated instruction students receive each week

Example of indicators for quality and reach (How well? How effective? How uniform?)  Extent of rigor, depth, and breadth of theme-based curriculum 

Mastery of concepts and skills as evidenced by student work



Common understanding of magnet theme and quality criteria for integration into coursework



All students access and experience program services and magnet content



Satisfaction of participants in training session



Level of teacher understanding of workshop concepts



Clarity of learning goals for training



Level of ongoing support



Number of workshop sessions made available to teachers and administrators



Frequency of professional development



Number of staff participating



Number of partnerships and outside resources brought in to support training



Number of recruitment activities executed



Level of interest and engagement of parents at recruitment events



Number of parents involved in recruitment events (and other activities)



Consistency in understanding magnet program and mission among parents and students



Clarity of application and enrollment process for participants

Which indicators are most useful, and why?

Source: Selected PowerPoint® slides from “Partners in data-driven decision making: Evaluators and districts working together to improve program practice,” by David Kikoler of American Education Solutions, at Magnet Schools of America national conference, April 2008, Chattanooga, TN.

2