Practice:
Get quality data into your evaluator’s hands
Key Action:
Manage logistics and communications for data collection
SAMPLE MATERIAL: Timeline for Data Collection and Analysis
Purpose:
This sample timeline outlines ongoing tasks, milestones, and deliverables for data collection and analysis during a district’s magnet program evaluation. To keep the process on track, the table clearly lays out all the key tasks for the evaluation team and allocates adequate time to develop instruments, collect data, and deal with issues that may impact data analysis. You may find this document useful as you create or evaluate your own timeline.
Source:
From 2007 Magnet Schools Assistance Program grant application, developed by St. Paul Public Schools (SPPS), in collaboration with their external evaluators, Mid-continent Research for Education and Learning (McREL) and Geoffrey Borman, professor of education, University of Wisconsin-Madison.
1
Practice:
Get quality data into your evaluator’s hands
Key Action:
Manage logistics and communications for data collection
Table 3: Data Collection and Analysis Timeline
Winter 2009
Spring 2010
--
--
--
Summer 2010
Fall 2009
Summer 2009
Ÿ
Spring 2009
--
Year 3
Winter 2008
Spring 2008
Winter 2007
--- ongoing work Ÿinterim milestone 8 final deliverable
Fall 2007
Tasks
Fall 2008
Year 2 Summer 2008
Year 1
Overall Study Design Finalize study design plans: research (Borman)
--
Finalize study design plans: evaluation (McREL)
--
Finalize randomization plan; construct and sign MOU (Borman, McREL, SPPS) Randomly assign students to treatment and control schools Instruments
--
Adapt / develop data collection instruments for assessing implementation (McREL / Borman) Adapt / develop data collection instruments: Teacher Outcomes (McREL / Borman) Adapt / develop data collection instruments: Student Isolation (McREL / Borman) Adapt / develop data collection instruments: Student Engagement (McREL / Borman) Adapt / develop data collection instruments: Student Preparedness for Future (McREL / Borman) Data Collection
--
--
8 (Mar 28)
--
--
8 (Mar 28)
--
--
8 (Mar 28)
--
--
8 (Mar 28)
--
--
8 (Mar 28)
Download student data (MCA-II); screen and clean (SPPS) Download student data (SAT-10); screen and clean (SPPS)
8 (Nov 30) 8 (Nov 30) 8 (Mar/Apr)
--
---
---
Ÿ (Aug)
--
Ÿ
--
---
---
Ÿ (Aug) Ÿ
--
--
--
Ÿ (Aug) Ÿ
2
Practice:
Get quality data into your evaluator’s hands
Key Action:
Manage logistics and communications for data collection
--
Collect TEACHER data (McREL, SPPS)
Ÿ
--
Ÿ
Collect STUDENT data (incl. attendance data, honors course enrollment data) Analysis
Ÿ
Ÿ
--
--
Summer 2010
Spring 2010
Winter 2009
--
Fall 2009
--
Summer 2009
--
Spring 2009
--
Year 3
Winter 2008
Spring 2008
Collect IMPLEMENTATION data (McREL, SPPS)
Winter 2007
--- ongoing work Ÿinterim milestone 8 final deliverable
Fall 2007
Tasks
Fall 2008
Year 2 Summer 2008
Year 1
Ÿ
Ÿ
Ÿ
Ÿ
Ÿ
Analyze descriptive data (McREL)
--
--
--
--
--
--
--
--
Ÿ
Analyze student achievement data (Borman, McREL)
--
--
--
--
--
--
--
--
Ÿ
--
8 (Jul) --
--
--
--
Ÿ
--
--
--
--
8
Reporting Write Summary Report (McREL, Borman) Write Annual Performance report (McREL, Borman) Integrate findings from descriptive and inferential analyses (McREL, Borman) Write final report (McREL, Borman)
8 (Feb)
8 (Feb) 8 (Jul)
As shown in Table 3, deliverables include the following: o Research Plan (Borman, November 30, 2007) o Evaluation Plan (McREL, November 30, 2007) o Randomization Plan (Borman, McREL & SPPS, February 29, 2008) o Memoranda of Understanding (SPPS, February 29, 2008) o Start-up Year Summary Report (per Rigorous Evaluation direction, February 29, 2008) o Instrument development will be collaborative and ongoing; the following are targeted due dates: Instrument: Implementation (McREL, March 28, 2008) Instrument: Teacher Outcomes (McREL, March 28, 2008) Instrument: Student Outcomes (isolation, engagement, preparedness for future) (McREL, March 28, 2008) o Year 1 Annual Performance Report (McREL, Borman, July 2008) o Year 2 Summary Report (per Rigorous Evaluation direction, February, 2009) o Interim Report (McREL, Borman, September, 2009) o Year 2 Annual Performance Report (McREL, Borman, July 2009) o Final Report (McREL, Borman, June 2010)
3