Administrator 2.0

Administrator 2.0 By Tammy Stephens Last week the Stephens Group finished baseline evaluations for five EETT baseline evaluations involving 25 school districts, 315 teachers, 52 library media specialists, 77 teachers and 12,708 students. One of the things I have been busy doing the last couple of months is traveling around and working with administrators to help us gather data by conducting walk-throughs in their buildings. The Stephens Group approaches evaluation and both a formative and summative process. The evaluation process is a very collaborative process that will involve the grant evaluator, grant manager, grant leaders, and teachers and administrators in the grant. “The role of local engagement, collaboration, and feedback is paramount. Teachers and administrators at the local site should be participants in, rather than recipients of the evaluation” (Means et.al., 2003, p. 6). Evaluation research that is responsive to local concerns, constraints, and priorities can be structured and synthesized to produce knowledge about effective uses of educational technology that has high face validity within local communities and still informs wider research as well as practitioner and policy audiences” (Means et.al.,p. 6). Walk-through data of a consortium of school districts we worked with last year were able to track strong gains in 21st century skills and ISTE NETS Standards (see graphs on next page). In order to gain interrater reliability we view online videos of classrooms utilizing technology and rate them using the walkthrough checklist. We also work with administrators to develop coaching strategies to talk with teachers about their observations to help them move to higher levels of technology integration. The coaching model that we use is situational. Different coaching techniques are used depending on the level of technology integration a teacher is at. Birman et al. (2001) also found that coaching is responsive to the way teachers learn, “… and may have more influence on changing teaching practice” (p. 921).

Continued on next page

% of classrooms observed

21st Century Skills 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Priori Prod Adap tizing ucing tabili Socia Tech Infor Multi Glob Pers , Relev Scien Econ Visua ty/M l and Basic nolo mati cultu al Risk- onal Plan ant, anagi Creat tific omic l Civic on ral Awar Liter gical takin Resp ning, High ng ivity Resp Liter Liter Liter Liter Liter Liter enes g onsib acy and Quali acy Com acy acy onsib acy acy s acy ility Man ty plexit ility aging Prod y for… ucts Pre

68% 15%

Post 98% 65%

8%

67% 36% 30% 16% 17% 20% 29% 22% 29% 20% 64% 20%

5% 100% 65% 85% 33% 25% 35% 75% 20% 75% 38% 40% 43%

% of classrooms observed

ISTE NETS Standards 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Creati vity: applyi ng existin g knowl edge to…

Create origin al works as a means of perso nal…

Use model s and simula tions to explor e com…

Identif y trends and foreca st possib ilities

Intera ct, collab orate, and publis h with peers, expe…

Comm unicat e inform ation and ideas effecti vely…

Contri bute to projec t teams to produ ce…

Locate , organi ze, analyz e, evalua te, synt…

Pre

20%

22%

10%

13%

9%

12%

14%

23%

10%

9%

23%

4%

Post

50%

45%

40%

23%

58%

55%

35%

55%

53%

38%

48%

8%

Troubl Solve Collect eshoo authe , t ntic Exhibit proces syste real leader s, and ms world ship analyz and proble e data applic ms ations

This year we are using a second walk-through checklist that measures ACOT Levels of technology integration and can track changes in teacher practice over time with two consortiums.

Administrator Name: District: School: Date of walk-through observation: Number of classrooms observed: Learning Environment Students have no interaction with other students Students collaborate with peers Students are provided opportunities to use higher order thinking skills Technology access is adequate to meet lesson objectives Students with special needs have access to appropriate hardware and software

Observed

Technology Use: Who used the technology in the lesson or activity observed? All students Some students One student The teacher

Observed

Teacher Role: What was the role of the teacher during the lesson or activity observed? Leader Facilitator Observer

Observed

Lesson Implementation

Technology use is not clearly related to lesson objectives The lesson is focused on learning a technology skill Traditional assessment methods including paper and pencil are used to measure student outcomes Technology use is somewhat related to lesson objectives Technology is used for drill and practice, tutorials, or as a free time activity Productivity tools and courseware are used to augment the lesson Technology is used with little or no management problems Student outcomes are often measured using teacher developed rubrics

Observed

Objective, Instruction, or Assessment O

Level

Adoption

I

Adoption

A

Adoption

O

Adoption

I

Adaptation

I

Adaptation

I

Adaptation

A

Adaptation

Lesson Implementation

Technology skills are learned in the context of the lesson objectives Would not be possible to meet lesson objectives without the use of technology Students use technology to engage in authentic tasks Lesson objectives including technology use encourage student choice and planning to complete assignments Lesson objectives are designed to provide students with opportunities to demonstrate learning outcomes using technology Lesson objectives permit students to initiate technology use for learning and assessment Technology use provides opportunities to expand student interactions beyond the classroom Students are encouraged to seek new uses of hardware and software for learning Student products are assessed using student developed rubrics and portfolios. Comments:

Observed

Objective, Instruction, or Assessment O

Level

Appropriation

O

Appropriation

I

Appropriation

O

Invention

O

Invention

O

Invention

I

Invention

I

Invention

A

Invention

This checklist was modified from the one created by Susan Brooks & Harvey Barnett (2002).

You can read more about the evaluations we are doing on our website on our research page at http://www.thestephensgroup.com/research.html . References: Birman, B. F., Desimone, L., Garet, M. S., Porter, A. C., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Research Journal, 38(4), 915-945. Means, B., Haertel, G., & Moses, L. (2003). Evaluating the Effects of Learning Technologies. In G. Haertel & B. Means (Eds.), Evaluating educational technology: Effective research designs for improving learning. New York: Teachers College Press.