Fipse Guide

“If we took the research on persistence seriously, we would, at a minimum, move to forms of academic organizations that require students to become actively involved with others in learning. We would construct educational settings that promote shared, connected learning.” Vincent Tinto in Colleges as Communities: Taking Research on Student Persistence Seriously. Review of Higher Education. 21.2(1998) 167-177.

“Teaching with S.T.Y.L.E. (situated technology yielding learning enhancement)”

A grant for Integrating Electronic Technology into Liberal Education & Professional Education Classes” FIPSE Grant 1998-2000 by Dr. Barbara K. Iverson • Columbia College Chicago 1

What's Inside

3 ........................................................................................................................................Overview 4 ....................Teaching with S.T.Y.L.E. (situated technology yielding learning enhancement) 5 ...........................................................................................What happens during the semester? 5 ....................................................... Measuring Student Teamwork and Student Achievement 6 ......................................................................................................... Why Teach with S.T.Y.L.E? 6 ..........................................................How do Faculty Learn about Technology “on-the-job?” 6 ...............................................Writing some learning objectives for the collaboration project 7 ....................................................................Project Objectives Evaluation Strategy Summary 8 .................................................................................................................... Digital Skills Invento 10 .......................................................................................................... Product Evaluation Form 10 .......................................................................................................... Evaluation Questionnaire 10 .......................................................................................................... Teacher Evaluation Form 11 .....................Student Evaluation Form -- The L.E.I. or Learning Environment Inventory 13 .............................................................................. Summary--Teaching with S.T.Y.L.E. Steps 13 .................................................................. What we have learned this far--benefits and risks 14 ................................................................................................................. What’s best & worst? 14 .......................................................................................Are faculty getting new digital skills? 15 ...........................................................................How did students fare in the collaborations? 15 .................................................................................................Tech Classes vs. Control Group 16 ............................. Latin American Art Music & Culture Content Class vs. Control Group 16 ................................................................Nutrition Science Content Class vs. Control Group 17 ..................................................................................................................................Conclusions 18 .......Highlights of F.I.P.S.E. Annual Progress Report (1998-1999) and Current Semester’s Progress 19 ........................................................................................................................... REFERENCES 21 ..................................................................................................................................Appendix A 21 ................................................................................................................... The Grant Proposal 21 .....................................................................................................................I. Project Narrative 23 ......................................................... II. Proposed strategies for improving current practices 27 ...........................III. Institutional commitment--contribution of the College to the project 27 ............................................................................................. IV. Evaluation and Dissemination

2

Overview Our project is about active learning -- for faculty and students -- as part of collaborative, team-based projects. We want to use digital technology tools (computers, the Internet, e-mail) to create better learning environments in our classes. The integration of the technology into teaching creates new ways of organizing instruction, assessing progress and learning, and of scheduling time and activities. Faculty learn to use the technology “on-the-job” in specific classes, which is time-saving and eases their learning curve. Students get to participate in an active, project-based assignment that draws on multiple learning styles, and opportunities to develop higher order thinking skills (perspective-taking, synthesis, analysis, evaluation.) As knowledge makers, students experience a sense of ownership and empowerment that they miss as passive knowledge consumers. Our model, funded in part by a Federal Improvement of Post-Secondary Education grant sets up a simulation of the contractor/client relationship common in business and professions today. It pairs up a “content” class (the clients) with a “technology” class (the contractors) to make an interactive hypermedia presentation for the web and CD-ROM. The “content” teacher gets help from the “tech” teacher in integrating use of the Internet, electronic presentations, e-mail, electronic forums and bulletin boards into their classroom routine. The students learn in context and by doing, rather than by rote. In general, the teamwork inherent in the model increases the cohesiveness of classes involved in the project, which is associated with higher achievement scores. The simulation and project represent a scalable shared assignment developed by the participating teachers to fit the constraints and needs of their specific classes. The introduction of the technology-supported project into existing classes provides a way to update teaching throughout the institution without creating new curricula or hiring new faculty. The grant provides funding for “replacement costs” for participating teachers to ease departmental concerns about workload issues. It funds teaching assistants for each class, as well. The teacher partners negotiate the specific topics that will be covered and what the final outcome project will be to suits their needs. Teachers structure the team-based project to achieve their“same old” learning objectives in new active, collaborative ways through the team project. They need to meet with the partner class, as well as their own, and to use their T.A. to help facilitate the collaboration. The project can be continued for several semesters, if that is agreeable to the participating faculty, or it might last a single semester. An outside evaluation consultant conducts qualitative and quantitative assessments of the project and its efforts (see Evaluation to Date.) Teachers and students try out tech tools as they work on the projects. After pariticipating, teachers can continue to collaborate. They can incorporate internet research into all their classes. They can create team projects presented as computer slideshows. They can use electronic forums to stimulate discussion or check on students’ understanding of reading assignments. The aim of the project is not to prescribe a single use or method of technology, but to give teachers the experience they need to integrate available tools into the dialogue, communication, and teaching/learning process. Project Director, Dr. Barbara K. Iverson

3

Teaching with S.T.Y.L.E. (situated technology yielding learning enhancement) To participate, you need a partner. If you teach a class with a technological focus, look for a partner whose class focus is on content, and vice versa. For example, a New Media Projects teacher teams with Senior Seminar for a project to create a virtual job placement and information center. The project director can help you find a partner. The partnered faculty meet to decide what form your project will take. A poet works with a time-based composing class on interactive literary collages. A psychologist finds multimedia students who can animate “non-locality.” The teachers decide if the project will be one of several assignments, or the main assignment. They set how many weeks to schedule for it. You plan for using resources like computers and the Internet , to get the assignment coordinated and provide opportunities for teamwork and collaboration within and between classes. The “tech” teacher will help the “content” teacher get up to speed with e-mail attachments, using the internet, and other tech tools. This is an exciting time where you can come up with creative ideas, and work to bring them to life. You need to share syllabii, and figure out what the project will look like or “be.” Will it be a website for future classes to use? Will it be an interactive compilation of art or poetry (or both?) Will it be a new way of making something? The project outcome is flexible and scalable, and wellsuited to a variety of cross-discipline collaborations. The partner teachers should set the dates when they will bring the classes together (ideally the class meeting times allow for face-to-face meetings) or when the teachers will visit their partners’ classes. How will information be passed from clients to contractors? How will clients monitor the progress of the contractors? The faculty partners set this up based on the scope and needs of their specific project. The project director will meet with as often as you like. As part of the collaboration, you will be responsible for producing a common product out of the collaboration. You and your partner will be expected to use and experiment with e-mail, electronic forums, computers and ways of sharing information and working together both face-toface and “asynchronously.” You can use the tools with each other, and with your students in ways that work in your situation. The project is evaluated through collecting faculty and student opinions about the project, and comparing that with a series of questions about the classroom learning environment (how cooperative and cohesive students feel about the class) and an inventory of faculty digital skills. Research shows that classes with a sense of cooperation, cohesiveness as a group, satisfaction, and a sense of having a say in the class (democracy) do better on achievement outcomes than classes without a positive sense of “groupness”. Part of the reason for building the project around a team-based collaboration is to build up the learning environment of your class so it will promote higher achievement. The team-based project assignments provide opportunities to increase the positive sense of “teamness” and collaboration, and by inference achievement. Motivation increases in active-learning situations, which leads to students spending more time on task, and learning in classes where they are engaged. You are still the one who evaluates achievment in your subject and class. However, we can compare disparate classes and draw conclusions about the team-based project approach to integrating technology into classes by measuring the learning environments we create, and attempting to determine if we are changing them in positive ways because of our team-based projects. 4

What happens during the semester? The semester begins and the project is introduced to the classes. When the initial learning in both classes permits, the project assignment begins. E-mail, electronic forums, networked computers and ftp and websites all facilitate how information passes between the two classes. The project is built and reviewed by the clients, revised and added to as comments and work comes in from the client class. Faculty need to remain flexible, as they determine what changes they may need to make during the weeks of class as they see how the plans they made play out in the everyday classroom dynamics. During the semester the evaluator will take measures via quick questionnaires of student and faculty opinions and perceptions. The project itself is an additional measure of the progress and success of the collaboration.

Measuring Student Teamwork and Student Achievement

Faculty continue to evaluate the academic achievement for their classes in the manner they see fit. For project evaluation, students complete a questionnaire which measures six aspects of the learning environment we create (see Table 1.) This questionnaire has been carefully developed and studied so that we know which characteristics of a classroom learning environment are associated with increased student achievement in any class (Fraser, Anderson & Walberg, 1982.) If we try and measure student achievement directly, and then compare our classes, we will run into an “apple and oranges problem” — how do we compare what students learned in Women’s History, with what they learned in CD-ROM Multimedia Production? However, if we measure the “learning environment” we consciously create, we can make reliable inferences about what is happening with student achievement in our classes. For example, we know classroom cohesiveness (a feeling of being a team) and student satisfaction are associated with subject-matter achievement gains. We assess the learning environments of project classes and compare them with similar, but traditionally organized control-group classes to see whether our team-based, collaborative approach is associated with higher student achievement. We rate the learning environment using a Learning Environment Inventory that measures: cohesiveness (sense of being a “team”), diversity (extent to which differences between students exist and are provided for), material environment, democracy (extent to which students share equally in decision-making related to the class), and satisfaction (extent of enjoyment of the class.) Cohesiveness refers to how favorably students see themselves and their work as a team. Diversity reflects whether students with different abilities and interests are encouraged. The material environment refers to availability appropriate of books, equipment, and space. Democracy measures the perceived student input in classroom decision-making. Finally, satisfaction gauges whether the students enjoy the class. These scales were selected to assess the fulfillment of the project objectives to improve cooperative and active learning. The Internet is available on campus, but not always integrated into classroom learning. Teacher participants and student participants use e-mail, electronic bulletin boards, listservs, and other Internet-based tools to keep in touch and collaborate. As the content teachers are guided in their use of this technology, they begin to apply what they learn in their research and 5

other assignments. By learning to use the particular resources at hand while they were teaching e.g. on-the-job, the cost of faculty training is kept low. Teachers who have used technology successfully in their classrooms become better judges of what new innovations might be useful, and which are not important to their discipline or teaching style.

Why Teach with S.T.Y.L.E? This model and how we assess it are based on research on ‘what works’ to improve classroom learning so it provides a framework to guide us in designing the team-based assignments that are flexible, adaptable to existing classes, and effective in promoting student learning and faculty technology use. It is based on what we know about retention and how learning takes place. We know now where we need to be headed (e.g. toward creating a sense of “team”, toward allowing for diversity within an environment where we have clearly stated learning goals) and we can structure a collaboration that lets students attain learning objectives in setting that supports active, potent, social learning. The fear of giving up “teacher control” is real, but the payback in terms of facilitating authentic higher order thinking and enjoyment of learning, coupled with the pleasure arising from collaboration with a fellow teacher in the team-based, collaborative learning environment makes it worth the risk.

How do Faculty Learn about Technology “on-the-job?” By trying out and using technology while they work, teachers learn tech skills easily, and in context. This is the “Trojan-horse” approach to teaching faculty to use technology to improve their teaching -- by focusing on the teaching, we reduce the anxiety around technology and shorten the learning curve. The Digital Skills Inventory we administer measures changes in faculty tech skills. The skills it measures are targeted to correspond to areas where content faculty are expected to gain confidence and expertise because of their involvement in the project.

Writing some learning objectives for the collaboration project Here are sample objectives that relate to the teamwork objectives of the project. Please include something like this on your syllabus. These can be the same for the tech and content teacher. By including these in your syllabus, they will become a part of the classroom learning environment:

•“Students will develop a sense of being part of team as a result of class activities” (cohesiveness)

•“Students will be encouraged to express and accept a range of opinions about the project and to work on topics of individual interest, as well as achieving group goals” (diversity)

6

•“Students will participate democratically in goal setting in creating the joint project” (democracy)

•“The project will combine goals set by the instructor and goals determined by the students, and the class will produce a clear list of these goals to guide the project” (goal direction) •“Students will express satisfaction with their participation in the team-based collaborative project as well as other class activities.” (satisfaction)

Project Objectives Evaluation Strategy Summary

7

Digital Skills Inventory There are no right or wrong answers on this checklist. We are assessing faculty computer skills related to a project we are conducting to integrate technology into classrooms. This information is provide us with baseline data for assessing our project. Your responses are appreciated. Please circle the response that best characterisizes your skills at this time. Thanks. Word Processing and Using Documents

1

Do you use a computer to:

Never

Write letters, notes, or flyers?

1

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Never 1

Seldom 2

Often 3

Always 4

Never

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Never Can you annotate bookmarks & make bookmark webpages in a browser? 1

Seldom

Often

Always

2

3

4

Never

Seldom

Often

Always

Never Type academic papers or journal submissions ?

1 Never

To write press releases and artist notes?

To create syllabi & course handouts?

To complete administrative tasks

1

1 Never

Telecommunications Do you use the on-line catalog of any library?

1 Never

Do you browse the WorldWide Web (Internet) ?

1 Never

Do you use the Internet as a research tool in your work?

1 Never

Have you created a web page?

1 Never

Do you have a personal webpage or web page for your classes?

1

8

Do you know what FTP is, and use it?

1 Never

2

3

4

Seldom

Often

Always

Do you use e-mail for personal communication? If no, skip to the next set of questions. Do you have your own e-mail account?

Use e-mail with friends?

Use e-mail to communicate with students and other professionals?

Do students submit homework for your class electronically?

Do you use a listserv or chatroom with students?

1

2

3

4

Never 1

Seldom 2

Often 3

Always 4

Never

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

2

3

4

Seldom

Often

Always

1 Never 1 Never 1 Never

Images & Graphics Do you scan images to use with a computer?

1 Never

Have you cropped or resized and manipulated digital images?

1 Never

Use computer images in your written communications?

1 Never

Create charts & graphs with a computer and use them in your work?

1 Never

Use clipart?

1 Never

Desktop Publishing Create computer newsletters, flyers, programs, or schedules?

Add and manipulate images in these documents easily?

1 Never 1 Never

Usee presentation software like PowerPoint?

1 Never

9

Product Evaluation Form Name________________________

Date__________________

Dept._______________________

Name of Product being evaluated:_________________________________ Instructions: You should have received a product (CD ROM or website address) produced by the Integrating Electronic Technology in Liberal and Professional Education Classrooms project. Please respond to the following questions and return this form to Barbara Iverson, Project Director, within one week. Thank you. 1. What is your opinion about the subject content of the product? 2. What is your impression about the multimedia aspects of the product? 3. What was the best thing about the product? 4. What was the worst thing about the product? 5. Do you have any suggestions to improve the product? 6. What is your overall impression of the product? 7. Any additional comments?

Evaluation Questionnaire Instructions: With respect to (course title) please answer the following questions and reply with your answers by (date). 1. In your opinion, what are the goals of the class? 2. What is the best thing about the class? 3. What is the worst thing about the class? 4. What do you suggest be done differently? 5. How would you compare this class to other courses in this discipline?

Teacher Evaluation Form Instructions: Please answer the following questions with respect to, “Integrating Electronic Technology in Liberal and Professional Education Classrooms.” 1. What were the goals of the project? 2. Were the goals met? 3. What was the best thing about the project? 4. What was the worst thing about the project? 5. What do you suggest should be done differently? 6. Do you think this project represents a cost-effective method of instruction? 7. Was this project an effective way to integrate computers into the classroom? 8. Were your computer and digital skills improved as a result of this project? 9. Has this project changed your teaching? 10. Did you or do you plan to share project ideas with those outside the project? 11. Do you think this project would benefit other campuses?

10

Student Evaluation Form -- The L.E.I. or Learning Environment Inventory Instructions: Please answer the following questions with respect to (course title) using the following scale: Strongly Agree = 4 Agree = 3 Disagree = 2 Strongly Disagree = 1 Please respond to all the questions and place your answer (whole numbers only 1, 2, 3, or 4) in the left column next to the question number. Please reply with your responses by (date). Thank you. 1. Members of the class do favors for one another. 2. The class has students with many different interests. 3. The books and equipment students need or want are easily available to them in the classroom. 4. The class knows exactly what it has to get done. 5. Class decisions tend to be made by all the students. 6. The students enjoy their class work. 7. A student has the chance to get to know all other students in the class. 8. Interests vary greatly within the group. 9. A good collection of books and magazines is available in the classroom for students to use. 10. The objectives of the class are not clearly recognized. 11. Decisions affecting the class tend to be made democratically. 12. Personal dissatisfaction with the class is too small to be a problem. 13. Members of the class are personal friends. 14. Some students are interested in completely different things than other students. 15. The students would be proud to show the classroom to a visitor. 16. Students have little idea of what the class is attempting to accomplish. 17. Certain students have more influence on the class than others. 18. Many students are dissatisfied with much that the class does. 19. All students know each other very well. 20. Class members tend to pursue different kinds of problems. 21. The room is bright and comfortable. 22. The objectives of the class are specific. 23. Certain students impose their wishes on the whole class. 24. There is considerable dissatisfaction with the work of the class. 25. Students are not in close enough contact to develop likes or dislikes for one another. 26. The class divides its efforts among several purposes. 27. There are displays around the room. 28. Each student knows the goals of the course. 29. Each member of the class has as much influence as any other

.

11

30. The members look forward to coming to class meetings. 31. The class is made up of individuals who do not know each other well. 32. The class is working toward many different goals. 33. The classroom is too crowded. 34. The class realizes exactly how much work it is required to do. 35. What the class does is determined by all the students. 36. After the class, the students have a sense of satisfaction. 37. Each student knows the other members of the class by their first names. 38. Different students vary a great deal regarding which aspects of the class they are interested in. 39. There is enough room for both individual and group work. 40. Each student in the class has a clear idea of the class goals. 41. A few members of the class have much greater influence than the other members. 42. Students are well-satisfied with the work of the class.

12

Summary--Teaching with S.T.Y.L.E. Steps 1. A pair of collaborators decide to work together, and meet with the project director 2. The pair share syllabii, look over suggested learning objectives for collaboration and team outcomes, and discuss a shared project. 3. The pair meet and create a shared project description. Project Director can write up the details as a product descriptor for each teacher to use with their class (the descriptors for each class will be different because their activities and contributions to the project are different.) 4. E-mail, electronic forums and FTP and websites are established, and faculty try these out and learn about them prior to meeting with their classes, as part of the planning of the project. 5. Dates for the classes to meet (synchronous or asynchronous) are set. When the teachers will visit each other’s classes is set and discussed. 6. Evaluation times are set at the beginning, middle and end of the project. 7. Semester begins, project begins as planned (not usually until the 4th or 5th week of the semester.) 8. Students share info and input about and for the project. The clients view the work in progress, and suggest revisions. Contractors prepare the work for shared viewing. 9. The project is completed. Final evaluations take place. 10. Teachers and project director “debrief” on how the project went. Revisions are made if the collaboration is going to be carried out again.

What we have learned this far--benefits and risks Since beginning this project at Columbia, teachers have increased and refined how they use e-mail with their students, experimented with electronic forums, and worked to get their departments to purchase tech resources they need to support the integration of tech into their teaching. As a direct result of participation by several faculty from Liberal Arts and the efforts of the project director, all Lib.Arts classrooms are wired for internet and two computer/tech carts are being assembled (computer, internet hookup, scanner, printer, DVD, CD-ROM, PowerPoint, web browsers, easily moveable robust cart with lots of electric outlets.) The lag between the initial collaborations with the promise of this supportive technology and its acquisition has attenuated the positive effects of the grant because the teachers who used the technology during the grant projects have been unable to continue to use the tools they learned in their subsequent teaching. With the late, but welcome acquisition of supportive hardware and software, the benefits of the grant will be obvious, and will begin to percolate throughout the institution. Teachers who have had access to computer and internet use in classrooms continued and have increased their use. Obviously those whose access is limited have not been able to use what they learned in the grant effectively--but new equipment will change this. 13

What’s best & worst? This table shows what teachers who participated in the program said were the best and worst aspects of being in a F.I.P.S.E. collaboration. The teachers rated the technology, the collaboration, and the shift in role from “sage on the stage” to facilitator postively. They indicated problems or shortcomings in the communication between classes, caused by the lack of overlapping meeting times, and the necessity for asychronous meetings facilitated by internet, e-mail and electronic communication tools. On our campus, this is not the common practice, nor within student expectations at this time. The difficulty scheduling these classes in overlapping timeslots seems insurmountable, so probably the novelty of asynchronous electronic communication will wear off before scheduling is easier, and this kind of communication will become routine as it is on many campuses.

Are faculty getting new digital skills? This table demonstrates that teachers in the collaborations were improving their digital skills. Little gain in word processing was measured because most teachers entered with good skills in word processing. We are revising this survey to try and measure the changes in teacher skills in more detail. Anecdotal evidence shows teachers especially liked the electronic bulletin boards, but the lack of computer facilities available to the teachers after they participated in the project interfered with the greater use of the this useful tool. In fact, the interactive products have not been used to the best advantage as resources in classes because of the lack of classroom internet access. 14

How did students fare in the collaborations? First we compared the learning environment scores for all of the grant classes and a comparable control group in traditional classes. Significant differences in the learning environments were found for “cooperation,” cohesiveness,” “material environment,” “goal direction,” and “satisfaction” between the experimental (content & tech classes) and the control groups. The F.I.P.S.E. classes had higher cooperation, cohesiveness, and material environment scores which is what we would expect because of our emphasis on teams and cooperation and collaboration in the S.T.Y.L.E. model. Higher learning environment scores on these variables are indicative of higher achievement. We can use the LEI scores because we can’t compare achievement scores directly in our various partnered situations. The material environment score probably reflects the use of lab and computer facilities in the experimental groups, while the control group met in a traditional classroom.

Tech Classes vs. Control Group The tech classes outcomes were compared to control group outcomes with results similar to the comparison of all experimental classes to control groups, except that there was no significant difference between satisfaction. Cohesiveness and material environment perceptions were higher for students in the tech classes, suggesting higher achievement of cognitive outcomes for the students in these classes. This analysis is also tentative because the control group consisted of students in content classes, but no tech classes were included in the control owing to some confusion about what tech classes might constitute comparable groups. In future analyses, the control group will include both content type classes and tech type classes. 15

Latin American Art Music & Culture Content Class vs. Control Group This class had a significantly higher cohesiveness score than the control group. These students were primarily Latino and in their comments expressed pride that others were interested in their culture, which could added to the perceptions of cohesiveness which would be created through the team approach and group projects. The direction of this correlation suggests that achievement was higher for these students than for students in the control classes.

Nutrition Science Content Class vs. Control Group The analysis of results for the Nutrition Science class reflect the situation for that class. That it met in a computer lab as well as a traditional classroom is reflected in its higher scores for Material Environment. The sense of cooperation was higher for this class, indicating a greater sense of team work and interdependence. However, the teacher of class was enthusiastic, but inexperienced with the use of Internet as a teaching tool in class. Some of the teaching objectives for the Nutrition Science class began to drift, as the class spent time surfing the net, but without guidelines or a concrete set of tasks to accomplish as a result of the surfing. The project director met with this class halfway through the course because of confusion about the nature and amount of work being required in the class and things went better, but some student dissatisfaction remained. These problems had not occurred in other collaboration settings because the content teachers maintained greater focus on the outcomes and objectives for their own classes, and had 16

used Internet and other digital technology to streamline what they already did. Teachers used Internet research, bulletin boards and e-mail to carry out their learning objectives. The Nutrition Science teacher inadvertently got off-track in her class, focusing on the tools of digital and electronic technology without guidance of learning objectives for a nutrition science class. The Nutrition Science teacher had assigned students to “research” nutrition on the Internet, almost exclusively, and because this was new territory, the parameters of what “research” entailed were unclear to students, who were browsing and reading sites. In traditional research, students identify relevant sources, read them, and then write or otherwise summarize, generalize, or analyze what they have learned from their research. For effective use of Internet research a similar process must be set up, as browsing and reading sites produces no evidence of the research or learning. The students felt they were working hard, but had little to show for their work, either to share with their collaborators, or to turn in for a grade. The dissatisfaction with the class was expressed by students, and remediated somewhat in meetings with the teacher and project director during the semester, but it was expected that this class would have less perceived satisfaction with the class than other experimental classes.

Conclusions The project goals involving teamwork and collaboration appear to be succeeding, as student perceptions of cooperation and cohesiveness were greater in experimental than control group classes. This suggests that cognitive learning may be greater in these classes than in the control group, which is consistent with the theoretical underpinnings of the model for improving education which is being tried out and evaluated. Student satisfaction with the collaborative pairings does appear to be greater for experimental classes, except where the teacher got “internet infatuation” and strayed from her learning objectives during the semester. Future partners will be told to be careful of this pitfall, and the project director will monitor not only written course materials like the syllabus, but also the initial class meetings of content teachers to assure the proper use of internet, etc. as tools, and not as “ends” in themselves. This practitioner research effort draws on “what works” to improve classroom learning. It helps faculty understand and use technology effectively as an educational tool. It provides a framework to guide the design of team-based assignments that is flexible, scalable, and adaptable to a range of teaching styles. By reengineering the teaching actions (e.g. toward creating a sense of “team”, toward allowing for diversity within an environment where we have clearly stated learning goals) teachers can use their experience and judgment to get there. The fear of not knowing exactly what the class will end up producing, and of giving up some “teacher control” is real, but the payback in terms of genuine student engagement and enjoyment of learning, and the pleasure of collaboration with a fellow teacher in the team-based, collaborative learning environment makes it worthwhile. There is danger in the hype and novelty of digital and electronic technologies, and in losing one’s pedogical way amidst the glitter of tools not serving teaching aims. However, the approach provides guidelines to keep the primary learning objectives in focus as you new teaching tools. This approach begins to ...”move to forms of academic organizations that require students to become actively involved with others in learning.” and to “... construct educational settings that promote shared, connected learning.” (Tinto, 1998)

17

Highlights of F.I.P.S.E. Annual Progress Report (1998-1999) and Current Semester’s Progress The technical difficulties of sharing the work between the participating classes when we were doing CD ROM-based projects in the first semester has caused us to move to the creation of projects for the internet (http://digdesign.colum.edu/fipse) from the get-go. The work is easier to share between the classes, and can thus be produced in a more interactive fashion. During the first year of the project, we completed 3 collaborations. At the end of Fall 1999 semester, the number will be six: • Women’s History class and Interactive Programming for Business and Training class (website) • a Nutrition Science Class and an Interactive Multimedia Producer Class (website) • a Latin American Art, Music & Literature class and Interactive Programming for Business and Training class (website) •Latin American Art, Music, & Literature and New Media Projects •Psychology of Creativity and Interactive Multimedia Producer (websites) •Time-based Composing and Literary Collage The completed projects and some works-in-progress are visible from: http:// digdesign.colum.edu/fipse. The site is being “renovated” by a web consultant who will master an interactive CD-ROM guide to the projects, information about the model and grant, and resources on teaching collaboratively and related topics. • As a result of the liberal arts/computing collaboration class, the College installed an internet drop into one liberal arts classroom (NOTE:now all Lib arts classrooms are wired through efforts of project director and Lib Arts chair. • We are completing the ordering and installation of 2 computers, projectors, and carts for the liberal arts faculty to use in the classroom with the Internet drop. For the first time, this department has added equipment requests like this to their budget request • One of the “non-technical” teachers has agreed to design & deliver a presentation to her department demonstrating how to use the Internet hook-up and the computer/projector on a cart. There are currently 5 faculty who have expressed an interest in using this technology in their classes • There are more collaborative projects for use as classroom resources for teachers at Columbia (Women’s History part 1, a Nutrition Science site and a site to accompany the Latin American Art, Music & Culture class.) • A collaboration between a Time-based composing class (digital movie production) and a poetry class called “Literary Collage” is in the works (NOTE: it will be completed this semester.) • Students and teachers are using e-mail to communicate with each other. Electronic 18

forums and web pages for sharing e-mail addresses and information are now a part of the participating classes, with “spill over” to participating faculty’s other classes, and with some colleagues now adopting the electronic forum bulletin boards to their classes. • Our first set of quantitative results, show significant trends in increased cooperation and cohesiveness for the classes involved in the collaboration. • Two fashion classes will participate in the Spring • A senior seminar class will participate in the Spring The project director must follow-up on the requests for the computer presentation stations for the Liberal Arts Department. They have been promised, but we must make sure the college rewards faculty who have taken a change, taught innovatively, and want to continue to integrate technology into their classes. (NOTE: Since this report was filed, all Lib Arts classrooms have been “wired” and two computer carts with projectors and software have been ordered for the Lib Arts Dept. with the help of the project director.)

REFERENCES Archambault, R. D. (Ed.) John Dewey on Education. University of Chicago Press, Chicago, IL, 1974, pp.150-1. Csikszentmihalyi, M. The Evolving Self A Psychology for the Third Millennium:. HarperCollins, New York, NY, 1993. Iverson, B. K. “Praxis What You Teach: Making Interactive Multimedia around Diversity Issues & Topics”. in Gardaphe, F. (Ed.) 1997. Jonnassen, D.H., Peck, K. L. & Wilson, B.G. Learning with Technology: A Constructivist Perspective. Prentice Hall, Upper Saddle River, N.J., 1999. Fraser, B.J., G.J. Anderson, and H.J. Walberg. (1991). Assessment of Learning Environments: Manual for Learning Environment Inventory (LEI) and My Class Inventory (MCI). Perth, Western Australia: Curtin University of Technology, Science and Mathematics Education Center. Iverson, B. K. & Prados-Torreira, T. Liberal Ed meets High Tech:Interdisciplinary Collaboration to Improve Instruction. World Association for Case Method Research and Application Conference, Extremadura , SpainI, July, 1999, in press. Iverson, B. K. Tutorial ABCs of Collaborative Multimedia Projects in the Classroom. Tutorial for EdMedia99, Seattle, Washington, June, 1999. Iverson, B. K. Making New Media in Everyday Classrooms. Presentation at Lilly Atlantic Conference, Towson College, Baltimore, MD, April, 1999.

19

Jonassen, D.H. & Grabinger, R.S. (1992). “Levels of processing in building expert systems.” In P. Kommers, D. Jonassen, & T. Mayes (Eds.), Cognitive tools for learning. Heidelberg, FRG: Springer-Verlag. McLuhan, M. in Southam Interactive Understanding McLuhan: In the Electric World Change is the only Stable Factor. Voyager CD-ROM, 1996. Schank , R. Engines for Education website, 1994 (http://www.ils.nwu.edu/~e_for_e/). Schank, R.C. Goal-Based Scenarios. The Institute for the Learning Sciences, Northwestern University, Evanston, Illinois., Technical Report #36 December 1992. Tinto, V. Colleges as Communities: Taking Research on Student Persistence Seriously. The Review of Higher Education 21.2 (1998) 167-177. URL: http://muse.jhu.edu/journals/ review_of_higher_education/v021/21.2tinto.html Walberg, H.J. (May 1984). “Improving the Productivity of America’s Schools.” Educational Leadership 41, 8: 19-27.

20

Iverson, B. K. Digital Technology & the Teaching of Women’s History. presented at “The Stuff of Women’s History: Using Artifacts, Landscapes & BuiltEnvironments to Research & Teach Women’s History in the Classroom,” Organization of American Historians & the National ParkService. Seneca Falls, N. Y.,August 21, 1998 with Teresa Prados-Torreira. Iverson, B. K. & Prados-Torreira, T. The making of warp & weft,might & magic, mettle & motherhood: an electronic exploration of AmericanWomen’s History: 1600s to 1870’s. SIGGRAPH98 Education Program,Orlando, Florida, July 1998. in SIGGRAPH98 Conference Abstracts and Applications, July, 1998. Iverson, B. K. & Prados-Torreira, T. Integrating Women’s History and CyberTechnology. Lecture & presentation at Women’s INterVENTIONS in Science, Art, and Technology,Fifth Women’s Studies Symposium. Purdue University, March, 1998. URL:http://digesign.colum.edu/ FIPSE/WHpaper.html

Appendix A The Grant Proposal The entire text of the original grant proposal is presented for your convenience in the following sections of this guide to Columbia’s FIPSE grant.

I. Project Narrative The project that Columbia College Chicago proposes will address three fundamental problems facing liberal arts colleges today. First, the problem of how to actively involve students in their own learning, and prepare them to make use of what they learn later in life is addressed. Secondly, it addresses the question of how computers and digital technology can be used and integrated into the curriculum rather than just added on to existing course offerings. And finally, the question of how to provide faculty with an effective means of acquiring computer and digital skills without adding on to their current workload is addressed. Pairing up traditional “content-rich” lecture / discussion classes with new “technologyrich” computer production classes to create collaborative interactive projects (websites and CDROMs) to supplement typical course work is an effective way to integrate digital technology into classrooms and to provide students with more opportunities for active learning through participation in the projects. Faculty seeking to learn how to use and integrate computers and digital technologies into their teaching can acquire new skills in a timely and cost effective manner by working with another teacher and class on a collaborative digital project. By acquiring skills in image capture, Internet use, and authoring, making “knowledge maps” (see Appendix A), and computer productivity while on the job, these teachers model how one “learns to learn” for their students. The ability to retrain oneself and to adapt to new technologies may be novel, but it will be a core skill in the next century. We will adopt this strategy to help our faculty optimize their time by teaching and learning concurrently, and encourage them to pass this skill on as part of their teaching repertoire. The proposed project asks for funding for T.A.s to assist each of the teachers in the paired classes. Graduate students are not readily available at our institution owing to its focus primarily on undergraduate studies. Therefore, the T.A.s will be undergraduate students chosen by the participating faculty because they have computer experience and demonstrate a degree of leadership and initiative necessary for the kind of liaison work expected of them. They do not require special training, as their main responsibilities will be in assisting with word processing and Internet 21 use by students who are new to computing, and making sure that non-digital information can be shared between the classes. They will work with individuals or small groups at the request of the

tive, train faculty in a cost effective, organizationally sound way, strengthen links between the general and professional curricula, and actively involve undergraduate students in research. The model can be adapted to various projects so that schools can fit it to available technology and the skills of faculty on their campuses. At Columbia, the project will strengthen and modernize our liberal and professional education courses which are largely lecture-based and utilize few computer-mediated communication tools. A cost effective feature of this project enabled by the use of computer-mediated communication including electronic mail (e-mail), Listserv technology, and the posting of shared materials on a website, is that the paired classes do not need to meet on the same day, at the same time, or in the same space, in order to collaborate. From any computer on campus, or from any remote computer with Internet browsing capability and access to e-mail, students can review each other’s work, suggest revisions, and discuss any aspect of the collaborative project. The project T.A.s can provide help to students new to computer-mediated communication and can facilitate the exchange of non-digital course-related material. Traditional teamwork with its important social and interpersonal dimensions will occur face-to-face in the primary classrooms. The teamwork between the paired classes will occur electronically and be nontraditional, in that students will be not be meeting in person. The faculty will introduce this idea to students, and where schedules permit, students will have the opportunity to meet in person at the beginning of the collaboration. The T.A.s and faculty will need to provide support for and incorporate the information from this new, asynchronous channel of communication into the discourse of the classroom so that it becomes integral to the overall project. The formative evaluation of the project will provide feedback on how the students relate to their real team members and to their “virtual” team members. A presentation session at the end of the semester will allow the collaborators of the traditional and the virtual kind to meet in person around a showing of the collaborative project. With a grant from the American Council of Learned Societies (ACLS) we have been testing out this model with Women’s History classes, and a CD ROM Multimedia production class by creating a website and related interactive CD-ROM. The pilot project will continue through this fall. Based on our experience, we are refining the model for this kind of cross-disciplinary, collaborative, teaching/learning experiment. We are requesting support from FIPSE in order to apply, evaluate, refine, and extend this model project to a range of classes spanning several departments in our college. The model is designed to change the pedagogy of lecture-based classes from passive to active learning environments, to integrate computer-mediated communication technology (Internet, computer presentation stations) into classrooms, and to provide faculty with cost effective, on-the-job computer training. It will benefit our college, and in doing so, will serve as a model that can be easily adapted to other post-secondary institutions facing similar problems and looking for cost effective ways to transform teaching, integrate computer technology and retrain their faculty without taking them out of their classrooms. The use of computer-mediated communication to conduct a collaborative, team-based project using virtual teams as well as real teams will be new to the proposed project. The interdisciplinary nature of proposed collaboration between courses will help the college integrate the use of digital technologies and computers throughout the curriculum. Participation faculty will be learning digital skills as they teach, and will become increasingly familiar and able to integrate digital technology like the Internet, electronic presentations, and the incorporation of digital images into their regular teaching. Also, faculty with newly acquired digital skills 22

can advise their departments regarding digital technology use. The incorporation of computermediated communication tools into classes to provide support for project-based learning outside the few hours of face-to-face contact time as a flexible extension of class activities can be adapted to individual courses, as well as those that are paired up for interdisciplinary projects. The active, constructivist orientation of the model classes stressing teamwork, perspective-taking in terms of communicating the results of research assignments, and the creation of a knowledge structure we call the “knowledge map” (see Appendix A) benefits students. Opportunities for sustained perspective-taking that come when students must prepare their research for an audience, and annotate and hyperlink it, foster development of higher-order reasoning skills. Research and technology skills transfer to other subjects and classes that students take. When they “learn by doing” and “think about their thinking, ” students create cognitive webs or nets that promote a deep and full understanding of a subject. Furthermore, because the students will be creating an educational piece, they have a sense of pride and ownership about their abilities as a scholar and their mastery of the topic. Their range of technical skills will be linked to the production of meaningful learning in their classes. The Project Director will teach the technical classes involved in the model collaborations, and has the skills to help the students and participating faculty learn to work with digital technology. The classes that will participate in the project are taken from a range of disciplines, and represent the arts, communication, and general studies areas of the college. They are discussed in greater detail in the next section of the paper.

II. Proposed strategies for improving current practices This model provides the advantages of a relatively low-cost intervention, that will result in changes in faculty knowledge and teaching strategies. Most faculty are familiar with teambased projects and project-based learning, though in some larger classes they may not have felt this strategy was practical to implement. This model does not strain organizational structures of the college and relies on low-cost infrastructure improvements (computers on carts, projection devices, Internet “drops” where the wiring is already in place) in order to continue beyond the time of the funding provided by the grant. It looks to the use of computer-mediated communication tools like e-mail, listservs, and websites to provide students with opportunities to collaborate in a flexible time frame. Faculty need to be shown how to use computer-mediated instruction as a means of communication for students both in and outside of the classroom. The training builds on what they know from their personal use of computers for registration and email, and does not require leave time or special training off campus. The resources the students need in order to use computer-mediated communication are already in place at our institution and most colleges across the country—computer labs with Internet access. All our full-time faculty have computers in their offices with Internet access. The changes in pedagogy from passive to active learning incorporating a constructivist approach to subject matter, and the new skills faculty gain from their participation in the proposed grant are not dependent on continued funding from outside sources. We believe that the positive reactions of students to classes where knowledge creation not passive knowledge acquisition is the focus, will reinforce the resolve of faculty to continue to use this method beyond their participation in the grant. Once the faculty have used computer-mediated communication tools, they will not want to relinquish their use, and they need not, because of the ease of use and access to these tools. Furthermore, as this use of computers to communicate proliferates in the workplace, students and 23

teachers will view it as a natural part of the learning process, not an exotic add-on. Additionally faculty become knowledgeable advocates for what kinds of technology are appropriate for their classes and department and are less vulnerable to technological fads or bandwagons as they become active users of the technology. The model will be described in general in this section. Its implementation in specific courses and the differentiation of the model in those settings will be discussed in the next section. In general, “technology-rich” classes where students are learning web design or computer production skills must draw content from outside themselves. Unless the course objectives directly address instructional design, information engineering, storyboarding or other content creation methodologies, the use of content provided to the class from an external source creates a level playing field as far as evaluating student acquisition of technical skills without controlling for their ability to be content providers. In many work settings, technical producers must work with clients to produce the client’s vision, not their own, so this experience mirrors real-life experiences the student will encounter. In this model, the content is prepared by the students who researched it, but in a form where it is ready to be made interactive with the use of media files (images, sound, etc.) The key to this transformation is the exercise where students work in groups to produce “knowledge maps” which chart out hyperlinks and interactivity in their traditional research papers (see Appendix A.) The project director teaches the content class faculty to use the knowledge map to recast the research paper into this annotated, pre-electronic, linked guide for the content producers. Funding for the faculty who participate in this grant serves as an incentive to meet with the program director regularly, and to make the required changes to course syllabi to accommodate the collaborative projects. In some cases, the funding will allow a teacher to have release time to participate in this special project. In order to get ready for the semester, each teacher needs to meet 4 to 5 times with the project director, and to exchange syllabi and other course materials (e.g. the topics and order in which they will be covered, required and supplemental readings, and other resources.) Each syllabus must be altered somewhat to include the collaborative project and provide for meeting times, computer lab sessions, and discussion of nontraditional assignments. Depending on the faculty member’s computer skills, the project director may schedule training in using Internet or e-mail or other computer skills with the faculty before the semester begins. As the semester progresses, the faculty member attends the computer lab sessions with the students and gets the same training they do. The project director is available to help with any other technology concerns as the semester progresses. During the semester, the teachers and project director need to regularly by e-mail and telephone, and to make sure that any knowledge maps or other materials are being shared in a timely manner. The project director provides an introductory computer lab session for the students enrolled in the content course, to make sure they can do word processing, use the Internet, and have an e-mail account. A follow-up session with the T.A.s is provided for students who need a bit of extra help. A series of no-cost workshops offered at Columbia on computer use in different areas is open to students and faculty. If the content producer class needs to ask content-related questions, the content teacher attends their class to clarify matters. The content teacher visits and lectures in the technology class at least twice during the semester—at the beginning to give an overview of the content area, and near the end to troubleshoot and critique the works-in-progress. Works-in-progress will be evaluated by faculty teaching in relevant disciplines who are not currently part of the project, using a simple evaluation form 24

(see Appendix B.) This is will provide the students and teachers involved in the project with some constructive criticism, and also provide a guide for use of the project during the dissemination phase of the project. Each project can be rated in terms of how it fits into the discipline it covers, and its degree of technical completeness. When the projects (CD-ROMS or websites) are completed, the project director, the content teacher, and all the students present a “showing” of the work for the college community in order to publicize the project and to showcase their project. Each semester, the faculty of the two classes will summarize their experiences and submit conference proposals to talk about the project and to demonstrate the student projects to appropriate educational and professional organizations. Here is a description of the particular classes that will be involved in the pairing of classes during the term of the grant (see Table 1). The technology classes involved will include a section of Designing for the Web, Programming for Interactive Business & Training Uses (formerly Courseware & Learning.) or other similar classes. In each of these classes the emphasis is on aspects of technical production of digital media, so the use of external content providers can be accommodated (and in some instances was built-into the course.) The “content-rich” classes will include: Women’s History I and II., each of which has been a part of a previous, similar grant from the American Council of Learned Societies (ACLS) for collaborative interdisciplinary work. These classes are the pilot version for the FIPSE grant. The objectives for the Women’s History class were altered in several ways to enable the collaboration. These alterations did not change the way the topics of Women’s History were taught substantively, but altered some classroom activities and assignments. For example, the class was shifted from a temporal view of the subject where some students did little writing until the 6th or 7th week of class, to a theme-based approach which meant students produced content on a more uniform basis over the semester. This was better for the tech classes and meant that the classes could meet to test and review the collaboration more frequently. The Women’s History students initially wrote two papers. In order to produce content for the tech classes at a more uniform rate, the requirement was changed to 4 shorter papers, spread out more equally over the semester. While this change was beneficial to the writing of the students, the history students as content providers did not get enough opportunity to review the interactive interpretation of their work by the tech class with only four sessions of meeting together. In the first iteration of this experiment, the tech students were simply “showing” their work to the content students. We redesigned the class so that the content provider students reworked the material themselves so they would experience the perspective-taking involved rendering a work for an audience by creating a “knowledge map” of their topic (see Appendix A). Additionally, we designed ways for the student writers to have the opportunity to try out the pieces, and suggest revisions to the students who were authoring the interactive pieces on an ongoing basis through the use of Internet and e-mail. With the proposed FIPSE grant, classes involved in the model collaboration project will include: Women’s History I and II, Nutrition (Science & Math Department), History of American Sign Language (Interpreter Training), Latin American Literature, Art & Music (Humanities) and Fashion (Fashion Management.) In the Nutrition class, the teacher wants to use certain professionally produced CD ROMs in her lectures. The project director can facilitate this. The collaboration project will focus on nutrition knowledge and present “public service” type web pages about good nutrition for the college community. The Interpreter Training Program (one of the only such programs in Illinois) will have its introductory survey course, “History of American Sign Lan25

guage (ASL)” participate. Students do a research paper in this class, and will adapt their research for an audience of “laymen” to help educate the public about ASL and issues of importance to this field. Segments of ASL can be animated as part of this effort. The Latin American Literature, Music, and Art class currently does one traditional research assignment and one group project, emphasizing performance. The class objectives already include items about teamwork and preparing content for an audience. The projects will be enhanced by having students consider how to show highlights of their projects in a context that will explain them in the interactive digital collaboration piece. Our college has a strong concentration in Fashion and an additional course that will be involved in this grant is a fashion course. Students in this class will transform their drawings and ideas about the future of fashion into knowledge maps which will be produced collaboratively by the tech classes. The team project aspect can include a virtual fashion show which would be interactive so that viewers look at patterns, drawing, and information about fabrics in the collaborative multimedia piece. The area of fashion and rendering fashions for “virtual humans” is of growing interest in several professions. For the classes involved in this series of collaborative efforts, there are questions of balance—between the traditional course content and the changes made to allow for the collaborative project and between the importance “ process” and “product.” The technical classes have a press for completion of a polished product, but there is tension in working with student content providers. The quality of the pieces will vary with the student producers, however, issues of visual literacy, production techniques, and aesthetics are addressed and students learn from near misses as well as from their successes. However, in all cases, the products (CDs or websites) can become resources for college. Thus, the CD ROM or website product of the Women’s History/CD ROM Multimedia Production class is a resource for the next CD ROM and Women’s History classes and for other production classes, as well as for American History which includes some readings and study of the role of women in American History. Similarly, the Sign Language website can become a resource for the interpreter training class, or for any class which might be considering questions regarding equity and disabilities. The Nutrition website can be a resource for student groups, and other science and health classes. It is important to remember all the projects are created by undergraduates in a single semester. They may be revised, corrected, and updated regularly by classes in subsequent semesters. Much more with than commercially made and produced software and classroom resources, the hallmark of these products is that students are knowledge producers, not simply knowledge consumers. There is always the incentive to improve the work, the challenge to take a shot at it yourself, that is missing when students use commercially prepared materials which are professionally edited, finished, and complete. The evaluation of the class products by faculty outside of the project will provide information for users within and outside our institution about the nature and completeness of the CD ROM or website resources produced by for this grant. This aspect of learning, wherein students are empowered to learn because they want to communicate their ideas is powerful and stands in a central role in the transformation of the classroom from passive to active, with a focus on constructivism. While the proposed interventions are not overly complex, nor costly, they are revolutionary, in that the teachers involved become active lobbyists for the kinds of technology which will make their classrooms more effective learning environments, and students themselves are the active authors of curricular reform and of classroom materials. The essence of digital information is that it is ‘assembled’ rather than ‘looked 26

up’. Being able to put together relevant information, make sense of it, and to communicate about it is a central task for the worker, thinker, and citizen of the next century. Students who participate in the proposed model classes practice this, and educational research shows we learn what we practice. The payoffs of the transformed classrooms need to be documented. The details of this are laid out in the evaluation section of this paper, however, it should be noted that dimensions of cooperative learning environments which will result from this model can be measured. Changes in perceived levels of cooperation can be correlated with achievement gains. This is exciting, because without such an empirically validated measure of effectiveness, it would be difficult to compare outcomes across such a range of classes with such diverse learning outcomes. In most cases, there will be comparable classes to use as comparison groups, so that changes in the target classes can be more accurately attributed to aspects of the proposed strategies to improve postsecondary instruction.

III. Institutional commitment--contribution of the College to the project Columbia College has much of the equipment needed to begin, carry out, and complete these kind of productions, including computer labs, a CD ROM press, and a capable faculty who are already teaching such classes as Elements of Multimedia, Multimedia Production and CD ROM Production. There is growing interest among faculty across departments in learning to use and integrate of digital technologies into the fabric of the classroom. This project will provide the needed structure within which the faculty collaboration can take place and resources can be used effectively. The series of projects produced in the classes—the CD ROMs and websites themselves—are an additional educational resource and will benefit students and faculty by emphasizing teamwork and project-based learning and providing them with cutting edge digital technology skills. We will be working to have the college create “smart” classrooms for participating subjectmatter classes where instructors will have access to an Internet connection, a computer and a presentation device. Currently these classrooms have no computer, projection device, nor Internet drop and this means the logistics of demonstrating software or showing the work-in-progress projects are difficult to manage. We hope to have at least one “smart” classroom by Fall 1998.

IV. Evaluation and Dissemination This section describes the evaluation and dissemination plan for Integrating Electronic Technology in Liberal and Professional Education Classrooms. This project has three main objectives: to promote active student learning, to integrate computers into the classroom, and to improve faculty computer and digital skills. The purpose of this evaluation is to assess whether project goals are met in a cost effective manner. The use of controlled comparisons will help reveal the impact of the project, disseminate findings, and expand the project to other areas. This evaluation has two main parts. The formative evaluation is designed to improve implementation and development and reform project activities. The summative evaluation assesses cost effectiveness, the fulfillment of project objectives, and project efficacy. Both the formative and summative evaluations may address administrative planning and policy making, reveal unintended consequences, inform the allocation of resources, and identify problems and/or costs. 27

The formative evaluation is designed to assess quickly and accurately the strengths and weaknesses of the project and provide suggestions for improvement. It is important to capitalize on the strengths, or address weaknesses, during the early stages of the project to improve development and implementation. Furthermore, the results of the formative evaluation should be disseminated in a timely manner so appropriate adjustments can be made to improve the project by the Project Director and participating faculty members. This plan facilitates a timely dissemination of formative evaluation information. The use of electronic technology will be employed for fast and accurate results. The evaluator will be provided with the e-mail addresses for the teachers and students after the first week of classes. During the third week of classes the evaluator will e-mail the Evaluation Questionnaire (see Appendix C) to all participating teachers and students. They will have one week to e-mail their responses back to the evaluator. The evaluator will write a clear and concise formative evaluation based on the input of the teachers and students. The appendices for these reports will include complete transcripts of the responses for further review. The reports will be sent to the Project Director for facilitating changes. The Evaluation Questionnaire is simple and effective. The items are open-ended to allow for freedom of response. The respondents are asked to state the goals of the project. Participant knowledge about the project objectives is important to promote learning and understanding. The participants are also asked to cite the best and worst things about the project. Obviously, the Project Director could try to enhance or continue the best things while reducing or eliminating the worst. Finally, the respondents are invited to suggest changes. This is an ideal way to identify ways to improve the project for its consumers. The summative evaluation, the second part of this plan, has three parts: student questionnaires, teacher interviews, and project product evaluations. Initially, the Student Evaluation Form (Appendix D) will be administered at during the last week of classes to the students participating in the project and, where possible, to students in comparable classes outside the project. This controlled comparison will identify the strengths and weaknesses of the project. The questionnaire is composed of six scales from the Learning Environment Inventory (LEI). The LEI measures classroom social climate which research has shown to be one of the key psychological factors in academic learning (Walberg 1994). If cooperative learning can be enhanced, for example, achievement gains are likely. The LEI consists of Likert scale items with the following responses: Strongly Disagree, Disagree, Agree, and Strongly Agree. The scales have been empirically proven to be both valid and reliable (Fraser, Anderson, Walberg, 1991). Six scales were selected to measure the attainment of the project objectives. Keeping the individual scales intact preserves their reliability. The scales include: cohesiveness, diversity, material environment, goal direction, democracy, and satisfaction. Cohesiveness refers to how favorably students see themselves and their work as a group. Diversity reflects whether students with different abilities and interests are encouraged. The material environment refers to availability appropriate of books, equipment, and space. Goal direction assesses whether the students know what they are doing. Democracy measures the perceived student input in classroom decisionmaking. Finally, satisfaction gauges whether the students enjoy the class. These scales were selected to assess the fulfillment of the project objectives to improve cooperative and active learning. The LEI will be administered to the students electronically. The evaluator will e-mail the Student Evaluation Form to the students during the last week of class. The students will have one 28

week to complete the questionnaire and reply with their answers. The evaluator will write a report based on the results of the LEI questionnaires. The evaluator will also conduct interviews with project teachers. The teachers home and work telephone numbers will be provided to the evaluator during the first week of classes. (An emailed version of the same interview schedule may be substituted depending on time constraints.) The teachers will be asked to respond to the questions on the Teacher Evaluation Form (Appendix E). This form is designed to assess the summative evaluation issues mentioned earlier. Specifically, the Teacher Evaluation Form will probe key areas of the project. Using the form, teachers will be asked what the goals were and whether they were met. Respondents will be asked to cite the best and worst things about the project and to make suggestions. They will be asked if the project is cost effective and whether it successfully integrated computers in the classroom. The teachers will be asked whether their computer and digital skills improved and whether the project changed their teaching. Lastly, the teachers will be asked if they shared or planned to share project ideas with others outside the project and whether the project would benefit other campuses. Finally, project products will be evaluated. Project CD ROMS and websites will be evaluated by teachers outside the project. These teachers will be within the same discipline of the product that is being evaluated. A non-project history teacher, for example, will evaluate a CD ROM on women’s history that is produced through the project. The teachers will be asked to complete the Product Evaluation Form (See Appendix B). They will assess the quality and content of project materials and provide suggestions for improvement. The Project Director will be responsible for distributing the evaluation forms and relevant project materials (e.g. website address or CD ROM). The teachers will complete the form and return it to the Project Director who will forward it to the evaluator. The evaluator will review all the Product Evaluation Forms and write a summary that will be included in the final report. The evaluator will write a final summative report for each year of the project. These reports will be due August 30 of each year and will be based on the findings of the student questionnaire results, teacher interviews, and project product evaluations. In addition, copies of the formative reports for that year will also be included. The final reports will, among other things, address whether the project goals were fulfilled, give suggestions for improvement, and provide an overall assessment of the project. In order to disseminate information about the project, faculty will attend conferences and submit papers about their work to professional and educational organizations. Currently as part of a similar project funded by the American Council of Learned Societies, faculty have presented at Purdue (on Women’s History), and will present at SIGGRAPH/SIG ED in Orlando in July. With FIPSE funding similar efforts to disseminate information about the model and its benefits will continue. Once the project is underway, and includes the evaluation of the class projects by faculty outside the project, the projects can be rated and information about them disseminated via the Internet by registering the project websites with Internet search engines, and by contacting websites which are educational resources and asking them to include links to selected sites. Explanatory material about the student-generated nature of the projects will be included. Thus, these sites could be used as research resources for other classes, as models for teachers wanting to conduct 29

similar knowledge construction processes. For other post-secondary educational institutions, the emphasis in this project on interventions which do not require huge capital outlays, nor profound organizational upheaval, but rely on cost-effective retraining of faculty , the adoption of simple but effective teaching strategies (team-based project assignments) and on the use of resources which are widely available in postsecondary institutions will be of interest, and through participation in FIPSE sponsored conferences, regional and other national conferences, and in publications in academic ( Journal of Educational Research. Educational Technology, etc.) and trade-type journals (SYLLABUS, THE Journal, etc.) the project and its findings will be disseminated. The detailed and systematic nature of the both the formative and summative evaluation of this project will present an important resource in and of itself. Research on the learning impact of interactive multimedia materials is sparse. This study, through its use of the LEI which is an established tool in educational research and helps provide a way to link changes in student attitude and perception of classroom climate to expected changes in intellectual and academic indicators, will provide a way of quantifying the effects of collaborative, interdisciplinary digital projects on measures of learning outcomes. The findings of this longitudinal study which will incorporate classes from a wide range of disciplines, and the provide a first attempt at the evaluation of the effect of computer-mediated communication tools on classroom climate, and thus student learning outcomes, will be of interest to post-secondary educators in a wide variety of institutions and settings.

30