Evaluation Strategies
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Evaluation Strategies as PDF for free.
More details
- Words: 3,783
- Pages: 9
Evaluation strategies
This study is about :An Evaluation of Two Courses for Graduate-Level Professors Designed to Improve Academic Internet Use in Research and Teaching
Study by: Anise A. G. D'O. FERREIRA Catholic University of São Paulo Brazil
Abstract This paper discusses insights gained from the author's experience in planning and teaching two courses aimed at improving academic Internet use in research and teaching by college teachers, professionals, and master's degree candidates. The courses involved different educational strategies. The first one had prescriptive goals and tasks; the second had a content-based, goaloriented activity, in which certain abilities, subgoals, and knowledge were implicitly required. The evaluation of both courses was based on students' effective engagement in course activities, measured by class attendance, dropout rate, students' reports, interaction, and performance observations in both online and in-class activities. The results highlighted three important influences on learning: institutional infrastructure and policies for faculty and graduate student improvement, students' previous experience (e.g., proficiency in English reading or computer use), and motivational engagement, which can be ascribed to a participatory and constructive learning environment at the cost of more time-consuming interaction among participants and with the instructor. This paper raises questions about institutional policies for faculty training, theoretical and methodological issues in motivational aspects of the learning environment, implications for future online courses, and their evaluation.
First course The first course had strategies that were based upon behavioral goals and tasks related to projected learner needs. The strategies had activities organized in units of knowledge. These units involved lectures and practice exercises. There was some flexibility in the practice sessions, since the content target of tasks had to be related to the participants' main area of interest and the pace of performance had to be based on their competence in computer use. The instructor was available to give in-class guidance and feedback during practice sessions. Pedagogically, this teaching plan reflects, in part, some of the behaviorist principles of educational methodology, such as previous specification of goals and tasks in sequential steps, and the belief that the educator can control and predict required learning At the end of the course, the participants would be expected to: • •
know the types of Internet connection and access; use Internet communication and information service clients;
• • • • • •
access a remote computer using telnet, file transfer protocol (FTP), gopher, and World Wide Web (WWW); communicate using e-mail; subscribe to and participate in academic e-conferences; send and receive files through e-mail and ftp; search and analyze relevant academic material in databases and available information services on the Internet; and draft an educational or research project using the Internet as a tool.
In order to attain the objectives, the participants were expected to: • •
• •
attend lectures and demonstrations; accomplish assignments: four blocks of exercises, working in pairs in the laboratory, which had to be turned in by e-mail or in printed form, directly to the instructor, every other week; answer a questionnaire; and turn in a final individual paper related to the last block of exercises.
Participants
Enrolled in the course were 30 male and female adults, consisting of faculty members (part-time college professors), invited coordinators, invited librarians, and administrative employees. This group was heterogeneous in academic level, English language proficiency, professional interests, and familiarity with both microcomputers and the Internet. The department of technological development wanted this opportunity to reach the greatest possible number of people. Internet access and the available infrastructure
The lectures were offered in a classroom in which a microcomputer (personal computer; PC) directly accessed the Internet through a 64 Kb line connecting to the Internet service provider (ISP).This microcomputer was a Pentium PC (133 MHz, 32 MB RAM, 1 MB Video RAM) equipped with CD-ROM drive and multimedia resources, running Win95 and the necessary Internet clients. A datashow was used to project the Microsoft PowerPoint slides and Internet applications during explanations of their features and academic usage. Since the connection was neither fast nor stable, the instructor sometimes had to present the material off-line. The institution provided Internet access on campus via direct connection, since it was not able to provide an off campus dial-up service. An Internet laboratory, with 40 identically configured PCs (without multimedia resources), was available for students who wanted to practice in the afternoons. However, they preferred to use the equipment available in the teachers' room, which was equally slow and nonstable but offered more privacy. The invited coordinators, librarians, and administrative employees had microcomputers in their offices with direct or dial-up connections to the Internet. Since those connections were not stable, and some machines had maintenance problems, some preferred to use their personal equipment and connections at home.
Procedure
The participants were given printed material that contained: • • •
course objectives; activity descriptions and accomplishment expectations; and copies of two Web pages (printed with URL [uniform resource locator] addresses) built specially to provide initial references for navigation in WWW, tutorial addresses, and software.
The course was divided into sessions and assignments: •
•
Eight weekly, one-and-a-half hour lecture classes conducted by the instructor for the entire group. o The content of the lecture sessions, based on introductory material (PowerPoint slides) freely distributed by The included: Networking, the Internet, and the Internet in Brazil Types of Internet access and accounts; software and hardware required Internet clients: WWW, e-mail, ftp, telnet, chat Information and communication tools and services Characteristics of Internet addresses Client server model Strategies of using information and communication services o Demonstrations, tips, and explanations were provided during the lecture classes, through online access whenever possible. Weekly two-hour (minimum) practice sessions for individual or pair work, tutored by the instructor o The practice sessions were guided by assignments, four blocks of exercises with answer forms. Here are some examples of exercise items: week 1 exercise: "Visit the information services listed in the Web page "x" (the one printed) and select an item of information among those which most interested you. Justify." week 3 exercise: "Select two academic lists and subscribe to them; then, write down a) the address of the respective list servers and the list addresses; b) their institutions which maintain these lists; c) the weekly volume of information; and d) some characteristics of the communication in these lists." o Also, the participants had to answer two questionnaires: one after the second week, and the other after the sixth week. Here are examples of questions: from the first questionnaire: "What is the configuration of the computer you are using in terms of hardware and software; are there communication software and hardware?" from the second questionnaire: "What do you do to find an email address?" "What do you do to subscribe to a list and to participate in a discussion group?"
o
The individual assignment was related to the fourth block of exercises which focused on the analysis of academic information sites. The participants had to evaluate those sites that they liked most and propose a draft of the structure and/or organization of their ideal information server in terms of their own academic interest.
Results and discussion
By the third week, those who had no familiarity whatsoever with computers began dropping out. They said they had no available time to practice. The dropout rate reached 66.6 percent. Only 10 (six graduate students, two coordinators, one administrative employee, and a librarian), from the initial 30, attended the lectures consistently. During the eight weeks, the presence of the instructor in practice sessions was requested three times by each pair of the more assiduous students. E-mail was also used to communicate with those participants. Nobody turned in all the assignments. All of them complained about lack of time and blamed their unproductive days on the nonstable infrastructure conditions. Moreover, some reported that they tried at least to attend the lecture sessions because they felt obligated by their superiors. On the other hand, there was some positive feedback. One of the coordinators reported having been very satisfied with being able to find documents he needed. The librarian and administrative employee started an informal interaction with instructor in order to solve specific problems with their professional activities on the Internet. While it is possible to imagine many reasons why the participants didn't satisfactorily complete their assignments or finish the course, the obvious reasons were institutional infrastructure problems and participants' time constraints, since some were part-time professors and had their own businesses. Of course, one could say that a better infrastructure and paid release time would help. In fact, two hours a week dedicated to practice falls short of learning needs. At least 10 hours a week would be necessary . The instructor noted that those who persisted in continuing the course, despite the difficulties, exhibited common characteristics: a) engagement in activities in which the Internet fulfilled immediate needs and b) English reading proficiency. Among those who continued were the coordinators, senior researchers, and professionals, who experienced immediate benefits. For the librarian and employee, the course provided insights into the use of Internet tools. On closer inspection, it turns out that the course may have started from the false premise that participants were equally as motivated as this latter group. Pedagogically, one can say that the teaching strategies, based on prescribed and structured exercises, were not adequate for all the learners. Even though the exercises were related to the solution of a genuine problem, they were not embedded in a meaningful or relevant project for the majority of the participants. The importance of English reading proficiency makes sense, since most sources of relevant information at that time were in English. Many still are, such as indexes, abstracts, publishers, electronic lists, newsletters, and general information websites. These results demonstrated that infrastructure and time can be powerful factors, but motivational aspects embedded in strategies may also be powerful.
2. Second course The second course had practically the same general objectives as the first one, focusing more on a critical analysis of academic performance through the Internet. However, task goals and specific contents were not previously established by the professor as in the first course. The design took into account the remaining unsolved problems of infrastructure and students' lack of time. In order to minimize these effects, course duration was increased, criteria for selection of participants were established, and course content and strategies were modified; the design became more flexible, according to students' difficulties and motivation. This course proposed content-based, goal-oriented activity, in which many abilities and subgoals and much knowledge were implicitly required. The final objective was to build an academic information resource website (individually and in group) in which specific contents were negotiated with the instructor and among the students, based on their real work and research issues. The students, along with the instructor, evaluated and selected a free service for Web page storage, such as Geocities, Xoom, and Starmedia, where they stored their assignments and structured their website. In summary, the position assumed by the instructor here gave importance to creating opportunities for the students to perform and accomplish tasks in a genuine situation, to perform collaborative tasks, and to reach goals according to group and individual needs and desires. Participants
Nine graduate students, one journalist, one business administrator, one social communication college teacher, one marketing college teacher, two teachers of English as foreign language, and three pedagogical advisors applied for this course. The enrollment requirements were a) to have their own Internet account and connection at home, b) to be a proficient English language reader, c) to have basic computer skills in the operating system, word processors, browser, and e-mail clients, and d) to have 10 extra hours a week available to practice. The students who judged that they could not fulfill all the requirements (either because their level of English reading was poor, or because their computer skills were inadequate) were accepted upon agreeing to work additional hours. A student who had more experience with Internet than his classmates did offered to help the others with technical problems. Although this student was experienced in the use of Internet tools, he was not confident using them as academic tools. Internet access and the available infrastructure
Aside from their own Internet access and accounts, students could use the undergraduate laboratory in the afternoons. The undergraduate campus was situated slightly less than one kilometer from the graduate department. The instructor could use a shared dial-up access available in one room in the graduate department, or use her own Internet dial-up access and account at home. The undergraduate laboratory was reserved for classes one morning every other week. It had 40 PCs (133 MHz, 32 MB RAM, HD 2 GB, 1 MB Video RAM) connected in a Novell local network with direct access to the Internet through a 64 MB line to the ISP. The other
sessions took place in a regular classroom on the graduate campus, with desks and a white board, where lectures and discussions were carried on. Procedure
The course was divided in classroom and remote activities. Sixty accredited hours were given: 15 weekly four-hour sessions with students and the instructor in a regular classroom (desks and white board) or with students and instructors at the laboratory every other week from the sixth session on, where some online activities took place in pairs or groups. The remote activity -- that is, homework and online work -- was not computed as instruction hours for administrative purposes. The general content involved the following topics in theoretical discussions and practical activities: • • • • • • • • • • • • • •
the infrastructure and general purpose of an academic network client server model, Internet software and hardware, protocols communication and information Internet services and addresses search tools and engines in the WWW publishing tools for the WWW types of academic information distributed by different Internet services and tools search of bibliographical data and specific academic services based on the WWW the Internet as a tool and object of study user-interface studies and research evaluation of information quality on the Internet cyberculture and virtual communities studies teaching and learning studies in the Internet context introduction to the planning of Web-based courses pedagogical issues regarding Web-based classes
Classroom and laboratory activities involved: • • • • • • • •
short instructor lectures student presentations discussion of themes and questions related to academic readings proposed by the instructor and students discussion of issues raised from online activities planning and organization of the Web project content planning of instructional activities and feedback mechanisms writing short papers about classroom discussions and questions practicing and discussing the online use of Internet clients, tools, and services.
Remote activities involved:
• • • • •
asynchronous communication through e-mail and list by the group and instructor gathering of online material reading and analysis of the material gathered material preparation for Web storage purposes Web storage and organization of online information
The first two sessions were dedicated to planning and organizing the work routine, based on discussions of students' research and work themes. The themes for readings were derived from instructor-student dialogue. Readings and discussions, alternating with short lectures, involved the selected topics mentioned above. The complexity of practice activities was negotiated with the students during the course. In order to help initiate practice, two Web pages were provided -one containing academic resource information addresses, the other containing guidelines for bibliography searches. In order to become more technically familiar with the tools, each student agreed to store homework in individual Web pages. Then, they had to edit all the individual material (handwritten or in HTML [hypertext markup language] format) and organize it in a collective website according to data and criteria they had read and discussed. Student evaluation was based on in-class and online participation and production, in both individual and collective publication of assignments negotiated with the instructor (some in HTML, others distributed through e-mail). Results and discussion Face-to-face attendance and dropout rate
There were no dropouts, although one of the students missed classes, was always 30 minutes late, and couldn't accomplish any of the work established by the instructor and group. He complained about his defective microcomputer and lack of time either to have it fixed or to work on campus. The average rate of attendance was 91 percent. Online interaction
By the fifth session, the site for the Web page storage was chosen. Since the site had a forum service, some of the messages exchanged by the group remained at the site and could be revisited anytime. However, not all the messages exchanged were stored at the site. Two hundred and forty messages were sent to the group list. Thirty-one percent were sent by the instructor, and 29.5 percent were sent by the student who was more experienced in the Internet. He also forwarded some peer messages to the list. Three percent came from another graduate professor who collaborated in finding course material once in a while. Thirty-nine percent were sent by the other students, who could be divided into two groups of three: a) average-level interaction, ranging from 0.7 to 14 percent, and b) minimum interaction with less than 0.7 percent. It is important to notice that some messages were not sent directly to the list by the students, but were sent to classmates. As the course progressed, online interaction decreased: 41
percent of the total messages sent during the course were exchanged in the first month, 28 percent in the second, 10 percent in the third, and 5 percent in the last. The content of these communications was mostly organizational (instructor's feedback, comments and suggestions to all students, and students' messages to all proposing and assuming tasks) and technical (students asking for help, instructor and students giving help). There was student-student communication, too. At the beginning of the course, the numbers showed an active list. As soon as the course evolved, the list activity decreased. This has a simple explanation. The individual tasks decreased and collective work was required in order to accomplish the collective site; the greater part of these messages were sent directly to the classmates with whom the student was sharing the activity. The level of remote, online interaction was less than that expected by the instructor, since the group had agreed that part of the theoretical discussions and work would be distributed to all in the class list. The students took more time to solve technical difficulties, and they didn't have time to deal electronically with the material gathered in the interval between classes. So, they printed the material, read it at home, and discussed it in the classroom. Performance observations and reports
The assignments exhibited in class and in the website formed the students' portfolio for evaluation. These materials showed that practically all of the students (89%) were able to: • • • •
look for Web page services and evaluate them in order to select useful academic information, look for academic material related to their area of interest, communicate and share interests with other researchers or students through e-mail and participation in discussion lists, and edit HTML and publish their reviews and other academic findings on an individual site using basic graphic resources practiced during the course and organize them in one collective site, according to certain concepts they read and discussed in class.
Priority was given to content organization and quality, instead of graphic embellishment. The collective site was remarkably superior to the individual pages both in quantity and quality (URL: http://members.xoom.com/tgcarmona/default.html). In terms of specific skills for using Internet software, tools, and services, they were able to get an Internet account, install communication and Internet software, communicate more frequently through e-mail, navigate the WWW, use search engines, download and upload material through an ftp client, and edit HTML. Of course, there were different levels of performance and achievements. Although most had already used e-mail and the WWW before the course, some took much more time to achieve results than others and still had to follow personal written notes and instructions.
Performance expectations
Judging by the commitment to course requirements, the instructor expected a higher level of accomplishment. However, the students emphasized that their ability to gather relevant academic material through Internet information and communication services increased (in some cases, from zero). According to discussions, assignments, and reports collected during sessions, all the students, except one, had insufficient previous experience with Internet clients or other software, and six were slow English readers. All students needed more time to complete activities, some as much as 15 hours. One group misjudged the time they had available for the courses, and in some cases, had as few as six hours a week to work. The most experienced student said he needed seven hours a week to complete his work. Even those who had more experience with word processors or graphic editors demonstrated difficulty in transferring these abilities to other tools as Internet clients. For example, the student who was a proficient English reader and had very good computer skills with text and graphic editors said that six to eight hours a week were necessary in order to achieve her goals. Although the students also complained about the slow Internet access they had in the laboratory sessions, having some classes in the Internet laboratory proved to be essential for those who had poorer computer skills. The quality and quantity of discussions of course content were negatively affected by the effort spent resolving technical problems. Some students reported that they wanted to explore and discuss subjects of interest in greater depth and quantity. Technical difficulties forced the instructor to be less demanding concerning the focus of the course. Changes were made to accommodate the majority of student needs. However, the students became aware of and put into practice important concepts of academic Internet use, such as criteria for establishing quality of information, usability, and peculiarities of the media in communication and education. At the end of the course, they reported that their technical level as well their comprehension of Internet technology use improved (some had serious prejudices about using the Internet in academic work). They also reported that the benefits they got from the Internet for their research and educational processes increased substantially during the course and that they were satisfied with their progress. Some reported that the Internet helped them to find theoretical and methodological trends in their research areas. These better results in the second course can be ascribed to modifications in course methodology motivated by the necessity of managing student lack of time, access difficulties on campus, and maintenance of student interest in Internet academic usage. Refrence: http://www.isoc.org/inet99/proceedings/2f/2f_4.htm
This study is about :An Evaluation of Two Courses for Graduate-Level Professors Designed to Improve Academic Internet Use in Research and Teaching
Study by: Anise A. G. D'O. FERREIRA
Abstract This paper discusses insights gained from the author's experience in planning and teaching two courses aimed at improving academic Internet use in research and teaching by college teachers, professionals, and master's degree candidates. The courses involved different educational strategies. The first one had prescriptive goals and tasks; the second had a content-based, goaloriented activity, in which certain abilities, subgoals, and knowledge were implicitly required. The evaluation of both courses was based on students' effective engagement in course activities, measured by class attendance, dropout rate, students' reports, interaction, and performance observations in both online and in-class activities. The results highlighted three important influences on learning: institutional infrastructure and policies for faculty and graduate student improvement, students' previous experience (e.g., proficiency in English reading or computer use), and motivational engagement, which can be ascribed to a participatory and constructive learning environment at the cost of more time-consuming interaction among participants and with the instructor. This paper raises questions about institutional policies for faculty training, theoretical and methodological issues in motivational aspects of the learning environment, implications for future online courses, and their evaluation.
First course The first course had strategies that were based upon behavioral goals and tasks related to projected learner needs. The strategies had activities organized in units of knowledge. These units involved lectures and practice exercises. There was some flexibility in the practice sessions, since the content target of tasks had to be related to the participants' main area of interest and the pace of performance had to be based on their competence in computer use. The instructor was available to give in-class guidance and feedback during practice sessions. Pedagogically, this teaching plan reflects, in part, some of the behaviorist principles of educational methodology, such as previous specification of goals and tasks in sequential steps, and the belief that the educator can control and predict required learning At the end of the course, the participants would be expected to: • •
know the types of Internet connection and access; use Internet communication and information service clients;
• • • • • •
access a remote computer using telnet, file transfer protocol (FTP), gopher, and World Wide Web (WWW); communicate using e-mail; subscribe to and participate in academic e-conferences; send and receive files through e-mail and ftp; search and analyze relevant academic material in databases and available information services on the Internet; and draft an educational or research project using the Internet as a tool.
In order to attain the objectives, the participants were expected to: • •
• •
attend lectures and demonstrations; accomplish assignments: four blocks of exercises, working in pairs in the laboratory, which had to be turned in by e-mail or in printed form, directly to the instructor, every other week; answer a questionnaire; and turn in a final individual paper related to the last block of exercises.
Participants
Enrolled in the course were 30 male and female adults, consisting of faculty members (part-time college professors), invited coordinators, invited librarians, and administrative employees. This group was heterogeneous in academic level, English language proficiency, professional interests, and familiarity with both microcomputers and the Internet. The department of technological development wanted this opportunity to reach the greatest possible number of people. Internet access and the available infrastructure
The lectures were offered in a classroom in which a microcomputer (personal computer; PC) directly accessed the Internet through a 64 Kb line connecting to the Internet service provider (ISP).This microcomputer was a Pentium PC (133 MHz, 32 MB RAM, 1 MB Video RAM) equipped with CD-ROM drive and multimedia resources, running Win95 and the necessary Internet clients. A datashow was used to project the Microsoft PowerPoint slides and Internet applications during explanations of their features and academic usage. Since the connection was neither fast nor stable, the instructor sometimes had to present the material off-line. The institution provided Internet access on campus via direct connection, since it was not able to provide an off campus dial-up service. An Internet laboratory, with 40 identically configured PCs (without multimedia resources), was available for students who wanted to practice in the afternoons. However, they preferred to use the equipment available in the teachers' room, which was equally slow and nonstable but offered more privacy. The invited coordinators, librarians, and administrative employees had microcomputers in their offices with direct or dial-up connections to the Internet. Since those connections were not stable, and some machines had maintenance problems, some preferred to use their personal equipment and connections at home.
Procedure
The participants were given printed material that contained: • • •
course objectives; activity descriptions and accomplishment expectations; and copies of two Web pages (printed with URL [uniform resource locator] addresses) built specially to provide initial references for navigation in WWW, tutorial addresses, and software.
The course was divided into sessions and assignments: •
•
Eight weekly, one-and-a-half hour lecture classes conducted by the instructor for the entire group. o The content of the lecture sessions, based on introductory material (PowerPoint slides) freely distributed by The included: Networking, the Internet, and the Internet in Brazil Types of Internet access and accounts; software and hardware required Internet clients: WWW, e-mail, ftp, telnet, chat Information and communication tools and services Characteristics of Internet addresses Client server model Strategies of using information and communication services o Demonstrations, tips, and explanations were provided during the lecture classes, through online access whenever possible. Weekly two-hour (minimum) practice sessions for individual or pair work, tutored by the instructor o The practice sessions were guided by assignments, four blocks of exercises with answer forms. Here are some examples of exercise items: week 1 exercise: "Visit the information services listed in the Web page "x" (the one printed) and select an item of information among those which most interested you. Justify." week 3 exercise: "Select two academic lists and subscribe to them; then, write down a) the address of the respective list servers and the list addresses; b) their institutions which maintain these lists; c) the weekly volume of information; and d) some characteristics of the communication in these lists." o Also, the participants had to answer two questionnaires: one after the second week, and the other after the sixth week. Here are examples of questions: from the first questionnaire: "What is the configuration of the computer you are using in terms of hardware and software; are there communication software and hardware?" from the second questionnaire: "What do you do to find an email address?" "What do you do to subscribe to a list and to participate in a discussion group?"
o
The individual assignment was related to the fourth block of exercises which focused on the analysis of academic information sites. The participants had to evaluate those sites that they liked most and propose a draft of the structure and/or organization of their ideal information server in terms of their own academic interest.
Results and discussion
By the third week, those who had no familiarity whatsoever with computers began dropping out. They said they had no available time to practice. The dropout rate reached 66.6 percent. Only 10 (six graduate students, two coordinators, one administrative employee, and a librarian), from the initial 30, attended the lectures consistently. During the eight weeks, the presence of the instructor in practice sessions was requested three times by each pair of the more assiduous students. E-mail was also used to communicate with those participants. Nobody turned in all the assignments. All of them complained about lack of time and blamed their unproductive days on the nonstable infrastructure conditions. Moreover, some reported that they tried at least to attend the lecture sessions because they felt obligated by their superiors. On the other hand, there was some positive feedback. One of the coordinators reported having been very satisfied with being able to find documents he needed. The librarian and administrative employee started an informal interaction with instructor in order to solve specific problems with their professional activities on the Internet. While it is possible to imagine many reasons why the participants didn't satisfactorily complete their assignments or finish the course, the obvious reasons were institutional infrastructure problems and participants' time constraints, since some were part-time professors and had their own businesses. Of course, one could say that a better infrastructure and paid release time would help. In fact, two hours a week dedicated to practice falls short of learning needs. At least 10 hours a week would be necessary . The instructor noted that those who persisted in continuing the course, despite the difficulties, exhibited common characteristics: a) engagement in activities in which the Internet fulfilled immediate needs and b) English reading proficiency. Among those who continued were the coordinators, senior researchers, and professionals, who experienced immediate benefits. For the librarian and employee, the course provided insights into the use of Internet tools. On closer inspection, it turns out that the course may have started from the false premise that participants were equally as motivated as this latter group. Pedagogically, one can say that the teaching strategies, based on prescribed and structured exercises, were not adequate for all the learners. Even though the exercises were related to the solution of a genuine problem, they were not embedded in a meaningful or relevant project for the majority of the participants. The importance of English reading proficiency makes sense, since most sources of relevant information at that time were in English. Many still are, such as indexes, abstracts, publishers, electronic lists, newsletters, and general information websites. These results demonstrated that infrastructure and time can be powerful factors, but motivational aspects embedded in strategies may also be powerful.
2. Second course The second course had practically the same general objectives as the first one, focusing more on a critical analysis of academic performance through the Internet. However, task goals and specific contents were not previously established by the professor as in the first course. The design took into account the remaining unsolved problems of infrastructure and students' lack of time. In order to minimize these effects, course duration was increased, criteria for selection of participants were established, and course content and strategies were modified; the design became more flexible, according to students' difficulties and motivation. This course proposed content-based, goal-oriented activity, in which many abilities and subgoals and much knowledge were implicitly required. The final objective was to build an academic information resource website (individually and in group) in which specific contents were negotiated with the instructor and among the students, based on their real work and research issues. The students, along with the instructor, evaluated and selected a free service for Web page storage, such as Geocities, Xoom, and Starmedia, where they stored their assignments and structured their website. In summary, the position assumed by the instructor here gave importance to creating opportunities for the students to perform and accomplish tasks in a genuine situation, to perform collaborative tasks, and to reach goals according to group and individual needs and desires. Participants
Nine graduate students, one journalist, one business administrator, one social communication college teacher, one marketing college teacher, two teachers of English as foreign language, and three pedagogical advisors applied for this course. The enrollment requirements were a) to have their own Internet account and connection at home, b) to be a proficient English language reader, c) to have basic computer skills in the operating system, word processors, browser, and e-mail clients, and d) to have 10 extra hours a week available to practice. The students who judged that they could not fulfill all the requirements (either because their level of English reading was poor, or because their computer skills were inadequate) were accepted upon agreeing to work additional hours. A student who had more experience with Internet than his classmates did offered to help the others with technical problems. Although this student was experienced in the use of Internet tools, he was not confident using them as academic tools. Internet access and the available infrastructure
Aside from their own Internet access and accounts, students could use the undergraduate laboratory in the afternoons. The undergraduate campus was situated slightly less than one kilometer from the graduate department. The instructor could use a shared dial-up access available in one room in the graduate department, or use her own Internet dial-up access and account at home. The undergraduate laboratory was reserved for classes one morning every other week. It had 40 PCs (133 MHz, 32 MB RAM, HD 2 GB, 1 MB Video RAM) connected in a Novell local network with direct access to the Internet through a 64 MB line to the ISP. The other
sessions took place in a regular classroom on the graduate campus, with desks and a white board, where lectures and discussions were carried on. Procedure
The course was divided in classroom and remote activities. Sixty accredited hours were given: 15 weekly four-hour sessions with students and the instructor in a regular classroom (desks and white board) or with students and instructors at the laboratory every other week from the sixth session on, where some online activities took place in pairs or groups. The remote activity -- that is, homework and online work -- was not computed as instruction hours for administrative purposes. The general content involved the following topics in theoretical discussions and practical activities: • • • • • • • • • • • • • •
the infrastructure and general purpose of an academic network client server model, Internet software and hardware, protocols communication and information Internet services and addresses search tools and engines in the WWW publishing tools for the WWW types of academic information distributed by different Internet services and tools search of bibliographical data and specific academic services based on the WWW the Internet as a tool and object of study user-interface studies and research evaluation of information quality on the Internet cyberculture and virtual communities studies teaching and learning studies in the Internet context introduction to the planning of Web-based courses pedagogical issues regarding Web-based classes
Classroom and laboratory activities involved: • • • • • • • •
short instructor lectures student presentations discussion of themes and questions related to academic readings proposed by the instructor and students discussion of issues raised from online activities planning and organization of the Web project content planning of instructional activities and feedback mechanisms writing short papers about classroom discussions and questions practicing and discussing the online use of Internet clients, tools, and services.
Remote activities involved:
• • • • •
asynchronous communication through e-mail and list by the group and instructor gathering of online material reading and analysis of the material gathered material preparation for Web storage purposes Web storage and organization of online information
The first two sessions were dedicated to planning and organizing the work routine, based on discussions of students' research and work themes. The themes for readings were derived from instructor-student dialogue. Readings and discussions, alternating with short lectures, involved the selected topics mentioned above. The complexity of practice activities was negotiated with the students during the course. In order to help initiate practice, two Web pages were provided -one containing academic resource information addresses, the other containing guidelines for bibliography searches. In order to become more technically familiar with the tools, each student agreed to store homework in individual Web pages. Then, they had to edit all the individual material (handwritten or in HTML [hypertext markup language] format) and organize it in a collective website according to data and criteria they had read and discussed. Student evaluation was based on in-class and online participation and production, in both individual and collective publication of assignments negotiated with the instructor (some in HTML, others distributed through e-mail). Results and discussion Face-to-face attendance and dropout rate
There were no dropouts, although one of the students missed classes, was always 30 minutes late, and couldn't accomplish any of the work established by the instructor and group. He complained about his defective microcomputer and lack of time either to have it fixed or to work on campus. The average rate of attendance was 91 percent. Online interaction
By the fifth session, the site for the Web page storage was chosen. Since the site had a forum service, some of the messages exchanged by the group remained at the site and could be revisited anytime. However, not all the messages exchanged were stored at the site. Two hundred and forty messages were sent to the group list. Thirty-one percent were sent by the instructor, and 29.5 percent were sent by the student who was more experienced in the Internet. He also forwarded some peer messages to the list. Three percent came from another graduate professor who collaborated in finding course material once in a while. Thirty-nine percent were sent by the other students, who could be divided into two groups of three: a) average-level interaction, ranging from 0.7 to 14 percent, and b) minimum interaction with less than 0.7 percent. It is important to notice that some messages were not sent directly to the list by the students, but were sent to classmates. As the course progressed, online interaction decreased: 41
percent of the total messages sent during the course were exchanged in the first month, 28 percent in the second, 10 percent in the third, and 5 percent in the last. The content of these communications was mostly organizational (instructor's feedback, comments and suggestions to all students, and students' messages to all proposing and assuming tasks) and technical (students asking for help, instructor and students giving help). There was student-student communication, too. At the beginning of the course, the numbers showed an active list. As soon as the course evolved, the list activity decreased. This has a simple explanation. The individual tasks decreased and collective work was required in order to accomplish the collective site; the greater part of these messages were sent directly to the classmates with whom the student was sharing the activity. The level of remote, online interaction was less than that expected by the instructor, since the group had agreed that part of the theoretical discussions and work would be distributed to all in the class list. The students took more time to solve technical difficulties, and they didn't have time to deal electronically with the material gathered in the interval between classes. So, they printed the material, read it at home, and discussed it in the classroom. Performance observations and reports
The assignments exhibited in class and in the website formed the students' portfolio for evaluation. These materials showed that practically all of the students (89%) were able to: • • • •
look for Web page services and evaluate them in order to select useful academic information, look for academic material related to their area of interest, communicate and share interests with other researchers or students through e-mail and participation in discussion lists, and edit HTML and publish their reviews and other academic findings on an individual site using basic graphic resources practiced during the course and organize them in one collective site, according to certain concepts they read and discussed in class.
Priority was given to content organization and quality, instead of graphic embellishment. The collective site was remarkably superior to the individual pages both in quantity and quality (URL: http://members.xoom.com/tgcarmona/default.html). In terms of specific skills for using Internet software, tools, and services, they were able to get an Internet account, install communication and Internet software, communicate more frequently through e-mail, navigate the WWW, use search engines, download and upload material through an ftp client, and edit HTML. Of course, there were different levels of performance and achievements. Although most had already used e-mail and the WWW before the course, some took much more time to achieve results than others and still had to follow personal written notes and instructions.
Performance expectations
Judging by the commitment to course requirements, the instructor expected a higher level of accomplishment. However, the students emphasized that their ability to gather relevant academic material through Internet information and communication services increased (in some cases, from zero). According to discussions, assignments, and reports collected during sessions, all the students, except one, had insufficient previous experience with Internet clients or other software, and six were slow English readers. All students needed more time to complete activities, some as much as 15 hours. One group misjudged the time they had available for the courses, and in some cases, had as few as six hours a week to work. The most experienced student said he needed seven hours a week to complete his work. Even those who had more experience with word processors or graphic editors demonstrated difficulty in transferring these abilities to other tools as Internet clients. For example, the student who was a proficient English reader and had very good computer skills with text and graphic editors said that six to eight hours a week were necessary in order to achieve her goals. Although the students also complained about the slow Internet access they had in the laboratory sessions, having some classes in the Internet laboratory proved to be essential for those who had poorer computer skills. The quality and quantity of discussions of course content were negatively affected by the effort spent resolving technical problems. Some students reported that they wanted to explore and discuss subjects of interest in greater depth and quantity. Technical difficulties forced the instructor to be less demanding concerning the focus of the course. Changes were made to accommodate the majority of student needs. However, the students became aware of and put into practice important concepts of academic Internet use, such as criteria for establishing quality of information, usability, and peculiarities of the media in communication and education. At the end of the course, they reported that their technical level as well their comprehension of Internet technology use improved (some had serious prejudices about using the Internet in academic work). They also reported that the benefits they got from the Internet for their research and educational processes increased substantially during the course and that they were satisfied with their progress. Some reported that the Internet helped them to find theoretical and methodological trends in their research areas. These better results in the second course can be ascribed to modifications in course methodology motivated by the necessity of managing student lack of time, access difficulties on campus, and maintenance of student interest in Internet academic usage. Refrence: http://www.isoc.org/inet99/proceedings/2f/2f_4.htm
Related Documents
Evaluation Strategies
May 2020 2
Vanguard - Evaluation Of Tactical Strategies
June 2020 4
Strategies
June 2020 20
Strategies
May 2020 33
Strategies
November 2019 41