Evaluating An Nline Ourse

Tech 4102

Course

Done by:

Overview

This course is Aysha Al-Shuaili provided by Alison that provides free online courses. It was developed by (67060) Learning Innovation co. In this course, learners are supposed to gain Zainab Al-Shidhani MS word 2007 basics including: the new tool bar tools, tables, drawings and pictures, advanced properties and texts formatting. This course is delivered as webbased. Presentation of this course is provided in different ways. There are flash movies prepared to describe the way the program works step by step. In addition, the course structure is presented in hieratical textual manner which shows the user what are the main topics and what goes under each topic. Users are given the choice to select the order they want to follows in learning this software. If the user finishes one topic, watching the video only, he will be considered as he finished that section. There are no exams given to learners to check their performance but once the learner registers with Alison; a record will be created to this learner. In this record, learner can monitor their advancement in Alison as whole (i.e. what courses he’d taken, what certificates, and how many points he gained.

The purpose: There many Alison courses in Alison.com. Arabic Training of MS Word 2007 is the only Arabic course in Alison until know. This report will try to measure the effectiveness of this course in teaching MS Word in Arabic in terms of content accuracy and course visual design.

Participants: This report instrument was uploaded in surveyconsole.com. 20 visitors had responded to this survey. People who want to answer the survey were given access to the evaluated course (we provided username and password to those who are going to answer the survey to make it easer to participant visiting the course (no need for registration). People who responded to this survey were invited via mail to solve the survey. All of them are studying the course of “Evaluation of Educational Technology”.

The evaluation instruments: Two surveys were used to develop this report survey (See the reference). This survey only includes items that evaluate content accuracy and course

design from students’ perspective. The survey consists of two main sections. The first section was examining students’ perspective of the course content accuracy (items (1-9)). This section has sub sections which are: content arrangement, goal statement in the content, assessment and content and content usefulness. The second section was examining students’ perspective of course design (items (10-22). This section, also, has sub sections which are: navigation and components clarity and functionality. At the end there is a question asks students about there overall satisfaction about the course. This survey apply 5 Likert scale (1.strongly disagree 2. Disagree 3.not sure 4. Agree 5.strongly agree).

The results Content Accuracy: a. Content arrangment: About 40% of participants think that the course organization is poor and the course is structured illogical. The rest (45%)agreed (agree (20%), strongly agree (25%)) that the course is well organized and logically structured. b. goals statement in the content most of the responses for this section were in the not sure – disagree area. 35% of participants didn’t understand the course goals and objectves while the same percentage understood the course objectives and goals. The rest were not sure. Similar result was concluded for goals and objectives matching to the taught content but the negative respones are more dominant. Those who are not sure still 30% of the whole group. c. Content usefulness:

Regarding course length appropriateness, most of the responses were in the “not sure” area. Then 30% of those who responded agree that the lenghth of the course wasn’t appropriate. The rest (25%) agreed in the appropriateness of the course lenghth. The difference between the two last groups only one participant response. For participants view of the online facilitation of the subjcet taught in the course, most of the responses fall in the “not sure” area but most of them diagree or strongly disagree that this course online materials have significant important in facilitating the learning of the subject. Four participnats out of tewnty only agree that the online course materials are useful. The third part of this

section emphasizes participants previouse responses. Most of participants (> 40%) disagree that the conetnt is useful and wealth with information and skills. d. Assessment & content: Responses to this section show that most of participant don’t know how they will be evaluated in this course (45%). Only 30% of participnats belive that they know how they will be evaluated in this course (20% of them are very sure of that). On the other hand, most of patricipants believe that the course assessment strategies are not approprite in terms of time and content. Only 25% of particpants agree (and some strongly agree) that course assessment are appropriate to their learning as students. Course Design: e. Navigation: The majority (55%) of responses to this item fall in the “disagree” and”not sure” area. But when comparing the whole disagree (disagree and strongly disagree) with whole agree (agree and strongly agree) the “Agree” gains more responses (40%) than disagree (35%). On the other hand, most of participnats (40%) agree that the course content is easy to download and access. Although this precentage is higher than other responses but the difference between this AGREE and DISAGREE only 10%. f. Components clarity & function: The highest rate response is “not sure” (>42%) for the clarity of the course title and instructor while more than the third agree that they see the course title and instrcutor in the course homepage. Regarding the other design compnents, icons are labled in unapprporiate way (50%) and they are not properly sized in the screen (50%). The number of these icons also seen as is excessively used. Many participant weren’’t able to judge the menu consistancy with course home page. Most of the responses related to the colors and text in the course design are positive. These responses indicates that text and colors make the content easy to read (50%) and the used colors doesn’t affect the clarity of content in the page. Majority of participants agree that this course design help deliver information clearly (>50%) but still there is similar presecntage that

said this ourse design doesn’t convey the meassage as it is (>45%). On the other hand, most of participants belive that the course pages are consistent visually & functionally. There preference of Alison courses wasn’t so obvious. 40% think that they will join more Alison courses whereas 35% don’t think so. The rest (25%) not sure. Similar results found regarding the overall course design.

Discussion: Most of the responses were nagative. In the content accuracy responses, the responses are conflicting with each other. While 40% said that the organization is poor, there are participants whole found the organization and structure is logical and well arranged. This response can be interpreted as participants didn’t understand the structure of the course. Also, many participants didn’t understand the goals stated in the course home page and content when presented. This maybe because these objectives werent written in “points” manner, ethier they were announced in verbally when the lessons are presented (flash presentation) or they were written within the introductory paragraph in the homepage. Participants responses to the usefulness part wasn’t clear enough because most of them responded as they are “NOT SURE” which means they don’t find it useful nethier useless. Finally, the assessment part. Participants responses consistant with what was mentioned in the introdution of this report, that this course evaluation is not clear. The only abvois way to know particpant progress is finishing watching the videos which will mark the finished one with blue bullet (if all watch) or red bullet (if patially watched). Participants responses to the navigation were slightly possitive. Participant maybe find the way the content arrangment some how traditional which may ease the navigation for others and no for others. Many responses shows that this course design components are not clear (title, instructor, icons lables, icons size and number in the screen). This is maybe because the very simplisity of the course design. Participants were satisfied with colors and text used in the course this is also may be refered to the desighn simplisity of the course. The last part of this evaluation instrument was examining the participants perception about the design role in delivering the information. Most of them agree in the functionallity of the course design in deliverig the content. This is conflict with what they said regarding the course organization and content clairty. All these conflicts in the results caused by the careless in responding to the survey. Most of the responses doesn’t refelct the course quality and the abvious evidince that the average time taken to answer the survey was only 1 minute. Advantages and limitations of the technology, based on the findings.

according to what was mentioned in the discussion, most of the responses were conflecting each other and don’t represent the course quality. But overall. This course have number of advantages and disadvantages which are: advantage: 1. The only course in arabic in Alison.com 2. The content is accurate and well organized 3. The goal is stated in the introdution (but not in bulleten form) 4. The content useful and its lenghth is appropraite in relation with the objectives. 5. The course design components are clear but they need more caption or clearer lables (for some icons) Limitations: 1. Course objectives are not mentioned clearly in textual manner 2. The course assessment not appropriate to assess students progress 3. The navigation through Alsion needs more practice untill someone will not be lost in the course website.

Reference: Mark, S., & Charles, W. S. (2007, 12 2). Student Course Evaluations: Common Themes across Courses and Years. Med Educ Online , pp. 1-9. Statewide, N. (n.d.). Course Rubric Checklist. Retrieved April 28, 2009, from Rubrics for Online Instruction: dev.scs.nevada.edu/committees/connectivity/06minutes/060606docs/RUBRIC%20C HECKLIST.doc

Appendices: Appendix A: screen shots of the course

Appendex B: Research Instrument: Survey: Please respond to these items according to what you perceive about the Arabic Training Course for MS Word. You response should represent your

opinion about the course. The used scale means the following:1= strongly disagree 2= disagree 3= not sure 4= agree 5= strongly agree Content Accuracy: a. Content arrangment: 1. The course was well organized and was presented in a logical sequence. b. goals statement in the content 2. I understood the goals and objectives of the course 3. What was taught matched the goals and objectives. c. Content usefulness: 4. The length of the course was appropriate. 5. What was presented in online materials facilitated my learning of the subject 6. The content is helpful and wealth with information and skills d. Assessment & content: 7. I understood how I was to be evaluated in this course. 8. Course has multiple timely & appropriate activities to assess student learning. Course Design: e. Navigation: 9. It is easy to navigate through the course 10. Course content is easy to access/download f. Components clarity & function: 11. Course title and instructor obvious on homepage 12. Icons are clearly labeled 13.Images/icons are sized appropriately 14.Aesthetic design presents & communicates course information clearly 15. Text type & color easy to read 16.Page background color or image does not interfere with font 17. Homepage does not have an excessive number of icons (12-16) 18.All pages are visually & functionally consistent 19.Course menu is consistent with course homepage 20. Course provides visual, textual, kinesthetic, and/or auditory activities to enhance learning. 21. I think, the overall look of the course is appropriate for all audience 22.I like the course design

23.I need to join other courses in the Alison site or other sites that provide online courses