Sistem Pemantauan Kecerdikan

INTELLIGENT TUTORING SYSTEM USING MULTI-AGENT RULE-BASED SYSTEM

Tema: Integrasi Dalam Pendidikan

Ahmad Rizwan Romli Faculty of Information Technology & Quantitative Sciences, Universiti Teknologi MARA, 40450, Shah Alam, Selangor Darul Ehsan, Malaysia [email protected]

Norzaidah Md Noh Faculty of Information Technology & Quantitative Sciences, Universiti Teknologi MARA, 40450, Shah Alam, Selangor Darul Ehsan, Malaysia [email protected]

Abstract Intelligent Tutoring System (ITS) is a multi-agent expert system developed specially to give a one-to-one tutoring to students which simulates the student-teacher learning environment. This system provides training and tutoring to students by giving notes, examples, exercise, hints and corrections just like how teachers do in teaching their students. The objective of the system is to teach and tutor the student to improve the student’s understanding and ability in mathematic similar to the teacher-student one-to-one tutoring. Student who uses the system will be given notes, examples and exercise. The tutoring method is different for different type of student. Student who is good in mathematic and will be given notes and exercise which is up to their level while weak student will be trained until they reached level for the good student. The system will train student until they are good and passed all the exercise successfully. Once a student passed all the exercise in the system, the training will end and the student will be classified as advance in the subject.

Keywords: Intelligent agent, expert system, artificial intelligent, intelligent system, intelligent tutoring system

1.0 Introduction Students’ understanding and perception on a certain subject at school depends on how much they are tutored directly by the teachers. Students especially from the primary school, needs special attention from the teachers where they need the one-to-one direct tutorial to make them learn faster and understand better. The more the students given direct tutorial, the better they will understand and the faster they learn new topics. Involving students in tutorial is an important process. Students need to participate in the tutorial to understand what they are learning. Tutoring students without students’ participation only could

let the students remember for a short time. In order to store something in their long term memory, students need to participate in their activity. The problem with the current educational method is that students are lacking one-to-one tutorial at school. Only several of the students afford to get personal teacher to come to their house after school to revise what they have learned in school. Teachers at school often find difficult to satisfy all the students need because of the limited time and work congestion. When in class, only students who asked or show their misunderstanding are given attention by the teacher for one-to-one tutorial. The others who stay at the back listening without ever really manages to understand what they are learning will miss the one-to-one tutorial. As a result these students are still hanging without answers and miss the boat to proceed to the next chapter. In search for the alternative solution for this problem, several tutoring approaches were developed including the Computer Based Training (CBT) and courseware. This computer software was designed to provide students with the alternative way of one-to-one tutorial and at the same time improving their understanding and perception. This solution at first was seen as success as more of this software was developed to provide education tutorial to the students. Students also accept this method easily as they prefer to interact with computer which they find it interesting and fun. However the problem for this method is that the system’s tutoring approach is prefix and that all user’s will get the tutoring approach from the system. This tutoring approach however is not the optimal solution and does not reflects the real tutoring between students and teachers at school. The approach does not adapt to students’ capability and that it assumes all students are the same. There are needs for a system that can really simulate the one-to-one tutoring between students and teachers and adapts to the students’ capability. An ITS is a system that not only provide tutoring materials to students, it also adapts to the students capability. Every student has different capabilities and has different type of understanding. Advance students, are less dependent to tutoring materials and able to learn new things faster than the weaker ones. This various type of students’ capability allow the needs of ITS to be initialized. Therefore, in order to address the issues mentioned above, we aim to: 1. determine the suitability of using rule base intelligent agent for web based intelligent tutoring system. 2. design and develop the intelligent tutoring system. The project is aim to produce a system that can tutor twelve years old students on subject mathematic focusing on the topic fractions. The system is able to provide students with notes, examples and exercises. Students as user of the system will experience one-to-one tutoring and the system will evaluate their exercises, gives corrections and asses their performance through out the course. The scope of the system is that the system was design for students who have basic computing skills or those who already have experience on using computers before. The system also requires the students to give full attention while on course in order to get efficient results. Any distractions or delay while on course will affect the result of the system.

2.0 Intelligent Tutoring System The ITS is a system that provide tutoring approaches to students based on their capabilities. It is designed using the AI components to first study the students’ capabilities and then come out with tutoring materials that matches with the students. The system is equipped with expert system where the expert knowledge came from school teachers who have experience in tutoring students on the particular subject. The expert system is use to support the intelligent agent to provide the students with matching tutoring materials. Basically the function of the system is to provide students with one-to-one tutoring approach with tutoring materials that match with the students’ capabilities. The system is equipped with tutoring materials like notes, examples and exercises. This materials is use to tutor the students but the students will not get to see all the materials as they will be shown materials that are only necessary to them. This means that advance students who are good in the subject and have better understanding will not be shown materials for the beginners. For example, if a student who uses the system is already good in the subject and

considered to be advance student, the student will be given more difficult exercise rather than basic exercise for the beginners. Beginner student meanwhile will be given basic exercise, notes and examples and will be train until the student reaches the level for advance students. ITS is a multi-agents system where there are 3 different agent working in the background interacting each. The agents have their specific roles and they provide information for each other. All user activities with the system will be capture by the agents via the user interface. User interacts with the system by giving inputs and these inputs are process by the agents to provide feedback to the user. The agents working in the system are student profile, tutor agent and evaluate agent. The agents in the background are separated to the user via the user interface. Both user and agents can control what appears on the user interface and this means that the system and the user are actually communicating each other. Intelligent Tutoring System User Interface Student Profile

Evaluation Agent

Tutor Agent

Figure 1: System Design Framework

3.0 Intelligent Agent The terms "agent" and "intelligent agent" are ambiguous and have been used in two different, but related senses, which are often confused. Intelligent agent can be viewed from two different angles which are in computer science and in artificial intelligence. In computer science, intelligent agent is software that assists users and will act on their behalf, in performing non-repetitive computer-related tasks based on pre-programming rules and the term ‘intelligent’ means its ability to learn and adapt. Meanwhile in artificial intelligence, an intelligent agent is used for intelligent actors, which observe and act upon an environment, to distinguish them from intelligent thinkers isolated from the world. An agent in this sense of the word is an entity that is capable of perception and action. There are many definitions and perceptions of intelligent agent but in as much as the definition, intelligent agents is usually a software-based computer system that contains the properties of autonomy, social ability, reactivity and proactiveness (Marakas, 2003). The word autonomy means agents operate without the intervention of human or other agents and have some kind of control over their actions and internal states. (Jennigns & Wooldrige, 1998). Social ability also known as cooperation refers to ability to interact with other agents via some communication language (Lin, 2005). In reactivity, agents perceive their environment and respond in a timely fashion to changes that occur in it (Marakas, 2003). Proactiveness means agents do not simpy act in response to their environment; they are able to exhibit goal-directed behavior by taking the initiative (Marakas, 2003). An agent topology was designed base on the properties of agents which is collaborative agents, collaborative learning agents, interface agents and smart agents (Lin, 2005). Intelligent agent can be in the form of single agent system or multiple agent system. Jennings and Wooldrige, 1998, in their article said that the multi-agent system is where a system is designed and implemented as several interacting agent. The multi-agent system is to be more complex system than the single agent system but there are cases that single agent system to be more appropriate. The system is a multi-agents system where the agents interacts each other as a whole unit. All agents are important and they receive inputs and provide outputs to each other. Each agent has their specific role and their task is to provide output for the other agents to process. The intelligent agents were constructed to process information received from user’s interactivity with the system interface.

3.1

Student Profile

Student agent is an agent for the user. It will capture the information relating to the user from the first time the user registers until the user finish the using the system. First time user who use the system will have to register their personal detail into the system. The agent will then use the data and pass the information to the other agents. During registration, the user need to key in their full name, date of birth, school, username and password into the system. The agent will pre-determine the new user as a level 1 student which means beginner. This level is however is not fixed as the user levels may varied depending on their progress. There three levels of students identified in the system which are level 1 (beginner), level 2 (intermediate) and level 3 (advance). The user may be in any level and their level depends on their progress in the system. 3.2

Tutor Agent

Tutor agent is the agent that determines the tutoring approach for the user. Each user has different tutor approach use on them. This agent’s task is to come out with sets of questions which are suitable for the user based on the user’s level. The agent will use information captured from the student agent to determine the approach to use to tutor the user. The questions in the question set are not fixed for the whole process as they will change depending on the user’s progress. For new user who has registered, they will get fifteen randomly selected questions suitable for level 1 user. The question however are pre-determined questions and does not reflects the user’s exact level. Once the user starts to answer the questions, the agent will study the user level. After first five questions answered by the user, the agent will be able to have glimpse view on the user’s level. The agent will then change the balance ten questions to set of questions that are suitable for the user’s level based on the first evaluation. This will continue for the next five questions until the user finish all fifteen questions. After the user complete the first fifteen questions, the agent then can predict thoroughly the exact level for the user. The user will then again receive another set of fifteen questions that matching the level determined by the agent. The next round for the user tutoring process will follow the user level and the agent will provide the user with more challenging sets of questions in order to increase the user level and capability. The agent will stop training the user when the user successfully complete all course of questions and reaches the highest level in the system. The rules for determining the student tutoring approach are described below:

Finish 5 questions

Yes Complete all within 100 seconds

Set user level to level 3

No Yes Complete all in between 100 to 200 seconds

Set user level to level 2

No Set user level to level 1

Get remaining questions based on level set

After another 5 questions

Figure 2: First 5 questions in first round

To ensure student’s consistency, the system will check if the user scores minimum 90% for all 3 levels before the user can finish the course. If not, the user will have to another round of 15 questions for the questions level which the user did not manage to score minimum 90%. 3.3

Evaluate Agent

Evaluate agent job is to evaluate the user progress throughout the tutoring process. Every time the user answers the questions, the evaluating agent captures the data and evaluates the user’s performance. The agent will look into student’s weakness and strength and provide information for the tutor agent to determine the set of questions to give to the user. Evaluating agent tasks also are to correct the user’s mistakes and make sure that the user knows where their mistakes are. The agent makes sure that the user answers their questions correctly before proceeding to the next question. Besides that the agent also record the time taken for the user to answer each question and sum up all the time taken to complete all the questions. The agent will provide information like total time taken and user’s strength and weakness to the tutor agent for further process. User who failed to answer a question correctly will have to do correction and the agent will provide them with hints or notes for the user reference. All user activities in answering the questions will be recorded by the agent to provide useful information to the tutor agent. For every questions the user answer, the user will have to go through the evaluation process. Evaluation process is as below:

User answers question

Move to next question

Marking Correct

Wrong

Do corrections

Figure 3: Evaluating process

Time taken is counted by the total time to pass 1 question. For example if the user correctly in the first attempt in 20 seconds, the time taken that will be recorded in the student model is 20. If the user answers wrong in the first attempt and correct in second attempt, the total time taken is time taken in first attempt plus time taken in the second attempt. st

1 attempt : 25 seconds nd 2 attempt (correction) : 20 seconds Total time taken : 45 seconds

Agent will then calculate the percentage of correctness for each level of questions based on the student model. If the user scores at least 90% for each question level, the user is considered as advance student. If not the student need more practices on the questions for the specific level. Correctness percentage is counted by: Total correct answers for specific level divide by total of questions answered for specific level multiply by 100. Example: Question level 3 answered: 25 Correctly answered: 23 23/25 * 100 = 92%

3.4 Expert System Expert system is the system which simulates the human expert. In the case of this project, school teachers have been identified by the researcher as the expert domain. The expert system consists of knowledge from the school teachers in tutoring students and evaluating their capabilities. The knowledge gained from the system are then stored in knowledge bank and rules were made to support the knowledge. The expert systems were used in the agent to help the agent to process the data gathered from the user activities.

In the tutor agent, the expert system supports the agent in terms of what approach to use in tutoring different type of user level. The expert system diagnoses the student’s progress and determines the level of the user. It will then provide information to the tutor agent to construct the questions to be given to the user for further tutoring. Knowledge for expert system were obtain from two school teachers that have experience in teaching standard six school students.

Start

Database

Registration

Student Profile

Read notes

Do exercise

Correction

Fail

Tutor Agent

Evaluate Pass

Evaluation Agent

Level adjustment

No Completed all exercises

Yes Finish Figure 6: Flow chart of the ITS

Figure 6 shows the process flow chart of the system. The system consists of 3 agents working together. Once registered into the system, user has the options to read notes and examples. This activity however is not recorded by the agent and the user a free to navigate through all the contents. When the user finishes reading the notes and examples, the user will then go through the exercise for tutorial process. Activity in this process will be capture by the agent and user evaluation will depend how well the user perform. The process will end after the user completed all three levels of questions within the time range given with minimum 90% score in each level. This process will iterate for several times depending on the user’s capability. Once the process is completed, the result will be showing the summary of the user’s performance.

4.0 Analysis The system result is the output from the system after going through the process of tutoring the user. User rd rd who uses the system will finish the course only after they reach the 3 level and completed the 3 level questions with 90% correctness in each level. After the user finished the course, the system will provide output that summarizes the progress of the user while using the system. The output then can be use to study the pattern of the user who use the system. In order to get the accurate result, the system needs to be tested by real user. The researcher made two type of testing for the system. First is by using simulation to test for the system accuracy and second using the real standard six students to be the user of the system. This allows the researcher to make comparisons between the output gained from the simulation and real students. From the comparisons, the researcher can analyze whether the system has the system accuracy in tutoring and evaluating students. High accuracy is important to prove that the system is ready to be use in the education sector. Further analysis can be described from the table below: Table 1: Result from real student User ID

Level

11 12 13 17 18 19 21 22 23 25 26 27 30 31 32 33 34 35 36 37 38 40 41 42 43 45

3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3

Total Time (minutes) 40 39 26 65 35 41 44 75 50 60 43 49 48 53 49 58 42 40 27 41 80 43 40 55 50 52

Legend: Q: Question answered

Q 10 11 5 30 10 15 30 30 30 35 20 25 16 20 20 50 15 16 25 30 31 20 31 15 21 30

Level 1 C 10 10 5 29 9 14 30 27 27 35 19 23 15 18 18 45 15 15 25 30 30 18 30 15 20 30

% 100 90.1 100 96.7 90 93.3 100 90 90 100 95 92 93.75 90 90 90 100 93.75 100 100 96.8 90 96.9 100 95.2 100

C: Correct answers

Progress Level 2 Q C % 35 35 100 37 35 94.6 20 18 90 45 41 91.1 30 28 93.3 30 28 93.3 15 14 93.3 30 28 93.3 30 29 96.7 40 39 97.5 25 23 92 20 18 90 30 27 90 40 36 90 33 30 90.1 31 30 96.8 32 30 93.75 33 30 90.1 21 20 95.2 20 18 90 50 45 90 33 30 90.1 20 18 90 48 45 93.75 27 25 92.6 31 30 96.7

Q 30 31 35 45 35 30 45 45 30 30 30 30 45 33 49 33 33 33 16 33 47 32 32 30 48 32

Level 3 C % 30 100 30 96.7 33 91.4 41 91.1 32 91.4 27 90 42 93.3 41 91.1 28 93.3 29 96.7 27 90 28 93.3 41 91.1 30 90.1 45 91.8 30 90.1 30 90.1 30 90.1 15 93.75 30 90.1 45 95.7 30 93.75 30 93.75 30 100 45 93.75 30 93.75

No of rounds 5 5 4 8 5 5 6 7 6 7 5 5 6 6 6 7 5 5 4 5 8 5 5 6 6 6

%: Percentage

Table above shows the result of real students. All students are randomly picked from different school and area but from the same domain which is 12 years old standard six students. The student were given short brief about how the system works and given pre-test to get use with the system. When the testing starts, the students were required to give full concentration to the system and not allowed to do other things until they finish. This is to ensure that the accuracy of the result. Result from the test done by the students

showed that all of them are able to finish the course within certain time period. The table shows the User ID, user level, total time taken to finish the course, progress of the students in each level and numbers of rounds required completing the course. Results from the test showed that students who are good in mathematic are able to finish the course within 4 to 6 rounds in approximately 40 minutes. The students were actually learning and improving their ability throughout the sessions. Some of them who made mistakes in the first few rounds were able to see their weaknesses and improved for the later questions. There also students who manage to solve lower level questions in short time but face difficulty as they move to harder questions. Although, these students proved that they are able to maintain their consistency in answering questions. Consistency is the key to the system where the students need to reduce their mistakes while answering as fast as they could. Students who are a little bit slow requires up to 8 rounds of exercises to complete the course. Although the students find it hard to complete the course, the longer the students use the system, the more the students improve their understanding. As they learned from their mistakes, they also able to improve their consistency thus reduce their mistakes. More questions given to the students help them to relate the previous questions they get with the new one. They also made improvement in terms of answering technique where the students learned to answer questions correctly in shorter time. The results also showed that students who finish the course are able to maintain their consistency in answering the questions. The requirement of 90% correctness in each level requires the students not to afford to make too many mistakes thus improving their consistency in answering correct answers. Results from the test proved that the system is capable to provide one-to-one tutoring to students and adapt to the different level of students. Students are given exercised base on their level and they are tutored until they are able to do more difficult questions. Generally, students who finish the course made improvement in terms of understanding, perception, consistency and time management. Result from the test can be justified further if more tests on real students are made. More results from real student can help to see more pattern of how the students response using the system. So far the results shown already prove that the system met its objective but with more real student data, more information can be gather.

References Abushar, S., & Hirata, N. (N.D.). Filtering with Intelligent Software Agents. Retrieved October 1, 2007 from http://www.engin.umd.umich.edu/CIS/course.des/cis479/projects/FISA.html Ali, S. S., & McRoy, S. (2000). What is an Intelligent Tutoring System? Retrieved September 1, 2007 from http://www.cs.niu.edu/~freedman/papers/link2000.pdf Beal, C. R. (N.D.) Intelligent tutoring for math achievement testing: A controlled evaluation. Retrieved September 1, 2007 from http://wayangoutpost.net/paper/WayangEvaluation.pdf Beal, C. R., & Lee, H. (N.D.) Creating a pedagogical model that uses student self reports of motivation and mood to adapt ITS instruction. Retrieved September 1, 2007 from http://k12.usc.edu/papers/Beal&LeeCRC.pdf Fuchs J., Heller I., Topilsky M., & Inbar M. (1999). CaDet, a computer-based clinical decision support system for early cancer detection. Cancer Detection Prev. PubMed. Beeri Community Health Center, Tel Aviv, Israel. Pages 78-87. th

Gauthier G., Frasson, C., VanLehn, Kurt., M. (Eds). (2000). Intelligent Tutoring Systems: 5 International Conference; proceedings / ITS 2000, Montreal, Canada, June 19-23, 2000. Springer-Verlag. New York, United States. Hafner, K. (2004). Software Tutors Offer Help and Customized Hints. New York Times. New York, USA. Hayes-Roth, B., & Larsson, J. E., (1998). Guardian: An Intelligent Autonomous Agent for Medical Monitoring and Diagnosis. IEEE Intelligent Systems. IEEE Educational Activities Department. New Jersey. USA. Pages 58-64

Jennings, N. R., & Wooldridge, M. (1998). Application of Intelligent Agents. Queen Mary & Westfield College, University of London. Retrieved October 1, 2007 from http://agents.umbc.edu/introduction/jennings98.pdf Jeremic, Z., Devedzic, V. (2004). Design Pattern ITS: Student Model Implementation. In Fourth IEEE International Conference on Advance Learning Technologies (ICALT ’04). Retrieved September 1, 2007 from http://csdl2.computer.org/persagen/DLAbsToc.jsp?resourcePath=/dl/proceedings/&toc=comp/proc eedings/icalt/2004/2181/00/2181toc.xml&DOI=10.1109/ICALT.2004.1357693 Lajoie, P. S., & Vivet, M. (2002). Artificial Intelligence in Education : Open Learning Environments . New Computational Technologies to Support Learning, Exloration and Collaboration. IOS Press. The Netherlands Kiong, S.W., Rahman, A. L. A., Zaiyadi, M. F., & Aziz, A. A. (2005). Expert System in Real World Applications. Generation 5. Retrieved October 5, 2007 from http://www.generation5.org/content/2005/Expert_System.asp Ljungberg, M., & Lucas, A. (1992). The OASIS Air Management System. In Proceedings of the Second Pacific Rim International Conference on Artificial Intelligence (PRICAI '92). Seoul, Korea. Retrieved October 1, 2007 from http://citeseer.ist.psu.edu/ljungberg92oasis.html Limoanco, T., & Sison, R. (2002). Use of Learner Agents as Student Modeling System and Learning Companion. In ICCE Proceedings of the International Conference on Computers in Education. IEEE Computer Society. Washington, DC, USA. Page 1331 Lin, F.O. (2005). Designing Distributed Learning Enviroments with Intelligent Software Agents. Information Science Publishing. United States. pages 162-180. st

nd

Marakas, G. M. (2003). Decision Support Systems in The 21 Century 2 Jersey, USA. Pages 530-570.

edition. Prentice Hall. New

Mazza, R. (2003). Using Open Student Models in Distance Learning Environments to Help Teachers Provide Adaptive Tutoring. In SCOMS- Studies in Communication Sciences, Special Issue New Media in Education. Retrieved September 1, 2007 from http://www.inf.unisi.ch/assistants/mazza/Web_area/Pubblicazioni/SCOMS2003/scoms_feb2003v2 .pdf nd

Negnevitsky, M. (2005). Artificial Intelligence: A Guide to Intelligent System 2 England. pages 25-54.

edition. Addison-Wesley.

Ong, J., & Ramachandran, S. (N.D.) Intelligent Tutoring Systems: Using AI to Improve Training Performance and ROI. Retrieved September 1, 2007 from http://www.stottlerhenke.com/papers/ITS_using_AI_to_improve_training_performance_and_ROI.p df Russel, S., & Norvig, P. (1995). Artificial Intelligence: A Modern Approach. Retrieved September 1, 2007 from http://www.cs.berkeley.edu/~russell/aima1e/chapter02.pdf Thaw, T. H., & Phon-Amnuiasuk, S. (N.D.) Intelligent Tutoring System for Mathematical Problems: Explanation Generations for Integration Problem (EGIP). Retrieved September 1, 2007 from http://pesona.mmu.edu.my/~somnuk/papers/EGIP.pdf