Virtual Reality, Games And Child Literacy

A Proposal for a Child Literacy Development Game Using Augmented Reality Hugo R. da Silva Filho Universidade Federal Rural de Pernambuco [email protected]

Lu´ıs G. R. Alves Universidade Federal Rural de Pernambuco [email protected]

Marizete S. Santos Universidade Federal Rural de Pernambuco [email protected] Abstract This paper introduces a recently started research project that aims for the development of a series of games based on Augmented Reality that will serve as entertainment and also as an educational tool. The main focus of this document is on the first game of the series which is currently being developed. Some information on Child Development and correlated works are also presented as they serve as a base to this project.

1. Introduction Computers have become part of the daily life of an increasing number of people all over the world in many ways: at work as a tool helping to get a variety of tasks done, for fun at home as an entertainment option or at school, helping with research and other activities. Another important observation is that computers are being introduced in children’s life much earlier than it was a few years ago [9]. The constant evolution of Information Technology has forced many institutions and industries to update their processes and the way they deal with the surrounding world and markets, in order to keep themselves competitive. Education is no different. It needs to be frequently taken to a next level of sophistication since educators recognize the need of exploring new technologies potential in the search for new ways of teaching. The evolution of the teaching and learning process is necessary due to the new abilities needed in the newly formed information society. A motivation to the use of games and entertainment tools is the ludic approach of it. It is also known that children implicitly learn and develop a variety of abilities by simply playing around. Two distinct goals (both very important) can be achieved by this ludic approach: entertainment and traditional school education [9] [6].

This paper does not suggest a solution to illiteracy; it describes part of a recently started research on Child Development and Augmented Reality (AR) that involves an investigatory work to find ways of using AR and games as tools to aid the development of some abilities in children. In the case of the first game of the series the focus is on child literacy process and reading and writing skills.

2. Child Development and Games A lot of research has been done on Child Development and there are many theories and studies that try to understand and describe it. One of the most accepted theories is the Theory of Cognitive Development by Jean Piaget. According to Piaget’s theory Child Development can be summarized in four main periods: the Sensorimotor Period (years 0-2), Preoperational Period (years 2-6), the Concrete Operational Period (years 6-12) and the Formal Operational Period (from 12 to adult age) [10]. Literacy development starts from the early ages, but it is mostly noticed and accelerated around years 5 and 7. This means the kid is walking between the Preoperational and the Concrete Operational periods, in which they appreciate symbolic plays and games with rules. Symbolic plays are activities like story telling, daily routine reproduction or simulation of feelings or imitation. Games with rules are based on a set of rules that must be applied to specific situations. The player’s non-compliance is usually somehow punished. During this transition between stages of development children start developing the logical thought in opposition to the intuitive thought that rule their acts earlier in life. Games have the capacity of retaining the children’s attention and can also create the sensation that learning is fun, providing to the player the opportunity to develop abilities like processing events, raise the understanding of contexts and situations and make logical inferences to solve problems [8].

3. Augmented Reality Augmented Reality is the process of overlapping virtual three-dimensional objects, generated by computer, with a real environment, through some technological device. It allows the user to see the real world around added of virtual objects to it as complementary devices. This causes the user to have the idea that both virtual and real content coexist in a determined environment [7] [4].

4. Correlated Works There are some researches and works done in education and Augmented Reality but we have no records of its use in games related to literacy, reading or writing development. At least two important and motivating works in Augmented Reality shall be mentioned. The first one is by Brett E. Shelton and uses AR to teach astronomy concepts of ”earth-sun relationships” in an application that lets students manipulate the virtual planets while viewing their respective information [11]. The second AR work is one of the HITlabs’ interface projects called Augmented Tangible Molecular Models. This project helps students to understand complex molecular structures and features a Magic Book with animations [1]. Regarding the reading and writing processes, the game Pegue as Letras, shown in Figure 1, has been an important reference.

the screen, and move the robot around to catch the letters that are part of it. There is an option that activates the easymode, in which the correct letters blink to indicate they are part of the current word. This is very useful to let younger children play even when they have not started recognizing the difference between capital letters [5].

5. Tools and Integration This section describes the tools used in the research, the most relevant features of them and also exemplifies and demonstrates their integrated use.

5.1. OGRE OGRE (Object-Oriented Graphics Rendering Engine) is a 3D engine written in C++. Its main goal is providing to developers the capability of creating applications with hardware accelerated graphics more easily. This is achieved by abstracting lowest level underlying technologies such as OpenGL and Direct3D [2]. Although people commonly misunderstand it, OGRE is not a game engine. It does not provide game related features such as collision detection, physics simulation and networking, among others. Because of this focus only on graphic’s features, it needs be used along other libraries (like sound and physics libraries) in order to meet the needs of game development. OGRE was the chosen engine because of its design-led architecture that makes it useful in many different kinds of graphical applications, from simulations to games. Another strong feature that was important was the cost and the license. It is free of charge and runs under the GNU Lesser Public License (LGPL).

5.2. ARToolKit

Figure 1. Beach scenario of Pegue as Letras.

Although Pegue as Letras employs the formation of words through letter catching, its focus is on younger children starting from the age of 3. All the player has to do is look at the grayed word, located in a box on the corner of

ARToolKit is a software library that helps on the development of Augmented Reality applications by providing facilities to deal with 3D objects, multiple camera position and orientation, marker patterns tracking, rendering, and interaction [3]. ARToolKit provides the video capture facility that is necessary to do markers tracking from webcam data. The process of markers tracking demands some processing time, consequently, it decreases the frame rating. Nevertheless, using a small number of markers at a time can guarantee that the software will run with satisfactory frame rates even on lower configuration machines. The performance of the library is directly related to the hardware used to run it; it varies according the type of video card, the amount of RAM memory and the capacity of the processor, for example. This had a great importance when

it came to the decision on what to implement, considering that many Brazilian schools have outdated equipment. In this sense, the team decided to use a short number of markers (around three at the most), and simple light weight 3D models.

”deep” into the screen; a notebook 3D model, positioned at a marker; a trash box, located at another marker; a scoreboard, positioned on the top-right corner of the screen; and the virtual letters objects that are thrown to the player.

5.3. Integrating OGRE and ARToolKit

6.2. Word List Editor

The use of ARToolKit and OGRE is complementary. The lack of common entities or interfaces to rule the relationship between these technologies does not generate the need of any special implementation or significative coding effort. The ARToolKit library has been used to do video capture and to track the makers while OGRE has been used to place the 3D models in the scene (e.g. Figure 2).

The Word List Editor is a tool used to update the list of words in the game’s dictionary. The database basically has two entities: Categories and Words. Categories represent the idea of a particular vocabulary such as words related to a beach or to a forest environment. The list of words is stored in a Database but there are plans to implement export and import functionalities to support the data exchange through XML files. This feature will allow educators to trade vocabularies and consequently adapt and improve their dictionaries.

6.3. Playing the Game

Figure 2. Sample code that uses OGRE and ARToolKit.

Concerning the positioning operations applied to the 3D models there has been no need of changes or calculations for the coordinates provided by ARToolKit to the OGRE objects because they both use the same right handed coordinate system.

6. Game Description This section briefly describes the game’s user interface and its controls, as well as the complementary Word List Editor software. The goal to be achieved by the player is: given a random sequence of letters, the player must put them together to form existing words. To meet this goal the player must decide which letters are going to be used and which ones shall be discarded.

6.1. User Interface The real world image captured by the video camera will be overlapped with the following interface elements: a letter throwing mechanism, positioned at the center and

There are basically two actions in the game: picking up the current thrown letter to use it in a word or discarding it. These actions are represented by a notebook model and a trash box model and each one of them is located over one marker. The game uses two markers: one to locate the trash box model (that holds the discarded letters) and the other one to locate the player’s notebook (that holds the letters chosen by the player). The markers must be placed side to side, within a distance that allows the player to comfortably place one hand on top each one of them. When the game starts the player must have both markers hidden underneath his hands. The throwing mechanism will start throwing the letters toward the center of the screen, in a parabola route going from deep inside the monitor towards the player. If the player wants to discard the letter, the hand over the trash box must be raised, when this happens the trash box model will appear and the letter will change its route and go into the trash. This letter won’t be thrown again, except in another match. If the player wants to use the letter, the hand over the notebook marker must be raised, when this happens the notebook model will appear and the letter will change its route and go into it. This letter will be thrown again so the player can use it more than once. When the player takes neither action, the letter goes back to the throwing mechanism, and sooner or later will be thrown again.

7. Conclusions The use of Augmented Reality for educational purposes is a recent tendency.Nevertheless, some research products are already being used in fields like chemistry and biology in high school classes. Augmented Reality software can interact with users in interesting ways and this can be explored to bring to the game a better gameplay (experiences and sensations of the players during the game). Concerning the tools and libraries that are being used, both OGRE and ARToolKit have been responding well to all expectations and fulfilling the requirements of the application development. OGRE and ARToolKit have proved to work well together, although they have no common entities or correlated interfaces. The proposed game is being developed on Criatronics’ labs since January 2008. A beta version is expected to be released later on June 2008 and, after all needed corrections and improvements, a variety of data will be collected through controlled experiments to gather information from children and teachers. Finally, the expectation is to acquire enough information from participating children and teachers to evaluate and verify whether they like playing the game, if they like the AR experience in this kind of game and other questions that serve as guideline to the project.

8. Acknowledgements The authors wish to acknowledge the Secretaria de Educac¸a˜ o a` Distˆancia (SEED)/MEC for the financial support that made it possible for us to work on this research. We would also like to thank the N´ucleo de Educac¸a˜ o a` Distˆancia of UFRPE for the complementary support.

References [1] HITLab Projects Homepage. Available: http://www.hitl.washington.edu. [2] (2008) OGRE 3D: Open source graphics engine. Available: http://www.ogre3d.org/. [3] ARToolKit Home Page on HITLab (2008) [Online]. Available: http://www.hitl.washington.edu/artoolkit/. [4] R. T. Azuma. A survey on augmented reality. Available: http://www.cs.unc.edu/ azuma/ARpresence.pdf. [5] H. da Silva Filho. Jogos educativos de computador na educac¸a˜ o infantil. Available: http://www.newhorizons.org/strategies/technology. [6] J. Lopes. A introduc¸a˜ o da inform´atica no ambiente escolar. Available: http://www.clubedoprofessor.com.br/artigos. [7] P. Milgram. Augmented reality: A class of displays on the reality-virtuality continuum. Telemanipulator and Telepresence Technologies, 2351(7), January 1994.

[8] P. Moratori. Por que utilizar jogos educativos no processo de ensino aprendizagem? Available: http://www.nce.ufrj.br/ginape/publicacoes. [9] V. B. Oliveira. O Brincar e a crianc¸a do nascimento aos seis anos. Editora Vozes, Petr´opolis, RJ, 2000. [10] J. Piaget. A Psicologia da Crianc¸a. Bertrand Brasil, Rio de Janeiro, RJ, 1998. [11] B. E. Shelton. Augmented reality and education: Current projects and the potential for classroom learning. Available: http://www.newhorizons.org/strategies/technology.