Internet Based Wireless Mobile Robot Network M. Farajmandi and J. Gu
Max Meng
Peter X. Liu
Yangquan Chen
Electrical and Computer Engineering Dalhousie University
Electronic Engineering
Systems and Computer Engineering Carleton University
Electrical and Computer Engineering Utah State University
Ottawa, Canada
[email protected]
Logan, Utah USA
[email protected]
Halifax, Canada
[email protected]
The Chinese University of Hong Kong Shatin, N.T. Hong Kong
[email protected]
Abstract -- This paper describes an Internet based wireless mobile robot. Using Internet, a user can command a wireless mobile robot in a remote place. The whole system consists of a mobile robot, a server workstation, radio frequency communication circuit and host software. The user can control a mobile robot through a friendly user interface, and receive the real-time feedback through a camera server. Details of the experimental set up and the implementation of the project are given and future directions are included.
Section II gives the architecture of the whole system as an overview. Section III describes the details of implementation, including software and hardware design. Experiment results are given in section IV and section V concludes this paper with conclusions and future works.
II. SYSTEM ARCHITECTURE Using Internet, a user can command a wireless mobile robot in a remote place. The whole system consists of a mobile robot, a server workstation, and radio frequency communication circuit and host software. The user can control a mobile robot through a friendly user interface, and receive the real-time feedback through a camera server. The system consists five major components
I. INTRODUCITON Today’s mobile robotic systems have been applied extensively in various areas such as delivery, robotics assembly and manufacturing, intelligent vehicles, robotics for survey and inspection, mining automation, military reconnaissance, underwater inspection, agriculture and forestry, entertainment, clearing robots, etc. For example, Pyxis HelpMate is a trackless, robotic courier system designed to perform material transport tasks throughout the hospital environment. Twenty-four hours a day, 365 days a year [1]. An autonomous driving system (NAVLAB) in CMU can drive autonomously on the highway for 6000 miles [2]. Nowadays various tele-operated mobile robots have been designed to perform remote sensing, space exploration, and hostile environment inspection [3-8]. Ren C. Luo et al constructed a remote supervisory control architecture, which combines computer network and autonomous mobile robot [3-6]. Tracy Engwirda et al facilitated cooperative behaviour remotely of an Internet mobile robot [7]. An interesting web-robot has been designed for a time critical task by Kensuke Tanaka [8]. A human robot interface has been designed in Simon Hartel’s paper [9]. In [10], Tsuyoshi Suzuki et al developed a human machine interface for multi-robot teleoperation using the Internet.
1. 2. 3. 4. 5.
Robot Program RF communication system Main host control program Camera Server Java Applets for Clients
Figure 1 shows the top-level description of the project, showing these components are connected together: Java Applet Control and Monitor Interface control
status
feedback
Internet
control
status
Camera Server
Host Software
Micro processor
The purpose of this paper is to design and implement an Internet controlled wireless robot. In our approach, our mobile robot linked to the host of the computer through a wireless channel, which makes it more reasonable, since it is not convenient to have a cable to connect the computer and the robot. The paper is organized as follows:
RF Robot
Fig. 1: System Overview
549
III. IMPLEMENTATION
C. Software Design
In this session, details of implementation are described. The mobile platform is introduced first. Then both software and hardware design of the RF receiver and transmitter are presented. Main control program of the system is given in detail. Also the java applet program for clients, camera server and client, and queue software are covered.
The design of the robot program must be done very carefully, because any of the five main tasks mentioned in section 2, have to be serviced immediately and especially in case of robot program and main host control program, if not being serviced immediately there is a big chance of losing the useful/critical information. Therefore, the program is designed to be fully interrupt driven, and implements multitasking environment for critical processes. The robot software is broken into three main parts, which have been tested and successfully installed.
A. Mobile Robot Platform
1.
Receive RF Data: Get commands/information from RF receiver
RF receiver circuit signals the HC11 processor that a complete data is available to be received. Four data bits of decoder holding the received data are connected to the Port of HC11. Received Interrupt Service routine (ISR) is then called on rising edge of VT. In ISR, the data is read simply by read and then queued for further processing in the main program.
Fig. 2: Tarik Mobile Robot The mobile robot used in this paper is called TalrikIITM. It offers a sophisticated, expandable, programmable, autonomous, mobile robot. TalrikIITM fits into a right circular cylinder of 10-inch diameter and 10 inch height. It rides on two 3-inch Du-Bro wheels and a rear caster. Two high quality servos, hacked as gear head motors and mounted underneath the platform, drive these wheels. It has twelve IR emitters, 12 IR detectors, 10 bump switches, 3 CdS lookdown "nose" light sensors, and 3 CdS scanning light sensors.
2.
Control the Robot: Control the robot according to the received commands
Controlling the robot can be done in the main function of the program by dequeuing the RF received data, and acting according to the received data. There are several ways to deal with the received data. E.g., using several if statements (or switch/case statement) or using table driven programming. The second option is chosen since it is much faster and occupied less code memory.
B. Maze Design
3. The environment of the Tarik mobile robot is a reconfigurable maze. A maze is custom designed to test the robot’s intelligence and functionality. A snap shot of the maze is shown in the following figure. Except for the borders, all the building blocks of the maze are removable and they are sitting on top of pressure sensors, which are connected to micro-controller. Whenever the configuration of the maze is changed, the pressure sensor information will be reported to the micro controller.
RF transmission: Transmit the sensor information to the Host using RF.
Transmission of the sensor data to the host computer can be done by dequeuing the information from the sensors and transmitting the data through the ports to the encoder. But one problem that is quite possible to occur is the interference between the transmitters, and receivers’ signals. This is due to the fact that the same Carrier frequency for both sets of receivers and transmitters are used. Thus, a protocol can be developed to handle this problem. Protocol works in the following manner that each side will transmits a FREE_TOKEN command after completing sending its data
D. Hardware Design 1. Fig. 3: Re-configurable Maze
550
RF Transmitters and Receivers
RF Transmitters and Receivers (AMRT5-315, AMRR315) are used to establish communication between robot and the Host computer. They operate at frequency of 315MHz. They are able to transmit data at up to 4 KHz. Data can be supplied directly from a microprocessor or an encoder.
On the other hand, RF transmission is even simpler. Figure 5, shows the circuit diagram of the transmitter side. Whenever logic ‘0’ is supplied to !TE then the encoder will transmit the data connected to its A8-A11 pins. Therefore, as shown in circuit diagram, PD0-PD4 need to be connected to data bits of encoder, and PD4 to !TE.
Encoders and decoders are used along with the Transmitters and receivers. This has several advantages; 1. The transmitted data is checked three times in the decoder (that is connected to the receiver) for valid data verification. 2. And also encoders and decoders as their names imply, they add an 8-bit address to the sent data, and only if the address selected on the decoder side matches that of encoder, data will be received. HT12E encoder chip is chosen, which consumes only about 0.1 uA in Standby mode, therefore, it can last with minimum power consumption. Both encoders and decoders sample eight lines (A0-A7), which act as device address inputs. Therefore, an encoder/decoder pair can be set to operate at one of 256 different addresses. The data packet sent by the encoder consists of the 8-bit address followed by a 4-bit data field corresponding to the state of up to four switches connected to inputs D8-D11.
Fig. 5: Transmitter’s Circuit Diagram
E. Main Control Program on Host PC Host software is the main control program of the system. The main control program will communicate with the robot using an RF circuit (connected to serial port of the Host PC). Therefore, the Main Control Program tasks can be categorized into the followings:
Unfortunately, there was no simple way to communicate between the Host computer’s serial port and the RF Transmitter and Receiver, therefore an Atmel AT8353 microcontroller is used for this purpose. Atmel Microprocessor works as a gateway between Host computer and the Robot. It gets the command from the host computer and sends it to Robot, and on the other side it waits for receiving data from Robot and sends it to the Host computer.
• • • • •
1.
Develop a graphical user interface environment Create a multi-threaded server that waits for clients over the internet Manage the clients in a First In First Out (FIFO) fashion Direct the robot to move to client’s desired direction Get the sensor information from the Robot to monitor its movement. Graphical User Interface Environment
It is very advantageous to develop graphical user interface (GUI) for the host software, because the host software will eventually be the core of this project, and many supporting tools can be added to it, to enhance the process of control and monitoring the robot. Glade development environment, which uses GTK+ as its bases, is used to create user interface. It will create the source code for the developed GUI using its powerful, and user-friendly tool kits. The programmer will then need to open the source code to modify and add functionally to UIs and events of the program.
Fig. 4: Receiver’s Circuit Diagram Figure 4 shows the circuit diagram of the receiver side. As it can be seen DOUT pin of the receiver is connected to Din of the Decoder (HT12D) where data is decoded and sent in the parallel fashion to the Atmel Microprocessor. One useful feature of the HT12D decoder is that it gives us Valid Transmission line that is set whenever a valid data is received by the decoder. Thus the VT line is connected to PD0 of the Atmel microprocessor that is interrupt driven and set up an Interrupt Service Routine (ISR) to be called on the rising edge of VT line. In the event of interrupt, the first four bits of PORTB is read, and then the message is en-queued. The main program then only will have to check the queue for new messages.
2.
Multi-threaded Server
There are several reasons why a multi-threaded server is needed: 1.When writing a server using sockets, there is a stage where accept function is called to accept an incoming call from client. Thus, if the accept function is called in
551
normal way. The whole program will be blocked and wait for a connection. Basically, the program will be busy-waiting for a connection. This is especially a problem when a server runs in a window environment, since nothing can be done when waiting for the connection.
2. and simply use read(), write() or other similar functions to read and write to the port through its file descriptor. 4.
The web-control of the robot was originally designed to be implemented using Perl/Cgi scripting. But the problem rises when two or more clients want to control the robot; the server has to be aware of state of all of its clients. Cgi/Perl scripts are called upon an action perform (or using a timer, by Javascript), which doesn’t result in an efficient way of handling this problem. It was much easier to monitor state of clients using Java Applets. Therefore, a Java Applet is designed to control the number of users who want to access the robot, and only allow one in. Java Applets and GUIs are simple to develop, and provide powerful libraries for developer.
2. The other reason is that this design forces continuous communication between clients and server, so that server is constantly aware of the state of its clients, and clients are informed immediately that access to the robot is allowed. There are at least two ways to create a Multithreads programs in Linux; one is the old traditional way POSIX's threads. The other way, is unique to Linux and is by using "clone"ing. In terms of usage, "clone" is very similar to pthread_create. Thus, to create a "clone", a function will be first defined to run concurrently with the parent program. Therefore, a process is created first to take care of clients. The process then creates a thread for each client that wants to access the robot. Every thread created, also en-queues itself –new client- in Userlist queue. Userlist is shared memory among all threads; therefore, accessing it, is a critical region that must be protected. Thus, a semaphore is created to protected the en-queuing and de-queuing. The first user will immediately be given the access to the robot (i.e. becomes the active user), the rest will be queued until their time arrives. Whenever the active user completes its task, it will automatically de-queue itself from the queue, and the next thread (in charge of next client) will be granted the access to the robot. Structure of the main host software is described in figure 6.
Java applets unlike CGI or Perl don’t run on server; they run on the client’s computer using their Brower. Thus, there is no way that a Java Applet (client) can find out by itself, how many clients are trying to gain access to the control page. Thus, a Java Applet is created that works as a client program, which talks to the main control program running on the server, which was described in details in previous subsection. Java Applet first asks for number of clients in the system. If nobody except one user is inside, then he is given the permission to control the robot. If another client tries to gain access, (it is queued in the main control program) it receives the number of clients in the system, and it will wait for a Ready command from the server. When the first client completes his task, then the next client will be informed, and automatically transferred to the control page.
Atmel Microcontroller RS232
The control is turned to control Java Applet whenever the access is granted by the server. The control Applet mainly sends different commands depending on what button is clicked.
Main Host Software
Wait for Java Clients (Child Process)
Serve the Client #N
Serve the Client #1
Java Client #N
Java Client #1
Java Applet Program for Clients
Fig. 6: Structure of Multi-threaded Server 3.
Serial Communication Microprocessor
with
Atmel
Last task of the host software is to send the clients’ received commands to Atmel via serial port. There are two ways to access serial port in Linux 1. Use file descriptor 2. Direct access to serial port. The most common way of accessing I/O ports is through a File descriptor. Unix-like systems allow the programmer to communicate with an I/O port through a file descriptor. Thus, the user has to: 1. open the port (i.e. create the file descriptor that is used to communicate with the port.) with the proper flag.
Fig. 7: Snapshot of Control Java Applet 5.
Camera Server and Client
Another part of the task was to visually monitor the movement of the robot by providing streaming video over the Internet. For this part a QuickCam Express is purchased (by Logitech), which was both cheap and more importantly compatible with Linux operating system. But the next important step was to send the real time images to a
552
Webpage so that users can control the robot. A GNU licensed program is found, to achieve this task; Camserv, is the program in which takes the image from the Webcam and serves the client with images in two ways: 1. Real-time images (JPEGs), served at port 91921. 2. Is by creating Single frame images. If browser was capable of handling raw JPEGs, then just use the port directly by saying http://domainname.domain:9192. Since Internet Explorer (IE) wasn’t. A simple JavaScript program has to be created using Camserv documentation, to get the constructed single frame images, and load it to the web page, and the keep refreshing it. There were several problems with this method. 1. The images were delayed for at least 2-3 seconds, which was a big disadvantage 2. Refreshing the images in JavaScript was causing the cursor to change periodically, which could have been irritating to the clients. Fortunately, there was a GNU licensed Java Applet that was written for this purpose. Only with few small modifications needs to be made for this project. Another big advantage this Java Applet was that it used the JPEG raw data to construct the images not the single frame images, which made the images almost real time with the least delay possible. Figure 8 gives a snap short of the webcamera interface, where the image is displayed and updated continuously.
The other problem that needs to be addressed is the possibility of being interrupted while incrementing tail. To overcome this problem, interrupts are disabled before critical region (i.e. incrementing tail) inside de-queue, this will insure that that critical region is uninterrupted.
IV. EXPERIMENTS Following sequential tasks have been tried to carry out the experimental studies: I. Mobile robot testing: Talrik mobile robot has been fully tested. Motions of the robot such as move forward, move backward, rotate clockwise and count clockwise have all been tested successfully. II. RF transmitter and receiver testing: Strings have been sent out of the transmitter and the received strings have been verified to match the sent out string. III. Host software testing: The communication between the host software and the microprocessor has been fully tested such that the host software can control the mobile robot wirelessly. IV.
Java Applets and Clients Interface: Communication between the client and the host software has been set up such that the client can talk to the server anywhere to control the robot remotely.
V. Web-camera display: as explained in section 3.5, web camera has been tested and worked properly. With all the components of the system working, the whole system has been tested successfully. The user can use any networked computer to control the mobile robot remotely. Figure 9-11 shows some snap shots of the interface, web-camera picture. A remote user has controlled the mobile robot to move from one corner of the maze to the other side of the corner.
Fig. 8: Snap-shot of Web-image 6.
Queue Software
In case of multiple users, the server will only allow one entry and the later user will be put to the wait list. When the user who is controlling the robot log out, the first one on the wait list get access to the system to control the mobile robot.
Queue software has been designed to store the packets received by RF interrupt service routines. Basic structure of the queue software was used in Atmel, HC11, and also host software. To construct queues, two indexes, head and tail have been used. Head is index of the character last enqueued, and tail is index of the character that is to be dequeued next. But the other problem to be addressed is how large should be the size of the array. It is impossible to have an infinite queue, because of memory limit. Therefore, circular queue is implemented. Circular queue, works in the following manner; it queues the data and increments the head, till head of the queue reaches the maximum size specified, it then resets head back to zero.
1
This port is static and defined in the camserv configuration file.
Fig. 9: Remote Control of Wireless Robot (Snapshot 1)
553
the ordinary users (or even clients over the internet) can test their own maze surfing algorithm on the robot, without having to recompile the code in to the system.
VI. REFERENCE [1] http://www.pyxis.com/products/newhelpmate.asp [2] Charles E. Thorpe(ed.) Vision and Navigation: The Carnegie Mellon Navlab. Kluwer Academic Publisher, Boston, MA, 1990 [3] Luo, R.C., Su, K.L., Jyh Hwa Tzou and Phang, S.H.H, “Multis ensor based control of pet robot through the Internet”, The 27th Annual Conference of the Industrial Electronics Society, IECON '01,Vol. 1, pp.416-421, 2001 [4] Luo, R.C. and Tse Min Chen, “Development of a multi-behavior based mobile robot for remote supervis ory control through the Internet” IEEE/ASME Transactions on Mechatronics, Vol. 5, No. 4, pp.376– 385, 2000. [5] Luo, R.C., Tse Min Chen and Chih-Chen Yih, “Intelligent autonomous mobile robot control through the Internet” Proceedings of the 2000 IEEE International Symposium on Industrial Electronics, pp.6-11, 2000. [6] Luo, R.C. and Tse Min Chen, “Remote supervisory control of a sensor based mobile robot via Internet”; Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS '97,Vol. 2, 7-11 Sept., pp.1163-1168. [7] Engwirda, T., Engwirda, A., Vlacic, L. and Dromey, G.,“Sharing cooperative mobile robots through the internet” Proceedings of IEEE International Conference on Industrial Technology 2000, pp.460– 465. [8] Tanaka, K.; Nakagawa, E., Ito, M., Mizuno, N., Yamada, T., Shimizu, E. and Kagayama, K.,“An Internet-based tele-robot environment for a time critical task”, 1999 IEEE International Conference on Systems, Man, and Cybernetics, SMC '99, pp.11061110. [9] Hartfiel, S. and Dunn, L.; “Global telerobotics: exploring effective Internet access to robots”, Sixth Australian Conference on Computer-Human Interaction, 24-27 Nov., pp. 326–327,1996. [10] Suzuki, T., Fujii, T., Yokota, K., Asama, H.; Kaetsu, H. and Endo, I., “Teleoperation of multiple robots through the Internet”, 5th IEEE International Workshop on Robot and Human Communication, 1114 Nov., pp. 84-89, 1996.
Fig. 10: Remote Control of Wireless Robot (Snapshot 2)
Fig. 11: Remote Control of Wireless Robot (Snapshot 3)
V. CONCLUSION AND FUTURE WORKS The main purpose of this paper was to design a robust system to control a Talrik robot over the Internet. A simple control of the robot over the Internet was implemented and successfully tested. As stated in each section of this paper, all the components of the system were designed to be flexible and extendible for future purposes: 1. Robot program is designed to be interrupt-driven software, that additional programs can be simply added to the main function of the code. 2. Atmel (RF Gateway) program was programmed in an interrupt driven fashion 3. Main Host program was designed as a multi-threaded program, so that additional features can simply be added to the program. 4. Finally, Java Applets in conjunction with multi-threaded server are designed in a way that with simple modification the program can even turn to a chat room among the clients of the Talrik! There are probably many extensions that could be added to this project. Several interesting future works are mentioned and discussed here. One of the most interesting that can be done now is to design an intelligent robot by adding some artificial intelligence to the Talrik. For this purpose, the robot has to obtain accurate movement data. The data is then to be sent to the Host PC. At this point the only thing is left, is to develop the required software to command the robot to move from a source to its destination. There are several algorithm developed so far for maze surfing purposes; First those algorithms can be implemented. Then an interpreter can be developed so that
554