Self Driving Cars In India

Implementing Self Driving Cars in India Shlok Khemani

Vidit Kothari

Shamoil Loliwala

Department of Computers

Department of Computers

Department of Computers

Mukesh Patel School of Technology, Management and Engineering (NMIMS) Mumbai, India

Mukesh Patel School of Technology, Management and Engineering (NMIMS) Mumbai, India

Mukesh Patel School of Technology, Management and Engineering (NMIMS) Mumbai, India

[email protected]

[email protected]

[email protected]

Abstract—Throughout the globe, there is an increasing interest in the deployment of self-driving (autonomous) vehicles, with the aim that they would eventually replace human-controlled vehicles. Unlike in Western countries, where they have hit the road in some form or the other, autonomous vehicles are rarely talked about, let alone implemented, in India. In this paper we have looked at various technical obstacles faced in India before self-driving cars can be implemented and how to overcome them.

I. INTRODUCTION Around the world, the interest, research and deployment of autonomous vehicles has significantly increased in the past decade. Global tech leaders like Google, Microsoft and Apple are all conducting research in the field. The advantages of autonomous vehicles are numerous. Perhaps the most significant one being increased on-road safety. As a society, we currently rely heavily on the driving ability of the individual: his or her experience, mental state, temperament and sobriety. This dependence is reflected in accident statistics: around 90% of road accidents are caused at least partially by human error [1]. This problem is even more relevant in India, where the process of obtaining a driving license is comparatively relaxed and often questionable. This, coupled with the burgeoning population, increasing number of vehicles on the road and the high number of unpaved roads in the country makes India one of the most dangerous countries in the world for drivers and pedestrians. Over 137,000 people were killed in road accidents in 2013 alone [2]. Autonomous cars are operated by a computer. Computers make decisions objectively. They aren’t influenced by the factors mentioned above, which often lead to accidents. This is why investing in the development of autonomous vehicles is potentially so rewarding. Autonomous cars make driving, which is one of the most quintessential parts of modern human life, a much safer experience for all those involved in it, directly and indirectly. Governments spend huge amounts of money to protect and monitor vehicular traffic. With the implementation of self-driving cars, this money can be well spent elsewhere. India as a country is lagging behind compared to the rest of the world in the research and implementation of selfdriving cars. This is in part due to government reservations on the technology and potential loss of livelihood of drivers [3]. However, autonomous vehicles are the future, India will eventually have to adopt it and we need to be prepared to adapt when the time comes.

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

India is a vast country. In its expanse, it covers every possible type of terrain and roadway. Like various other Asian countries, India faces some unique problems in the implementation of self-driving cars. These concerns are very different from those faced in Western countries, where autonomous vehicles have been extensively tested. Some of these concerns include the absence of proper road signs, traffic lights and lane markings, a high number of unexpected obstacles, unpaved highways and dirt roads and the absence of a network of extensive highways. We have referenced the work done by researchers in Pakistan [4] and Bangladesh [5], countries which are geographically and demographically similar to India, to find solutions for the implementation of autonomous vehicles in India. We have worked towards finding solutions for five major concerns in the implementation of self-driving cars in India: traffic light and road sign detection, stationary and unexpected obstacle detection, navigation techniques, traffic scenarios (changing lanes, driving in congestion) and controlled parking. II. TRAFFIC LIGHT AND ROAD SIGN DETECTION A. Traffic Light Detection Our roads are filled with traffic lights. They are a part and parcel of road safety. To detect and follow these lights is of paramount importance in an autonomous vehicle. In the paper [7], the researchers have enabled an autonomous vehicle to successfully navigate multiple intersections in heavy traffic, while correctly detecting and following the lights. To detect a single traffic light, the following steps are followed: 1. The system is supplied with a predefined list of locations of traffic lights. 2. The system, through cameras, begins looking for lights according to the resolution of the camera and a dynamic distance accounting for braking. 3. The expected location of the traffic light is focussed onto a region of interest. 4. A template matching algorithm is applied to the region of interest. 5. The result is passed through a histogram filter and blurred, to remove uncertainty (possibility of a light source being elsewhere). 6. The most prominent cell is detected from the filter and the corresponding coordinates are transmitted. 7. The light’s colour, red, green or yellow, is reported based on the results from the filter.

B. Road Sign Detection Sign detection, like traffic light detection, is one of the more significant actions an autonomous vehicle needs to perform to ensure safe navigation. In [7], researchers have used a laser-based sign detector to locate the position and orientation of signs encountered by the vehicle. A supervised learning method (SVM) is trained to detect signs. A sliding window detector with a fixed square size is used for sign detection. After using non-maximal suppression algorithms, a weighted voting method and accumulation and projection on a RANSAC plane, we are able to detect the type of sign and whether the vehicle needs to stop or not. This method has yielded a success rate of 89%, significantly reducing the human effort required to manually annotate a map. However, in India, traffic is still being controlled manually in many areas. So as suggested by [5] an image recognition software has to be developed which can identify the signs and signals given by traffic policemen and effectively make an autonomous car India friendly. III. STATIONARY AND UNEXPECTED OBSTACLE DETECTION The detection of small-sized and unexpected obstacles can cause an alarming situation for cars on the road as it could lead to severe accidents. In countries like India where there is litter and potholes on the road, it is important that we need to be able to detect this accurately to prevent mishaps. This is one major aspect that needs to be considered in the development of self-driving cars which are going to become road legal soon. Hence we need accurate results. The researchers in the paper [6] have observed that in the previously used technology the methods that tackled this problem made use of stereo cameras so as to use geometric means to detect and locate small 3D obstacles. As we require accurate results in our line of research we would need to use machine learning techniques so as to supplement the geometrical means that are already being used. The advancements in artificial intelligence has given a push to deep learning methods, in the field of computer vision, deep Convolutional Neural Networks (CNN) has proved to be an excellent source for low level as well as high-level image processing. The contextual properties of the small unexpected obstacles can be linked with the properties of CNN so as to detect the obstacles irrespective of its shape and size. There would surely be drivable road area surrounding the obstacle hence making the maneuvering of the car also effective. This paper [6] makes use of this advancement in CNN technology in order to improve their accuracy in the detection of small and unexpected obstacles on the road so as to provide a safer environment for driving these cars. CNN has helped the researchers to supplement the already present geometrical methods in order to provide more accurate results as there is a larger data set from which obstacles can be detected and processed. Furthermore, the researchers in this paper have also started working on implementing a probabilistic fusion approach that integrates their research with the already existing best in class stereo-based systems.

In the paper [6], researchers have explored algorithms for the detection of unexpected obstacles, focusing on small road hazards like lost cargo. Since these objects cover only small areas on scanned images, traditional obstacle detection algorithms and techniques, which are suitable for large obstacles, prove inefficient. The researcher have used CNNs to learn context from training data and generalize information. After using multiple algorithms over the same data sets, researchers found the Fusion-Prob technique to be the most efficient in detecting small obstacles. The FusionProb technique uses information from the UON-Stixel and FPHT methods, combines it with Fusion-OR and FusionAND techniques to generate results. The results from each of these algorithms are given in Fig. 1. Researchers in [7] use depth information to conduct Velodyne scans. This removes the ground plane and the remaining points are clustered. After feeding this through a Kalman tracker and applying boosting classifiers, track classification is complete. The accuracy of the classifier in detecting obstacles such as pedestrians is 98% and the system runs in real time. This method of obstacle detection is highly reliable with one drawback which the researchers are working on: when obstacles overlap (a pedestrian crossing a pole), they aren’t detected as two separate obstacles.

Fig. 1. Quantitative results from applying various techniques on the data set

Researchers in [5] have suggested using an algorithm which is demonstrated in Fig. 2. This algorithm works well with the usual style of Indian driving where driving regulations are not as stringent as in the western countries. IV. PATTERN DETECTION AND NAVIGATION TECHNIQUES A. LIDAR Based Navigation In [7], researchers have developed a method for the calibration of multiple laser beams to obtain a map of the surroundings. This calibration is completely unsupervised. For intrinsic calibration, points are aggregated and iterative optimization methods are applied. Further performing extrinsic calibration, we recover the position within accuracy of 1cm. The final results are shown in Fig. 3. B. Path Learning through CNNs Since the advent of CNNs, instead of manually teaching vehicles paths, they use convolution operations to not only recognize patterns, but also learn the process pipeline

needed to navigate a motor vehicle in its entirety. This concept was described by researchers in [9]. The block diagram in Fig. 4. shows how the training system works. Images from various cameras on the vehicle are fed into the CNN. Once trained, the network, which has control of the steering, can steer the vehicle according to the data generated from the images.

When self-driving vehicles are implemented in India, the methods prescribed in this paper to overcome navigation problems will be most crucial. C. Trajectory Planning

If we are to implement self-driving cars in real time traffic, we have to consider the impact of other cars while driving. These include everyday driving operations like merging into traffic, changing lanes, avoiding collisions etc. Researchers in [7] use an algorithm that transfers distance and velocity control to the system and also actively avoids obstacles controlling the steering, acceleration and breaking. Researchers use the Frenet-Serret formulation and mimic human like driving behavior by combining horizontal (Eq. 2) and vertical (Eq. 3) cost functions. They seek to eliminate any kinds of jerks while changing trajectories, much like how humans operate vehicles. Lateral and longitudinal curves are then combined (Eq. 4). Fig. 2. Object Detection Algorithm for Indian Roads

D. Speed Adaptive, Ratio Based Lane Detection System

Fig. 3. We achieve a very accurate calibration after optimization

The training data for teaching the CNN was collected by driving on a variety of roads, in various weather conditions. This is very important for a country like India, where climate varies significantly and weather conditions are very unpredictable (especially in the tropical parts of the country). The data also included roads lacking lane markings, another distinctive characteristic of Indian roads. Furthermore, the amount training data was relatively small and the vehicle operated in multiple diverse conditions.

As the lane detection and lane keeping of a car is done by cameras, a huge amount of reliability of the self-driving car is dependent on the frame rate of the camera. Considering that at high speeds, a particular camera’s frame rate induces a delay in giving the next frame to the processor. This small delay can lead to a huge accident in Indian conditions. Thus, researchers in [10] have developed an algorithm where they can predict the next frame with the help of the current frame and velocity of the car. Using equation 5 and 6, one can plot a graph of where the car will be in the next frame and find out the ratio of the distance from the left lane to the distance of the right lane and accordingly navigate through to keep the car in the lane. (5)

(6)

Fig. 5. shows how this algorithm was tested in 3 different combinations of speed and frame rate of camera. The horizontal yellow line is the place where the car will next be and the vertical blue line is how the car will continue to drive on the current trajectory. Fig. 4. Training the Neural Network.

The algorithm now uses an algorithm similar to Dijkstra's algorithm to find the nearest available parking slot. It follows these two points: 1. Generate the shortest path to every P-node from every E-node and save them in a list such that every E-node has its own list of closest paths with the values sorted in ascending order of their cost. 2. Generate the shortest path to every E-node from every P-node and save them in a list such that every P-node has its own list of closest paths with the values sorted in ascending order of their cost.

Fig. 5. Predicting future frames

V. CONTROLLED PARKING As autonomous cars get increasingly popular, one has to devise algorithms to ensure smooth and safe parking. Which is the closest available parking? How to navigate to the empty spot? All these questions can be answered by the solution provided in [8] in which they used an Automated Parking System with a graphical approach to solve the parking problem. a.

The reason for generating these two sets of path separately is that a shortest path from a particular E-node to some P-node, while parking, may not be the shortest path while exiting, since it can have more options for E-nodes. c.

Experimental Analysis. Consider that the car does not use the Automated Parking System. As per Fig 4., let’s say that a car follows the unguided approach, i.e, it does not use the automated parking system and it finds that the parking slot 16 is empty. Now to reach the slot, it may have taken the path 0-3-2-7-12-17-16 or 0-34-9-14-19-18-17-16 to reach the slot.

Graph Generation: First, the parking slots (P-node), pathway (TNode), and entry/exit points(E-node) are mapped and converted into a graph. Fig. 6. Coverts to Fig. 7.

Fig. 6. The parking lot

Fig .8. The non-guided approach(left) has car 5 parked at P7 and guided approach has the car parked at P15.

In the second scenario, consider that it takes help from the automated parking system and the system finds the nearest empty slot and gives directions to the car. Initially, when the slots near the entrance are empty, both approaches are equally similar and simple, however, as the parking lot fills up, the guided approach becomes easier as shown below. VI. CONCLUSIONX

Fig. 7. The parking lot is converted to a graph with P-nodes in gray, T-nodes in white and E-nodes in Black.

b.

Nearest Slot Parking:

Even though the implementation of autonomous vehicles in India currently seems like a pipe dream, this paper goes on to explore how potential problems and difficulties can be overcome and solved. The algorithms described by researchers in [7] have generated high traffic light detection rates, as well as has had good success in detecting road signs. However, as

suggested by [5], hand gesture recognition software need to be developed. Researchers in [6] have used CNNs for the detection of unexpected small objects. In a country like India, this is very essential. Moreover, they have devised an extremely accurate algorithm which should be implemented in autonomous vehicles in India. In [7], researchers have used Velodyne scans to detect large obstacles like poles and pedestrians. They have achieved an accuracy of 98%, which is excellent. For the mapping of roads, researchers in [7] propose the use of LIDAR based technology. This mapping is completely unsupervised and generates an extremely accurate topological image of the surroundings in real time. In a country like India, where there are unexpected obstacles and changes in the vehicle surroundings, this kind of real time mapping is critical. In [9], researchers have used CNNs to train vehicles in navigation. With a relatively small amount of training data, the vehicle was able to completely navigate the path, in various weather conditions. This paper will be vital in training vehicles to drive in India, which is vast and has varying and often unpredictable weather. In [10], researchers have devised methods to predict future frames using information from the current frame. This helps a vehicle’s navigation remain stable. In [8], researchers have explored the domain of parking and how autonomous vehicles can most efficiently park themselves. Owing to the size, demography and diversity of the country, the implementation of autonomous vehicles will definitely be a huge task. However, we have tried to identify specific potential problems and used research done in the West, as well as by our neighboring countries, to look for solutions. As future work, we plan on collecting real time traffic footage to train an autonomous vehicle. We will also try to

administer various methods, already implemented in other countries, to a vehicle in Indian traffic. REFERENCES “Human error as a cause of vehicle crashes,” Center for Internet and Society, 30-Oct-1970. [Online]. Available: https://cyberlaw.stanford.edu/blog/2013/12/human-error-causevehicle-crashes. [Accessed: 21-Oct-2018]. [2] National Crime Records Bureau, Ministry of Road Transport & Highway, Law Commission of India, Global status report on road safety 2013. [3] D. K. Dash, “Driverless tech will leave millions without jobs, won't allow it: Gadkari - Times of India,” The Times of India, 25-Jul-2017. [Online]. Available: https://timesofindia.indiatimes.com/auto/cars/government-wontallow-driverless-cars-on-indian-roads-nitingadkari/articleshow/59741458.cms. [Accessed: 21-Oct-2018]. [4] Q. Memon , M. Ahmed, S. Ali, A. R. Memon, and W. Shah, “SelfDriving and Driver Relaxing Vehicle,” 2016 2nd International Conference on Robotics and Artificial Intelligence (ICRAI). [5] M. S. U. Miah, M. F. B. Ahmed, M. M. A. Timu, S. Akter, and M. J. Sarker, “The issues and the possible solutions for implementing selfdriving cars in bangladesh,” 2017 IEEE Region 10 Humanitarian Technology Conference (R10-HTC), 2017. [6] S. Ramos, S. Gehrig, P. Pinggera, U. Franke, and C. Rother, “Detecting unexpected obstacles for self-driving cars: Fusing deep learning and geometric modeling,” 2017 IEEE Intelligent Vehicles Symposium (IV), 2017. [7] J. Levinson, J. Askeland, J. Becker, J. Dolson, D. Held, S. Kammel, J. Z. Kolter, D. Langer, O. Pink, V. Pratt, M. Sokolsky, G. Stanek, D. Stavens, A. Teichman, M. Werling, and S. Thrun, “Towards fully autonomous driving: Systems and algorithms,” 2011 IEEE Intelligent Vehicles Symposium (IV), 2011. [8] S. Tariq, H. Choi, C. Wasiq, and H. Park, “Controlled parking for self-driving cars,” 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2016. [9] M. Bojarski , D. D. Testa, D. Dwarakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, Z. Zhang, X. Zhang, J. Zhao, and K. Zieba, “End to End Learning for Self Driving Cars.” [10] S. Kim, J. Lee, and Y. Kim, “Speed-adaptive ratio-based lane detection algorithm for self-driving vehicles,” 2016 International SoC Design Conference (ISOCC), 2016. [1]