Sh Kim

A Robust License-Plate Extraction Method under Complex Image Conditions Sunghoon Kim, Daechul Kim, Younbok Ryu, and Gyeonghwan Kim Dept. of Electronic Engineering Sogang University CPO Box 1142, Seoul 100-611, Korea
doncopy, s950812, roy21, gkim@sogang.ac.kr Abstract A robust approach for extracting car license plate from images with complex background and relatively poor quality is presented in this paper. The approach focuses on dealing with images taken under weak lighting condition. The proposed method is divided into two steps: 1) searching candidate areas from the input image using gradient information, and 2) determining the plate area among the candidates and adjusting the boundary of the area by introducing a plate template. A set of experiments has been performed to prove the robustness and accuracy of the approach. For many images collected from a large underground parking place the result shows that 90% of them are correctly segmented.

1. Introduction As the ITS(Intelligent Transport Systems) becomes getting bigger attention, automatic identification of a vehicle has turned out to be an important research issue [5]. In general, there are two ways in identifying a vehicle: one approach is to use of electronic devices to identify each individual car. The RF ID tag, which has been attached to a car for the purpose, is a popular example. The unique information from the tag, when microwave signal arrives from an external antenna, is decoded and used for identifying the car. Another way is to retrieve the information on the license plate. In the approach, a camera captures the image of a car and a computer processes the image and recognizes the information on the plate by applying various image processing and optical pattern recognition techniques [6]. Recognition-based approaches reported in previous research generally consist of several procedures, including extracting a license plate region, segmenting characters from the region, and recognizing each character. Among the procedures, accurate extraction of the license plate region from a variable of scenes is of crucial importance because it di-

rectly affects system’s overall accuracy. To make the extraction process successful, many difficult problems should be dealt with, such as poor image quality due to various ambient lighting conditions and image distortion stems mainly from various combination of visual angles between the camera and the car. The image capturing process is largely depending on several aspects, including camera angle, dynamic range, ambient lighting condition, background complexity, existence of reflection and plate fracture [2]. Therefore, it is necessary to make the extraction process robust so that it can work on any situation in the real world environment. For extracting the plate region, techniques such as edge extraction [1][7], Hough transform [8], histogram analysis [4] and morphological operators [10] have been applied. An edge-based approach is normally simple and fast. However, it is too sensitive to the unwanted edges, which may happen to appear in the front part of a car. Therefore, this method cannot be used independently. Hough transform for line detection gives positive effect on images with a large plate region where it can be assumed the shape of license plate is defined by lines. However, it needs a large memory space and considerable amount of computing time. The histogram based approach does not working properly on the image with noises and the image with tilted plate. Morphology has been known to be strong to noise signals, but it is rarely used in real time system because of its slow operation. The approach presented in this paper is to extract the region of the license plate from images taken from indoor parking lots, which suffer from various real world problems. We have encountered many images with various problems, including poor ambient lighting, diverse plate locations, tilt of the plate, and influence from strong light beams. The proposed approach consists of two parts: 1) searching candidate areas using a combination of gradient features obtained from the input image, and 2) determining the plate area among the candidates and adjusting the boundary of the area by introducing a plate template.

1051-4651/02 $17.00 (c) 2002 IEEE

Figure 2. The variance is plotted based on
for a plate image

Figure 1. The overview of the algorithm

2. Outline of the approach A two-stage search process is proposed in the approach: the global search in which candidates of the plate region are sought using the combination of the gradient features, and the local search where the correct plate region is selected from the candidates and the boundary of the region is adjusted using a plate template. Details on each step shown in Figure 1 are described in this section.

2.1 Global search In the beginning part of the stage, histogram stretching is performed as a pre-processing, using Eq.(1), in order to increase the contrast of the image and protect the vital edges from subsequent operations.

  



(1)

where,  represents the intensity value of the input image,  is the variance of the image, and  is a constant (set to 30 in this case). Step 1: Obtaining three statistical features Conventional approaches, which rely only on the variance of the gradient while ROIs being searched, could result in limited success when the image under consideration contains a tilted plate and/or complicated background. Therefore three statistical features are introduced in the proposed approach and combined by a simple neural network.

¯ Feature1 - the gradient variance: The gradient of the image is obtained using the Sobel operator. Based on the observation that gray scale changes are more frequent in the plate area, local variances are obtained

Figure 3. Feature images formed using  (left) and  (right) for two plate images from windows of size 1¢9 pixels using Eq.(2). The window size is determined experimentally. Figure 2 shows the changes of variance in a plate image.

  
   










 






  





(2)

where  represents the gradient at
  and  is the number of pixels involved while the variance is obtained.

¯ Feature2 - the density of edges: Plate regions tend to have a high density of edges. The density is measured in a block of size 1¢9 pixels by summing all edge pixels using Eq.(3). 
  




    
  


 

(3)

where 
  represents the edge magnitude at
 , and  is the number of pixels involved while the feature is obtained. Bottom left image in Figure 3 is formed using the feature described above for the upper left image in the figure.

1051-4651/02 $17.00 (c) 2002 IEEE

¯ Feature3 - the density variance: The feature is based on the concept that if a candidate area indeed contains a plate, the foreground pixels are distributed evenly comparing to the areas with simple structures [3]. Therefore, this feature can be used to discriminate text regions from background regions. To obtain the feature, a block of size 9¢9, centered at
 , is divided into nine equal-sized sub-blocks. For each sub-block

, let  denote the total number of edge points in the sub-block. Then the feature, the density variance, is defined by Eq.(4).

¿
  

 




 

(4)



where   is the average of   s. Bottom right image in Figure 3 is formed using the feature described above for the upper right image in the figure. Those three features are combined with a neural network to determine if a particular pixel under examination could belong to the plate region. To train the network, around 300,000 patterns were collected from 50 images. The output of the network forms a map, which is called the feature map hereafter. Figure 5. The search procedure

Step 2: Finding candidate areas In the feature map, areas with higher variance are grouped and labeled using chaincode-based contour follower [9]. While the processing is being performed, careful examination is carried out to apply certain limitation on the size of candidates. The examination prevents from creating candidates with too big or small aspect ratio comparing to that of the license plate.

After the merge is completed, the boxes with both the aspect ratio of less than 1/2 and horizontal length of less than 200 pixels are used for the next step. A region of interest is determined by another back-propagation neural network which inputs box diagonal indices, average value of intensities and box filling ratio.

Step 3: Selecting an ROI

2.2 Local search

Candidate areas are represented by bounding boxes and a plate region may consist of one or more than one boxes as shown in Figure 4(left). To merge the boxes, a simple neural network is employed and it determines whether two boxes under examination need to be merged or not. As inputs to the neural network, corner points of the boxes, aspect ratios, and box filling ratios are provided.

Once an ROI is selected, it is necessary to make sure that the box contains the plate. The similarity transform is performed to the plate template so that it fits into the ROI selected. Then, projection profiles on both directions are examined to judge whether the ROI contains the plate or not. When the parameters collected from the projection profiles satisfy the conditions shown in Figure 5, the ROI is regarded as the correct plate region. If the ROI does not satisfy either of the conditions, the process is repeated until the selected ROIs are consumed. Figure 6 illustrates intermediate results of some steps explained in this section for an input image.

3 Experiments Figure 4. Merging bounding boxes: before (left) and after (right)

Many number of images with various types of vehicles, not only passenger cars but also vans and trucks, were col-

1051-4651/02 $17.00 (c) 2002 IEEE

(a)

(b) Figure 7. The gradient variance method (left) and the proposed method (right)

(c)

(d)

(e)

(f)

whether it contains the license plate by introducing a template of the license plate. Based on the experimental result, the performance of the proposed approach is promising. Comparing to the conventional edge-based approach, the overall processing speed is a little bit slower, but the approach is more robust in terms of accuracy. Ways to make the approach more robust and faster are being sought by combining a couple of techniques, including the active contour algorithm.

Figure 6. Processing steps: (a) an input image, (b) the feature map, (c) bounding boxes, (d) selected ROI, (e) initial setting for the local search, and (f) the local search result

lected from a large indoor parking place with different angles and different lightening conditions. The image acquisition system consists of a CCD camera, a photo sensor to detect presence of a car, and a personal computer with a image grabber in it. Size of the image is 640492. 1,000 of them were used for the evaluation purpose. The experimental result shows that the proposed algorithm locates correctly the plate area for 90% of the images used. Sources of the failure can be classified into three major categories: 1) existence of other text blocks, 2) bounding box containing the plate is merged into other box(es), and (3) weak gradient information from the plate area. As shown in Figure 7, by combining the features using the neural network the system becomes more robust against the noise and tilt comparing to the other method [5].

4 Summary and Conclusion A robust approach for extracting license plate from a car image is presented in this paper. The two-step approach is designed to deal with images taken under various real world conditions. In the first step, the whole image is searched and candidate areas are located based on the gradient features. In the second stage, the candidates are examined to verify

References [1] D. H. Ballard. Computer Vision. Prentice-Hall Inc., 1991. [2] M. W. Burke. Image acquisition. Chapman & Hall, 1996. [3] W.-Y. Chen and S.-Y. Chen. Adaptive page segmentation for color technical journals’ cover images. Image and Vision Computing, 16(12-13):855–877, 1998. [4] D. U. Cho and Y. H. Cho. Implementation of preprocessing independent of environment and recognition of car number plate using histogram and template matching. The Journal of the Korean Institute of Communication Sciences, 23(1):94– 100, 1998. [5] D. Gao and J. Zhou. Car License Plate Detection from Complex Scene. In Proceedings of International Conference on Signal Processing, pages 1409–1414, 2000. [6] P. Hu, Y. Zhao, J. Zhu, and J. Wang. An Effective Automatic License Plate Recognition System. In The proceedings of CISST2000, pages 80–84, 2000. [7] K. Kanayama, Y. Fujikawa, K. Fujimoto, and M. Horino. Development of vehicle-license number recognition system using real-time image processing and its application to travel-time measurement. In Proceedings of IEEE Vehicular Technology Conference, pages 798–804, 1991. [8] K. M. Kim, B. J. Lee, K. Lyou, and G. T. Park. The automatic recognition of the plate of vehicle using the correlation coefficient and Hough transform. Journal of Control, Automation and System Engineering, 3(5):511–519, 1997. [9] A. Rosenfeld and A. C. Kak. Digital Picture Processing. Academic Press, 1982. [10] M. Shridhar, J. W. Miller, G. Houle, and L. Bijnagte. Recognition of license plate images: Issues and perspectives. In Proceedings of International Conference on Document Analysis and Recognition, pages 17–20, 1999.

1051-4651/02 $17.00 (c) 2002 IEEE