Opencv Ing

Tutorial: OpenCV haartraining (Rapid Object Detection With A Casca...

1 di 8

http://note.sonots.com/SciSoftware/haartraining.html

Naotoshi Seo Tutorial: OpenCV haartraining (Rapid Object Detection With A Cascade of Boosted Classifiers Based on Haar-like Features) Table of Contents Objective Data Prepartion Create Samples (Increase Images) createsamples Create Description File EXTRA: random seed Create Training Samples Create Testing Samples Training haartraining Generate xml Testing performance facedetect Discussion Download References

Objective The OpenCV library gives us a greatly interesting demo for a face detection. Furthermore, it provides us programs (or functions) which they used to train classifiers for their face detection system (called HaarTraining) so that we can create our own object classifiers using these functions. It is interesting. However, I could not follow how OpenCV developers performed the haartrainig for their face detection system exactly because they did not provide us several information such as what images they used for training. The objective of this report is to provide step-by-step procedures for following people. My working environment is Visual Studio + cygwin on Windows XP, or on Linux. The cygwin is required because I use several Linux commands.

A picture from the OpenCV website

11/07/2008 12.54

Tutorial: OpenCV haartraining (Rapid Object Detection With A Casca...

2 di 8

http://note.sonots.com/SciSoftware/haartraining.html

First Edition: 03/12/2006. Last Modified: 06/05/2007. Tag: SciSoftware ComputerVision FaceDetection OpenCV

Data Prepartion Positive (Face) Images I downloaded from The UMIST Face Database. FYI: There are database lists on Face Recognition Homepage - Databases. and Computer Vision Test Images. Negative (Background) Images Kuranov et. al. [3] states as they used 3000 negative images. Fortunately, I found http://face.urtho.net/ (Negatives sets, Set 1 - Various negatives ) which has about 3500 images. But, this collection was used for eye detection, and includes some faces in some pictures. Therefore, I deleted all suspicious images which looked including faces. About 3000 images were remained, thus they should be enough. The collection is available at the Download section (But, it may take forever to download.)

Create Samples (Increase Images) We can create traning samples and testing samples with the createsamples software.

createsamples In this section, I describe functionalities of the createsamples software because the Tutorial [1] did not explain them clearly (but please see the Tutorial [1] also for further options). The createsamples software has mainly four functionalities. 1. Create training samples (a vec file) from one image applying distortions 2. Create training samples (a vec file) from some images without applying distortions 3. Create test samples (images) and their ground truth (a description file) from single image applying distortions 4. Show images within a vec file 1. Create training samples from one image applying distortions. This function is launched when options, -img, -bg, and -vec were specified. -img [a positive image] -bg [a description file of negatives] -vec [name of the output file containing the generated samples] For example, $ createsamples -img face.png -num 10 -bg negatives.dat -vec samples.vec -maxxangle 0.6 -ma

Only the first negative images in the <description file of negatives> are used. 2. Create training samples from some images without applying distortions You may think this function is a file format conversion function. This is launched when options, -info, and -vec were specified.

11/07/2008 12.54

Tutorial: OpenCV haartraining (Rapid Object Detection With A Casca...

3 di 8

http://note.sonots.com/SciSoftware/haartraining.html

-info [a description file of positives] -vec [name of the output file containing the generated samples] In this case, the option -num is used only to reduce the number of samples to generate, this does not generate many samples from one image applying distortions. For example, $ createsamples -info samples.dat -vec samples.vec -w 20 -h 20

3. Create test samples and their ground truth from single image applying distortions This is triggered when options, -img, -bg, and -info were specified. -img [a positive image] -bg [a description file of negatives] -info [a generated description file for the generated test iamges] In this case, -w and -h are used to determine the minimal size of positives to be embeded in the test images. $ createsamples -img face.png -num 10 -bg negatives.dat -info test.dat -maxxangle 0.6 -maxy

This generates tons of jpg files such as

The output file name format is as _<x>__<width>_.jpg, where x, y, width and height are the coordinates of placed object bounding rectangle. Only the first negative images in the <description file of negatives> are used. 4. Show images within a vec file This is triggered when only an option, -vec, was specified (no -info, -img, -bg). For example, $ createsamples -vec samples.vec -w 20 -h 20

Create Description File The format of a description file which is used by createsamples is [1]

11/07/2008 12.54

Tutorial: OpenCV haartraining (Rapid Object Detection With A Casca...

4 di 8

http://note.sonots.com/SciSoftware/haartraining.html

[filename] [filename] [filename] . .

such as img/img1.jpg img/img2.jpg

or if it is used for testing, [filename] [# of objects] [[x y width height] [... 2nd object] ...] [filename] [# of objects] [[x y width height] [... 2nd object] ...] [filename] [# of objects] [[x y width height] [... 2nd object] ...] . .

such as img/img1.jpg 1 140 100 45 45 img/img2.jpg 2 100 200 50 50 50 30 25 25

The first format can be easily created with the find command as $ cd [your working directory] $ find [image dir] -name '*.[image ext]' > [description file]

such as $ find ../../data/negatives/ -name '*.jpg' > negatives.dat $ find ../../data/umist_cropped/ -name '*.pgm' > positives.dat

EXTRA: random seed The createsamples software applys the same sequence of distortions for each image. We may want to apply the different sequence of distortions for each image because, otherwise, our resulting detection may work only for specific distortions. This can be done by modifying createsamples slightly as: Add below in the top #include

Add below in the main function srand(time(NULL));

The modified source code is available at svn:createsamples.cpp

Create Training Samples

11/07/2008 12.54

Tutorial: OpenCV haartraining (Rapid Object Detection With A Casca...

5 di 8

http://note.sonots.com/SciSoftware/haartraining.html

Kuranov et. al. [3] mentions as they used 5000 positive frontal face patterns and 3000 negatives for training, and 5000 positive frontal face patterns were derived from 1000 original faces. However, you may have noticed that none of 4 functionalities of createsamples software provide us the functionality to generate 5000 positive images from 1000 images at burst. We have to use the 1st functionality of the createsamples to generate 5 (or some) positives form 1 image, repeat the procedures 1000 (or some) times, and finally merge the generated output vec files. I wrote a program, mergevec.cpp, to merge vec files. I also wrote a script, createtrainsamples.pl, to repeat the procedures 1000 (or some) times. Please modify the path to createsamples and its option parameters directly written in the file. I specified 7000 instead of 5000 because the Tutorial [1] states as "the reasonable number of positive samples is 7000." The input format of createtrainsamples.pl is $ perl createtrainsamples.pl <positives.dat>

And, the input format of mergevec is $ mergevec

Example) $ $ $ $

perl createtrainsamples.pl positives.dat negatives.dat samples find samples/ -name '*.vec' > samples.dat # to create a collection file for vec files mergevec samples.dat samples.vec createsamples -vec samples.vec -show -w 20 -h 20 # Extra: If you want to see inside

Create Testing Samples Testing samples are images which include positives in negative images and locations of positives are known in the images. We can use the 3rd functionality of createsamples to do it. But, we can specify only one image using it, thus, creating a script to repeat the procedure would help us. The script is available at svn:createtestsamples.pl. Please modify the path to createsamples and its option parameters directly in the file. The input format of the createtestsamples.pl is as $ perl createtestsamples.pl <positives.dat>

This generates lots of jpg files and info.dat. The jpg file name format is as _<x>__<width>_.jpg, where x, y, width and height are the coordinates of placed object bounding rectangle. Example) $ perl createtestsamples.pl positives.dat negatives.dat tests $ find tests/ -name 'info.dat' -exec cat \{\} \; > tests.dat # merge info files $ # find tests/ -name 'info.dat' | xargs cat > tests.dat # should be faster, but may not wo

Training Training is done by the haartraining software.

haartraining 11/07/2008 12.54

Tutorial: OpenCV haartraining (Rapid Object Detection With A Casca...

6 di 8

http://note.sonots.com/SciSoftware/haartraining.html

Kuranov et. al. [3] states as 20x20 of sample size achieved the highest hit rate. Furthermore, they states as "For 18x18 four split nodes performed best, while for 20x20 two nodes were slightly better. The difference between weak tree classifiers with 2, 3 or 4 split nodes is smaller than their superiority with respect to stumps." Furthermore, there was a description as "20 stages were trained. Assuming that my test set is representative for and a hit rate about ." the learning task, I can expect a false alarm rate about Therefore, I used 20x20 of sample size with nsplit = 2, nstages = 20, minhitrate = 0.9999, maxfalselarm = 0.5 such as $ haartraining -data trainout -vec samples.vec -bg negatives.dat -nstages 20 -nsplits 2 -mi

The "-mode ALL" uses Extended Sets of Haar-like Features [2]. Default is BASIC and it uses only upright features, while ALL uses the full set of upright and 45 degree rotated feature set [1]. The "-mem 512" is the available memory in MB for precalculation [1]. Default is 200MB, so increase if more memory is available. We should not use all system RAM because otherwise it will result in a considerable training slow down. #If you use OpenMP (multi-processing) supporting compilers such as Intel C++ compiler and MS Visual Studio 8.0, the haartraining is automatically built with the OpenMP. This is effective when your computer has multi-CPUs. #One training took two days.

Generate xml The haartraing software did not generate a cascade.xml used by the facedetect software. Although the Tutorial [1] does not mention how to generate a cascade.xml, OpenCV has a software to convert a haartraining output dir tree into a xml file at the OpenCV/samples/c/convert_cascade.c (that is, in your installation directory). Compile it. The input format is as $ convert_cascade --size="<sample_width>x<sampe_height>"
Example) $ convert_cascade --size="20x20" trainout cascade.xml

Testing under construction. I am not achieving sufficient results....

performance We will evaluate the performance of the generated classifier using the performance software. The input format of the software is $ performance -data -info <description_file_for_test_samples>

such as

11/07/2008 12.54

Tutorial: OpenCV haartraining (Rapid Object Detection With A Casca...

7 di 8

http://note.sonots.com/SciSoftware/haartraining.html

$ performance -data trainout -info tests.dat

The output format is as +================================+======+======+======+ | File Name | Hits |Missed| False| +================================+======+======+======+

'Hits' shows the # of correctly found objects, and 'Missed' shows the # of missed objects (there must exist but not found, as known as false negatives), and 'False' shows the # of false alarm (there must not exist but found, as known as false positives).

facedetect Fun with a USB camera or some image files. $ facedetect --cascade=<xml_file> []

Discussion The generated classifier did not work well. Specializing classifiers for only the frontal faces may achieve a better result (UMIST images include 90 degree faces for each person.) If we are supposed to use the trained face detection system only indoor, using simpler negative images (pictures of plain walls, pc, chairs, desks only, etc) would provide a better accuracy for the constrained environment.

Download The files are available at http://tutorial-haartraining.googlecode.com/svn/trunk/ (mirror) http://opensvn.csie.org/sonots/SciSoftware/haartraining/ . HaarTraining Source Code, Binary, Makefile (VC++ Project File) data The collected Image Datasets result Generated Files This is a svn repository, so you can download files at burst if you have a svn client. (Sorry, but I will not explain how to install or use the svn client here because it is a general knowledge.) For example, svn co http://tutorial-haartraining.googlecode.com/svn/trunk/HaarTraining/ .

Sorry, but downloading (checkout) image datasets may take forever.... I created a zip file once, but google code repository did not allow me to upload such a big file (100MB). The list of my additional utilities (I put them in HaarTraining/ directory): mergevec.cpp † - Further Details (How to Use, Compile) vec2img.cpp - Further Details (How to Use, Compile) createtrainsamples.pl † createtestsamples.pl † The following additional utilities can be obtained from OpenCV/samples/c in your OpenCV install directory (I also put them in HaarTraining/ directory). convert_cascade.c facedetect.c

11/07/2008 12.54

Tutorial: OpenCV haartraining (Rapid Object Detection With A Casca...

8 di 8

http://note.sonots.com/SciSoftware/haartraining.html

References [1] Rapid Object Detection With A Cascade of Boosted Classifiers Based on Haar-like Features - OpenCV haartraining Tutorial (This can be obtained from OpenCV/apps/HaarTraining/doc on your OpenCV install directory). [2] Rainer Lienhart and Jochen Maydt. An Extended Set of Haar-like Features for Rapid Object Detection. Submitted to ICIP2002. ( http://www.lienhart.de/ICIP2002.pdf ) [3] Alexander Kuranov, Rainer Lienhart, and Vadim Pisarevsky. An Empirical Analysis of Boosting Algorithms for Rapid Objects With an Extended Set of Haar-like Features. Intel Technical Report MRL-TR-July02-01, 2002. ( http://www.lienhart.de/Publications/DAGM2003.pdf ) OpenCV haartraining (Rapid Object Detection With A Cascade of Boosted Classifiers Based on Haar-like Features) - Naotoshi Seo Learning-Based Computer Vision with Intel's Open Source Computer Vision Library How-to build a cascade of boosted classifiers based on haar like features [pdf] Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection [pdf] 5000 positive frontal face patterns and 3000 negative. 5000 positive frontal face patterns were derived from 1000 original face patterns by random rotation, scaling, mirroring, shifting. 20x20 of sample size with nsplit = 2. nstages = 20 OpenCV for Linux　Haar-like 特徴を用いる高速物体検知 OpenCV: オブジェクト検出器関数 http://ai-www.naist.jp/people/kohei-t/OpenCV/reference/ref/OpenCVRef_Experimental.htm 和訳 An Extended Set of Haar-like Features for Rapid Object Detection [pdf] MERL – TR2003-096 – Fast Multi-view Face Detection Haar状特徴に基づくブースト分類器のカスケードを利用する高速物体検知 haartraining Tutorial 和訳 Object Detection Using Haar-like Features with Cascade of Boosted Classifiers Manual to create cascade xml file. Same with the file located in OpenCV directory. OpenCV: Object Detection Functions FaceDetection - OpenCV Library Wiki Boosting Haar-like features. The basic classifiers are decision-tree classifiers with at least 2 leaves.

11/07/2008 12.54