[back]

Classification of Road Vehicles
Robert Fergus, Bradley Phillips, Paul Updike

Abstract. We have worked to apply the probabilistic recognition techniques developed in our lab to the classification of road vehicles in busy traffic scenes. We have demonstrated that the model can successfully determine the presence or absence of cars in a given road scene using a detection algorithm that is translation and scale invariant, and can deal with cluttered scenes and occlusions.

Motivation and Aims. The automotive industry is, for safety reasons, interested in giving cars awareness of the environment around them. They would like cars to be able to monitor the position and relative speeds of other nearby vehicles and to be able to tell what kind of vehicle they are. Our objective is to apply our recognition models to this problem giving reliable classification of images from an in-car camera.

Research. The Idea. Other research has demonstrated that the constellation model has promise, and it seemed well suited for this application. Thus far, our particular focus was to understand how to set the constellation model parameters such that performance is maximized.

The Constellation Model Parameters. In applying the constellation model to a data set, there are many parameters that determine just how the model will behave. We focused on three parameters that seemed particularly relevant; the average number of detections per feature, the number of features in the model, and the size of the individual features.

Detections per Feature. In the current version of the code used for the model learning and detections, we have used a hard cutoff for determining what areas of an image are classified as containing a given feature. This means that increasing the average number of detected features per image is equivalent to accepting a "less convincing" area of the image as a features. As a result, on average the highest quality features can be found when the number of detections per feature is relatively low - an average of four detections per image worked quite well. (see figure 1. below.) It seems likely that overall performance could be improved with better feature detectors; particularly a detector scheme that includes soft detections - i.e. gives a quality rating for each detection.

Figure 1(a). (above left) Many detections per feature, but in this case the car was not detected.

Figure 1(b). (above right) Few average feature detections per image, and yet this car was correctly detected by the model.

Number of Features in the Model. Another parameter we set out to learn about was the number of features used in the model. In this case we tried many different models with anywhere from two to six features in them (we did not allow repeated parts in our experimentation.) We found that three part models gave adequate performance, and that the learning time was far better than for models with more parts. Models with five or six parts took many, many times longer to run with only slightly better performance overall. In the end we settled on the three part model as the best choice for current use. (see figure 2. below.)

Figure 2. Performance for one of the better three part models can be seen in the ROC curve above. Click for the full size image.

Feature Size. In the beginning it was unclear what size patch should be used to obtain the best performance. Our experiments indicated that smaller patches represented more "atomic" elements of the images - such as lines and simple gradients, whereas larger patches also included very distinctive parts of the images - rear view mirrors, license plates, and corners of bumpers to name a few. The larger patches generally lead to better overall model performance. This can be seen by looking at the set of patches used to determine the model, known as the codebook. (see figure 3. below.)

Figure 3(a). (above left) An excerpt from a codebook constructed using 7x7 pixel patches. Note the lack of anything specifically car-like.

Figure 3(b). (above right) An excerpt from a codebook constructed using 15x15 pixel patches. Note the many car-like bits, such as tail lights.

Model enhancements. In addition to the above experiments we are looking at ways to improve the probabilistic recognition model. One approach is to use a number of smaller models in conjunction rather than one large complex one. Preliminary results from this are very encouraging, with an error on our test sets of under 1%.

Achievements. We have begun to apply the constellation model for use in the classification of road vehicles. We have demonstrated that it works well for cars, and we have begun to apply the model in detecting different types of trucks and other vehicles.