[back]

VLSI for feature detection and tracking
Christophe Basset, Bedabrata Pain (JPL), Pietro Perona

Abstract. We are developing an integrated visual tracking system. The goal of this collaborative work with the Jet Propulsion Laboratory is a single chip serving as a camera (1024x1024 pixel imager array) able to find and track a small (7x7 pixel) target whose image has previously been provided by the user.

Motivation. A system capable of tracking any target in real-time in an unknown environment finds numerous applications: object avoidance or interception, autonomous navigation (machine vision, nano-rovers, robots, docking...), recognition (tracking of eyes, nose...), etc. A hardware implementation is very well suited for such real-time vision systems. A software-based approach would lack miniaturization (need of a camera, a computer with a frame grabber) and would run too slowly (~1 frame/second) for most of these applications requiring a real-time flow of data.

The on-going collaboration with the Jet Propulsion Laboratory has brought to this project expertise in Active Pixel Sensors. This CMOS technology for building imager chips allows on-focal plane signal processing (as opposed to their CCD counterparts that need to serially output the flow of pixels to an external processing chip). The tracking can therefore be implemented as a fast, low-power analog circuit.

Research and Achievements. Tracking is achieved by matching a template to an image. The chip has an integrated imager array and a 7x7 pixel digital memory to store the template. The template is chosen off-chip through a separate learning process. This part will therefore not be discussed here. The initial "best match" is located during a slow scan of the entire imager array (anticipated size of 1024x1024 pixel) allowing us to restrict the search for the following frames to a smaller, 25x25 pixel sub-window.

At each frame, the template is to be correlated with the search window. The highest correlation response is determined in a decision-maker circuit. This feeds back to an addressing unit that moves the search window, ensuring that it remains centered on the target. Below is the block diagram of the feature-tracking chip.

The core of the tracking system relies on a 7x7 pixel cell whose task is to perform a correlation. The image provided by the array reaches the correlating block one row at a time. Hence, the 25x25 pixel search window can be processed by only 25 correlating blocks operating in parallel. Each of these cells computes the matching between the analog image (I) and the template (T):

The image is fed to the correlating block one row at a time (making the system parallel in this direction). By implementing a series of delay lines (shown below), we can reconstruct the complete 25x25 correlation map in 25 cycles (one image line processed in parallel per cycle).

The imager provides the pixels as analog currents to the correlator while the template comes as 8-bit digital lines. Correlation is obtained by mirroring and scaling the image currents (binary-scaled: by x2, x4, x8...). The scaled currents are dumped, through switches controlled by the template bits, into capacitors for read-out. To keep the transistors from getting too large (the most significant bit would be 128 times bigger than the least significant bit), the template is divided into two halves (mirrors scaled x1 to x8 each) and dumped into the accumulating capacitors with different integration times (50ns and 800ns respectively).


A first prototype of the correlator chip was built and tested. It consists of a 64x64 pixel imager array, a 49-byte digital memory to store the template, and a single 7x7 pixel correlating cell tied to the center of the image. Preliminary results demonstrated a correlation and allowed identification of adjustments to be made for the next fabrication run. Below is a photograph of the chip placed on the test board.

This project is currently in the next phase. The goal is now to complete the functional circuits. This includes normalizing the correlation (necessary to be able to track the target accurately) and making a decision (ie. finding the highest correlation from the computed 25x25 correlation map).

An effort is also being made to explore other circuits for performing the correlation (with improved performance in terms of spped, accuracy, size of the circuit...) as well as alternate template-matching techniques.