[back]

Configurable Architectures and Systems for Real-Time Low-level Vision
Arrigo Benedetti, Pietro Perona

Abstract. The long-term goal of this project is to build an infrastructure for the design and implementation of real-time computer vision systems. Since vision algorithms are compute-bound we have chosen the technology of Field Programmable Gate Array (FPGAs), that allow to exploit the instruction level parallelism inherent to the first stages of vision tasks. The first problem that we have considered is the real-time computation of the optical flow measured from the sequence of images captured by a video camera. We have designed, built and demonstrated a system able to select in real-time 2-D visual features on a commercially available reconfigurable platform. During this process we have learned that the system level architectures of off-the-shelf reconfigurable computers are not optimized for low level vision tasks, therefore, we have designed a novel architecture dedicated to real-time processing of video streams. A system based on this architecture has been built and is currently being tested. More recently, we have studied the problem of bit-width computation for the optimization of the data paths found in digital video signal processors.

Motivation. The practical realization of many computer vision applications such as vehicles which guides themselves (e.g. autonomous robots moving on a factory floor or automatically driving along a highway) is often slowed down by the computational complexity of the low-level vision tasks that need to be carried on. Among the goals of this project is the development of a framework for the deployment of systems, based on configurable logic devices, able to perform these signal processing tasks in real-time. An additional motivation to our research is the lack of commercially available configurable platforms and tools optimized to signal processing tasks on video signals.

Research and Achievements. We have designed and implemented a system able to select salient features from a sequence of video frames in real-time (30 frames/sec). In the figure below we show the point features selected in a single frame by an algorithm inspired by the method proposed by Tomasi and Kanade in a way that does not not require any floating-point operation. This algorithm has been implemented on a Giga Operations G800 Reconfigurable Computer hosting 4 XC4028EX and 2 XC4020E Xilinx Field Programmable Gate Arrays. Reconfigurable hardware allows these tasks to be performed quickly while still leaving room for future modification and enhancements.

The algorithm that we employed for real-time feature selection has also been implemented in an analog VLSI integrated circuit in a collaborative effort by Alberto Pesavento in the Koch group. More recently, we have designed a novel architecture dedicated to real-time processing of videostreams. A system based on this architecture has been built and is currently being tested. In a joint project with Irvine Sensors Corp. a very compact solid state imager tightly coupled with computing and memory resources has been designed and is currently being built. The aim of of this project is the exploration of the design challanges and tradeoffs involved in performing complex vision tasks in a very limited space. We are also studying the theoretical problem of bit-width computation for the optimization the datapaths found in digital video signal processors.

Publications/References
Real-time 2-D Feature Detection on a Reconfigurable Computer. A Benedetti, P Perona, Proc. of the 1998 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'98), Santa Barbara, CA, June 1998.

A Novel System Architecture for Real-Time Low-Level Vision. A Benedetti, P Perona, Proc. of the 1999 IEEE Symposium on Circuit and Systems (ISCAS'99), Orlando, FL, June 1999.

Bit-width optimization for configurable DSP's by multi-interval analysis. A Benedetti, P Perona, presented at the 34th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, Oct 2000.