22-06-2012, 01:58 PM
Energy-efficient Feedback Tracking on Embedded Smart Cameras by Hardware-level Optimization
[attachment=25072]
INTRODUCTION
The spread of embedded systems has increased enormously
worldwide in recent years. Embedded systems equipped with
smart camera sensors are employed in many different applications
including environmental monitoring [1], stereo matching
[2], as well as a plethora of applications related to video
surveillance, such as foreground object detection [7], [5], [3],
face detection [4], people detection [6] etc.
When a camera sensor is added to an embedded system,
several problems arise due to the large amount of image data
to be stored and processed. This impacts the overall efficiency,
the memory requirements, the communication bandwidth and
the energy consumption of the system.
THE EMBEDDED SMART CAMERA PLATFORM
The wireless embedded smart camera platform employed
in our experiments is a CITRIC mote [9] shown in Fig. 1. It
consists of a camera board and a wireless mote. The camera
board is composed of a CMOS image sensor, a microprocessor,
external memories and other supporting circuits. The
microprocessor PXA270 is a fixed-point processor with a
maximum speed of 624 MHz and 256 KB of internal SRAM.
The microprocessor is connected to a 64 MB of SDRAM and
16MB of NOR FLASH. The image sensor is an OmniVision
OV9655. An embedded Linux system runs on the camera
board. Attached to the camera board is a TelosB mote with a
maximum data rate of 250 Kbps.
IMAGE SCALING AND CROPPING
The main goal of our work is to decrease the processing time
and energy consumption. To achieve this goal, we perform two
main operations at hardware level: (i) the change of the image
resolution and (ii) image cropping based on a search region
obtained from the tracking stage. The hardware subsystem
composed of the image sensor and the quick capture interface
is highly configurable. The exploitation of this flexibility by
performing these functions at hardware level provides a reduction
in the amount of transferred data. This, in turn, leads to
significant savings in energy consumption thanks to the better
use of the memory controller and the memory resources and
freeing the main microprocessor from the tasks of performing
image down-sampling and cropping at software-level. Downsampling,
scaling and cropping operations are accomplished
by changing the hardware registers of the OV9655.
DETECTION AND TRACKING
Traditional tracking systems perform foreground object detection
and tracking at each frame independently, and in a
sequential manner. This will henceforth be referred to as the
sequential method. We have presented the feedback method
[10], which is a lightweight foreground object detection and
tracking algorithm suitable for embedded platforms.
SAVINGS IN ENERGY CONSUMPTION
In this section, we provide a quantitative comparison
showing the advantages of performing hardware-level downsampling
and cropping at the micro-controller of the OV9655
sensor for tracking purposes rather than processing whole
frames and performing these tasks at software level on the
main micro-processor of the camera board.
We will present savings in energy consumption when we
perform hardware-level down-sampling and cropping, and
use the feedback method for object detection and tracking.
As stated in [10], the feedback method provides significant
savings in processing time, and thus allows us to increase
idle state durations of cameras to increase the battery-life.
[attachment=25072]
INTRODUCTION
The spread of embedded systems has increased enormously
worldwide in recent years. Embedded systems equipped with
smart camera sensors are employed in many different applications
including environmental monitoring [1], stereo matching
[2], as well as a plethora of applications related to video
surveillance, such as foreground object detection [7], [5], [3],
face detection [4], people detection [6] etc.
When a camera sensor is added to an embedded system,
several problems arise due to the large amount of image data
to be stored and processed. This impacts the overall efficiency,
the memory requirements, the communication bandwidth and
the energy consumption of the system.
THE EMBEDDED SMART CAMERA PLATFORM
The wireless embedded smart camera platform employed
in our experiments is a CITRIC mote [9] shown in Fig. 1. It
consists of a camera board and a wireless mote. The camera
board is composed of a CMOS image sensor, a microprocessor,
external memories and other supporting circuits. The
microprocessor PXA270 is a fixed-point processor with a
maximum speed of 624 MHz and 256 KB of internal SRAM.
The microprocessor is connected to a 64 MB of SDRAM and
16MB of NOR FLASH. The image sensor is an OmniVision
OV9655. An embedded Linux system runs on the camera
board. Attached to the camera board is a TelosB mote with a
maximum data rate of 250 Kbps.
IMAGE SCALING AND CROPPING
The main goal of our work is to decrease the processing time
and energy consumption. To achieve this goal, we perform two
main operations at hardware level: (i) the change of the image
resolution and (ii) image cropping based on a search region
obtained from the tracking stage. The hardware subsystem
composed of the image sensor and the quick capture interface
is highly configurable. The exploitation of this flexibility by
performing these functions at hardware level provides a reduction
in the amount of transferred data. This, in turn, leads to
significant savings in energy consumption thanks to the better
use of the memory controller and the memory resources and
freeing the main microprocessor from the tasks of performing
image down-sampling and cropping at software-level. Downsampling,
scaling and cropping operations are accomplished
by changing the hardware registers of the OV9655.
DETECTION AND TRACKING
Traditional tracking systems perform foreground object detection
and tracking at each frame independently, and in a
sequential manner. This will henceforth be referred to as the
sequential method. We have presented the feedback method
[10], which is a lightweight foreground object detection and
tracking algorithm suitable for embedded platforms.
SAVINGS IN ENERGY CONSUMPTION
In this section, we provide a quantitative comparison
showing the advantages of performing hardware-level downsampling
and cropping at the micro-controller of the OV9655
sensor for tracking purposes rather than processing whole
frames and performing these tasks at software level on the
main micro-processor of the camera board.
We will present savings in energy consumption when we
perform hardware-level down-sampling and cropping, and
use the feedback method for object detection and tracking.
As stated in [10], the feedback method provides significant
savings in processing time, and thus allows us to increase
idle state durations of cameras to increase the battery-life.