21-06-2012, 04:15 PM
IMPLEMENTATION OF IMAGE PROCESSING ALGORITHMS ON FPGA HARDWARE
INTRODUCTION
Recently, Field Programmable Gate Array (FPGA) technology has become a viable target for the
implementation of algorithms suited to video image processing applications. The unique architecture of the
FPGA has allowed the technology to be used in many such applications encompassing all aspects of video
image processing [1,2]. The goal of this thesis is to develop FPGA realizations of three such algorithms on
two FPGA architectures.
As image sizes and bit depths grow larger, software has become less useful in the video processing
realm. Real-time systems such as those that are the target of this project are required for the high speeds
needed in processing video. In addition, a common problem is dealing with the large amount of data
captured using satellites and ground-based detection systems. DSP systems are being employed to
selectively reduce the amount of data to process, ensuring that only relevant data is passed on to a human
analyst. Eventually, it is expected that most video processing can and will take place in DSP systems, with
little human interaction. This is obviously advantageous, since human data analysts are expensive and
perhaps not entirely accurate.
PC Digital Signal Processing Programs
Signal processing programs used on a PC allow for rapid development of algorithms, as well as
equally rapid debug and test capabilities. It is common for many hardware designers to use some sort of
PC programming environment to implement a design to verify functionality prior to a lengthy hardware
design.
MATLAB [6] is such an environment. Although it was created for manipulating matrices in
general, it is well suited to some image processing applications. MATLAB treats an image as a matrix,
allowing a designer to develop optimized matrix operations implementing an algorithm. However, if the
eventual goal is a hardware device, the algorithms are instead often written to operate similarly to the
proposed hardware system, which results in an even slower algorithm.
Systems such as IDL [7] and its graphical component ENVI [8] are more specifically geared to
image processing applications, and include many pre-written algorithms commonly used to process images.
However, even specialized image processing programs running on PCs cannot adequately process large
amounts of high-resolution streaming data, since PC processors are made to be for general use. Further
optimization must take place on a hardware device.
Application Specific Integrated Circuits
Application Specific Integrated Circuits (ASICs) represent a technology in which engineers create
a fixed hardware design using a variety of tools. Once a design has been programmed onto an ASIC, it
cannot be changed. Since these chips represent true, custom hardware, highly optimized, parallel
algorithms are possible. However, except in high-volume commercial applications, ASICs are often
considered too costly for many designs. In addition, if an error exists in the hardware design and is not
discovered before product shipment, it cannot be corrected without a very costly product recall.
Dedicated Digital Signal Processors
Digital Signal Processors (DSPs) such as those available from Texas Instruments [9] are a class of
hardware devices that fall somewhere between an ASIC and a PC in terms of performance and design
complexity. They can be programmed with either assembly code or the C programming language, which is
one of the platform’s distinct advantages. Hardware design knowledge is still required, but the learning
curve is significantly lower than some other design choices, since many engineers have knowledge of C
prior to exposure to DSP systems. However, algorithms designed for a DSP cannot be highly parallel
without using multiple DSPs. Algorithm performance is certainly higher than on a PC, but in some cases,
ASIC or FPGA systems are the only choice for a design. Still, DSPs are a very common and efficient
method of processing real-time data [10].
One area where DSPs are particularly useful is the design of floating point systems. On ASICs
and FPGAs, floating-point operations are rather difficult to implement. For the scope of this project, this is
not an issue because all images consist of only integer data.
Recent advances in DSP technology have resulted in very high-speed algorithm implementations
[11]. While the advantages of ASICs and FPGAs are still applicable, this new generation of DSPs has
made some engineers reconsider FPGA development. Still, as new DSPs arrive to the market, so do new
FPGAs, and it is expected that the two architectures will have similarly increasing performance for each
new generation of processors.
Field Programmable Gate Arrays
Field Programmable Gate Arrays (FPGAs) represent reconfigurable computing technology [12],
which is in some ways ideally suited for video processing. Reconfigurable computers are processors which
can be programmed with a design, and then reprogrammed (or reconfigured) with virtually limitless
designs as the designer’s needs change. FPGAs generally consist of a system of logic blocks (usually look
up tables and flip-flops) and some amount of Random Access Memory (RAM), all wired together using a
vast array of interconnects. All of the logic in an FPGA can be rewired, or reconfigured, with a different
design as often as the designer likes.
INTRODUCTION
Recently, Field Programmable Gate Array (FPGA) technology has become a viable target for the
implementation of algorithms suited to video image processing applications. The unique architecture of the
FPGA has allowed the technology to be used in many such applications encompassing all aspects of video
image processing [1,2]. The goal of this thesis is to develop FPGA realizations of three such algorithms on
two FPGA architectures.
As image sizes and bit depths grow larger, software has become less useful in the video processing
realm. Real-time systems such as those that are the target of this project are required for the high speeds
needed in processing video. In addition, a common problem is dealing with the large amount of data
captured using satellites and ground-based detection systems. DSP systems are being employed to
selectively reduce the amount of data to process, ensuring that only relevant data is passed on to a human
analyst. Eventually, it is expected that most video processing can and will take place in DSP systems, with
little human interaction. This is obviously advantageous, since human data analysts are expensive and
perhaps not entirely accurate.
PC Digital Signal Processing Programs
Signal processing programs used on a PC allow for rapid development of algorithms, as well as
equally rapid debug and test capabilities. It is common for many hardware designers to use some sort of
PC programming environment to implement a design to verify functionality prior to a lengthy hardware
design.
MATLAB [6] is such an environment. Although it was created for manipulating matrices in
general, it is well suited to some image processing applications. MATLAB treats an image as a matrix,
allowing a designer to develop optimized matrix operations implementing an algorithm. However, if the
eventual goal is a hardware device, the algorithms are instead often written to operate similarly to the
proposed hardware system, which results in an even slower algorithm.
Systems such as IDL [7] and its graphical component ENVI [8] are more specifically geared to
image processing applications, and include many pre-written algorithms commonly used to process images.
However, even specialized image processing programs running on PCs cannot adequately process large
amounts of high-resolution streaming data, since PC processors are made to be for general use. Further
optimization must take place on a hardware device.
Application Specific Integrated Circuits
Application Specific Integrated Circuits (ASICs) represent a technology in which engineers create
a fixed hardware design using a variety of tools. Once a design has been programmed onto an ASIC, it
cannot be changed. Since these chips represent true, custom hardware, highly optimized, parallel
algorithms are possible. However, except in high-volume commercial applications, ASICs are often
considered too costly for many designs. In addition, if an error exists in the hardware design and is not
discovered before product shipment, it cannot be corrected without a very costly product recall.
Dedicated Digital Signal Processors
Digital Signal Processors (DSPs) such as those available from Texas Instruments [9] are a class of
hardware devices that fall somewhere between an ASIC and a PC in terms of performance and design
complexity. They can be programmed with either assembly code or the C programming language, which is
one of the platform’s distinct advantages. Hardware design knowledge is still required, but the learning
curve is significantly lower than some other design choices, since many engineers have knowledge of C
prior to exposure to DSP systems. However, algorithms designed for a DSP cannot be highly parallel
without using multiple DSPs. Algorithm performance is certainly higher than on a PC, but in some cases,
ASIC or FPGA systems are the only choice for a design. Still, DSPs are a very common and efficient
method of processing real-time data [10].
One area where DSPs are particularly useful is the design of floating point systems. On ASICs
and FPGAs, floating-point operations are rather difficult to implement. For the scope of this project, this is
not an issue because all images consist of only integer data.
Recent advances in DSP technology have resulted in very high-speed algorithm implementations
[11]. While the advantages of ASICs and FPGAs are still applicable, this new generation of DSPs has
made some engineers reconsider FPGA development. Still, as new DSPs arrive to the market, so do new
FPGAs, and it is expected that the two architectures will have similarly increasing performance for each
new generation of processors.
Field Programmable Gate Arrays
Field Programmable Gate Arrays (FPGAs) represent reconfigurable computing technology [12],
which is in some ways ideally suited for video processing. Reconfigurable computers are processors which
can be programmed with a design, and then reprogrammed (or reconfigured) with virtually limitless
designs as the designer’s needs change. FPGAs generally consist of a system of logic blocks (usually look
up tables and flip-flops) and some amount of Random Access Memory (RAM), all wired together using a
vast array of interconnects. All of the logic in an FPGA can be rewired, or reconfigured, with a different
design as often as the designer likes.