25-10-2016, 09:11 AM
[attachment=73792]
Introduction
A smart camera performs real-time analysis to recognize scenic elements. Smart cameras
are useful in a variety of scenarios: surveillance, medicine, etc.We have built a real-time
system for recognizing gestures. Our smart camera uses novel algorithms to recognize gestures
based on low-level analysis of body parts as well as hidden Markov models for the moves that
comprise the gestures. These algorithms run on a Trimedia processor. Our system can recognize
gestures at the rate of 20 frames/second. The camera can also fuse the results of multiple
cameras.
A "smart camera" is basically a video camera coupled to a computer vision system in a tiny
package. This communication begins stating the main differences between smart cameras and
standard smart vision systems. Smart camera architecture is described whereby a combination of
an on-board microprocessor and PLD‘s allow for the embedding of image processing algorithms
in the camera. A static thresholding algorithm is presented which demonstrates the ability to
track non-uniformity in the inspection target. A multi camera inspection system application is
presented where a maximum of twenty smart cameras may be networked together to a single
host computer. Finally, a prediction is made of technological and applicational future evolution
on smart cameras.
The smart camera – a whole vision system contained in one neat housing – can be used
anywhere, in any industry where image processing can be applied. Companies no longer need a
cabinet in which to keep all their computing equipment: the computer is housed within the smart
camera. In the pharmaceutical industry and in clean rooms – when not even dust is allowed – this
can be a big advantage. A single square meter of space can be comparatively very expensive – if
there is no need for a component rack or cabinet, simply a smart camera, and then this could save
a lot of money. In particular, there would not be the usual cabling involved for other vision
systems, and set-up is simple. Later in this communication are stated some advantages of using
smart cameras or PC-based systems in vision applications.
In usual vision systems scenarios, only a small fraction of a picture frame will be the region of
interest (ROI). In fact, often no visual image of the ROI is even required. The object of a vision
system, after all, is to make a decision: "Is there a blob"? "Where is the blob"? "Is this a
defect"?What if all that pixel pre-processing and decision-making could be done within the
camera? If all the processing were done inside the camera, the blob analysis of a gigabit image
might result in only a few hundred bytes of data which need to be sent somewhere. Such
compact packets of data could be easily transmitted directly to a machine control without even
passing through a PC.
Information should be processed where the information occurs – i.e. the brain should be behind
the eyes!
The answer to the problem stated earlier is a stand-alone, smart camera.To illustrates this, a
smart camera‘s embedded system architecture is shown along with an example of a hardware
embedded image processing algorithm: “Static Gray Scale Thresholding”.
Many global companies including Glaxo and Ciba-Geigy, and motor companies such as Ford
and Volkswagen are users of smart cameras.
A small example of a vision system for web inspection is shown where twenty smart cameras are
connected together.
Many changes are yet to come concerning smart cameras as in technology as well as in there
future applications. An overview, concerning these issues, is made closing this communication.
We have to recognize however that smart cameras will not be the answer for all vision applications.
OVERVIEW OF SMART CAMERA
A smart camera is an integrated machine vision system which, in addition to image capture
circuitry, includes a processor, which can extract information from images without need for an
external processing unit, and interface devices used to make results available to other devices.
A smart camera or "intelligent camera" is a self-contained, standalone vision system with built-in
image sensor in the housing of an industrial video camera. It contains all necessary
communication interfaces, e.g. Ethernet, as well as industry-proof 24V I/O lines for connection
to a PLC, actuators, relays or pneumatic valves. It is not necessarily larger than an industrial or
surveillance camera.
This architecture has the advantage of a more compact volume compared to PC-based vision
systems and often achieves lower cost, at the expense of a somewhat simpler (or missing
altogether) user interface.
Although often used for simpler applications, modern smart cameras can rival PCs in terms of
processing power and functionalities. Smart cameras have been marketed since the mid 80s, but
only in recent years have they reached widespread use, once technology allowed their size to be
reduced while their processing power has reached several thousand MIPS (devices with 1GHz
processors and up to 8000MIPS are available as of end of 2006).
Having a dedicated processor in each unit, smart cameras are especially suited for applications
where several cameras must operate independently and often asynchronously, or when
distributed vision is required (multiple inspection or surveillance points along a production line or within an assembly machine).
Smart Cameras vs. Standard Smart Vision Systems
The question often comes up as to what is the most appropriate approach to take in implementing
a vision system - using a smart camera or using some sort of PC-based approach. There is no
question that as the microprocessors, DSPs and FPGAs are getting faster and, therefore, more
capable, smart cameras are getting smarter. Therefore, they are a challenge to more ''traditional''
approaches to vision systems. Significantly, however, ''traditional'' approaches are also taking
advantage of the advances and so, too, are faster and smarter.
Traditional approaches usually mean a PC-based implementation. This could be either using a
camera with the capability to interface directly to the PC (IEEE 1394/Fire wire, Camera Link,
LVDS, USB, etc.), or a system based on a frame grabber or other intelligent image processing
board or vision engine that plugs into the PC. In this latter case, more conventional analog
cameras are used as the input device.
A smart camera, on the other hand, is a self-contained unit. It includes the imager as well as the
''intelligence'' and related I/O capabilities. Because this format resembles the format of many
intelligent sensors, these products are often referred to as ''vision sensors.'' However, a vision
sensor often has a limited and fixed performance envelope, while a smart camera has more
flexibility or tools, inherently capable of being programmed to handle many imaging algorithms
and application functions. A PC-based vision system is generally recognized as having the
greatest flexibility and, therefore, capable of handling a wider range of applications.
2.3.1 PC-Based Vision Systems Advantages
The PC-based vision systems advantages include:
1) Flexibility –
The PC offers greater flexibility in the number of options that can be selected. For
example one can use a line scan versus an area scan camera with the PC. One can use
third party software packages with the PC approach (smart cameras tend to be single
source software).
2) Power –
PC's tend to offer greater power and speed due in large part to the speed of the Intel
processors used internally. This power in turn means that PC's are used to handle the
''tougher'' applications in vision systems.
2.3.2 Smart Cameras Advantages
The smart cameras advantages include:
1) Cost –
Smart cameras are generally less expensive to purchase and set up than the PC based solution,
since they include the camera, lenses, lighting (sometimes), cabling and processing.
2) Simplicity –
Software tools available with smart cameras are of the point-and-click variety and are easier to
use than those available on PC's. Algorithms come pre-packaged and do not need to be
developed, thus making the smart camera quicker to setup and use.
3) Integration –
Given their unified packaging, smart cameras are easier to integrate into the manufacturing
environment.
4) Reliability –
With fewer moving components (fans, hard drives) and lower temperatures, smart cameras are
more reliable than PC's.
In general the performance of the smart camera will continue to increase. This will mean that the
smart camera will be used for more difficult applications, slowly displacing the PC approach.
Smart Camera Architecture
The smart camera presented in this communication reduces the amount of data generated to the
„data of interest‟ by making use of embedded image processing algorithms. The data of interest
might be, for example, defective areas of the product being inspected. Multiple cameras can
route their data to a single frame grabber and computer due to the reduction of data stream, thus
dramatically reducing system cost and increasing inspection bandwidth capability. This smart
camera also makes use of an on-board microprocessor for communication with the inspection
systems‘ host computer and for internal control functions.
The following block diagram illustrates the camera architecture.
Fig 2.4. Smart Camera Architecture Block Diagram
A detailed explanation of the camera architecture follows, starting with the image sensor.
2.4.1 Image Sensor Basics
In this smart camera, a CCD (Charge Coupled Device) image sensor converts photons (light)
into electrons (charge). When photons hit an image sensor, the sensor accumulates electrons.
This is called charge integration. The brighter your light source, the more photons available for
the sensor to integrate, and the smaller the amount of time required to collect a given amount of
light energy.
Finally, the sensor transfers its aggregate charge to readout registers, which feed each pixel‘s
charge from the image sensor into an output node that converts the charges into voltages. After
this transfer and conversion, the voltages are amplified to become the camera‘s analog output
2.4.2 Analog to Digital Conversion Electronics
The analog output of the CCD is c
onverted to a digital output for further processing. The camera presented here sub-divides the
CCD analog output into eight channels of 256 pixel elements each. Analog to digital conversion
is performed at a 20 MHz data rate for each channel thus yielding an effective camera data rate
of 160 MHz. The digital data is then passed along to the image processing electronics for
processing and analysis.
2.4.3 Image Processing Electronics
Image processing is performed by embedded algorithms on a per channel basis. The
following block diagram illustrates the basic processing architecture for each channel.
Introduction to embedded systems:
An embedded system is a special-purpose computer system designed to perform one or a few
dedicated functions, often with real-time computing constraints. It is usually embedded as part of
a complete device including hardware and mechanical parts. In contrast, a general-purpose
computer, such as a personal computer, can do many different tasks depending on programming.
Embedded systems control many of the common devices in use today.
Since the embedded system is dedicated to specific tasks, design engineers can optimize it,
reducing the size and cost of the product, or increasing the reliability and performance. Some
embedded systems are mass-produced, benefiting from economies of scale.
Physically, embedded systems range from portable devices such as digital watches and MP4
players, to large stationary installations like traffic lights, factory controllers, or the systems
controlling nuclear power plants. Complexity varies from low, with a single microcontroller
chip, to very high with multiple units, peripherals and networks mounted inside a large chassis or
enclosure.
In general, "embedded system" is not an exactly defined term, as many systems have some
element of programmability. For example, Handheld computers share some elements with
embedded systems — such as the operating systems and microprocessors which power them —
but are not truly embedded systems, because they allow different applications to be loaded and
peripherals to be connected.
Embedded computing systems
• Computing systems embedded within electronic devices
• Hard to define. Nearly any computing system other than a desktop computer
• Billions of units produced yearly, versus millions of desktop units
• Perhaps 50 per household and per automobile.
Introduction
A smart camera performs real-time analysis to recognize scenic elements. Smart cameras
are useful in a variety of scenarios: surveillance, medicine, etc.We have built a real-time
system for recognizing gestures. Our smart camera uses novel algorithms to recognize gestures
based on low-level analysis of body parts as well as hidden Markov models for the moves that
comprise the gestures. These algorithms run on a Trimedia processor. Our system can recognize
gestures at the rate of 20 frames/second. The camera can also fuse the results of multiple
cameras.
A "smart camera" is basically a video camera coupled to a computer vision system in a tiny
package. This communication begins stating the main differences between smart cameras and
standard smart vision systems. Smart camera architecture is described whereby a combination of
an on-board microprocessor and PLD‘s allow for the embedding of image processing algorithms
in the camera. A static thresholding algorithm is presented which demonstrates the ability to
track non-uniformity in the inspection target. A multi camera inspection system application is
presented where a maximum of twenty smart cameras may be networked together to a single
host computer. Finally, a prediction is made of technological and applicational future evolution
on smart cameras.
The smart camera – a whole vision system contained in one neat housing – can be used
anywhere, in any industry where image processing can be applied. Companies no longer need a
cabinet in which to keep all their computing equipment: the computer is housed within the smart
camera. In the pharmaceutical industry and in clean rooms – when not even dust is allowed – this
can be a big advantage. A single square meter of space can be comparatively very expensive – if
there is no need for a component rack or cabinet, simply a smart camera, and then this could save
a lot of money. In particular, there would not be the usual cabling involved for other vision
systems, and set-up is simple. Later in this communication are stated some advantages of using
smart cameras or PC-based systems in vision applications.
In usual vision systems scenarios, only a small fraction of a picture frame will be the region of
interest (ROI). In fact, often no visual image of the ROI is even required. The object of a vision
system, after all, is to make a decision: "Is there a blob"? "Where is the blob"? "Is this a
defect"?What if all that pixel pre-processing and decision-making could be done within the
camera? If all the processing were done inside the camera, the blob analysis of a gigabit image
might result in only a few hundred bytes of data which need to be sent somewhere. Such
compact packets of data could be easily transmitted directly to a machine control without even
passing through a PC.
Information should be processed where the information occurs – i.e. the brain should be behind
the eyes!
The answer to the problem stated earlier is a stand-alone, smart camera.To illustrates this, a
smart camera‘s embedded system architecture is shown along with an example of a hardware
embedded image processing algorithm: “Static Gray Scale Thresholding”.
Many global companies including Glaxo and Ciba-Geigy, and motor companies such as Ford
and Volkswagen are users of smart cameras.
A small example of a vision system for web inspection is shown where twenty smart cameras are
connected together.
Many changes are yet to come concerning smart cameras as in technology as well as in there
future applications. An overview, concerning these issues, is made closing this communication.
We have to recognize however that smart cameras will not be the answer for all vision applications.
OVERVIEW OF SMART CAMERA
A smart camera is an integrated machine vision system which, in addition to image capture
circuitry, includes a processor, which can extract information from images without need for an
external processing unit, and interface devices used to make results available to other devices.
A smart camera or "intelligent camera" is a self-contained, standalone vision system with built-in
image sensor in the housing of an industrial video camera. It contains all necessary
communication interfaces, e.g. Ethernet, as well as industry-proof 24V I/O lines for connection
to a PLC, actuators, relays or pneumatic valves. It is not necessarily larger than an industrial or
surveillance camera.
This architecture has the advantage of a more compact volume compared to PC-based vision
systems and often achieves lower cost, at the expense of a somewhat simpler (or missing
altogether) user interface.
Although often used for simpler applications, modern smart cameras can rival PCs in terms of
processing power and functionalities. Smart cameras have been marketed since the mid 80s, but
only in recent years have they reached widespread use, once technology allowed their size to be
reduced while their processing power has reached several thousand MIPS (devices with 1GHz
processors and up to 8000MIPS are available as of end of 2006).
Having a dedicated processor in each unit, smart cameras are especially suited for applications
where several cameras must operate independently and often asynchronously, or when
distributed vision is required (multiple inspection or surveillance points along a production line or within an assembly machine).
Smart Cameras vs. Standard Smart Vision Systems
The question often comes up as to what is the most appropriate approach to take in implementing
a vision system - using a smart camera or using some sort of PC-based approach. There is no
question that as the microprocessors, DSPs and FPGAs are getting faster and, therefore, more
capable, smart cameras are getting smarter. Therefore, they are a challenge to more ''traditional''
approaches to vision systems. Significantly, however, ''traditional'' approaches are also taking
advantage of the advances and so, too, are faster and smarter.
Traditional approaches usually mean a PC-based implementation. This could be either using a
camera with the capability to interface directly to the PC (IEEE 1394/Fire wire, Camera Link,
LVDS, USB, etc.), or a system based on a frame grabber or other intelligent image processing
board or vision engine that plugs into the PC. In this latter case, more conventional analog
cameras are used as the input device.
A smart camera, on the other hand, is a self-contained unit. It includes the imager as well as the
''intelligence'' and related I/O capabilities. Because this format resembles the format of many
intelligent sensors, these products are often referred to as ''vision sensors.'' However, a vision
sensor often has a limited and fixed performance envelope, while a smart camera has more
flexibility or tools, inherently capable of being programmed to handle many imaging algorithms
and application functions. A PC-based vision system is generally recognized as having the
greatest flexibility and, therefore, capable of handling a wider range of applications.
2.3.1 PC-Based Vision Systems Advantages
The PC-based vision systems advantages include:
1) Flexibility –
The PC offers greater flexibility in the number of options that can be selected. For
example one can use a line scan versus an area scan camera with the PC. One can use
third party software packages with the PC approach (smart cameras tend to be single
source software).
2) Power –
PC's tend to offer greater power and speed due in large part to the speed of the Intel
processors used internally. This power in turn means that PC's are used to handle the
''tougher'' applications in vision systems.
2.3.2 Smart Cameras Advantages
The smart cameras advantages include:
1) Cost –
Smart cameras are generally less expensive to purchase and set up than the PC based solution,
since they include the camera, lenses, lighting (sometimes), cabling and processing.
2) Simplicity –
Software tools available with smart cameras are of the point-and-click variety and are easier to
use than those available on PC's. Algorithms come pre-packaged and do not need to be
developed, thus making the smart camera quicker to setup and use.
3) Integration –
Given their unified packaging, smart cameras are easier to integrate into the manufacturing
environment.
4) Reliability –
With fewer moving components (fans, hard drives) and lower temperatures, smart cameras are
more reliable than PC's.
In general the performance of the smart camera will continue to increase. This will mean that the
smart camera will be used for more difficult applications, slowly displacing the PC approach.
Smart Camera Architecture
The smart camera presented in this communication reduces the amount of data generated to the
„data of interest‟ by making use of embedded image processing algorithms. The data of interest
might be, for example, defective areas of the product being inspected. Multiple cameras can
route their data to a single frame grabber and computer due to the reduction of data stream, thus
dramatically reducing system cost and increasing inspection bandwidth capability. This smart
camera also makes use of an on-board microprocessor for communication with the inspection
systems‘ host computer and for internal control functions.
The following block diagram illustrates the camera architecture.
Fig 2.4. Smart Camera Architecture Block Diagram
A detailed explanation of the camera architecture follows, starting with the image sensor.
2.4.1 Image Sensor Basics
In this smart camera, a CCD (Charge Coupled Device) image sensor converts photons (light)
into electrons (charge). When photons hit an image sensor, the sensor accumulates electrons.
This is called charge integration. The brighter your light source, the more photons available for
the sensor to integrate, and the smaller the amount of time required to collect a given amount of
light energy.
Finally, the sensor transfers its aggregate charge to readout registers, which feed each pixel‘s
charge from the image sensor into an output node that converts the charges into voltages. After
this transfer and conversion, the voltages are amplified to become the camera‘s analog output
2.4.2 Analog to Digital Conversion Electronics
The analog output of the CCD is c
onverted to a digital output for further processing. The camera presented here sub-divides the
CCD analog output into eight channels of 256 pixel elements each. Analog to digital conversion
is performed at a 20 MHz data rate for each channel thus yielding an effective camera data rate
of 160 MHz. The digital data is then passed along to the image processing electronics for
processing and analysis.
2.4.3 Image Processing Electronics
Image processing is performed by embedded algorithms on a per channel basis. The
following block diagram illustrates the basic processing architecture for each channel.
Introduction to embedded systems:
An embedded system is a special-purpose computer system designed to perform one or a few
dedicated functions, often with real-time computing constraints. It is usually embedded as part of
a complete device including hardware and mechanical parts. In contrast, a general-purpose
computer, such as a personal computer, can do many different tasks depending on programming.
Embedded systems control many of the common devices in use today.
Since the embedded system is dedicated to specific tasks, design engineers can optimize it,
reducing the size and cost of the product, or increasing the reliability and performance. Some
embedded systems are mass-produced, benefiting from economies of scale.
Physically, embedded systems range from portable devices such as digital watches and MP4
players, to large stationary installations like traffic lights, factory controllers, or the systems
controlling nuclear power plants. Complexity varies from low, with a single microcontroller
chip, to very high with multiple units, peripherals and networks mounted inside a large chassis or
enclosure.
In general, "embedded system" is not an exactly defined term, as many systems have some
element of programmability. For example, Handheld computers share some elements with
embedded systems — such as the operating systems and microprocessors which power them —
but are not truly embedded systems, because they allow different applications to be loaded and
peripherals to be connected.
Embedded computing systems
• Computing systems embedded within electronic devices
• Hard to define. Nearly any computing system other than a desktop computer
• Billions of units produced yearly, versus millions of desktop units
• Perhaps 50 per household and per automobile.