08-08-2012, 01:25 PM
Study on citrus fruit image data separability by segmentation
methods
Study on citrus fruit image data separability.pdf (Size: 533.57 KB / Downloads: 41)
Introduction
Machine vision sensor scheme
In horticultural industry, the automatic fruit harvester has been investigated and researched for years
since Schertz & Brown proposed in 1968[1]. They proposed the guidance for the manipulator by a
machine vision system to detect the fruit by the photometric comparison. The concepts have been further
developed and practiced in the upcoming large projects and research[2-10]for example. The vision system
used in the automatic harvester aims to detect the fruits and provide the information of the location and
the distance to the fruits to the robotic controller. In vision system, the cameras are mainly the solution to
communicate with the environment. The major achievements of the vision systems and the performance
of the various sensors in the harvesting have been reviewed by Jimenez et al[11]. On top of the survey, an
economical portable cold mirror and bi-camera acquisition system has been reported and detailed[12] for
a multispectral photographing purpose. This data acquisition and the synchronization on two cameras
have been practiced to photograph the citrus fruit image data at the Alverstoke orchard at Waite campus
of the University of Adelaide and School of Agriculture, Food & Wine. In this work the focus is paid on
the segmentation methods behind the citrus image data.
Methods to identify the fruit
In practice, the use of threshold or index is considered as a fast way to segment the target fruit out of
the background. The physical optical filters have been selected to enhance the spectral area between fruit
and background [2, 8, 13-16]. On top of the data enhancement, the color index has been used to segment
the image fast. The index r = 3R - (G+ B) gave high contrast values for red apples[17]. An index R - B
was used on citrus fruit image segmentation by Xu and Ying[18]. A dynamic threshold value was
calculated based on the image maximum and minimum intensity to classify the citrus fruits and the
background after segmentation. Stajnko et al[19] used a thermal imaging camera to estimate the number
and diameter of the apple. A normalized difference index NDI (g - r) / (g +r) was adopted to contrast the
image[20]. The global threshold was then applied to segment the fruits form the background. Based on
the study on the spectral measurement using spectrophotometer by Kane and Lee[21], a normalized
difference index similar to NDI was applied on three waveband images instead of RGB[22].
Shape based method such as the circular Hough transform on image segmentation has some issues
from the application. The interference of background, leaves, or curvature from the other non-fruit
features would be misinterpreted. Another issue was the intensive computation by CHT[23]. The shape
analysis could be integrated with the advanced technique for the acquisition of multichannel images on
the same scene[24].
The unpredictable distribution of the empirical data has been studied and endeavoured to find a more
precise estimation method. One of the multivariate statistical pattern segmentation techniques, Bayes
parametric optimal classifier, has been widely applied on the segmentation based on posterior probability
theory[8, 25]. Ferri et al[26] presented a method based on the nearest neighbor segmentation and
multiedit-condensing technique in a vision system for citrus harvesting. The multiedit algorithm allowed
to select prototypes which belong to Bayes accepted region while the condensing algorithm attempted to
eliminate those multiedit references which embedded in the internal regions of Voronoi polygons of each
class. A attenuated K-means algorithm approach was presented by Weeks et al[27] on orange image
segmentation. About 67% orange were detected with 54% accurate detection in the results. The
PCA(Principle Component Analysis) was employed by Mehl et al for apple surface defects analysis[28].
Kane et al[21] used a spectrometer to measure the different citrus fruits through seasons. PCA was
applied to discriminate the classes between the fruits and the background. The Fisher linear discriminant
was also used to find the direction of the major eigenvector for the projection of the lower dimension
space.
On top of that, more advanced methods have been practiced such as the function approximation by
using the neural network architecture. A multilayer perceptron trained by backpropagation error algorithm
was employed on the segmentation of orange tree images[29]. In the experiment, about 88% fruits were
detected. Bulanon et al[30] reported that both artificial neural network(ANN) and decision theoretic
classifier were used on the segmentation of apples in image for the harvesting robot. Both methods
achieved 80% fruits detected. Regunathan et al[31] employed a color camera and an ultrasonic sensor
with a neural network to compare with Bayesian and Fisher linear discriminant methods. In the
experiments, the neural network performed better in fruit size estimation. Ye et al[32] employed a neural
network via backpropagation learning algorithm on airborne hyperspectral images to estimate the citrus
yield. The study indicated that the hyperspectral data in May had high correlation with the yield. Jian-jun
et al[33] studied and compared a linear vector quantization(LVQ) network based segmentation method
with other three methods: dynamic threshold segmentation, extended Otsu method, and improved Otsu
method combined with genetic algorithm on fruit image. The result showed that the traditional Otsu
method was faster than other two Otsu methods. The segmentation based on LVQ generated a poor result
due to the nonuniform reflection and the high bright area. In addition, the time cost of LVQ was higher
than Otsu methods. The author concluded that the LVQ was not adaptable for real time applications.
However, the parametric adjustment for the estimation functions in learning algorithm barrier the use of
such advanced method in real time harvesting application. On the contrary, more literature can be found
for analytical purpose or inspection purpose applications using artificial neural network[34].
Motivation of this work
From the literature, the successful fruit identification rate is averagely around 70% and up to 90% with
variation in case. To cooperate with the vision system development, the prior rich measurement has been
performed on citrus kinds through seasons to find the illumination function of the fruit and the neighbor
leave on the spectral space[21]. Therefore the cold mirror bi-camera system has been proposed and
prototyped for a multispectral acquisition purpose on citrus fruit tree. By using the function of the cold
mirror with 50% reflection in visible area and 50% transmission in NIR area, this practice attempts to
capture both the visible and the certain NIR area on the natural citrus fruit tree canopy. In addition, the
synchronization on both cameras can be practiced to solve the registration issue in dynamic spatially and
temporally. The precision of the alignment of the images from two cameras has been quantified by the
software based calibration. With flexibility of interchangeability, more combination of the citrus image
data can be collected by interchanging some physical optical attenuation filters on both cameras.
Therefore from the same channel the data for the visible area could be studied and compared between the
non-attenuated and the attenuated image data. On top of that, the use of the pairwise registered image data
considers the fusion technique by combining the visible component from one camera and the near
infrared image component from the second camera. In this study, the data from camera 1 has been studied
by some segmentation methods. The segmentation methods have been conceptually selected including
some of the color indices, Fisher linear discriminant analysis, and hyperplane construction using single
perceptron along with the multilayer perceptron, and the competitive self-organizing map. Finally the
non-attenuated and the attenuated data, and the accuracy of segmentation methods are compared in this
paper.