09-10-2012, 01:19 PM
Flower Recognition
flowerrecognition.pdf (Size: 2.76 MB / Downloads: 139)
Introduction
Imagine someone hiking in the Swiss mountains who nds a beautiful ower.
This person has always been bad in biology but would like to know more about
that ower. What's its name? Its main features? Is it rare? Is it protected?
etc. By simply taking a picture of the ower with a phone, he or she could get
all these informations through an automatic ower recognition application.
As part of the semester project, the elaboration of such an application has
been aimed. The recognition of owers from photographs implies several steps,
starting with the localization of the ower in the image, followed by identifying
and extracting the specic characteristics of this ower, and nally nding the
best match.
The solution proposed in this report includes the following elements; specic
research on plants and on existing method, a segmentation algorithm based
on user's inputs, implementation of several visual features suitable for owers
dierentiation. It was eventually implemented in android phone.
Other applications could include educational purposes and nature preservation
programs.
State of the art
Some research conducted in ower recognition and general features extraction
are listed in this section.
In 1999, Das et al. [1] suggested to use domain knowledge of owers colors
to index the images. In this context they developed an iterative segmentation
algorithm to isolate the ower from the background. There are indeed colors
that are rarely present in owers and the background of photographs taken in
the nature could be quite similar. They used only color names and their relative
proportions within the ower region as features which is a good but not sucient
mean for full recognition.
Then, Saitoh and Kaneko [2] have proposed a method that uses two input
images, one of the ower and one of the leaf. In order to do so the user should
place a black cloth behind the ower which is not so convenient. And even with
this approach the background separation isn't straightforward, they actually
used a k-means clustering method in color space (with multiple integrations).
Feature-wise they considered color and shape informations for both the ower
and the leaf.
Some interactive methods such as CAVIAR (Computed Assisted InterActive
Recognition) [3] have been developed [4] to exploit the human perceptibility. A
rose-curve is generated for the test ower and the top three candidates are
proposed to the user who can then either select the right ower (if present) or
modify the rose-curve to get a new set of propositions. This is done iteratively
until the right ower is found, which is not suitable for our application because
of these multiple user interactions.
Biology point of view
Importance
In items recognition research a lot have been done about general features extraction
or recognition between dierent classes of objects. In case of a specic
domain recognition, taking into account the unique characteristics that belong
to this category may very much improves the performance of the system.
Despite the high technical aspect of this project, dealing with owers gives
it a biological connotation. Some basic knowledges about owers have to be
learned and concepts about how the biologists themselves recognize ower to
be studied. To this purpose some books have been borrowed and a meeting
with a real biologist was booked. The two next paragraphs are devoted to these
experiences.
Books
Dierent books have been loaned from the EPFL library in Lausanne. Aspects
of morphogenesis of leaves, owers and somatic embryos is a documentation
edited by the National Botanic Garden of Belgium and treats advanced themes
as genetic aspects of ower development or molecular aspects of somatic embryogenesis
in conifers. Organization des plantes à eurs also appears to be
quite a bit too much micro-biology oriented, a good nd to note anyway: the
plants with owers are part of the class named the Angiosperms which hold the
majority of the plants on earth with more than 200'000 species! The last book
entitled Botanique Systématique des plantes à eurs relates the history of the
botanic classication, the evolutions of dierent parts of the owers through time
before providing descriptive tables of families. A family, as the one described
in Figure 1 holds more than 400 genus and about 12'000 species. Looking at
the denition of the family gives a good idea about the complexity of dening a
plant, e.g. under the section ower more than 20 qualicatifs are used (such as
cyclic, polystemon, shortened sepals, staminodes present, etc.) and by looking
at the other page it seems the plants are looking completely dierent despite
being in the same family.
Meeting with biologist
A meeting with Pascal Vittoz, biologist and Chief of Research Project in UNIL,
has been arranged on the 23rd of March. He rst of all introduced the categorization
of owers, organized in a hierarchy : Embranchement, Class, Family,
Genius and nally Species. He then explained a few methods used on the eld
to recognize owers, one of them is the use of a key i.e. a guide with a set of
iterative questions with multiple answers that leads to the wanted ower. This
Swiss book is called Flora Helvetica and a sample is exhibited in Figure 2.
System Overview
The whole process starts with the user taking a picture of a ower, after having
indicated the ower region (see Section 5 ), these informations are sent to the
server. Then the segmentation is done, it outputs the mask of the ower which
is a binary image with ones on the ower region and zeros in the background.
The original image and the mask are needed for the features extraction which
stores the values for each feature separately. Then the data are compared to the
database's for the matching. Finally the result is sent back to the user. This
scheme is illustrated in Figure 6.
Application
One can think of using this markers facility in an iterative way i.e. the result of
segmentation can be shown to the user and if the result does not satisfy him,
he could add new markers, then the result is shown, and so on until he's happy.
The main drawback of the watershed, and not a negligible one, is that the
whole database has to be treated manually. Indeed to extract the features the
mask output by the segmentation is needed and thus has to be done a priori.
This could represent a very tiring job for a large database !
Note that this algorithm is already implemented in OpenCV which is this
time a signicant advantage.
Features
This section lists the all 18 features extracted for the recognition. First a quick
explanation about the FELib is presented, then the features have been grouped
by dierent categories: the features based on colors (which may also come from
the Felib), the features based on the contour of the ower, the features based
on the texture and nally two others. This section is structured according to
this organization which is illustrated in Figure 10.