05-11-2012, 02:34 PM
Sketch4Match – Content-based Image Retrieval System Using Sketches
Sketch4Match.pdf (Size: 439.44 KB / Downloads: 119)
Abstract
The content based image retrieval (CBIR) is one of
the most popular, rising research areas of the digital image processing.
Most of the available image search tools, such as Google
Images and Yahoo! Image search, are based on textual annotation
of images. In these tools, images are manually annotated with
keywords and then retrieved using text-based search methods.
The performances of these systems are not satisfactory. The goal
of CBIR is to extract visual content of an image automatically,
like color, texture, or shape.
This paper aims to introduce the problems and challenges
concerned with the design and the creation of CBIR systems,
which is based on a free hand sketch (Sketch based image
retrieval – SBIR). With the help of the existing methods, describe
a possible solution how to design and implement a task spesic
descriptor, which can handle the informational gap between a
sketch and a colored image, making an opportunity for the
efcient search hereby. The used descriptor is constructed after
such special sequence of preprocessing steps that the transformed
full color image and the sketch can be compared. We have
studied EHD, HOG and SIFT. Experimental results on two
sample databases showed good results. Overall, the results show
that the sketch based system allows users an intuitive access to
search-tools.
INTRODUCTION
Before the spreading of information technology a huge
number of data had to be managed, processed and stored.
It was also textual and visual information. Parallelly of the
appearance and quick evolution of computers an increasing
measure of data had to be managed. The growing of data
storages and revolution of internet had changed the world. The
efciency of searching in information set is a very important
point of view. In case of texts we can search exibly using
keywords, but if we use images, we cannot apply dynamic
methods. Two questions can come up. The rst is who yields
the keywords. And the second is an image can be well
represented by keywords.
The Purpose of the System
Even though the measure of research in sketch-based image
retrieval increases, there is no widely used SBIR system. Our
goal is to develop a content-based associative search engine,
which databases are available for anyone looking back to
freehand drawing. The user has a drawing area, where he can
draw all shapes and moments, which are expected to occur in
the given location and with a given size. The retrieval results
are grouped by color for better clarity. Our most important task
is to bridge the information gap between the drawing and the
picture, which is helped by own preprocessing transformation
process. In our system the iteration of the utilization process is
possible, by the current results looking again, thus increasing
the precision.
The Preprocessing Subsystem
The system was designed for databases containing relatively
simple images, but even in such cases large differences can
occur among images in le size or resolution. In addition,
some images may be noisier, the extent and direction of
illumination may vary (see Fig. 3), and so the feature vectors
cannot be effectively compared. In order to avoid it, a multistep
preprocessing mechanism precedes the generation of
descriptors.
The Feature Vector Preparation Subsystem
In this subsystem the descriptor vectors representing the
content of images are made. Basically three different methods
were used, namely the edge histogram descriptor (EHD) [4],
the histogram of oriented gradients (HOG) [2] and the scale
invariant feature transform (SIFT) [16].
Our system works with databases containing simple images.
But even in such cases, problems can occur, which must be
handled. If the description method does not provide perfect
error handling, that is expected to be robust to the image
rotation, scaling and translation. Our task is to increase this
safety.
Another problem was encountered during the development
and testing. Since own hand-drawn images are retrieved, an
information gap arises between retrieved sketch and color
images of database. While an image is rich of information,
in contrast at a binary edge image only implicit content and
explicit location of pixels can be known.
The Displaying Subsystem
Because drawings are the basis of the retrieval, thus a
drawing surface is provided, where they can be produced. Also
a database is needed for search, which also must be set before
the search. In case of large result set the systematic arrangement
of search results makes much easier the overviews, so it
is guaranteed. The methods in our system cannot work without
parameters, and therefore an opportunity is provided to set
these as well.
The number of results to show in the user interface is an
important aspect. Prima facie the rst n pieces of results
can be displayed, which conveniently can be placed in the
user interface. This number depends on the resolution of
the monitor, and forasmuch the large resolution monitors are
widely used, so this number can move between 20 and 40.
Another approach is to dene the maximum number of results
(n), but we also observe that how the goodness of individual
results can vary. If the retrieval effectiveness is worse by only
a given ratio, the image can be included in the display list.
CONCLUSIONS
Among the objectives of this paper performed to design,
implement and test a sketch-based image retrieval system. Two
main aspects were taken into account. The retrieval process has
to be unconventional and highly interactive. The robustness of
the method is essential in some degree of noise, which might
also be in case of simple images.
The drawn image without modication can not be compared
with color image, or its edge representation. Alternatively a
distance transform step was introduced. The simple smoothing
and edge detection based method was improved, which had a
similar importance as the previous step.