15-10-2016, 02:17 PM
[attachment=73109]
ABSTRACT
A novel mechanism to simulate visual attention mechanisms for content-based image retrieval, based on saliency structure histogram method was proposed in this paper. In CBIR, images are indexed by their visual content, such as color, texture, shapes. A color volume with edge information together is used to detect saliency regions. The texture image features, such as energy, inverse difference moment, contrast are extracted. To simulate orientation-selective mechanism for image representation within CBIR framework, saliency structure histogram is used. The performances of the proposed algorithm was evaluated based on two datasets. The proposed algorithm outperforms the standard BOW baseline and micro-structure descriptor.
1.Introduction
Image retrieval is the process of searching and retrieving image from a dataset, but the extraction of features from the image data is a challenging problem. Images contain a rich variety of semantic information.
Human's visual system[1,2] has visual attention mechanism that helps humans to select the highly relevant information from a dataset. Content-based image retrieval (CBIR) can benefit from visual attention mechanisms by using the saliency information.
Image retrieval techniques can be classified into two categories: (1) the methods based on global features and (2) the methods based on local features. In image retrieval the extraction of the primary visual features is a challenging one. Current CBIR technologies are mainly based on global features (e.g., color, texture, edges and spatial information).
In earlier works, micro-structures model [3] is developed for content-based image retrieval. Micro-structures are defined as the collection of certain underlying colors, where the idea of micro-structures model derived from Treisman's feature integration theory and Julesz’ texton theory[4]. Micro-structures model has not simulated visual attention mechanisms well and also saliency orientation mechanism. In standard BOW[5] baseline object-based image retrieval, object recognition achieve good performance, but the visual words obtained by using the vector quantization of local features descriptors results in the loss of information. This can be used in object based image retrieval rather than CBIR. To address this problem, a novel computational visual-attention model, namely saliency structure model, for content-based image retrieval is proposed. There are three highlights in this model: (1) a novel visual cue, namely color volume, with edge information together is used to detect saliency regions. (2) the energy feature of texture image features such as energy, inverse difference is used for globally suppressing maps (3) A novel method, namely saliency structure histogram, is proposed to stimulate orientation-selective mechanism for image representation.
2. Related works
In the following subsections, a review of visual attention models, image retrieval, feature extraction and image representation are given.
2.1. Visual attention model
It follows the bottom-up model based on the characteristics of a visual scene. In saliency model, the image visual features such as color, intensity and orientation are extracted [8]. The features are computed by a set of center-surround operation. Finally, various features are combined into a saliency map.
2.2 Techniques related to Image Retrieval
The image retrieval techniques are based on two global features. It aims at the whole image as visual content, e.g. color, texture. Texture features can be combined with color feature to improve the discrimination power to obtained better performance.
3. Energy features
In this proposed saliency model ,Haralick's gray level co-occurrence matrix[9], is used to extract a set of texture images features, such as energy, inverse difference moment, contrast, entropy etc.,
4. The Saliency Structures Model and Descriptor
Saliency structure model is proposed to content-based image retrieval according to Treisman's feature integration theory and Julesz’ texton theory[4]. In feature extraction and image representation, Orientation-selective mechanism which derived from the works of Hubel and Wiesel is used in this model. The primary features are Color, intensity and orientation. To detect saliency regions color volume and edge information detection are used.
In the proposed saliency model within CBIR framework focuses on: (1) the construction of saliency structure model (2) image representation. Where the construction of saliency structure model mainly consists of three stages: (a) extraction of the primary visual features, (b) the saliency map detection and © the combination of bar-shaped structure and oriented Gabor filters for saliency structure detection.
4.1 Extraction of the Primary Visual Features
The primary features are implemented in HSV color space. H, S and V color channels are uniform quantized into 6, 3 and 3 bins, totally 6x3x3=54 color combinations are obtained, MC (x, y) denotes the color combinations or color map, as MC (x, y)= w w ϵ{0, 1, …, NC -1}, where NC = 54 in this paper.Intensity information is obtain the intensity map M1 (x, y), as M1 (x, y)= s; sϵ{0, 1, …, N1 -1}, where N1 = 16..In this system, intensity information is also used to detect edge orientation map O(x, y) and gradient image g (x, y) by using Sobel operator. After uniform quantization to obtain the edge orientation map MO (x, y), as MO (x, y), = θ, θ ϵ{0, 1, …, NO -1}, where NO = 60.
In this system, MC (x, y), M1 (x, y) and MO(x, y) used to detect saliency structures and image representation.
4.2 Saliency Map, Oriented Gabor filters and structure Histogram
In proposed system the shape of the HSV color space can be interpreted as cylinder coordinate point cv=πr2h’, where r denotes radius and h denotes height of the cylinder.
Gabor filters are used to detect saliency structures and describe image features for further processing.
Bar-shaped structures are defined as three consecutive adjacent pixels which have the same pixel values.
5. Experiments
In this section, the performances of the proposed algorithm are evaluated on two datasets .In experiments, images randomly chosen, were used as query images, and the system performs the similarity evaluation with respect to each query image.
5.1 Distance metric
Distance metric between feature vector and query image is calculated as
Example:
1 1 2 2 4 4 4 4
2 3 3 3 1 1 2 2
3 3 4 4 3 3 4 4
4 4 4 4 2 3 3 3
2 3 5 6 2 3 5 6
DH = 1 – (2+3+5+6)/min(16,16) =1-16/16=1-1 = 0
5.2 Performance metrics
In the field of information retrieval, two primary metrics are precision and recall. The two metrics are often combined as the weighted harmonic mean, F-measure[10,11]. In the experiments of image retrieval, precision(P) is the ratio of the number of retrieved similar images to the number of retrieved images, while recall® is the ratio of the number of retrieved similar images to the total number of similar images.
6. Conclusion
In this paper a computational visual attention model to improve the performances of content-based image retrieval is introduced. Color volume is used to detect saliency regions and energy features are used for suppressing the maps globally. An oriented Gobar filters are embedded into bar-shaped structures to simulate orientation-selective mechanism and image representation.
The features like color, edge-orientation and intensity informations are mapped into histogram. The proposed algorithm is a bottom-up component of visual attention and orientation-selective mechamism.
But BOW techniques can be extended in future work with saliency model within CBIR framework.
ABSTRACT
A novel mechanism to simulate visual attention mechanisms for content-based image retrieval, based on saliency structure histogram method was proposed in this paper. In CBIR, images are indexed by their visual content, such as color, texture, shapes. A color volume with edge information together is used to detect saliency regions. The texture image features, such as energy, inverse difference moment, contrast are extracted. To simulate orientation-selective mechanism for image representation within CBIR framework, saliency structure histogram is used. The performances of the proposed algorithm was evaluated based on two datasets. The proposed algorithm outperforms the standard BOW baseline and micro-structure descriptor.
1.Introduction
Image retrieval is the process of searching and retrieving image from a dataset, but the extraction of features from the image data is a challenging problem. Images contain a rich variety of semantic information.
Human's visual system[1,2] has visual attention mechanism that helps humans to select the highly relevant information from a dataset. Content-based image retrieval (CBIR) can benefit from visual attention mechanisms by using the saliency information.
Image retrieval techniques can be classified into two categories: (1) the methods based on global features and (2) the methods based on local features. In image retrieval the extraction of the primary visual features is a challenging one. Current CBIR technologies are mainly based on global features (e.g., color, texture, edges and spatial information).
In earlier works, micro-structures model [3] is developed for content-based image retrieval. Micro-structures are defined as the collection of certain underlying colors, where the idea of micro-structures model derived from Treisman's feature integration theory and Julesz’ texton theory[4]. Micro-structures model has not simulated visual attention mechanisms well and also saliency orientation mechanism. In standard BOW[5] baseline object-based image retrieval, object recognition achieve good performance, but the visual words obtained by using the vector quantization of local features descriptors results in the loss of information. This can be used in object based image retrieval rather than CBIR. To address this problem, a novel computational visual-attention model, namely saliency structure model, for content-based image retrieval is proposed. There are three highlights in this model: (1) a novel visual cue, namely color volume, with edge information together is used to detect saliency regions. (2) the energy feature of texture image features such as energy, inverse difference is used for globally suppressing maps (3) A novel method, namely saliency structure histogram, is proposed to stimulate orientation-selective mechanism for image representation.
2. Related works
In the following subsections, a review of visual attention models, image retrieval, feature extraction and image representation are given.
2.1. Visual attention model
It follows the bottom-up model based on the characteristics of a visual scene. In saliency model, the image visual features such as color, intensity and orientation are extracted [8]. The features are computed by a set of center-surround operation. Finally, various features are combined into a saliency map.
2.2 Techniques related to Image Retrieval
The image retrieval techniques are based on two global features. It aims at the whole image as visual content, e.g. color, texture. Texture features can be combined with color feature to improve the discrimination power to obtained better performance.
3. Energy features
In this proposed saliency model ,Haralick's gray level co-occurrence matrix[9], is used to extract a set of texture images features, such as energy, inverse difference moment, contrast, entropy etc.,
4. The Saliency Structures Model and Descriptor
Saliency structure model is proposed to content-based image retrieval according to Treisman's feature integration theory and Julesz’ texton theory[4]. In feature extraction and image representation, Orientation-selective mechanism which derived from the works of Hubel and Wiesel is used in this model. The primary features are Color, intensity and orientation. To detect saliency regions color volume and edge information detection are used.
In the proposed saliency model within CBIR framework focuses on: (1) the construction of saliency structure model (2) image representation. Where the construction of saliency structure model mainly consists of three stages: (a) extraction of the primary visual features, (b) the saliency map detection and © the combination of bar-shaped structure and oriented Gabor filters for saliency structure detection.
4.1 Extraction of the Primary Visual Features
The primary features are implemented in HSV color space. H, S and V color channels are uniform quantized into 6, 3 and 3 bins, totally 6x3x3=54 color combinations are obtained, MC (x, y) denotes the color combinations or color map, as MC (x, y)= w w ϵ{0, 1, …, NC -1}, where NC = 54 in this paper.Intensity information is obtain the intensity map M1 (x, y), as M1 (x, y)= s; sϵ{0, 1, …, N1 -1}, where N1 = 16..In this system, intensity information is also used to detect edge orientation map O(x, y) and gradient image g (x, y) by using Sobel operator. After uniform quantization to obtain the edge orientation map MO (x, y), as MO (x, y), = θ, θ ϵ{0, 1, …, NO -1}, where NO = 60.
In this system, MC (x, y), M1 (x, y) and MO(x, y) used to detect saliency structures and image representation.
4.2 Saliency Map, Oriented Gabor filters and structure Histogram
In proposed system the shape of the HSV color space can be interpreted as cylinder coordinate point cv=πr2h’, where r denotes radius and h denotes height of the cylinder.
Gabor filters are used to detect saliency structures and describe image features for further processing.
Bar-shaped structures are defined as three consecutive adjacent pixels which have the same pixel values.
5. Experiments
In this section, the performances of the proposed algorithm are evaluated on two datasets .In experiments, images randomly chosen, were used as query images, and the system performs the similarity evaluation with respect to each query image.
5.1 Distance metric
Distance metric between feature vector and query image is calculated as
Example:
1 1 2 2 4 4 4 4
2 3 3 3 1 1 2 2
3 3 4 4 3 3 4 4
4 4 4 4 2 3 3 3
2 3 5 6 2 3 5 6
DH = 1 – (2+3+5+6)/min(16,16) =1-16/16=1-1 = 0
5.2 Performance metrics
In the field of information retrieval, two primary metrics are precision and recall. The two metrics are often combined as the weighted harmonic mean, F-measure[10,11]. In the experiments of image retrieval, precision(P) is the ratio of the number of retrieved similar images to the number of retrieved images, while recall® is the ratio of the number of retrieved similar images to the total number of similar images.
6. Conclusion
In this paper a computational visual attention model to improve the performances of content-based image retrieval is introduced. Color volume is used to detect saliency regions and energy features are used for suppressing the maps globally. An oriented Gobar filters are embedded into bar-shaped structures to simulate orientation-selective mechanism and image representation.
The features like color, edge-orientation and intensity informations are mapped into histogram. The proposed algorithm is a bottom-up component of visual attention and orientation-selective mechamism.
But BOW techniques can be extended in future work with saliency model within CBIR framework.