22-09-2016, 12:33 PM
1455691225-ENHANCEMENT.pdf (Size: 2.27 MB / Downloads: 6)
Abstract— Latent fingerprint identification plays an important
role for identifying and convicting criminals in law enforcement
agencies. Latent fingerprint images are usually of poor quality
with unclear ridge structure and various overlapping patterns.
Although significant advances have been achieved on developing
automated fingerprint identification system, it is still challenging
to achieve reliable feature extraction and identification for latent
fingerprints due to the poor image quality. Prior to feature
extraction, fingerprint enhancement is necessary to suppress various
noises, and improve the clarity of ridge structures in latent
fingerprints. Motivated by the recent success of sparse representation
in image denoising, this paper proposes a latent fingerprint
enhancement algorithm by combining the total variation model
and multiscale patch-based sparse representation. First, the total
variation model is applied to decompose the latent fingerprint into
cartoon and texture components. The cartoon component with
most of the nonfingerprint patterns is removed as the structured
noise, whereas the texture component consisting of the weak
latent fingerprint is enhanced in the next stage. Second, we
propose a multiscale patch-based sparse representation method
for the enhancement of the texture component. Dictionaries
are constructed with a set of Gabor elementary functions to
capture the characteristics of fingerprint ridge structure, and
multiscale patch-based sparse representation is iteratively applied
to reconstruct high-quality fingerprint image. The proposed
algorithm cannot only remove the overlapping structured noises,
but also restore and enhance the corrupted ridge structures.
In addition, we present an automatic method to segment the
foreground of latent image with the sparse coefficients and
orientation coherence. Experimental results and comparisons on
NIST SD27 latent fingerprint database are presented to show the
effectiveness of the proposed algorithm and its superiority over
existing algorithms.
INTRODUCTION
LATENT fingerprints are the finger skin impressions left
at the crime scene by accident. Usually, such impressions
are not directly visible to human eyes unless some physical or
chemical techniques are used to process and enhance them
Latent fingerprints have been used as an important evidence
to identify criminals in law enforcement agencies for more
than a century. Before introduction of automated fingerprint
identification system (AFIS), latent fingerprints were manually
matched against previously enrolled full (rolled or plain) fingerprints
by latent examiners to find the suspects [2], [3]. The
emergence of AFIS significantly improved the speed of fingerprint
identification and made the latent identification against
a large fingerprint database feasible. After over thirty years
of development, tremendous advances have been made on
developing AFIS for full print to full print matching [4]–[10].
However, compared to the rolled and plain fingerprints, latent
fingerprints are usually of low image quality, caused by unclear
ridge structure, uneven image contrast, and various overlapping
patterns such as lines, printed letters, handwritings or
even other fingerprints, etc. [11]. Fig. 1 (a), (b) and © show a
sample of the rolled, plain and latent fingerprints, respectively.
Due to the low image quality, automatic feature extraction is
still undesirable for latent fingerprints and features (such as
minutiae and singular points) need to be manually marked
by latent examiners for identification [11]. However, manual
markup of minutiae features is not only time-consuming but
also short of repeatability and compatibility [12]. First, the
minutiae features in the same fingerprint marked by different
latent examiners or by the same examiner but at different times
may not be same, which results in making different matching
decisions on the same latent-exemplar pair [12]. Second, in
current practice, minutiae features in latent fingerprints are
manually marked while the minutiae features in enrolled
fingerprints are automatically extracted, which may cause a
compatibility problem [11]. Thus, manually marking minutiae
features is not the best solution for latent fingerprint
identification. Before input to AFIS, latent fingerprints need
to go through an enhancement stage which removes various
overlapping patterns, connects broken ridges and separates
joined ridges [13].
The interleaved ridge and valley flows of fingerprint form
a sinusoidal-shaped plane wave with well-defined frequency and orientation in a local neighborhood. A number of methods
were proposed to take advantage of this information to
enhance the poor quality fingerprints [14]–[17]. A Gabor filter,
which is defined with a sinusoidal plane wave tapered by a
Gaussian, can capture the periodic, yet non-stationary nature
of fingerprint ridge structure. Gabor filtering is widely used for
fingerprint enhancement [14], [15]. In this method, the local
ridge orientation and frequency are first estimated at each pixel
based on a local neighborhood. Then a Gabor filter is tuned to
the local orientation and frequency and applied on the image
pixel to suppress the undesired noise and improve the clarity
of ridge structure. This method requires reliable estimation of
local ridge orientation and frequency, which is challenging
for poor quality fingerprint. Another interesting technique
based on Short Time Fourier Transform (STFT) analysis
was proposed to perform contextual filtering in the Fourier
domain for fingerprint enhancement [16]. The traditional 1D
(one dimensional) time-frequency analysis is extended to 2D
fingerprint images for short (time/space)-frequency analysis.
The probabilistic estimates of the foreground region mask,
ridge orientation and frequency are simultaneously computed
from STFT analysis. The full contextual information including
local orientation, frequency and angular coherence is utilized
for fingerprint enhancement.
The above enhancement algorithms were usually developed
for the rolled and plain fingerprints. They cannot work well
on latent images with difficulty in two aspects. First, since
the latent fingerprint impression is usually left by accident,
the latent fingerprint itself is of poor quality with smudged,
blurred or broken ridge structure. Second, the latent fingerprint
is usually overlapped with various types of non-fingerprint
patterns such as arch, line, character, handwriting, speckle and
stain etc., which are considered as structured noises. Compared
to the relatively weak fingerprint pattern, the structured noises
are usually of much larger scale in various forms and form the
dominant image component. Due to these difficulties, reliable
estimation of the contextual information such as the local ridge
orientation and frequency is challenging and the enhancement
performance is far from satisfactory with the above algorithms
for latent fingerprints.
Some smoothing and global modeling methods have been
proposed to enable reliable estimation of orientations for
enhancement of latent fingerprints [2], [13], [18]. A robust
method was proposed for orientation field estimation to
improve the performance of latent fingerprint enhancement
with Gabor filters [18]. This method applied STFT method
to obtain multiple orientation elements in each image block,
and a set of hypothesized orientation fields were generated
with a hypothesize-and-test paradigm based on randomized
RANSAC. It required manual markup of singular points
and the region of interest (ROI) for enhancement. Recently,
Feng et al. [13] proposed a robust method to estimate orientation
field using the prior knowledge of fingerprint ridge
structure, which is represented by a dictionary of reference
orientation patches and the compatibility constraint between
neighboring orientation patches. The fingerprint orientation
field was computed by a combination of candidates which
minimizes an energy function using loopy belief propagation With robust orientation estimation, Gabor filtering was applied
to achieve improvement for enhancement of latent fingerprints.
However, the above methods set a fixed ridge frequency to
tune Gabor filters for enhancement. In practical applications,
it is not always constant in fingerprint image, which limits the
enhancement performance of Gabor filtering.
The total variation (TV) image models, which aim at
minimizing the total variation of an image, have been widely
studied for image decomposition [19]–[22]. Usually, the TV
model decomposes an image into two components: texture
and cartoon. The texture component is characterized
as repeated, oscillatory and meaningful structure of small
patterns. Noise is characterized as uncorrelated random patterns.
The rest of an image, i.e., the cartoon component,
consists of the object hues, sharp edges and piecewise-smooth
components. Zhang et al. [23] proposed an adaptive TV model
to remove the structured noises for latent fingerprint segmentation.
They further proposed an adaptive directional total
variation (ADTV) model by integrating the local orientation
and scale for fingerprint segmentation and enhancement [21].
These TV based methods decompose latent image into
texture and cartoon components. The texture component
mainly consists of the oscillatory fingerprint ridge patterns
while the cartoon component contains the left unwanted contents,
i.e., structured noises. Latent fingerprint segmentation
and enhancement are performed on the texture component
with the structured noises removed. However, it is not easy to
reliably estimate the local parameters (i.e., ridge orientation
and scale) of ADTV model for latent images of poor quality.
In addition, the noise corrupted regions are not restored and
the extracted fingerprint pattern is usually very weak, which
will limit the performance of latent fingerprint identification.
A dictionary-based method was proposed to enable reliable
estimation of ridge orientation and frequency fields and facilitate
the automatic segmentation and enhancement of latent
fingerprints [22]. The TV model was first applied to remove
the piecewise-smooth and structured noises. Then, both coarse
and fine ridge structure dictionaries were learnt from a set
of high quality fingerprint patches and used to reconstruct
the ridge structure of latent image. Finally, the orientation
and frequency fields were estimated with the reconstructed
patches and used for latent fingerprint enhancement by Gabor
filtering. Although reliable estimation of ridge orientation and
frequency can improve the performance of latent fingerprint
enhancement, there are two inherent limitations in these methods.
First, in the regions of high curvature, the assumption
of a single dominant ridge orientation is not valid. As a
result, the Gabor filters with fixed orientation will be likely to
destroy the ridge structure and lead to spurious ridge artifacts.
Second, although the Gabor filtering with correct orientation
and frequency parameters can work well to enhance the ridge
clarity, it fails to restore the ridge structure destroyed by heavy
structured noises.
The main challenging problem for latent fingerprint
enhancement is to remove various types of image noises while
reliably restoring the corrupted regions and enhancing the
ridge clarity and details. Sparse representation on redundant
dictionary is a promising method for image reconstruction especially from the noisy image. As a powerful statistical
image modeling technique, sparse representation has been
successfully used in various image processing and recognition
applications [24]–[26]. Motivated by the recent success of
sparse representation in image denoising [25], [27], this paper
proposes a latent fingerprint enhancement algorithm via multiscale
patch based sparse representation, which consists of
two main stages. First, the TV model is used to decompose
latent image into cartoon and texture components. The
cartoon component with most of the irrelevant contents is
discarded, while the texture component contains the weak
latent fingerprint and is extracted for further enhancement.
Second, instead of using Gabor filtering, a set of Gabor
elementary functions with various parameters are used to build
the basis atoms of dictionary, and the texture component is
reconstructed via sparse representation for latent fingerprint
enhancement. The patch size is a critical parameter for fingerprint
reconstruction via sparse representation. Large patch
can suppress the noise better while small patch can preserve
the details of ridge structure. To achieve both noise robustness
and detail preserving, we propose a multi-scale patch based
sparse representation by gradually increasing the patch size
and dictionary scale for iterative reconstruction of high quality
fingerprint.
The dictionary based method has the advantage in utilizing
the prior knowledge of ridge structures in fingerprints.
The ridge structure dictionaries have been investigated to
reliably estimate orientation and frequency fields for fingerprint
enhancement and segmentation [13], [22]. In these
methods, the dictionaries are learned from a set of high quality
fingerprint patches and then used to restore the ridge structure
in latent patches, which facilitates the reliable estimation of
local ridge orientation and frequency for fingerprint enhancement
by Gabor filtering. Our proposed method is different
from these methods in two aspects. First, instead of learning
the dictionary from high quality fingerprints, our method
generates the dictionary atoms with a set of Gabor functions,
which not only features a fast implicit implementation but
also has high adaptivity. If we change the patch size, the
dictionary atoms can be easily adapted by varying the scale
parameter and the corresponding atoms (to the same column)
on different scales can be generated with the same orientation
and frequency parameters. These facilitate the multi-scale
iterative enhancement by gradually and adaptively increasing
the patch size and dictionary scale to preserve the ridge details
and restore the noise corrupted regions. On the other hand, the
dictionary learned from the image patch itself may provide
various representative ridge structures, but it needs to be
relearned if we change the patch size and the corresponding
dictionary atoms may have different ridge structures (i.e., with
respect to different orientation and frequency), which limits
the application of multi-scale iterative sparse representation.
Second, instead of enhancing the latent fingerprint with Gabor
filtering, our proposed method gradually and iteratively reconstructs
the high quality fingerprints with the sparse coefficients
and Gabor dictionaries. Gabor filtering with fixed orientation
and frequency not only fails to enhance the regions of high
ridge curvature with abrupt changes, but also cannot work well to restore the ridge structure destroyed by heavy structured
noises. As shown in Fig. 2, although the orientations in the
corrupted area by digit ‘9’ are correctly estimated with the
method [13] (see Fig. 2a), the corrupted ridge structure is not
well restored by Gabor filtering (see Fig. 2b). Our proposed
multi-scale iterative method can solve this problems by gradually
and adaptively increasing the patch size and dictionary
scale so that the ridge structure in high-curvature area is well
reconstructed while the corrupted areas are correctly restored
(see Fig. 2c). The rest of the paper is organized as follows.
Section II presents in detail the proposed latent fingerprint
enhancement algorithm as well as the automatic segmentation
method. Experimental results and comparison are provided in
Section III. Finally, we conclude the paper and suggest future
directions in Section IV.
II. LATENT FINGERPRINT ENHANCEMENT
A. Method Overview
Latent fingerprint images are usually of poor quality,
corrupted by various kinds of noises and overlapped with
non-fingerprint patterns. The objective of latent fingerprint
enhancement is to remove the irrelevant contents while
enhancing the clarity of fingerprint ridge and valley structure,
which can be considered as an image denoising and
restoration problem. To address this problem, we propose
a latent fingerprint enhancement algorithm which includes
image decomposition and fingerprint reconstruction (see the
flowchart in Fig. 3). First, the TV model is applied to
decompose latent image into texture and cartoon components.
The cartoon component, which contains most of the irrelevant
contents and structured noises, is discarded. Second,
latent fingerprint image is reconstructed via multi-scale sparse
representation from the texture component. The fingerprint
image is initially reconstructed with sparse representation on
small patch to preserve the ridge details. To restore the noise corrupted ridge structure, a multi-scale patch based sparse
representation method is proposed to iteratively reconstruct
high-quality fingerprint image by gradually increasing the
patch size. In addition to the manually marked region of
interest (ROI), we also propose an automatic segmentation
method to generate ROI for enhancement. In the following
sections, we will present the details of the main processing
steps.