23-08-2014, 04:33 PM
Super Resolution
Super.docx (Size: 2.16 MB / Downloads: 11)
1. Introduction
Super Resolution is estimate the high resolution image form low resolution input [1].The need of high resolution is in common in computer vision application for better performance. Super Resolution is used in wide range of applications like medical imaging, remote sensing, forensic imaging, satellite imaging. In super resolution there are mainly two techniques are used 1. Multi-frame super resolution2.single-frame super resolution and mainly three terminologies are used that is 1. Low resolution 2.High resolution 3.Super resolution. Super-resolution (SR) image reconstruction is currently a very active area of research, as it offers the promise of overcoming some of the inherent resolution limitations of low-cost imaging sensors (e.g. cell phone or surveillance cameras) allowing better utilization of the growing capability of high-resolution displays (e.g. high-definition LCDs)[2].
The resolution of an image is lost due to different factors such as limited hardware, noise, scaling, defocus, etc. The immediate form of augmenting the resolution of an image is by using sensors of greater resolution, i.e. having a greater number of small-size photo sensors. This, however, implies a greater cost. Another, more economic form of augmenting the resolution of images is using algorithms especially design for this task, which apart from obtaining images with a resolution similar to that obtained from a high quality sensor, they must be fast and robust.
There are mainly three ways to increasing image resolution 1. Reduce pixel size 2. Increase the chip size 3. Super resolution. The very first step of super resolute on is to register the images.
2. Basis of super resolution
2.1 Classification of Algorithms:
There are a number of different algorithms developed to perform super-resolution reconstruction. Here some of the algorithms are listed 1.non-uniform interpolation, 2.frequency domain deterministic and stochastic regularization, 3.Projection onto convex sets(POCS), 4.hybrid techniques, 5.optical flow and other approaches[4].
2.1.1 Non uniform interpolation
In non uniform interpolation there are basic three steps
1) Registration
2) Interpolation
3) De blurring
2.1.1.1 Registration
Estimating the completely arbitrary in real world image scenes is extremely difficult with almost no guarantees of estimator performance. Incorrect estimates of motion have disastrous implications on overall SR performance.
2.1.1.2 Interpolation
Since the shifts between the LR images are arbitrary, the images will not always match up to a uniformly to the HR grid. Non uniform interpolation is obtaining a uniformly spaced HR image from a none uniformly spaced of LR images. No uniform interpolation between LR images are used to improve resolution
2.1.1.3 De blurring
In SR, blur is usually modeled as a spaced averaging operator as shown below.
2.1.2 Frequency domain deterministic and stochastic regularization
This approach was proposed by Tsai and Huang [5] where aliasing in the low resolution images is used to reconstruct high resolution image. The relationship between low resolution images and the high resolution image is described by them using relative motion between the low resolution images. This approach is based on following three principles,
i) The shifting property of Fourier transforms.
ii) The aliasing relationship between the Continuous Fourier transform (CFT) of an Original HR image and the discrete Fourier Transform (DFT) of observed LR images.
iii) The assumption that an original HR image is band- limited It is thus possible to formulate the system equation relating the aliased DFT Coefficients of the observed low resolution images to a sample of the CFT of an unknown image[6].
2.1.2 Frequency domain deterministic and stochastic regularization
This approach was proposed by Tsai and Huang [5] where aliasing in the low resolution images is used to reconstruct high resolution image. The relationship between low resolution images and the high resolution image is described by them using relative motion between the low resolution images. This approach is based on following three principles,
2.1.2 Frequency domain deterministic and
stochastic regularization
This approach was proposed by Tsai and Huang [5] where aliasing in the low resolution images is used to reconstruct high resolution image. The relationship between low resolution images and the high resolution image is described by them using relative motion between the low resolution images. This approach is based on following three principles,
resolution test image, referred to as the super-resolution image (SRI) is sequentially projected onto constraint sets constructed around observed component images called low-resolution images (LRI). Such constraint sets are composed of all high-resolution images that could have produced the low-resolution observation, after the effect of the PSF, imperfect sampling and aliasing. The projection operator identifies the feasible high-resolution image that is closest to the current SRI. This algorithm uses both the spatial and frequency domains, but does most of what would ordinarily be identified as projection in the frequency domain. For our purposes, an important benefit of POCS is that the running estimate is affected by only one observation at a time. The projection operation is based on the relationship of a high-resolution image to a low-resolution one, and so it involves the smoothing effect of the image sensor, which is more efficiently computed in the frequency domain than in the spatial domain.
Additionally, the aliasing relationship is very straightforward in the frequency domain. However, in order to address the rotation of the LRI.s with respect to each other, which is not easily formulated in the frequency domain, FDSR-POCS uses resembling of the SRI in the spatial domain, which means that the SRI cannot be maintained solely in the frequency domain during the computation. Therefore we add to every image projection a reverse FFT, a spatial domain resembling operation and a forward FFT.
2.1.3.2 Spatial Domain Methods
2.1.5 Optical flow
Existing approaches to super resolution are not a videos of faces because faces are non-planer, non-rigid, non-lambertian, and are subject to self occlusion. We present super-resolution optical flow as a solution to these problems. Implementation of super resolution optical flow. Recently there has been considerable interest in the problem of extracting high resolution images from a lower resolution video. This process is usually referred to as super-resolution because the output images have higher resolution than any of the images used to create them. Can we extend any of these super-resolution techniques to maximize the resolution of our face images?[8].
3. Super-resolution techniques
3.1 Multi-frame super resolution
In multi-frame Super-Resolution (SR) image reconstruction a single High-Resolution (HR) image is created from a sequence of Low-Resolution (LR) frames. This work considers stochastic regularized multi-frame SR image reconstruction from the data-fidelity point of view. In fact, a novel estimator named norm is proposed for assuring fidelity to the measured data. This estimator presents the hybrid form of both error norm and logarithm ln. The introduced norm is combined with the Bilateral Total Variation (BTV) regularization. The proposed SR method is directly
3.2 single-frame super resolution
Several methods for the super-resolution problem, in most of them the input is a set of LR images from which a HR image is obtained. For the limited case, where the only input is the LR image to be super resolved, is almost impossible to recover high-frequency information accurately. However, in the last five years, methods have been proposed that base their learning stage in the correlation between regions with low and high
3.2.1 Kernel Hebbian Algorithm for Single Frame Super-Resolution
The problem of image super-resolution, where one achieves high-resolution enlargements of pixel-based images, has a lot of potential applications in graphics, image processing, and computer vision. Methods of image super-resolution can be conveniently divided into three complimentary classes: 1) interpolation and sharpening enlarges the low resolution image using generic image interpolation techniques and sharpen the resulting image for better visibility; 2) aggregation from multiple frames extracts a single high-resolution frame from a sequence of low-resolution images; 3) single-frame super-resolution extracts high-resolution image details from a single low-resolution image, which cannot be achieved by simple sharpening. All three approaches rely on a certain type of prior knowledge about the image class to be reconstructed. The third approach, in particular, needs a specific characterization of its respective image class which is often available in the form of example patterns. Whereas the first two methods have already been extensively studied the third method has been introduced only recently proposed a patch-wise reconstruction technique. Both algorithms already demonstrated impressive super-resolution results.
An alternative approach to super-resolution based on an unsupervised learning technique. Instead of encoding a fixed relationship between pairs of high- and low-resolution image prely on a generic model of the high-resolution images that is obtained from Kernel Principal Component Analysis (KPCA).
4. Observation model
Super-resolution (SR) techniques make use of sub pixel shifts between frames in an image sequence to yield higher resolution images. We propose an original observation model devoted to the case of non isometric inter-frame motion as required, for instance, in the context of airborne imaging sensors. First, the main observation models used in the SR literature deal with motion, and they are not suited for non isometric motion. Then, an extension of the observation model by Elad and Feuer adapted to affine motion. This model is based on a decomposition of affine transforms into successive shear transforms, each one efficiently implemented
6. Image observation model
The digital imaging system is not perfect due to hardware limitations, acquiring images with various kinds of degradations. For example, the finite aperture size causes optical blur, modeled by Point Spread Function (PSF). The finite aperture time results in motion blur, which is very common in videos. The finite sensor size leads to sensor blur; the image pixel is generated by integration over the sensor area instead of impulse sampling. The limited sensor density leads to aliasing effects, limiting the spatial resolution of the achieved image. These degradations are modeled fully or partially in different SR techniques.
Fig. 5.2 shows a typical observation model relating the HR image with LR video frames, as introduced in the literature [12, 13]. The input of the imaging system is continuous natural scenes, well approximated as band-limited signals.
These signals may be contaminated by atmospheric turbulence before reaching the imaging system. Sampling the continues signal beyond the Nyquist rate generates the high resolution digital image (a) we desire. In our SR setting, usually there exists
8. Super-resolution procedure
As mentioned in the introduction, the aim of super-resolution is to reverse the effects associated with the image acquisition, and construct an approximation of the original high resolution data source. Ideally, the constructed high resolution (HR) frame would be an exact copy of the original HR frame; however, even if the forward model perfectly describes the acquisition process the effects of noise still limit the accuracy of the reconstruction.
The forward model provides a mapping from a given high resolution frame directly to a corresponding low resolution frame. However the forward model does not provide a unique one to one mapping between high resolution inputs and low resolution outputs. Instead a many to one mapping is provided. This means that it is possible to find multiple high resolution images which when used as the input to the forward model will result in the same low resolution image being produced.
This property of the forward model poses problems for super-resolution approaches since there is no unique mapping from a given low resolution frame to a high resolution frame – put another way, the super-resolution problem is an underdetermined problem, i.e. there are more unknown variables than there are known variables and equations linking them to the unknowns.
In order for this approach to be successful these frames must contain additional information about the original source (the scene). The method that is generally used to capture different information in the various low resolution images is to introduce a small amount of motion at the image capturing system. This introduced motion is taken into account in the warp matrix (F) in Equation 2. If enough additional information is provided by extra low resolution frames then it becomes possible for super-resolution to be performed and for the high resolution frame to be reconstructed. Equation 3 below shows how the model is extended in order to take into account the additional low resolution frames that are used.
Equation 3: Extended Model – Linking all LR Images in Input Set to a Single HR Image
The forward model for the acquisition of images, and showed how the transformation from high resolution to low resolution can be broken down into a series of transformations, which when successively applied to a high resolution frame will result in a low resolution output frame being generated. However before we can attempt to apply super resolution and reverse the effects the image acquisition process had on the original data source must first mathematically model the linear transformations that make up the image acquisition model.
The blurring and decimation transformations (for a given acquisition system), are assumed to be both spatially and time invariant (the transformation is the same for all images captured using a device). The warp transformation however varies depending on the position of the imaging system at the time the low resolution frames are captured, therefore each low resolution frame will have a different warp matrix. Before super-resolution can be applied, the warp matrix for each low resolution frame must first be determined
9.2 Motion free super resolution
Many different methods that use motion as the cue to generate the high frequency details. All these methods require a dense point correspondence among frames. Any error in establishing the correspondence affects the quality of super resolution. Although the bulk of the work on super-resolution does use motion cue, of late, there has been work on using other possible cues. Motion-free super-resolution techniques try to obtain the spatial Enhancement by using the cues which do not involve a motion among low resolution observations, thus avoiding the correspondence problem. One may expect an improved result since there would be no correspondence. However, find out what other cues can possibly be used as a substitute for the motion cue to bring in the high frequency details. We need to study how useful are these cues and what additional difficulties do they introduce during the super-resolution process. Another issue that comes out is how we should compare the performances of these methods with those of the motion-based methods. We simply cannot compare the methods as the data generation process is very different in both the cases. Further, the volume of work in this area is still quite small. Use of cues other than motion is the subject matter of this monograph.
9.3.2 LMS estimate
After the two images to be considered have been decomposed into pyramids, the motion parameters are estimated iteratively, by minimizing the error after the first image is warped toward the second. The following process is repeated five times1 at each level of the pyramid:
1. Warp the first image toward the second, using the current estimate (which is initially zero).
2. For each pixel (x, y), set up the Following matrix:
9.5 SIFT algorithm
This algorithm consists in a very efficient method to identify and to describe image key points, which is done by performing a mapping with different views of an object or scene, resulting in a vector with 128 values that describes each image key point. The algorithm consists of the following steps:
9.5.1 Scale-space extrema detection
The key points are detected applying a cascade filtering that identifies candidates that are invariant to scale.
D(x,y, σ) = (G(x,y,kσ) – G(x,y, σ)) * I(x,y)
The DoG is an approximation of the scale-normalized Laplacian of Gaussian σ2∆2G. The maxima and minima of σ 2∆2G produce the most stable image features.
9.5.2. Local extrema detection
From D(x,y, σ), it is suggested that the local maxima and minima must be detected by comparing each pixel with its eight neighbors in the current image and nine neighbors in the scale above and below (26 neighbors). SIFT guarantees that the key points are located at regions and scales of high variations, which make these locations stable for characterizing the image.
9.5.3. Key point description
The next step is to compute a descriptor for the local image region that is distinctive and invariant to additional variations, such as change in illumination or 3D viewpoint. In [Shrestha & Arai, 2003; Lowe, 2004; Dakun et al., 2010] is suggested that the best approach is to determine the magnitudes and directions of the gradients around the keypoint location. In this approach the Gaussian image on the keypoint scale is used.
9.6 Matching between two images
In order to find the match between two images it is possible to use the keypoints detected with the SIFT algorithm. To avoid an exhaustive search in [Lowe, 2004] the use of a data structure k-d tree [Brown & Lowe, 2007] that supports a balanced binary search is suggested to find the closest neighbor of the features and the heuristic algorithm Best-Bin-First (BBF) is used for the search.
9.7. RANSAC
RANSAC algorithm proposed by Fischler & Bolles (1981) is a robust estimation method designed to identify the inliers (data points that fit a particular model within an error tolerance) and outliers (data points that do not fit a particular model within an error tolerance) from the set of key points detected by the SIFT algorithm. RANSAC is widely used for object recognition [Okabe & Sato, 2003; Dakun et al., 2010; Lukashevich et al., 2011]. In addition, it makes it possible to find the geometrically consistent correspondences to solve the problem of joining pairs of images. RANSAC is a robust estimator, so much so that it shows fine results, even in extreme conditions, or with some kind of outlier.
As mentioned by Fischler & Bolles (1981), unlike the conventional techniques that use a lot of data to obtain an initial solution, and then eliminate the outliers, RANSAC uses only a set with a minimum number of required and sufficient points for a first estimate, and it continues the process by increasing the set of data points consistent.
10. Image interpolation
we consider the problem of high-quality interpolation of a single noise-free image. Several aspects of the corresponding super-resolution algorithm are investigated: choice of regularization term, dependence of the result on initial approximation, convergence speed, and heuristics to facilitate convergence and improve the visual quality of the resulting image. Linear methods for image interpolation are usually constructed to deal wcith band limited signals. The interpolated one-dimensional signal is defined as:
F'(x) = ∑+∞i=-∞ F(ih)K (ih- x) ,
Where,
K(x) – is the interpolation filter, h is the sampling step. In a two dimensional case, the interpolation is typically performed separately for each axis. The most popular weight functions are box filter (or nearest neighbor), tent function (or bilinear), ideal lowpass filter, Lanczos filter, Gaussian filter, and bicubic interpolation[23].
For every algorithm which is using linear interpolation there are some typical artifacts: blurriness, ringing effect, and jagged edges. Reduction of one of these artifacts increases the others.
As usual, non-linear algorithms are used to scale two-dimensional images with a fixed ratio without constructing continuous image. Interpolated pixel values are calculated as a linear combination of nearest sampled values, but the main difference with the linear interpolation is the variability of coefficients which depend on surrounding pixel intensities.
The main idea of gradient algorithms is the fact that directed interpolation along edges results in better interpolation than no directed linear interpolation. The direction and the intensity of an edge in a point are defined by the local gradient information.
One of these algorithms is WADI [24], which is based on the modification of bilinear interpolation. It computes the derivate along the normal to every side of a square formed by four sampled pixels and modifies coefficients of bilinear interpolation in a special way: the side with greater derivative results in smaller coefficients for points of this side.
Gradient algorithms are fast in the class of non-linear algorithms and produce better results than linear interpolation; it makes edges less jagged and more realistic.
NEDI algorithm (New Edge-Directed Interpolation) is a typical non-linear algorithm, which doubles the resolution of images [25]. It uses the concept of self-similarity. The assumption is that coefficients of linear combination used for destination pixel interpolation are the same as coefficients used for interpolation of source image pixels by pixels of the decimated source image. This algorithm provides very good interpolation quality but it is very complex, so it is often executed only in small areas with strong edges while simpler algorithms process the rest of area.
11. Image reconstruction
In the classic restoration problem in image processing, a blurred and noisy image is given and the purpose is to somehow restore the ideal image prior to the degradation effects. Such problem is typically modeled using the linear vector-matrix equation (using lexicographic ordering for the images):
Y = HX +N ;N ~ G {0,W−1 }
The three main tools that have been proposed to solve the above restoration problem are the Maximum Likelihood Estimator (ML) and the Maximum A-posteriori Probability Estimator (MAP), which apply stochastic perception to the problem, and Projection Onto Convex Sets (POCS), which applies set theory tools instead [26].A utilization of the above three estimation tools for the (SRR) problem. In this problem, an improved resolution image is reconstructed based on several geometrically warped, linearly blurred, uniformly down-sampled and noisy measured images. N such measured images are given, and the purpose is to reconstruct a single super-resolution image, which fuses all the measurements into it. The SRR problem have been proposed and treated by several authors in the last decade. Among the various proposed methods, the most general and thorough approaches are the Iterative Back Propagation algorithm proposed by Irani & Peleg [27], and the POCS based reconstruction, proposed by Patti, Sezan and Tekalp [28].
11.1 Principle
Super-resolution image reconstruction is based on the theory of Analytic Continuation, which means reconstruction of the whole analytic function according to its values in certain area. Because of diffraction of lights, spectrum distribution of certain image is infinite in space and optical system truncates its frequency to obtain frequency-truncated image that is finite in space. Generally, truncation function cannot be band-limited, but a diffraction limited optical system’s truncation is band-limited, therefore, the reconstruction of whole spectrum function or just spectrum function above certain frequency is possible.
11.2.3 Auto Regressive Moving Average
(ARMA) Estimate.
This method regards the original image as a 2 dimensional Autoregressive (AR) process and PSF model as a 2 dimensional Moving Average (MA) . So the blurred image can be described as a noised observation of Auto Regressive Moving Average (ARMA). Therefore, blind de convolution is translated into the problem of determining the parameter of ARMA. There are several algorithms, including Maximum Likelihood, Generalized Cross Validation (GCV), Neural Network and High Order Statistics (HOS) and so on. They all have good robustness on noise, but when there are too many parameters, they cannot convergent to global optimality.
11.2.4 Nonparametric Finite Support Restoration Techniques (NFSRT)
Nonparametric Finite Support Restoration Techniques don't need to establish the parametric model of original or blurred image, and there are not too many strict restraints, so they are widely used in image restoration.
11.3 Simplified Reconstruction Illustration
Figure 12 below illustrates a simplified case of super-resolution, in which the reconstruction process reduces to an interleaving of the pixel values of the low resolution images contained in the input set
Figure 11.1: Simplified Image Reconstruction [29]
This simplification of the reconstruction process occurs when we are presented with an ideal set of low resolution input frames, and corresponding models. This ideal enhancement occurs if we are presented with a set of 4 low resolution input frames, each of which has been subjected to only to translation and decimation transformations (no blurring). The translations transformations applied in order to generate each low resolution frame is also subjected to constraints. If the first image in the set is assumed to be the reference frame with a translation of 0 in each direction, the second image must have a translation of 1 HR pixel horizontally, the third must have 1 HR pixel vertically, and the fourth image must have been translated by 1 HR pixel in both directions.
11.5 Parallel Algorithm
The parallel implementation of the SR algorithm is straightforward. For example, for a 1024 x 1024 image and a 128 x 128 array of PES, each PE can be assigned an 8 x 8 block of data. Since the Fourier and convolution steps are completely independent for each 8 x 8 block, perfect partitioning is achieved and the speedup over a single PE for these two steps is 16,384. The most difficult part of implementing in parallel the SR algorithm is not in the algorithm, but in realigning the data between the PES so that correct output operations can be performed with a minimum of number of communication steps. After the Fourier step each PE contains an array of bytes whose number is dependent on the image data in the associated 8 x 8 block. The write primitives available for the MP-1 do not have the capability of simultaneously writing data, which is stored in the PES in arrays of varying length. In this paper we will be specifically examining efficiently writing
13. Image restoration
The theory of restoration of a image from liner blur and additive noise has drawn a lot of research attention in the last three decades. Many algorithms were proposed in the literature for this classic and related problems, contributing to the construction of a unified theory that ties together many of the existing methods. In the single image restoration theory, three major and distinct approaches are extensively used in order to get practical restoration algorithms: 1) maximum likelihood (ML) estimator,2) maximum a posteriori (MAP) probability estimator, and 3) projection onto convex sets (POCS)approach. The consistent development of computer technology in recent years has led to a growing interest in image restoration theory. The main directions are nontraditional treatments to the classic problem and looking at new, second-generation restoration problems, allowing for more complicated and more computationally intensive algorithms. Among these new second generation problems are multiple image restoration and super resolution image restoration. focuses on the latter problem of super resolution restoration. Application of such restoration methods arises in the following areas [30].
1) Remote sensing: where several images of the same area are given, and an improved resolution image is sought.
2) Frame freeze in video: where typical single frame in video signal is generally of poor quality and is not suitable for hard-copy printout. Enhancement of a freeze image can be done by using several successive image merged together by a super resolution algorithm.
3) Medical imaging (CT, MRI, ultrasound, etc.): these enable the acquisition of several images, yet are limited in resolution quality.
The super resolution restoration idea was first presented by T say and Huang. They used the frequency domain approach to demonstrate the ability to reconstruct one improved resolution image from several down sampled noise-free versions of it, based on the spatial aliasing effect. Other results suggested a simple generalization of the above idea to noisy and blurred images [5].
Continuous image sequences restoration –
Where the possibility of loss of resolution is not treated. Many of the existing results in this sub-category also limit their treatment to the case of noisy sequence without blur .
Classic super-resolution restoration –
The restoration of single improved resolution image from several measured low-resolution warped, blurred and noised images.
Multiple image restoration –
The restoration of several images simultaneously using the cross correlation between them. The proposed algorithms in this sub-category generally assume no motion between the images, and do not treat the continuous flow of images in time.
1x3.1 Nonparametric Finite Support Restoration
In the degraded model of real image, as presented, the existed linear image restoration algorithms all assume that the PSF is given, and try to obtain its inverse and make use of a lot of information about PSF, real image and noise to decrease noise. However, PSF is often unknown to us, and we don't have much information about the original image. For this problem, researchers put forward methods that restore image and obtain the PSF at the same time. Different from the priori blur identification methods, in nonparametric finite support restoration techniques, the parametric model of original or blurred image is not necessary to assume.
There are several methods need to discuss, including Iterative Blind De convolution, Richard
14.7 Super Resolution from Single Frame Image
Example based super-resolution methods have been proposed with the aim to reconstruct a high resolution image given a single low resolution image. In this approach, the correlation between low resolution images and corresponding high resolution images is learnt from a database of known low and high resolution image pairs. This learning is then applied to a new low resolution image to obtain its most likely high resolution image. Higher factors of super-resolution have been obtained by repeated application of this process.
14.7.1 Sparse Signal Representation
Recently sparse representation has been applied to many other related inverse problems in image processing, denoising,restoration, super- resolution, etc., often improving on the state of-the-art. Proves that LMMSE estimate is better where the parameter vector is known to be sparse, hence the LMMSE estimate given by NEDI (New Edge Directed Interpolation) can be improved based on sparse representation.
Define a dictionary (matrix) of size ? ∈ ?*? (with k ¿ n, implying that it is redundant, n is
the size of the image patch). The basic idea here is that every signal instance from the family we consider can be represented as a linear combination of few columns (atoms) from the redundant dictionary D. That is the signal vector X can be written as ? = ? where ? ∈ ?*? is a vector with very few (≪ ?) nonzero entries, then it is undermined for the unknown coefficient. The sparse representation can be as follows,
? = ?∥?∥0 ? ? ? ≈ ?.The term ∥?∥0 stands for the count of the non-zero entries in ?.
We can use an alternative terminology which the constraint becomes a penalty, then can be changed to be? = {∥? − ?∥22 + ?∥?0∥}. The term ∥?∥0 encourages the sparsity of the fitted coefficient vector, and the parameter ?controls the tradeoff between the reconstruction error and the sparsity.
14.7.2 Super-resolution from Sparse Geometric Similarity
Now describe the algorithm for image super-resolution via sparse representation. For the interesting unknown HR pixel , we define its neighborhood as the training window. From the description of ANEDI (Arbitrary New Edge Directed Interpolation), the local geometry is defined as the contribution with which data is reconstructed from its neighborhood in the training window. It can be represented as follows,
? = {∥? − ?∥22}.
Where ?, ? are ? ∗ 1vectors, = {?1, ?2, ⋅ ⋅ ⋅ , ?}? ,? = {?1, ?2, ⋅ ⋅ ⋅ , ?}? ,,(n is the size of the training window),? is composed of the nearest neighbors corresponding to each elements of X. After the coefficient ? calculated using, The interesting unknown HR pixel can be directly
calculated as the weighted average of its nearest LR neighbors. i.e.,
14.7.2 Super-resolution from Sparse Geometric Similarity
Now describe the algorithm for image super-resolution via sparse representation. For the interesting unknown HR pixel , we define its neighborhood as the training window. From the description of ANEDI (Arbitrary New Edge Directed Interpolation), the local geometry is defined as the contribution with which data is reconstructed from its neighborhood in the training window. It can be represented as follows,
? = {∥? − ?∥22}.
Where ?, ? are ? ∗ 1vectors, = {?1, ?2, ⋅ ⋅ ⋅ , ?}? ,? = {?1, ?2, ⋅ ⋅ ⋅ , ?}? ,,(n is the size of the training window),? is composed of the nearest neighbors corresponding to each elements of X. After the coefficient ? calculated using, The interesting unknown HR pixel can be directly
? = ?.
14.8 Single-Frame text super-resolution: a Bayesian approach
Some early work on fully automatic super-resolution of text was done by Ulichney and Troxel [32]. They use a fixed set of heuristic templates to model local continuous shape. While doing so, they verify that neighboring shape assignments are mutually compatible and make local straight-line approximations. Several pieces of more-recent work on super-resolution and compression have centered on inferring shape by clustering glyphs seen in a document. For a document containing Latin characters, generate nearly 300 glyph clusters. Ideally, each cluster contains all instances of a single glyph. During the clustering process, a high-resolution model of each glyph is generated by averaging polygonal shape approximations. Some care is taken to try to preserve sharp corners. Bern and Goldberg expand on approach by modeling the scanning process probabilistically and using slightly different algorithms at each step. Non linear optimization on a grayscale input image to minimize a Bimodal Smoothness Average (BSA) score. To be bimodal, the high-resolution image should be black and white with few gray pixels. To be considered smooth, its locally-estimated second derivatives should be small. For the “average” measure to be small
15. Simultaneous super-resolution
Imaging devices have limited achievable resolution due to many theoretical and practical restrictions. An original scene with a continuous intensity function o(x,y) warps at the camera lens because of the scene motion and/or change of the camera position. In addition, several external effects such as atmospheric turbulence, camera lens, relative camera-scene motion, etc, can cause images to be blurred. Super resolution blurs the images and also use external effects as above. call these effects volatile blurs to emphasize their unpredictable and transitory behavior, yet we will assume that we can model them as convolution with an unknown point spread function (PSF) h(x, y). This is a reasonable assumption if the original scene is °at and perpendicular to the optical axis. Finally, the CCD discreteness the images and produces a digitized noisy image g(i; j), which we refer to as a low-resolution (LR) image, since the spatial resolution is too low to capture all the details of the original scene. For one single observation g(i; j) the problem of reconstructing o(x,y) is heavily underdetermined
16. Super resolution image Enhancement
Image super-resolution creates an enhanced high-resolution (HR) image using multiple low-resolution (LR) images of the same object. A typical image formation model introduces blurring, aliasing, and added noise. Super-resolution (SR) algorithms jointly reduce or remove all three. The first SR algorithm was based on an image model that only introduced
Aliasing (no blurring or noise), and explicitly performed de-aliasing in the frequency domain. Subsequently, most SR algorithms have operated in the spatial domain, including Iterative Back-Projection (IBP), Projection Onto Convex Sets (POCS), Maximum A Posteriori (MAP) optimization. Approaches based on wavelets have also been presented. A comprehensive overview of super-resolution algorithms can be found in. One primary drawback of many papers on SR algorithms is that they do not quantify performance. Sometimes, the algorithms are applied to .synthetic. LR images generated from an original HR image.
Recently people have begun exploring the performance of SR algorithms for translational motion. Limits on the up sampling factor for a box-filter PSF have been explored. However, these all evaluate only objective MSE performance. explore the subjective performance of super resolution image enhancement. We examine subjective quality
as a function of up sampling factor, down sampling factor, and SR algorithm for three images. We show that, in contrast to the claim in, image quality improves as the up sampling factor increases. also show that PSNR does not effectively predict quality of SR images. to examine the ability of some existing quality metrics (including SSIM and Marziliano's blurring and ringing metrics) to characterize SR performance. focus on synthetic examples so the original image is available for reference. In addition, we extend our subjective
tests to include three more images. The perceptual impact of image scaling, a problem closely related to super-resolution enhancement, has been explored in. They catalog five impairment types introduced by image scaling. These are un sharpness (or blurring), ringing, and three impairments caused by aliasing. They use expert viewers to judge the severity of each of these five impairment types to obtain an overall quality value for the scaled images. However, the requirement of expert viewers limits the applicability of this method.
Here describe image formation model for super-resolution enhancement using only one dimension for simplicity. The kth observed low-resolution (LR) frame is gk(n) = [h(x) * f(x- Tk) + wk(x)|x=n∆ ,k = 1,…,K,
Where f(x) is the continuous image scene, the sampling interval of the LR image is ∆, and wk(x) is additive white Gaussian noise with variance σ2. assume a translational shift Tk between the LR images. The camera applies a continuous point-spread function (PSF) h(x). use the model, where the continuous HR image is histogram-shaped, so the continuous HR image is formed by applying sample-and-hold reconstruction to the discrete samples of the HR image. also assume that the continuous PSF is space-invariant. Typically, the image formation process is expressed in matrix form. Denote Pk as the matrix operator which
20. Conclusion
Now a days to convert high resolution from low resolution there are many different types of algorithms and techniques are available. the success of super resolution is depends on noisy image and image pixel density. We have provided a review of the current state of super-resolution research, covering the past, present, a. Use all of these algorithms and approach and methods and all techniques which is used in super resolution an image from one or more images that is blur images, noisy images and also use the image registration