Very high resolution image matching based on local__ features and k-means clustering

The Photogrammetric Record30(150):166–186(June2015)

DOI:10.1111/phor.12101

VERY HIGH RESOLUTION IMAGE MATCHING BASED ON LOCAL FEATURES AND K-MEANS CLUSTERING

Amin S EDAGHAT(am.sedaghat@https://www.sodocs.net/doc/4c11335271.html,)

Hamid E BADI(ebadi@kntu.ac.ir)

K.N.Toosi University of Technology,Tehran,Iran

Abstract

Image matching is a critical process in photogrammetry and remote sensing.

Automatic and reliable feature matching using well-distributed points in very high resolution images is a dif?cult task due to signi?cant relief displacement caused by tall buildings and ground relief.In this paper a robust and ef?cient image-matching approach is proposed,consisting of two main steps.In the?rst step, three sets of local features–Harris points,UR-SIFT and MSER–are extracted over the entire image.A SIFT(scale-invariant feature transform)descriptor is then created for each extracted feature,and an initial cross-matching veri?cation is performed using the Euclidean distance between feature descriptors.In the second step,an approach based on k-means clustering is performed to achieve accurate matching without mismatched features,followed by a consistency check using a local af?ne transformation model for each cluster.The proposed method is successfully applied to matching various aerial and satellite images and the results demonstrate its robustness and capability.

Keywords:Harris corners,image matching,k-means clustering,MSER,UR-SIFT,

very high resolution image

Introduction

I MAGE MATCHING IS THE PROCESS of?nding corresponding points in two or more images of the same scene taken at different times,from different viewpoints and/or by different sensors.It is one of the most important and challenging subjects in photogrammetry and remote sensing and is a crucial process in a wide range of applications such as image registration (Gianinetto,2012;Parmehr et al.,2014),change detection(Theiler and Wohlberg,2012), image fusion(Kurz et al.,2011),and in the modelling and mapping sciences(Ekhtari et al., 2009;Barazzetti et al.,2010;Tack et al.,2012;Mohammadi and Malek,2014).

Image-matching methods are generally classi?ed as area-based matching(ABM)or feature-based matching(FBM).In ABM methods,the pixel intensities of small image patches are used to establish the correspondence by estimating a similarity measure such as cross correlation,least squares matching(LSM)(Gruen,1985)and mutual information(Suri and Reinartz,2010;Gong et al.,2014).FBM techniques utilise salient features to establish correspondences or estimate the transformation parameters between two images.

?2015The Authors The Photogrammetric Record?2015The Remote Sensing and Photogrammetry Society and John Wiley&Sons Ltd

The Photogrammetric Record ABM methods require good approximate values to assure convergence,and problems occur in areas with occlusions,areas with a lack of(or repetitive)texture,or if the surface does not correspond to the assumed model(Remondino et al.,2008).FBM methods are more robust and more reliable than ABM methods,but they are usually less accurate and are problematic in mismatch elimination(Sedaghat et al.,2011).In remote sensing applications,ABM methods are normally suitable for open terrain areas,but FBM methods can provide more accurate results in urban areas(Xiong and Zhang,2009).

In this paper the focus is on FBM in very high resolution remote sensing images.In previous research,many image-matching methods have been proposed for photogrammetry and remote sensing applications.A variety of these algorithms have been reviewed and classi?ed in detail in the literature(Le Moigne et al.,2011;Goshtasby,2012;Gruen,2012; Remondino et al.,2014).

Most FBM algorithms consist of two main steps(Remondino et al.,2008):

(1)Feature detection,which extracts distinctive image features and their attributes

(“descriptors”to characterise and match them)in two images(a reference image and an input image),such as corners,lines,blobs and regions.

(2)Feature matching,which determines the correspondence between the features in the

two images using particular similarity measures.It then uses a consistency check process for outlier rejection.

Xiong and Zhang(2009)proposed an algorithm for interest-point matching in high-resolution satellite images based on control network construction using so-called“super points”(Harris corners,which represent the most prominent features).In their research,an iterative closest point algorithm was applied to search for correspondences based on the position that had been assigned to each interest point.In their approach,only translation and rotation are considered;therefore,this algorithm can only be used for images that were captured with a short baseline and have negligible local distortions.

Local feature detectors and descriptors are commonly used for image matching in the computer vision literature(Mikolajczyk and Schmid,2005;Mikolajczyk et al.,2005; Tuytelaars and Mikolajczyk,2008;Brown et al.,2011;Gauglitz et al.,2011).The most prominent methods are the scale-invariant feature transform(SIFT)(Lowe,2004),shape context(Belongie et al.,2002)and their extensions.These have been shown to be invariant to image rotation and scale;furthermore,they are robust across a substantial range of af?ne distortion,the presence of noise and change in illumination.SIFT-based methods have been widely applied in remote sensing image matching and registration(Goncalves et al.,2011; Sedaghat et al.,2011;Chureesampant and Susaki,2014;Han et al.,2014;Song et al., 2014).For example,Sedaghat et al.(2011)proposed a uniform robust SIFT(UR-SIFT) algorithm for remote sensing image matching based on a selection strategy of SIFT features over the full distribution of location and scale.

Wang et al.(2012)presented a study of a multisource image automatic registration system(MIARS)based on the SIFT algorithm.Their approach consisted of three main steps:image division;histogram equalisation;and the elimination of false point matches by a subregion least squares iteration.Cheng et al.(2012)presented a new technical framework for remote sensing image matching by integrating maximally stable extremal region (MSER)features(Matas et al.,2004),SIFT descriptors and random sample consensus (RANSAC)(Fischler and Bolles,1981).The main property of their framework was an automatic optimisation strategy for af?ne invariant feature matching based on RANSAC. Other remote sensing image-matching methods based on SIFT are presented in Schwind et al.(2010),Goncalves et al.(2011),Zhang et al.(2011),Han et al.(2014)and Song et al.?2015The Authors

The Photogrammetric Record?2015The Remote Sensing and Photogrammetry Society and John Wiley&Sons Ltd167

(2014).Huang and Li (2010)proposed a feature-based image-matching method for an airborne multi-sensor image using shape context.

Even the best algorithms for image matching make some mistakes and output some mismatches (Adam et al.,2001).Perhaps the most dif ?cult process in FBM is discarding wrong matches from initial matches (Heikkil €a et al.,2009).There are several methods for identifying and discarding matched points as mismatches.Generally,a geometric transformation model is estimated between the two images.The feature matches that are not consistent with the estimated model are identi ?ed as false matches and rejected.Two main issues need to be considered in order to establish a transformation model and mismatch elimination (Wong and Clausi,2007):(a)the type of geometric models that represent the spatial relationship between the two images;and (b)the method of estimating the parameters of the selected transformation model.The transformation model used depends on the types of geometric distortions;common spatial transformation models include af ?ne,projective and polynomial (Wong and Clausi,2007).The most prominent methods for transformation-model estimation and performing outlier rejection are the RANSAC-based methods.

The main dif ?culty of FBM of very high resolution remote sensing images from different viewpoints is signi ?cant relief displacement,which causes localised distortion.Such images of the same 3D scenes have different appearances.Fig.1shows two UltraCam aerial images of a building area acquired from different viewpoints.The features marked in the images are conjugate points,which are selected manually.It shows highly complicated local distortions due to the relief displacement.For example,P 1P 3distance in the left image is approximately 174pixels,while this distance is 37pixels in the right image (a difference of 137pixels).Due to these considerable local distortions,a global 2D transformation cannot reliably model geometric consistency,especially for areas with large elevation differences (Sedaghat et al.,2012).These methods,for example,af ?ne or projective models,cannot overcome signi ?cant local distortions and hence can lead to the elimination of a large number of true matches due to inconsistency with the model

adopted.

Fig.1.Signi ?cant local distortions due to the relief displacement in two very high resolution UltraCam aerial

images.Enlargements on the right show the distances between points P 1,P 2and P 3in pixels.

Sedaghat and Ebadi.Very high resolution image matching based on local features and k -means clustering ?2015The Authors

The Photogrammetric Record ?2015The Remote Sensing and Photogrammetry Society and John Wiley &Sons Ltd

168

The Photogrammetric Record An alternative geometric model,particularly used in computer vision,is the epipolar constraint based on the fundamental matrix.This model,however,is potentially not without problems(Kim,2000).Generally,there are a few false matches in the selected matching results based on epipolar geometry(Zhao et al.,2010).The outlier rejection,based on this constraint,only considers the distance of each matched point from its estimated epipolar line.Therefore,false matches located in the vicinity of epipolar lines cannot be identi?ed using such an epipolar constraint.Another effective outlier rejection method is based on graph transformation matching(GTM)(Aguilar et al.,2009;Izadi and Saeedi,2012).This is an iterative process that discards one outlying correspondence at a time,according to a graph-similarity measure.After each iteration,the graphs are recon?gured in order to re?ect the new state of the remaining correspondences.

In a recent study,the current authors introduced a method for image matching of satellite data based on quadrilateral control networks which was based on a2D piecewise transformation model(Sedaghat et al.,2012).This method provides good results,but it depends on the number of local distortions.In fact,the matching of images containing highly complicated local distortions cannot be solved by this method.

Designing and developing an effective approach to overcome the aforementioned problems in very high resolution images is an important task.In this paper a robust FBM strategy is presented which can extract a large number of corresponding points for very high resolution aerial or satellite images with signi?cant terrain or building relief.The next section discusses the two steps of the proposed methodology.This is followed by an analysis of the experimental results and,?nally,conclusions are drawn.

Methodology of the Proposed Method

In this section,an effective and robust automatic approach is presented for reliable matching in very high resolution images with signi?cant relief displacement.The proposed method can be divided into two main steps,as illustrated in Fig.2,where one image is considered the reference image and the other the input image.

In the?rst step,the feature extraction and description processes are performed in both reference and input images.In order to increase the reliability of image-matching results,a combination of three well-known image features–Harris corners,UR-SIFT blobs and MSER regions–are used.Then,for each extracted feature,the SIFT descriptor is generated based on local image gradients.The?nal process of this?rst step is initial cross matching, which is performed using the Euclidean distance between feature descriptors.

In the second step,a?ltering strategy is applied for outlier rejection.For removing all mismatched points a novel method based on a local consistency check strategy is proposed. This process is automatically started with the k-means clustering process for extracted initial matched point pairs based on their location in the reference image.Then the algorithm uses a consistency check process based on an af?ne transformation model for each cluster to identify and remove the mismatched points.During this process,it is possible that some correct matches are mistakenly removed,particularly those clustered incorrectly.These removed correct matches are returned to?nal matches by a recheck process.Details of the proposed method are presented in the following sections.

Local Feature Extraction and Description

In this research,three types of local features–corners,blobs and irregular regions–are used for a full description of image features.Corners are prominent image points that

?2015The Authors

The Photogrammetric Record?2015The Remote Sensing and Photogrammetry Society and John Wiley&Sons Ltd169

show a strong two-dimensional intensity change,and provide important information on image feature structures.Blobs are the regions that are either brighter or darker than the surrounding area that can be approximated using a circle.The third type –MSER features –are enclosed boundaries that have the properties required for af ?ne invariant feature extraction.

The main reason behind this combined feature extraction using three types is that there does not exist one single feature detector algorithm which outperforms the other detectors for all scene types and all classes of transformations.Each of these image features has different properties,and the overlap between them is small if non-existent (Mikolajczyk et al.,2005).Therefore,the simultaneous combination of these features offers a reliable and robust distinctive extraction structure over the majority of the image scene.

In the following subsections the details of the implementation of these three different local feature extraction methods are presented.

Harris Corner Point Extraction

The Harris operator is based on the auto-correlation matrix and computes a measure that indicates the presence of a corner point.In this paper,a local extraction strategy is used for a good spatial distribution of Harris corner points according to the authors ’previous research (Sedaghat et al.,2012).This strategy consists of two parts:distance constraint and grid-distribution constraint.The distance constraint uses a minimum-distance

threshold

Fig.2.The main stages of the proposed method.

Sedaghat and Ebadi.Very high resolution image matching based on local features and k -means clustering

?2015The Authors

The Photogrammetric Record ?2015The Remote Sensing and Photogrammetry Society and John Wiley &Sons Ltd

170

The Photogrammetric Record between the extracted corners which can prevent ambiguity in the matching process.The grid-distribution constraint applies a regular grid to spread the total number of required corner points,N c,over the entire image.This gridding process is based on a Harris measure

–entropy–and the number of available corner points in each grid.For a detailed explanation of this method,see Sedaghat et al.(2012).

UR-SIFT Feature Extraction

Various methods for blob image feature extraction have been proposed,of which the SIFT algorithm is one of the most prominent.The feature extraction process in the SIFT algorithm is based on scale space theory and the DoG(difference of Gaussian)function. The extracted features using SIFT algorithms are reasonably invariant to rotation,scaling and illumination changes.However,it has some problems with remote sensing images, particularly in the feature-extraction module(Sedaghat et al.,2011).

As mentioned in the Introduction,the UR-SIFT algorithm is an advanced improvement of the standard SIFT algorithm for uniform robust feature matching in remote sensing images.The local feature-extraction process in the UR-SIFT algorithm uses a selection strategy of SIFT features over the full distribution of location and scale,where the feature qualities are quarantined based on stability and distinctiveness constraints(Sedaghat et al., 2011).Due to the outstanding characteristics of the UR-SIFT algorithm in extracting a uniform and robust feature set in an image,it has been recently used for medical image registration(Ghassabi et al.,2013).

MSER Feature Extraction

MSER denotes a set of distinguished irregular regions,which are de?ned by an extremal property(see below)of its intensity function over the region and on its outer boundary(Matas et al.,2004).

MSERs are identi?ed by analysing a unique image representation denoted as a component tree that is built by thresholding the image at all possible values.Each node of the component tree contains a single extremal region.The word extremal refers to the property that all the pixel values in the regions are either strictly darker or strictly brighter than those on the boundary.Only the extremal regions that remain unchanged over a range of thresholded images,namely,the most stable ones,are selected as MSERs.Finally,an ellipse is?tted to each irregular MSER.For a detailed explanation of this method,see Matas et al.(2004).

In this paper,ED-MSER(entropy and spatial dispersion constraints MSER)(Cheng et al.,2008)was used for a robust and well-distributed region extraction using MSER.In this approach,a hierarchical?ltering strategy for MSER extraction is adopted that is based on the information entropy and spatial dispersion quality constraints.The features extracted by the MSER algorithm are invariant to geometric and photometric transformation,and according to the research carried out by Mikolajczyk et al.(2005),this operator has the best results compared with other approaches.

Feature Descriptor Generation and Initial Correspondence

After feature extraction,the next step is the descriptor generation process.The descriptor is computed on an image region de?ned by the feature detector.Based on the Mikolajczyk and Schmid(2005)comparative study,a SIFT descriptor offers the best

?2015The Authors

The Photogrammetric Record?2015The Remote Sensing and Photogrammetry Society and John Wiley&Sons Ltd171

matching results in comparison with other descriptors,including steerable ?lters,differential invariants,moment invariants,complex ?lters and principal components analysis SIFT (PCA-SIFT).

The SIFT descriptor is a 3D histogram of gradient locations and orientations.The location is quanti ?ed into a 494location grid,and the gradient angle is quanti ?ed into eight orientations,resulting in a 128-dimensional descriptor.

Because the corner features do not have scale parameters,a constant 41941patch (proposed in Mikolajczyk and Schmid,2005)was used,centred over each corner point for descriptor generation.In contrast,UR-SIFT and MSER detectors provide circular and elliptical regions of different sizes.All the regions are resampled to a circular region of constant radius (a 41941patch)to obtain scale and af ?ne invariance.In order to achieve orientation invariance,before the SIFT descriptor generation one or more orientations are assigned to each feature point based on local image gradient directions.Then the normalised regions are rotated in the direction of the dominant gradient orientation.Fig.3shows the process of SIFT descriptor generation for a sample MSER area.

In the proposed method it is assumed that all of the salient features of the image scene are extracted by a combined feature extraction method which can signi ?cantly increase the feature matching density and reliability.For example,Fig.4shows the extracted combined features in a panchromatic IKONOS image using the proposed method.As seen in this ?gure,almost all the salient structures of the image scene are extracted in one of the corner,blob or region feature forms.

The combined feature-extraction algorithm requires a set of threshold values to be determined as the input parameters.These parameters include the grid cells ’dimensions and the total number of required features.The dimension of the grid cells determines

the

(a)(b)(c)(d)

Fig.3.The SIFT descriptor generation process:(a)a sample MSER elliptical area;(b)region normalisation

and orientation assignment;(c)rotation of normalised region;(d)generated

descriptor.

(a)(b)(c)(d)

https://www.sodocs.net/doc/4c11335271.html,bined feature extraction:(a)IKONOS image;(b)extracted Harris corners;(c)extracted UR-SIFT

blobs;(d)extracted MSER ellipses.

Sedaghat and Ebadi.Very high resolution image matching based on local features and k -means clustering ?2015The Authors

The Photogrammetric Record ?2015The Remote Sensing and Photogrammetry Society and John Wiley &Sons Ltd

172

distribution condition of the extracted features in each algorithm.In this paper a grid cell of

25925pixels was used.Additionally,the total number of required Harris corner features,

N c,UR-SIFT blob features,N b,and MSER features,N r,was set to0á7%,0á6%and0á2%of the image pixel area,respectively.For example,for a sample image with size100091000

pixels,the total number of required features of each type was N c=7000,N b=6000and N r=2000.It is clear that if,in the input image,these numbers of reliable features do not exist,the maximum number of available reliable features will be extracted.

The feature correspondence between extracted features in both the reference and input

images is achieved through a Euclidean-distance ratio based on a nearest neighbour approach(Lowe,2004).To increase the matching reliability,a cross-matching constraint that con?rms the feature matching by a reverse certi?cation is also used(Sedaghat et al., 2011).The initially matched features whose distance ratio is greater than T ED=0á85 (Euclidean-distance ratio threshold(Lowe,2004))are rejected.

K-means Clustering for Outlier Rejection

In this section,an effective approach for mismatch elimination based on k-means clustering and an af?ne model is presented.The input of the proposed approach is one-to-one match point sets.Suppose two point sets from two images are denoted by P={p1,p2,...,p i,...,p n}and Q={q1,q2,...,q i,...,q n}where p i and q i are the initial match points.There might be some mismatches in these initial sets which should be removed.

The main idea of the proposed approach is based on the following assumption.In very high resolution image pairs with signi?cant relief displacement and occlusion,there are local areas approximating a plane which can be related to a2D model such as an af?ne transformation.For example,Fig.5shows two UltraCam aerial images acquired from different viewpoints in which?ve local-area pairs are selected manually.As seen in this ?gure,because each of these local areas is approximately a plane,a local af?ne model can be used to establish the correspondence between extracted features.

In the previous step,a set of well-distributed combined features were extracted and matched.These initial matches contain some outliers which should be removed.Fig.6 shows a?owchart of the proposed method for outlier rejection.To this end,?rst of all the clustering process of the initial matched features is performed based on the feature locations

in the reference image using the k-means method.This is a method of cluster

analysis

Fig.5.Conjugate local areas that are approximately planar in very high resolution images.A local af?ne model can be used to establish correspondences between extracted features in each pair’s local area.

The Photogrammetric Record

?2015The Authors

The Photogrammetric Record?2015The Remote Sensing and Photogrammetry Society and John Wiley&Sons Ltd173

which aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean.This process is shown in Fig.7(a),where it can be seen that some features have wrong matches in each cluster.It should be noted that the proposed outlier-rejection method does not use the image radiometric information for clustering;rather,the clustering process of the initially matched features is performed based only on the feature locations in the reference image.

In the next step,the initial matched pairs for each cluster are checked in an af ?ne transformation model;those with the highest errors are eliminated one by one until a root mean square error (RMSE)is achieved that is less than a threshold parameter T A which controls the accuracy of the image matching (RMSE

All likely mismatches are rejected by the local consistency check process in the af ?ne transformation model.However,there are some true matches which are incorrectly removed in this process.A rechecking process is therefore applied to return the majority of these

true

Fig.6.Outlier rejection ?owchart.T A is a threshold parameter.

Sedaghat and Ebadi.Very high resolution image matching based on local features and k -means clustering

?2015The Authors

The Photogrammetric Record ?2015The Remote Sensing and Photogrammetry Society and John Wiley &Sons Ltd

174

matches to the ?nally selected matches.These removed,yet correctly matched,points can be divided into three types,as illustrated in Fig.8.

The features of the ?rst type are those that are assigned to an inappropriate cluster (Fig.8(a)).When points that are located on the ground are clustered with points on elevated objects they will,most likely,be rejected with the 1pixel rejection threshold.However,when they are clustered properly with the relevant ground points they will pass the test if they are valid matches.In fact,if these points are assigned to an appropriate cluster,then they will not be removed in the local consistency check process.In this process,an af ?ne model is estimated for each cluster,which is used to return this type of incorrectly removed feature to the matched points.For this purpose,the matching accuracies of all initially removed matched pairs are re-examined in the estimated af ?ne model of their neighbourhood clusters.The matched pairs that are consistent with one of the neighbourhood clusters are returned to the ?nal matched point set.The features of the second type (Fig.8(b))are those that are not consistent in any existing neighbourhood clusters but can be considered as a new cluster with a smaller size.(a)

(b)

Fig.7.Local consistency check based on k -means clustering and an af ?ne model:(a)feature-point clustering

in the reference image space;(b)re ?nement clusters after outlier rejection.

The Photogrammetric Record

?2015The Authors

The Photogrammetric Record ?2015The Remote Sensing and Photogrammetry Society and John Wiley &Sons Ltd 175

In order to return these points,a circular window of radius r is moved in the reference image;the number of all initially removed matches are computed in each position.The local consistency check process is applied in those positions where the number of initially removed matches is higher than a threshold number n t .

The features of the third type are other miscellaneous cases.Fig.8(c)shows an example of this type.These are solitary points,located in isolation on small elevated objects such as sparse trees or electricity posts.Because the number of matched pairs of this type is very small in comparison to the numerous other selected points,these true matches are ignored in the method proposed in this paper.

The number of clusters,CN ,is one of the most important input parameters of the proposed algorithm.This parameter is dependent on various factors such as:the spatial resolution;the number of initial matches;the image scene height variation;and the image baseline.Each factor,which has an impact on the amount of geometric distortion,is important in determining the number of clusters.To compute CN ,this procedure uses the ratio between the areas covered with all initial matches,A t ,and the average area of each cluster,A m .Here A t is computed based on the convex hull boundary of the initial matches in the reference image;A m is considered as a circle with a radius of between 60and 80pixels,which has been determined empirically.

It is assumed that initial correctly matched features inside the area of each cluster are consistent with an af ?ne model because of the approximately planar local area.The correctness of this assumption depends on the object size and its height variation,and also the amount of the local distortion in the very high resolution image.Due to the fact that in

(a)(b)

(c)

Fig.8.Rechecking the consistency of matched pairs.Removed true matches due to:(a)inappropriate

clustering;(b)inconsistent with neighbourhood clusters;(c)miscellaneous cases,such as isolated points.Sedaghat and Ebadi.Very high resolution image matching based on local features and k -means clustering

?2015The Authors

The Photogrammetric Record ?2015The Remote Sensing and Photogrammetry Society and John Wiley &Sons Ltd

176

Table I.Input image pairs.See also Fig.9.

Pair number Image

pair

Spectral

mode

Image size

(pixels)

GSD

(m)

Bits per

pixel

Acquisition

year

Location

1UltraCam-D RGB1256912780á1082005Tehran,Iran UltraCam-D RGB1256912780á1082005

2UltraCam-D RGB1161911690á1082005Tehran,Iran UltraCam-D RGB1161911690á1082005

3UltraCam-D RGB1197912030á1082005Tehran,Iran UltraCam-D RGB1197912030á1082005

4WorldView-2Panchromatic1286913090á5112007San Francisco,USA QuickBird Panchromatic1072910910á6112011

5WorldView-2Panchromatic1286913090á5112007San Francisco,USA QuickBird Panchromatic1072910910á6112011

6WorldView-2Panchromatic1286913090á5112007San Francisco,USA QuickBird Panchromatic1072910910á6112011

RGB is red/green/blue;GSD is ground sample distance.

(a)(b)

(c)(d)

(e)(f)

Fig.9.Test datasets.(a)to(f)are the?rst to sixth test image pairs(see Table I).

The Photogrammetric Record

?2015The Authors

The Photogrammetric Record?2015The Remote Sensing and Photogrammetry Society and John Wiley&Sons Ltd177

Sedaghat and Ebadi.Very high resolution image matching based on local features and k-means clustering the?nal step of the proposed method,a consistency check process is applied to return wrongly removed matches to the solution,the impact of the number of clusters and their area is considerably decreased in the?nal matching results.

It should be noted that the number of initial matches for each cluster should not be less than a reasonable number to reach reliable results.Based on the authors’experiments,10 features is an appropriate minimum number.The radius of the circular window for the consistency check process is empirically set to half the radius of the A m region.The n t threshold number for this process is set to10features.

Experimental Results

The capability of the proposed algorithm is evaluated with different types of very high resolution images.In the following image datasets,evaluation criteria and experimental results are described.The results obtained with the proposed method are also compared with those from the following algorithms:RANSAC+epipolar;RANSAC+projective;and GTM (Aguilar et al.,2009).In addition to examining the effectiveness of the proposed combined feature extraction algorithm,it was compared with the UR-SIFT detector(UR-SIFT+k-means).

Datasets

To evaluate the applicability of the proposed algorithm for different types of imagery, six aerial and satellite image pairs were chosen.The properties of these input images are shown in Table I.

All selected image pairs are very high resolution images;their ground sample distance (GSD)varies from0á1to0á6m.The selected image pairs cover different image textures in two urban areas,including highly structured buildings,trees and natural terrain.Because each image pair is acquired from different viewpoints of elevated objects(mainly buildings), the images have considerable relief displacement and local geometric distortions.Fig.9 shows the image sets used to evaluate the proposed method.

The?rst,second and third aerial stereopairs are from an UltraCam-D sensor captured over Tehran,Iran in2005.The fourth,?fth and sixth satellite stereopairs are taken from WorldView-2in2007and QuickBird in2011over San Francisco,USA and were related to the2012IEEE GRSS data fusion contest.

Evaluation Criteria and Implementation Details

The quality of the proposed method is evaluated using two common criteria Recall and Precision(Mikolajczyk and Schmid,2005):

Recall?TruePositives=TotalPositives

Precision?TruePositives=eTruePositivestFalsePositivesT

where TruePositives is the number of correctly matched point pairs in the matching results, FalsePositives is the number of falsely matched point pairs in the matching results and TotalPositives is the total number of existing correctly matched point pairs in the initial matched point sets(Mikolajczyk and Schmid,2005;Aguilar et al.,2009;Liu et al.,2012).To compute TotalPositives for each image pair after the feature extraction and initial matching

?2015The Authors 178

The Photogrammetric Record?2015The Remote Sensing and Photogrammetry Society and John Wiley&Sons Ltd

The Photogrammetric Record process,the number of correct matches is determined manually by carefully checking every

matched pair by an expert.Therefore,the number of TruePositives and FalsePositives can be

easily computed by comparing the?nal matching results with the manually selected matches.

The results of the proposed method were compared with the corresponding results

obtained using the following four algorithms:RANSAC+epipolar;RANSAC+projective;

GTM;and UR-SIFT+k-means.In the RANSAC+epipolar algorithm,the epipolar geometry model based on estimating the fundamental matrix is used.The RANSAC algorithm has

two parameters:a distance threshold t for?nding outliers;and the desired con?dence p for ?nding the maximum number of inliers.Empirically selected values of t=0á5and p=0á99 were used in the implementation process.In the RANSAC+projective algorithm,a2D

projective transformation model with eight parameters was used,and the RANSAC

parameters were set as t=1and p=0á99.In the case of the GTM algorithm,the value of k

was also set to4empirically.

The proposed image-matching algorithm and RANSAC were implemented in

MATLAB.The input parameters of the proposed method,as related in the previous section,

were selected as follows.The number of required Harris,N c,UR-SIFT,N b,and MSER,N r, features were set to0á7%,0á6%and0á2%of the image pixel area,respectively.For a uniform feature-extraction process,the cell size was empirically set to25925pixels for each of the three algorithms.The radius of the average cluster area,A m,was set to40pixels for the satellite images and60pixels for the aerial images.The T A threshold for the local consistency check process was set to1pixel for all image pairs.The radius of the circular window for the re-consistency check process was set to half of the radius of the A m region. The n t threshold number for this process was considered to be10features.

Results and Discussion

The proposed combined feature extraction algorithm was used to obtain robust,dense and uniform features from all test image pairs.Then the initial cross-matching process was performed by using the Euclidean distance similarity measure between the extracted feature descriptors.Finally,based on these extracted initial matched pairs,an outlier rejection process was conducted by the proposed cluster matching algorithm and also by the RANSAC+epipolar,RANSAC+projective,GTM and UR-SIFT+k-means methods.

Fig.10shows the results of the image matching by the proposed algorithms on the ?rst image pair.This test image covered an urban area with considerable relief displacement due to building height variations.The visual inspection of the matching results in this?gure shows the capability of the proposed method in?nding numerous accurate and well-distributed matched points.Similar results were obtained by the proposed method for other test images in Table I(not shown due to space limitations).

The comparative matching results of the proposed approach and the four other techniques are shown in Table II.It can be seen that the proposed outlier rejection algorithm based on k-means clustering outperforms the other techniques under evaluation in terms of Recall and Precision for all image pairs.These results demonstrate the effectiveness of the proposed method in removing mismatches in very high resolution image matching.Generally, in terms of the Recall criteria,the proposed method performs better than other methods, especially RANSAC+projective.The following observations on each method can be made:

(1)RANSAC+projective.Due to the considerable relief displacement present in the

images,the global2D projective transformation model applied in the RANSAC+projective method cannot reliably establish the spatial relationship

?2015The Authors

The Photogrammetric Record?2015The Remote Sensing and Photogrammetry Society and John Wiley&Sons Ltd179

Sedaghat and Ebadi.Very high resolution image matching based on local features and k-means clustering

(a)

(b)

(c)

Fig.10.Matching results for?rst image pair:(a)original image;(b)clustering process;(c)?nal matches.

?2015The Authors 180

The Photogrammetric Record?2015The Remote Sensing and Photogrammetry Society and John Wiley&Sons Ltd

between image pairs and hence a large number of true matches are removed with this approach.

(2)RANSAC +epipolar .With this method,the epipolar constraint has certain dif ?culties

when applied to satellite imagery due to their geometric characteristics.In addition,there are some mismatches in all test results which is due to the line constraint of the RANSAC +epipolar method,which applies the distance between each point and its estimated epipolar line.Consequently,some false matched points that have a small distance from the epipolar line are erroneously selected as true matches.

(3)Graph transformation matching (GTM).This method applies the average distance

between initial points and local adjacent relations to remove outliers.However,due to signi ?cant relief displacement in very high resolution images,not all the local neighbour relations are similar.Consequently,many correct points are removed as outliers by the GTM method.

(4)Proposed method .Because the density and the number of reliable local features

extracted using only UR-SIFT algorithms is not suf ?cient for the local consistency check process in the k -means clustering,the recall value of the UR-SIFT +k -means is signi ?cantly less than that of the proposed combined feature-extraction method based on the Harris,UR-SIFT and MSER methods.

In contrast to methods (1)to (3),the proposed method in this research generates approximately local areas over the entire image and uses an af ?ne consistency check Table II.Final matching results.

Pair Number

1

23

456Image Pair

U l t r a C a m ‐D U l t r a C a m ‐D

U l t r a C a m ‐D U l t r a C a m ‐D U l t r a C a m ‐D U l t r a C a m ‐D W o r l d V i e w ‐2

Q u i c k B i r d

W o r l d V i e w ‐2Q u i c k B i r d

W o r l d V i e w ‐2Q u i c k B i r d

Number of extracted features*N c 1123611236950095001007910079117838186117838186117838186N b 963196318143814386398639101007017101007017101007017N r 321032102714271428792879336623393366233933662339Number of initial matches 645765525954382937553758Outliers (%)13á1611á3816á8327á6828á1625á78R e c a l l (%)RANSAC +epipolar 96á993á985á478á476á279á1

RANSAC +projective 41á239á938á849á845á242á2

GTM 95á993á179á377á482á785á1

UR-SIFT +k -means 85á281á582á774á276á973á6

Proposed method 97á995á395á992á592á191á5

P r e c i s i o n (%)RANSAC +epipolar 97á494á391á971á967á269á5

RANSAC +projective 10010099á199á499á798á9

GTM 91á990á889á071á269á668á9

UR-SIFT +k -means 10010097á998á391á893á7

Proposed method 100100100100100100

Prop ’d method RMSE (pixel)0á9720á9930á9970á9760á9820á984*Nc is the number of Harris corner features;Nb is the number of UR-SIFT blob features;Nr is the number of MSER features.

The Photogrammetric Record

?2015The Authors

The Photogrammetric Record ?2015The Remote Sensing and Photogrammetry Society and John Wiley &Sons Ltd 181

process for each local area to remove mismatches.Furthermore,it applies post-processing to return wrongly removed matches.

Because mismatches in matching results can degrade subsequent photogrammetric processes such as relative orientation and space intersection,it is important to remove all mismatched points.The main drawback of the RANSAC +epipolar,GTM and other similar algorithms is the presence of some mismatches in the ?nal results.As seen in Table II,in any image pairs the Precision of the RANSAC algorithm is not 100%.The main goal of the proposed method is to overcome this drawback.

As seen in Table II,the results of the proposed method are completely accurate;the Precision ?gures for all test image pairs being 100%.Although some correctly matched features are removed by the proposed method,all the mismatches are completely removed,which is a very valuable result.

The fourth,?fth and sixth image pairs of the test datasets are satellite images;these were used to investigate the effectiveness of the proposed method in matching images acquired from different sensors (WorldView-2and QuickBird)and at differing dates (2007and 2011).The different sensors means the images have different intensity mappings and modalities,and are thus considered as dif ?cult cases for matching purposes.Despite the success of the proposed method in matching of these images,the Recall values are lower than those of the three aerial pairs.

The average value of all RMSEs related to all the clusters is an appropriate value to indicate the accuracy of the proposed method.As seen in Table II,the RMSE is less than 1pixel for all image pairs,which is an appropriate accuracy for matching results.It should be noted that the accuracy of the proposed method can be increased by selecting a smaller value for the T A threshold in the local consistency check process.However,such a lower value can decrease the number of ?nal matches.

In order to assess the capability of the proposed method against various outlier proportions,random mismatches were manually added to the ?rst image pair in 10%steps from 10%to 200%of the matches.Fig.11shows the performance of the proposed method and other algorithms in terms of Recall and Precision.It can be seen that the Precision of the proposed method is virtually invariant of these levels of outlier proportions;the

Recall

(a)(b)

Fig.11.The performance of the proposed method and the four other algorithms against outlier proportions:

(a)Recall ;(b)Precision .

Sedaghat and Ebadi.Very high resolution image matching based on local features and k -means clustering ?2015The Authors

The Photogrammetric Record ?2015The Remote Sensing and Photogrammetry Society and John Wiley &Sons Ltd

182

The Photogrammetric Record is slightly decreased with an increasing proportion of outliers.However,both the Recall and Precision of the other four methods signi?cantly decrease as the outlier percentage increases,especially for noise proportions higher than100%.These experiments were also repeated for the other test images and approximately similar results were obtained.These results indicate the robustness of the proposed method against a large proportion of outliers.

The proposed local af?ne consistency check computation is a relatively fast process. Thus,the ef?ciency of the proposed matching algorithm depends directly on the local feature extraction and clustering https://www.sodocs.net/doc/4c11335271.html,ing a2á4GHz computer,about80s was required to perform the feature extraction and clustering,and2s for the local af?ne consistency check process,in the test image pairs.

The proposed algorithm requires certain local(approximately)planar areas with dense extraction of local features.Therefore,it cannot be applied to images with very small elevated objects(building roofs,for example)or a very small number of local features.

Conclusions

In this paper,an ef?cient and robust approach for reliable and uniform image matching for very high resolution remote sensing images is introduced.The proposed method utilises various algorithms,including the Harris operator,UR-SIFT,MSER,k-means clustering,a local consistency check based on an af?ne model and a rechecking process.Experimental results on a variety of very high resolution aerial and satellite image pairs demonstrate the algorithm’s capability in terms of Recall and Precision.The proposed method offers higher values of Recall and Precision compared with other methods including RANSAC+epipolar,RANSAC+projective,GTM and UR-SIFT+k-means.The most important characteristic of the proposed method is its capability to extract large numbers of correct matches with the minimum number of outliers.The proposed matching approach uses the simple k-means methods.Clearly,various existing extensions of the k-means may also be applied to the proposed approach in future work.Based on the matching results,the proposed method can be applied to a variety of photogrammetric and remote sensing applications such as image registration,digital elevation model generation and image mosaicking.

references

Adam,A.,Rivlin,E.and Shimshoni,I.,2001.ROR:rejection of outliers by rotations.IEEE Transactions on Pattern Analysis and Machine Intelligence,23(1):78–84.

Aguilar,W.,Frauel,Y.,Escolano,F.,Martinez-Perez,M.E.,Espinosa-Romero,A.and Lozano, M. A.,2009.A robust graph transformation matching for non-rigid registration.Image and Vision Computing,27(7):897–910.

Barazzetti,L.,Scaioni,M.and Remondino, F.,2010.Orientation and3D modelling from markerless terrestrial images:combining accuracy with automation.Photogrammetric Record,25(132):356–381. Belongie,S.,Malik,J.and Puzicha,J.,2002.Shape matching and object recognition using shape contexts.

IEEE Transactions on Pattern Analysis and Machine Intelligence,24(4):509–522.

Brown,M.,Hua,G.and Winder,S.,2011.Discriminative learning of local image descriptors.IEEE Transactions on Pattern Analysis and Machine Intelligence,33(1):43–57.

Cheng,L.,Gong,J.,Yang,X.,Fan,C.and Han,P.,2008.Robust af?ne invariant feature extraction for image matching.IEEE Geoscience and Remote Sensing Letters,5(2):246–250.

Cheng,L.,Li,M.,Liu,Y.,Cai,W.,Chen,Y.and Yang,K.,2012.Remote sensing image matching by integrating af?ne invariant feature extraction and https://www.sodocs.net/doc/4c11335271.html,puters&Electrical Engineering,38(4): 1023–1032.

Chureesampant,K.and Susaki,J.,2014.Automatic GCP extraction of fully polarimetric SAR images.IEEE Transactions on Geoscience and Remote Sensing,52(1/1):137–148.

?2015The Authors

The Photogrammetric Record?2015The Remote Sensing and Photogrammetry Society and John Wiley&Sons Ltd183

Sedaghat and Ebadi.Very high resolution image matching based on local features and k-means clustering

Ekhtari,N.,Zoej,M.J.V.,Sahebi,M.R.and Mohammadzadeh, A.,2009.Automatic building extraction from LIDAR digital elevation models and WorldView imagery.Journal of Applied Remote Sensing,3(1):033571–033571-12.

Fischler,M. A.and Bolles,R. C.,1981.Random sample consensus:a paradigm for model?tting with applications to image analysis and automated https://www.sodocs.net/doc/4c11335271.html,munications of the ACM,24(6):381–395. Gauglitz,S.,H€o llerer,T.and Turk,M.,2011.Evaluation of interest point detectors and feature descriptors for visual tracking.International Journal of Computer Vision,94(3):335–360.

Ghassabi,Z.,Shanbehzadeh,J.,Sedaghat, A.and Fatemizadeh, E.,2013.An ef?cient approach for robust multimodal retinal image registration based on UR-SIFT features and PIIFD descriptors.EURASIP Journal on Image and Video Processing,2013(1):1–16.

Gianinetto,M.,2012.Automatic co-registration of satellite time series.Photogrammetric Record,27(140): 462–470.

Goncalves,H.,Corte-Real,L.and Goncalves,J.A.,2011.Automatic image registration through image segmentation and SIFT.IEEE Transactions on Geoscience and Remote Sensing,49(7):2589–2600.

Gong,M.,Zhao,S.,Jiao,L.,Tian,D.and Wang,S.,2014.A novel coarse-to-?ne scheme for automatic image registration based on SIFT and mutual information.IEEE Transactions on Geoscience and Remote Sensing,52(7):4328–4338.

Goshtasby,A.A.,2012.Image Registration:Principles,Tools and Methods.Springer,Berlin,Germany.442 pages.

Gruen, A.,1985.Adaptive least squares correlation:a powerful image matching technique.South African Journal of Photogrammetry,Remote Sensing and Cartography,14(3):175–187.

Gruen, A.,2012.Development and status of image matching in photogrammetry.Photogrammetric Record, 27(137):36–57.

Han,Y.,Choi,J.,Byun,Y.and Kim,Y.,2014.Parameter optimization for the extraction of matching points between high-resolution multisensor images in urban areas.IEEE Transactions on Geoscience and Remote Sensing,52(9):5612–5621.

Heikkil€a,M.,Pietik€a inen,M.and Schmid, C.,2009.Description of interest regions with local binary patterns.Pattern Recognition,42(3):425–436.

Huang,L.and Li,Z.,2010.Feature-based image registration using the shape context.International Journal of Remote Sensing,31(8):2169–2177.

Izadi,M.and Saeedi,P.,2012.Robust weighted graph transformation matching for rigid and nonrigid image registration.IEEE Transactions on Image Processing,21(10):4369–4382.

Kim,T.,2000.A study on the epipolarity of linear pushbroom images.Photogrammetric Engineering& Remote Sensing,66(8):961–966.

Kurz,T.H.,Buckley,S.J.,Howell,J. A.and Schneider, D.,2011.Integration of panoramic hyperspectral imaging with terrestrial lidar data.Photogrammetric Record,26(134):212–228.

Le Moigne,J.,Netanyahu,N.S.and Eastman,R. D.(Eds.),2011.Image Registration for Remote Sensing.Cambridge University Press,Cambridge,UK.484pages.

Liu,Z.,An,J.and Jing,Y.,2012.A simple and robust feature point matching algorithm based on restricted spatial order constraints for aerial image registration.IEEE Transactions on Geoscience and Remote Sensing,50(2):514–527.

Lowe, D.G.,2004.Distinctive image features from scale-invariant keypoints.International Journal of Computer Vision,60(2):91–110.

Matas,J.,Chum,O.,Urban,M.and Pajdla,T.,2004.Robust wide-baseline stereo from maximally stable extremal regions.Image and Vision Computing,22(10):761–767.

Mikolajczyk,K.and Schmid,C.,2005.A performance evaluation of local descriptors.IEEE Transactions on Pattern Analysis and Machine Intelligence,27(10):1615–1630.

Mikolajczyk,K.,Tuytelaars,T.,Schmid, C.,Zisserman,A.,Matas,J.,Schaffalitzky,F.,Kadir, T.and van Gool,L.,2005.A comparison of af?ne region detectors.International Journal of Computer Vision,65(1–2):43–72.

Mohammadi,N.and Malek,M.,2014.VGI and reference data correspondence based on location-orientation rotary descriptor and segment matching.Transactions in GIS,DOI:10á1111/tgis.12116.

Parmehr,E.G.,Fraser,C.S.,Zhang,C.and Leach,J.,2014.Automatic registration of optical imagery with 3D LiDAR data using statistical similarity.ISPRS Journal of Photogrammetry and Remote Sensing,88:28–40. Remondino, F.,El-Hakim,S. F.,Gruen, A.and Zhang,L.,2008.Turning images into3-D models–development and performance analysis of image matching for detailed surface reconstruction for heritage objects.IEEE Signal Processing Magazine,25(4):55–65.

Remondino, F.,Spera,M.G.,Nocerino, E.,Menna, F.and Nex, F.,2014.State of the art in high density image matching.Photogrammetric Record,29(146):144–166.

?2015The Authors 184

The Photogrammetric Record?2015The Remote Sensing and Photogrammetry Society and John Wiley&Sons Ltd

The Photogrammetric Record

Schwind,P.,Suri,S.,Reinartz,P.and Siebert,A.,2010.Applicability of the SIFT operator to geometric SAR image registration.International Journal of Remote Sensing,31(8):1959–1980.

Sedaghat,A.,Mokhtarzade,M.and Ebadi,H.,2011.Uniform robust scale-invariant feature matching for optical remote sensing images.IEEE Transactions on Geoscience and Remote Sensing,49(11/2):4516–4527. Sedaghat, A.,Ebadi,H.and Mokhtarzade,M.,2012.Image matching of satellite data based on quadrilateral control networks.Photogrammetric Record,27(140):423–442.

Song,Z.,Zhou,S.and Guan,J.,2014.A novel image registration algorithm for remote sensing under af?ne transformation.IEEE Transactions on Geoscience and Remote Sensing,52(8):4895–4912.

Suri,S.and Reinartz,P.,2010.Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas.IEEE Transactions on Geoscience and Remote Sensing,48(2):939–949.

Tack, F.,Goossens,R.and Buyuksalih,G.,2012.Assessment of a photogrammetric approach for urban DSM extraction from tri-stereoscopic satellite imagery.Photogrammetric Record,27(139):293–310. Theiler,J.and Wohlberg,B.,2012.Local coregistration adjustment for anomalous change detection.IEEE Transactions on Geoscience and Remote Sensing,50(8):3107–3116.

Tuytelaars,T.and Mikolajczyk,K.,2008.Local invariant feature detectors:a survey.Foundations and Trends in Computer Graphics and Vision,3(3):177–280.

Wang,L.,Niu,Z.,Wu,C.,Xie,R.and Huang,H.,2012.A robust multisource image automatic registration system based on the SIFT descriptor.International Journal of Remote Sensing,33(12):3850–3869.

Wong, A.and Clausi, D. A.,2007.ARRSI:automatic registration of remote-sensing images.IEEE Transactions on Geoscience and Remote Sensing,45(5):1483–1493.

Xiong,Z.and Zhang,Y.,2009.A novel interest-point-matching algorithm for high-resolution satellite images.

IEEE Transactions on Geoscience and Remote Sensing,47(12/2):4189–4200.

Zhang,Y.,Xiong,J.and Hao,L.,2011.Photogrammetric processing of low-altitude images acquired by unpiloted aerial vehicles.Photogrammetric Record,26(134):190–211.

Zhao,F.,Huang,Q.,Wang,H.and Gao,W.,2010.MOCC:a fast and robust correlation-based method for interest point matching under large scale changes.EURASIP Journal on Advances in Signal Processing, DOI:10á1155/2010/410628.

R e sum e

L’appariement d’image est une op e ration critique en photogramm e trie et en t e l e d e tection.Apparier des primitives de mani e re?able et automatique en utilisant des points bien r e partis dans des images a tr e s haute r e solution est une t^a che dif?cile en raison d’importants d e placements dus aux b^a timents et au relief naturel.Cet article propose une approche robuste et ef?cace d’appariement d’images,compos e e de deux e tapes principales. La premi e re e tape consiste a extraire dans l’image enti e re trois ensembles de primitives locales–points de Harris,UR-SIFT et MSER.Un descripteur SIFT est alors cr e e pour chaque primitive extraite,et une premi e re v e ri?cation est effectu e e au moyen d’un appariement utilisant la distance euclidienne entre les descripteurs extraits.Lors de la deuxi e me e tape,un algorithme de partitionnement bas e sur les k-moyennes permet d’obtenir un appariement pr e cis sans aucune primitive non appari e e,suivi d’un test de consistance utilisant un mod e le local de transformation af?ne pour chaque r e https://www.sodocs.net/doc/4c11335271.html, m e thode propos e e a permis d’apparier avec succ e s plusieurs images a e riennes et spatiales et les r e sultats montrent sa robustesse et ses performances.

Zusammenfassung

Die digitale Bildzuordnung ist ein kritischer Prozess in der Photogrammetrie und der Fernerkundung.Wegen signi?kanter Reliefversetzung durch hohe Geb€a ude und bewegtem Gel€a nde,ist eine automatische und zuverl€a ssige merkmalsbasierte Zuordnung von gut verteilten Punkten in sehr hoch au?€o senden Bilddaten schwierig.Dieser Beitrag stellt als L€o sung einen robusten und ef?zienten Bildzuordnungsansatz vor,der aus zwei Schritten besteht. In einem ersten Schritt werden drei S€a tze lokaler Merkmale–Harris Punkte,UR-SIFT und MSER im gesamten Bild extrahiert.F€u r jedes extrahierte Merkmal wird eine SIFT Beschreibung erzeugt,und eine initiale Korrelation mittels der Euklidischen Distanz zwischen den Merkmalsbeschreibungen durchgef€u hrt.Im zweiten Schritt wird, basierend auf K-Means Clusteranalyse,eine genaue,robuste Zuordnung durchgef€u hrt,gefolgt von einer

?2015The Authors

The Photogrammetric Record?2015The Remote Sensing and Photogrammetry Society and John Wiley&Sons Ltd185

JPEG图像的编解码实现

毕业论文论文题目（中文）JPEG图像的编解码实现 论文题目（外文）Encoding and decoding of JPEG image

摘要 JPEG是一种十分先进的图像压缩技术，它用有损压缩方式去除冗余的图像数据，在获得极高的压缩率的同时能展现十分丰富生动的图像。本文设计和实现一个JPEG图像编解码器来进行图像转换，利用离散余弦变换、熵编码、Huffman编码等图像压缩技术将BMP图像转换成JPEG图像，即进行图像的压缩。验证JPEG压缩编码算法的可行性。通过比对图像压缩前后实际效果，探讨压缩比，峰值信噪比等评价图像数据压缩程度及压缩质量的关键参数，对JPEG 压缩编码算法的实用性和优越性进行研究。 关键词：JPEG；编码；解码；图像压缩

Abstract JPEG is a very advanced image compression technology, it uses lossy compression to remove redundant image data, in obtaining a very high compression rate can show a very rich and vivid image. In this project, a JPEG image codec is designed and implemented to transform image, using discrete cosine transform, entropy coding, Huffman coding and other image compression techniques to convert BMP images into JPEG images. Verifies the feasibility of JPEG compression coding algorithm. Through the comparison of the actual effect of image compression, the key parameters of compression ratio, peak Snr, and the compression quality of image data are discussed, and the practicability and superiority of JPEG compression coding algorithm are researched. Key words: JPEG; encoding; decoding; image compression

智能图像分析系统

智能图像分析系统 解 决 方 案

北京恒泰同兴科技有限公司北京恒泰同兴科技有限公司是注册在中关村科技园区的高科技企业，成立于2004年，具有稳定的研发、生产、销售、服务队伍。恒泰同兴坚持自主开发之路，以“创造最大核心价值”为目标，以数字化、网络化、智能化为发展方向，专业从事图像智能识别、分析判断及自动处理产业化研究；公司研发的智能图像处理系统，与传统监控系统配合，为视频监控系统提供具有智能图像识别分析和告警的功能。可实现周界警戒与入侵检测、警戒线穿越检测、重要物品看护、遗留/遗弃物品检测、人体行为识别、道路交通检测等功能，可在各种恶劣气候、环境条件下进行目标识别和检测，避免了人工监控存在的易疲劳、易疏忽、反应速度慢、人工费用高等诸多不足，为客户提供了最佳安全监控系统解决方案。同时公司成功地开发大型行业联网解决方案，并有大量的实际案例，在视频监控行业积累了丰富的经验，智能监控和联网平台为用户提供了全方位的解决方案。公司本着诚实守信的经营之道，整合各种先进的技术资源，为客户定制最先进的行业解决方案，与各界用户一道，共同推进图像视频监控数字化、智能化和网络化进程。 恒泰同兴：持之以恒、稳如泰山 诚实、守信、专业、共赢

一、智能产品简介 智能视频分析系统是由位于前端或后端视频分析服务器，对监控摄像机所拍摄的视频图像进行分析，能将影像中的人、车或者物体的状态从任何背景中分离出来，加以辨认、分析与追踪。比对出所追踪对象的行为模式与预设的诸项安全规则，若发现违规之处，立刻进行报警通知，同时由使用平台进行信息记录或显示。 二、智能分析的功能 目前，智能视频分析系统在视频监控方向的应用主要在对运动目标的识别、分类和追踪。可以设置的规则、功能为以下几种：1、绊线检测 针对人、车通过特定运动方向绊线的监控；其应用如：警戒线、单向闸门流向、 栅栏攀爬…等；支持警戒区内多个目标同时告警、显示、报警图片抓拍、而且有 声音提示

图像编码技术的研究和应用

图像编码技术的研究和应用 一幅二维图像可以表示为将一个二维亮度函数通过采样和量化而得到的一个二维数组。这样一个二维数组的数据量通常很大,从而对存储、处理和传输都带来了许多问题,提出了许多新的要求。为此人们试图采用对图像新的表达方法以减少表示一幅图像需要的数据量,这就是图像编码所要解决的主要问题。压缩数据量的主要方法是消除冗余数据,从数学角度来讲是要将原始图像转化为从统计角度看尽可能不相关的数据集。这个转换要在图像进行存储、处理和传输之前进行,而在这之后需要将压缩了的图像解压缩以重建原始图像或其近似图像.图像压缩和图像解压缩,通常也分别称为图像编码和图像解码。 图像编码系统模型模型主要包括2个通过信道级连接的结构模块 :编码器和解码器。当一幅输入图像送入编码器后 ,编码器根据输入数据进行信源编码产生一组信号。这组信号在进一步被信道编码器编码后进入信道。通过信道传输后的码被送入信道解码器和信源解码器 ,解码器重建输出的图像。一般来说 ,输出图是输入图的精确复制 ,那么系统是无失真的或者信息保持型的 ;否则 ,称系统是信息损失的。 现代编码方法 这里介绍了几种比较热的编码方法:第二代编码方法、分形编码、模型编码、神经网络编码、小波变换编码。 1.第二代图像编码方法 第二代图像编码方法是针对传统编码方法中没有考虑人眼对轮廓、边缘的特殊敏感性和方向感知特性而提出的。它认为传统的第一代编码技术以信息论和数字信号处理技术为理论基础 ,出发点是消除图像数据的统计冗余信息 ,包括信息熵冗余、空间冗余和时间冗余。其编码压缩图像数据的能力已接近极限 ,压缩比难提高。第二代图像编码方法充分利用人眼视觉系统的生理和心理视觉冗余特性以及信源的各种性质以期获得高压缩比,这类方法一般要对图像进行预处理，将图像数据根据视觉敏感性进行分割。 2．分形图像编码 分形图像编码是在分形几何理论的基础上发展起来的一种编码方法。分形理论是欧氏几何相关理论的扩展,是研究不规则图形和混沌运动的一门新科学。它描述了自然界物体的自相似性,这种自相似性可以是确定的,也可以是统计意义上的。这一理论基础决定了它只有对具备明显自相似性或统计自相似性的图像,例如海岸线，云彩，大树等才有较高的编码效率。而一般图像不具有这一特性,因此编码效率与图像性质学特性有关 ,而且分形图像编码方法实质上是通过消除图像的几何冗余度来压缩数据的 ,根本没有考虑人眼视觉特性的作用。 3.基于模型的图像编码 基于模型的图像编码技术是近几年发展起来的一种很有前途的编码方法。它利用了计算机视觉和计算机图形学中的方法和理论 ,其基本出发点是在编、解码两端分别建立起相同的模型 ,针对输入的图像提取模型参数或根据模型参数重建图像。模型编码方法的核心是建模和提取模型参数,其中模型的选取、描述和建立是决定模型编码质量的关键因素。为了对图像数据建模, 一般要求对输入图像要有某些先验知识。目前研究最多、进展最快的是针对可视电话应用中的图像序列编码。这类应用中的图像大多为人的头肩像。 4.神经网络图像编码

图像处理在航天航空中的应用-结业论文

图像处理在航天航空中的应用-结业论文

论文题目：图像处理在航天和航空技术方面的运用 学院：机械电气工程学院 班级： 2012级机制3班 姓名：张娜 学号： 20125009077

摘要：图像处理技术的研究和应用越来越受到社会发展的影响，并以自身的技术特点反过来影响整个社会技术的进步。本文主要简单概括了数字图像处理技术的特点、优势，列举了数字图像处理技术的应用领域并详细介绍了其在航天航空领域中的发展。 关键字：图像处理简介技术的优点发展技术应用 一、引言 数字图像处理是通过计算机采用一定的算法对图像图形进行处理的技术，它已经在各个领域上都有了较广泛的应用。图像处理的信息量很大，对处理速度要求也很高。本文就简单的介绍图像处理技术及其在各个领域的应用，详细说明图像处理在航天航空技术方面的应用。 二、数字图像处理简介 （一）图像的概念 图像包含了它所表达的物体的描述信息。我们生活在一个信息时代，科学研究和统计表明，人类从外界获得的信息约有百分之七十来自视觉系统，也就是从图像中获得，即我们平常所熟知的照片，绘画，动画。视像等。 （二）数字图像处理技术 数字图像处理又称为计算机图像处理，它是指将图像信号转换成数字信号并利用计算机对其进行处理的过程。图像处理技术着重强调在图像之间进行的变换，主要目标是要对图像进行各种加工以改善图像的视觉效果并为其后的目标自动识别打基础，或对图像进行压缩编码以减少图像存储所需要的空间或图像传输所需的时间。图像处理是比较低层的操作，它主要在图像像素级上进行处理，处理的数据量非常大。数字图像处理的早期应用是对宇宙飞船发回的图像所进行的

智能视频分析系统

智能视频分析系统

目录 一、项目背景及建设目标 (3) 1.1 项目背景 (3) 1.2 技术优势 (4) 二、厂区智能视频分析整体设计方案 (5) 2.1传统对射系统与智能视频分析系统比较 (5) 2.2厂房周界入侵报警系统 (6) 2.2.1 周界入侵检测 (7) 2.2.2 周界警戒线警戒区预警 (8) 2.3厂房仓库物资看护 (8) 2.3.1 可疑人员接近仓库提醒 (8) 2.3.2 仓库物品看护 (9) 2.3.3 夜间停车场、厂区内部、附近可疑逗留检测 (9) 2.4夜间厂区办公楼内可疑人员检测 (10) 2.5生产车间危险区域或者夜间下班后人员检测 (10) 2.6系统拓扑结构 (11)

一、项目背景及建设目标 1.1 项目背景 慧视科技智能视频分析系统是以软件的形式实现智能视频分析功能，拥有自主的软件知识产权，可满足各行业的需要，也满足各厂家设备的接入，同时可以与各种监控平台进行二次对接。传统报警设备的误报多漏报多操作复杂不直观已经成为行业共识，且传统的视频监控系统数量庞大画面单一，工作人员很难从视频中发现问题，往往更多用于事后取证，智能图像分析通过图像中目标的识别和规则运用来进行预警，报警速度快且精确度高，可辅助工作人员从繁琐重复的工作中解放出来，真正体现科技为人服务的理念。 国内现有厂房的视频监控系统主要由摄像机、光缆、矩阵、硬盘录像机和电视墙等组成。由于视频监控图像数量大，内容枯燥，现有系统即使配备值班人员，在大多数情况下仍处于无人观看的状态下。当犯罪事件发生时，从硬盘录像机中调取录像回放、取证变成系统主要的价值之一。即使值班人员在岗，由于人的生理特点，不可能长时间有效观察多路图像，很可能造成遗漏可疑事件，对安全形式产生错误判断。 智能视频监控技术可以理解为用计算机来帮助值班人员"看"监控录像。现代计算机的高可靠性可以提供24小时不间断地保护。从根本上杜绝由于人员疲劳造成的遗漏问题。同时也可以防止出现监控人员内外勾结的可能性。

数字视频技术及应用复习题

第一章数字视频概述 1．什么是复合视频？2页，可改为填空题 例如：黑白视频信号是一个已经经过加工处理并包含扫描同步和消隐的图像信号，通常也叫做复合视频，简称视频。由于频带范围在1-6MHZ人们又把它叫做电视基带视频。 2．什么是视频技术？它主要应用在哪些领域？3页，可以改为填空题 例如：在不考虑电视调制发射和接收等诸多环节时，单纯考虑和研究电视基带信号的摄取、改善、传输、记录、编辑、显示的技术就叫做视频技术。 主要应用领域：广播电视的摄录编系统、安全及监控、视频通信和视频会议、远程教育及视听教学、影像医学、影音娱乐和电子广告。 3．什么是数字视频？5页 广义的数字视频表述为数字视频是指依据人的视觉暂留特性，借着计算机或微处理器芯片的高速运算，加上Codec技术、传输存储技术等来实现的以比特流为特征的，能按照某种时基规律和标准在显示终端上再现活动影音的信息媒介。狭义的数字视频时指与具体媒体格式所对应的数字视频。 第二章彩色数字视频基础 1．彩色电视系统是根据色光三基色原理来再现彩色图像的。按照此原理，任何一种色光颜色都可以用R G B三个彩色分量按一定的比例混合得到。7页 2．匹配兼容制彩色电视亮度信号的公式是：8页（2-2） 3．两个色差信号正交调制的目的是什么？10页 4．电视扫描分为逐行扫描和隔行扫描两种。 5．电视基带视频有复合视频、亮色分离视频和分量视频三种。13页 6．彩色电视制式有哪三种？制式差异主要体现在哪些方面？14页或改为填空 世界上现行的彩色电视制式有NTSC制式、PAL制式和SECAM制式三大制式。制式差异主要体现在亮度合成公式、色差信号提取、色副载选取及其正交调制类型、扫描方式、同步时基确定等方面的参数。 7．彩色电视图像的数字化有信号上游数字化和信号下游数字化两种。 8．A/D转换主要包括哪些环节？量化的实质是什么？编码的实质是什么？17，18页，可改为填空 A/D转换就是指对幅值连续变化的模拟视频电信号进行脉冲抽样保持、量化、编码等环节后形成二进制码流的技术处理过程。 9．一般常用的线性D/A转换器，其输出模拟电压U和输入数字量D之间成正比关系。19页 10．YCbCr信号和YUV信号是正比关系。21页，或选择A正比B反比C非线性D平方11．CCIR601标准为NTSC、PAL、和SECAM制式规定了共同的图像采样频率是13.5MHZ。21页 12．PAL制NTSC制的现行标准数字电视有效显示分辨率（清晰度）各为720X576像素和720X480像素。公用中分辨率为352X288像素。23页 第三章广义数字视频及分类 1．广义数字视频的定义？28页 2．广义的数字视频是依据人的视觉暂留特性，借助计算机或微处理器芯片的高速运算加上Codec编解码技术、传输存储技术等来实现的比特流为特征的全新的信息媒介。 3．图像序列的特点有哪些？33页 特点是每帧的分辨率相同，图像内容相关、图像文件名连续编号，而且有表示开始的图像序列头和表示结束的图像终止码。

图像处理技术原理及其在生活中的应用探讨

图像处理技术原理及其在生活中的应用探讨 摘要在社会生活实践中，图像处理技术获得了广泛的应用。这种技术之所以可以得到广泛应用，与其极强的功能所分不开的。在计算机算法不断改善的过程中，图像处理技术的发展前景是非常广阔的。笔者对图像处理技术的原理进行了分析，并其对在生活中的应用进行了探究[1]。 关键词图像处理技术原理；生活；应用 1 图像处理技术的原理分析 所谓的图像处理技术，就是通过计算机技术以及相关的技术来对图像进行处理，从而使图像更好地为我们所利用的一种技术。在这个过程中，需要运用到几个技术要点。第一个就是使图像进行转换，从而得到计算机容易识别的矩阵，这种矩阵被称为是“数字矩阵”。这样得到的矩阵更容易被计算机所存储。第二就是通过多种算法来实现对计算机所存储的图像进行有关处理，其中用到的常用算法就有基于人眼视觉特性的阈值算法、具有去噪功能的图像增强算法等。第三就是在进行了一些技术性的处理，然后获取图像信息。通过中国知网、万方数据库等平台所查阅到的图像类型相关资料可知，图像的类型主要可以分为两大类，一类是数字化图像，另一类是模拟图像。前者不仅处理便捷，而且精度较高，能够适应现代社会的发展要求，后者在现实生活中的应用更为常见，比如在相机图片中的应用。模拟图像输出较为简单，灵活性和精度不太高，因此其使用的限制性较大[2]。 2 图像处理技术原理在生活中的应用探讨 2.1 图像处理技术原理在安全防范中的应用 在安全防范监控系统不断发展的过程中，系统从模拟向数字的方向发展，这跟人们要求图像的精准度越来越高有关。在安防领域，图像处理技术如果能够得到很好的利用，那么就可以实现对图像的去噪声处理，对失真的图像进行矫正处理。在公安部门破案的过程中，有时会根据犯罪现场的指纹特征来对视频采集参数进行调节，比如色彩补偿就是一种很好的调節方法，这样方便公安部门更快地破案。尽管现在的监控系统越来越完善，但是如果遇到暴风暴雨和雾霾或者光线较弱的天气，那么监控得到的视频图像往往还是比较模糊的，对于这些模糊的图像，可以通过图像增强技术进行一些处理，从而为后续的公安部门调查和取证提供便利，模糊图像处理技术这时就排上了用场[3]。 2.2 图像处理技术原理在娱乐休闲领域的应用 在娱乐休闲领域，图像处理技术原理主要的应用场合就是平时我们利用手机或数码相机摄影以及电影特效制作等场合。在数码相机出现以前，图像只能使用传统相机通过胶片的形式保存。在数码相机出现之后，人们就可以短时间内对相

图像处理技术及其应用

图像处理技术及其应用 姓名： （班级：学号：） 【摘要】图像处理技术的研究和应用越来越收到社会发展的影响，并以自身的技术特点反过来影响整个社会技术的进步。本文主要简单概括了数字图像处理技术近期的发展及应用现状，列举了数字图像处理技术的主要优点和制约其发展的因素，同时设想了图像处理技术在未来的应用和发展。 【关键字】图像处理；发展；技术应用 1 引言 计算机图像处理技术是在20世纪80年代后期，随着计算机技术的发展应运而生的一门综合技术。图像处理就是利用计算机、摄像机及其它有关数字技术，对图像施加某种运算和处理，使图像更加清晰，以提取某些特定的信息，从而达到特定目的的技术。随着多媒体技术和网络技术的快速发展，数字图像处理已经广泛应用到了人类社会生活的各个方面，如：遥感，工业检测，医学，气象，通信，侦查，智能机器人等。无论在哪个领域中，人们喜欢采用图像的方式来描述和表达事物的特性与逻辑关系，因此，数字图像处理技术的发展及对其的要求就越来显得重要。 2 图像处理技术发展现况 进入21世纪，随着计算机技术的迅猛发展和相关理论的不断完善，数字图像处理技术在许多应用领域受到广泛重视并取得了重大的开拓性成就。随着计算机技术和人工智能、思维科学研究的迅速发展，数字图像处理向更高、更深层次发展。人们已开始研究如何用计算机系统解释图像，实现类似人类视觉系统理解外部世界，这被称为图像理解或计算机视觉。 从图像变换方面来讲，目前新兴研究的小波变换在时域和频域中都具有良好的局部化特性，它在图像处理中也有着广泛而有效的应用；而图像增强和复原图像增强和复原的目的是为了提高图像的质量，如去除噪声，提高图像的清晰度等，目前主要在指纹图像增强处理技术，医学影像学方面有显著的成果。这项技术使得各自图像的空间分辨率和对比度有了更大的提高，而最新的医学图像融合则是指对医学影像信息如CT、MRI、SPECT和PET所得的图像,利用计算机技术将它们综合在一起,实现多信息的同步可视化,对多种医学影像起到互补的作用。图像分割图像分割是数字图像处理中的关键技术之一。图像分割是将图像中有意义的特征部分提取出来，这是进一步进行图像识别、分析和理解的基础。虽然目前已研究出不少边缘提取、区域分割的方法，但还没有一种普遍适用于各种图像的有效方法。因此，对图像分割的研究还在不断深入之中，是目前图像处理中研究的热点之一。 图像描述图像描述是图像识别和理解的必要前提。作为最简单的二值图像可采用其几何特性描述物体的特性，一般图像的描述方法采用二维形状描述，它有边界描述和区域描述两类方法。对于特殊的纹理图像可采用二维纹理特征描述。随着图像处理研究的深入发展，已经开始进行三维物体描述的研究，提出了体积描述、表面描述、广义圆柱体描述等方法；图像分类（识别）图像分类（识别）属于模式识别的范畴，其主要内容是图像经过某些预处理（增强、复原、压缩）后，进行图像分割和特征提取，从而进行判决分类。近年来新发展起来的模糊模式识别和人工神经网络模式分类在图像识别中也越来越受到重视。 3 图像处理技术应用现状 图像是人类获取和交换信息的主要来源，因此，图像处理的应用领域必然涉及到人类生活和工作的方方面面。随着人类活动范围的不断扩大，图像处理的应用领域也将随之不断扩大。 3.1航天和航空技术方面的应用 数字图像处理技术在航天和航空技术方面的应用，许多国家每天派出很多侦察飞

智能视频分析系统解决方案

智能视频分析系统解决方案 1.1 系统概述 智能视频（Intelligent Video）技术源自计算机视觉（Computer Vision）与人工智能（Artificial Intelligent）的研究，其发展目标是在图像与事件描述之间建立一种映射关系，使计算机从纷繁的视频图像中分辩、识别出关键目标物体。这一研究应用于安防视频监控系统，将能借助计算机强大的数据处理能力过滤掉图像中无用的或干扰信息，自动分析、抽取视频源中的关键有用信息，从而使传统监控系统中的摄像机成为人的眼睛，使“智能视频分析”计算机成为人的大脑，并具有更为“聪明”的学习思考方式。这一根本性的改变，可极大地发挥与拓展视频监控系统的作用与能力，使监控系统具有更高的智能化，大幅度节省资源与人员配置，同时必将全面提升安全防范工作的效率。因此，智能视频监控不仅仅是一种图像数字化监控分析技术，而是代表着一种更为高端的数字视频网络监控应用。 智能视频分析包含视频诊断、视频分析和视频增强等，它们各自又包含了大量的功能算法，比如清晰度检测、视频干扰检测、亮度色度检测、PTZ（云台）控制功能检测，以及视频丢失、镜头遮挡、镜头喷涂、非正常抖动等检测都属于视频诊断内容，而视频分析算法则包含区域入侵、绊线检测、遗留遗失检测、方向检测、人群计数、徘徊检测、流量统计、区域稠密度统计、人脸识别、车牌识别、烟火烟雾检测、自动 PTZ 跟踪等功能，视频图像增强则包括稳像、去雾、去噪、全景拼接等算法。由此组合衍生出的算法种类又有很多，应用方式也千变万化，所以智能视频分析的应用范围很广。 在以往的视频监控系统中，操作人员盯着屏幕电视墙超过 10 分钟后将漏掉90％的视频信息，而使视频监控工作失去意义。随着社会发展，视频监控被越来越广泛地应用到各行各业中，摄像机数量越来越庞大，这给传统的视频监控带来严峻的挑战。针对行业发展推出智能视频分析系统，主要解决以下问题：一个是将安防操作人员从繁杂而枯燥的“盯屏幕”任务解脱出来，由机器来完成分析识别工作；另外一个是为在海量的视频数据中快速搜索到想要找的的图象。 1.2 系统组成 智能视频分析系统以数字化、网络化视频监控为基础，用户可以设置某些特定的规则，系统识别不同的物体，同时识别目标行为是否符合这些规则，一旦发现监控画面中的异常情况，系统能够以最快和最佳的方式发出警报并提供有用信息，从而能够更加有效的协助安全人员处理危机，最大限度的降低误报和漏报现象。智能视频分析是在传统的监控系统中，加入智能视频技术，在整个系统中，系统分布图如下：

图像编解码技术及应用

图像编解码技术及应用 1．图像编解码技术概论： 在当前的图像压缩领域中常用的技术有： BMP、EPS、GIF、JPG、PDF、PIC、PNG、PSD、TIF。上述技术间的差异主要存在于图像编解码的算法不同，通过对算法的研究可以使我们更加容易的理解图像压缩的原理。 位图格式（BMP）是在DOS时代就出现的一种元老级文件格式，因此它是DOS和WINDOWS操作系统上的标准的WINGDOWS点阵图像格式，以此文件格式存储时，采用一种非破坏性的RLE压缩,不会省略任何图像的细部信息。 EPS是最常见的线条稿共享文件格式，它是以PostScript语言为开发基础，所以EPS文件能够同时兼容矢量和点阵图形，所有的排版或图像处理软件如PageMaker或Illustrator等，都提供了读入或置入EPS格式文件的能力，而且RGB和CMYK对象也可以保有各自的原始的色彩模式。 GIF应该是在网络上最常见的一种压缩文件格式，它的英文全名Graphic Interchange format，当初研发的目的是为了最小化电缆上的传输，因此能采用LZW方式进行压缩，但可显示的颜色范围只局限于256索引色，目前所采用 的GIF图形共有两种格式：87a和89a，常见于网页上建议的小动画制作，其中GIF89a还可提供透明色效果，点阵图形，灰度图形或者索引颜色模式皆可存储为此种文件格式 JPG跟GIF一样为网络上最常见道的图像格式，其英文正式名称为Joint Photographic Experts Group，它是以全彩模式进行显示色彩，是目前最有效率的一种压缩格式，常用于照片或连续色调的显示，而且没有GIF去掉图像细 部信息的缺点，但需要注意的是此类图像需要自行设置压缩程度，在打开时JPG 图像会自动解压缩，不过要注意的是JPG采用的压缩是破坏性的压缩，因此会在一定程度上减损图像本身的品质。

图像处理技术的应用论文

图像处理技术的应用先展示一下自己用Photoshop处理的图片（做的不好望见谅）

摘要：图像处理技术的研究和应用越来越收到社会发展的影响，并以自身的技术特点反过来影响整个社会技术的进步。本文主要简单概括了数字图像处理技术近期的发展及应用现状，列举了数字图像处理技术的主要优点和制约其发展的因素，同时设想了图像处理技术在未来的应用和发展。 关键字：图像处理发展技术应用 1.概述 1.1图像的概念 图像包含了它所表达的物体的描述信息。我们生活在一个信息时代，科学研究和统计表明，人类从外界获得的信息约有百分之七十来自视觉系统，也就是从图像中获得，即我们平常所熟知的照片，绘画，动画。视像等。 1.2图像处理技术 图像处理技术着重强调在图像之间进行的变换，主要目标是要对图像进行各种加工以改善图像的视觉效果并为其后的目标自动识别打基础，或对图像进行压缩编码以减少图像存储所需要的空间或图像传输所需的时间。图像处理是比较低层的操作，它主要在图像像素级上进行处理，处理的数据量非常大。 1.3优点分析 1．再现性好。数字图像处理与模拟图像处理的根本不同在于，它不会因图像的存储、传输或复制等一系列变换操作而导致图像质量的退化。 2．处理精度高。按目前的技术，几乎可将一幅模拟图像数字化为任意大小的二维数组，这主要取决于图像数字化设备的能力。现代扫描仪可以把每个像素的灰度等级量化为16位甚至更高，这意味着图像的数字化精度可以达到满足任一应用需求。 3．适用面宽。图像可以来自多种信息源，它们可以是可见光图像，也可以是不可见的波谱图像（例如X射线图像、射线图像、超声波图像或红外图像等）。从图像反映的客观实体尺度看，可以小到电子显微镜图像，大到航空照片、遥感图像甚至天文望远镜图像。即只要针对不同的图像信息源，采取相应的图像信息采集措施，图像的数字处理方法适用于任何一种图像。 4．灵活性高。图像处理大体上可分为图像的像质改善、图像分析和图像重建三大部分，每一部分均包含丰富的内容。而数字图像处理不仅能完成线性运算，而且能实现非线性处理，即凡是可以用数学公式或逻辑关系来表达的一切运算均可用数字图像处理实现。 2.应用领域 2.1图像技术应用领域

超高分辨率显微镜技术

超高分辨率显微镜技术 为了更好地理解生命过程和疾病发生机理，生物学研究需要观察细胞内器官等细微结构的精确定位和分布，阐明蛋白等生物大分子如何组成细胞的基本结构，重要的活性因子如何调节细胞的主要生命活动等，而这些体系尺度都在纳米量级，远远超出了常规的光学显微镜（激光共聚焦显微镜等）的分辨极限。为了解决生命科学研究面临的这一难题，超高分辨率显微技术应时而生，并且一经问世就得到了广泛的响应，2008年Nature Methods将这一技术列为年度之最。 为了达到纳米量级的分辨率和极快速的成像，超高分辨率显微镜引入了许多突破时代的创新技术，了解这些技术将带领我们走入超高分辨率显微镜的奇妙世界。 3D-SIM(结构照明技术): 荧光样品通过不同方向和相位的光源照射，并且在成像后利用特点的运算方法重构，产生突破光学极限的超高分辨率图像。 ●完全兼容现有荧光分子和荧光染料、不改变任何实验流程 ●轴向分辨率提高到80-120nm，空间分辨率提高到激光共聚焦显微镜观察极限的8 倍。 搭载3D-SIM技术的DeltaVision OMX超高分辨率显微镜已经成功运用到了很多样品，比如微生物、脊椎动物细胞、组织切片甚至整个胚胎等。大大提高的分辨率在鉴定和研究亚细胞结构中成效显著，比如对微管和肌动蛋白的观察中可以解析到单根微管纤维。 Monet (单分子成像与定位技术): 通过在极短时间内对单个或几个荧光分子的激发和获取发射光信号，上千次获取后重构图像，从而获得突破百纳米极限的超高分辨率图像。这种技术需要使用独特的光敏蛋白来做荧光染料，通过独特的算法可以分辨衍射极限上重叠的荧光团位置。 ●搭载PALM的DeltaVision OMX可在极短时间内完成图像获取和重构 ●能够处理极大密度的图像，使高浓度标记的和更高激活能量的样品的成像变成 可能。 超高速成像： 研究者对于成像速度进入“亚秒时代”的需求已经十分的迫切。以往的速度瓶颈主要在曝光时间以及CCD成像速度，利用高效光路和改进的新型照相机，大大提高了成像速度。

智能视频行为分析平台建设方案

基于智能视频分析的监控平台建设方案 随着国家经济的提高，城市和城市化进程在不断的发展，各种社会矛盾和暴力事件逐渐增多，政府和相关部分对加强城市各地联网型监控系统越来越重视，当前城市和小区监控系统建设使用监控录像存储，事件发生后调取查阅的方式，这种方式在一定程度上满足了社会的需求，但是无法避免事态趋于恶化，在此背景下，具有智能视频行为分析的监控平台建设就显得尤为重要。 智能视频技术让安全警卫部门能通过摄像机实时自动“发现警情”并主动“分析”视野中的监视目标，同时判断出这些被监视目标的行为是否存在安全威胁，对已经出现或将要出现的安全威胁，及时向安全防卫人员通过文字信息、声音、快照等发出警报，极大地避免工作人员因倦怠、脱岗等因素造成情况误报和不报，切实提高监控区域的安全防范能力。 现有各大监控系统厂商和信息化科技公司都研发出大量的智能视频分析软件，可以分为两大类，基于嵌入式DSP 智能分析系统和基于计算机末端处理的智能分析系统。 一．基于嵌入式DSP的处理优点

1、DSP方式可以使得视觉分析技术采用分布式的架构方式。在此方式下，视觉分析单元一般位于视觉采集设备附近（摄像机或编码器），这样，可以有选择的设置系统，让系统只有当报警发生的时候才传输视觉到控制中心或存储中心，相对于计算机末端处理方式，大大节省的网络负担及存储空间。 2、DSP方式下视觉分析单元一般位于视觉采集设备附近（摄像机或编码器），此方式可以使得视觉分析单元直接对原始或最接近原始的图象进行分析，而后端计算机方式，计算机器得到的图象经过网络编码传输后已经丢失了部分信息，因此精确度难免下降。 3、视觉分析是复杂的过程，需要占用大量的系统计算资源，因此计算机方式可以同时进行分析的视觉路数非常有限，而DSP方式没有此限制。 二．在对比上述两种处理模式的优缺点基础上，提出基于DSP嵌入式处理和末端计算机处理两种系统结构.

视频监控智能分析技术应用分析

视频监控智能分析技术应用分析 一、概述 在视频监控飞速发展的今天，海量视频画面已经大大超过了人力有效处理的范围。而智能视频分析技术极大地发挥与拓展了视频监控系统的作用与能力，使监控系统具有更高的智能化，大幅度降低资源与人员配置，全面提升安全防范工作的效率。目前已广泛应用于平安城市、智能交通、金融行业、政法监管、商业等领域。 智能视频分析技术是计算机视觉技术在安防领域应用的一个分支，是一种基于目标行为的智能监控技术。它能够在图像或图像序列与事件描述之间建立映射关系，从而使计算机从纷繁的视频图像中分辩、识别出关键目标的行为，过滤用户不关心的信息，其实质是自动分析和抽取视频源中的关键信息。 按照智能分析算法实现的方式进行区分，可以概括为以下几种类型的智能分析： 识别类分析：该项技术偏向于对静态场景的分析处理，通过图像识别、图像比对及模式匹配等核心技术，实现对人、车、物等相关特征信息的提取与分析。如人脸识别技术、车牌识别技术及照片比对技术等。 行为类分析：该项技术侧重于对动态场景的分析处理，典型的功能有车辆逆行及相关交通违章检测、防区入侵检测、围墙翻越检测、绊线穿越检测、物品偷盗检测、客流统计等。 图像检索类分析：该技术能按照所定义的规则或要求，对历史存储视频数据进行快速比对，把符合规则或要求的视频浓缩、集中或剪切到一起，这样就能快速检索到目标视频。 图像处理类分析：主要是对图像整体进行分析判断及优化处理以达到更好的效果或者将不清楚的内容通过算法计算处理达到看得清的效果。如目前的视频增强技术(去噪、去雾、锐化、加亮等)、视频复原技术(去模糊、畸变矫正等)。 诊断类分析:该项分析主要是针对视频图像出现的雪花、滚屏、模糊、偏色、增益失衡、云台PTZ失控、画面冻结等常见的摄像头故障进行准确分析、判断和报警，如视频质量诊断技术。 二、智能分析核心算法介绍 1. 运动检测算法 帧差法

h265(HEVC)编解码相关技术概述

H265(HEVC Heigh Efficiency Video Coding)介绍 1 概要 H.265(高效率视频编码（HEVC）)是现行“H.264/MPEG-4 AVC”标准于2003年实现标准化以来时隔10年推出的新标准，将成为支撑未来十年的影像服务和产品的视频压缩技术。其特点是，支持1080p以上的4K×2K和8K×4K分辨率，将视频压缩率提高至H.264的约2倍。也就是说，能以原来一半的编码速度发送相同画质的视频。例如，按照20Mbit/秒发送的H.264格式视频内容，在相同画质的条件下用HEVC格式只需10Mbit/秒的速度。 1.1 H.265发展背景 H.264虽然是一个划时代的数字视频压缩标准，但是随着数字视频产业链的高速发展，H.264的局限性逐步显现，并且由于H.264标准核心压缩算法的完全固化，并不能够通过调整或扩充来更好地满足当前高清数字视频应用。 视频应用向以下几个方面发展的趋势愈加明显： (1)高清晰度(Higher Definition)：数字视频的应用格式从720P向1080P全面升级，在一些视频应用领域甚至出现了4K*2K、8K*4K的数字视频格式 (2)高帧率(Higher frame rate)：数字视频帧率从30fps向60fps、120fps甚至240fps的应用场景升级 (3)高压缩率(Higher Compression rate)：传输带宽和存储空间一直是视频应用中最为关键的资源，因此，在有限的空间和管道中获得最佳的视频体验一直是用户的不懈追求。 由于数字视频应用在发展中面临上述趋势，如果继续采用H.264编码就出现如下一些局限性： (1)宏块个数的爆发式增长，会导致用于编码宏块的预测模式、运动矢量、参考帧索引和量化级等宏块级参数信息所占用的码字过多，用于编码残差部分的码字明显减少。即：单个宏块所表示的图像内容的信息大大减少，导致4*4或8*8块变换后的低频率相似程度也大大提高，会出现大量的冗余 (2)分辨率的大幅增加，表示同一个运动的运动矢量的幅值将大大增加，H.264中采用一个运动矢量预测值，对运动矢量差编码使用的是哥伦布指数编码，该编码方式的特点是数值越小使用的比特数越少。因此，随着运动矢量幅值的大幅增加，H.264中用来对运动矢量进行预测以及编码的方法压缩率将逐渐降低。 (3)并行度比较低 H.264的一些关键算法，例如采用CAVLC和CABAC两种基于上下文的熵编码方法、deblock滤波等都要求串行编码，并行度比较低。针对GPU/DSP/FPGA/ASIC等这种并行化程序非常的CPU，H.264的这种串行化处理越来越成为制约运算性能的瓶颈。 基于以上视频应用的发展趋势和H.264的局限性，面向更高清晰度、更高帧率、更高压

浅谈计算机图像处理技术的应用

浅谈计算机图像处理技术的应用 摘要：随着人类正加快步入信息时代，使得计算机技术得到了飞速发展。计算机技术的广泛应用，使人们越来越多地开始将将先进的计算机技术应用到我们捕捉到的图像上，并希望通过计算机图像处理技术的应用，获得理想的效果并提取我们想要的信息。相信在先进图像处理技术发展的推动下，计算机图像处理技术的应用也将渗透到社会的各个领域。 关键词：计算机；图像处理技术；应用 The Application of Computer Image Processing Technology Abstract:As human being accelerated into the information age,making computer technology has been rapid development.Extensive application of computer technology,more and more people began to be advanced computer technology to our captured image,and that through computer image processing technology,to obtain the desired results and extract the informat ion we want.I believe the development of advanced image processing technology,driven by the application of computer image processing technology will permeate all spheres of society. Keywords:Computer;Image processing technology;Application

视频图像智能检测分析系统

视频图像智能检测分析系统 一、系统概述 智能视频监控系统具有图像内容智能识别与智能分析处理管理功能，并可通过联网实现。智能视频监控系统是一种先进的智能视频分析系统。摄像头信号通过视频服务器(视频采集卡）进行采集，基于我们的智能分析与管理平台，对采集的数据进行实时分析，及时报告可疑事件（如闯入禁区、逆行、滞留等）的发生，并对提出来的事件信息和视频数据一起记录，从而达到实时报警和事后视频有效检索的目的；能有效检测、分类、跟踪和记录非法过往行人、车辆及其它可疑物体，能够判断是否有行人及车辆在指定区域内长时间徘徊、停留或逆行；还可通过控制台摄像机放大并抓拍移动目标等。 二、系统主要功能介绍 1、物品的移动或失窃检测 自动识别出监控区域内的物品被盗等行为并发出报警信号（也可发送至用户手机或小灵通等通讯设备），自动录下相关信息。 ◆对办公室（重要人物或物品放置的地点）实行监控 ◆对博物馆、展览馆等珍贵物品的公共场所 ◆高档小区或别墅 上图中红框区域中的画为重点监控点，如该区内的画有移动迹象，即显示警示信息。多用于博物馆、展览馆等珍贵资料的保护。 2、人体行为识别

◆对视频图像进行分析，能检测警戒区域范围内以下各类人体行为并报警 ◆徘徊、滞留：在禁区或监控场景内停留超过设定时长 ◆突然加速、突然减速：由静止或匀速运动变为高速运动 ◆突然倒地或卧倒；人体直立姿势突然改变为卧地姿势 ◆车辆行为分析，识别车辆的逆行、跃线、违章乱停车并产生报警信号 3、遗留物识别 ◆可在监控区域内，一旦出现遗留物（包裹、碎块、行李等）或被蓄意放置物 体（如危险爆炸物品）立即发出告警，并自动弹出画报告遗留物的位置。 ◆在要塞地区进行可疑物品的侦测（反恐行为） ◆于机场或铁路等环境底下寻找被遗留的行李 ◆在繁忙的公路或隧道里监控故障的车辆 ◆超市或机场的地方侦测到空置的手推车以便清理 4、周界闯入、离开检测

基于压缩感知的图像视频编解码技术研究摘要

基于压缩感知的图像视频编解码技术研究 摘要： 随着多媒体和互联网技术的迅速发展，人们对数字图像视频信息（如数码图片、流媒体视频、监控视频）的需求越来越高。而数码相机的广泛应用，又衍生出高分辨率、低功耗、低成本图片和视频的需求。传统的图像视频采集方式，通常是通过获得超过有效维度的高密度观测样本。这种过度采样方式，不仅对硬件设备要求高，要求高密度的传感器；而且为了满足后续存储和传输的需求，采集得到的大量观测样本需要经过复杂的编码过程。压缩感知理论的提出，为新型低复杂度编码器的设计提供了可能性。它指出，如果一个信号在某个变换域下是稀疏的，那么仅需极少量的观测值，就可以通过优化方法高质量地恢复出原始信号。这样，基于压缩感知理论进行图像视频的采集，从而大幅度降低了为了精确重建原始信号所需要的观测值数目，在信号采集阶段就隐含着降低了信号维度，从而大大简化了后续的压缩编码过程。压缩感知理论，改变了图像视频的采集、编码和重建方式，在低功耗编码器（如分布式视频）、低成本采集设备（如高速摄像机）等应用方面具有明显优势。 本文基于压缩感知理论，针对图像和视频的编解码技术进行研究，设计出高效的编码和重建算法；并将其与传统的图像视频编解码方案相结合，在保证标准兼容的基础上，利用压缩感知技术提高编码效率。本文的创新点包括以下四个方面： 1.基于条带差分预测的压缩感知图像编码算法 在图像压缩感知系统的编码端，图像信号的采集方式，以及如何有效地对采集到的观测值进行压缩，是影响整个系统性能的关键性因素。本文提出了一种基于条带差分预测编码的压缩感知图像编码方法。将一幅图像划分为由多行像素点组成的条带，以条带为单位进行采样和编码。与图像块的划分方式相比，具有同样像素点个数的条带更扁平，从而相邻条带之间更为接近，它们的观测值也就具有更高的相关性。实验还表明，在同样的重建算法下，条带划分相比于块划分能够在保证同等重建质量的前提想，产生更小的码率。将这种条带划分方式与差分预测编码框架相结合，设计出一种条带观测值之间的预测机制。由此计算得到的观测值的残差相比原始观测值具有更低的能量，从而达到更高的压缩效率。压缩

数字图像处理技术的应用综述--课程论文

《数字图像处理》课程论文 题目：数字图像处理技术的应用综述

1 绪论 1.1数字图像处理简介 数字图像处理又称为计算机图像处理，它是指将图像信号转换成数字信号并利用计算机对其进行处理的过程。数字图像处理的早期应用是对宇宙飞船发回的图像所进行的各种处理。到了70年代，图像处理技术的应用迅速从宇航领域扩展到生物医学、信息科学、资源环境科学、天文学、物理学、工业、农业、国防、教育、艺术等各个领域与行业，对经济、军事、文化及人们的日常生活产生重大的影响。 1.2数字图像处理技术的基本特点 1）处理信息量很大。数字图像处理的信息大多是二维信息，处理信息量很大。如一幅256×256低分辨率黑白图像，要求约64kbit的数据量；对高分辨率彩色512×512图像，则要求768kbit数据量；如果要处理30帧/秒的电视图像序列，则每秒要求500kbit～22.5Mbit数据量。因此对计算机的计算速度、存储容量等要求较高。 2）占用频带较宽。数字图像处理占用的频带较宽。与语言信息相比，占用的频带要大几个数量级。如电视图像的带宽约5.6MHz，而语音带宽仅为4kHz左右。所以在成像、传输、存储、处理、显示等各个环节的实现上，技术难度较大，成本亦高，这就对频带压缩技术提出了更高的要。 3）各像素相关性大。数字图像中各个像素是不独立的，其相关性大。在图像画面上，经常有很多像素有相同或接近的灰度。就电视画面而言，同一行中相邻两个像素或相邻两行间的像素，其相关系数可达0.9以上，而相邻两帧之间的相关性比帧内相关性一般说还要大些。因此，图像处理中信息压缩的潜力很大。 4）无法复现三维景物的全部几何信息。由于图像是三维景物的二维投影，一幅图象本身不具备复现三维景物的全部几何信息的能力，很显然三维景物背后部分信息在二维图像画面上是反映不出来的。因此，要分析和理解三维景物必须作合适的假定或附加新的测量，例如双目图像或多视点图像。在理解三维景物时需要知识导引，这也是人工智能中正在致力解决的知识工程问题。 5）受人的因素影响较大。数字图像处理后的图像一般是给人观察和评价的，因此受人的因素影响较大。由于人的视觉系统很复杂，受环境条件、视觉性能、人的情绪爱好以及知识状况影响很大，作为图像质量的评价还有待进一步深入的研究。另一方面，计算机视觉是模仿人的视觉，人的感知机理必然影响着计算机