Centre dedocumentation
scientifique

Deep learning feature representation for image matching under large viewpoint and viewing direction change / Lin Chen in ISPRS Journal of photogrammetry and remote sensing, vol 190 (August 2022)  

Public [article]  inISPRS Journal of photogrammetry and remote sensing > vol 190 (August 2022) . - pp 94 -112 Titre : Deep learning feature representation for image matching under large viewpoint and viewing direction change Type de document : Article/Communication Auteurs : Lin Chen, Auteur ; Christian Heipke, Auteur Année de publication : 2022 Article en page(s) : pp 94 -112 Note générale : bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Traitement d'image optique [Termes IGN] appariement d'images [Termes IGN] apprentissage profond [Termes IGN] classification par réseau neuronal convolutif [Termes IGN] extraction de traits caractéristiques [Termes IGN] image aérienne oblique [Termes IGN] orientation d'image [Termes IGN] reconnaissance de formes [Termes IGN] réseau neuronal siamois [Termes IGN] SIFT (algorithme) Résumé : (auteur) Feature based image matching has been a research focus in photogrammetry and computer vision for decades, as it is the basis for many applications where multi-view geometry is needed. A typical feature based image matching algorithm contains five steps: feature detection, affine shape estimation, orientation assignment, description and descriptor matching. This paper contains innovative work in different steps of feature matching based on convolutional neural networks (CNN). For the affine shape estimation and orientation assignment, the main contribution of this paper is twofold. First, we define a canonical shape and orientation for each feature. As a consequence, instead of the usual Siamese CNN, only single branch CNNs needs to be employed to learn the affine shape and orientation parameters, which turns the related tasks from supervised to self supervised learning problems, removing the need for known matching relationships between features. Second, the affine shape and orientation are solved simultaneously. To the best of our knowledge, this is the first time these two modules are reported to have been successfully trained together. In addition, for the descriptor learning part, a new weak match finder is suggested to better explore the intra-variance of the appearance of matched features. For any input feature patch, a transformed patch that lies far from the input feature patch in descriptor space is defined as a weak match feature. A weak match finder network is proposed to actively find these weak match features; they are subsequently used in the standard descriptor learning framework. The proposed modules are integrated into an inference pipeline to form the proposed feature matching algorithm. The algorithm is evaluated on standard benchmarks and is used to solve for the parameters of image orientation of aerial oblique images. It is shown that deep learning feature based image matching leads to more registered images, more reconstructed 3D points and a more stable block geometry than conventional methods. The code is available at https://github.com/Childhoo/Chen_Matcher.git. Numéro de notice : A2022-502 Affiliation des auteurs : non IGN Thématique : IMAGERIE Nature : Article nature-HAL : ArtAvecCL-RevueIntern DOI : 10.1016/j.isprsjprs.2022.06.003 Date de publication en ligne : 14/06/2022 En ligne : https://doi.org/10.1016/j.isprsjprs.2022.06.003 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=101000 [article]

Application oriented quality evaluation of Gaofen-7 optical stereo satellite imagery / Jiaojiao Tian in ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol V-1-2022 (2022 edition)  

Public [article]  inISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences > vol V-1-2022 (2022 edition) . - pp 145 - 152 Titre : Application oriented quality evaluation of Gaofen-7 optical stereo satellite imagery Type de document : Article/Communication Auteurs : Jiaojiao Tian, Auteur ; Xiangyu Zhuo, Auteur ; Xiangtian Yuan, Auteur ; Corentin Henry, Auteur ; Pablo d' Angelo, Auteur ; Thomas Krauss, Auteur Année de publication : 2022 Article en page(s) : pp 145 - 152 Note générale : bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Traitement d'image optique [Termes IGN] Allemagne [Termes IGN] détection du bâti [Termes IGN] extraction du réseau routier [Termes IGN] image Gaofen [Termes IGN] image optique [Termes IGN] orientation d'image [Termes IGN] reconstruction 3D [Termes IGN] scène urbaine [Termes IGN] segmentation Résumé : (auteur) GaoFen-7 (GF-7) satellite mission is further expanding the very high resolution 3D mapping application. Carrying the first civilian Chinese sub-meter resolution stereo satellite sensors, GF-7 satellite was launched on November 7, 2019. With 0.65 meter resolution on backward view and 0.8 meter resolution forward view, GF-7 has been designed to meet the demand of natural resource monitoring, land surveying, and other mapping applications in China. The use of GF-7 for 3D city reconstruction is unfortunately restricted by the fixed large stereo view angle of forward and backward cameras with +26 and −5 degrees respectively which is not optimal for dense stereo matching in urban regions. In this paper, we intensively evaluate the quality of the GF-7 datasets by performing a series of urban monitoring applications, including road detection, building extraction and 3D reconstruction. In addition, we propose a 3D reconstruction workflow which uses the land cover classification result to refine the stereo matching result. Six sub-urban regions are selected from the available datasets in the middle of Germany. The results show that basic elements in urban scenes like buildings and roads could be detected from GF-7 datasets with high accuracy. With the proposed workflow, a 3D city model with a visually observed good quality can be delivered. Numéro de notice : A2022-442 Affiliation des auteurs : non IGN Thématique : IMAGERIE/INFORMATIQUE Nature : Article DOI : 10.5194/isprs-annals-V-1-2022-145-2022 Date de publication en ligne : 17/05/2022 En ligne : https://doi.org/10.5194/isprs-annals-V-1-2022-145-2022 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=100776 [article]

Cooperative image orientation considering dynamic objects / P. Trusheim in ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol V-1-2022 (2022 edition)  

Public [article]  inISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences > vol V-1-2022 (2022 edition) . - pp 169 - 177 Titre : Cooperative image orientation considering dynamic objects Type de document : Article/Communication Auteurs : P. Trusheim, Auteur ; Max Mehltretter, Auteur ; Franz Rottensteiner, Auteur ; Christian Heipke, Auteur Année de publication : 2022 Article en page(s) : pp 169 - 177 Note générale : bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Traitement d'image optique [Termes IGN] compensation par faisceaux [Termes IGN] orientation d'image [Termes IGN] point d'appui [Termes IGN] points homologues [Termes IGN] réseau neuronal artificiel [Termes IGN] scène urbaine [Termes IGN] séquence d'images Résumé : (auteur) In the context of image orientation, it is commonly assumed that the environment is completely static. This is why dynamic elements are typically filtered out using robust estimation procedures. Especially in urban areas, however, many such dynamic elements are present in the environment, which leads to a noticeable amount of errors that have to be detected via robust adjustment. This problem is even more evident in the case of cooperative image orientation using dynamic objects as ground control points (GCPs), because such dynamic objects carry the relevant information. One way to deal with this challenge is to detect these dynamic objects prior to the adjustment and to process the related image points separately. To do so, a novel methodology to distinguish dynamic and static image points in stereoscopic image sequences is introduced in this paper, using a neural network for the detection of potentially dynamic objects and additional checks via forward intersection. To investigate the effects of the consideration of dynamic points in the adjustment, an image sequence of an inner-city traffic scenario is used; image orientation, as well as the 3D coordinates of tie points, are calculated via a robust bundle adjustment. It is shown that compared to a solution without considering dynamic points, errors in the tie points are significantly reduced, while the median of the precision of all 3D coordinates of the tie points is improved. Numéro de notice : A2022-441 Affiliation des auteurs : non IGN Thématique : IMAGERIE/INFORMATIQUE Nature : Article DOI : 10.5194/isprs-annals-V-1-2022-169-2022 Date de publication en ligne : 17/05/2022 En ligne : https://doi.org/10.5194/isprs-annals-V-1-2022-169-2022 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=100775 [article]

Feature detection and description for image matching: from hand-crafted design to deep learning / Lin Chen in Geo-spatial Information Science, vol 24 n° 1 (March 2021)  

Public [article]  inGeo-spatial Information Science > vol 24 n° 1 (March 2021) . - pp 58 - 74 Titre : Feature detection and description for image matching: from hand-crafted design to deep learning Type de document : Article/Communication Auteurs : Lin Chen, Auteur ; Franz Rottensteiner, Auteur ; Christian Heipke, Auteur Année de publication : 2021 Article en page(s) : pp 58 - 74 Note générale : Bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Traitement d'image optique [Termes IGN] appariement automatique [Termes IGN] appariement d'images [Termes IGN] appariement de formes [Termes IGN] apprentissage automatique [Termes IGN] apprentissage profond [Termes IGN] extraction de traits caractéristiques [Termes IGN] image aérienne oblique [Termes IGN] orientation d'image [Termes IGN] SIFT (algorithme) Résumé : (Auteur) In feature based image matching, distinctive features in images are detected and represented by feature descriptors. Matching is then carried out by assessing the similarity of the descriptors of potentially conjugate points. In this paper, we first shortly discuss the general framework. Then, we review feature detection as well as the determination of affine shape and orientation of local features, before analyzing feature description in more detail. In the feature description review, the general framework of local feature description is presented first. Then, the review discusses the evolution from hand-crafted feature descriptors, e.g. SIFT (Scale Invariant Feature Transform), to machine learning and deep learning based descriptors. The machine learning models, the training loss and the respective training data of learning-based algorithms are looked at in more detail; subsequently the various advantages and challenges of the different approaches are discussed. Finally, we present and assess some current research directions before concluding the paper. Numéro de notice : A2021-297 Affiliation des auteurs : non IGN Thématique : IMAGERIE Nature : Article nature-HAL : ArtAvecCL-RevueIntern DOI : 10.1080/10095020.2020.1843376 Date de publication en ligne : 17/11/2020 En ligne : https://doi.org/10.1080/10095020.2020.1843376 Format de la ressource électronique : URL Article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=97379 [article]

Deep learning for feature based image matching / Lin Chen (2021)  

Public Titre : Deep learning for feature based image matching Type de document : Thèse/HDR Auteurs : Lin Chen, Auteur ; Christian Heipke, Directeur de thèse Editeur : Munich : Bayerische Akademie der Wissenschaften Année de publication : 2021 Collection : DGK - C, ISSN 0065-5325 num. 867 Importance : 159 p. Format : 21 x 30 cm Note générale : bibliographie Diese Arbeit ist gleichzeitig veröffentlicht in: Wissenschaftliche Arbeiten der Fachrichtung Geodäsie und Geoinformatik der Leibniz UniversitätHannoverISSN 0174-1454, Nr. 369, Hannover 2021 Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Photogrammétrie numérique [Termes IGN] appariement d'images [Termes IGN] chaîne de traitement [Termes IGN] classification par réseau neuronal convolutif [Termes IGN] descripteur [Termes IGN] image aérienne oblique [Termes IGN] orientation d'image [Termes IGN] orthoimage Résumé : (auteur) Feature based image matching aims at finding matched features between two or more images. It is one of the most fundamental research topics in photogrammetry and computer vision. The matching features area prerequisite for applications such as image orientation, Simultaneous Localization and Mapping (SLAM) and robot vision. A typical feature based matching algorithm is composed of five steps: feature detection, affine shape estimation, orientation, description and descriptor matching. Today, the employment of deep neural network has framed those different steps as machine learning problems and the matching performance has been improved significantly. One of the main reasons why feature based image matching may still prove difficult is the complex change between different images, including geometric and radiometric transformations. If the change between images exceeds a certain level, it will also exceed the tolerance of those aforementioned separate steps and, in turn, cause feature based image matching to fail. This thesis focuses on improving feature based image matching against large viewpoint and viewing direction change between images. In order to improve the feature based image matching performance under these circumstances, affine shape estimation, orientation and description are solved with deep learning architectures. In particular, Convolutional Neural Networks (CNN) are used. For the affine shape and orientation learning, the main contribution of this thesis is two fold. First, instead of a Siamese CNN, only one branch is needed and the loss is built based on the geometric measures calculated from the mean gradient or second moment matrix. Therefore, for each of the input patches, a global minimum, namely the canonical feature, exists. Second, both the affine shape and orientation are solved simultaneously within one network by combining the loss used for affine shape and orientation learning. To the best of the author’s knowledge, this is the first time these two modules are reported to have been successfully trained simultaneously. For the descriptor learning part, a new weak match is defined. For any input feature patch, a slightly transformed patch that lies far from the input feature patch in descriptor space is defined as a weak match feature. A weak match finder network is proposed to actively find these weak match features. In a following step, the found weak matches are used in the standard descriptor learning framework. In this way, the intra-variance of the appearance of matched feature patch pairs is explored in depth and, accordingly, the invariance of feature descriptors against viewpoint and viewing direction change is improved. The proposed feature based image matching method is evaluated on standard benchmarks and is used to solve for the parameters of image orientation. For the image orientation task, aerial oblique images are taken into account. Through analysis of the experiments conducted for small image blocks, it is shown that deep learning feature based image matching leads to more registered images, more reconstructed 3D points and a more stable block connection. Note de contenu : 1- Introduction 2- Basics 3- Related work 4- Deep learning feature representation 5- Experiments and results 6- Discussion 7- Conclusion and outlook Numéro de notice : 17673 Affiliation des auteurs : non IGN Thématique : IMAGERIE Nature : Thèse étrangère Note de thèse : PhD dissertation : Fachrichtung Geodäsie und Geoinformatik : Hanovre : 2021 En ligne : https://dgk.badw.de/fileadmin/user_upload/Files/DGK/docs/c-867.pdf Format de la ressource électronique : URL Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=97999

Robust and fast global image orientation / Xin Wang (2021)  

Permalink

Post‐filtering with surface orientation constraints for stereo dense image matching / Xu Huang in Photogrammetric record, vol 35 n° 171 (September 2020)  

Permalink

Automatic sensor orientation using horizontal and vertical line feature constraints / Yanbiao Sun in ISPRS Journal of photogrammetry and remote sensing, vol 150 (April 2019)  

Permalink

Close-range photogrammetry and 3D imaging / Thomas Luhmann (2019)  

Permalink

A fully automatic approach to register mobile mapping and airborne imagery to support the correction of plateform trajectories in GNSS-denied urban areas / Phillipp Jende in ISPRS Journal of photogrammetry and remote sensing, vol 141 (July 2018)  

Permalink

Second iteration of photogrammetric processing to refine image orientation with improved tie-points / Truong Giang Nguyen in Sensors, vol 18 n° 7 (July 2018)    

Permalink

Automatic measurement of control points for aerial image orientation / Adilson Berveglieri in Photogrammetric record, vol 32 n° 158 (June - july 2017)  

Permalink

Triangulating social multimedia content for event localization using Flickr and Twitter / George Panteras in Transactions in GIS, vol 19 n° 5 (October 2015)  

Permalink

Simultaneous registration of gnomonic projections and central perspectives / Luigi Barazzetti in Photogrammetric record, vol 29 n° 147 (September - November 2014)  

Permalink

Automatic orientation and 3D modelling from markerless rock art imagery / J. Lerma in ISPRS Journal of photogrammetry and remote sensing, vol 76 (February 2013)  

Permalink