Centre de documentation
scientifique

Effect of label noise in semantic segmentation of high resolution aerial images and height data / Arabinda Maiti in ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol V-2-2022 (2022 edition)  

Public [article]  inISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences > vol V-2-2022 (2022 edition) . - pp 275 - 282 Titre : Effect of label noise in semantic segmentation of high resolution aerial images and height data Type de document : Article/Communication Auteurs : Arabinda Maiti, Auteur ; Sander J. Oude Elberink, Auteur ; M. George Vosselman, Auteur Année de publication : 2022 Article en page(s) : pp 275 - 282 Note générale : bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Traitement d'image optique [Termes IGN] apprentissage profond [Termes IGN] bruit (théorie du signal) [Termes IGN] données altimétriques [Termes IGN] données d'entrainement (apprentissage automatique) [Termes IGN] image à très haute résolution [Termes IGN] image aérienne [Termes IGN] orthoimage [Termes IGN] segmentation sémantique Résumé : (auteur) The performance of deep learning models in semantic segmentation is dependent on the availability of a large amount of labeled data. However, the influence of label noise, in the form of incorrect annotations, on the performance is significant and mostly ignored. This is a big concern in remote sensing applications, wherein acquired datasets are spatially limited, labeling is done by domain experts with possible sources of high inter-and intra-observer variability leading to erroneous predictions. In this paper, we first simulate the label noise while conducting experiments on two different datasets with very high-resolution aerial images, height data, and inaccurate labels, responsible for the training of deep learning models. We then focus on the effect of these noises on the model performance. Different classes respond differently to the label noise. The typical size of an object belonging to a class is a crucial factor regarding the class-specific performance of the model trained with erroneous labels. Errors caused by relative shifts of labels are the most influential label errors. The model is generally more tolerant of the random label noise than other label errors. It has been observed that the accuracy gets reduced by at least 3% while 5% of label pixels are erroneous. In this regard, our study provides a new perspective of evaluating and quantifying the propagation of label noise in the model performance that is indeed important for adopting reliable semantic segmentation practices. Numéro de notice : A2022-434 Affiliation des auteurs : non IGN Thématique : IMAGERIE/INFORMATIQUE Nature : Article DOI : 10.5194/isprs-annals-V-2-2022-275-2022 Date de publication en ligne : 17/05/2022 En ligne : https://doi.org/10.5194/isprs-annals-V-2-2022-275-2022 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=100741 [article]

Weakly supervised semantic segmentation of airborne laser scanning point clouds / Yaping Lin in ISPRS Journal of photogrammetry and remote sensing, vol 187 (May 2022)  

Public [article]  inISPRS Journal of photogrammetry and remote sensing > vol 187 (May 2022) . - pp 79 - 100 Titre : Weakly supervised semantic segmentation of airborne laser scanning point clouds Type de document : Article/Communication Auteurs : Yaping Lin, Auteur ; M. George Vosselman, Auteur ; Michael Ying Yang, Auteur Année de publication : 2022 Article en page(s) : pp 79 - 100 Note générale : Bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Lasergrammétrie [Termes IGN] apprentissage profond [Termes IGN] chevauchement [Termes IGN] classification dirigée [Termes IGN] données étiquetées d'entrainement [Termes IGN] données laser [Termes IGN] données localisées 3D [Termes IGN] hétérogénéité sémantique [Termes IGN] segmentation sémantique [Termes IGN] semis de points Résumé : (Auteur) While modern deep learning algorithms for semantic segmentation of airborne laser scanning (ALS) point clouds have achieved considerable success, the training process often requires a large number of labelled 3D points. Pointwise annotation of 3D point clouds, especially for large scale ALS datasets, is extremely time-consuming work. Weak supervision that only needs a few annotation efforts but can make networks achieve comparable performance is an alternative solution. Assigning a weak label to a subcloud, a group of points, is an efficient annotation strategy. With the supervision of subcloud labels, we first train a classification network that produces pseudo labels for the training data. Then the pseudo labels are taken as the input of a segmentation network which gives the final predictions on the testing data. As the quality of pseudo labels determines the performance of the segmentation network on testing data, we propose an overlap region loss and an elevation attention unit for the classification network to obtain more accurate pseudo labels. The overlap region loss that considers the nearby subcloud semantic information is introduced to enhance the awareness of the semantic heterogeneity within a subcloud. The elevation attention helps the classification network to encode more representative features for ALS point clouds. For the segmentation network, in order to effectively learn representative features from inaccurate pseudo labels, we adopt a supervised contrastive loss that uncovers the underlying correlations of class-specific features. Extensive experiments on three ALS datasets demonstrate the superior performance of our model to the baseline method (Wei et al., 2020). With the same amount of labelling efforts, for the ISPRS benchmark dataset, the Rotterdam dataset and the DFC2019 dataset, our method rises the overall accuracy by 0.062, 0.112 and 0.031, and the average F1 score by 0.09, 0.178 and 0.043 respectively. Our code is publicly available at ‘https://github.com/yaping222/Weak_ALS.git’. Numéro de notice : A2022-227 Affiliation des auteurs : non IGN Thématique : IMAGERIE Nature : Article nature-HAL : ArtAvecCL-RevueIntern DOI : 10.1016/j.isprsjprs.2022.03.001 Date de publication en ligne : 11/03/2022 En ligne : https://doi.org/10.1016/j.isprsjprs.2022.03.001 Format de la ressource électronique : URL Article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=100197 [article]

Exemplaires(3)

Code-barres	Cote	Support	Localisation	Section	Disponibilité
081-2022051	SL	Revue	Centre de documentation	Revues en salle	Disponible
081-2022053	DEP-RECP	Revue	LASTIG	Dépôt en unité	Exclu du prêt
081-2022052	DEP-RECF	Revue	Nancy	Dépôt en unité	Exclu du prêt

Photogrammetric point clouds: quality assessment, filtering, and change detection / Zhenchao Zhang (2022)  

Public Titre : Photogrammetric point clouds: quality assessment, filtering, and change detection Type de document : Thèse/HDR Auteurs : Zhenchao Zhang, Auteur ; M. George Vosselman, Auteur ; Markus Gerke, Auteur ; Michael Ying Yang, Auteur Editeur : Enschede [Pays-Bas] : International Institute for Geo-Information Science and Earth Observation ITC Année de publication : 2022 Note générale : bibliographie NB : EMBARGO SUR LE TEXTE JUSQU'AU 1ER JUILLET 2022 Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Applications photogrammétriques [Termes IGN] appariement dense [Termes IGN] détection de changement [Termes IGN] données lidar [Termes IGN] données localisées 3D [Termes IGN] qualité des données [Termes IGN] réseau neuronal convolutif [Termes IGN] segmentation sémantique [Termes IGN] semis de points Résumé : (auteur) 3D change detection draws more and more attention in recent years due to the increasing availability of 3D data. It can be used in the fields of land use / land cover (LULC) change detection, 3D geographic information updating, terrain deformation analysis, urban construction monitoring et al. Our motivation to study 3D change detection is mainly related to the practical need to update the outdated point clouds captured by Airborne Laser Scanning (ALS) with new point clouds obtained by dense image matching (DIM). The thesis has three main parts. The first part, chapter 1, explains the motivation, providing a review of current ALS and airborne photogrammetry techniques. It also presents the research objectives and questions. The second part including chapter 2 and chapter 3 evaluates the quality of photogrammetric products and investigates their potential for change detection. The third part including chapter 4 and chapter 5 proposes two methods for change detection that meet different requirements. To investigate the potential of using point clouds derived by dense matching for change detection, we propose a framework for evaluating the quality of 3D point clouds and DSMs generated by dense image matching. Our evaluation framework based on a large number of square patches reveals the distribution of dense matching errors in the whole photogrammetric block. Robust quality measures are proposed to indicate the DIM accuracy and precision quantitatively. The overall mean offset to the reference is 0.1 Ground Sample Distance (GSD); the maximum mean deviation reaches 1.0 GSD. We also find that the distribution of dense matching errors is homogenous in the whole block and close to a normal distribution based on many patch-based samples. However, in some locations, especially along narrow alleys, the mean deviations may get worse. In addition, the profiles of ALS points and DIM points reveal that the DIM profile fluctuates around the ALS profile. We find that the accuracy of DIM point cloud improves and that the noise level decreases on smooth ground areas when oblique images are used in dense matching together with nadir images. Then we evaluate whether the standard LiDAR filters are effective to filter dense matching points in order to derive accurate DTMs. Filtering results on a city block show that LiDAR filters perform well on the grassland, along bushes and around individual trees if the point cloud is sufficiently precise. When a ranking filter is used on the point clouds before filtering, the filtering will identify fewer but more reliable ground points. However, some small objects on the terrain will be filtered out. Since we aim at obtaining accurate DTMs, the ranking filter shows its value in identifying reliable ground points. Based on the previous findings in DIM quality, we propose a method to detect building changes between ALS and photogrammetric data. Firstly, the ALS points and DIM points are split out and concatenated with the orthoimages. The multimodal data are normalized to feed into a pseudo-Siamese Neural network for change detection. Then, the changed objects are delineated through per-pixel classification and artefact removal. The change detection module based on a pseudo-Siamese CNN can quickly localize the changes and generate coarse change maps. The next module can be used in precise mapping of change boundaries. Experimental results show that the proposed pseudo-Siamese Neural network can cope with the DIM errors and output plausible change detection results. Although the point cloud quality from dense matching is not as fine as laser scanning points, the spectral and textural information provided by the orthoimages serve as a supplement. Considering that the tasks of semantic segmentation and change detection are correlated, we propose SiamPointNet++ model to combine the two tasks in one framework. The method outputs a pointwise joint label for each ALS point. If an ALS point is unchanged, it is assigned a semantic label; If an ALS point is changed, it is assigned a change label. The sematic and change information are included in the joint labels with minimum information redundancy. The combined Siamese network learns both intra-epoch and inter-epoch features. Intra-epoch features are extracted at multiple scales to embed the local and global information. Inter-epoch features are extracted by Conjugated Ball Sampling (CBS) and concatenated to make change inference. Experiments on the Rotterdam data set indicate that the network is effective in learning multi-task features. It is invariant to the permutation and noise of inputs and robust to the data difference between ALS and DIM data. Compared with a sophisticated object-based method and supervised change detection, this method requires much less hyper-parameters and human intervention but achieves superior performance. As a conclusion, the thesis evaluates the quality of dense matching points and investigates its potential of updating outdated ALS points. The two change detection methods developed for different applications show their potential in the automation of topographic change detection and point cloud updating. Future work may focus on improving the generalizability and interpretability of the proposed models. Numéro de notice : 20403 Affiliation des auteurs : non IGN Thématique : IMAGERIE/INFORMATIQUE Nature : Thèse étrangère Note de thèse : PhD thesis : Geo-Information Science and Earth Observation : Enschede, university of Twente : 2022 DOI : 10.3990/1.9789036552653 Date de publication en ligne : 14/01/2022 En ligne : https://research.utwente.nl/en/publications/photogrammetric-point-clouds-quality [...] Format de la ressource électronique : URL Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=100963

Enhanced trajectory estimation of mobile laser scanners using aerial images / Zille Hussnain in ISPRS Journal of photogrammetry and remote sensing, vol 173 (March 2021)  

Public [article]  inISPRS Journal of photogrammetry and remote sensing > vol 173 (March 2021) . - pp 66 - 78 Titre : Enhanced trajectory estimation of mobile laser scanners using aerial images Type de document : Article/Communication Auteurs : Zille Hussnain, Auteur ; Sander J. Oude Elberink, Auteur ; M. George Vosselman, Auteur Année de publication : 2021 Article en page(s) : pp 66 - 78 Note générale : bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Lasergrammétrie [Termes IGN] appariement de points [Termes IGN] atténuation du signal [Termes IGN] balayage laser [Termes IGN] canyon urbain [Termes IGN] centrale inertielle [Termes IGN] données lidar [Termes IGN] erreur [Termes IGN] image captée par drone [Termes IGN] mesurage par GNSS [Termes IGN] semis de points [Termes IGN] trajectoire (véhicule non spatial) [Termes IGN] trajet multiple Résumé : (auteur) Multipath effects and signal obstruction by buildings in urban canyons can lead to inaccurate GNSS measurements and therefore errors in the estimated trajectory of Mobile Laser Scanning (MLS) systems; consequently, derived point clouds are distorted and lose spatial consistency. We obtain decimetre-level trajectory accuracy making use of corresponding points between the MLS data and aerial images with accurate exterior orientations instead of using ground control points. The MLS trajectory is estimated based on observation equations resulting from these corresponding points, the original IMU observations, and soft constraints on the pitch and yaw rotations of the vehicle. We analyse the quality of the trajectory enhancement under several conditions where the experiments were designed to test the influence of the number and quality of corresponding points and to test different settings for a B-spline representation of the vehicle trajectory. The method was tested on two independently acquired MLS datasets in Rotterdam by enhancing the trajectories and evaluating them using checkpoints. The RMSE values of the original GNSS/IMU based Kalman filter results at the checkpoints were 0.26 m, 0.30 m, and 0.47 m for the X-, Y- and Z-coordinates in the first dataset and 1.10 m, 1.51 m, and 1.81 m in the second dataset. The latter dataset was recorded with a lower quality IMU in an area with taller buildings. After trajectory adjustment these RMSE values were reduced to 0.09 m, 0.11 m, and 0.16 m for the first dataset and 0.12 m, 0.14 m, and 0.18 m for the second dataset. The results confirmed that, if sufficient tie points between the point cloud and aerial imagery are available, the method supports geo-referencing of MLS point clouds in urban canyons with a near-decimetre accuracy. Numéro de notice : A2021-102 Affiliation des auteurs : non IGN Thématique : IMAGERIE Nature : Article nature-HAL : ArtAvecCL-RevueIntern DOI : 10.1016/j.isprsjprs.2021.01.005 Date de publication en ligne : 17/01/2021 En ligne : https://doi.org/10.1016/j.isprsjprs.2021.01.005 Format de la ressource électronique : url article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=96877 [article]

Exemplaires(3)

Code-barres	Cote	Support	Localisation	Section	Disponibilité
081-2021031	SL	Revue	Centre de documentation	Revues en salle	Disponible
081-2021033	DEP-RECP	Revue	LASTIG	Dépôt en unité	Exclu du prêt
081-2021032	DEP-RECF	Revue	Nancy	Dépôt en unité	Exclu du prêt

Dynamic scene understanding using deep neural networks / Ye Lyu (2021)  

Public Titre : Dynamic scene understanding using deep neural networks Type de document : Thèse/HDR Auteurs : Ye Lyu, Auteur ; M. George Vosselman, Directeur de thèse ; Michael Ying Yang, Directeur de thèse Editeur : Enschede [Pays-Bas] : International Institute for Geo-Information Science and Earth Observation ITC Année de publication : 2021 Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Traitement d'image [Termes IGN] attention (apprentissage automatique) [Termes IGN] chaîne de traitement [Termes IGN] champ aléatoire conditionnel [Termes IGN] compréhension de l'image [Termes IGN] détection d'objet [Termes IGN] image captée par drone [Termes IGN] image vidéo [Termes IGN] poursuite de cible [Termes IGN] régression [Termes IGN] segmentation sémantique Résumé : (auteur) Scene understanding is an important and fundamental research field in computer vision, which is quite useful for many applications in photogrammetry and remote sensing. It focuses on locating and classifying objects in images, understanding the relationships between them. The higher goal is to interpret what event happens in the scene, when it happens and why it happens, and what should we do based on the information. Dynamic scene understanding is to use information from different time to interpret scenes and answer the above related questions. For modern scene understanding technology, deep learning has shown great potential for such task. "Deep" in deep learning refers to the use of multiple layers in the neural networks. Deep neural networks are powerful as they are highly non-linear function that possess the ability to map from one domain to another quite different domain after proper training. It is the best solution for many fundamental research tasks regarding scene understanding. This ph.D. research also takes advantage of deep learning for dynamic scene understanding. Temporal information plays an important role for dynamic scene understanding. Compared with static scene understanding from images, information distilled from the time dimension provides values in many different ways. Images across consecutive frames have very high correlation, i.e., objects observed in one frame have very high chance to be observed and identified in nearby frames as well. Such redundancy in observation could potentially reduce the uncertainty for object recognition with deep learning based methods, resulting in more consistent inference. High correlation across frames could also improve the chance for recognizing objects correctly. If the camera or the object moves, the object could be observed in multiple different views with different poses and appearance. The information captured for object recognition would be more diverse and complementary, which could be aggregated to jointly inference the categories and the properties of objects. This ph.D. research involves several tasks related to the dynamic scene understanding in computer vision, including semantic segmentation for aerial platform images (chapter 2, 3), video object segmentation and video object detection for common objects in natural scenes (chapter 4, 5), and multi-object tracking and segmentation for cars and pedestrians in driving scenes (chapter 6). Chapter2 investigates how to establish the semantic segmentation benchmark for the UAV images, which includes data collection, data labeling, dataset construction, and performance evaluation with baseline deep neural networks and the proposed multi-scale dilation net. Conditional random field with feature space optimization is used to achieve consistent semantic segmentation prediction in videos. Chapter3 investigates how to better extract the scene context information for etter object recognition performance by proposing the novel bidirectional multiscale attention networks. It achieves better performance by inferring features and attention weights for feature fusing from both higher level and lower level branches. Chapter4 investigates how to simultaneously segment multiple objects across multiple frames by combining memory modules with instance segmentation networks. Our method learns to propagate the target object labels without auxiliary data, such as optical flow, which simplifies the model. Chapter5 investigates how to improve the performance of well-trained object detectors with a light weighted and efficient plug&play tracker for object detection in video. This chapter also investigates how the proposed model performs when lacking video training data. Chapter6 investigates how to improve the performance of detection, segmentation, and tracking by jointly considering top-down and bottom-up inference. The whole pipeline follows the multi-task design, i.e., a single feature extraction backbone with multiple heads for different sub-tasks. Overall, this manuscript has delved into several different computer vision tasks, which share fundamental research problems, including detection, segmentation, and tracking. Based on the research experiments and knowledge from literature review, several reflections regarding dynamic scene understanding have been discussed: The range of object context influence the quality for object recognition; The quality of video data affect the method choice for specific computer vision task; Detection and tracking are complementary for each other. For future work, unified dynamic scene understanding task could be a trend, and transformer plus self-supervised learning is one promising research direction. Real-time processing for dynamic scene understanding requires further researches in order to put the methods into usage for real-world applications. Numéro de notice : 12984 Affiliation des auteurs : non IGN Thématique : IMAGERIE/INFORMATIQUE Nature : Thèse étrangère Note de thèse : PhD thesis : Geo-Information Science and Earth Observation : Enschede, university of Twente : 2021 DOI : 10.3990/1.9789036552233 Date de publication en ligne : 08/09/2021 En ligne : https://library.itc.utwente.nl/papers_2021/phd/lyu.pdf Format de la ressource électronique : URL Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=100962

Active and incremental learning for semantic ALS point cloud segmentation / Yaping Lin in ISPRS Journal of photogrammetry and remote sensing, vol 169 (November 2020)  

Permalink

Automatic extraction of accurate 3D tie points for trajectory adjustment of mobile laser scanners using aerial imagery / Zille Hussnain in ISPRS Journal of photogrammetry and remote sensing, vol 154 (August 2019)  

Permalink

Co‐registration of panoramic mobile mapping images and oblique aerial images / Phillipp Jende in Photogrammetric record, vol 34 n° 166 (June 2019)  

Permalink

The reviewing process for ISPRS events / Clément Mallet in ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol IV-5 (November 2018)  

Permalink

A deep learning approach to DTM extraction from imagery using rule-based training labels / Caroline M. Gevaert in ISPRS Journal of photogrammetry and remote sensing, vol 142 (August 2018)  

Permalink

A fully automatic approach to register mobile mapping and airborne imagery to support the correction of plateform trajectories in GNSS-denied urban areas / Phillipp Jende in ISPRS Journal of photogrammetry and remote sensing, vol 141 (July 2018)  

Permalink

The variants of an LOD of a 3D building model and their influence on spatial analyses / Filip Biljecki in ISPRS Journal of photogrammetry and remote sensing, vol 116 (June 2016)  

Permalink

Flexible building primitives for 3D building modeling / B. Xiong in ISPRS Journal of photogrammetry and remote sensing, vol 101 (March 2015)  

Permalink

CityGML implementation specifications for a countrywide 3D dataset: The case of the Netherlands / Jantien E. Stoter in Photogrammetric Engineering & Remote Sensing, PERS, vol 80 n° 11 (November 2014)  

Permalink

A graph edit dictionary for correcting errors in roof topology graphs reconstructed from point clouds / B. Xiong in ISPRS Journal of photogrammetry and remote sensing, vol 93 (July 2014)  

Permalink