Centre de documentation
scientifique

Learning multi-view aggregation in the wild for large-scale 3D semantic segmentation / Damien Robert (2022)  

Public Titre : Learning multi-view aggregation in the wild for large-scale 3D semantic segmentation Type de document : Article/Communication Auteurs : Damien Robert , Auteur ; Bruno Vallet , Auteur ; Loïc Landrieu , Auteur Editeur : Ithaca [New York - Etats-Unis] : ArXiv - Université Cornell Année de publication : 2022 Projets : 3-projet - voir note / Conférence : CVPR 2022, IEEE Conference on Computer Vision and Pattern Recognition 19/06/2022 24/06/2022 New Orleans Louisiane - Etats-Unis OA Proceedings Importance : pp 5575 - 5584 Note générale : bibliographie This work was funded by ENGIE Lab CRIGEN and carried on in the LASTIG research unit of Universite Paris-Est. The authors wish to thank AI4GEO for sharing their computing resources. Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Applications photogrammétriques [Termes IGN] chaîne de traitement [Termes IGN] données localisées 2D [Termes IGN] données localisées 3D [Termes IGN] données massives [Termes IGN] pixel [Termes IGN] segmentation sémantique [Termes IGN] semis de points Résumé : (auteur) Recent works on 3D semantic segmentation propose to exploit the synergy between images and point clouds by processing each modality with a dedicated network and projecting learned 2D features onto 3D points. Merging large-scale point clouds and images raises several challenges, such as constructing a mapping between points and pixels, and aggregating features between multiple views. Current methods require mesh reconstruction or specialized sensors to recover occlusions, and use heuristics to select and aggregate available images. In contrast, we propose an end-to-end trainable multi-view aggregation model leveraging the viewing conditions of 3D points to merge features from images taken at arbitrary positions. Our method can combine standard 2D and 3D networks and outperforms both 3D models operating on colorized point clouds and hybrid 2D/3D networks without requiring colorization, meshing, or true depth maps. We set a new state-of-the-art for large-scale indoor/ outdoor semantic segmentation on S3DIS (74.7 mIoU 6-Fold) and on KITTI360 (58.3 mIoU). Our full pipeline is accessible at https: //github.com/drprojects/DeepViewAgg, and only requires raw 3D scans and a set of images and poses. Numéro de notice : C2022-006 Affiliation des auteurs : UGE-LASTIG (2020- ) Autre URL associée : vers CVF/vers ArXiv Thématique : IMAGERIE/INFORMATIQUE Nature : Communication nature-HAL : ComAvecCL&ActesPubliésIntl DOI : 10.48550/arXiv.2204.07548 Date de publication en ligne : 15/04/2022 En ligne : https://doi.org/10.1109/CVPR52688.2022.00549 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=100490