Catalogue en ligne IGN

Multi-modal temporal attention models for crop mapping from satellite time series / Vivien Sainte Fare Garnot (2021)

Public

Titre :	Multi-modal temporal attention models for crop mapping from satellite time series
Type de document :	Article/Communication
Auteurs :	Vivien Sainte Fare Garnot , Auteur ; Loïc Landrieu , Auteur ; Nesrine Chehata , Auteur
Editeur :	Saint-Mandé : Institut national de l'information géographique et forestière - IGN (2012-)
Année de publication :	2021
Langues :	Anglais (eng)
Descripteur :	[Vedettes matières IGN] Traitement d'image mixte [Termes IGN] base de données d'images [Termes IGN] carte agricole [Termes IGN] image optique [Termes IGN] image radar [Termes IGN] Pastis [Termes IGN] segmentation d'image
Résumé :	(auteur) Optical and radar satellite time series are synergetic: optical images contain rich spectral information, while C-band radar captures useful geometrical information and is immune to cloud cover. Motivated by the recent success of temporal attention-based methods across multiple crop mapping tasks, we propose to investigate how these models can be adapted to operate on several modalities. We implement and evaluate multiple fusion schemes, including a novel approach and simple adjustments to the training procedure, significantly improving performance and efficiency with little added complexity. We show that most fusion schemes have advantages and drawbacks, making them relevant for specific settings. We then evaluate the benefit of multimodality across several tasks: parcel classification, pixel-based segmentation, and panoptic parcel segmentation. We show that by leveraging both optical and radar time series, multimodal temporal attention-based models can outmatch single-modality models in terms of performance and resilience to cloud cover. To conduct these experiments, we augment the PASTIS dataset with spatially aligned radar image time series. The resulting dataset, PASTIS-R, constitutes the first large-scale, multimodal, and open-access satellite time series dataset with semantic and instance annotations.
Numéro de notice :	P2021-005
Affiliation des auteurs :	UGE-LASTIG (2020- )
Thématique :	IMAGERIE
Nature :	Preprint
nature-HAL :	Préprint
DOI :	sans
Date de publication en ligne :	14/12/2021
En ligne :	https://arxiv.org/abs/2112.07558v1
Format de la ressource électronique :	URL article
Permalink :	https://documentation.ensg.eu/index.php?lvl=notice_display&id=99392

Voir aussi

Multi-modal temporal attention models for crop mapping from satellite time series / Vivien Sainte Fare Garnot in ISPRS Journal of photogrammetry and remote sensing, vol 187 (May 2022)

Public

[article]
inISPRS Journal of photogrammetry and remote sensing > vol 187 (May 2022) . - pp 294 - 305

Titre :	Multi-modal temporal attention models for crop mapping from satellite time series
Type de document :	Article/Communication
Auteurs :	Vivien Sainte Fare Garnot , Auteur ; Loïc Landrieu , Auteur ; Nesrine Chehata , Auteur
Année de publication :	2022
Projets :	3-projet - voir note /
Article en page(s) :	pp 294 - 305
Note générale :	bibliographie This work was partly supported by ASP, the French Payment Agency.
Langues :	Anglais (eng)
Descripteur :	[Vedettes matières IGN] Traitement d'image mixte [Termes IGN] attention (apprentissage automatique) [Termes IGN] bande C [Termes IGN] carte agricole [Termes IGN] fusion d'images [Termes IGN] image Sentinel-MSI [Termes IGN] image Sentinel-SAR [Termes IGN] parcelle agricole [Termes IGN] Pastis [Termes IGN] segmentation d'image [Termes IGN] série temporelle [Termes IGN] surface cultivée
Résumé :	(auteur) Optical and radar satellite time series are synergetic: optical images contain rich spectral information, while C-band radar captures useful geometrical information and is immune to cloud cover. Motivated by the recent success of temporal attention-based methods across multiple crop mapping tasks, we propose to investigate how these models can be adapted to operate on several modalities. We implement and evaluate multiple fusion schemes, including a novel approach and simple adjustments to the training procedure, significantly improving performance and efficiency with little added complexity. We show that most fusion schemes have advantages and drawbacks, making them relevant for specific settings. We then evaluate the benefit of multimodality across several tasks: parcel classification, pixel-based segmentation, and panoptic parcel segmentation. We show that by leveraging both optical and radar time series, multimodal temporal attention-based models can outmatch single-modality models in terms of performance and resilience to cloud cover. To conduct these experiments, we augment the PASTIS dataset (Garnot and Landrieu, 2021a) with spatially aligned radar image time series. The resulting dataset, PASTIS-R, constitutes the first large-scale, multimodal, and open-access satellite time series dataset with semantic and instance annotations. (Dataset available at: https://zenodo.org/record/5735646)
Numéro de notice :	A2022-157
Affiliation des auteurs :	UGE-LASTIG+Ext (2020- )
Autre URL associée :	vers ArXiv
Thématique :	IMAGERIE/INFORMATIQUE
Nature :	Article
nature-HAL :	ArtAvecCL-RevueIntern
DOI :	10.1016/j.isprsjprs.2022.03.012
Date de publication en ligne :	24/03/2022
En ligne :	https://doi.org/10.1016/j.isprsjprs.2022.03.012
Format de la ressource électronique :	URL article
Permalink :	https://documentation.ensg.eu/index.php?lvl=notice_display&id=100365

Voir aussi

Exemplaires(3)

Code-barres	Cote	Support	Localisation	Section	Disponibilité
081-2022051	SL	Revue	Centre de documentation	Revues en salle	Disponible
081-2022053	DEP-RECP	Revue	LASTIG	Dépôt en unité	Exclu du prêt
081-2022052	DEP-RECF	Revue	Nancy	Dépôt en unité	Exclu du prêt

IGN

Centre de documentation
scientifique

Accueil

Sélection de la langue

Adresse

Se connecter

Actualité

L'actu ! Horaires du CDOS

Informations pratiques

Exemplaires(3)

IGN / ENSG

L'IGN a pour vocation

Accès directs

2014 -2024 IGN

IGN

Centre de documentationscientifique

Accueil

Sélection de la langue

Adresse

Se connecter

Actualité

L'actu ! Horaires du CDOS

Informations pratiques

Exemplaires(3)

IGN / ENSG

L'IGN a pour vocation

Accès directs

2014 -2024 IGN

Centre de documentation
scientifique