Centre de documentation
scientifique

Unsupervised pansharpening based on self-attention mechanism / Ying Qu in IEEE Transactions on geoscience and remote sensing, vol 59 n° 4 (April 2021)  

Public [article]  inIEEE Transactions on geoscience and remote sensing > vol 59 n° 4 (April 2021) . - pp 3192 - 3208 Titre : Unsupervised pansharpening based on self-attention mechanism Type de document : Article/Communication Auteurs : Ying Qu, Auteur ; Razieh Kaviani Baghbaderani, Auteur ; Hairong Qi, Auteur ; et al., Auteur Année de publication : 2021 Article en page(s) : pp 3192 - 3208 Note générale : bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Traitement d'image optique [Termes IGN] apprentissage profond [Termes IGN] attention (apprentissage automatique) [Termes IGN] classification non dirigée [Termes IGN] image multibande [Termes IGN] pansharpening (fusion d'images) [Termes IGN] pouvoir de résolution géométrique [Termes IGN] précision infrapixellaire [Termes IGN] reconstruction d'image [Termes IGN] segmentation d'image Résumé : (auteur) Pansharpening is to fuse a multispectral image (MSI) of low-spatial-resolution (LR) but rich spectral characteristics with a panchromatic image (PAN) of high spatial resolution (HR) but poor spectral characteristics. Traditional methods usually inject the extracted high-frequency details from PAN into the upsampled MSI. Recent deep learning endeavors are mostly supervised assuming that the HR MSI is available, which is unrealistic especially for satellite images. Nonetheless, these methods could not fully exploit the rich spectral characteristics in the MSI. Due to the wide existence of mixed pixels in satellite images where each pixel tends to cover more than one constituent material, pansharpening at the subpixel level becomes essential. In this article, we propose an unsupervised pansharpening (UP) method in a deep-learning framework to address the abovementioned challenges based on the self-attention mechanism (SAM), referred to as UP-SAM. The contribution of this article is threefold. First, the SAM is proposed where the spatial varying detail extraction and injection functions are estimated according to the attention representations indicating spectral characteristics of the MSI with subpixel accuracy. Second, such attention representations are derived from mixed pixels with the proposed stacked attention network powered with a stick-breaking structure to meet the physical constraints of mixed pixel formulations. Third, the detail extraction and injection functions are spatial varying based on the attention representations, which largely improves the reconstruction accuracy. Extensive experimental results demonstrate that the proposed approach is able to reconstruct sharper MSI of different types, with more details and less spectral distortion compared with the state-of-the-art. Numéro de notice : A2021-285 Affiliation des auteurs : non IGN Thématique : IMAGERIE Nature : Article nature-HAL : ArtAvecCL-RevueIntern DOI : 10.1109/TGRS.2020.3009207 Date de publication en ligne : 23/07/2020 En ligne : https://doi.org/10.1109/TGRS.2020.3009207 Format de la ressource électronique : url article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=97394 [article]