Centre dedocumentation
scientifique

3D hand mesh reconstruction from a monocular RGB image / Hao Peng in The Visual Computer, vol 36 n° 10 - 12 (October 2020)  

Public [article]  inThe Visual Computer > vol 36 n° 10 - 12 (October 2020) . - pp pages2227 - 2239 Titre : 3D hand mesh reconstruction from a monocular RGB image Type de document : Article/Communication Auteurs : Hao Peng, Auteur ; Chuhua Xian, Auteur ; Yunbo Zhang, Auteur Année de publication : 2020 Article en page(s) : pp pages2227 - 2239 Note générale : bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Traitement d'image optique [Termes descripteurs IGN] classification par réseau neuronal convolutif [Termes descripteurs IGN] estimation de pose [Termes descripteurs IGN] image de synthèse [Termes descripteurs IGN] image RVB [Termes descripteurs IGN] maillage [Termes descripteurs IGN] modélisation 3D [Termes descripteurs IGN] réalité augmentée [Termes descripteurs IGN] réalité virtuelle [Termes descripteurs IGN] reconstruction 3D [Termes descripteurs IGN] reconstruction d'objet [Termes descripteurs IGN] vision monoculaire Résumé : (auteur) Most of the existing methods for 3D hand analysis based on RGB images mainly focus on estimating hand keypoints or poses, which cannot capture geometric details of the 3D hand shape. In this work, we propose a novel method to reconstruct a 3D hand mesh from a single monocular RGB image. Different from current parameter-based or pose-based methods, our proposed method directly estimates the 3D hand mesh based on graph convolution neural network (GCN). Our network consists of two modules: the hand localization and mask generation module, and the 3D hand mesh reconstruction module. The first module, which is a VGG16-based network, is applied to localize the hand region in the input image and generate the binary mask of the hand. The second module takes the high-order features from the first and uses a GCN-based network to estimate the coordinates of each vertex of the hand mesh and reconstruct the 3D hand shape. To achieve better accuracy, a novel loss based on the differential properties of the discrete mesh is proposed. We also use professional software to create a large synthetic dataset that contains both ground truth 3D hand meshes and poses for training. To handle the real-world data, we use the CycleGAN network to transform the data domain of real-world images to that of our synthesis dataset. We demonstrate that our method can produce accurate 3D hand mesh and achieve an efficient performance for real-time applications. Numéro de notice : A2020-596 Affiliation des auteurs : non IGN Thématique : IMAGERIE Nature : Article DOI : 10.1007/s00371-020-01908-3 date de publication en ligne : 14/07/2020 En ligne : https://doi.org/10.1007/s00371-020-01908-3 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=95936 [article]