Centre dedocumentation
scientifique

Spatial data management in apache spark: the GeoSpark perspective and beyond / Jia Yu in Geoinformatica, vol 23 n° 1 (January 2019)  

Public [article]  inGeoinformatica > vol 23 n° 1 (January 2019) . - pp 37 - 78 Titre : Spatial data management in apache spark: the GeoSpark perspective and beyond Type de document : Article/Communication Auteurs : Jia Yu, Auteur ; Zongsi Zhang, Auteur ; Mohamed Sarwat, Auteur Année de publication : 2019 Article en page(s) : pp 37 - 78 Note générale : bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Bases de données localisées [Termes IGN] analyse comparative [Termes IGN] Apache (serveur) [Termes IGN] arbre k-d [Termes IGN] arbre quadratique [Termes IGN] arbre-R [Termes IGN] données massives [Termes IGN] Hadoop [Termes IGN] index spatial [Termes IGN] performance [Termes IGN] Spark [Termes IGN] traitement réparti Résumé : (auteur) The paper presents the details of designing and developing GeoSpark, which extends the core engine of Apache Spark and SparkSQL to support spatial data types, indexes, and geometrical operations at scale. The paper also gives a detailed analysis of the technical challenges and opportunities of extending Apache Spark to support state-of-the-art spatial data partitioning techniques: uniform grid, R-tree, Quad-Tree, and KDB-Tree. The paper also shows how building local spatial indexes, e.g., R-Tree or Quad-Tree, on each Spark data partition can speed up the local computation and hence decrease the overall runtime of the spatial analytics program. Furthermore, the paper introduces a comprehensive experiment analysis that surveys and experimentally evaluates the performance of running de-facto spatial operations like spatial range, spatial K-Nearest Neighbors (KNN), and spatial join queries in the Apache Spark ecosystem. Extensive experiments on real spatial datasets show that GeoSpark achieves up to two orders of magnitude faster run time performance than existing Hadoop-based systems and up to an order of magnitude faster performance than Spark-based systems. Numéro de notice : A2019-225 Affiliation des auteurs : non IGN Thématique : GEOMATIQUE Nature : Article nature-HAL : ArtAvecCL-RevueIntern DOI : 10.1007/s10707-018-0330-9 Date de publication en ligne : 22/10/2018 En ligne : http://dx.doi.org/10.1007/s10707-018-0330-9 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=92621 [article]