Centre de documentation
scientifique

Spatial data management in apache spark: the GeoSpark perspective and beyond / Jia Yu in Geoinformatica, vol 23 n° 1 (January 2019)  

Public [article]  inGeoinformatica > vol 23 n° 1 (January 2019) . - pp 37 - 78 Titre : Spatial data management in apache spark: the GeoSpark perspective and beyond Type de document : Article/Communication Auteurs : Jia Yu, Auteur ; Zongsi Zhang, Auteur ; Mohamed Sarwat, Auteur Année de publication : 2019 Article en page(s) : pp 37 - 78 Note générale : bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Bases de données localisées [Termes IGN] analyse comparative [Termes IGN] Apache (serveur) [Termes IGN] arbre k-d [Termes IGN] arbre quadratique [Termes IGN] arbre-R [Termes IGN] données massives [Termes IGN] Hadoop [Termes IGN] index spatial [Termes IGN] performance [Termes IGN] Spark [Termes IGN] traitement réparti Résumé : (auteur) The paper presents the details of designing and developing GeoSpark, which extends the core engine of Apache Spark and SparkSQL to support spatial data types, indexes, and geometrical operations at scale. The paper also gives a detailed analysis of the technical challenges and opportunities of extending Apache Spark to support state-of-the-art spatial data partitioning techniques: uniform grid, R-tree, Quad-Tree, and KDB-Tree. The paper also shows how building local spatial indexes, e.g., R-Tree or Quad-Tree, on each Spark data partition can speed up the local computation and hence decrease the overall runtime of the spatial analytics program. Furthermore, the paper introduces a comprehensive experiment analysis that surveys and experimentally evaluates the performance of running de-facto spatial operations like spatial range, spatial K-Nearest Neighbors (KNN), and spatial join queries in the Apache Spark ecosystem. Extensive experiments on real spatial datasets show that GeoSpark achieves up to two orders of magnitude faster run time performance than existing Hadoop-based systems and up to an order of magnitude faster performance than Spark-based systems. Numéro de notice : A2019-225 Affiliation des auteurs : non IGN Thématique : GEOMATIQUE Nature : Article nature-HAL : ArtAvecCL-RevueIntern DOI : 10.1007/s10707-018-0330-9 Date de publication en ligne : 22/10/2018 En ligne : http://dx.doi.org/10.1007/s10707-018-0330-9 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=92621 [article]

Generic and efficient framework for search trees on flash memory storage systems / Mohamed Sarwat in Geoinformatica, vol 17 n° 3 (July 2013)  

Public [article]  inGeoinformatica > vol 17 n° 3 (July 2013) . - pp 489 - 519 Titre : Generic and efficient framework for search trees on flash memory storage systems Type de document : Article/Communication Auteurs : Mohamed Sarwat, Auteur ; Mohamed F. Mokbel, Auteur ; Xun Zhou, Auteur ; Suman Nath, Auteur Année de publication : 2013 Article en page(s) : pp 489 - 519 Note générale : Bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Informatique [Termes IGN] arbre (mathématique) [Termes IGN] arbre-R [Termes IGN] indexation spatiale [Termes IGN] mémoire d'ordinateur [Termes IGN] mémoire flash Résumé : (Auteur) Tree index structures are crucial components in data management systems. Existing tree index structure are designed with the implicit assumption that the underlying external memory storage is the conventional magnetic hard disk drives. This assumption is going to be invalid soon, as flash memory storage is increasingly adopted as the main storage media in mobile devices, digital cameras, embedded sensors, and notebooks. Though it is direct and simple to port existing tree index structures on the flash memory storage, that direct approach does not consider the unique characteristics of flash memory, i.e., slow write operations, and erase-before-update property, which would result in a sub optimal performance. In this paper, we introduce FAST (i.e., Flash-Aware Search Trees) as a generic framework for flash-aware tree index structures. FAST distinguishes itself from all previous attempts of flash memory indexing in two aspects: (1) FAST is a generic framework that can be applied to a wide class of data partitioning tree structures including R-tree and its variants, and (2) FAST achieves both efficiency and durability of read and write flash operations through memory flushing and crash recovery techniques. Extensive experimental results, based on an actual implementation of FAST inside the GiST index structure in PostgreSQL, show that FAST achieves better performance than its competitors. Numéro de notice : A2013-381 Affiliation des auteurs : non IGN Thématique : INFORMATIQUE Nature : Article DOI : 10.1007/s10707-012-0164-9 Date de publication en ligne : 30/08/2012 En ligne : https://doi.org/10.1007/s10707-012-0164-9 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=32519 [article]

Exemplaires(1)

Code-barres	Cote	Support	Localisation	Section	Disponibilité
057-2013031	RAB	Revue	Centre de documentation	En réserve L003	Disponible