Centre dedocumentation
scientifique

Spatial data management in apache spark: the GeoSpark perspective and beyond / Jia Yu in Geoinformatica [en ligne], vol 23 n° 1 (January 2019)  

Public [article]  inGeoinformatica [en ligne] > vol 23 n° 1 (January 2019) . - pp 37 - 78 Titre : Spatial data management in apache spark: the GeoSpark perspective and beyond Type de document : Article/Communication Auteurs : Jia Yu, Auteur ; Zongsi Zhang, Auteur ; Mohamed Sarwat, Auteur Année de publication : 2019 Article en page(s) : pp 37 - 78 Note générale : bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Bases de données localisées [Termes descripteurs IGN] analyse comparative [Termes descripteurs IGN] Apache (serveur) [Termes descripteurs IGN] arbre k-d [Termes descripteurs IGN] arbre quadratique [Termes descripteurs IGN] Arbre-R [Termes descripteurs IGN] données massives [Termes descripteurs IGN] Hadoop [Termes descripteurs IGN] index spatial [Termes descripteurs IGN] performance [Termes descripteurs IGN] Spark [Termes descripteurs IGN] traitement réparti Résumé : (auteur) The paper presents the details of designing and developing GeoSpark, which extends the core engine of Apache Spark and SparkSQL to support spatial data types, indexes, and geometrical operations at scale. The paper also gives a detailed analysis of the technical challenges and opportunities of extending Apache Spark to support state-of-the-art spatial data partitioning techniques: uniform grid, R-tree, Quad-Tree, and KDB-Tree. The paper also shows how building local spatial indexes, e.g., R-Tree or Quad-Tree, on each Spark data partition can speed up the local computation and hence decrease the overall runtime of the spatial analytics program. Furthermore, the paper introduces a comprehensive experiment analysis that surveys and experimentally evaluates the performance of running de-facto spatial operations like spatial range, spatial K-Nearest Neighbors (KNN), and spatial join queries in the Apache Spark ecosystem. Extensive experiments on real spatial datasets show that GeoSpark achieves up to two orders of magnitude faster run time performance than existing Hadoop-based systems and up to an order of magnitude faster performance than Spark-based systems. Numéro de notice : A2019-225 Thématique : GEOMATIQUE Nature : Article nature-HAL : ArtAvecCL-RevueIntern DOI : 10.1007/s10707-018-0330-9 date de publication en ligne : 22/10/2018 En ligne : http://dx.doi.org/10.1007/s10707-018-0330-9 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=92621 [article]

Experiments to distribute and parallelize map generalization processes / Guillaume Touya in Cartographic journal (the), Vol 54 n° 4 (November 2017)  

Public [article]  inCartographic journal (the) > Vol 54 n° 4 (November 2017) . - pp 322 - 332 Titre : Experiments to distribute and parallelize map generalization processes Type de document : Article/Communication Auteurs : Guillaume Touya , Auteur ; Justin Berli, Auteur ; Imran Lokhat , Auteur ; Nicolas Regnauld , Auteur Année de publication : 2017 Projets : 1-Pas de projet / Article en page(s) : pp 322 - 332 Note générale : Bibliographie Langues : Anglais (eng) Descripteur : [Termes descripteurs IGN] partitionnement [Termes descripteurs IGN] traitement parallèle [Termes descripteurs IGN] traitement réparti [Vedettes matières IGN] Généralisation Résumé : (Auteur) Automatic map generalization requires the use of computationally intensive processes often unable to deal with large datasets. Distributing the generalization process is the only way to make them scalable and usable in practice. But map generalization is a highly contextual process, and the surroundings of a generalized map feature needs to be known to generalize the feature, which is a problem as distribution might partition the dataset and parallelize the processing of each part. This paper proposes experiments to evaluate the past propositions to distribute map generalization, and to identify the main remaining issues. The past propositions to distribute map generalization are first discussed, and then the experiment hypotheses and apparatus are described. The experiments confirmed that regular partitioning was the quickest strategy, but less effective when taking context into account. The geographical partitioning, though less effective for now, is quite promising regarding the quality of the results as it better integrates the geographical context. Numéro de notice : A2017-827 Thématique : GEOMATIQUE Nature : Article nature-HAL : ArtAvecCL-RevueIntern DOI : 10.1080/00087041.2017.1413787 date de publication en ligne : 19/02/2018 En ligne : https://doi.org/10.1080/00087041.2017.1413787 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=89359 [article]

Réservation

Réserver ce document

Exemplaires (1)

Code-barres	Cote	Support	Localisation	Section	Disponibilité
030-2017041	SL	Revue	Centre de documentation	Revues en salle	Disponible

Graph mapping: Multi-scale community visualization of massive graph data / David Jonker in Information visualization, vol 16 n° 3 (July 2017)  

Public [article]  inInformation visualization > vol 16 n° 3 (July 2017) Titre : Graph mapping: Multi-scale community visualization of massive graph data Type de document : Article/Communication Auteurs : David Jonker, Auteur ; Scott Langevin, Auteur ; David Giesbrecht, Auteur ; et al., Auteur Année de publication : 2017 Note générale : bibliographie Langues : Anglais (eng) Descripteur : [Termes descripteurs IGN] graphe [Termes descripteurs IGN] regroupement de données [Termes descripteurs IGN] traitement réparti [Termes descripteurs IGN] visualisation multiéchelle [Vedettes matières IGN] Géovisualisation Résumé : (auteur) Graph visualizations increase the perception of entity relationships in a network. However, as graph size and density increases, readability rapidly diminishes. In this article, we present an end-to-end, tile-based visual analytic approach called graph mapping that utilizes cluster computing to turn large-scale graph (node–link) data into interactive visualizations in modern web browsers. Our approach is designed for end-user analysis of community structure and relationships at macro- and micro scales. We also present the results of several experiments using alternate methods for qualitatively improving comprehensibility of hierarchical community detection visualizations by proposing constraints to state-of-the-art modularity maximization algorithms. Numéro de notice : A2017-758 Thématique : GEOMATIQUE Nature : Article DOI : 10.1177/1473871616661195 En ligne : https://doi.org/10.1177/1473871616661195 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=89117 [article]

Distributed processing of big mobility data as spatio-temporal data streams / Zdravko Galić in Geoinformatica [en ligne], vol 21 n° 2 (April - June 2017)  

Public [article]  inGeoinformatica [en ligne] > vol 21 n° 2 (April - June 2017) . - pp 263 - 291 Titre : Distributed processing of big mobility data as spatio-temporal data streams Type de document : Article/Communication Auteurs : Zdravko Galić, Auteur ; Emir Mešković, Auteur ; Dario Osmanović, Auteur Année de publication : 2017 Article en page(s) : pp 263 - 291 Note générale : Bibliographie Langues : Anglais (eng) Descripteur : [Vedettes matières IGN] Analyse spatiale [Termes descripteurs IGN] données massives [Termes descripteurs IGN] données spatiotemporelles [Termes descripteurs IGN] environnement de développement [Termes descripteurs IGN] flux de données [Termes descripteurs IGN] mobilité territoriale [Termes descripteurs IGN] mobilité urbaine [Termes descripteurs IGN] objet mobile [Termes descripteurs IGN] prototype [Termes descripteurs IGN] temps réel [Termes descripteurs IGN] traitement réparti Résumé : (Auteur) Recent rapid development of wireless communication, mobile computing, global navigation satellite systems (GNSS), and spatially enabled sensors are leading to an exponential growth of available mobility data produced continuously at high speed. Due to these advancements, a new class of monitoring applications has come to the focus, including real-time intelligent transportation systems, traffic monitoring and mobile objects tracking. These new information flow processing (IFP) application domains need to process huge volume of mobility data arriving in the form of continuous data streams from mobile objects. IFP applications are pushing traditional database technologies beyond their limits due to their massively increasing data volumes and demands for real-time processing. Mobility data, i.e. real-time, transient, time-varying sequences of spatio-temporal data items, generated by embedded positioning sensors demonstrates at least two Big Data core features: volume and velocity. Existing distributed data stream management systems (DSMS), real-time computing systems (RTCS) and their processing models are dominantly based on relational paradigm and continuous operator model. Thus, they have rudimentary spatio-temporal capabilities, provide expensive fault recovery requiring either hot replication or long recovery times, and do not handle faults and slow nodes. The framework proposed in this paper is a cornerstone towards efficient real-time managing and monitoring of mobile objects through distributed spatio-temporal streams processing on large clusters. A prototype implementation is rooted in a new stream processing model that overcomes the challenges of current distributed stream processing models and enable seamless integration with batch and interactive processing like MapReduce. Numéro de notice : A2017-069 Thématique : GEOMATIQUE Nature : Article En ligne : http://dx.doi.org/10.1007/s10707-016-0264-z Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=84300 [article]

Experiments to distribute map generalization processes / Justin Berli (2017)  

Public contenu dans ICC 2017, the 28th International Cartographic Conference, à Washington, USA, 2–7 July 2017, proceedings / International cartographic association = association cartographique internationale (2017)     Titre : Experiments to distribute map generalization processes Type de document : Article/Communication Auteurs : Justin Berli, Auteur ; Guillaume Touya , Auteur ; Imran Lokhat , Auteur ; Nicolas Regnauld , Auteur Congrès : Congrès: ICC 2017 28th International Cartographic Conference ICA (2 - 7 juillet 2017; Washington DC, Etats-Unis) , Commanditaire Editeur : International Cartographic Association ICA Année de publication : 2017 Importance : 10 p. Note générale : Bibliographie Ce papier a reçu "2018 Henry Johns Award" for the most outstanding paper published in 2017 Langues : Anglais (eng) Descripteur : [Termes descripteurs IGN] généralisation automatique de données [Termes descripteurs IGN] partitionnement [Termes descripteurs IGN] prise en compte du contexte [Termes descripteurs IGN] traitement réparti [Vedettes matières IGN] Généralisation Résumé : (Auteur) Automatic map generalization requires the use of computationally intensive processes often unable to deal with large datasets. Distributing the generalization process is the only way to make them scalable and usable in practice. But map generalization is a highly contextual process, and the surroundings of a generalized map feature needs to be known to generalize the feature, which is a problem as distribution might partition the dataset and parallelize the processing of each part. This paper proposes experiments to evaluate the past propositions to distribute map generalization, and to identify the main remaining issues. The past propositions to distribute map generalization are first discussed, and then the experiment hypotheses and apparatus are described. The experiments confirmed that regular partitioning was the quickest strategy, but also the less effective in taking context into account. The geographical partitioning, though less effective for now, is quite promising regarding the quality of the results as it better integrates the geographical context. Numéro de notice : C2017-010 Thématique : GEOMATIQUE Nature : Communication nature-HAL : ComAvecCL&ActesPubliésIntl DOI : 10.5194/ica-proc-1-8-2018 date de publication en ligne : 16/05/2018 En ligne : http://dx.doi.org/10.5194/ica-proc-1-8-2018 Format de la ressource électronique : URL article Permalink : https://documentation.ensg.eu/index.php?lvl=notice_display&id=86222

Documents numériques

en open access Experiments to distribute map generalization processes - pdf éditeur Adobe Acrobat PDF

en open access Experiments to distribute map generalization processes - pdf éditeur nouvelle version Adobe Acrobat PDF

Un protocole basé sur des mobiles sécurisés pour une collecte participative de données spatiales en mobilité réellement anonyme / Dai Hai Ton That in Revue internationale de géomatique, vol 26 n° 2 (avril - juin 2016)  

Permalink

Big spatial data analytics / Colin Henderson in GEO: Geoconnexion international, vol 13 n° 8 (september 2014)

Permalink

Big Data, concepts et mise en oeuvre de Hadoop / Laurent Jolia-Ferrier (2014)

Permalink

Análisis y visualización de nubes de puntos LiDAR en internet / Ivan Ulloa Enriquez (2013)  

Permalink

WebGIS performance tests / H. Dai in GIM international, vol 26 n° 1 (January 2012)

Permalink

Automated processing for map generation using web services / M. Neun in Geoinformatica, vol 13 n° 4 (December 2009)

Permalink

Service chaining architectures for applications implementing distributed geographic information processing / Anders Friis-Christensen in International journal of geographical information science IJGIS, vol 23 n° 5 (may 2009)

Permalink

TeraGrid GIScience Gateway: Bridging cyberinfrastructure and GIScience / Shuo Wang in International journal of geographical information science IJGIS, vol 23 n° 5 (may 2009)

Permalink

Use of grid computing modeling virtual geospatial products / A. Chen in International journal of geographical information science IJGIS, vol 23 n° 5 (may 2009)

Permalink

DEXA 2004, Fifteenth International Workshop on Database and Expert Systems Applications / Fernando Galindo (2004)

Permalink