Distributed Spatial Data Clustering as a New Approach for Big Data Analysis
Refereed Conference Meeting Proceeding
In this paper we propose a new approach for Big Data mining and analysis. This new approach works well on distributed datasets and deals with data cluster- ing task of the analysis. The approach consists of two main phases: the first phase executes a clustering algorithm on local data, assuming that the datasets was already distributed among the system process- ing nodes. The second phase deals with the local clusters aggregation to generate global clusters. This approach not only generates local clusters on each processing node in parallel, but also facilitates the formation of global clusters without prior knowledge of the number of the clusters, which many partition- ing clustering algorithm require. In this study, this approach was applied on spatial datasets. The pro- posed aggregation phase is very efficient and does not involve the exchange of large amounts of data between the processing nodes. The experimental results show that the approach has super-linear speed-up, scales up very well, and can take advantage of the recent programming models, such as MapReduce model, as its results are not affected by the types of communi- cations. Keywords: Distributed data mining, distributed com- puting, synchronous communication, asynchronous communication, super-speedup, spacial data mining
The 15th Australian Data Mining Conference (AusDM-2017), Melbourne, Australia, August 19-20 2017.
Digital Object Identifer (DOI):
National University of Ireland, Dublin (UCD)
Open access repository: