Clustering Algorithms based Noise Identification from Air Pollution Monitoring Data

The development of data science has brought about many discussions of noise detection, and so far, there is no universal best method. In this paper, we propose a clustering-algorithm-based solution to identify and remove noise from air pollution data collected with mobile portable sensors. The test dataset is the air pollution data collected by the portable sensors throughout three seasons at the campus in Macao. We have applied and compared six clustering algorithms to identify the most appropriate clustering algorithm to achieve this goal: Simple K-means, Hierarchical Clustering, Cascading K-means, X-means, Expectation Maximization, and Self-Organizing Map. The performance is evaluated by their accuracy and the best number of clusters calculated by the Silhouette Coefficient. Additionally, a classification algorithm J48 tree can extract the key attributes and identify the noise cluster for future unlabeled data that may contain noise. The experiment results indicate that the Expectation Maximization and Cascading Simple K-Means perform the best. Moreover, temperature and carbon dioxide are vital attributes in identifying the noise cluster.

• Publication year

  2022

• Venue

  2022 IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE)

• Publication date

  2022-12-18

• Fields of study

  Not labeled

• Identifiers
  DOI 10.1109/CSDE56538.2022.10089276
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• A hybrid model for class noise detection using k-means and classification filtering algorithms

  2020cited by this paper
• Application of k-means and hierarchical clustering techniques for analysis of air pollution: A review (1980–2019)

  2020cited by this paper
• Comparison of clustering methods for identification of outdoor measurements in pollution monitoring

  2019cited by this paper
• Data preprocessing in predictive data mining

  2019cited by this paper
• Brief review of self-organizing maps

  2017cited by this paper
• Noise detection on ECG based on agglomerative clustering of morphological features

  2017cited by this paper
• A portable Wireless Sensor Network system for real-time environmental monitoring

  2016cited by this paper
• Cluster Segregation for Indoor and Outdoor Environmental Monitoring System

  2016cited by this paper
• Cluster quality based performance evaluation of hierarchical clustering method

  2015cited by this paper
• A Comparative Analysis of Various Clustering Techniques used for Very Large Datasets

  2013cited by this paper
• A Review of K-mean Algorithm

  2013cited by this paper
• Decision Tree Analysis on J48 Algorithm for Data Mining

  2013cited by this paper
• Cascading K-means Clustering and K-Nearest Neighbor Classifier for Categorization of Diabetic Patients

  2012cited by this paper
• Efficient BackProp

  2012cited by this paper
• A Comparative Study of Various Clustering Algorithms in Data Mining

  2012cited by this paper
• Accounting for noise when clustering biological data

  2012cited by this paper
• Mining With Noise Knowledge: Error-Aware Data Mining

  2008cited by this paper
• Incorporating quality aspects in sensor data streams

  2007cited by this paper
• Selecting variables for k-means cluster analysis by using a genetic algorithm that optimises the silhouettes

  2004influential reference
• Mining distance-based outliers in near linear time with randomization and a simple pruning rule

  2003cited by this paper
• X-means: Extending K-means with Efficient Estimation of the Number of Clusters

  2000cited by this paper
• The expectation-maximization algorithm

  1996cited by this paper
• A dendrite method for cluster analysis

  1974cited by this paper

• No citing papers are available for this paper.