Open source software for visualization and quality control of continuous hydrologic and water quality sensor data

It is common for in situ hydrologic and water quality data to be collected at high frequencies and for extended durations. These data streams, which may also be collected across many monitoring sites require infrastructure for data storage and management. The Observations Data Model (ODM), which is part of the Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) Hydrologic Information System (HIS), was developed as a standard data model in which to organize, store, and describe point observations data. In this paper we describe ODM Tools Python, an open source software application that allows users to query and export, visualize, and perform quality control post processing on time series of environmental observations data stored in an ODM database using automated Python scripting that records the corrections and adjustments made to data series in the quality control process and ensures data editing steps are traceable and reproducible. We developed a workflow for scripting of time series data quality control.ODM Tools automatically scripts manual quality control data edits in Python.ODM Tools preserves provenance of quality control edits.ODM Tools is open source and cross platform compatible.ODM Tools demonstrates options for desktop application development and deployment using Python.

• Publication year

  2015

• Venue

  Environmental Modelling & Software

• Publication date

  2015-08-01

• Fields of study

  Computer Science, Environmental Science

• Identifiers
  DOI 10.1016/j.envsoft.2015.04.002
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• A centralized tool for managing, archiving, and serving point-in-time data in ecological research laboratories

  2014cited by this paper
• Guidelines and Standard Procedures for Continuous Water-Quality Monitors: Station Operation, Record Computation, and Data Reporting

  2014cited by this paper
• Data visualization and analysis within a Hydrologic Information System: Integrating with the R statistical computing environment

  2014cited by this paper
• Quantity is Nothing without Quality: Automated QA/QC for Streaming Environmental Sensor Data

  2013cited by this paper
• Automated quality control methods for sensor data: a novel observatory approach

  2012cited by this paper
• Moderate and Deep Sedation in Clinical Practice: Guidelines and standards

  2012cited by this paper
• The Initial Design of Data Sharing Infrastructure for the Critical Zone Observatory

  2011cited by this paper
• Components of an environmental observatory information system

  2011cited by this paper
• Python: An Ecosystem for Scientific Computing

  2011cited by this paper
• Python for Scientists and Engineers

  2011cited by this paper
• REAL-TIME QUALITY CONTROL (QC) PROCESSING, NOTIFICATION, AND VISUALIZATION SERVICES, SUPPORTING DATA MANAGEMENT OF THE INTELLIGENT RIVER©

  2010cited by this paper
• Quality Assurance Procedures for Mesoscale Meteorological Data

  2010cited by this paper
• Hydroserver: A Platform for Publishing Space-Time Hydrologic Datasets

  2010cited by this paper
• Development of a Community Hydrologic Information System

  2009cited by this paper
• Rattle: A Data Mining GUI for R

  2009cited by this paper
• An integrated system for publishing environmental observations data

  2009cited by this paper
• Some Simple Guidelines for Effective Data Management

  2009cited by this paper
• A relational model for environmental and water resources data

  2008cited by this paper
• Real‐time Bayesian anomaly detection in streaming environmental data

  2007cited by this paper
• Probabilistic Models for Anomaly Detection in Remote Sensor Data Streams

  2007cited by this paper
• A method for automatic validation of long time series of data in urban hydrology.

  2002cited by this paper
• A quality-control method for physical and chemical monitoring data. Application to dissolved oxygen levels in the river Loire (France)

  2001cited by this paper

• A critical review of statistical, signal processing and machine learning methods for continuous and high-frequency water quality data improvement

  2026cites this paper
• Hydrologic information systems: An introductory overview

  2025cites this paper
• Interpretable Quality Control of Sparsely Distributed Environmental Sensor Networks Using Graph Neural Networks

  2025cites this paper
• HydroServer: A software stack supporting collection, communication, storage, management, and sharing of data from in situ environmental sensors

  2025cites this paper
• ContDataQC: An R package and Shiny app for quality control of continuous water quality sensor data

  2025cites this paper
• Protocol for extracting flow hydrograph shape metrics for use in time-series flood hydrology analysis

  2025cites this paper
• Advancing freshwater science with sensor data collected by community scientists

  2024cites this paper
• Construction of an Art Education Teaching System Assisted by Artificial Intelligence

  2024cites this paper
• MitH: A framework for Mitigating Hygroscopicity in low-cost PM sensors

  2024cites this paper
• SentemQC - A novel and cost-efficient method for quality assurance and quality control of high-resolution frequency sensor data in fresh waters

  2024cites this paper
• MassWateR: Improving quality control, analysis, and sharing of water quality data

  2023cites this paper
• System for automated Quality Control (SaQC) to enable traceable and reproducible data streams in environmental science

  2023cites this paper
• Operationalizing Real-Time Monitoring Data in Simulation Models Using the Public Domain HEC-DSSVue Software Platform

  2023cites this paper
• A data-driven persistence test for robust (probabilistic) quality control of measured environmental time series: constant value episodes

  2023cites this paper
• Assimilating Low‐Cost High‐Frequency Sensor Data in Watershed Water Quality Modeling: A Bayesian Approach

  2023cites this paper
• K-span: Open and reproducible spatial analytics using scientific workflows

  2023cites this paper
• Head-mounted display-based augmented reality for water quality visualisation

  2023cites this paper
• React Library: A Case Study on the Effective Instruments of the Design of an Environmental Monitoring System

  2023cites this paper
• Automated Quality Control for In-Situ Water Temperature Sensors Automated Quality Control for In-Situ Water Temperature Sensors

  2022cites this paper
• Forward-Looking Roadmaps for Long-Term Continuous Water Quality Monitoring: Bottlenecks, Innovations, and Prospects in a Critical Review

  2022cites this paper
• Using Springs as Sentinels of Climate Change in Nature Parks North and South of the Alps: A Critical Evaluation of Methodological Aspects and Recommendations for Long-Term Monitoring

  2022cites this paper
• An innovative approach to correct data from in-situ turbidity sensors for surface water monitoring

  2022cites this paper
• AbspectroscoPY, a Python toolbox for absorbance-based sensor data in water quality monitoring

  2022cites this paper
• Toward automating post processing of aquatic sensor data

  2021cites this paper
• An Open-source Wireless Platform for Real-time Water Quality Monitoring with Precise Global Positioning

  2021cites this paper
• A New Automatic Monitoring Network of Surface Waters in Greece: Preliminary Data Quality Checks and Visualization

  2021cites this paper
• Automated high frequency monitoring of Lake Maggiore through in situ sensors: system design, field test and data quality control

  2021cites this paper
• Data acquisition and control system design for PET cyclotron and negative hydrogen ion source

  2020cites this paper
• Evolution of hydroinformatics at a state water management agency

  2020cites this paper
• Managing collaborative research data for integrated, interdisciplinary environmental research

  2020cites this paper
• An Observational Process Ontology-Based Modeling Approach for Water Quality Monitoring

  2020cites this paper
• State Tagging for Improved Earth and Environmental Data Quality Assurance

  2020cites this paper
• An educational rule-based expert system to determine water quality for environmental engineering and biotechnology students

  2020cites this paper
• Detecting technical anomalies in high-frequency water-quality data using Artificial Neural Networks.

  2020cites this paper
• Using maintenance records from a long-term sensor monitoring network to evaluate the relationship between maintenance schedule and data quality

  2019cites this paper
• Estimating Suspended Solids and Phosphorus Loading in Urban Stormwater Systems Using High-Frequency, Continuous Data

  2019cites this paper
• An R package for correcting continuous water quality monitoring data for drift

  2019cites this paper
• Challenges in Building an End-to-End System for Acquisition, Management, and Integration of Diverse Data From Sensor Networks in Watersheds: Lessons From a Mountainous Community Observatory in East River, Colorado

  2019cites this paper
• Assessing the quality of biogeochemical coastal data: a step-wise procedure

  2019cites this paper
• Anomaly Detection in Streaming Time Series Data

  2019influential citation
• A short review of contemporary developments in aquatic ecosystem modelling of lakes and reservoirs

  2019cites this paper
• Towards a taxonomy for quality control in Environmental Sciences

  2018cites this paper
• Control system design for new electronic cooler of HIRFL-CSRm

  2018cites this paper
• A data mining approach to evaluate suitability of dissolved oxygen sensor observations for lake metabolism analysis

  2018influential citation
• Predicting Sediment and Nutrient Concentrations in Rivers Using High Frequency Water Quality Surrogates

  2018cites this paper
• A framework for automated anomaly detection in high frequency water-quality data from in situ sensors.

  2018cites this paper
• Towards a water quality database for raw and validated data with emphasis on structured metadata

  2018cites this paper
• Control system design for new electronic cooler of HIRFL-CSRm

  2018cites this paper
• Assessing subjectivity in environmental sensor data post processing via a controlled experiment

  2018cites this paper
• Designing and Implementing a Network for Sensing Water Quality and Hydrology across Mountain to Urban Transitions

  2017influential citation
• Shallow water benthic imaging and substrate characterization using recreational-grade sidescan-sonar

  2017cites this paper
• STATISTICAL SOFTWARE FOR RAW WATER QUALITY ASSESMENT

  2017influential citation
• High-frequency measurements reveal spatial and temporal patterns of dissolved organic matter in an urban water conveyance

  2017cites this paper
• Resolving Spatial And Temperal Variability in Dissolved Organic Matter Characteristics Within Combined Agricultural and Stormwater Conveyances

  2017cites this paper
• Condition-based joint maintenance optimization for a large-scale system with homogeneous units

  2017cites this paper
• TopoZeko: A MATLAB function for 3-D and 4-D topographical visualization in geosciences

  2017cites this paper
• A cloud-enabled remote visualization tool for time-varying climate data analytics

  2016cites this paper
• Design and Application Research of OA system for Small and Medium Sized Enterprise Management based on SQL Platform

  2016cites this paper
• Control strategies used in the control software for the Heavy Ion Research Facility in Lanzhou

  2016cites this paper
• A data management and publication workflow for a large-scale, heterogeneous sensor network

  2015cites this paper