Analysis of self-describing gridded geoscience data with netCDF Operators (NCO)

The netCDF Operator (NCO) software facilitates manipulation and analysis of gridded geoscience data stored in the self-describing netCDF format. NCO is optimized to efficiently analyze large multi-di-mensional data sets spanning many files. Researchers and data centers often use NCO to analyze and serve observed and modeled geoscience data including satellite observations and weather, air quality, and climate forecasts. NCO’s functionality includes shared memory threading, a message-passing in- terface, network transparency, and an interpreted language parser. NCO treats data files as a high level data type whose contents may be simultaneously manipulated by a single command. Institutions and data portals often use NCO for middleware to hyperslab and aggregate data set requests, while scientific researchers use NCO to perform three general functions: arithmetic operations, data permutation and compression, and metadata editing. We describe NCO’s design philosophy and primary features, illus-trate techniques to solve common geoscience and environmental data analysis problems, and suggest ways to design gridded data sets that can ease their subsequent analysis.

• Publication year

  2008

• Venue

  Environmental Modelling & Software

• Publication date

  Unknown publication date

• Fields of study

  Geology, Computer Science, Environmental Science

• Identifiers
  DOI 10.1016/j.envsoft.2008.03.004
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Author ' s personal copy

  2009cited by this paper
• Server-Side Parallel Data Reduction and Analysis

  2007cited by this paper
• NetCDF-4 : Software Implementing an Enhanced Data Model for the Geosciences

  2007cited by this paper
• Scaling Properties of Common Statistical Operators for Gridded Datasets

  2007cited by this paper
• Response of model simulated weather parameters to round-off-errors on different systems

  2007cited by this paper
• Climate change 2007 : the physical science basis : contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change

  2007cited by this paper
• The Physical Science Basis : Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change

  2007cited by this paper
• Thematic Real-time Environmental Distributed Data Services (THREDDS): Incorporating Interactive Analysis Tools into NSDL

  2006cited by this paper
• Prediction of ozone levels in London using the MM5-CMAQ modelling system

  2006cited by this paper
• Long-term simulation of global dust distribution with the GOCART model: correlation with North Atlantic Oscillation

  2004cited by this paper
• OPeNDAP: Accessing data in a distributed, heterogeneous environment

  2003cited by this paper
• Parallel netCDF: A High-Performance Scientific I/O Interface

  2003cited by this paper
• A Web Service Model for Climate Data Access on the Grid

  2003cited by this paper
• J3.3 The Earth System Grid II: Turning Climate Datasets Into Community Resources

  2001cited by this paper
• Using MPI-2: Advanced Features of the Message Passing Interface

  1999cited by this paper
• The Cathedral & the Bazaar

  1999cited by this paper
• MPI - The Complete Reference: Volume 1, The MPI Core

  1998influential reference
• The cathedral and the bazaar

  1998cited by this paper
• The Accumulation of Rounding Errors and Port Validation for Global Atmospheric Models

  1997cited by this paper
• Learning the bash Shell

  1995cited by this paper
• NetCDF: an interface for scientific data access

  1990cited by this paper
• Using MPI-2 : Advanced Features of the Message Passing Interface ( Scientific and Engineering Computation ) by Ewing L . Lusk

  year unknowncited by this paper

• Zooming in: SCREAM at 100 m using regional refinement over the San Francisco Bay Area

  2026cites this paper
• Error bounded compression for weather and climate applications

  2025cites this paper
• Comparison of simulations from a state-of-the-art dynamic global vegetation model (LPJ-GUESS) driven by low- and high-resolution climate data

  2025cites this paper
• Exploring the dynamics of past and future climate extremes under CMIP6: implications for rainfed agriculture

  2025cites this paper
• The Impacts of Rotational Mixing on the Precipitation Simulated by a Convection Permitting Model

  2025influential citation
• A Comprehensive Chemistry Evaluation and Diagnostics Package for E3SM - ChemDyg Version 1.1.0

  2025cites this paper
• Shifting Patterns in the Weather Regimes That Drive Regional Drought: Demonstration for South Africa

  2025cites this paper
• Climate Change and Its Effect on Temperature and Precipitation Trends: Case Study in Surabaya Using RegCM5

  2025cites this paper
• Toward Dynamical Annual to Decadal Climate Prediction Using the IAP‐CAS Model

  2025cites this paper
• An updated non-intrusive, multi-scale, and flexible coupling interface in WRF 4.6.0

  2025cites this paper
• Chemistry–climate feedback of atmospheric methane in a methane-emission-flux-driven chemistry–climate model

  2025cites this paper
• An overview of cloud–radiation denial experiments for the Energy Exascale Earth System Model version 1

  2024cites this paper
• Exploring a Data‐Driven Approach to Identify Regions of Change Associated With Future Climate Scenarios

  2024cites this paper
• Using variable-resolution grids to model precipitation from atmospheric rivers around the Greenland ice sheet

  2024cites this paper
• Projections of precipitation and temperatures in Greenland and the impact of spatially uniform anomalies on the evolution of the ice sheet

  2024cites this paper
• Hades: A Context-Aware Active Storage Framework for Accelerating Large-Scale Data Analysis

  2024cites this paper
• Changes in United States Summer Temperatures Revealed by Explainable Neural Networks

  2024influential citation
• Simulation of Compound Flooding Using River‐Ocean Two‐Way Coupled E3SM Ensemble on Variable‐Resolution Meshes

  2024cites this paper
• ncompare: A Python package for comparing netCDF structures

  2024cites this paper
• Leveraging regional mesh refinement to simulate future climate projections for California using the Simplified Convection-Permitting E3SM Atmosphere Model Version 0

  2024cites this paper
• Wave climate conditions of the MUSAN underwater park in Ayia Napa with emphasis in extreme events

  2024cites this paper
• Revealing trends in extreme heatwave intensity: applying the UNSEEN approach to Nordic countries

  2024cites this paper
• Impacts of spatial heterogeneity of anthropogenic aerosol emissions in a regionally refined global aerosol–climate model

  2024cites this paper
• DSCompare: Unleashing the potential of ocean and atmospheric data with a comparative analysis software

  2023cites this paper
• The Importance of Dynamic Iron Deposition in Projecting Climate Change Impacts on Pacific Ocean Biogeochemistry

  2023cites this paper
• Restructuring and serving web-accessible streamflow data from the NOAA National Water Model historic simulations

  2023cites this paper
• Evaluating the Water Cycle Over CONUS at the Watershed Scale for the Energy Exascale Earth System Model Version 1 (E3SMv1) Across Resolutions

  2023cites this paper
• Navigating the Noise: Bringing Clarity to ML Parameterization Design With O $\boldsymbol{\mathcal{O}}$ (100) Ensembles

  2023cites this paper
• Global warming and arctic terns: Estimating climate change impacts on the world's longest migration

  2023cites this paper
• Modeling dust mineralogical composition: sensitivity to soil mineralogy atlases and their expected climate impacts

  2023cites this paper
• The fully coupled regionally refined model of E3SM version 2: overview of the atmosphere, land, and river results

  2023cites this paper
• Assessing Clouds Using Satellite Observations Through Three Generations of Global Atmosphere Models

  2023cites this paper
• Using Machine Learning With Partial Dependence Analysis to Investigate Coupling Between Soil Moisture and Near‐Surface Temperature

  2023influential citation
• Observational constraints reduce model spread but not uncertainty in global wetland methane emission estimates

  2023cites this paper
• Idealized hydrodynamical numerical model dataset with no-river runoff at the western tropical North Atlantic

  2023cites this paper
• Identifying the regional emergence of climate patterns in the ARISE-SAI-1.5 simulations

  2023influential citation
• Blending high-resolution satellite rainfall estimates over urban catchment using Bayesian Model Averaging approach

  2023cites this paper
• Climate Change Impacts the State of Winter Roads Connecting Indigenous Communities: Case Study of Sakha (Yakutia) Republic

  2023cites this paper
• Hydrodynamic and TS Structure Dataset of the São Francisco and Parnaiba Brazilian Rivers

  2022influential citation
• The E3SM Diagnostics Package (E3SM Diags v2.7): a Python-based diagnostics package for Earth system model evaluation

  2022influential citation
• A Flood Forecasting Framework Coupling a High Resolution WRF Ensemble With an Urban Hydrologic Model

  2022cites this paper
• Quick Compression and Transmission of Meteorological Big Data in Complicated Visualization Systems

  2022cites this paper
• Data-Driven Artificial Intelligence for Calibration of Hyperspectral Big Data

  2022cites this paper
• Diurnal Rainfall Response to the Physiological and Radiative Effects of CO2 in Tropical Forests in the Energy Exascale Earth System Model v1

  2022cites this paper
• Analysing the PMIP4-CMIP6 collection: a workflow and tool (pmip_p2fvar_analyzer v1)

  2022cites this paper
• Monitoring of Hydrological Resources in Surface Water Change by Satellite Altimetry

  2022cites this paper
• A map of global peatland extent created using machine learning (Peat-ML)

  2022cites this paper
• Database of oceanographic anomalies and atmospheric surface fluxes for the study of climate change in the Brazilian Northeast.

  2022cites this paper
• Metrics for Intercomparison of Remapping Algorithms (MIRA) protocol applied to Earth system models

  2022cites this paper
• dataset of Oceanographic and biogeochemical anomalies in the Caribbean Sea.

  2022cites this paper
• Comparison of Climate Model Large Ensembles With Observations in the Arctic Using Simple Neural Networks

  2022cites this paper
• PostWRF: Interactive tools for the visualization of the WRF and ERA5 model outputs

  2022cites this paper
• Towards HPC and Big Data Analytics Convergence: Design and Experimental Evaluation of a HPDA Framework for eScience at Scale

  2021cites this paper
• An N-dimensional Fortran interpolation programme (NterGeo.v2020a) for geophysics sciences – application to a back-trajectory programme (Backplumes.v2020r1) using CHIMERE or WRF outputs

  2021cites this paper
• Processing methodology of global anthropogenic emissions for air quality modeling

  2021cites this paper
• Processing methodology of global anthropogenic for air quality modeling

  2021cites this paper
• TempestExtremes v2.1: A Community Framework for Feature Detection, Tracking and Analysis in Large Datasets

  2021cites this paper
• The Regional Importance of Oxygen Demand and Supply for Historical Ocean Oxygen Trends

  2021cites this paper
• Predicting Slowdowns in Decadal Climate Warming Trends With Explainable Neural Networks

  2021cites this paper
• PARASO, a circum-Antarctic fully-coupled ice-sheet - ocean - sea-ice - atmosphere - land model involving f.ETISh1.7, NEMO3.6, LIM3.6, COSMO5.0 and CLM4.5

  2021influential citation
• Snowmelt and Snow Sublimation in the Indus Basin

  2021cites this paper
• Metrics for Intercomparison of Remapping Algorithms (MIRA) applied to Earth System Models

  2021cites this paper
• TempestExtremes v2.1: a community framework for feature detection, tracking, and analysis in large datasets

  2021cites this paper
• Analyzing the PMIP4-CMIP6 ensemble: a workflow and tool (pmip_p2fvar_analyzer v1)

  2021cites this paper
• MPR 1.0: A stand-alone Multiscale Parameter Regionalization Tool for Improved Parameter Estimation of Land Surface Models

  2021cites this paper
• Performance evaluation and correction of precipitation data using the 20-year IMERG and TMPA precipitation products in diverse subregions of China

  2021cites this paper
• Introduction to Freva – A Free Evaluation System Framework for Earth System Modeling

  2021cites this paper
• Inundation prediction in tropical wetlands from JULES-CaMa-Flood global land surface simulations

  2021influential citation
• Greenland ice sheet mass balance from 1840 through next week

  2021cites this paper
• Integrating hydrologic modeling web services with online data sharing to prepare, store, and execute hydrologic models

  2020cites this paper
• Overcoming Data Scarcity in Earth Science

  2020cites this paper
• A global-scale evaluation of extreme event uncertainty in the eartH2Observe project

  2020cites this paper
• HydroDS: Data services in support of physically based, distributed hydrological models

  2020cites this paper
• CMIP5 Models Predict Rapid and Deep Soil Warming Over the 21st Century

  2020cites this paper
• Kenyan tea is made with heat and water: how will climate change influence its yield?

  2020cites this paper
• Kenyan tea is made with heat and water: how will climate change influence its yield?

  2020cites this paper
• Tools and Techniques for High Performance Computing: Selected Workshops, HUST, SE-HER and WIHPC, Held in Conjunction with SC 2019, Denver, CO, USA, November 17–18, 2019, Revised Selected Papers

  2020cites this paper
• Collaborative sharing of multidimensional space-time data in a next generation hydrologic information system

  2020cites this paper
• A N-dimensional Fortran Interpolation Program (NterGeo.v2020a) for Geophysics Sciences – Application to a back-trajectory program (BACKPLUMES.v2020r1) using CHIMERE or WRF outputs

  2020cites this paper
• Assessing the Impacts of Two Averaging Methods on AIRS Level 3 Monthly Products and Multiyear Monthly Means

  2020cites this paper
• High Resolution Chemistry Transport Modeling with the On-Line CHIMERE-WRF Model over the French Alps—Analysis of a Feedback of Surface Particulate Matter Concentrations on Mountain Meteorology

  2020cites this paper
• Emission inventory processing of biomass burning from a global dataset for air quality modeling

  2020cites this paper
• A High Spatiotemporal Resolution Global Gridded Dataset of Historical Human Discomfort Indices

  2020influential citation
• A New End-to-End Workflow for the Community Earth System Model (version 2.0) for CMIP6

  2020cites this paper
• A new end-to-end workflow for the Community Earth System Model (version 2.0) for the Coupled Model Intercomparison Project Phase 6 (CMIP6)

  2020influential citation
• Greenland liquid water discharge from 1958 through 2019

  2020cites this paper
• A Schwarz iterative method to evaluate ocean–atmosphere coupling schemes: implementation and diagnostics in IPSL-CM6-SW-VLR

  2020cites this paper
• py-meteo-num: Dockerized Python Notebook environment for portable data analysis workflows in Indonesian atmospheric science communities

  2020cites this paper
• The DOE E3SM Coupled Model Version 1: Overview and Evaluation at Standard Resolution

  2019cites this paper
• A statistical physics approach to perform fast highly-resolved air quality simulations - A new step towards the meta-modelling of chemistry transport models

  2019cites this paper
• TheDiaTo (v1.0) – A new diagnostic tool for water, energy and entropy budgets in climate models

  2019cites this paper
• A collaborative analysis framework for distributed gridded environmental data

  2019cites this paper
• Advancing Cyberinfrastructure for Collaborative Data Sharing and Modeling in Hydrology

  2019influential citation
• Towards High Performance Data Analytics for Climate Change

  2019cites this paper
• LIVVkit 2.1: automated and extensible ice sheet model validation

  2019cites this paper
• Understanding Monsoonal Water Cycle Changes in a Warmer Climate in E3SMv1 Using a Normalized Gross Moist Stability Framework

  2019cites this paper
• The Pangeo Ecosystem: Interactive Computing Tools for the Geosciences: Benchmarking on HPC

  2019cites this paper
• Historical global gridded degree‐days: A high‐spatial resolution database of CDD and HDD

  2019influential citation
• A High-Resolution Global Gridded Historical Dataset of Climate Extreme Indices

  2019influential citation
• Reply to reviewers

  2018cites this paper