Semi-Supervised Learning-Based Calibration Model Building of NIR Spectroscopy for In Situ Measurement of Biochemical Processes Under Insufficiently and Inaccurately Labeled Samples

To deal with the typical problem of inaccurately and insufficiently labeled samples involved with real measurement of biochemical process components, a semi-supervised learning-based calibration model building method is proposed to use the up-to-date near-infrared (NIR) spectroscopy for in situ measurement of such a process. First, a supervised ladder network is designed to delete the inaccurately labeled samples, such that a cleaned training set is used for establishing the NIR calibration model. To address the issue of insufficiently labeled samples for modeling with the aim of online measurement, a semi-supervised ladder network is explored to establish a regression learning algorithm, by representing the process characteristics from all the unlabeled samples in combination with the cleaned labeled samples. In the ladder network, encoding is used to extract the features and decoding is used for denoising. Meanwhile, the skip connections allow higher layers to concentrate on those features related to the regression task, therefore facilitating the model accuracy. A numerical example and experiments on a glucose fermentation process are shown to demonstrate the effectiveness and advantage of the proposed method.

• Publication year

  2021

• Venue

  IEEE Transactions on Instrumentation and Measurement

• Publication date

  Unknown publication date

• Fields of study

  Chemistry, Engineering, Environmental Science, Computer Science

• Identifiers
  DOI 10.1109/TIM.2020.3029456
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• An effective framework based on local cores for self-labeled semi-supervised classification

  2020cited by this paper
• Semisupervised Robust Modeling of Multimode Industrial Processes for Quality Variable Prediction Based on Student's t Mixture Model

  2020cited by this paper
• Supervised Nonlinear Dynamic System for Soft Sensor Application Aided by Variational Auto-Encoder

  2020cited by this paper
• Semi-supervised mixture of latent factor analysis models with application to online key variable estimation

  2019cited by this paper
• Mixture modeling for industrial soft sensor application based on semi-supervised probabilistic PLS

  2019influential reference
• A survey on semi-supervised learning

  2019cited by this paper
• Calibration Model Building for Online Monitoring of the Granule Moisture Content during Fluidized Bed Drying by NIR Spectroscopy

  2019cited by this paper
• Near-infrared spectroscopy (NIRS) for a real time monitoring of the biogas process.

  2018cited by this paper
• A Cluster-then-label Semi-supervised Learning Approach for Pathology Image Classification

  2018cited by this paper
• Deep Learning of Semisupervised Process Data With Hierarchical Extreme Learning Machine and Soft Sensor Application

  2018cited by this paper
• Reg-Gan: Semi-Supervised Learning Based on Generative Adversarial Networks for Regression

  2018cited by this paper
• Semi‐supervised dynamic latent variable modeling: I/O probabilistic slow feature analysis approach

  2018cited by this paper
• Near infrared spectroscopy: A mature analytical technique with new perspectives - A review.

  2018cited by this paper
• Semi-supervised regression: A recent review

  2018influential reference
• A brief introduction to weakly supervised learning

  2018cited by this paper
• Self-training semi-supervised classification based on density peaks of data

  2018influential reference
• Semi-supervised Text Regression with Conditional Generative Adversarial Networks

  2018cited by this paper
• Data Mining and Analytics in the Process Industry: The Role of Machine Learning

  2017cited by this paper
• Semi-supervised Learning with Ladder Networks

  2015influential reference
• Learning from normalized local and global discriminative information for semi-supervised regression and dimensionality reduction

  2015cited by this paper
• Deep learning applications and challenges in big data analytics

  2015cited by this paper
• Co-training partial least squares model for semi-supervised soft sensor development

  2015cited by this paper
• From neural PCA to deep unsupervised learning

  2014influential reference
• A New Optical-Based Device for Online Black Powder Detection in Gas Pipelines

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• A novel approach for change detection of remotely sensed images using semi-supervised multiple classifier system

  2014cited by this paper
• Mixture analysis using reverse searching and non-negative least squares

  2014cited by this paper
• A graph Laplacian based approach to semi-supervised feature selection for regression problems

  2013cited by this paper
• Comparative Study of Multivariate Methods to Identify Paper Finishes Using Infrared Spectroscopy

  2012cited by this paper
• Critical review of near-infrared spectroscopic methods validations in pharmaceutical applications.

  2012cited by this paper
• Semi-supervised pattern classification of medical images: Application to mild cognitive impairment (MCI)

  2011cited by this paper
• Support Vector Machines Under Adversarial Label Noise

  2011cited by this paper
• Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion

  2010cited by this paper
• An Iterative Algorithm for Extending Learners to a Semi-Supervised Setting

  2008cited by this paper
• Semi-supervised learning of compact document representations with deep networks

  2008cited by this paper
• Semi-supervised SVM batch mode active learning for image retrieval

  2008cited by this paper
• Robust Truncated Hinge Loss Support Vector Machines

  2007cited by this paper
• A boosting approach to remove class label noise

  2005cited by this paper
• Indirect Spectral Hard Modeling for the Analysis of Reactive and Interacting Mixtures

  2004cited by this paper
• Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping

  2000cited by this paper
• An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

  2000cited by this paper
• Semi-Supervised Support Vector Machines

  1998cited by this paper
• Discovering Informative Patterns and Data Cleaning

  1996cited by this paper
• Asymptotic Properties of Nearest Neighbor Rules Using Edited Data

  1972cited by this paper
• Journal of Machine Learning Research Submitted 03/04; Revised Denoising Source Separation

  year unknowninfluential reference

• Semi-Supervised Calibration Modeling of Near-Infrared Spectroscopy via Just-In-Time Learning With Data Augmentation for In-Situ Measurement of Multiple Component Contents During the Fermentation Process

  2025cites this paper
• Near-Infrared Spectrum Calibration by Deep Dictionary Learning Regression With Data Augmentation for In-Situ Moisture Measurement of Fluidized Bed Drying Process

  2025cites this paper
• Stacked Target‐Related Autoencoder‐Extreme Learning Machine: A Novel Soft Measurement Modeling Approach for Near‐Infrared Spectroscopy

  2025cites this paper
• A Low-Dimensional Space-Based Redundancy Minimization Semisupervised Modeling Method for Total Nitrogen Detection

  2024cites this paper
• NIRWatchdog: Cross-Domain Product Quality Assessment Using Miniaturized Near-Infrared Sensors

  2024cites this paper
• Borehole lithology modelling with scarce labels by deep transductive learning

  2024cites this paper
• Semi-Supervised Deep Learning-Based Multi-component Spectral Calibration Modeling for UV-vis and Near-Infrared Spectroscopy without Information Loss.

  2023cites this paper
• Just-In-Time Learning Based Functional Spectral Data Modeling for In-Situ Measurement of Slurry Component Concentrations via Infrared Spectroscopy

  2023cites this paper
• Developing a Conditional Variational Autoencoder to Guide Spectral Data Augmentation for Calibration Modeling

  2022cites this paper
• A self-calibration model building method for rapid total nitrogen detection based on semi-supervised learning

  2022cites this paper
• A Semi-Supervised Ladder Network-Based Indoor Localization Using Channel State Information

  2022influential citation
• In Situ Testing Method for Laser Cleaning 7075 Aluminum Alloy Based on Grating Diffraction

  2022cites this paper
• Development of a calibration model for near infrared spectroscopy using a convolutional neural network

  2022cites this paper