Semi-Supervised Conditional Random Fields for Improved Sequence Segmentation and Labeling

We present a new semi-supervised training procedure for conditional random fields (CRFs) that can be used to train sequence segmentors and labelers from a combination of labeled and unlabeled training data. Our approach is based on extending the minimum entropy regularization framework to the structured prediction case, yielding a training objective that combines unlabeled conditional entropy with labeled conditional likelihood. Although the training objective is no longer concave, it can still be used to improve an initial model (e.g. obtained from supervised training) by iterative ascent. We apply our new training algorithm to the problem of identifying gene and protein mentions in biological texts, and show that incorporating unlabeled data improves the performance of the supervised CRF in this case.

• Publication year

  2006

• Venue

  Annual Meeting of the Association for Computational Linguistics

• Publication date

  2006-07-17

• Fields of study

  Computer Science

• Identifiers
  DOI 10.3115/1220175.1220202
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Accelerated training of conditional random fields with stochastic gradient methods

  2006cited by this paper
• Data-Dependent Regularization

  2006cited by this paper
• Maximum Margin Semi-Supervised Learning for Structured Variables

  2005cited by this paper
• Elements of Information Theory

  2005cited by this paper
• Semi-Supervised Sequence Modeling with Syntactic Topic Models

  2005cited by this paper
• Identifying gene and protein mentions in text using conditional random fields

  2005cited by this paper
• Learning from labeled and unlabeled data on a directed graph

  2005cited by this paper
• Understanding the Yarowsky Algorithm

  2004cited by this paper
• Numerical optimization

  2004cited by this paper
• Semi-supervised Learning by Entropy Minimization

  2004cited by this paper
• Learning with Local and Global Consistency

  2003cited by this paper
• Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions

  2003cited by this paper
• Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

  2001influential reference
• Maximum certainty data partitioning

  2000influential reference
• Text Classification from Labeled and Unlabeled Documents using EM

  2000cited by this paper
• Text Classi(cid:12)cation from Labeled and Unlabeled Documents using EM

  1998cited by this paper
• Combining labeled and unlabeled data with co-training

  1998cited by this paper
• The relative value of labeled and unlabeled samples in pattern recognition with an unknown mixing parameter

  1996cited by this paper
• Unsupervised Word Sense Disambiguation Rivaling Supervised Methods

  1995cited by this paper
• A Classification EM algorithm for clustering and two stochastic versions

  1992cited by this paper
• Pattern classification and scene analysis

  1974cited by this paper
• Pattern classification and scene analysis

  1974cited by this paper

• Named entity recognition based on anchor span for manufacturing knowledge extraction

  2025cites this paper
• Partial Sequence Labeling With Structured Gaussian Processes

  2022cites this paper
• Modeling Sequential Annotations for Sequence Labeling With Crowds

  2021influential citation
• Don’t Parse, Insert: Multilingual Semantic Parsing with Insertion Based Decoding

  2020cites this paper
• Weak Disambiguation for Partial Structured Output Learning

  2020cites this paper
• Don’t Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing

  2020cites this paper
• Weakly Supervised Sequence Tagging from Noisy Rules

  2020cites this paper
• Semi-Supervised Method using Gaussian Random Fields for Boilerplate Removal in Web Browsers

  2019cites this paper
• Variational Sequential Labelers for Semi-Supervised Learning

  2019cites this paper
• Extraction of Temporal Information from Clinical Narratives

  2019cites this paper
• Piecewise Large Margin Learning for Partially Annotated Sequences

  2019cites this paper
• Thread Structure Learning on Online Health Forums With Partially Labeled Data

  2019cites this paper
• Neural Aspect and Opinion Term Extraction with Mined Rules as Weak Supervision

  2019cites this paper
• Automatically classifying user requests in crowdsourcing requirements engineering

  2018cites this paper
• An Empirical Investigation on Fine-Grained Syndrome Segmentation in TCM by Learning a CRF from a Noisy Labeled Data

  2018cites this paper
• MLP – MomenT 2018 : The Second Workshop on Multi-Language Processing in a Globalising World The First Workshop on Multilingualism at the intersection of Knowledge Bases and Machine Translation

  2018cites this paper
• Part‐Based Mesh Segmentation: A Survey

  2018cites this paper
• Empirical evaluation of semi-supervised naïve Bayes for active learning

  2018cites this paper
• Semi-supervised learning for structured regression on partially observed attributed graphs

  2018cites this paper
• @UMass Amherst

  2018cites this paper
• Improving Language Understanding by Generative Pre-Training

  2018cites this paper
• Extraction of Temporal Events from Clinical Text Using Semi-supervised Conditional Random Fields

  2017cites this paper
• Ontology-based semi-supervised conditional random fields for automated information extraction from bridge inspection reports

  2017cites this paper
• A semi-supervised approach for temporal information extraction from clinical text

  2016cites this paper
• Graph-based Semi-supervised Gene Mention Tagging

  2016cites this paper
• A Comparative Study of Minimally Supervised Morphological Segmentation

  2016influential citation
• Patient-Centered and Experience-Aware Mining for Effective Information Discovery in Health Forums

  2016cites this paper
• Data collection & preprocessing Feature extraction Learning Classification Regression Retrieval Segmentation Reconstruction Synthesis Processed or generated data with semantic information Clustering Inference ... ... Feature learning Belief propagation Bayesian inference Gibbs sampling

  2016cites this paper
• A Probabilistic Address Parser Using Conditional Random Fields and Stochastic Regular Grammar

  2016cites this paper
• Semi-supervised Chinese Word Segmentation based on Bilingual Information

  2015influential citation
• Transfer Learning for Bibliographic Information Extraction

  2015cites this paper
• Automatic Entity Recognition and Typing from Massive Text Corpora: A Phrase and Network Mining Approach

  2015cites this paper
• Evolutionary non-linear modelling for selecting vaccines against antigenically variable viruses

  2015cites this paper
• Data‐Driven Shape Analysis and Processing

  2015cites this paper
• Semi-supervised Sequence Labeling for Named Entity Extraction based on Tri-Training: Case Study on Chinese Person Name Extraction

  2014cites this paper
• Lexicon expansion for latent variable grammars

  2014cites this paper
• Corpus-based Translation of Ontologies for Improved Multilingual Semantic Annotation Ontology-based Extraction of Structured Information from Publications on Preclinical Experiments for Spinal Cord Injury Treatments Semi-supervised Sequence Labeling for Named Entity Extraction Based on Tri-training:

  2014cites this paper
• Online discriminative learning for machine translation with binary-valued feedback

  2014cites this paper
• Unsupervised Dependency Parsing with Transferring Distribution via Parallel Guidance and Entropy Regularization

  2014influential citation
• Named Entity Extraction via Automatic Labeling and Tri-training: Comparison of Selection Methods

  2014cites this paper
• Automatic Corpus Expansion for Chinese Word Segmentation by Exploiting the Redundancy of Web Information

  2014cites this paper
• Semi-Supervised Learning

  2014cites this paper
• Joint semi-supervised learning of Hidden Conditional Random Fields and Hidden Markov Models

  2014influential citation
• Semantic Segmentation and Labeling of 3D Garments

  2014cites this paper
• Semi-Supervised Chinese Word Segmentation Using Partial-Label Learning With Conditional Random Fields

  2014influential citation
• Digital Library Technologies: Complex Objects, Annotation, Ontologies, Classification, Extraction, and Security

  2014cites this paper
• Log-Linear Models , Extensions and Applications

  2014cites this paper
• Painless Semi-Supervised Morphological Segmentation using Conditional Random Fields

  2014cites this paper
• Semi-Supervised Learning of Statistical Models for Natural Language Understanding

  2014cites this paper
• Mining biomedical information from scientific literature

  2013influential citation
• Machine Learning Paradigms for Speech Recognition: An Overview

  2013cites this paper
• Structured Output Learning with Candidate Labels for Local Parts

  2013cites this paper
• Improved Information Structure Analysis of Scientific Documents Through Discourse and Lexical Constraints

  2013cites this paper
• Graph-based Semi-Supervised Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging

  2013influential citation
• Wikipedia entity expansion and attribute extraction from the web using semi-supervised learning

  2013cites this paper
• Semi-supervised training for infrastructure mediated sensing: disaggregated hot and cold water sensing with minimal calibration

  2013cites this paper
• Co-regularizing character-based and word-based models for semi-supervised Chinese word segmentation

  2013cites this paper
• Discipline-Independent Text Information Extraction from Heterogeneous Styled References Using Knowledge from the Web

  2013cites this paper
• Feature Noising for Log-Linear Structured Prediction

  2013cites this paper
• Towards generalization of semi-supervised place classification over generalized Voronoi graph

  2013cites this paper
• A Joint Model to Identify and Align Bilingual Named Entities

  2013cites this paper
• Predicting Linguistic Structure with Incomplete and Cross-Lingual Supervision

  2013cites this paper
• Blog comments classification using tree structured conditional random fields

  2012influential citation
• Transductive Learning of Structural SVMs via Prior Knowledge Constraints

  2012cites this paper
• Semi-Supervised and Latent-Variable Models of Natural Language Semantics

  2012cites this paper
• Extension of TSVM to Multi-Class and Hierarchical Text Classification Problems With General Losses

  2012cites this paper
• Event Detection using Log-linear Models for Coronary Contrast Agent Injections

  2012cites this paper
• Technical Term Recognition with Semi-supervised Learning Using Hierarchical Bayesian Language Models

  2012cites this paper
• Entropic Regularization of Mixed-membership Network Models using Pseudo-observations

  2012cites this paper
• Regularized Structured Output Learning with Partial Labels

  2012cites this paper
• Conditional Random Fields for Information Extraction

  2012cites this paper
• Named Entity Recognition for Vietnamese Documents Using Semi-supervised Learning Method of CRFs with Generalized Expectation Criteria

  2012cites this paper
• Semi-supervised learning for acoustic and prosodic modeling in speech applications

  2012cites this paper
• Extracting Knowledge from Twitter and The Web

  2012cites this paper
• A hybrid two-stage approach for discipline-independent canonical representation extraction from references

  2012cites this paper
• Semi‐supervised Mesh Segmentation and Labeling

  2012cites this paper
• Comments Classification using Tree Structured Conditional Random Fields

  2012cites this paper
• Graph-Based Lexicon Expansion with Sparsity-Inducing Penalties

  2012cites this paper
• Integrating unsupervised and supervised word segmentation: The role of goodness measures

  2011cites this paper
• Prediction-based failure management for supercomputers

  2011cites this paper
• Aspects of Semi-supervised and Active Learning in Conditional Random Fields

  2011cites this paper
• Biomedical named entity recognition using generalized expectation criteria

  2011cites this paper
• Improving context-aware query classification via adaptive self-training

  2011cites this paper
• Disease named entity recognition using semisupervised learning and conditional random fields

  2011cites this paper
• Weakly supervised learning of information structure of scientific abstracts - is it accurate enough to benefit real-world tasks in biomedicine?

  2011cites this paper
• A vector space model of semantics using Canonical Correlation Analysis

  2011cites this paper
• Semi-supervised multi-task learning of structured prediction models for web information extraction

  2011cites this paper
• A Weakly-supervised Approach to Argumentative Zoning of Scientific Documents

  2011influential citation
• Semi-Supervised CONTRAfold for RNA Secondary Structure Prediction: A Maximum Entropy Approach

  2011influential citation
• Adapting n-gram maximum entropy language models with conditional entropy regularization

  2011cites this paper
• Discriminative semi-supervised learning of dynamical systems for motion estimation

  2011cites this paper
• Generalized expectation criteria for lightly supervised learning

  2011cites this paper
• Semi-Supervised Learning via Generalized Maximum Entropy

  2010cites this paper
• Discriminative training and variational decoding in machine translation via novel algorithms for weighted hypergraphs

  2010cites this paper
• Semi-supervised training of Gaussian mixture models by conditional entropy minimization

  2010cites this paper
• Training Conditional Random Fields Using Transfer Learning for Gesture Recognition

  2010cites this paper
• Structured Probabilistic Models for Natural Language Semantics

  2010influential citation
• Generalized Expectation Criteria for Semi-Supervised Learning with Weakly Labeled Data

  2010influential citation
• Morpho Challenge 2005-2010: Evaluations and Results

  2010cites this paper
• Contributions to the estimation of probabilistic discriminative models: semi-supervised learning and feature selection. (Contributions à l'estimation de modèles probabilistes discriminants: apprentissage semi-supervisé et sélection de caractéristiques)

  2010cites this paper