Does Baum-Welch Re-estimation Help Taggers?

In part of speech tagging by Hidden Markov Model, a statistical model is used to assign grammatical categories to words in a text. Early work in the field relied on a corpus which had been tagged by a human annotator to train the model. More recently, Cutting et al. (1992) suggest that training can be achieved with a minimal lexicon and a limited amount of a priori information about probabilities, by using an Baum-Welch re-estimation to automatically refine the model. In this paper, I report two experiments designed to determine how much manual training information is needed. The first experiment suggests that initial biasing of either lexical or transition probabilities is essential to achieve a good accuracy. The second experiment reveals that there are three distinct patterns of Baum-Welch reestimation. In two of the patterns, the re-estimation ultimately reduces the accuracy of the tagging rather than improving it. The pattern which is applicable can be predicted from the quality of the initial model and the similarity between the tagged training corpus (if any) and the corpus to be tagged. Heuristics for deciding how to use re-estimation in an effective manner are given. The conclusions are broadly in agreement with those of Merialdo (1994), but give greater detail about the contributions of different parts of the model.

• Publication year

  1994

• Venue

  Applied Natural Language Processing Conference

• Publication date

  1994-10-13

• Fields of study

  Computer Science

• Identifiers
  DOI 10.3115/974358.974371arXiv cmp-lg/9410012
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Tagging English Text with a Probabilistic Model

  1994cited by this paper
• Building a Large Annotated Corpus of English: The Penn Treebank

  1993cited by this paper
• Tagging an Unfamiliar Text With Minimal Human Supervision

  1992cited by this paper
• A Simple Rule-Based Part of Speech Tagger

  1992cited by this paper
• A Practical Part-of-Speech Tagger

  1992influential reference
• Robust part-of-speech tagging using a hidden Markov model

  1992influential reference
• Hidden Markov Models for Speech Recognition

  1991cited by this paper
• Probabilistic Models of Short and Long Distance Word Dependencies in Running Text

  1989cited by this paper
• The computational analysis of English : a corpus-based approach

  1989cited by this paper
• Grammatical Category Disambiguation by Statistical Optimization

  1988cited by this paper
• A Stochastic Parts Program and Noun Phrase Parser for Unrestricted Text

  1988cited by this paper
• The Computational Analysis of English—A Corpus‐Based Approach

  1988cited by this paper

• Video Segmentation and Tokenization for Model-Based Video Scene Classification

  2025cites this paper
• An empirical evaluation of deep semi-supervised learning

  2025cites this paper
• Spectrally Transformed Kernel Regression

  2024cites this paper
• Developing a Novel Methodology by Integrating Deep Learning and HMM for Segmentation of Retinal Blood Vessels in Fundus Images

  2023cites this paper
• A Survey on Dynamic Fuzzy Machine Learning

  2022cites this paper
• Structured Models for Semantic Analysis of Audio Content

  2022cites this paper
• A Cascaded Unsupervised Model for PoS Tagging

  2021cites this paper
• Improved Inference for Imputation-Based Semisupervised Learning Under Misspecified Setting

  2021cites this paper
• Speech Emotion Recognition Based on Acoustic Segment Model

  2021cites this paper
• Predicting learners' effortful behaviour in adaptive assessment using multimodal data

  2020cites this paper
• Semi-Supervised Robust Mixture Models in RKHS for Abnormality Detection in Medical Images

  2020cites this paper
• Adversarially Robust Generalization Just Requires More Unlabeled Data

  2019cites this paper
• A Hybrid Approach to Acoustic Scene Classification Based on Universal Acoustic Models

  2019cites this paper
• Robust Hidden Markov Model based intelligent blood vessel detection of fundus images

  2017cites this paper
• The Pessimistic Limits and Possibilities of Margin-based Losses in Semi-supervised Learning

  2016cites this paper
• The Pessimistic Limits of Margin-based Losses in Semi-supervised Learning

  2016cites this paper
• POS-tagging of Historical Dutch

  2016cites this paper
• Unsupervised learning of semantic relations of a morphologically rich language

  2016cites this paper
• Semi-supervised learning for genomic prediction of novel traits with small reference populations: an application to residual feed intake in dairy cattle

  2016cites this paper
• UvA-DARE ( Digital Academic Repository ) POS-tagging of Historical Dutch Hupkes

  2016cites this paper
• Unsupervised Analysis of Structured Human Artifacts

  2015cites this paper
• New data-driven approaches to text simplification

  2015cites this paper
• Robust semi-supervised least squares classification by implicit constraints

  2015cites this paper
• Implicitly Constrained Semi-supervised Linear Discriminant Analysis

  2014cites this paper
• Unsupervised Learning for Syntactic Disambiguation

  2014cites this paper
• A novel and robust parameter training approach for HMMs under noisy and partial access to states

  2014cites this paper
• A hybrid selection method of helpful unlabeled data applicable for semi-supervised learning algorithms

  2014cites this paper
• A Hybrid Selection Method of Helpful Unlabeled Data Applicable for Semi-Supervised Learning Algorithm

  2014cites this paper
• A Ruled-Based Part of Speech (RPOS) Tagger for Malay Text Articles

  2013cites this paper
• Stagger: an Open-Source Part of Speech Tagger for Swedish

  2013cites this paper
• Semi-Supervised Classification Techniques in Big Data Text Analytics

  2013cites this paper
• Software state monitoring model studies based on multivariate HPM

  2013cites this paper
• Morpho-Syntactic Analysis Framework for Tone Language Text-to-Speech Systems

  2012cites this paper
• A Novel Training Algorithm for HMMs with Partial and Noisy Access to the States

  2012cites this paper
• Adaptive Bayesian HMM for Fully Unsupervised Chinese Part-of-Speech Induction

  2012cites this paper
• Особенности обучения программирования будуших учителей информатики Украины с учетом требований современности

  2012cites this paper
• Hidden Markov Model training with side information

  2012cites this paper
• Statistical malay part-of-speech (POS) tagger using Hidden Markov approach

  2011cites this paper
• Machine Learning for Natural Language Processing

  2011cites this paper
• Structured Models for Audio Content Analysis

  2011cites this paper
• Unsupervised Structure Prediction with Non-Parallel Multilingual Guidance

  2011cites this paper
• Minimal supervision for language learning: bootstrapping global patterns from local knowledge

  2011cites this paper
• Structured Models for Audio Content Analysis Ph.D. Thesis Proposal

  2011cites this paper
• Lateen EM: Unsupervised Training with Multiple Objectives, Applied to Dependency Grammar Induction

  2011cites this paper
• From Baby Steps to Leapfrog: How “Less is More” in Unsupervised Dependency Parsing

  2010influential citation
• New directions in semi-supervised learning

  2010cites this paper
• Semi-supervised self-learning on imbalanced data sets

  2010cites this paper
• Painless Unsupervised Learning with Features

  2010cites this paper
• Contributions to the estimation of probabilistic discriminative models: semi-supervised learning and feature selection. (Contributions à l'estimation de modèles probabilistes discriminants: apprentissage semi-supervisé et sélection de caractéristiques)

  2010cites this paper
• Recognition method of behavioral pattern by sequential learning for life logs

  2010cites this paper
• Viterbi Training Improves Unsupervised Dependency Parsing

  2010cites this paper
• Syntactic Disambiguation by Learning Weighted Government Patterns from a Large Corpus

  2010cites this paper
• A comparison of unsupervised methods for Part-of-Speech Tagging in Chinese

  2010cites this paper
• Hierarchical Semantic Tagger for Robust Spoken Language Understanding

  2009influential citation
• A semi-supervised learning method to classify grant support zone in web-based medical articles

  2009cites this paper
• HMMs, GRs, and N-Grams as Lexical Substitution Techniques – Are They Portable to Other Languages?

  2009cites this paper
• Probabilistic Modeling of Korean Morphology

  2009cites this paper
• Arabic Part Of Speech Disambiguation: A Survey

  2009cites this paper
• Baby Steps: How “Less is More” in Unsupervised Dependency Parsing

  2009cites this paper
• Refining the most frequent sense baseline

  2009cites this paper
• Introduction to Semi-Supervised Learning

  2009cites this paper
• One in the bush Low-density language technology

  2009cites this paper
• Unlabelled extra data do not always mean extra performance for semi‐supervised fault prediction

  2009cites this paper
• Part-of-Speech Tagging for Bengali

  2009cites this paper
• Unsupervised Approaches to Part-of-Speech Tagging Five methodologies surveyed

  2008cites this paper
• Optimising the speed and accuracy of a statistical GLR parser

  2008cites this paper
• Using Prior Domain Knowledge to Build Robust HMM-Based Semantic Tagger Trained on Completely Unannotated Data

  2008cites this paper
• EM Can Find Pretty Good HMM POS-Taggers (When Given a Good Start)

  2008influential citation
• Part-of-speech tagging of Modern Hebrew text

  2008cites this paper
• Proceedings of the Workshop on Prior Knowledge for Text and Language Processing

  2008cites this paper
• Processing Tools and Services from iLexIR Ltd

  2008cites this paper
• Semi-supervised learning on large complex simulations

  2008cites this paper
• A Fault Prediction Model with Limited Fault Data to Improve Test Process

  2008cites this paper
• Unsupervised Approaches to Part Unsupervised Approaches to Part Unsupervised Approaches to Part Unsupervised Approaches to Part---of--Tagging

  2008cites this paper
• Building Domain-Specific Taggers without Annotated (Domain) Data

  2007cites this paper
• Novel estimation methods for unsupervised discovery of latent structure in natural language text

  2007cites this paper
• Semi-supervised learning of the hidden vector state model for extracting protein-protein interactions

  2007cites this paper
• Learning on Complex Simulations

  2007cites this paper
• from Unlabeled Partially-bracketed Data

  2007influential citation
• Semi-supervised Training of a Statistical Parser from Unlabeled Partially-bracketed Data

  2007cites this paper
• Knowledge Sources for WSD

  2007cites this paper
• Comparison of Unigram, Bigram, HMM and Brill's POS tagging approaches for some South Asian languages

  2007cites this paper
• 1 Neural Networks , Part-of-Speech Tagging and Lexicons

  2007cites this paper
• Semantic Tagging for Medical Documents Using a Hidden Markov Model

  2006cites this paper
• Part-of-speech tagging models for parsing

  2006cites this paper
• Comparison of different POS tagging techniques for some South Asian languages

  2006cites this paper
• The Second Release of the RASP System

  2006cites this paper
• Semantic Tagging for Medical Knowledge Tracking

  2006cites this paper
• Text Mining for Medical Documents Using a Hidden Markov Model

  2006cites this paper
• An introduction to tag sequence grammars and the RASP system parser

  2006cites this paper
• Probabilistic word sense disambiguation: analysis and techniques for combining knowledge sources

  2006cites this paper
• Integration of Low Level Linguistic Information for Clinical Document Semantic Tagging

  2006cites this paper
• An Unsupervised Morpheme-Based HMM for Hebrew Morphological Disambiguation

  2006influential citation
• Mining for Medical Documents Using a Hidden Markov Model

  2006cites this paper
• LeXFlow: A System for Cross-Fertilization of Computational Lexicons

  2006cites this paper
• Part-of-Speech Tagging

  2006cites this paper
• Hybrid Syntactic Category Induction

  2005cites this paper
• Improved estimation for unsupervised part-of-speech tagging

  2005cites this paper
• Etiquetage morpho-syntaxique du français à base d’apprentissage supervisé

  2005cites this paper
• The importance of the lexicon in tagging biological text

  2005cites this paper