HMM Specialization with Selective Lexicalization

We present a technique which complements Hidden Markov Models by incorporating some lexicalized states representing syntactically uncommon words. Our approach examines the distribution of transitions, selects the uncommon words, and makes lexicalized states for the words. We performed a part-of-speech tagging experiment on the Brown corpus to evaluate the resultant language model and discovered that this technique improved the tagging accuracy by 0.21% at the 95% level of confidence.

• Publication year

  1999

• Venue

  Conference on Empirical Methods in Natural Language Processing

• Publication date

  1999-12-01

• Fields of study

  Linguistics, Computer Science

• Identifiers
  arXiv cs/9912016
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Estimating Markov model structures

  1996influential reference
• A hierarchical Dirichlet language model

  1995cited by this paper
• Tagging English Text with a Probabilistic Model

  1994cited by this paper
• Equations for Part-of-Speech Tagging

  1993cited by this paper
• Robust part-of-speech tagging using a hidden Markov model

  1992cited by this paper
• A Stochastic Parts Program and Noun Phrase Parser for Unrestricted Text

  1988cited by this paper
• Grammatical Category Disambiguation by Statistical Optimization

  1988cited by this paper
• An introduction to hidden Markov models

  1986cited by this paper

• GWPT: A Green Word-Embedding-based POS Tagger

  2024cites this paper
• The State of the Art of Natural Language Processing—A Systematic Automated Review of NLP Literature Using NLP Techniques

  2023cites this paper
• OFrLex: A Computational Morphological and Syntactic Lexicon for Old French

  2020cites this paper
• Improving neural tagging with lexical information

  2017cites this paper
• Étiquetage multilingue en parties du discours avec MElt (Multilingual part-of-speech tagging with MElt)[In French]

  2016cites this paper
• External Lexical Information for Multilingual Part-of-Speech Tagging

  2016cites this paper
• Using unsupervised corpus-based methods to build rule-based machine translation systems

  2011cites this paper
• A Simple and Effective Closed Test for Chinese Word Segmentation Based on Sequence Labeling

  2010cites this paper
• Empleo de m etodos no supervisados basados en corpus para construir traductores autom aticos basados en reglas Using unsupervised corpus-based methods to build rule-based machine translation systems

  2009cites this paper
• Using unsupervised corpus-based methods to build rule-based machine translation systems Tesis doctoral Empleo de metodos no supervisados basados en corpus para construir traductores automaticos basados en reglas

  2008cites this paper
• Chinese word segmentation as morpheme-based lexical chunking

  2008cites this paper
• 基於特製隱藏式馬可夫模型之中文斷詞研究 (Chinese Word Segmentation using Specialized HMM) [In Chinese]

  2006cites this paper
• 基於特製隱藏式馬可夫模型之中文斷詞研究 (Chinese Word Segmentation using Specialized HMM) [In Chinese]

  2006cites this paper
• Voting Between Multiple Data Representations for Text Chunking

  2005cites this paper
• VOTING BETWEEN MULTIPLE DATA REPRESENTATIONS FOR TEXT CHUNKING by Hong Shen

  2004cites this paper
• Improving part-of-speech tagging using lexicalized HMMs

  2004cites this paper
• Self-Organizing Markov Models and Their Application to Part-of-Speech Tagging

  2003cites this paper
• Improving part-of-speech tagging using lexicalized HMMs

  2003cites this paper
• Lexicalized Hidden Markov Models for Part-of-Speech Tagging

  2002influential citation
• Shallow Parsing using Specialized HMMs

  2002cites this paper
• Lexicalized Hidden Markov Models for Part-of-Speech Tagging

  2000cites this paper