Automatic Evaluation and Uniform Filter Cascades for Inducing N-Best Translation Lexicons

This paper shows how to induce an N-best translation lexicon from a bilingual text corpus using statistical properties of the corpus together with four external knowledge sources. The knowledge sources are cast as filters, so that any subset of them can be cascaded in a uniform framework. A new objective evaluation measure is used to compare the quality of lexicons induced with different filter cascades. The best filter cascades improve lexicon quality by up to 137% over the plain vanilla statistical method, and approach human performance. Drastically reducing the size of the training corpus has a much smaller impact on lexicon quality when these knowledge sources are used. This makes it practical to train on small hand-built corpora for language pairs where large bilingual corpora are unavailable. Moreover, three of the four filters prove useful even when used with large training corpora.

• Publication year

  1995

• Venue

  VLC@ACL

• Publication date

  1995-05-01

• Fields of study

  Computer Science

• Identifiers
  arXiv cmp-lg/9505044
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Using Bi-textual Alignment for Translation Validation: the TransCheck system

  1994cited by this paper
• Improving Chinese Tokenization With Linguistic Filters on Statistical Lexical Acquisition

  1994cited by this paper
• Accurate Methods for the Statistics of Surprise and Coincidence

  1993cited by this paper
• Using cognates to align sentences in bilingual corpora

  1993cited by this paper
• But Dictionaries Are Data Too

  1993influential reference
• The Mathematics of Statistical Machine Translation: Parameter Estimation

  1993cited by this paper
• A Program for Aligning Sentences in Bilingual Corpora

  1993cited by this paper
• A Simple Rule-Based Part of Speech Tagger

  1992cited by this paper
• Identifying word correspondence in parallel texts

  1991cited by this paper
• Identifying Word Correspondences in Parallel Texts

  1991cited by this paper
• The String-to-String Correction Problem

  1974cited by this paper

• HyperSumm-RL: A Dialogue Summarization Framework for Modeling Leadership Perception in Social Robots

  2025cites this paper
• Investigating Unsupervised Neural Machine Translation for Low-resource Language Pair English-Mizo via Lexically Enhanced Pre-trained Language Models

  2023influential citation
• CharSpan: Utilizing Lexical Similarity to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages

  2023cites this paper
• SelectNoise: Unsupervised Noise Injection to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages

  2023cites this paper
• Short Sequence Chinese-English Machine Translation Based on Generative Adversarial Networks of Emotion

  2022cites this paper
• Machine Translation from Standard German to Alemannic Dialects

  2022cites this paper
• Simple measures of bridging lexical divergence help unsupervised neural machine translation for low-resource languages

  2021cites this paper
• An Unsupervised Normalization Algorithm for Noisy Text: A Case Study for Information Retrieval and Stance Detection

  2021influential citation
• A Large-scale Evaluation of Neural Machine Transliteration for Indic Languages

  2021cites this paper
• Identify Bilingual Patterns and Phrases from a Bilingual Sentence Pair

  2021cites this paper
• Detection and Rephrasing for Chinese Clickbait News Headlines

  2021cites this paper
• Utilizing Language Relatedness to improve Machine Translation: A Case Study on Languages of the Indian Subcontinent

  2020cites this paper
• Automatic Identification and Production of Related Words for Historical Linguistics

  2020cites this paper
• Contact Relatedness can help improve multilingual NMT: Microsoft STCI-MT @ WMT20

  2020cites this paper
• Bilingual Lexicon Induction across Orthographically-distinct Under-Resourced Dravidian Languages

  2020cites this paper
• Toward Formalization of Comprehensive Bilingual Dictionaries Creation Planning as Constraint Optimization Problem

  2020cites this paper
• A deep learning approach to bilingual lexicon induction in the biomedical domain

  2018cites this paper
• Neural Text Normalization for Turkish Social Media

  2018cites this paper
• Bilingual Lexicon Induction by Learning to Combine Word-Level and Character-Level Representations

  2017cites this paper
• A Generalized Constraint Approach to Bilingual Dictionary Induction for Low-Resource Language Families

  2017cites this paper
• Empirical evaluation of NMT and PBSMT quality for large-scale translation production.

  2017cites this paper
• Utilizing Lexical Similarity for pivot translation involving resource-poor, related languages

  2017cites this paper
• On expected score of cellwise alignments

  2017cites this paper
• Augmenting Translation Lexica by Learning Generalised Translation Patterns

  2017cites this paper
• Neural Network-based Word Alignment through Score Aggregation

  2016cites this paper
• Lower Bounds on the Generalized Central Moments of the Optimal Alignments Score of Random Sequences

  2016cites this paper
• Improving KantanMT Training Efficiency with fast_align

  2016cites this paper
• Automatic cognate classification with a Support Vector Machine

  2016cites this paper
• Lower bounds for moments of global scores of pairwise Markov chains

  2016cites this paper
• AUTOMATIC EVALUATION OF MACHINE TRANSLATION QUALITY OF A SCIENTIFIC TEXT

  2016cites this paper
• Bilingual Lexicon Extraction from Arabic-English Parallel Corpora with a View to Machine Translation

  2016cites this paper
• Orthographic Syllable as basic unit for SMT between Related Languages

  2016cites this paper
• Learning variable length units for SMT between related languages via Byte Pair Encoding

  2016influential citation
• An algorithm of word indexing model for document summarization based on perspective of document

  2016cites this paper
• Modernising historical Slovene words

  2015cites this paper
• Pivot-Based Bilingual Dictionary Creation for Low-Resource Languages

  2015influential citation
• Lower Bounds on the Generalized Central Moments of the Optimal Alignments Score of Random Sequences

  2015cites this paper
• Unsupervised Text Normalization Using Distributed Representations of Words and Phrases

  2015cites this paper
• Classification and Selection of Translation Candidates for Parallel Corpora Alignment

  2015cites this paper
• Automatic Detection of Cognates Using Orthographic Alignment

  2014cites this paper
• An Etymological Approach to Cross-Language Orthographic Similarity. Application on Romanian

  2014cites this paper
• Evaluation Metrics for the Summarization Task

  2014cites this paper
• Building a Dataset of Multilingual Cognates for the Romanian Lexicon

  2014cites this paper
• On the Romance Languages Mutual Intelligibility

  2014cites this paper
• A Dictionary-Based Approach for Evaluating Orthographic Methods in Cognates Identification

  2013cites this paper
• Proportion of Gaps and Fluctuations of the Optimal Score in Random Sequence Comparison

  2013cites this paper
• US 7 , 624 , 005 B 2 Page 2 OTHER PUBLICATIONS

  2013cites this paper
• High-Accuracy Phrase Translation Acquisition Through Battle-Royale Selection

  2013cites this paper
• General approach to the fluctuations problem in random sequence comparison

  2012cites this paper
• Improving Statistical Machine Translation for a Resource-Poor Language Using Related Resource-Rich Languages

  2012influential citation
• Minimally supervised techniques for bilingual lexicon extraction

  2012influential citation
• Towards high-accuracy bilingual phrase acquisition from parallel corpora

  2012influential citation
• STATISTICAL MACHINE TRANSLATION WITHOUT PARALLEL DATA

  2012cites this paper
• Development of shahmukhi to gurmukhi transliteration system

  2011cites this paper
• Using SVMs for Filtering Translation Tables for Parallel Corpora Alignment

  2011influential citation
• Lexical selection for machine translation

  2011cites this paper
• Acquiring Word-Meaning Mappings for Natural Language Interfaces

  2011cites this paper
• Data Driven Models for Language Evolution

  2011influential citation
• Statistical machine translation

  2010cites this paper
• Dependency-based translation equivalents for factored machine translation

  2010cites this paper
• The rate of the convergence of the mean score in random sequence comparison

  2010cites this paper
• Information Retrieval Techniques for Pattern Matching: Managing and Searching Textual and XML Information in 21st Century Applications

  2010cites this paper
• Cria¸c˜ao de L´exicos Bilingues Para Tradu¸c˜ao Autom´atica Estat´ıstica: Extended

  2010cites this paper
• Bilingual Lexicon Induction: Effortless Evaluation of Word Alignment Tools and Production of Resources for Improbable Language Pairs

  2010cites this paper
• Contribution des basses fréquences à l'alignement sous-phrastique multilingue : une approche différentielle

  2010influential citation
• Improved Statistical Machine Translation for Resource-Poor Languages Using Related Resource-Rich Languages

  2009influential citation
• L'alignement sous-phrastique multilingue pour les nuls

  2009cites this paper
• Dynamic Bayesian Networks for Transliteration Discovery and Generation

  2009cites this paper
• Hapax Legomena: Their Contribution in Number and Efficiency to Word Alignment

  2009cites this paper
• Exploiting linguistic and statistical knowledge in a text alignment system

  2009influential citation
• NUS at WMT09: Domain Adaptation Experiments for English-Spanish Machine Translation of News Commentary Text

  2009cites this paper
• Effective retrieval techniques for Arabic text

  2008cites this paper
• BAKAL A RSK A PR ACE

  2008cites this paper
• ALIGNMENT OF BILINGUAL ANTHROPONYMS AND TOPONYMS IN PARALLEL CORPORA FRENCH – ARABIC

  2008influential citation
• Differential Effect of Correct Name Translation on Human and Automated Judgments of Translation Acceptability: A Pilot Study

  2008cites this paper
• Ontology based semantic query processing in database systems

  2008cites this paper
• Alignment Of Bilingual Named Entities In French– Arabic Parallel Corpora

  2008cites this paper
• Efficient concept clustering for ontology learning using an event life cycle on the web

  2008cites this paper
• Automatic Construction of Chinese-english Translation Lexicons Final Project Report under Nsa Grant Mda904-97-c-3055 Introduction and Executive Summary

  2007cites this paper
• Improved Word Alignments Using the Web as a Corpus

  2007cites this paper
• Finding Variants of Out-of-Vocabulary Words in Arabic

  2007cites this paper
• An Investigation of the Relationship Between Automated Machine Translation Evaluation Metrics and User Performance on an Information Extraction Task

  2007cites this paper
• Generalized Parsers for Machine Translation

  2007cites this paper
• Meta-Structure Transformation Model for Statistical Machine Translation

  2007cites this paper
• Extracting Translation Lexicons from Bilingual Corpora : Application to South-Slavonic Languages

  2007cites this paper
• Cognate or False Friend ? Ask the Web !

  2007cites this paper
• Creating a Bilingual Psychology Lexicon for Cross Lingual Question Answering - A Pilot Study

  2007cites this paper
• Indução de léxicos bilíngües e regras para a tradução automática

  2007cites this paper
• Loose phrase extraction with n -best alignments

  2007cites this paper
• Tree-Structured Models of Multitext: Theory, Design and Experiments

  2007cites this paper
• Bilingual Word and Chunk Alignment: A Hybrid System for Amharic and English

  2007cites this paper
• Ontology-based Semantic Query Processing in Database Systems

  2007cites this paper
• A novel English/Chinese information retrieval approach in hotel website searching

  2007cites this paper
• Automatic Processing of Parallel Corpora(cid:2) A Swedish Perspective

  2007cites this paper
• WebSim : A Novel Term Similarity Metric based on a Web Search Technology

  2006cites this paper
• Semi-Supervised Training for Statistical Word Alignment

  2006cites this paper
• WebSim : A Pathway to Unveiling Term Relationships using a Web Search Technology

  2006cites this paper
• Practical Translation Pattern Acquisition with Utilizing Language Resources

  2006cites this paper
• Data-driven Amharic-English Bilingual Lexicon Acquisition

  2006cites this paper
• EXTRA: a system for example-based translation assistance

  2006cites this paper