Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates a correct translation from this representation. In this paper, we focus on analyzing the properties of the neural machine translation using two models; RNN Encoder--Decoder and a newly proposed gated recursive convolutional neural network. We show that the neural machine translation performs relatively well on short sentences without unknown words, but its performance degrades rapidly as the length of the sentence and the number of unknown words increase. Furthermore, we find that the proposed gated recursive convolutional network learns a grammatical structure of a sentence automatically.
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Kyunghyun Cho,B. V. Merrienboer,Dzmitry Bahdanau,Yoshua Bengio
Published 2014 in SSST@EMNLP
ABSTRACT
PUBLICATION RECORD
- Publication year
2014
- Venue
SSST@EMNLP
- Publication date
2014-09-03
- Fields of study
Mathematics, Computer Science
- Identifiers
- External record
- Source metadata
Semantic Scholar
CITATION MAP
EXTRACTION MAP
LINKED PAPERS
- Neural Machine Translation by Jointly Learning to Align and Translate
- encoder-decoder architecture is a · An RNN Encoder--Decoder is a recurrent neural machine translation model that follows the encoder-decoder architecture of encoding a source sentence and decoding a target translation.
- neural machine translation related to · Both neural machine translation concepts describe the same neural-network-based translation approach that maps a source sentence to a target sentence.
CLAIMS
CONCEPTS
- gated recursive convolutional neural network
A neural network architecture proposed for machine translation that combines gating with recursive convolutional structure.
Aliases: grConv
박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review - grammatical structure
The syntactic organization of a sentence into ordered and hierarchical parts.
Aliases: syntax
박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review - neural machine translation
A translation approach that uses neural networks to map a source sentence to a target sentence.
Aliases: NMT
박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review - rnn encoder--decoder
A recurrent encoder-decoder translation model used as one of the systems analyzed in the paper.
Aliases: RNN encoder-decoder
박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review - sentence length
The number of words or tokens in an input sentence, used as a factor in translation behavior.
박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review - short sentences
Input sentences with relatively few words or tokens, used here as a length-based subset for evaluation.
박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review - unknown words
Words or tokens that are not covered by the model vocabulary and are treated as out-of-vocabulary items.
Aliases: UNK words, out-of-vocabulary words
박진우 (dztg5apj7m) extractionAnonymous (12632b8b5f) review
REFERENCES
Showing 1-12 of 12 references · Page 1 of 1