Graph Convolutional Encoders for Syntax-aware Neural Machine Translation

We present a simple and effective approach to incorporating syntactic structure into neural attention-based encoder-decoder models for machine translation. We rely on graph-convolutional networks (GCNs), a recent class of neural networks developed for modeling graph-structured data. Our GCNs use predicted syntactic dependency trees of source sentences to produce representations of words (i.e. hidden states of the encoder) that are sensitive to their syntactic neighborhoods. GCNs take word representations as input and produce word representations as output, so they can easily be incorporated as layers into standard encoders (e.g., on top of bidirectional RNNs or convolutional neural networks). We evaluate their effectiveness with English-German and English-Czech translation experiments for different types of encoders and observe substantial improvements over their syntax-agnostic versions in all the considered setups.

• Publication year

  2017

• Venue

  Conference on Empirical Methods in Natural Language Processing

• Publication date

  2017-04-15

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.18653/v1/D17-1209arXiv 1704.04675
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Convolutional Neural Networks

  2018cited by this paper
• Neural Machine Translation with Source-Side Latent Graph Parsing

  2017cited by this paper
• Modeling Relational Data with Graph Convolutional Networks

  2017cited by this paper
• Neural Monkey: An Open-source Tool for Sequence Learning

  2017cited by this paper
• Towards String-To-Tree Neural Machine Translation

  2017cited by this paper
• Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling

  2017influential reference
• Neural Message Passing for Quantum Chemistry

  2017cited by this paper
• Learning to Parse and Translate Improves Neural Machine Translation

  2017cited by this paper
• Syntax-aware Neural Machine Translation Using CCG

  2017cited by this paper
• Syntactically Guided Neural Machine Translation

  2016cited by this paper
• Neural Machine Translation in Linear Time

  2016cited by this paper
• Tree-to-Sequence Attentional Neural Machine Translation

  2016cited by this paper
• Molecular graph convolutions: moving beyond fingerprints

  2016cited by this paper
• Recurrent Neural Network Grammars

  2016cited by this paper
• Linguistic Input Features Improve Neural Machine Translation

  2016cited by this paper
• Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

  2016cited by this paper
• Edinburgh Neural Machine Translation Systems for WMT 16

  2016cited by this paper
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

  2016cited by this paper
• A Convolutional Encoder Model for Neural Machine Translation

  2016influential reference
• Does String-Based Neural MT Learn Source Syntax?

  2016cited by this paper
• Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies

  2016cited by this paper
• Learning to Compose Words into Sentences with Reinforcement Learning

  2016cited by this paper
• Effective Approaches to Attention-based Neural Machine Translation

  2015cited by this paper
• Evaluating MT systems with BEER

  2015cited by this paper
• Deep Residual Learning for Image Recognition

  2015cited by this paper
• Multi-task Sequence to Sequence Learning

  2015cited by this paper
• Convolutional Networks on Graphs for Learning Molecular Fingerprints

  2015cited by this paper
• Neural Machine Translation of Rare Words with Subword Units

  2015cited by this paper
• Opinion Mining with Deep Recurrent Neural Networks

  2014cited by this paper
• Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

  2014influential reference
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• On the Properties of Neural Machine Translation: Encoder–Decoder Approaches

  2014cited by this paper
• Fitting Sentence Level Translation Evaluation with Many Dense Features

  2014cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014influential reference
• Sequence to Sequence Learning with Neural Networks

  2014cited by this paper
• Recurrent Continuous Translation Models

  2013influential reference
• Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

  2013cited by this paper
• Recurrent neural networks

  2013cited by this paper
• Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability

  2011cited by this paper
• Learning to Translate with Source and Target Syntax

  2010cited by this paper
• The CoNLL 2008 Shared Task on Joint Parsing of Syntactic and Semantic Dependencies

  2008cited by this paper
• Syntax Augmented Machine Translation via Chart Parsing

  2006cited by this paper
• A Study of Translation Edit Rate with Targeted Human Annotation

  2006cited by this paper
• Quasi-Synchronous Grammars: Alignment by Soft Projection of Syntactic Dependencies

  2006cited by this paper
• A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION

  2005cited by this paper
• Bleu: a Method for Automatic Evaluation of Machine Translation

  2002cited by this paper
• Bidirectional recurrent neural networks

  1997cited by this paper
• Long Short-Term Memory

  1997influential reference
• Finding Structure in Time

  1990cited by this paper

• Multi-Task Disaster Tweet Classification Using Hybrid TF-IDF and Graph Convolutional Networks

  2026cites this paper
• Graph Neural Networks Improve Quantized Transformers by Incorporating Global Relationships

  2026cites this paper
• Dependency-aware self-attention for robust neural machine translation

  2026cites this paper
• Graph Neural Network (GNN) and its Application: A State-of-the-Art Survey

  2025cites this paper
• MLGNN: a metric learning and graph neural network based approach for fake news detection in online social networks

  2025cites this paper
• Regularizing Softmax With Graph Similarity for Enhanced Node Classification in Semisupervised Settings

  2025cites this paper
• Graph Representation Learning for Infrared and Visible Image Fusion

  2025cites this paper
• BAT: A Versatile Bipartite Attention-Based Approach for Comprehensive Truth Inference in Mobile Crowdsourcing

  2025cites this paper
• Graph Neural Networks: Architectures, Applications, and Future Directions

  2025cites this paper
• CompressGNN: Accelerating Graph Neural Network Training via Hierarchical Compression

  2025cites this paper
• Do We Really Need GNNs with Explicit Structural Modeling? MLPs Suffice for Language Model Representations

  2025cites this paper
• OWL-S Grounding Parameters Matching by Means of LLM: Preliminary Investigation

  2025cites this paper
• Optimizing Federated Learning using Remote Embeddings for Graph Neural Networks

  2025cites this paper
• Pre-Training a Graph Recurrent Network for Text Understanding

  2025cites this paper
• Low-Resource Neural Machine Translation Using Recurrent Neural Networks and Transfer Learning: A Case Study on English-to-Igbo

  2025cites this paper
• Graph Laplacian Wavelet Transformer via Learnable Spectral Decomposition

  2025cites this paper
• Intelligent Bilingual Reading Translation System Based on Natural Language Processing

  2025cites this paper
• Disaster related tweet classification method based on BERT and GAT

  2025cites this paper
• ALSEM: aspect-level sentiment analysis with semantic and emotional modeling

  2025cites this paper
• Learning to Approximate Adaptive Kernel Convolution on Graphs

  2024cites this paper
• Synergizing Machine Learning & Symbolic Methods: A Survey on Hybrid Approaches to Natural Language Processing

  2024cites this paper
• Regularized Spatial–Temporal Graph Convolutional Networks for Metro Passenger Flow Prediction

  2024cites this paper
• PipeNet: Question Answering with Semantic Pruning over Knowledge Graphs

  2024cites this paper
• Investigating the Impact of Different Graph Representations for Relation Extraction with Graph Neural Networks

  2024cites this paper
• Meaning Representations for Natural Languages: Design, Models and Applications

  2024cites this paper
• Input Representation on Text Data for E-Commerce Product Review Summarization Using Graph Convolutional Network

  2024cites this paper
• An analysis of urban land subsidence susceptibility based on complex network

  2024cites this paper
• Adversarial Attacks Targeting Point-to-Point Wireless Networks

  2024cites this paper
• A multi-aggregator graph neural network for backbone exaction of fracture networks

  2024cites this paper
• Graph Mining under Data scarcity

  2024cites this paper
• Graph Neural Re-Ranking via Corpus Graph

  2024cites this paper
• GCRL: a graph neural network framework for network connectivity robustness learning

  2024cites this paper
• Fine grained sentiment analysis on microblogs based on graph convolution and self attention graph pooling

  2024cites this paper
• Abstractive summarization incorporating graph knowledge

  2024cites this paper
• BERT-CNN based evidence retrieval and aggregation for Chinese legal multi-choice question answering

  2024cites this paper
• GAINER: Graph Machine Learning with Node-specific Radius for Classification of Short Texts and Documents

  2024cites this paper
• Generated Contents Enrichment

  2024cites this paper
• A syntactic evidence network model for fact verification

  2024cites this paper
• RumorSAGE: Semantic Augmentation Graph for Early Rumor Detection

  2024cites this paper
• A Dual-Module Information Fusion Aspect-Level Sentiment Classification Model

  2024cites this paper
• GASCOM: Graph-based Attentive Semantic Context Modeling for Online Conversation Understanding

  2024cites this paper
• Dependency Parse Graph Neural Network for Text Classification

  2024cites this paper
• Vulnerability Detection via Multiple-Graph-Based Code Representation

  2024cites this paper
• Graph Similarity Regularized Softmax for Semi-Supervised Node Classification

  2024cites this paper
• Enhancing Fake News Detection Using Graph Autoencoders and LSTMs

  2024cites this paper
• Spatial and temporal attention embedded spatial temporal graph convolutional networks for skeleton based gait recognition with multiple IMUs

  2024cites this paper
• 基于深度动态语义关联的短视频事件检测

  2024cites this paper
• Morality in the mundane: Categorizing moral reasoning in real-life social situations

  2023influential citation
• Towards Understanding Generalization of Graph Neural Networks

  2023cites this paper
• Fake news detection: A survey of graph neural network methods

  2023cites this paper
• Community-aware graph embedding via multi-level attribute integration

  2023cites this paper
• ChestXRayBERT: A Pretrained Language Model for Chest Radiology Report Summarization

  2023cites this paper
• Recognizing Unseen States of Unknown Objects by Leveraging Knowledge Graphs

  2023cites this paper
• Graph Clustering with High-Order Contrastive Learning

  2023cites this paper
• Towards Understanding the Generalization of Graph Neural Networks

  2023cites this paper
• Syntactic-Informed Graph Networks for Sentence Matching

  2023cites this paper
• The Third Common Interface for Graph Neural Networks

  2023cites this paper
• SynJax: Structured Probability Distributions for JAX

  2023cites this paper
• Word Grounded Graph Convolutional Network

  2023cites this paper
• End-to-End Aspect-Level Sentiment Analysis Based on Directed Syntactic Dependency Trees

  2023cites this paper
• ViCGCN: Graph Convolutional Network with Contextualized Language Models for Social Media Mining in Vietnamese

  2023cites this paper
• Permutation-Invariant Set Autoencoders with Fixed-Size Embeddings for Multi-Agent Learning

  2023cites this paper
• Neural Machine Translation with Dynamic Graph Convolutional Decoder

  2023influential citation
• Detecting influential nodes with topological structure via Graph Neural Network approach in social networks

  2023cites this paper
• HIORE: Leveraging High-order Interactions for Unified Entity Relation Extraction

  2023cites this paper
• Emotion Classification in Texts Over Graph Neural Networks: Semantic Representation is Better Than Syntactic

  2023cites this paper
• Text classification on heterogeneous information network via enhanced GCN and knowledge

  2023cites this paper
• HGCH: A Hyperbolic Graph Convolution Network Model for Heterogeneous Collaborative Graph Recommendation

  2023cites this paper
• Enhancing Neural Machine Translation with Semantic Units

  2023cites this paper
• Which Sentence Representation is More Informative: An Analysis on Text Classification

  2023influential citation
• A deep graph convolutional neural network architecture for graph classification

  2023cites this paper
• Learning to Distill Graph Neural Networks

  2023cites this paper
• BERTGACN: Text Classification by Combining BERT and GCN and GAT

  2023cites this paper
• Knowledge-Fusion-Based Iterative Graph Structure Learning Framework for Implicit Sentiment Identification

  2023cites this paper
• Integrating Relational Knowledge With Text Sequences for Script Event Prediction

  2023cites this paper
• Intelligent fault diagnosis of rolling bearings based on the visibility algorithm and graph neural networks

  2023cites this paper
• SIGMA++: Improved Semantic-Complete Graph Matching for Domain Adaptive Object Detection

  2023cites this paper
• Leveraging Knowledge Graphs for Zero-Shot Object-agnostic State Classification

  2023cites this paper
• Investigating the Impact of Syntax-Enriched Transformers on Quantity Extraction in Scientific Texts

  2023cites this paper
• DSISA: A New Neural Machine Translation Combining Dependency Weight and Neighbors

  2023cites this paper
• Recognizing Real-World Intentions using A Multimodal Deep Learning Approach with Spatial-Temporal Graph Convolutional Networks

  2023cites this paper
• Sparse graph matching network for temporal language localization in videos

  2023cites this paper
• NeutronOrch: Rethinking Sample-based GNN Training under CPU-GPU Heterogeneous Environments

  2023cites this paper
• A syntactic multi-level interaction network for rumor detection

  2023cites this paper
• Adversarial Attacks on Graph Neural Networks Based Spatial Resource Management in P2P Wireless Communications

  2023cites this paper
• Graph Representation Learning for Infrared and Visible Image Fusion

  2023cites this paper
• GATSY: Graph Attention Network for Music Artist Similarity

  2023cites this paper
• Claim Extraction via Subgraph Matching over Modal and Syntactic Dependencies

  2023cites this paper
• GASCOM: Graph-based Attentive Semantic Context Modeling for Online Conversation Understanding

  2023cites this paper
• Dual-view graph neural network with gating mechanism for entity alignment

  2023cites this paper
• Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Models

  2023cites this paper
• S-KMN: Integrating semantic features learning and knowledge mapping network for automatic quiz question annotation

  2023cites this paper
• Graph Representation of Chinese EMRs and its Short-text Classification Application1

  2023cites this paper
• Semantic Graph Neural Network: A Conversion from Spam Email Classification to Graph Classification

  2022cites this paper
• Knowledge Graph Augmented Network Towards Multiview Representation Learning for Aspect-Based Sentiment Analysis

  2022cites this paper
• Chinese Word Segmentation with Heterogeneous Graph Neural Network

  2022cites this paper
• Attention enhanced capsule network for text classification by encoding syntactic dependency trees with graph convolutional neural network

  2022cites this paper
• Keyphrase Extraction with Dynamic Graph Convolutional Networks and Diversified Inference

  2022cites this paper
• Text Classification Using a Graph Based on Relationships Between Documents

  2022cites this paper
• SCGraph: Accelerating Sample-based GNN Training by Staged Caching of Features on GPUs

  2022cites this paper